Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [RFC] /dev/ioasid uAPI proposal
To:     Jason Gunthorpe <jgg@nvidia.com>
Cc:     "Tian, Kevin" <kevin.tian@intel.com>,
        Lu Baolu <baolu.lu@linux.intel.com>,
        Liu Yi L <yi.l.liu@linux.intel.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        Jonathan Corbet <corbet@lwn.net>,
        "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
        LKML <linux-kernel@vger.kernel.org>,
        "Alex Williamson (alex.williamson@redhat.com)\"\"" 
        <alex.williamson@redhat.com>, David Woodhouse <dwmw2@infradead.org>
References: <MWHPR11MB1886FC7A46837588254794048C3E9@MWHPR11MB1886.namprd11.prod.outlook.com>
 <05d7f790-870d-5551-1ced-86926a0aa1a6@redhat.com>
 <MWHPR11MB1886269E2B3DE471F1A9A7618C3E9@MWHPR11MB1886.namprd11.prod.outlook.com>
 <42a71462-1abc-0404-156c-60a7ee1ad333@redhat.com>
 <20210601173138.GM1002214@nvidia.com>
 <f69137e3-0f60-4f73-a0ff-8e57c79675d5@redhat.com>
 <20210602172154.GC1002214@nvidia.com>
 <c84787ec-9d8f-3198-e800-fe0dc8eb53c7@redhat.com>
 <20210608132039.GG1002214@nvidia.com>
 <f4d70f28-4bd6-5315-d7c7-0a509e4f1d1d@redhat.com>
 <20210610114751.GK1002214@nvidia.com>
From:   Jason Wang <jasowang@redhat.com>
Message-ID: <d2193dbd-0d55-7315-4e76-eea7f8cc8f5b@redhat.com>
Date:   Fri, 11 Jun 2021 13:43:59 +0800
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <20210610114751.GK1002214@nvidia.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
Precedence: bulk


在 2021/6/10 下午7:47, Jason Gunthorpe 写道:
> On Thu, Jun 10, 2021 at 10:00:01AM +0800, Jason Wang wrote:
>> 在 2021/6/8 下午9:20, Jason Gunthorpe 写道:
>>> On Tue, Jun 08, 2021 at 09:10:42AM +0800, Jason Wang wrote:
>>>
>>>> Well, this sounds like a re-invention of io_uring which has already worked
>>>> for multifds.
>>> How so? io_uring is about sending work to the kernel, not getting
>>> structued events back?
>>
>> Actually it can. Userspace can poll multiple fds via preparing multiple sqes
>> with IORING_OP_ADD flag.
> Poll is only a part of what is needed here, the main issue is
> transfering the PRI events to userspace quickly.


Do we really care e.g at most one more syscall in this case? I think the 
time spent on demand paging is much more than transferring #PF to 
userspace. What's more, a well designed vIOMMU capable IOMMU hardware 
should have the ability to inject such event directly to guest if #PF 
happens on L1.


>
>> This means another ring and we need introduce ioctl() to add or remove
>> ioasids from the poll. And it still need a kind of fallback like a list if
>> the ring is full.
> The max size of the ring should be determinable based on the PRI
> concurrance of each device and the number of devices sharing the ring


This has at least one assumption, #PF event is the only event for the 
ring, I'm not sure this is the case.

Thanks


>
> In any event, I'm not entirely convinced eliding the PRI user/kernel
> copy is the main issue here.. If we want this to be low latency I
> think it ends up with some kernel driver component assisting the
> vIOMMU emulation and avoiding the round trip to userspace
>
> Jason
>