On 3/15/22 10:38 AM, Jason Gunthorpe wrote:
> On Tue, Mar 15, 2022 at 09:49:01AM -0400, Matthew Rosato wrote:
>
>> The rationale for splitting steps 1 and 2 are that VFIO_SET_IOMMU doesn't
>> have a mechanism for specifying more than the type as an arg, no? Otherwise
>> yes, you could specify a kvm fd at this point and it would have some other
>> advantages (e.g. skip notifier). But we still can't use the IOMMU for
>> mapping until step 3.
>
> Stuff like this is why I'd be much happier if this could join our
> iommfd project so we can have clean modeling of the multiple iommu_domains.
>
I'd certainly be willing to collaborate so feel free to loop me in on
the discussions; but I got the impression that iommufd is not close to
ready (maybe I'm wrong?) -- if so I really don't want to completely
delay this zPCI support behind it as it has a significant benefit for
kvm guests on s390x :(
On Tue, Mar 15, 2022 at 12:29:02PM -0400, Matthew Rosato wrote:
> On 3/15/22 10:38 AM, Jason Gunthorpe wrote:
> > On Tue, Mar 15, 2022 at 09:49:01AM -0400, Matthew Rosato wrote:
> >
> > > The rationale for splitting steps 1 and 2 are that VFIO_SET_IOMMU doesn't
> > > have a mechanism for specifying more than the type as an arg, no? Otherwise
> > > yes, you could specify a kvm fd at this point and it would have some other
> > > advantages (e.g. skip notifier). But we still can't use the IOMMU for
> > > mapping until step 3.
> >
> > Stuff like this is why I'd be much happier if this could join our
> > iommfd project so we can have clean modeling of the multiple iommu_domains.
> >
>
> I'd certainly be willing to collaborate so feel free to loop me in on the
> discussions;
Sure, I have you on my list. I've been waiting for Eric to get a bit
further along on his ARM work so you have something appropriate to
look at.
In the mean time you can certainly work out the driver details as
you've been doing here and hacking through VFIO. The iommu_domain
logic is the big work item in this series, not the integration with
the uAPI.
> but I got the impression that iommufd is not close to ready (maybe
> I'm wrong?)
Well, quite alot has been done already and I think we are getting
close to having something that can start moving forward, but yes it
will not be "tomorrow".
> -- if so I really don't want to completely delay this zPCI support
> behind it as it has a significant benefit for kvm guests on s390x :(
Every platform vendor is trying to do this exact same performance
optimization. s390 is doing it a bit different, but as we've seen, not
fundamentally so compard to ARM and Intel IOMMU's with HW nesting.
I'm not sure that s390 should jump the queue and hacky hacky uAPIs all
over the place. ARM platform has been working on this for literal
years now.
Jason
On 3/15/22 1:25 PM, Jason Gunthorpe wrote:
> On Tue, Mar 15, 2022 at 12:29:02PM -0400, Matthew Rosato wrote:
>> On 3/15/22 10:38 AM, Jason Gunthorpe wrote:
>>> On Tue, Mar 15, 2022 at 09:49:01AM -0400, Matthew Rosato wrote:
>>>
>>>> The rationale for splitting steps 1 and 2 are that VFIO_SET_IOMMU doesn't
>>>> have a mechanism for specifying more than the type as an arg, no? Otherwise
>>>> yes, you could specify a kvm fd at this point and it would have some other
>>>> advantages (e.g. skip notifier). But we still can't use the IOMMU for
>>>> mapping until step 3.
>>>
>>> Stuff like this is why I'd be much happier if this could join our
>>> iommfd project so we can have clean modeling of the multiple iommu_domains.
>>>
>>
>> I'd certainly be willing to collaborate so feel free to loop me in on the
>> discussions;
>
> Sure, I have you on my list. I've been waiting for Eric to get a bit
> further along on his ARM work so you have something appropriate to
> look at.
>
> In the mean time you can certainly work out the driver details as
> you've been doing here and hacking through VFIO. The iommu_domain
> logic is the big work item in this series, not the integration with
> the uAPI.
>
A subset of this series (enabling some s390x firmware-assist facilities)
is not tied to the iommu and would still provide value while continuing
to use vfio_iommu_type1 for all mapping -- so I think I'll look into a
next version that shrinks down to that subset (+ re-visit the setup API).
Separate from that, I will continue looking at implementing the nested
iommu_domain logic for s390, and continue to hack through VFIO for now.
I'll use an RFC series when I have something more to look at, likely
starting with the fully-pinned guest as you suggest; ultimately I'm
interested in both scenarios (pinned kvm guest & dynamic pinning during
shadow)
Thanks,
Matt