Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
IronPort-SDR: krv5VOT2S1t/QuELlC07dBwGd26/WLxlyo2QxQWE6yo6DNxr+hOEARJjHmFJpLmpc/CxW7TGkM
 ALTJvGqcvEwg==
IronPort-SDR: SybIKhLo68vX3sB/Dv0c2eIQg1JFRFdTnesqEqt5fIKZNYO9N4/zh6Pm5SBAVyAzaqZu5IBjqg
 mgAzZ3MU31jQ==
From:   "Tian, Kevin" <kevin.tian@intel.com>
To:     Jason Gunthorpe <jgg@nvidia.com>
CC:     Jean-Philippe Brucker <jean-philippe@linaro.org>,
        "Alex Williamson (alex.williamson@redhat.com)" 
        <alex.williamson@redhat.com>, "Raj, Ashok" <ashok.raj@intel.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "Jonathan Corbet" <corbet@lwn.net>,
        Robin Murphy <robin.murphy@arm.com>,
        LKML <linux-kernel@vger.kernel.org>,
        Kirti Wankhede <kwankhede@nvidia.com>,
        "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
        "David Gibson" <david@gibson.dropbear.id.au>,
        "Jiang, Dave" <dave.jiang@intel.com>,
        David Woodhouse <dwmw2@infradead.org>,
        Jason Wang <jasowang@redhat.com>
Subject: RE: [RFC] /dev/ioasid uAPI proposal
Thread-Topic: [RFC] /dev/ioasid uAPI proposal
Thread-Index: AddSzQ97BhCb3gd8AUyldaDZ6yOMNABTUTCAAQiuDaAADwOFgAJSmyVgAA0fjwAAEFKEEAAASwWAAAAgcXA=
Date:   Tue, 15 Jun 2021 23:09:37 +0000
Message-ID: <MWHPR11MB1886A0CAB3AFF424A4A090038C309@MWHPR11MB1886.namprd11.prod.outlook.com>
References: <MWHPR11MB1886422D4839B372C6AB245F8C239@MWHPR11MB1886.namprd11.prod.outlook.com>
 <20210528233649.GB3816344@nvidia.com>
 <MWHPR11MB188621A9D1181A414D5491198C3C9@MWHPR11MB1886.namprd11.prod.outlook.com>
 <20210603130519.GY1002214@nvidia.com>
 <MWHPR11MB1886BA2258AFD92D5249AC488C309@MWHPR11MB1886.namprd11.prod.outlook.com>
 <20210615150630.GS1002214@nvidia.com>
 <MWHPR11MB1886E9553A5054DF7D51F27D8C309@MWHPR11MB1886.namprd11.prod.outlook.com>
 <20210615230215.GA1002214@nvidia.com>
In-Reply-To: <20210615230215.GA1002214@nvidia.com>
Accept-Language: en-US
Content-Language: en-US
dlp-version: 11.5.1.3
dlp-product: dlpe-windows
dlp-reaction: no-action
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?eguk5xzLesmU+lXxrdWSe00mdCe9SbgV/Q3qtzNkvR9blpOfR+clolWkDg3B?=
 =?us-ascii?Q?bRni14K0EZfhgAgYBNsawqvfFqwGHSmnv4Ltyx5M2uZwlh5zyybRBrkBmJc8?=
 =?us-ascii?Q?UOfLDmYW1qqElxbXMdITXrPd5jc4/EG8hCQC+Zhhg3G4kQhD6LPO6hJv9zR5?=
 =?us-ascii?Q?8BIVpwG3FaQZvsP5pT/JgG1djYGJOcbahFjDhPCtauw7v0El7qrxQFmbmKIb?=
 =?us-ascii?Q?Rd0v5ePTwG5qgUB+i1mIsW1ZcMYHB/tmr2lRlZAFAtepzDa6jFhMrznXfWWF?=
 =?us-ascii?Q?vgYfXAh2FRa6tElAxJ7Q4763WaZMJbVf/yvJtWIscalnO3jCgDHLSmY/3y4x?=
 =?us-ascii?Q?W7OCu+uGKE8H17H7mSCh6uc9rc7pRwYarKJp8Dw9tr9FNYs0XGfZdJgBPAHF?=
 =?us-ascii?Q?rN8dKR9QDQkGghL9iliL1+ce1AgMdmP2zAJWRS6mcpJS6/2VBrVeIC08iIDl?=
 =?us-ascii?Q?JgDTpp4NUP9eXXkfSVjQBcq6unQfLMlYUpubzX10OwT8jf7X0+W627cUz8jQ?=
 =?us-ascii?Q?F1+VuO/oXMr2GLHCTFjSki9v8hlLjFE4y+UvCIGE5t3Xk0fV0bPQ3G/ZUBwO?=
 =?us-ascii?Q?0RdSfd++xUylVbF0z/n1EQbZEVudstNecW/x5+Lftcjv/yCsenzFlUwxbZba?=
 =?us-ascii?Q?S9x4w0NymQ1VVt/mjXPbdig5iTYl1Y9PZprJwBekzXNUUcP4fK9vizYI+OcT?=
 =?us-ascii?Q?VV8668zDqKSRv0bf0zWgzHWg253UflcCprDHKnFu3BFt6CS5PfiuigKwB5xt?=
 =?us-ascii?Q?+nAG519ftUgnpWul24U9F1M2xYjC0+15H2ObYsjaUYpNULwr5pii9DjR/TbG?=
 =?us-ascii?Q?w+VBJw4hhIn3ednIEL3BUmgDsxsHCHynyyeCO5DS+bjESCV/pltKtkR7y9Bt?=
 =?us-ascii?Q?yBcaZukW6gW9dabOtrSMHgeI/PK43iUtrHYfmuSa26lH57MTj+XWhD74CcSD?=
 =?us-ascii?Q?s679AduVGOl691OmO8nfGicJnK6HeU23Y6aY/nnrdSgMwTJbT6QZYbNIcLWb?=
 =?us-ascii?Q?v/+9qSTfGYTrV6Zl+ePUQTz+WGqt3vjfIiPM6RPmfm0+/evi+X57f1C14H9b?=
 =?us-ascii?Q?UdGXAkGHF8mHkRsYFskLyxw4CFtJDP0YhTy/EcQT5XdD/A0v3Cs2ff7OVdFN?=
 =?us-ascii?Q?BVWYsFZL9+tkSK1W3Fj1zY5dI9RJ+aJJwGb0DPk+iSOmW2mE2KdH/MpcIper?=
 =?us-ascii?Q?6Bt7pyKEF7mqbXIhu9+cUxEIj2k1MAqNNfrHtrOvnD9/1tK8qlTWprr9HJHD?=
 =?us-ascii?Q?2REQDoqLyIeSjDyfLl2pt2zuX2m8nNn3S4fUawOHJgVHjOPNmyqBsKh/DfVm?=
 =?us-ascii?Q?SFsVSRdt8a6+czQ31+a4ZZHu?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR11MB1886.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bb3e58d5-19f5-476f-ce02-08d93052a5e2
X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Jun 2021 23:09:37.6622
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: rkA5dkH4JFf5QIZZQZgOk/y+nuyV6EnqMTZLo2rABgehoOJ0w2n4G9M7RQEvcZTZCN64W9TbMJHDv/H477hW4w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR11MB4963
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Wednesday, June 16, 2021 7:02 AM
>=20
> On Tue, Jun 15, 2021 at 10:59:06PM +0000, Tian, Kevin wrote:
> > > From: Jason Gunthorpe <jgg@nvidia.com>
> > > Sent: Tuesday, June 15, 2021 11:07 PM
> > >
> > > On Tue, Jun 15, 2021 at 08:59:25AM +0000, Tian, Kevin wrote:
> > > > Hi, Jason,
> > > >
> > > > > From: Jason Gunthorpe
> > > > > Sent: Thursday, June 3, 2021 9:05 PM
> > > > >
> > > > > On Thu, Jun 03, 2021 at 06:39:30AM +0000, Tian, Kevin wrote:
> > > > > > > > Two helper functions are provided to support
> VFIO_ATTACH_IOASID:
> > > > > > > >
> > > > > > > > 	struct attach_info {
> > > > > > > > 		u32	ioasid;
> > > > > > > > 		// If valid, the PASID to be used physically
> > > > > > > > 		u32	pasid;
> > > > > > > > 	};
> > > > > > > > 	int ioasid_device_attach(struct ioasid_dev *dev,
> > > > > > > > 		struct attach_info info);
> > > > > > > > 	int ioasid_device_detach(struct ioasid_dev *dev, u32 ioasi=
d);
> > > > > > >
> > > > > > > Honestly, I still prefer this to be highly explicit as this i=
s where
> > > > > > > all device driver authors get invovled:
> > > > > > >
> > > > > > > ioasid_pci_device_attach(struct pci_device *pdev, struct
> ioasid_dev
> > > *dev,
> > > > > > > u32 ioasid);
> > > > > > > ioasid_pci_device_pasid_attach(struct pci_device *pdev, u32
> > > > > *physical_pasid,
> > > > > > > struct ioasid_dev *dev, u32 ioasid);
> > > > > >
> > > > > > Then better naming it as pci_device_attach_ioasid since the 1st
> > > parameter
> > > > > > is struct pci_device?
> > > > >
> > > > > No, the leading tag indicates the API's primary subystem, in this=
 case
> > > > > it is iommu (and if you prefer list the iommu related arguments f=
irst)
> > > > >
> > > >
> > > > I have a question on this suggestion when working on v2.
> > > >
> > > > Within IOMMU fd it uses only the generic struct device pointer, whi=
ch
> > > > is already saved in struct ioasid_dev at device bind time:
> > > >
> > > > 	struct ioasid_dev *ioasid_register_device(struct ioasid_ctx *ctx,
> > > > 		struct device *device, u64 device_label);
> > > >
> > > > What does this additional struct pci_device bring when it's specifi=
ed in
> > > > the attach call? If we save it in attach_data, at which point will =
it be
> > > > used or checked?
> > >
> > > The above was for attaching to an ioasid not the register path
> >
> > Yes, I know. and this is my question. When receiving a struct pci_devic=
e
> > at attach time, what should IOMMU fd do with it? Just verify whether
> > pci_device->device is same as ioasid_dev->device? if saving it to per-d=
evice
> > attach data under ioasid then when will it be further used?
> >
> > I feel once ioasid_dev is returned in the register path, following oper=
ations
> > (unregister, attach, detach) just uses ioasid_dev as the main object.
>=20
> The point of having the pci_device specific API was to convey bus
> specific information during the attachment to the IOASID.

which information can you elaborate? This is the area which I'm not
familiar with thus would appreciate if you can help explain how this
bus specific information is utilized within the attach function or=20
sometime later.

>=20
> The registration of the device to the iommu_fd doesn't need bus
> specific information, AFIAK? So just use a normal struct device
> pointer
>=20

yes.

Thanks
Kevin