Received-SPF: pass (google.com: domain of linux-kernel+bounces-17411-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
From: "Tian, Kevin" <kevin.tian@intel.com>
To: Jason Gunthorpe <jgg@nvidia.com>, "Liu, Yi L" <yi.l.liu@intel.com>
CC: "joro@8bytes.org" <joro@8bytes.org>, "alex.williamson@redhat.com"
	<alex.williamson@redhat.com>, "robin.murphy@arm.com" <robin.murphy@arm.com>,
	"baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>, "cohuck@redhat.com"
	<cohuck@redhat.com>, "eric.auger@redhat.com" <eric.auger@redhat.com>,
	"nicolinc@nvidia.com" <nicolinc@nvidia.com>, "kvm@vger.kernel.org"
	<kvm@vger.kernel.org>, "mjrosato@linux.ibm.com" <mjrosato@linux.ibm.com>,
	"chao.p.peng@linux.intel.com" <chao.p.peng@linux.intel.com>,
	"yi.y.sun@linux.intel.com" <yi.y.sun@linux.intel.com>, "peterx@redhat.com"
	<peterx@redhat.com>, "jasowang@redhat.com" <jasowang@redhat.com>,
	"shameerali.kolothum.thodi@huawei.com"
	<shameerali.kolothum.thodi@huawei.com>, "lulu@redhat.com" <lulu@redhat.com>,
	"suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com>,
	"iommu@lists.linux.dev" <iommu@lists.linux.dev>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-kselftest@vger.kernel.org" <linux-kselftest@vger.kernel.org>, "Duan,
 Zhenzhong" <zhenzhong.duan@intel.com>, "joao.m.martins@oracle.com"
	<joao.m.martins@oracle.com>, "Zeng, Xin" <xin.zeng@intel.com>, "Zhao, Yan Y"
	<yan.y.zhao@intel.com>
Subject: RE: [PATCH v7 1/3] iommufd: Add data structure for Intel VT-d stage-1
 cache invalidation
Thread-Topic: [PATCH v7 1/3] iommufd: Add data structure for Intel VT-d
 stage-1 cache invalidation
Thread-Index: AQHaGViOWXGe59nutU2rEx/0zRDvrrCozh6AgCE2IQCAALd70A==
Date: Fri, 5 Jan 2024 02:16:31 +0000
Message-ID: <BN9PR11MB527699A02C25431DC3438E468C662@BN9PR11MB5276.namprd11.prod.outlook.com>
References: <20231117131816.24359-1-yi.l.liu@intel.com>
 <20231117131816.24359-2-yi.l.liu@intel.com>
 <c967e716-9112-4d1a-b6f7-9a005e28202d@intel.com>
 <20240104143658.GX50406@nvidia.com>
In-Reply-To: <20240104143658.GX50406@nvidia.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?/vWzOn887aMZs5ty484usgSJ2rQf9DnPLxPlNnQrVircQBcWu/SYGeyetzuP?=
 =?us-ascii?Q?sUrMFgjVymrEHehH2vpchjwWjK19i2ZDyDwZFUf+lwQSghzvQan6lGYzz1om?=
 =?us-ascii?Q?C6955yfPeZGCTZ9YdnGOn4ptaQGMB6q8IvtJc7Le9AYV1yxo1L8+0nZh8U/Q?=
 =?us-ascii?Q?cRtAGApXNwrjPcVgbPFfT0cF0+r5MM7I5455OMYTA+2k0sEjNoAmUQDwELrf?=
 =?us-ascii?Q?JfJjv+tWqPtDv6hO8Ya4GG2bcjI2p0KNQZrpvKnZ3Y06wHdy74RGRAV41K9i?=
 =?us-ascii?Q?PWBRiF26Slv5tYSV0HZoF7ZFReeZuNPeLpMHPXTCq7CuWp4hZ+yQdiFdlnr6?=
 =?us-ascii?Q?68ZdtWrSSlaUqiBGUiX1kypiJ3cCMgz4/MonVpLzcHeFp4GKuCsZt7B0AaxH?=
 =?us-ascii?Q?Scrh7adcXgA+54NR8v9ZjRlJOkI1Eod0tEByWJdJETLmwBrvTcpSRq0vts1C?=
 =?us-ascii?Q?5ULgxR7vQzbHBki0yfkqTaxOM8mHnVCYUqujae+cq/eYk+ucJqgMcOofJHAE?=
 =?us-ascii?Q?1Me3KyflddFAEE7rnCN9oPvgwrsX17VJIIlYLOy0aWmEqQ7xZo0m1PX1ly6q?=
 =?us-ascii?Q?URPis5fCOlfFuHFDN+OCOmT1JevxGri5mMIxbXUwfry8r+eYfrqlN7s9iHDz?=
 =?us-ascii?Q?3GZtgCm8wrLfgnAwA4RSyNg6sGI8PwDWICsX1Bi/XfH7QIBc3sBw0KfgILbM?=
 =?us-ascii?Q?pvDMYehUY/HdWy9gnjLZtrqcAk7oBDSLobWZE4cG1xM7xbhfT1sfSbhBVD1P?=
 =?us-ascii?Q?qSicL0B+CqsnTpw0euaVYO0/tlkXTZ/1xBnji/73QyOcvFktEnP8INg993Jc?=
 =?us-ascii?Q?GoXwCz4gFixBxFTmKLr7NndWRxzbpCHA91Ft8HhUFGFcrD9ar/CzOFeCmItu?=
 =?us-ascii?Q?GBDSnwA9DRsJ5RpWNPRqQi39UxsTvufFYfGJRuTile3fcLRD2BNe1GKLGyi7?=
 =?us-ascii?Q?gLukU5CbIS4VseodSDHLJ5Va+BJKflX52wJ89f9d5gFk0wkLMHBwjwa0hgqA?=
 =?us-ascii?Q?yFHQxnfUy3RJYzGsAILjrOH+7lgPLW6bHplQayomCcGjA5T4b2oNJzxbt5zl?=
 =?us-ascii?Q?ZmcgszPoFU0mnWMdFQtvC0qkyeN+c0WATus3ohiYNu0+BfL2gsMBekCQjEjI?=
 =?us-ascii?Q?Exyk5gUAReGSMBy/CCOXRA/LWczYedfPr2ZpO40iYxvo9i1xRXdibvdU4xID?=
 =?us-ascii?Q?/hJYjXJz+zjE+sd4y4rNLMA5QqtZN7oKxmM/rkI23cB8+IWiyEdAkMWJN4mC?=
 =?us-ascii?Q?N3uepWYf5YMXoftjGJ5vrrZlhLp5BwtmY/F8/Q6sNE90KOvukYtiQFUNpTEj?=
 =?us-ascii?Q?QoIsBjMFM1GuHitMHVd085KgNKqHIb9Z/HWBIQ2P1bVJUhtgxSselgbVIL1i?=
 =?us-ascii?Q?Sy6C4jWb5fHjm3n22DHjGOAi+Xb0ei3/HpwO8I6JCk254CXx9k3cb3DUQY7m?=
 =?us-ascii?Q?GNsGkdve58XovAp2//lJz4UiEyICITF/ZtUnLfqMIsjUlVLVMgtl6cn1r1Xd?=
 =?us-ascii?Q?QOWPNjpjVrI6LMUfPsFRNXh9CzaFeBd8b3BU05j5UdltgajJ3kq2qcpIkRTe?=
 =?us-ascii?Q?VvP/JzppletPkU2R0sinxYiE2HPYECPSjx6iA+Ll?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
List-Id: <linux-kernel.vger.kernel.org>
List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org>
List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org>
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 609ec3f7-0db9-435e-9386-08dc0d945530
X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Jan 2024 02:16:31.5176
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: As62HaFQ5KQZls+1rFQTTUqUZC5B8gbuorF7bfaTWiBk2ca75U8zmk9HUpdSgknHGMUedG9jU5RxQ/1iKtYRoQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR11MB6687
X-OriginatorOrg: intel.com

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Thursday, January 4, 2024 10:37 PM
>=20
> On Thu, Dec 14, 2023 at 07:26:39PM +0800, Yi Liu wrote:
> > Per the prior discussion[1], we agreed to move the error reporting into=
 the
> > driver specific part. On Intel side, we want to report two devTLB
> > invalidation errors: ICE (invalid completion error) and ITE (invalidati=
on
> > timeout error). Such errors have an additional SID information to tell
> > which device failed the devTLB invalidation. I've got the below structu=
re.
>=20
> IMHO all of this complexity is a consequence of the decision to hide
> the devtlb invalidation from the VM..
>=20
> On the other hand I guess you want to do this because of the SIOV
> troubles where the vPCI function in the VM is entirely virtual and
> can't be trivially mapped to a real PCI function for ATC invalidation
> like ARM and AMD can do (but they also can't support SIOV because of
> this). :(
>=20
> However it also makes it very confusing about how the VM would
> perceive an error - eg if it invalidates an SIOV device single PASID
> and that devtlb fails then the error should be connected back to the
> vPCI function for the SIOV's specific PASID and not back to the
> physical PCI function for the SIOV owner.
>=20
> As the iommu driver itself has no idea about the vPCI functions this
> seems like it is going to get really confusing. The API I suggested in
> the other email is not entirely going to work as the vPCI function for
> SIOV cases will have to be identified by the (struct device, PASID) -
> while it would be easy enough for the iommu driver to provide the
> PASID, I'm not sure how the iommufd core will relate the PASID back to
> the iommu_device to understand SIOV without actually being aware of
> SIOV to some degree :\

we plan to add such awareness with a new binding helper:

https://lore.kernel.org/lkml/20231009085123.463179-4-yi.l.liu@intel.com/

and with metadata to track association between PASID's and iommufd vdev.

but in reality the relation could be identified in an easy way due to a SIO=
V
restriction which we discussed before - shared PASID space of PF disallows
assigning sibling vdev's to a same VM (otherwise no way to identify which
sibling vdev triggering an iopf when a pasid is used on both vdev's). That
restriction implies that within an iommufd context every iommufd_device
object should contain a unique struct device pointer. So PASID can be
instead ignored in the lookup then just always do iommufd_get_dev_id()
using struct device.