Received-SPF: pass (google.com: domain of linux-kernel+bounces-12476-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
From: "Tian, Kevin" <kevin.tian@intel.com>
To: "Liu, Yi L" <yi.l.liu@intel.com>, "joro@8bytes.org" <joro@8bytes.org>,
	"alex.williamson@redhat.com" <alex.williamson@redhat.com>, "jgg@nvidia.com"
	<jgg@nvidia.com>, "robin.murphy@arm.com" <robin.murphy@arm.com>,
	"baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>
CC: "cohuck@redhat.com" <cohuck@redhat.com>, "eric.auger@redhat.com"
	<eric.auger@redhat.com>, "nicolinc@nvidia.com" <nicolinc@nvidia.com>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>, "mjrosato@linux.ibm.com"
	<mjrosato@linux.ibm.com>, "chao.p.peng@linux.intel.com"
	<chao.p.peng@linux.intel.com>, "yi.y.sun@linux.intel.com"
	<yi.y.sun@linux.intel.com>, "peterx@redhat.com" <peterx@redhat.com>,
	"jasowang@redhat.com" <jasowang@redhat.com>,
	"shameerali.kolothum.thodi@huawei.com"
	<shameerali.kolothum.thodi@huawei.com>, "lulu@redhat.com" <lulu@redhat.com>,
	"suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com>,
	"iommu@lists.linux.dev" <iommu@lists.linux.dev>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-kselftest@vger.kernel.org" <linux-kselftest@vger.kernel.org>, "Duan,
 Zhenzhong" <zhenzhong.duan@intel.com>, "joao.m.martins@oracle.com"
	<joao.m.martins@oracle.com>, "Zeng, Xin" <xin.zeng@intel.com>, "Zhao, Yan Y"
	<yan.y.zhao@intel.com>, "j.granados@samsung.com" <j.granados@samsung.com>
Subject: RE: [PATCH v8 10/10] iommu/vt-d: Add iotlb flush for nested domain
Thread-Topic: [PATCH v8 10/10] iommu/vt-d: Add iotlb flush for nested domain
Thread-Index: AQHaON+9l+zA44VLDUyKcHm82dBOTbC+R0mg
Date: Thu, 28 Dec 2023 07:11:48 +0000
Message-ID: <BN9PR11MB527696416E2B1F33900007968C9EA@BN9PR11MB5276.namprd11.prod.outlook.com>
References: <20231227161354.67701-1-yi.l.liu@intel.com>
 <20231227161354.67701-11-yi.l.liu@intel.com>
In-Reply-To: <20231227161354.67701-11-yi.l.liu@intel.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?zENgsyEkZfiQtdg3T003FPv/eFjewiEbpznKLemm6iq42Huha9yjQ1/jYv9F?=
 =?us-ascii?Q?XfXo9z0N3Ufbfku2mm9m30HJEeC07TXZfpffJXOZvf0yQAva930jZiM84y37?=
 =?us-ascii?Q?8M4qoGnPceOebrx64cgpr1sXCwlkeo4LGab64JhFuFAqqzIUgVLCNSM55rlG?=
 =?us-ascii?Q?HrSlUk48VlLpLLyKqJv0H/VS5IdkA9s3LGKEKpcpLnaIRAsnz5WTx75+VXVY?=
 =?us-ascii?Q?s17U07uvY4pzOBB9TPOlSSHZydg85RNyZLqyQpIYYaWBuw7aXpm0aCcbkx3u?=
 =?us-ascii?Q?uXNFlc8twslQSaihRojxIyLys+K4kRT4E4QILjtFz93Wm+6yD4OyFKvU8bHb?=
 =?us-ascii?Q?sWLNvXsmLxBRfOKVcVrayOx6ueeNIX7+a+yEoP0E8bM/ruKHJ4hVOhsG/BB1?=
 =?us-ascii?Q?NYVdWEg7JZ5cVnIjgNaBGlY9Gnb3E/ApRwmRMOk6tgDKPQ3i2Q5bEyGVUnCc?=
 =?us-ascii?Q?b1kZWog9Bxi+6VOj/Zk4gEbqWJWTkS2bV3yJhfrKbqXi8r1/QerDtULLbtxJ?=
 =?us-ascii?Q?nij4h1LsKck5Noa8hhqlsvLomiDnF+UBTzA8ljYdisaNaVaHvmY5BgsSx5qZ?=
 =?us-ascii?Q?VZqwHdqPSVn48E4Dd8iovsPaJ5kDgLkBiHT1PJSLcVsfda8T37/IWqLdL81g?=
 =?us-ascii?Q?DRLdFE+NmTPYvHENWrS9I6tz67iK0dVxKwzDeEyTd1qGlG4l8kKNRrB8FVm5?=
 =?us-ascii?Q?FbtCSpTkQyDG5j6A/iDymoJEeOX22p4nLDN2WyaFWdOQFnX7n7yGZ71NV0Lz?=
 =?us-ascii?Q?6bD2Buy0S+JOLi53duCoRG3cEDRhKcwOMbrlAkxOtOns6Vn+yk320PX9BZs6?=
 =?us-ascii?Q?H+LZiLXfNZaLiKbjpexDeSml7uK6+7luw4w0VxBjtryAUJQJ8030PCM/NNjH?=
 =?us-ascii?Q?UugltlMpBBbWErNaj4og5MFagvmmEccxxMJwXJDWXNyOtucryqm6jCPcc0ac?=
 =?us-ascii?Q?3YO4L5jIHnwDUl4Es0pI7saKDuFJ83iboI/6AlUB7TlMLHnFIWS5UlZsxny7?=
 =?us-ascii?Q?e9lDkzvJsSYwLZ5F7ndFYU8mVFhdKtNbw/dXayvnQy1FTiSJ0xAYKk+Nmzzc?=
 =?us-ascii?Q?coZAV6aLezggljtzFkJOkWsSn5C38aTWX1FH025DLPNx9nVSdXoqmmeOxhBW?=
 =?us-ascii?Q?1VH/aZUJzFocy+LedtwQ+J7W5aqXdFRS+ydp7IZULW331Y2yCEKBpdVgII7H?=
 =?us-ascii?Q?7cFvPVoe7Yged7lZaehxnZPmPHlod6sVPZ+jbjzEyiyTHep+CNzBq5cJfP1y?=
 =?us-ascii?Q?MWRFzl0UeufIv2vw/0KfpmYnH9SXYOZ9FvjTdAKh9Ek1f6DpSstpG6y/t653?=
 =?us-ascii?Q?boiTF6iPa0ElBlijLacOBvYSdC3GV+2gQGH2CfqOdWkf+3ZS3LpdQ1iNq15w?=
 =?us-ascii?Q?L2s9dA+8P4o1156+2smbQk6GiU5mGGu8RIZjDk+k/HEbpAcwhS1M8cA0Wwp6?=
 =?us-ascii?Q?jR7hpp04Q+4yQCs9gyhUR6jh5U9VajDa8rkDizHkLEZzjoKwoSqIMRkfwxke?=
 =?us-ascii?Q?BVvPY3CwZRVaoqxzpxf6hl//PFbHkWA/7ETp4h1ZGBy2cWb90G/7G6C5wtB1?=
 =?us-ascii?Q?AJzeVUS3ssRS30f52HQKykk13LbGn3kLit5latSG?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
List-Id: <linux-kernel.vger.kernel.org>
List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org>
List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org>
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d44abe7a-6c19-462b-6d19-08dc0774424c
X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Dec 2023 07:11:48.9261
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: vkyInhm93D39BdIpHnDX4Weehi9hkM08Blu01EweNQSTHHQMG3P8DzdWnis5ALv2qEuWAS5dHM7QWACU2O4qBg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB8202
X-OriginatorOrg: intel.com

> From: Liu, Yi L <yi.l.liu@intel.com>
> Sent: Thursday, December 28, 2023 12:14 AM
>=20
>=20
> +static void nested_flush_pasid_iotlb(struct intel_iommu *iommu,
> +				     struct dmar_domain *domain, u64 addr,
> +				     unsigned long npages, bool ih)
> +{
> +	u16 did =3D domain_id_iommu(domain, iommu);
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&domain->lock, flags);
> +	if (!list_empty(&domain->devices))
> +		qi_flush_piotlb(iommu, did, IOMMU_NO_PASID, addr,
> +				npages, ih, NULL);

Not sure the check of domain->devices makes sense here. The outer
loop is xa_for_each(&domain->iommu_array, i, info) which, if found,
then implies certain devices behind this iommu must be attached to
this domain.

I wonder whether it's clearer to remove this function and just call
qi_flush_piotlb() from intel_nested_flush_cache().

> +
> +	/*
> +	 * Invalidation queue error (i.e. IQE) will not be reported to user
> +	 * as it's caused only by driver internal bug.
> +	 */

here lack of:

	*error =3D 0;

> +	if (fault & DMA_FSTS_ICE)
> +		*error |=3D IOMMU_HWPT_INVALIDATE_VTD_S1_ICE;
> +	if (fault & DMA_FSTS_ITE)
> +		*error |=3D IOMMU_HWPT_INVALIDATE_VTD_S1_ITE;
> +}
> +