Received-SPF: pass (google.com: domain of linux-kernel+bounces-26317-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Date: Mon, 15 Jan 2024 13:24:30 -0400
From: Jason Gunthorpe <jgg@nvidia.com>
To: Yi Liu <yi.l.liu@intel.com>
Cc: joro@8bytes.org, alex.williamson@redhat.com, kevin.tian@intel.com,
	robin.murphy@arm.com, baolu.lu@linux.intel.com, cohuck@redhat.com,
	eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org,
	mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com,
	yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com,
	shameerali.kolothum.thodi@huawei.com, lulu@redhat.com,
	suravee.suthikulpanit@amd.com, iommu@lists.linux.dev,
	linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org,
	zhenzhong.duan@intel.com, joao.m.martins@oracle.com,
	xin.zeng@intel.com, yan.y.zhao@intel.com
Subject: Re: [PATCH 3/8] iommufd: Support attach/replace hwpt per pasid
Message-ID: <20240115172430.GN734935@nvidia.com>
References: <20231127063428.127436-1-yi.l.liu@intel.com>
 <20231127063428.127436-4-yi.l.liu@intel.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20231127063428.127436-4-yi.l.liu@intel.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?Od18bQGSlWS5MfZqDokOyyUs15So/74Fga4HNYdaGzJWnr89bKP7pqHBio7l?=
 =?us-ascii?Q?d83BVgV0DwQcllQBQgtMiErh4+jMrq8lwWkYqbmeX1NWcPbkq6kYG7cfW3cs?=
 =?us-ascii?Q?YGGrU/KslY80XZEselTMgXFyHoTv3nRgf+AjRex7SLD56ZSlxHM5u4FBpjI4?=
 =?us-ascii?Q?HOQ6jmhzsUC9V+Jkt96xoiRgM71EX1S6qPXigwg5RMnO4N/l3NAi/sKWWUHk?=
 =?us-ascii?Q?66ylOEX/asnFuuTvWJSL413sUJ3Huz5vrNqQCOqi5p84dkesUchAEluN/p0p?=
 =?us-ascii?Q?60dMNydP0plpL2TS7iiY4HMPMhSQx8lQV6+70/SxXk9Ed/A628LlUcVQFRSm?=
 =?us-ascii?Q?WklcijnXaaiKGMg2JyK4LTIrU2xY00hA08gJqPGodFf9aeFyalD63VmUVxte?=
 =?us-ascii?Q?ANeg7wWyxvcxhNo/v9+Y8ixYAjRZ4rrZyz/hpq7UIHbXDwbhvQgKMVzQ0LCc?=
 =?us-ascii?Q?mZ0jq/TUvlze0DoOf+cKLBgYbsR4kUD6NRRY51QGvLjM2tbDibWHiQhjP/Tj?=
 =?us-ascii?Q?WRyWaqsSeD9J12DzQH8TPqrZLkZqvDH8JJpKNDWCxIHvUz/jQWDshLqZ2vWO?=
 =?us-ascii?Q?bVHf9Q9r1GuT54TDVNTuVwCsNWOL8ie7pQW/PkHX2qsYm5rupYyTa26j2Sz+?=
 =?us-ascii?Q?MR3XZrrbqToinvyX/h1IBAKYSLvC60mgK4FczzDvPJwspMXm9Ao3urUA3Rrk?=
 =?us-ascii?Q?M43/blW1uzpNpTcfIcHKm6fIEpQrDJ5UHF2OrIcdUGQWpcYWh8Pw9hNIjvhD?=
 =?us-ascii?Q?8v8hp6uSxGErA42D/ZdsxKd9WtFDJUUcr7dKNXAU6qtdNnplDeqIiGXxZDRv?=
 =?us-ascii?Q?zBE3CxFnEv9AYB/AXO0fCbTdmT3KhmTk2YKwj+QO5TODS7GUBQTQfuNH9bZw?=
 =?us-ascii?Q?eH3lkrIhVuq6eCbmup8g0ViO62REYKGzwQvvApmEmNlfi48LaG8+VRDul3zs?=
 =?us-ascii?Q?6YXHNhzX556ksEeETu1IPHu2BTpa9jB4bpyCXJQmmkyIwLa+AaPXm+lNPyYQ?=
 =?us-ascii?Q?T+2FMuFYySnVO5HtanE9BYH1QDVgl1aLYUcdrbJxBJU49YTgrVD3iniwuYNs?=
 =?us-ascii?Q?41Rt13WHRbG5gS1bEN9no/E5YcwTKDQyXRtDE5icqlvh0IDbL3BFFkO10gEs?=
 =?us-ascii?Q?rMCbr2GkXbyLQW15LdtPbkKGzv5HyDmgOeb0hI6VfPKCGtQxWfDr4SlmlgPK?=
 =?us-ascii?Q?bYsZF3U87EOsUdRMqqDlMNbG5OZz36/+x8rXsea8gSmAr8IH7CoeHVAf9KeQ?=
 =?us-ascii?Q?QkMZIWRw//HOxTolwWv5eH0Ecd1unAbaGgj9rI8MqQFdLlnFssKT3k5pyaE9?=
 =?us-ascii?Q?5MzSQGp0GWIe0jLcHbZfI3lFZkKs0xUAVizaZlbRv7fz2csDBno3COWoJ4mf?=
 =?us-ascii?Q?2VACSWCtT1HtdygUjVsAqZl1D/zqoo7g5TXZHPUXVJRxWaNnh6JhRXPX3ldn?=
 =?us-ascii?Q?JF9EBXi7UtiR2H7Hn2JTyVUWci4Pr/UQDX3ZL1y/qiyCnPdyBG4OrR2L29DA?=
 =?us-ascii?Q?w+1RtBjk+Bi+qJddOVF+i9bU+w1aiSQ0Jl60xtUHnu+/grSz/aLX3g6lRWfL?=
 =?us-ascii?Q?lLabWIt+1btTAEQkRdV1zbHgdFrN0KAwS//dq0XO?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: eed8b8bd-a797-436a-9fca-08dc15eed5a1
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jan 2024 17:24:31.1819
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 65ZSkGLNsas1XJumnyPFGsED8roHz1DZThMIrS2CMWw3iBM5E1SOKD7apatBBEcY
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR12MB6292

On Sun, Nov 26, 2023 at 10:34:23PM -0800, Yi Liu wrote:
> @@ -534,7 +537,17 @@ iommufd_device_do_replace(struct iommufd_device *idev,
>  static struct iommufd_hw_pagetable *do_attach(struct iommufd_device *idev,
>  		struct iommufd_hw_pagetable *hwpt, struct attach_data *data)
>  {
> -	return data->attach_fn(idev, hwpt);
> +	if (data->pasid == IOMMU_PASID_INVALID) {
> +		BUG_ON((data->attach_fn != iommufd_device_do_attach) &&
> +		       (data->attach_fn != iommufd_device_do_replace));
> +		return data->attach_fn(idev, hwpt);
> +	} else {
> +		BUG_ON((data->pasid_attach_fn !=
> +			iommufd_device_pasid_do_attach) &&
> +		       (data->pasid_attach_fn !=
> +			iommufd_device_pasid_do_replace));
> +		return data->pasid_attach_fn(idev, data->pasid, hwpt);
> +	}

Seems like the BUG_ON's are pointless

> +/**
> + * iommufd_device_pasid_detach - Disconnect a {device, pasid} to an iommu_domain
> + * @idev: device to detach
> + * @pasid: pasid to detach
> + *
> + * Undo iommufd_device_pasid_attach(). This disconnects the idev/pasid from
> + * the previously attached pt_id.
> + */
> +void iommufd_device_pasid_detach(struct iommufd_device *idev, u32 pasid)
> +{
> +	struct iommufd_hw_pagetable *hwpt;
> +
> +	hwpt = xa_load(&idev->pasid_hwpts, pasid);
> +	if (!hwpt)
> +		return;
> +	xa_erase(&idev->pasid_hwpts, pasid);
> +	iommu_detach_device_pasid(hwpt->domain, idev->dev, pasid);
> +	iommufd_hw_pagetable_put(idev->ictx, hwpt);
> +}

None of this xarray stuff looks locked properly

Jason