Received-SPF: pass (google.com: domain of linux-kernel+bounces-28995-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Date: Wed, 17 Jan 2024 08:56:25 -0400
From: Jason Gunthorpe <jgg@nvidia.com>
To: Yi Liu <yi.l.liu@intel.com>
Cc: "Tian, Kevin" <kevin.tian@intel.com>,
	"joro@8bytes.org" <joro@8bytes.org>,
	"alex.williamson@redhat.com" <alex.williamson@redhat.com>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>,
	"baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>,
	"cohuck@redhat.com" <cohuck@redhat.com>,
	"eric.auger@redhat.com" <eric.auger@redhat.com>,
	"nicolinc@nvidia.com" <nicolinc@nvidia.com>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"mjrosato@linux.ibm.com" <mjrosato@linux.ibm.com>,
	"chao.p.peng@linux.intel.com" <chao.p.peng@linux.intel.com>,
	"yi.y.sun@linux.intel.com" <yi.y.sun@linux.intel.com>,
	"peterx@redhat.com" <peterx@redhat.com>,
	"jasowang@redhat.com" <jasowang@redhat.com>,
	"shameerali.kolothum.thodi@huawei.com" <shameerali.kolothum.thodi@huawei.com>,
	"lulu@redhat.com" <lulu@redhat.com>,
	"suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com>,
	"iommu@lists.linux.dev" <iommu@lists.linux.dev>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-kselftest@vger.kernel.org" <linux-kselftest@vger.kernel.org>,
	"Duan, Zhenzhong" <zhenzhong.duan@intel.com>,
	"joao.m.martins@oracle.com" <joao.m.martins@oracle.com>,
	"Zeng, Xin" <xin.zeng@intel.com>,
	"Zhao, Yan Y" <yan.y.zhao@intel.com>
Subject: Re: [PATCH 3/8] iommufd: Support attach/replace hwpt per pasid
Message-ID: <20240117125625.GF734935@nvidia.com>
References: <20231127063428.127436-1-yi.l.liu@intel.com>
 <20231127063428.127436-4-yi.l.liu@intel.com>
 <20240115172430.GN734935@nvidia.com>
 <BN9PR11MB52761349DFB5DAD2797C3EBF8C732@BN9PR11MB5276.namprd11.prod.outlook.com>
 <20240116125756.GB734935@nvidia.com>
 <BN9PR11MB52763DDDE39C211E761A05168C722@BN9PR11MB5276.namprd11.prod.outlook.com>
 <88e46f6c-4d64-4357-be2a-833797e6de15@intel.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <88e46f6c-4d64-4357-be2a-833797e6de15@intel.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?d43FTabOT0V8ojAC6fhjlKqv5ThPoiGYIyOhnWZVBUDtcsicHoY4nrJXgG0u?=
 =?us-ascii?Q?OTW83G8jeTQ2A3NXC2zYDRZwYZsgvPmqpPj2lLn3/CUrWdr/dADe43O2uZnm?=
 =?us-ascii?Q?MUpAWugtBWz0L2Zxo9qQpdLzjC7AlI+gHFtlTdlzjWxAUJeV8g891whs8bx8?=
 =?us-ascii?Q?gTJV/AuIrf1Fqw4rqM4YPdFvRUtCxllZC9zSxmpkReoBHWpN83B824Q0GB3k?=
 =?us-ascii?Q?tybSaId+ARARITsVyG4SuUZrBiC5WzlU5fRZMDugZXVXWxLAIzO26AKMqPOu?=
 =?us-ascii?Q?sP9XQ9wGdPEDM9p4XYUu9LFQMOgMi4Z9450wm7lBhh7/8vQq7WypANXqffCK?=
 =?us-ascii?Q?67akWyOVUCdwPdzm4zBggZTzFTFSrQjJ9HGT3gKfsp3OKUnOwPB/0ehWI0Tz?=
 =?us-ascii?Q?zOGBkkGLB2HB/GYl0T0mjXZrGPUNUUB0sBxdSDkuNUY5bjiWnCeajHCPawl8?=
 =?us-ascii?Q?F5GnpulqrysVE583F3gbSBLQibOLJimrJIX235J9F+RxshOqwMDIdZmFYmi2?=
 =?us-ascii?Q?zd8wcydP303jfRZ+xNfM+GU4tzfx0LLNG3IxVXhCR/a9noDUTQdJzTiHiNlq?=
 =?us-ascii?Q?ATlEVMVhUPMcGK10tnkrJMFLm9VLHVMX2ZWcxmgkcdPQgaOVwAWSDbACdwzW?=
 =?us-ascii?Q?rlFwqo6y8OkAxYcBf+2gp8G/frqv9tx1nDAOh+Rvr7tP/b342okbUxdAwSlF?=
 =?us-ascii?Q?hTZvuWp+VZonv2UbPLmeueTNvzTMFxPCwogTMcCJtHCEoZClcJc7CTxM7bar?=
 =?us-ascii?Q?VKPublZJnJucctR14U+Yhd7KZIAuj9hcp+dCcLW4jxjwen6bPY5g9jurw6zX?=
 =?us-ascii?Q?gVrerEe93r0qIIU0GCcD98tWdHxB8+4205GbHT0kEU5SLjJ6mqBY6i4Ln6hz?=
 =?us-ascii?Q?wBo0pdmUoqPjldPE3HR/P6nKnQbssEJ7V+tEVQTaNb+tSVFmfan1lPQhSnjN?=
 =?us-ascii?Q?q7MrNPi+lu1UFWVIhdG2+w3QtfXr06fjluzonf3WikkRu1DEXvHWbkXDsD+n?=
 =?us-ascii?Q?c1qeP7SSx0o9j1tLkJ+WQk9JQuZQUWYDxvNNq6GsuqZhUuWlBXkcrAVcY1RH?=
 =?us-ascii?Q?ZB0z/RPPowWJ5rayjn7W2XGvNB3RZEg/qH8dwbdEsKgsIuix1kGvaB9Hdquy?=
 =?us-ascii?Q?NZpYm4xzIT2NgGtVznYeyVHyIzWhjP3DIdd92SBUAnel9oZY2U8V7wvpj+pK?=
 =?us-ascii?Q?GZW2ImWh+9b7M0QGIXbR992xD3hq9lh4Y1CmnwhawQ6moDUpL4dcbVillsba?=
 =?us-ascii?Q?FZjthfcXYZkwHuNjunOEli2UiK6pZdcWEj+KUaMvOc0+249H8ZesiZeGx36g?=
 =?us-ascii?Q?KkI7iXtkHT9bs7/+IhbKiP0hMrHbyiovARXo6qm5SsRP5YSwxvbo8fqzjbwr?=
 =?us-ascii?Q?zvXPeN7vvnD7iftDtxtpTMPP6eH7+tcyOYpG1TJfUwbQVLgpXMaRgvNW1/Xq?=
 =?us-ascii?Q?w+Mxzvj8k3mnuUeo1mk6GgFxr3e38LMrm6Jmb0rdigVg4iZO7Y5fSWU9WeM4?=
 =?us-ascii?Q?zg0Dv+Hu3ik3YcJbBYhWIj/dVqODNwSlJ7LYCzpiRheyUtoTvZf5Cqg400xy?=
 =?us-ascii?Q?9jNJnSRq07uga6onTOQuire6kAV09K7Nw5hJxO0Z?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6e7feb9f-1aba-4aee-9876-08dc175bb717
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jan 2024 12:56:26.2018
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ZzQCauS/ybDS3/zRBs2E0/jpAlM0MODsDGlfh9MSfzpovoOsZNJYPHTxBWVWa0/H
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4934

On Wed, Jan 17, 2024 at 04:24:24PM +0800, Yi Liu wrote:
> Above indeed makes more sense if there can be concurrent attach/replace/detach
> on a single pasid. Just have one doubt should we add lock to protect the
> whole attach/replace/detach paths. In the attach/replace path[1] [2], the
> xarray entry is verified firstly, and then updated after returning from
> iommu attach/replace API. It is uneasy to protect the xarray operations only
> with xa_lock as a detach path can acquire xa_lock right after attach/replace
> path checks the xarray. To avoid it, may need a mutex to protect the whole
> attach/replace/detach path to avoid race. Or maybe the attach/replace path
> should mark the corresponding entry as a special state that can block the
> other path like detach until the attach/replace path update the final hwpt to
> the xarray. Is there such state in xarray?

If the caller is not allowed to make concurrent attaches/detaches to
the same pasid then you can document that in a comment, but it is
still better to use xarray in a self-consistent way.

Jason