Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Tue, 21 Sep 2021 14:44:38 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Liu Yi L <yi.l.liu@intel.com>
Cc:     alex.williamson@redhat.com, hch@lst.de, jasowang@redhat.com,
        joro@8bytes.org, jean-philippe@linaro.org, kevin.tian@intel.com,
        parav@mellanox.com, lkml@metux.net, pbonzini@redhat.com,
        lushenming@huawei.com, eric.auger@redhat.com, corbet@lwn.net,
        ashok.raj@intel.com, yi.l.liu@linux.intel.com,
        jun.j.tian@intel.com, hao.wu@intel.com, dave.jiang@intel.com,
        jacob.jun.pan@linux.intel.com, kwankhede@nvidia.com,
        robin.murphy@arm.com, kvm@vger.kernel.org,
        iommu@lists.linux-foundation.org, dwmw2@infradead.org,
        linux-kernel@vger.kernel.org, baolu.lu@linux.intel.com,
        david@gibson.dropbear.id.au, nicolinc@nvidia.com
Subject: Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE
Message-ID: <20210921174438.GW327412@nvidia.com>
References: <20210919063848.1476776-1-yi.l.liu@intel.com>
 <20210919063848.1476776-12-yi.l.liu@intel.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20210919063848.1476776-12-yi.l.liu@intel.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?ODcqI36tS0FVjupGscPem6SzTLUQlMNFfA7u81BB1hT/E8QxdXBg7eGnp4r3?=
 =?us-ascii?Q?XnVWucAeKinhkeRUsqByfdw3/drbCXQDFfCAPn+vmeAfXZqyaZGTMc0uHhtL?=
 =?us-ascii?Q?DUfbpA2CBrFGn1C721o+URM3cyhrhtlwm6R4vQvv//dLnRn8ULJC2Xn+4oCN?=
 =?us-ascii?Q?yLmZk+AlLUzqZXJUIVVonkzdNrWK+owE6Ry2evuoGiBeuBwO+5Fn1qnDFTWg?=
 =?us-ascii?Q?S2FlMmr3/7lCuTxYFji/xXMpm0LPmv3w2InlAN1tLiChnv27/AIs/2KYWf8P?=
 =?us-ascii?Q?XlexSgmbMQHchBKvLPliqJxyg9hNTJU6qqju8VAsovI8/BzrZLyYqcJzjYPH?=
 =?us-ascii?Q?6cyzV+KoIGY/L+2vfavuSqf5zRgrNMmmQQ5Op9I7qfNwbfdospIEoXMAmmKd?=
 =?us-ascii?Q?8DCvS163VKu12j4V+wD+tcbgg79EoMxVsLSQb0rrt5q1zJVtKOR4WmnAEg1i?=
 =?us-ascii?Q?J1qByJBD/Di9+P+6gynEoRBrq86pgiBeO1iS2Y/eocnd+j06W3BI7/3dTdul?=
 =?us-ascii?Q?1XjfzRGmapdoK606usaa84vNTry3bVGmaYVfdr5TjSU8f0jIexnIvVmwKFhY?=
 =?us-ascii?Q?b31FC/Kcf3u9l6LwDQy0AqDy6zAmvPKFXRlFvsVArcYuf2A5rZTzCvM0RVxS?=
 =?us-ascii?Q?b4v4jkZjWEYyjJz5aMse57+2K5CFOVBpwrALEAlZKRSwMrOfwVxXIgHfwTXz?=
 =?us-ascii?Q?TtBAwDZkJRJkUzbY8diyDVnq1dC3WWMBQvYYkqbBwGp5Kck/L+GpNNKvMszT?=
 =?us-ascii?Q?urlKu3Ot9d7+/cDxIDQi8omnrWyR9FRGtYqppWe7TRfCHwqzSuLB1shMsMM2?=
 =?us-ascii?Q?ggNg6ncbikOpn9uYWKjOvr9XU6KPV+Pxo5ne4mm3CdMXGqSzEhqV4cKFkGm7?=
 =?us-ascii?Q?M142VJrNBsHBJuxTkgQxwqYMCF6ioDbIp+iFeHJbnTCiQIWubdpXTjj1+2n/?=
 =?us-ascii?Q?zx9G0ieKBmL/p3sh9j42gpEb2ws3mE8DQ/vURHk+vG/8fgFQORUnRO/LowMB?=
 =?us-ascii?Q?9t88oMZaEaC7BX+ZyHHJpqjeQxbAwczjImaRhTXvqkrKGAwdURSqForv5aqb?=
 =?us-ascii?Q?3F8Ew+QiXYDzJjtIlKJS+1bS2uGDGFYGhaa6L7XivygkYEqhtZs/zw2j2fV6?=
 =?us-ascii?Q?e3sXfJ3FKPPq+N5em5WgpA/fLbcH8EHyftcz/gUuHn2oxnj09sCBY8rQ2z4n?=
 =?us-ascii?Q?mjO0o1ntSFmBD72+7WMzrvsi1aj+4OCu1Jm+BRGI/e9s+vXlwjFo8yX4AYCT?=
 =?us-ascii?Q?vn1x6WWyg8oMEykVbxc/E2Wqy+J2dOcno8Yl1D4fVcavDd5UqDWE6Ctlknup?=
 =?us-ascii?Q?YHLOnBTARKGOitGnW3chYcEc?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 437b6f8e-ff70-496f-71bb-08d97d277c63
X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB5506.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 17:44:39.6621
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ZxiHgtY5UDp3RtxsXwn5NZWjKYxrWfp6jTE7fUOcqLg4Hm1sI0SADHF3/p9EuVH5
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5303
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Sun, Sep 19, 2021 at 02:38:39PM +0800, Liu Yi L wrote:
> This patch adds IOASID allocation/free interface per iommufd. When
> allocating an IOASID, userspace is expected to specify the type and
> format information for the target I/O page table.
> 
> This RFC supports only one type (IOMMU_IOASID_TYPE_KERNEL_TYPE1V2),
> implying a kernel-managed I/O page table with vfio type1v2 mapping
> semantics. For this type the user should specify the addr_width of
> the I/O address space and whether the I/O page table is created in
> an iommu enfore_snoop format. enforce_snoop must be true at this point,
> as the false setting requires additional contract with KVM on handling
> WBINVD emulation, which can be added later.
> 
> Userspace is expected to call IOMMU_CHECK_EXTENSION (see next patch)
> for what formats can be specified when allocating an IOASID.
> 
> Open:
> - Devices on PPC platform currently use a different iommu driver in vfio.
>   Per previous discussion they can also use vfio type1v2 as long as there
>   is a way to claim a specific iova range from a system-wide address space.
>   This requirement doesn't sound PPC specific, as addr_width for pci devices
>   can be also represented by a range [0, 2^addr_width-1]. This RFC hasn't
>   adopted this design yet. We hope to have formal alignment in v1 discussion
>   and then decide how to incorporate it in v2.

I think the request was to include a start/end IO address hint when
creating the ios. When the kernel creates it then it can return the
actual geometry including any holes via a query.

> - Currently ioasid term has already been used in the kernel (drivers/iommu/
>   ioasid.c) to represent the hardware I/O address space ID in the wire. It
>   covers both PCI PASID (Process Address Space ID) and ARM SSID (Sub-Stream
>   ID). We need find a way to resolve the naming conflict between the hardware
>   ID and software handle. One option is to rename the existing ioasid to be
>   pasid or ssid, given their full names still sound generic. Appreciate more
>   thoughts on this open!

ioas works well here I think. Use ioas_id to refer to the xarray
index.

> Signed-off-by: Liu Yi L <yi.l.liu@intel.com>
>  drivers/iommu/iommufd/iommufd.c | 120 ++++++++++++++++++++++++++++++++
>  include/linux/iommufd.h         |   3 +
>  include/uapi/linux/iommu.h      |  54 ++++++++++++++
>  3 files changed, 177 insertions(+)
> 
> diff --git a/drivers/iommu/iommufd/iommufd.c b/drivers/iommu/iommufd/iommufd.c
> index 641f199f2d41..4839f128b24a 100644
> +++ b/drivers/iommu/iommufd/iommufd.c
> @@ -24,6 +24,7 @@
>  struct iommufd_ctx {
>  	refcount_t refs;
>  	struct mutex lock;
> +	struct xarray ioasid_xa; /* xarray of ioasids */
>  	struct xarray device_xa; /* xarray of bound devices */
>  };
>  
> @@ -42,6 +43,16 @@ struct iommufd_device {
>  	u64 dev_cookie;
>  };
>  
> +/* Represent an I/O address space */
> +struct iommufd_ioas {
> +	int ioasid;

xarray id's should consistently be u32s everywhere.

Many of the same prior comments repeated here

Jason