From:   "Liu, Yi L" <yi.l.liu@intel.com>
To:     Jason Gunthorpe <jgg@nvidia.com>
CC:     "joro@8bytes.org" <joro@8bytes.org>,
        "alex.williamson@redhat.com" <alex.williamson@redhat.com>,
        "Tian, Kevin" <kevin.tian@intel.com>,
        "robin.murphy@arm.com" <robin.murphy@arm.com>,
        "cohuck@redhat.com" <cohuck@redhat.com>,
        "eric.auger@redhat.com" <eric.auger@redhat.com>,
        "nicolinc@nvidia.com" <nicolinc@nvidia.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "mjrosato@linux.ibm.com" <mjrosato@linux.ibm.com>,
        "chao.p.peng@linux.intel.com" <chao.p.peng@linux.intel.com>,
        "yi.y.sun@linux.intel.com" <yi.y.sun@linux.intel.com>,
        "peterx@redhat.com" <peterx@redhat.com>,
        "jasowang@redhat.com" <jasowang@redhat.com>,
        "shameerali.kolothum.thodi@huawei.com" 
        <shameerali.kolothum.thodi@huawei.com>,
        "lulu@redhat.com" <lulu@redhat.com>,
        "suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-kselftest@vger.kernel.org" <linux-kselftest@vger.kernel.org>,
        "baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>
Subject: RE: [PATCH 09/17] iommufd: Add kernel-managed hw_pagetable allocation
 for userspace
Thread-Topic: [PATCH 09/17] iommufd: Add kernel-managed hw_pagetable
 allocation for userspace
Thread-Index: AQHZPD+BoQeVhsA6akuRGDX+dgTekK7HFkkAgADsacA=
Date:   Fri, 10 Feb 2023 10:52:40 +0000
Message-ID: <DS0PR11MB7529D9A66459E54DC7A64EA2C3DE9@DS0PR11MB7529.namprd11.prod.outlook.com>
References: <20230209043153.14964-1-yi.l.liu@intel.com>
 <20230209043153.14964-10-yi.l.liu@intel.com> <Y+VbUrpPQWkOVyW/@nvidia.com>
In-Reply-To: <Y+VbUrpPQWkOVyW/@nvidia.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?ILL8P5Ch9mtEJ5hesUjxVTQKlN91nshJ4N1htLJgFS366pJvmSTOO9VA7Sv6?=
 =?us-ascii?Q?lsR3eScXEoLQvphd9V6IfSfB+JeE82PXHNiyd1i3/HfDc6wtyJheTXLyEawD?=
 =?us-ascii?Q?ZBUJig64ktx/Ui8SDI2uWQh4Gk8IZzcJrb3VrRKt5iIWhgBFsyGimnyj4DY6?=
 =?us-ascii?Q?X77/HfKVdwcq/+ChF7jlkpoHJM//OTPXGZ0hKetcz3x9+Vevt9tB3CCxdpSp?=
 =?us-ascii?Q?h4UqUM1uQ4gVxX4tYGUDFoibMIGa0tev1CpcVQYTDcUZCRJuMjtPGt0Lu4ee?=
 =?us-ascii?Q?smN2F/ByVvTIUToYXc/OuzdM5G8lWq33dYCyQKaL85Il2S0FDvRhx34gmLZQ?=
 =?us-ascii?Q?5EoqAZyHdfTTDFbjJ2aEPaZvXL8gwH+6C4hoWX+lkMSzmAnYN/kkwp63jaS0?=
 =?us-ascii?Q?t2eHLzQwF2aAsGTp2GQcKasCnlKXXciRaYvEx2TX9dHBLPiexSmYqVszi+j4?=
 =?us-ascii?Q?NhNLBWzsLOjZHXRW/nkV9yTe6ruLlx/3Z/DZ/fal22AoCWT5SBZUlG7+TKCx?=
 =?us-ascii?Q?duhckTo50vEsEa47JC1cgG39Z/rkJk45PNQo1UyNtMJg38sFLfJtzPVY1P0K?=
 =?us-ascii?Q?gbprIlkWXstgEyUvG8p0xZmlNEoLWAEIGrVMZn8F+aPzxuvoKSUgZziNLG4N?=
 =?us-ascii?Q?ATdGLzZqhRHwC3ImaoQCKg1bgARr4SfzMp82wgH5nGHiw490+FqyeuI2MFu7?=
 =?us-ascii?Q?EALTYKb/Pv4O2p0PJq3y3n8OG++9+6lNhCVaKxOSYsGQQHL8e59NICHAKf7O?=
 =?us-ascii?Q?qaD/UUZDbdZLLoxQ2Llak9dEHIVHqm5WJ92CUk2+8uaf26EL1+5fjdtOsRQ8?=
 =?us-ascii?Q?oKHKgBf6txsHxYdIuEg54FYJAKze1pQAymP/LM7lcU0ytTy8Un0Pj5IogdSK?=
 =?us-ascii?Q?0g6JHcT4sqtgLJEpwaqMiDqr8+QspfPXWuC2TzVsV6z0TW0APAF2vThTLWTD?=
 =?us-ascii?Q?c/x7A4pmhaCPqA0Z2dupuS/rac/sRLulfC4gQrigCVWQ2DsMg3BBCuL23+bF?=
 =?us-ascii?Q?mJupQvib89xexf9qxHr9HAagikaH0PgNXvut9vxNdewAzgZHjYflWiSBA9e4?=
 =?us-ascii?Q?1SMv6V8tvWXBOTvQ8reVg0B+idl6eSsJZWayO73lYJQaoCe9faj4pp8zG00Q?=
 =?us-ascii?Q?bL1mAOK3VSEUh4XuM6WD22GDBLgdLji7QyIzF71rBdGJiwLfOO7WtoCb3GDE?=
 =?us-ascii?Q?3KkL/C+ZJYntP6npOw7U+CnXY6f7HAsebLTvPyJYxm3bfypwZZLOknmrKw1T?=
 =?us-ascii?Q?Hg0Othb4EbZa05OWT+VpeDne6TOuf4fcA+2qw2T/OGRf/ajn2yXKWdjOiVcQ?=
 =?us-ascii?Q?4OoqwziKe6mK33qlAnHw+O6bRg3o6R/5ZS254vxk8STVhFPWm6dMGMvtjDM0?=
 =?us-ascii?Q?/ICMfAMFEdQ5FeeyjOwpL81eMZf6vgxeG2x/hEW3/yFqztDGgWLVhwb6yKGI?=
 =?us-ascii?Q?CONLFXc2N1G92rfTrHu23kMgwy+HesTV9WUOHwfGvmuXO9wE9v2lE19Ob7s0?=
 =?us-ascii?Q?Ih8uknKa/74FASz0C531tH8QLTGCeA7hesOBcKz0lVTDTQenM6pbtR29XuGd?=
 =?us-ascii?Q?+cWVEkfZNFF9hNrnzQJ6dIbqGIQwlQrKfxJw7nfB?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7529.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 79f5fd62-e4e0-4159-195a-08db0b54ee17
X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Feb 2023 10:52:40.2551
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: P7lg7e5F1I8/gj28itTGGfC9a/Td3aLsTL0/R62Gpz5q+rIYkMx+OzcZSzO1eJK/ke2yCuVme0h0Z1YHd0rNWQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR11MB7438
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Friday, February 10, 2023 4:45 AM
>=20
> On Wed, Feb 08, 2023 at 08:31:45PM -0800, Yi Liu wrote:
> > Today iommufd allocates the kernel-managed hw_pagetabe implicitly
> when
> > device is attached to an IOAS. This links the hw_pagetable to IOPT with=
in
> > IOAS.
> >
> > However, this is not the perfect way. It makes much sense to let
> userspace
> > explicitly request hw_pagtable allocation via iommufd. The reason is ev=
en
> > though the hw_pagetable is kernel-managed, the mappings are feed by
> > userspace. Also, this makes the lifecircle of kernel-managed
> hw_pagetable
> > more clear during usage. This is very important in the usage of nested
> > translation, in which the kernel-managed hw_pagetable would be used as
> the
> > stage-2 hw_pagetable. In such case, both stage-1 and stage-2
> hw_pagetable
> > should be allocated by userspace to ensure the life-circle.
> >
> > This adds an ioctl IOMMU_HWPT_ALLOC for the hw_pagetable allocation.
> For
> > kernel-managed hw_pagetable, userspace should provide an IOAS ID in
> the
> > allocation request.
> >
> > Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> > Signed-off-by: Yi Liu <yi.l.liu@intel.com>
> > ---
> >  drivers/iommu/iommufd/device.c          |  11 ++-
> >  drivers/iommu/iommufd/hw_pagetable.c    | 116
> ++++++++++++++++++++++++
> >  drivers/iommu/iommufd/iommufd_private.h |  15 +++
> >  drivers/iommu/iommufd/main.c            |   3 +
> >  include/uapi/linux/iommufd.h            |  48 ++++++++++
> >  5 files changed, 191 insertions(+), 2 deletions(-)
>=20
> This patch and its requirements should all be first in the series. A
> mini series who's only job is to add IOMMU_HWPT_ALLOC
>=20
> Then patches to add IOMMU_HWPT_INVALIDATE
>=20
> Then the vt-d implementation of all this, including the vt-d specific
> changes to the uapi/etc.

Ok. will reorder in next version.

Regards,
Yi Liu