Date:   Thu, 9 Feb 2023 16:45:06 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Yi Liu <yi.l.liu@intel.com>
Cc:     joro@8bytes.org, alex.williamson@redhat.com, kevin.tian@intel.com,
        robin.murphy@arm.com, cohuck@redhat.com, eric.auger@redhat.com,
        nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com,
        chao.p.peng@linux.intel.com, yi.y.sun@linux.intel.com,
        peterx@redhat.com, jasowang@redhat.com,
        shameerali.kolothum.thodi@huawei.com, lulu@redhat.com,
        suravee.suthikulpanit@amd.com, iommu@lists.linux.dev,
        linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org,
        baolu.lu@linux.intel.com
Subject: Re: [PATCH 09/17] iommufd: Add kernel-managed hw_pagetable
 allocation for userspace
Message-ID: <Y+VbUrpPQWkOVyW/@nvidia.com>
References: <20230209043153.14964-1-yi.l.liu@intel.com>
 <20230209043153.14964-10-yi.l.liu@intel.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20230209043153.14964-10-yi.l.liu@intel.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?DR1/Vkv798NuPMHaiFQwiG/THM/8q/7IBgz9sn5ynVtqYaXEBxB3CNg5ogbH?=
 =?us-ascii?Q?XaDWrkfF+4QV3ZKlm6O+/uzsUV2gminjI9gwW8elcCwqvoCzd1HJNPQyvhPu?=
 =?us-ascii?Q?PCPDAbU1bV+o057K6hpMbw0W1KYLBV8s/+OwISZCmjSCDflEjGJ6lIm4fL7h?=
 =?us-ascii?Q?o4juwP7wxnfUuRL2JrNH9/0al0ctPVM9mfmfpa+BqD9UOLcBOn7biDzZENrS?=
 =?us-ascii?Q?/fD6bPc3yHWT6keTYxOR3R7k6KJc3fsss4/Foh3m8w3/WdS+g9eRALLSiICi?=
 =?us-ascii?Q?a2RawkPLSojkioHIpcXFer1S6ioVoVGMhpqPYeTzX89Sm90EuUt8wVAwGGl/?=
 =?us-ascii?Q?+CJhk4MPcRG2PcR6N5CBE2jIA8l6S8Y2+pwnSrbAwXi5kqLJrzEzqXML/AHV?=
 =?us-ascii?Q?2czbrQVX9yerFGlJXx2o4CyshcF0RamGRd7UJNIKNKFBpkfRBftDl/8ExbjU?=
 =?us-ascii?Q?2KtIJpjtXNMS3LPo9heWpZ1UGZRyIkL5u1NUe/wmM6H3PBzbPHqSU1hEweO8?=
 =?us-ascii?Q?CqC5YnYt63Q0KbVEYtrFkXT+ghCCaNuXD5HV5xCsPKYd8cMMn+I/CL+OCTRS?=
 =?us-ascii?Q?xSZ5IUslW/0a74GjP4PZRFcIHfao/yfpEU6JrM8gbpe8gEBF45i4D8tLhiTi?=
 =?us-ascii?Q?PUz/DT+Ba0VeL6rrNKg9OjHs29CZRn/0GhEMwOVby9uOXUcTPeJOyhRyAzG4?=
 =?us-ascii?Q?QdN0KtPJ51OITIybpjs4Lpquc+7/Lkgqmmgu2eUa/a4sQg9besKb1xFNsNvk?=
 =?us-ascii?Q?pXTGSJyjw8OUoVAhA4aAKtLMWwXe/7DqtpLvEHAC2yQQ3XHzJBu1SGQHlNlf?=
 =?us-ascii?Q?MdVvanHLImpal2XU02k4Ox/F/BGIwIg5lgQqYmdtRhDxKVPdPh2hn1XJQXGz?=
 =?us-ascii?Q?wy4kcjlFiHtW+WHIslULc0DOa/XdZYlNw9bTcPv9QR91zh498SntRj2jmIkZ?=
 =?us-ascii?Q?ToBxJDt0pqPcnB6O5h5DJUvCyI3cRGmuya8E2ld3LuzubuMb+BqxoHw3DPkf?=
 =?us-ascii?Q?vfSFTLxKVqUZdBWAco6qXnCX6+BGRa7VGttskPGm2x+ePN1cqojWzD/dZnQx?=
 =?us-ascii?Q?/vgd+Lp2jQLUPb+dm5+znPkHR/T18RyreX+0nqWyuSlHRcERPZHnyFSElKwc?=
 =?us-ascii?Q?CwQOVF6e4ANnXVVlWnjRI/mPcVmQwTuBfdDEOHy08wWcy0+bbl319twiXd15?=
 =?us-ascii?Q?zmH7YeUV0364LvNvTCH1Q8u/CLtx0SBiPv4t6VG8aKNxFuPwXViavM1Koz0f?=
 =?us-ascii?Q?l9JUdHrm+aNri+L/qhPoj06xXi/gF7d7bxjB0ej4QJ048gcrECJqs8DSEK5l?=
 =?us-ascii?Q?EmcLnx7/tc80r3clVeCeH447WigUZctX65Hkehn2yqnpun38elStbZoes9EO?=
 =?us-ascii?Q?WEO42+BPTQwh+K2iJGwVZFjMUOnFtohyC96OgDPdK+GTkHfaHlZMYUOsLWOE?=
 =?us-ascii?Q?V1Sw3vMIocDQrMAUCyqTRj0gS37mT/BjVXea9dpLC3sVEGP9ozD8+2nBI4BU?=
 =?us-ascii?Q?xfjz/OtJ/fY5J7iEGs05rNl/abCr7Xv5YgdDRfjCEN4cAhlNlZkposxtVug4?=
 =?us-ascii?Q?TFn1YFc/U/nwmko+i7w5CBDZa7QsLtplObl+Isfx?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: dd027b51-2844-4fdf-91e4-08db0ade8804
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Feb 2023 20:45:08.6262
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: f9SFa2FVej0oju9ShCWvpim7u/QAVPqvE9AHlRKqW/P1gehRZeJ/ILhyueNSy6Xl
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4125
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Wed, Feb 08, 2023 at 08:31:45PM -0800, Yi Liu wrote:
> Today iommufd allocates the kernel-managed hw_pagetabe implicitly when
> device is attached to an IOAS. This links the hw_pagetable to IOPT within
> IOAS.
> 
> However, this is not the perfect way. It makes much sense to let userspace
> explicitly request hw_pagtable allocation via iommufd. The reason is even
> though the hw_pagetable is kernel-managed, the mappings are feed by
> userspace. Also, this makes the lifecircle of kernel-managed hw_pagetable
> more clear during usage. This is very important in the usage of nested
> translation, in which the kernel-managed hw_pagetable would be used as the
> stage-2 hw_pagetable. In such case, both stage-1 and stage-2 hw_pagetable
> should be allocated by userspace to ensure the life-circle.
> 
> This adds an ioctl IOMMU_HWPT_ALLOC for the hw_pagetable allocation. For
> kernel-managed hw_pagetable, userspace should provide an IOAS ID in the
> allocation request.
> 
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> Signed-off-by: Yi Liu <yi.l.liu@intel.com>
> ---
>  drivers/iommu/iommufd/device.c          |  11 ++-
>  drivers/iommu/iommufd/hw_pagetable.c    | 116 ++++++++++++++++++++++++
>  drivers/iommu/iommufd/iommufd_private.h |  15 +++
>  drivers/iommu/iommufd/main.c            |   3 +
>  include/uapi/linux/iommufd.h            |  48 ++++++++++
>  5 files changed, 191 insertions(+), 2 deletions(-)

This patch and its requirements should all be first in the series. A
mini series who's only job is to add IOMMU_HWPT_ALLOC

Then patches to add IOMMU_HWPT_INVALIDATE

Then the vt-d implementation of all this, including the vt-d specific
changes to the uapi/etc.

Jason