Date:   Thu, 9 Mar 2023 11:40:16 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
Cc:     Jean-Philippe Brucker <jean-philippe@linaro.org>,
        Nicolin Chen <nicolinc@nvidia.com>,
        "robin.murphy@arm.com" <robin.murphy@arm.com>,
        "will@kernel.org" <will@kernel.org>,
        "eric.auger@redhat.com" <eric.auger@redhat.com>,
        "kevin.tian@intel.com" <kevin.tian@intel.com>,
        "baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>,
        "joro@8bytes.org" <joro@8bytes.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "yi.l.liu@intel.com" <yi.l.liu@intel.com>
Subject: Re: [PATCH v1 02/14] iommufd: Add nesting related data structures
 for ARM SMMUv3
Message-ID: <ZAn94BXkjiJKQ66O@nvidia.com>
References: <cover.1678348754.git.nicolinc@nvidia.com>
 <364cfbe5b228ab178093db2de13fa3accf7a6120.1678348754.git.nicolinc@nvidia.com>
 <20230309134217.GA1673607@myrica>
 <213a05ef06944a55a148995053b187f8@huawei.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <213a05ef06944a55a148995053b187f8@huawei.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?xCrpAjvriR2ILp+kxw/7Zc8zvtw89k5F6h/8Qx8Qwo+DP+g4isQFOfoUC+M6?=
 =?us-ascii?Q?oYnv95uTqkVswk85JvBZAtRy+LW5WttAkN2H9CvOKpo+UuWS/9xKvcj5Ia9t?=
 =?us-ascii?Q?4aKGN8jXKqDbgRzOgQ2MpIdWoCDATSqSWAa9j+63GmBfGrUSpUC4RQSorvLm?=
 =?us-ascii?Q?91ypuKeIcEsMUtKSDoc19Wv+MWOQQSRtarDtD+9Y9crP5ITgUxhrG94BoJpl?=
 =?us-ascii?Q?hq5P7Q5fp0NXjhyw4/GHjKPW+XNaYTIaH31IyrhV8FosI+gsipIoFKt3vMJA?=
 =?us-ascii?Q?6fzMzsUwZDa3GIWZ/kovBvqJYrp/zTKgyMiKZtEeRsHDQaua6yuLuARlXAH5?=
 =?us-ascii?Q?0pl9LRIhNf1dmSIW8j5iv1zZrz8PjBZi6Z8VvWAQDBaMZMhoB+edeVylDSJU?=
 =?us-ascii?Q?U7jc/y/eR7WpxBxT36OwCDXFCDSyVVlYUswjDg7x/aPnkGFSuEePYBx0o/Ci?=
 =?us-ascii?Q?D7N+sRAM1ZpIxN7spKW/M9UHYnCU8O5atDoG/xjN2wuZUVRE1HcZ8vH8YhJ3?=
 =?us-ascii?Q?C4QkPp9AgZmVskGAGT0qda7JtHGV5AeekEerXvLTjEN4yzk8Cbjqqene18o1?=
 =?us-ascii?Q?aR/s0DMKd/HzoKDWA55t6TUFvqHgUR++X4ixMdJySmaqEjXe4uKzbl5RVwhR?=
 =?us-ascii?Q?6qdAqCmhGLQ0y8aJ+nL75ajqXMrKgebrmbvVotMA4oI7B26q6Pk3n8qHYwvT?=
 =?us-ascii?Q?QEusLRX7psfEHj7rQNUTlew6noHcqm7NU4vdF1S8e6I0ijsWRhw3G6GbIDLj?=
 =?us-ascii?Q?YXujUKXxuW0cZJV65ocJAL1mUu8/LYouRKc2m58W2Qa+G8oDjzhzDxldytL/?=
 =?us-ascii?Q?HRiEJrZIddkXSyYGxDF73YE5X1aHJt7qoAPZhga5RMBrfGQL/zYcpvC8P5Vw?=
 =?us-ascii?Q?CH/xshE7gbvQkU9ekP8vfBDphAEkVqkdqHL5PU83ZxQ+s7O/Mi05lPJpKl1y?=
 =?us-ascii?Q?iAwfV/jDW0vUbY3AtFbMvoZWpMFfoLOfQ81L1hyuW51OAPLhOwhSbPAlDdwV?=
 =?us-ascii?Q?Ex44nbprV9vRHOY9iurfWnhlAeOnbtUUW4C3QrS9h9M199qVLdaFW2aU5qRh?=
 =?us-ascii?Q?Mhns2TOqV/ZFOqyjG/WOXdq7fKLX3hYy36ucH+ksNSuGWRUQePtrGF/oBpnP?=
 =?us-ascii?Q?5WkGAQkCchCiw56k80oy3eZFOI1oOv7tEtD4li2QK3omM4yN3aJfGi5Egqcf?=
 =?us-ascii?Q?LVo2IIgin1dz5R1UwM4zzh8fjT6udfQUaohpRM47DGOZD3PJISjq+srOjU2Z?=
 =?us-ascii?Q?R8DKbyuFSHeZ8QRm7yLmfJ1PBhu8e5mmCv+8BtFuBaPltnNbMI/E8UCfKraZ?=
 =?us-ascii?Q?24SnoqdAMnewCgz3viaohUczcVMBeO3Q44DNm29pry465qwCjeSgr2/omU1R?=
 =?us-ascii?Q?psWOz5+iNIvVffiDaclX6xSVlY6xaKcItKGNxaYycFu+aGlmnlXXH0qn3ZFM?=
 =?us-ascii?Q?G1UBNJwJLlZ+8bFXnGlhLg4/GxsJC+fJoKtBEp5L4q250OgwXO6g0KIBIX73?=
 =?us-ascii?Q?/jVrznryz8Z+zuZL1ciUxHwA8zDHogeqsG4d3HM+RmZjE6oeb6Ed0Xpm/lFm?=
 =?us-ascii?Q?ybgNhyLA4phP7AxFAtrXuckm7aw/biX48VWuZFbO?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 154702dd-1ea5-41dd-dcfb-08db20b49607
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Mar 2023 15:40:18.7606
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 3ZBX3gkI+QAkkzKzdIR9GlU4FXDvJraToofRQp4Q7vg/eQervXP1n4Wfd7uRhJBT
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6139
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Mar 09, 2023 at 03:26:12PM +0000, Shameerali Kolothum Thodi wrote:

> On platforms that supports BTM [1], we may need the VMID allocated by KVM.
> But again getting that from user pace doesn't look safe. I have attempted to revise
> the earlier RFC to pin and use the KVM VMID from SMMUv3 here[2].

Gurk

> " In the new design we can require from the start that creating a nesting IOMMU
> container through /dev/iommu *must* come with a KVM context, that way
> we're sure to reuse the existing VMID. "

I've been dreading this but yes I execpt we will eventually need to
connect kvm and iommufd together. The iommu driver can receive a kvm
pointer as part of the alloc domain operation to do any setups like
this.

If there is no KVM it should either fail to setup the domain or setup
a domain disconnected from KVM.

If IOMMU HW and KVM HW are using the same ID number space then
arguably the two kernel drivers need to use a shared ID allocator in
the arch, regardless of what iommufd/etc does. Using KVM should not be
mandatory for iommufd.

For ARM cases where there is no shared VMID space with KVM, the ARM
VMID should be somehow assigned to the iommfd_ctx itself and the alloc
domain op should receive it from there.

Nicolin, that seems to be missing in this series? I'm not entirely
sure how to elegantly code it :\

Jason