Received-SPF: pass (google.com: domain of linux-kernel+bounces-188713-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Date: Fri, 24 May 2024 10:12:06 -0300
From: Jason Gunthorpe <jgg@nvidia.com>
To: "Tian, Kevin" <kevin.tian@intel.com>
Cc: Nicolin Chen <nicolinc@nvidia.com>, "will@kernel.org" <will@kernel.org>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>,
	"suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com>,
	"joro@8bytes.org" <joro@8bytes.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"iommu@lists.linux.dev" <iommu@lists.linux.dev>,
	"linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>,
	"linux-tegra@vger.kernel.org" <linux-tegra@vger.kernel.org>,
	"Liu, Yi L" <yi.l.liu@intel.com>,
	"eric.auger@redhat.com" <eric.auger@redhat.com>,
	"vasant.hegde@amd.com" <vasant.hegde@amd.com>,
	"jon.grimm@amd.com" <jon.grimm@amd.com>,
	"santosh.shukla@amd.com" <santosh.shukla@amd.com>,
	"Dhaval.Giani@amd.com" <Dhaval.Giani@amd.com>,
	"shameerali.kolothum.thodi@huawei.com" <shameerali.kolothum.thodi@huawei.com>
Subject: Re: [PATCH RFCv1 07/14] iommufd: Add viommu set/unset_dev_id ops
Message-ID: <20240524131206.GS20229@nvidia.com>
References: <ZkOI8ztR1mUMJ8oe@nvidia.com>
 <ZkQW6/OAQ8MzN6Go@nvidia.com>
 <20240521182448.GN20229@nvidia.com>
 <BN9PR11MB5276EEC2A1DDA9C92F96737C8CF42@BN9PR11MB5276.namprd11.prod.outlook.com>
 <20240523150120.GO20229@nvidia.com>
 <BN9PR11MB5276A896D215E1FA61DECDC38CF52@BN9PR11MB5276.namprd11.prod.outlook.com>
 <ZlAI5d6ktg7yncLE@nvidia.com>
 <BN9PR11MB5276422CE4B77D262CD292208CF52@BN9PR11MB5276.namprd11.prod.outlook.com>
 <ZlAsXatxsopbz738@nvidia.com>
 <BN9PR11MB52765A6F6044EEF52F74230C8CF52@BN9PR11MB5276.namprd11.prod.outlook.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <BN9PR11MB52765A6F6044EEF52F74230C8CF52@BN9PR11MB5276.namprd11.prod.outlook.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?ccDLoTU9N9tP97tLPsl/a2EdhIZV75Et1yCJQQgS1718awwU3cG6gNFQ4VBG?=
 =?us-ascii?Q?0R14hoVstK3HPAkvjVoye7HLATk8Qso3GpAPlb43gvpIohcGryTBHkYXevF+?=
 =?us-ascii?Q?ERN1inWbth8XACUF/JBaaHzltT30v1R0NQNSUsQQzwBe/ZQRk4XD4kTCQt2W?=
 =?us-ascii?Q?9Mpbx7HAc2dut4Z6L5+Zgzj33z7oe1Wuw4okQlpcektxvW+/65mOEdCTRAGS?=
 =?us-ascii?Q?q4F0nsIx675w4Me+01gANHwv++Pvx7gJ3AobnuEycoZmj3u33TkD98zrwnZu?=
 =?us-ascii?Q?OvayopHiwffzI3aO/aGKUoS6hh5VH+4yvBYyMuQei2hM3e7bQGLlqrXk1MzX?=
 =?us-ascii?Q?resCMMYf2OxmuBjmQyh4HBxeMuvDWJ/aazBc3jttbjMkXJ2H1pSRv+gKhK5x?=
 =?us-ascii?Q?amjZmYEaR8q3p68uVrTK5cH4203offx6H2c97aUktQeCyqXIl+fiOsv+AyQx?=
 =?us-ascii?Q?UT2h2ED675c7SOytiiOK4+5q6HV5Z51LI9UE+Q4SHYkIIHWXIUrzh5hfSepI?=
 =?us-ascii?Q?aW66AeNeO91KQaA198ABwHboEcK9bSLhOB2+eXv+GQ5KHmTUznRhjhvJ0stE?=
 =?us-ascii?Q?C2cXEgiLgRxuPJ8ODlANXTeKdIvho9zzdQCmd4kkd1gTQBREoDK35kI7xWqu?=
 =?us-ascii?Q?Nq+hs7FC6B6wRZ2a+Kziv6rkse5DgM2RRL4joAREwLT8CczwDTG7VzT5E/CG?=
 =?us-ascii?Q?/g+6xKd/3m6qWF0r/6/MTf2Ev7Oe25ysTXa/OPWkkkZLfFYJ+ZDfMpY2Cez7?=
 =?us-ascii?Q?UOe1uWANvFxMKwmB9KVvS9OYS20PeDjDQGu8+bqWrR/Kef/drmDR1TT3XHKf?=
 =?us-ascii?Q?iBM9SXhIAQaBYSVgR54IFZ2303xiPt4s4MhAWLzvoO3sHnACSUx5jKOWSVTL?=
 =?us-ascii?Q?ZfnTJoPZiF7qbS2y0zocTiDUGMpIGvCGqj6LOOrLBG+HwF1kJD+wjk9mmuHs?=
 =?us-ascii?Q?bnexQMh2l2hKBZlKshIjM4Q3C9LnS1DRk2qI8CG23LETzYktKjffGH8TGZLB?=
 =?us-ascii?Q?bwT9Uv+i4foC9nKLOW3BYvSmyCZJsI9fN2kY1EbxfgiLbgJsOW09mpCuQSHA?=
 =?us-ascii?Q?CeRf7HapHjArorRjBbroW0uvg7b66HcMpP6UpCJ79q5RvUII61VmdFbgzhFb?=
 =?us-ascii?Q?LClGcDCA5SWd1O/DASOzVfbJ0StAGYIkhVa4t7q3MHBfApgj1tZp3TH2EnLF?=
 =?us-ascii?Q?wNxW3OZXLsSOHGG4RTrk+nqLoGNYOjfMRdMR2bq3aUARswFwWaMWIf9ifjIT?=
 =?us-ascii?Q?vghpIUnr39hZ8NRX/oF74BFg0Ur+EcitBRhN8Rxzpe0Kr/0dzgrPihVwu7NI?=
 =?us-ascii?Q?9ydlj27liXCVWEetbs/aT5HldUwXenehx0fGNDf36q5t9slphRaimklGXu92?=
 =?us-ascii?Q?bYdWXpzqUqMUgrS1U4OMDlTDUUS0CqQeQvGhlIbHnAdLDarsx9s7RtS+s0U+?=
 =?us-ascii?Q?H4fkTijJ/gg/2VqHto0ahxbZvvb2glktZKJHARgThyhFLv3+iSThv7jyl8LJ?=
 =?us-ascii?Q?dNv1Yn4gsLRkhGaueM54EqTGpi8bn1qjwYWdgD7LyT8D2FHVImqZ39D4FeA6?=
 =?us-ascii?Q?gwaOGE1SZve9N6KIQR895tR657ZOgDkI6UAHF0+v?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 40c09d9f-f7c0-4ab2-8a29-08dc7bf31d81
X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 13:12:08.4218
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: pc1mybJfwKoccYtu/dV6rpVgOIHVqW+pwMiNZBPqJlidtiYwi5kk40FXUPg/9Gv5
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB6079

On Fri, May 24, 2024 at 07:21:59AM +0000, Tian, Kevin wrote:

> My point was based on Jason's example about 3 VMIDs:
> 
>    hwpt_alloc(deva, nesting_parent=true) = shared_s2
>    viommu_alloc(deva, shared_s2) = viommu1
>    viommu_alloc(devb, shared_s2) = viommu2
>    hwpt_alloc(deva, viommu1, vste) = deva_vste
>    hwpt_alloc(devb, viommu2, vste) = devb_vste
>    attach(deva, deva_vste)
>    attach(devb, devb_vste)
>    attach(devc, shared_s2)
> 
> for devc it could be:
>    hwpt_alloc(deva, viommu1, vproxy_s1) = devc_proxys1
>    attach(devc, devc_proxys1)
> 
> then devc will reuse VMID of viommu1 and we save one VMID.

I think that can work, at least I haven't thought of a reason why the
non-vIOMMU device's VMID couldn't be shared with one of the
vIOMMUs. 

It is all quite a contrived case, I dont't expect people to want to do
anything like this, it is just showing that the API is properly setup
that it could be done. Userspace is not restricted to a single VIOMMU.

> Does that not work so we need create another viommu to hold the
> proxy identity s1 then still need a 3rd VMID?

Yes, I think that can be OK.

Jason