Received-SPF: pass (google.com: domain of linux-kernel+bounces-186507-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Date: Wed, 22 May 2024 13:46:23 -0300
From: Jason Gunthorpe <jgg@nvidia.com>
To: Nicolin Chen <nicolinc@nvidia.com>
Cc: will@kernel.org, robin.murphy@arm.com, kevin.tian@intel.com,
	suravee.suthikulpanit@amd.com, joro@8bytes.org,
	linux-kernel@vger.kernel.org, iommu@lists.linux.dev,
	linux-arm-kernel@lists.infradead.org, linux-tegra@vger.kernel.org,
	yi.l.liu@intel.com, eric.auger@redhat.com, vasant.hegde@amd.com,
	jon.grimm@amd.com, santosh.shukla@amd.com, Dhaval.Giani@amd.com,
	shameerali.kolothum.thodi@huawei.com
Subject: Re: [PATCH RFCv1 05/14] iommufd: Add IOMMUFD_OBJ_VIOMMU and
 IOMMUFD_CMD_VIOMMU_ALLOC
Message-ID: <20240522164623.GA20229@nvidia.com>
References: <cover.1712978212.git.nicolinc@nvidia.com>
 <3aa9bc1df6a2ee58a03c6ea6ededbc210a2d23a8.1712978212.git.nicolinc@nvidia.com>
 <ZkDR4Rp57cy9qSqP@nvidia.com>
 <ZkGYN36N7sh2vUmv@nvidia.com>
 <ZkOFkfHhG2h2fv/c@nvidia.com>
 <ZkQNxkYv23z7i6e0@nvidia.com>
 <20240521180555.GM20229@nvidia.com>
 <Zk04vnqJPKsVtrdY@nvidia.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Zk04vnqJPKsVtrdY@nvidia.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?AQSZ44eadGE5q86pizPfpkKwY+XKTW4O74sGoanV9EbYoLTXfZnZDIMteAAD?=
 =?us-ascii?Q?jWQiVFLw967ggECFq7lxFxL6p2Yfn+zFBZtjde2w8BDcLI/ZX9IqRTf8UCIQ?=
 =?us-ascii?Q?zmIFjdO8/+ixq26qH3A0i53+8pBVtzejPZUoDy4lC7G72/Bf5bjXwD8VVxJh?=
 =?us-ascii?Q?DiXx/Am30UrYBfvsOYAiwxvkU5mC8C37qYToAj7U+MQXOrbEjkayMzBYOU0t?=
 =?us-ascii?Q?/+Wmw5/lSHXLCBy3hPGuleI0SeZyi7n5kBJrck78reoA5OtKWndikSEKjMvY?=
 =?us-ascii?Q?47eBukbLsAmT9rjese6QUrSiMW8Qe7tdj9ATFsQdBfKnejFAZOgxOZcybsBy?=
 =?us-ascii?Q?O9NuIOVjdAePMOKHKPlgAFV+BuJVvyaLhN7yhdoBKJbe01kOdpAuK0z5rQpW?=
 =?us-ascii?Q?iCwXH/E5N4lolYB4sacrYgoRAFh0zAOWLKMzVxutgoPjKtt0IxdT9Lvn0kxx?=
 =?us-ascii?Q?QH4K9/85tTM6rnMZv+eqdoZKkEWQBZG4Iqp+CbRkDAKynncyIETh6GfeXCRh?=
 =?us-ascii?Q?TxZUMvlYp39ZMqutBr8OmdeF8UMqEkgiihq/8jUKMNF30e0iuHZeZUCFWOFk?=
 =?us-ascii?Q?QmLPsfWAPrdnI3tWX2HCl6mSvNnDbtP7N4MFhgK4RMoxSqQbl0fxMsPoVlg0?=
 =?us-ascii?Q?VzhToXYN+U1QW1vHM6r5GpS2FQaXpDcukTN2QhXieqKHOR2KJ4EMJgvgVJy0?=
 =?us-ascii?Q?hQ/5gaok2jxgnfx/Xt/Oo0MsSHIRQ7mIeCbM/IMjRakJ/wjDWc5Hfzf+mlKy?=
 =?us-ascii?Q?14aDmb9RRoZf4HoRFp/AMfeDjXaEW2JeyxE7EKx7I9sR/HD8phNioqZv0pkS?=
 =?us-ascii?Q?S1githuE+1/QOP4VvpETiUW1Q68l6ku9XU4evE7xMj50XnY1PayIYyTeftIm?=
 =?us-ascii?Q?68bsPwMMPPt3OkvnH5Qb1x27mHrTA8+KV2BUyyW4djUJpcQbNwcr3oh85NgQ?=
 =?us-ascii?Q?CTONJuPai+wzELk8zKB9sVRFU27Kqe2eR5AfqAqU7UxzxjIJQW2ztcj1VZI/?=
 =?us-ascii?Q?E5UtAGU7nyjDwuAvs1s00Ja0xY+B6z4K1VwqhL2SwtkcJ0sBskBDmrWAJfCn?=
 =?us-ascii?Q?PFuuhgbNFxTUBcQtDMDuti5M+mTnMuPpFrUUND9T5yl2F7pzidaM/y6zK9Va?=
 =?us-ascii?Q?iIa3MC1jJ2DKUq+sPCFFOYsX8JXOrCsBreAX0FJOoD5AzbB3Gxjs5xkurLM8?=
 =?us-ascii?Q?3BSJZ3UW8xbXFuc822CVNoZ+oqh9/5PjH4EbPBGiTteBMeqUhrBHUeC92UK7?=
 =?us-ascii?Q?sEmTYq92Pk7zxdlL85bNDtaM9jDVCvdYoUQ2RJtJxy9lopWSWnC6L9ysdwt4?=
 =?us-ascii?Q?5glAYQairkIYRWqK0/9h3moWmXsEm5We3u1cTwOrFOZs3sWqJVbnBFOP09O1?=
 =?us-ascii?Q?RSpU3GNijYa4WiIKRY7KWOKtEzoXIygf101tYxtQYUlg3dc4Kp0C87G/dH9k?=
 =?us-ascii?Q?2Jqk3J6vyTsjN9nuvcO+C4sL01EGqwYmOohuXfHVQ0r1zGqem7U+1Hn6i28i?=
 =?us-ascii?Q?L/2/6NbdxpG/z+EoXsT4fuEc2qQUYf87rfN4othsP92Kyatj9SIz0ifngvyT?=
 =?us-ascii?Q?2UDjE5ppwAqYT9cRtmA=3D?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 61de9dbb-a70d-4d88-9b16-08dc7a7eb86a
X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 16:46:25.9612
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 6uIjB6y3quVyMkyaKt+sldsND8fYcpEx3UJSa8pbfyjDJcCjv3qIX+Nv6ikwfCiH
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4307

On Tue, May 21, 2024 at 05:13:50PM -0700, Nicolin Chen wrote:
> Yea. VMM is always allowed to create a viommu to wrap an S2
> HWPT. Then, I assume iommufd in this case should allocate a
> viommu object if !domain_ops->viommu_alloc.

Yeah

> On one side, it may not be straightforward for a qemu viommu
> driver to hold a shared S2 hwpt, as the driver is typically
> per instance, though I think it can keep viommu to its own.
> So passing the S2 hwpt back to qemu core and tie to iommufd
> handler (ictx) makes sense.

Yes, qemu will need some per-driver-type but not per-instance storage
to make this work. Ie the ARM per-driver-type shared storage would
hold the ARM specific list of S2 hwpts.

> On the other side, there can be some future HW potentially
> supporting two+ kinds of IO page tables so a VM may have two+
> S2 hwpts? Then the core would hold a list of S2 hwpts and the
> viommu driver would need to try-n-allocate viommu against the
> list..

Yes, it is supported in the API. Userspace should try to create
viommus with all the S2 hwpts available and build a new one if it
can't, just like hwpt attachment to a device.

Jason