Received-SPF: pass (google.com: domain of linux-kernel+bounces-185306-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Date: Tue, 21 May 2024 15:05:55 -0300
From: Jason Gunthorpe <jgg@nvidia.com>
To: Nicolin Chen <nicolinc@nvidia.com>
Cc: will@kernel.org, robin.murphy@arm.com, kevin.tian@intel.com,
	suravee.suthikulpanit@amd.com, joro@8bytes.org,
	linux-kernel@vger.kernel.org, iommu@lists.linux.dev,
	linux-arm-kernel@lists.infradead.org, linux-tegra@vger.kernel.org,
	yi.l.liu@intel.com, eric.auger@redhat.com, vasant.hegde@amd.com,
	jon.grimm@amd.com, santosh.shukla@amd.com, Dhaval.Giani@amd.com,
	shameerali.kolothum.thodi@huawei.com
Subject: Re: [PATCH RFCv1 05/14] iommufd: Add IOMMUFD_OBJ_VIOMMU and
 IOMMUFD_CMD_VIOMMU_ALLOC
Message-ID: <20240521180555.GM20229@nvidia.com>
References: <cover.1712978212.git.nicolinc@nvidia.com>
 <3aa9bc1df6a2ee58a03c6ea6ededbc210a2d23a8.1712978212.git.nicolinc@nvidia.com>
 <ZkDR4Rp57cy9qSqP@nvidia.com>
 <ZkGYN36N7sh2vUmv@nvidia.com>
 <ZkOFkfHhG2h2fv/c@nvidia.com>
 <ZkQNxkYv23z7i6e0@nvidia.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ZkQNxkYv23z7i6e0@nvidia.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?acBHvC3IHsc99Bxde/IglpNJ4i7e7RBFQYfKOySthy6lRQ6fTvoWDx29R5vC?=
 =?us-ascii?Q?W948Q3Hs+srCu15iUwZxcsixC0PABSxoSRfT8r8Z8C/ae8Cnhlkn5ErXaMcI?=
 =?us-ascii?Q?XWdRrTzLN2sWxaWoCjeFPyvtAgPE3OZpfLWWLZ6P1Y/31Wes8sh34sZbAR10?=
 =?us-ascii?Q?37fdaRJzlymzQf38FIL1n3dWjaratBUSURv6w0Z6uls/qrHDT+/WnhoAjURn?=
 =?us-ascii?Q?/AYSG2ME3f7HtI4laeRNXs4zBmBOcAEkgkimYNaiJpkXnTEFwAG8WrWH+oxh?=
 =?us-ascii?Q?TYNjk9udSFVil4Nj2Q2U2w5b+0Go94KJ9gztW0Iw4woAmyWq+/CjTqpEdWxL?=
 =?us-ascii?Q?HPQe/29oJiAB2Z9viQYmg+tZixIa68F+3aX+T2Yj9BS7Zm7CsaAEAfK4Phkw?=
 =?us-ascii?Q?LRCtV8DXsSP6POOy4hDWVhMtpHGlsbgML6XxlCeBRV5v3fv28n96qZaOXmXI?=
 =?us-ascii?Q?40f/ctrUzKexVkWuZsbYiKI3ceZqEDQa5KQ2X7hRxKBCdExwlcJNKMW8/KyW?=
 =?us-ascii?Q?5/RJImjYj00Yz+nbT7vDVHAX21tiIBiMjZjKOIOS1hS7U0bLgNAajqCgO3kA?=
 =?us-ascii?Q?pVEGy0dVZ78TFJFdAEnoxTKNUPTYgZ7F1JSMXx79g9fqoWoIm/3rqw1NVXZs?=
 =?us-ascii?Q?OxnOaLGTTMjD6Ez/hX64F06dUrN+hXEStmZx4DzP+DL5DHow3TMMYRh5dY0+?=
 =?us-ascii?Q?W7WOtrcvP1xJ9kg52zcCD4Aaq4ycVnEnSEXIWDp+jH7NlrP83axM1TiKyCA0?=
 =?us-ascii?Q?MdOn4Lz27iFP3phKF/MtZL5yWzDVZoz+PA/JeUJ/G0FO1u8Sz8fPGjjV3WSG?=
 =?us-ascii?Q?/LW2qGUYMeGAkt7vODF9nnpctGY0p70QgnR2GW3y0eBrGFbi0abcgEOTS8si?=
 =?us-ascii?Q?Gf0CghXqipNsYIGyzqyRohSEEmj3hTMq4BwPG5F6/x7uH0nsG066gJmCrfNn?=
 =?us-ascii?Q?tDwJ9ZZuTvqp5ygGM2636dYoAz+oGpG9fZXSC2r2WugJ37LrWaZgCE0v38EU?=
 =?us-ascii?Q?YTH/Y3f9piif6p2DCBdCWsvQrYGzjwdugDXGKv9X/srcOrvoUJ4TUPIm5btN?=
 =?us-ascii?Q?/4yXeCHgr8S/10oxorEpvpr3wUT6fsjVeUHPdMBmSo78N/iMCS7qreoCccgj?=
 =?us-ascii?Q?nkJIUBU/AtNFhzrUXyIp3lCcggkQIpn00iNpVOOrRj9fMbx8TwzcIE4Ox/To?=
 =?us-ascii?Q?PE3FbR5YNpurXrYeSFIKg7zq3DMXSKXWgJKDejf0vLWqeEhvPLI0h8f3lOBI?=
 =?us-ascii?Q?E3X8rysF7p4YGP1ly7LMoG/Bmb44jEG3tGKPt+S73j5+0BcvSZ3+ChgEDbn2?=
 =?us-ascii?Q?s/0RHkn/TbmwApzLDzGVS0SVoVyer9Q5fVbVHP+3Wbu2rC81IgCPDwFC2QFN?=
 =?us-ascii?Q?nwAU/XH3CYzHRx1WPzMHLM1BkqgdbNw67Iu/V2zjE5ghsb7BX8+Ax0uiQVrD?=
 =?us-ascii?Q?FKAgMPnpJT7NAEbAvNK7iVo2zY2S0kP/9rAIocdd36Md9oS6q6gJEbzKHQ0K?=
 =?us-ascii?Q?01XwHvRuE9GTDFkA3LjXVAicj6JEcZoLjR8Um33RFTimvWlCoXL1T+wmlJWJ?=
 =?us-ascii?Q?ARgvEylRBEVbsJgvODg=3D?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 72ebb558-685b-4f1b-dd5e-08dc79c0aa98
X-MS-Exchange-CrossTenant-AuthSource: BY5PR12MB3843.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 18:05:58.4008
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: QH/9foUSZSIU7r3ROldXT/Et2C9/+lOyH9Ti4J+mqWySSXVJpnyPIa2t1QInn+tQ
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB7184

On Tue, May 14, 2024 at 06:20:06PM -0700, Nicolin Chen wrote:
> > I suspect 0 should be reserved as a non-set value for some
> > basic sanity in all these driver type enums.
> 
> We have an IOMMU_HWPT_DATA_NONE for HWPT_ALLOC to compatible
> with an S2 hwpt, since it doesn't need a data.
> 
> Maybe we can have an IOMMU_VIOMMU_TYPE_DEFAULT to be 0, for
> an IOMMU driver (e.g. VT-d) that doesn't need to handle nor
> be aware of any viommu object?

Seems like a good practice, and perhaps userspace will find value in a
generic viommu object that is always present.

> That makes a lot sense. I'd need to go through QEMU code and
> see how we will accommodate these two more naturally: likely
> the QEMU core should allocate an S2 HWPT for a VM, while the
> viommu code should allocate a VIOMMU for each instance.

I'd suggest that core qemu should allocate the S2 IOAS and pass that
to the qemu viommu driver

The qemu viommu driver should create the hwpt and then the viommu and
perhaps return the viommu or hwpt back to the core code.

If the vSTE flow above is used for identity then the qemu viommu
driver would also have to go an create vSTEs for identity and attach
them to all devices before the VM starts up. Then when the OS
activates the SMMU it would have to mirror the real vSTE from guest
memory to the kernel.

Not sure there is value in having the core qemu code directly access
the hwpt/viommu?

Jason