Received-SPF: pass (google.com: domain of linux-kernel+bounces-188706-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Date: Fri, 24 May 2024 10:03:34 -0300
From: Jason Gunthorpe <jgg@nvidia.com>
To: "Tian, Kevin" <kevin.tian@intel.com>
Cc: Nicolin Chen <nicolinc@nvidia.com>, "will@kernel.org" <will@kernel.org>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>,
	"suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com>,
	"joro@8bytes.org" <joro@8bytes.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"iommu@lists.linux.dev" <iommu@lists.linux.dev>,
	"linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>,
	"linux-tegra@vger.kernel.org" <linux-tegra@vger.kernel.org>,
	"Liu, Yi L" <yi.l.liu@intel.com>,
	"eric.auger@redhat.com" <eric.auger@redhat.com>,
	"vasant.hegde@amd.com" <vasant.hegde@amd.com>,
	"jon.grimm@amd.com" <jon.grimm@amd.com>,
	"santosh.shukla@amd.com" <santosh.shukla@amd.com>,
	"Dhaval.Giani@amd.com" <Dhaval.Giani@amd.com>,
	"shameerali.kolothum.thodi@huawei.com" <shameerali.kolothum.thodi@huawei.com>
Subject: Re: [PATCH RFCv1 04/14] iommufd: Add struct iommufd_viommu and
 iommufd_viommu_ops
Message-ID: <20240524130334.GQ20229@nvidia.com>
References: <cover.1712978212.git.nicolinc@nvidia.com>
 <8610498e3fc00000e78bb9cef6fac9f6a54978a4.1712978212.git.nicolinc@nvidia.com>
 <ZkDMSfd9BXmsFfb+@nvidia.com>
 <ZkGKKoHnykAdGJq6@nvidia.com>
 <ZkOJgvMNFaZZ06OO@nvidia.com>
 <BN9PR11MB5276423A0BFBDA8346E1ED3C8CEB2@BN9PR11MB5276.namprd11.prod.outlook.com>
 <20240522133905.GX20229@nvidia.com>
 <BN9PR11MB5276267BB1CC4AA7008AFBFC8CF42@BN9PR11MB5276.namprd11.prod.outlook.com>
 <20240523125856.GL20229@nvidia.com>
 <BN9PR11MB5276461630564C74C9369CB88CF52@BN9PR11MB5276.namprd11.prod.outlook.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <BN9PR11MB5276461630564C74C9369CB88CF52@BN9PR11MB5276.namprd11.prod.outlook.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?1oWZn3ZNGyyrlPnm+ayWvU/dpJN731zdbnxqq7iZ17jdEbfz3kfUAiN7Tt5n?=
 =?us-ascii?Q?ae7LhuOR0ih/KnZvjnO6Z3q9GbcDdLeUCezU7VH/5HBE0lwsR/L+ax6xBv40?=
 =?us-ascii?Q?a5D5N9v38HAW+gCoBKAH8o5x1erolNDmbg/t9X5XkPpKOtTUWwzf82PMqung?=
 =?us-ascii?Q?nSk8eltKRIN6VGmtQ3ApQBYKjlpslV/4MJdSLbR6uxWdOxYCqHjZ2xpZnk52?=
 =?us-ascii?Q?OGIMGA1WprVWvovWiv4vDxP87+8LoY46/7SdrSDDXQ8wOGLvUGbWTkQaTByC?=
 =?us-ascii?Q?B44VVkXCnReGi0TaxRtREu81TB7kunC6GwcMATPv8fqMrpfjmRAhib1DBk7K?=
 =?us-ascii?Q?SPltnJnz7KzZVwjH68j5wz1CLDpK8R88ARPAYoKtBv252VFwsS6/WyFboxp1?=
 =?us-ascii?Q?JfD1sNE4KpS8qW4mvTmPWWV/lznGvyFPTKaRqZ1xthg80k8WRNQuUQlgVAIw?=
 =?us-ascii?Q?PtIM2o7FIKHHoS6ZExr4uHpjRrPkzRgFKy7SOaRU430GSri0bkt+z/DQN7or?=
 =?us-ascii?Q?4RdqStm9+kv7D9qE09YcjRx87yaNU3o0i/HwbWuwwE3Ytaoc6EErrLt2rz4U?=
 =?us-ascii?Q?He68Ox3FDtinDHFXsvTsCAC2ubUfTt4jrcJnNM4QJ+vfiCMQZvIGD3whpUa9?=
 =?us-ascii?Q?jFGhBm0d7Z6JCz3ymVKTtqC9MKSAIYHVw6jDqQwaLk2pE6Hi18O0UQjwFEJp?=
 =?us-ascii?Q?Ycgwe+eE4er3YDW+aBLwkjdURr9pKIZgYrge5/wj/ed1UEiOOze639ck/jXS?=
 =?us-ascii?Q?zn5trSNdktfFTqYyc1/fpBOsK1rsmWolhmrlJOHTo7QN0NrWZmDVlvWHAWaS?=
 =?us-ascii?Q?mSeKiF6jxOCPWN+0wSp5AfZMV2BCEOfkTYoMnazE2KM9sI2tXsmd6/0/VESw?=
 =?us-ascii?Q?J0RNETtPoqzW8i606gvSlM9Qt1zjyuN7WQmYSA8Z74AO/kS+yg4YIs00xr4w?=
 =?us-ascii?Q?EV64sBC+lMU+6OGQvtYrBQokFR57HwAM69Ds4gdozgx/RVcUs2pNtga+FSgL?=
 =?us-ascii?Q?7oDvGro0jI6RenLGiKwe9fUgbkoeLJOiSu81/6/CA/I5528Dr2lJPC/n1bOK?=
 =?us-ascii?Q?CHdwRCeT8XsZ7bJOvDzi4unIitM9nl+47TLvlml1NiTVuctm+ojeAgAc85/M?=
 =?us-ascii?Q?v3XfciVA2289J9JIjtr6gKfB9rLQzLKvRb1NXTeMbSLHeR/jAMdY9QK4TKw8?=
 =?us-ascii?Q?+/SLoyo+X07GseoQ0UFakXKbAZJDOWUjOj8xjbygujP49Vq7HKTypFCOiv0w?=
 =?us-ascii?Q?381l7irDd7B1uw9WHGMTeRLSOLOAjaG9Dkh+RpI4nYc2v+ykG6LhSBl0HEsg?=
 =?us-ascii?Q?Hv8OugHm2gTPW7jLQjyA+bsHZofzNVnKWQm+TM22hAYXI4GyDu0cDPNXhR86?=
 =?us-ascii?Q?jrEjraWUIvFPWmrPBe8dZrrL1/yJIDApQkqZSbm3nUg5tBMAux9OQ8SJ0mJu?=
 =?us-ascii?Q?ede7qgs/pcLpVnwLHs3WmxviIuLtQua/Yn5HKw7jneYmnQv10dUqmmko+wSj?=
 =?us-ascii?Q?YWb2OHu+1jnIfZy6Rvxi3rxYoh/9Rsa6E8mJ8BECAVZzvcAyRcCE3J2vkqCs?=
 =?us-ascii?Q?YcbWM9IdEPdceugQYME+IYIW8SL+hGgHK+0HNhJO?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 02c56586-634c-4fa0-e98f-08dc7bf1ec51
X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 13:03:36.3652
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: vxYmvKOz9Zg0P+IrXOtna+UdVJRHFQYfJM9mlhUwLDn+/HZo8TSk2WksS1vVYd4E
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB9126

On Fri, May 24, 2024 at 02:16:34AM +0000, Tian, Kevin wrote:
> > From: Jason Gunthorpe <jgg@nvidia.com>
> > Sent: Thursday, May 23, 2024 8:59 PM
> > On Thu, May 23, 2024 at 01:43:45AM +0000, Tian, Kevin wrote:
> > > > From: Jason Gunthorpe <jgg@nvidia.com>
> > > > Sent: Wednesday, May 22, 2024 9:39 PM
> > > >
> > > > The driver will then know it should program three different VMIDs for
> > > > the same S2 page table, which matches the ARM expectation for
> > > > VMID. That is to say we'd pass in the viommu as the pt_id for the
> > > > iommu_hwpt_alloc. The viommu would imply both the S2 page table and
> > > > any meta information like VMID the driver needs.
> > >
> > > Can you elaborate the aspect about "three different VMIDs"?
> > 
> > In SMMUv3 the cache is tagged by (VMID,ASID) where ASID is completely
> > controlled by the guest.
> > 
> > Every time the guest observes a SMMUv3 instance it is allowed to
> > creates its own private ASID number space for that instance. The guest
> > could re-use ASID #1 on two instances.
> > 
> > So every SMMUv3 instance plugged into the guest needs to have its own
> > unique VMID so that the overlapping ASID's are disambiguate. The above
> > would create a VM where:
> > 
> >  deva -> vSMMUv3 #1
> >  devb -> vSMMUv3 #2
> >  devc -> No IOMMU
> 
> This assumes that each vSMMUv3 instance has only one ASID space
> i.e. the guest cannot create multiple VMID's itself?

Right, the vSMMUv3 in the guest has no support VMID at all.

> > I assume the vPPRLog will steal all the PRI before it reaches the
> > kernel, so once this is turned on Lu's path won't see anything.
> 
> Okay, then we expect this vqueue object only for HW acceleration
> while software-based fault logging is still routed via Baolu's work.

Yeah, I think it mirrors the invalidation that way, we have a SW
invalidation path and a HW path. The PRI is standards based so it is
easier to make a generic API for it.

Jason