Received-SPF: pass (google.com: domain of linux-kernel+bounces-187581-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Date: Thu, 23 May 2024 10:33:58 -0300
From: Jason Gunthorpe <jgg@nvidia.com>
To: "Tian, Kevin" <kevin.tian@intel.com>
Cc: Nicolin Chen <nicolinc@nvidia.com>, "will@kernel.org" <will@kernel.org>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>,
	"suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com>,
	"joro@8bytes.org" <joro@8bytes.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"iommu@lists.linux.dev" <iommu@lists.linux.dev>,
	"linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>,
	"linux-tegra@vger.kernel.org" <linux-tegra@vger.kernel.org>,
	"Liu, Yi L" <yi.l.liu@intel.com>,
	"eric.auger@redhat.com" <eric.auger@redhat.com>,
	"vasant.hegde@amd.com" <vasant.hegde@amd.com>,
	"jon.grimm@amd.com" <jon.grimm@amd.com>,
	"santosh.shukla@amd.com" <santosh.shukla@amd.com>,
	"Dhaval.Giani@amd.com" <Dhaval.Giani@amd.com>,
	"shameerali.kolothum.thodi@huawei.com" <shameerali.kolothum.thodi@huawei.com>
Subject: Re: [PATCH RFCv1 07/14] iommufd: Add viommu set/unset_dev_id ops
Message-ID: <20240523133358.GM20229@nvidia.com>
References: <cover.1712978212.git.nicolinc@nvidia.com>
 <6e57d7b5aa1705bdd547b1cd2aca93d3bf70dfa4.1712978212.git.nicolinc@nvidia.com>
 <ZkDWXnPW7CaX5TtA@nvidia.com>
 <ZkGZc5dvLigXcWib@nvidia.com>
 <BN9PR11MB52764728D4A8F54CCDE9B6C38CF42@BN9PR11MB5276.namprd11.prod.outlook.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <BN9PR11MB52764728D4A8F54CCDE9B6C38CF42@BN9PR11MB5276.namprd11.prod.outlook.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?qGRorE3gUWCDv75iuBg5w0BLGMEXyz0oGLmISPOn1jEPQE7wMxT5DvBFrXh9?=
 =?us-ascii?Q?Jo10AwxzpOxp7CIeiML/4U2l6MfufuJD3lfZYma33puEMI6b2YGKFSWO+n0T?=
 =?us-ascii?Q?B7z2mM61kWRdI0rObB1IE6lOi4yRO/znIBw/sGXUfQrrsNAO7X8Nr0f2jdyh?=
 =?us-ascii?Q?3MutD0Rpf9bpRmJr7h8cek8DuZ0Q1s0aiQMr534jYFXphFk0nKA+NRaUTweu?=
 =?us-ascii?Q?9KjTuvfc3q6SapqRNlKDZSF8JW6YJxK2lK2MAghF7oNk7tNK8hqtUwl4QfjQ?=
 =?us-ascii?Q?wErhJdUXZY/S7OduQ6tCPuYneyEOaGdZWeeb+EKSRkMYWuwfITU1+k5j4Ren?=
 =?us-ascii?Q?1wT/6gJlJzoYiNFu68cH0/KZ28p70wD2w8AGvpF8obdRYeDOj9oyuvgZ7pDD?=
 =?us-ascii?Q?YlD2ivVaJ3Gla/PdHBO8fBlFYxe7cS4eR2U29eaPZ8it9kdo35xCR7WdzGh7?=
 =?us-ascii?Q?yNgYOi5fQO8+REGQeE63G1gDDtZJiOBQIaPgqQWcKpb0oR+BwY4l1wr996s9?=
 =?us-ascii?Q?12+foET0YjWjXb5oIWNsAOok13MtGui1XP/gbglSCRrvIXKb2xNx5d4uA0JY?=
 =?us-ascii?Q?qH0r8VjhaaOoYkhzjxAeMyoem0yQygRHy1GIh9yNqbNodNzxyH5GIZlQKBqP?=
 =?us-ascii?Q?lmGd/S5e2p9ypHUCUFhIUfOBFvS+kpfjbdwK/D3RqVJ54vQzF5eeKE43/XdA?=
 =?us-ascii?Q?VJpiKlh/Zua9gZmLXDY40hke+OC87BMDElnotJL7iEjtBouvLG5sdLxqb3lp?=
 =?us-ascii?Q?ZWIKfNI9dpBcZQkMhMrhTPeQksBUgqSPPhlnKk6VkVy5LwJ0uzAete8Idq0P?=
 =?us-ascii?Q?zDLwwTc92769eCwpxqmjCmvE19yXGqOMKf1d638cDA5Ug7REMMcQx/dVNXmz?=
 =?us-ascii?Q?3HcsNYh10XOwb1B6UrEX03CDQA63VydSnhhPiAnQhnChYJmq+xODxQwXlwoG?=
 =?us-ascii?Q?ycuIr18YstMlaYrq4dR+OkeBh1tjjYvz+w78g6b3KCsx/k8V7BjB4E9va9fD?=
 =?us-ascii?Q?K2rwAxWeFBs57eKTUw8uGnC6RCSbWtagpAjRhPRIPZX6jDMQ9AfFpGhX568/?=
 =?us-ascii?Q?4e+IJAILFFARDvEgbT5quVlzoh4oboghGzgGmMYqnAJCBY4x3xRjxtxymuGe?=
 =?us-ascii?Q?4qFQygpDO2VQs9YXSRmBMcROwL5MtXalfIlZexhfMySM8OA5yuSBV/03NGVp?=
 =?us-ascii?Q?CmYYZzwtpGWas12RcEd6R+3RkrXCFVp0D8UWk8M/COFTPZ02Y+7TCzKVhGae?=
 =?us-ascii?Q?qaW3waOPYG1tJEAXZ/AOHZrBr+pufH7104yi4Dx56ArXVa9/bRMqnWwFJOiI?=
 =?us-ascii?Q?+QqzND9+oQTE64cBVMdRi0ThOERFrXg47pQOFHTWRIb9p5Zup3CqNYGQbW/q?=
 =?us-ascii?Q?ib9n6rkhSf54Qcm9VUptpEsC6GOjVKMF10snH6qGKAYNxwJP8aKGFcrWtKU0?=
 =?us-ascii?Q?h6v8wZrLhBWwin/Q1UKBkEEF713SeOIyIQemeCIZxaWIjGmGtuFZ4AtesHnL?=
 =?us-ascii?Q?kghm08BEU4Ywu/z7k68wOuwNzV0+SEzBOr4Qpju5i1vaHrpqeTiuk6vcJnIX?=
 =?us-ascii?Q?CMatfAZiIo8R5Ea9e6M=3D?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8aaf65fd-32ac-4cc4-b39d-08dc7b2d0135
X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 May 2024 13:34:00.5079
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Uxfi2rCD6EB1ALgEJGi9q20rhaxz/9/Q9dx0fNdpXbeVLGqaF1TrQqo2JOI64cuu
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR12MB6266

On Thu, May 23, 2024 at 05:44:40AM +0000, Tian, Kevin wrote:
> > On Sun, May 12, 2024 at 11:46:54AM -0300, Jason Gunthorpe wrote:
> > > On Fri, Apr 12, 2024 at 08:47:04PM -0700, Nicolin Chen wrote:
> > > > Add a pair of ops to set and unet device's virtual ID that belongs to
> > > > a viommu object. They will be used, in the following patch, by iommufd
> > > > to support some HW-acceleration feature from the host level.
> > > >
> > > > For instance, every device behind an ARM SMMU has a Stream ID. The ID
> > > > is used by ATC invalidation commands so SMMU HW can direct
> > invalidation
> > > > requests to the corresponding PCI device where the ID belongs to. In a
> > > > virtualization use case, a passthroughed device in the VM will have a
> > > > virtuail Stream ID, used by the ATC invalidation commands in the guest
> > > > system. NVIDIA's CMDQV extension for SMMUv3 provides a v-interface
> > to
> > > > execute the guest-level ATC invalidation commands directly, yet needs
> > > > the HW to be aware of its virtual Stream ID so it can replace with its
> > > > physical Stream ID.
> > >
> > > I imagine using this as well for the ATC invalidation commands. It
> > > would be very easy and simplifying if the command fixup just extracted
> > > the vSID from the ATC invalidation and used an xarray to turn it into
> > > a pSID and then pushed the resulting command.
> > 
> > You mean the nested SMMU series right? Actually the set_dev_id
> > ioctl was a part of that until we wanted to try DEV_INVALIDATE.
> > 
> > So again, yes, it makes sense to me that we move viommu and the
> > set_dev_id to the nested series, and then drop DEV_INVALIDATE.
> > 
> 
> I'm right about to ask how the nesting series is going. Per earlier
> discussion iirc the nesting series will go in before VCMDQ?

It has been sitting on my github, since it is now the N+1 series to
merge I will post it in a few weeks. There are still some details
yet to be typed in unfortunately.

Jason