Received-SPF: pass (google.com: domain of linux-kernel+bounces-187015-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
From: "Tian, Kevin" <kevin.tian@intel.com>
To: Nicolin Chen <nicolinc@nvidia.com>, Jason Gunthorpe <jgg@nvidia.com>
CC: "will@kernel.org" <will@kernel.org>, "robin.murphy@arm.com"
	<robin.murphy@arm.com>, "suravee.suthikulpanit@amd.com"
	<suravee.suthikulpanit@amd.com>, "joro@8bytes.org" <joro@8bytes.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"iommu@lists.linux.dev" <iommu@lists.linux.dev>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>, "linux-tegra@vger.kernel.org"
	<linux-tegra@vger.kernel.org>, "Liu, Yi L" <yi.l.liu@intel.com>,
	"eric.auger@redhat.com" <eric.auger@redhat.com>, "vasant.hegde@amd.com"
	<vasant.hegde@amd.com>, "jon.grimm@amd.com" <jon.grimm@amd.com>,
	"santosh.shukla@amd.com" <santosh.shukla@amd.com>, "Dhaval.Giani@amd.com"
	<Dhaval.Giani@amd.com>, "shameerali.kolothum.thodi@huawei.com"
	<shameerali.kolothum.thodi@huawei.com>
Subject: RE: [PATCH RFCv1 07/14] iommufd: Add viommu set/unset_dev_id ops
Thread-Topic: [PATCH RFCv1 07/14] iommufd: Add viommu set/unset_dev_id ops
Thread-Index: AQHajVVq/X0HjqCDvU+a/LkdwhCQ/LGT28sAgADojoCAD8kmgA==
Date: Thu, 23 May 2024 05:44:40 +0000
Message-ID: <BN9PR11MB52764728D4A8F54CCDE9B6C38CF42@BN9PR11MB5276.namprd11.prod.outlook.com>
References: <cover.1712978212.git.nicolinc@nvidia.com>
 <6e57d7b5aa1705bdd547b1cd2aca93d3bf70dfa4.1712978212.git.nicolinc@nvidia.com>
 <ZkDWXnPW7CaX5TtA@nvidia.com> <ZkGZc5dvLigXcWib@nvidia.com>
In-Reply-To: <ZkGZc5dvLigXcWib@nvidia.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?5Dw3qoBm9YcHo4WfRB7CKqRNLsI3SI4GVYfU/I4Zy5LZolTS4azhm6qs+yu4?=
 =?us-ascii?Q?AUVtxiNkwpcm9Urah99yOo1AlG+9mH+22cAJZgYWGtt/Z1Hvh9usW02Rrh2z?=
 =?us-ascii?Q?jBuetzjZqisTio9Qv5nlkKYEBSTPsZKRIIZFnTSNONCb8Ww1VsFkUl/n1NqX?=
 =?us-ascii?Q?I8oxvoydLjAELaqHn9ZSaxsvVgYFr6AcxjFYORWhFEG96kfVWDyR8G61nCWM?=
 =?us-ascii?Q?0YLsDG67LwIA3FeaPcVNE1UysJO4Ghjo5ONvQC8/6ch0QRqT/Xkq9Bfo0Ppt?=
 =?us-ascii?Q?odbUUSzqRXpBb4iODBgk3qku45UUVTJiHePSwocIBmZ+8jo/v5Pn3uKj6JG5?=
 =?us-ascii?Q?FZsM7tmQFMoaHdT0SfsCJqsNvEZc2I/pCTE3X1ckuOPliHXadOGjcwPPfrYt?=
 =?us-ascii?Q?Pml5ey0BgHEspWqZSBvgfhXr9Y6MnfzYQOpi39UkI8p02/Ff+hyQV8xQ8H/9?=
 =?us-ascii?Q?J9XsEOuPMjUq/wfc54FaPyo77Ut8I6uFepnJZGALVTIGW4cvJ4Z8JlfDbKmI?=
 =?us-ascii?Q?bZO3Mqy4G61tZKL2+thzUKBZnWlhdLzwEJa0hXbhx2DPfh7JaoWfCx55xltb?=
 =?us-ascii?Q?IQk722fnpAx/PTJELwrAWbfu8ti7ulCSF8pVPYvYW7oIxWSJTI2ESvmqeIW9?=
 =?us-ascii?Q?fa4aAbF1kZESckQwSFVyJvgbRTLTCzuZucFTA/vgM5JGyz62/gJDsiNxgI7D?=
 =?us-ascii?Q?Hh+ovf5US/hTMMJYWU72A5RxY/LWzIaDl3adyo8VdOVpaTVuq7vpI/2eLO11?=
 =?us-ascii?Q?qbczLfoBFqx5zQK5pTy9zYgGnT6HP06kcTMjGXmm5tHjglvszvzv/rj9gnLr?=
 =?us-ascii?Q?51f9VpP54iqUqu8/oVw/83qJgNRL3M2oyeX/4J7ifeweEvY6f2fzL+dUNd4/?=
 =?us-ascii?Q?0mJjKpJbllmD3TdaKvXHYEuh28FKuCpX4xiRum+2TtCd21bRcJMQf31c+I21?=
 =?us-ascii?Q?vVa31eNoLzqZOS3TS7RxG2hV5Y+qEvjVN+aYDGyfYG/uCnDsTDDWOx3Kv9YH?=
 =?us-ascii?Q?fo8I+ZU0FHzuVqCV/XC6gRNokzexu5PntYRC9+bd13A7fJnsrNIJnrYrIGVJ?=
 =?us-ascii?Q?TXCv07ZjKRIkN2tPmmulvke4vhlJbEvJPbtDFTQ1i8ZKuLDFVoji1fEd4rJ6?=
 =?us-ascii?Q?KypgJUuv0Hq7s2J842QRTKJDM7eRaPSFPS4HgK2a9fZ8ZsVktpznUMYPluXf?=
 =?us-ascii?Q?rceEu5M4oVkb/8+OaJBk4p3REELKmT6zXLXPmEQT0ycwP2VaxjUyvnOPqCfa?=
 =?us-ascii?Q?7+AKKHcQpnxP7GVeyuYKqDWow07g37Y9jjGbcdL1gMpTo4ICKsSX7vEZ6NeQ?=
 =?us-ascii?Q?wcekl2MuZjxuOKauHimx67HbQjRMv1ePELC+AK64Box37Uf6BOEs/u+4MwkV?=
 =?us-ascii?Q?CWwl3h5GJ0XopnqhTsjlvv5Y3pK/ErVru0OIO8f/OGgoauyWkIPIkEetzKyX?=
 =?us-ascii?Q?ftvpx5lEH2VMQu0FmG7K2OYvAT3xQygMTr+xXJoXYur9ZH3ew3JnyIv2M9s8?=
 =?us-ascii?Q?S0HQZu4lzMDBUBuCv55UQQ6c2/dXQ94eciM7p5ogSBolAo54xsWcr4DCN4IG?=
 =?us-ascii?Q?jdbbReryqFpLrk2eKjSlGHY0+N0K06xIluml0k5b?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
List-Id: <linux-kernel.vger.kernel.org>
List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org>
List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org>
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e1525e81-6f2e-4fbc-1f3c-08dc7aeb709a
X-MS-Exchange-CrossTenant-originalarrivaltime: 23 May 2024 05:44:40.4604
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: emBmnH+sQ/lh/h25mVOlRFFy/NapD378PL9FAQzhSyOXhNuzOuzGg0gCOfQmqgb/yNbevIu+wZEgZYUa5D2hHw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR11MB7927
X-OriginatorOrg: intel.com

> From: Nicolin Chen <nicolinc@nvidia.com>
> Sent: Monday, May 13, 2024 12:39 PM
>=20
> On Sun, May 12, 2024 at 11:46:54AM -0300, Jason Gunthorpe wrote:
> > On Fri, Apr 12, 2024 at 08:47:04PM -0700, Nicolin Chen wrote:
> > > Add a pair of ops to set and unet device's virtual ID that belongs to
> > > a viommu object. They will be used, in the following patch, by iommuf=
d
> > > to support some HW-acceleration feature from the host level.
> > >
> > > For instance, every device behind an ARM SMMU has a Stream ID. The ID
> > > is used by ATC invalidation commands so SMMU HW can direct
> invalidation
> > > requests to the corresponding PCI device where the ID belongs to. In =
a
> > > virtualization use case, a passthroughed device in the VM will have a
> > > virtuail Stream ID, used by the ATC invalidation commands in the gues=
t
> > > system. NVIDIA's CMDQV extension for SMMUv3 provides a v-interface
> to
> > > execute the guest-level ATC invalidation commands directly, yet needs
> > > the HW to be aware of its virtual Stream ID so it can replace with it=
s
> > > physical Stream ID.
> >
> > I imagine using this as well for the ATC invalidation commands. It
> > would be very easy and simplifying if the command fixup just extracted
> > the vSID from the ATC invalidation and used an xarray to turn it into
> > a pSID and then pushed the resulting command.
>=20
> You mean the nested SMMU series right? Actually the set_dev_id
> ioctl was a part of that until we wanted to try DEV_INVALIDATE.
>=20
> So again, yes, it makes sense to me that we move viommu and the
> set_dev_id to the nested series, and then drop DEV_INVALIDATE.
>=20

I'm right about to ask how the nesting series is going. Per earlier
discussion iirc the nesting series will go in before VCMDQ?