Received-SPF: pass (google.com: domain of linux-kernel+bounces-206308-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Date: Fri, 7 Jun 2024 11:49:17 -0300
From: Jason Gunthorpe <jgg@nvidia.com>
To: "Tian, Kevin" <kevin.tian@intel.com>
Cc: Nicolin Chen <nicolinc@nvidia.com>, "will@kernel.org" <will@kernel.org>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>,
	"suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com>,
	"joro@8bytes.org" <joro@8bytes.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"iommu@lists.linux.dev" <iommu@lists.linux.dev>,
	"linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>,
	"linux-tegra@vger.kernel.org" <linux-tegra@vger.kernel.org>,
	"Liu, Yi L" <yi.l.liu@intel.com>,
	"eric.auger@redhat.com" <eric.auger@redhat.com>,
	"vasant.hegde@amd.com" <vasant.hegde@amd.com>,
	"jon.grimm@amd.com" <jon.grimm@amd.com>,
	"santosh.shukla@amd.com" <santosh.shukla@amd.com>,
	"Dhaval.Giani@amd.com" <Dhaval.Giani@amd.com>,
	"shameerali.kolothum.thodi@huawei.com" <shameerali.kolothum.thodi@huawei.com>
Subject: Re: [PATCH RFCv1 08/14] iommufd: Add IOMMU_VIOMMU_SET_DEV_ID ioctl
Message-ID: <20240607144917.GK19897@nvidia.com>
References: <BN9PR11MB52762EA9B444DA71F551C3868CF22@BN9PR11MB5276.namprd11.prod.outlook.com>
 <ZlafEktUu5znDsNt@Asurada-Nvidia>
 <BN9PR11MB5276AC43120376A2502D3D148CF32@BN9PR11MB5276.namprd11.prod.outlook.com>
 <ZlfPPwsnY6P4SoGF@Asurada-Nvidia>
 <ZluWXYQjroz0fqqn@nvidia.com>
 <Zl03rsgZl/msa3I+@nvidia.com>
 <20240606182423.GF19897@nvidia.com>
 <ZmIDqgfINXfB0i3L@Asurada-Nvidia>
 <20240607002707.GJ19897@nvidia.com>
 <BN9PR11MB527604637EC37B3B03FBBE468CFB2@BN9PR11MB5276.namprd11.prod.outlook.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <BN9PR11MB527604637EC37B3B03FBBE468CFB2@BN9PR11MB5276.namprd11.prod.outlook.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?uP+XD6whdwKn1KtTIspcqxH82fgnh1Ahhy0TncM9x4p2X9kP9arlgBVe605D?=
 =?us-ascii?Q?UpB8yi7Klae2RvuzLns8uhhQ8Inu7WBV44jtTnZt0it7Y61IN7qYBAlcL01t?=
 =?us-ascii?Q?gh49FHkks15/cNsM7xGg6FXx3ymK1ASJ0D2IjxOtoLnc/nI0PcgXIengvSsP?=
 =?us-ascii?Q?UyRQU1mN1hqiuuYhsWl4LjNd53EzmQRNBmjv5y+xGqepNINOksRn9GBLjzjo?=
 =?us-ascii?Q?tkNp5I63G5ayftaXRjdcHLqjLe7rLb/u53KLrajHqFP0oW+9+CG2GvSg1viO?=
 =?us-ascii?Q?vmjw2frJbPnjsSLlNSVK6Uswvovbzfcb2tGNrV6Oledtq57rSC9ORKJ8EAnN?=
 =?us-ascii?Q?oHE9qD7g7gwTe48Zni3/1ll2RNGKAfpTYPHblbAXX193YFpQP97dnUtOiwzg?=
 =?us-ascii?Q?GrrMpT5r6WM6itQzGLZFmlPMLDHXvn6prYnV+Q/gvURL6+TbQ7titbvUj0nH?=
 =?us-ascii?Q?n/h5xdby6/JvSnD9p0LFxjIuYqgympkhmR4DWGjkHTKj88MShpVi6SzZiIwR?=
 =?us-ascii?Q?UeDLYax30rOTfBqC8kHXBWgxDTsNjrcuNXeJ5c1m+NyN6ymOUlWN1SjrvoKy?=
 =?us-ascii?Q?kAubfghP942xDzxUZ6IftTdjN3p39DBFEcCwacglUDC3sqYIvQLHoTvpEvCm?=
 =?us-ascii?Q?nLIcPN130Z21UuNm+4O5lJ7eZoGvCgyolF8caxyE6YwjfWLD5/yYQk/nq1YV?=
 =?us-ascii?Q?rUymk9cykGo2j3wznBboX9nA+4/xtv/JNKb0TTOVafoC10MKuMMT4O/I879f?=
 =?us-ascii?Q?w3r13/sspKIA+8cbr0QlGwyc7K11pL7zYmsP2vytS/XgYtwH3Tglf7Jpli77?=
 =?us-ascii?Q?qby3C1uy24iWxbTUvyFBxa8xKysnHV5BN7WtwZBfXuIFzK8o1/eOcNMDqspK?=
 =?us-ascii?Q?gA5JNVOq6pMOB3aM5rneRQvKXF+3IU7Lr/ZlM5LvREni7ohA66FEeEyFAW8k?=
 =?us-ascii?Q?uAP0RlP5rL++mOlS2Tyk38PkzEqis1qSzOv2RYwOr/doh3x/s1wN33Praf6a?=
 =?us-ascii?Q?qIIZghBd7z0KjYPob7ya0V+R5fbKrfYBIs15pbeVVs+TOH+vCaLtAWrLwhZa?=
 =?us-ascii?Q?zh3sCX3qJzPD37TdIFEPTsWrVI7wSc3tHIzBx9pta9ajckjdsmaAX3/oDajE?=
 =?us-ascii?Q?ELullEPgCjXhE5mJduMQ6I9w2wp3w5ZxEWKQd8C6MdTfZgbQTf+ud120ObtE?=
 =?us-ascii?Q?JNs7HR7L5Ylb6ihwUmqZzj73j97yjZxvETBH/lN9jY33zt/DRDk7JkaDszBk?=
 =?us-ascii?Q?j0eVoLianCQDSnRH6MDiBlG7tqDkxld+37CHAwo4bOZmvxnHfDveQgD2ZP26?=
 =?us-ascii?Q?MosDHLmFU6Z5iyhFRMiT0Tkoy/BOvC+1EcJHDblWhQAWPj5F/vg5qSSJHL90?=
 =?us-ascii?Q?Kwct5gT+BgyZsPyjF2IOR1S54BwwVmYXKWqDTF9QriPg3DGluiuG1hka5QWe?=
 =?us-ascii?Q?BfWdLp+z56UNu4bmMvDgNnFGsYVxR+0sNf0SdNbRwOo9SC5ptVNQe2E4tyrO?=
 =?us-ascii?Q?ncWMciUB1DPLqsjvmelzmTIgiV0MiKeZrQCJm2QzUg2rVDTVpA1f9PDm9m3S?=
 =?us-ascii?Q?p7I7thhFSWy3ssDBnQ5h1mdBIWIwdQlJyxs97zWl?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 192f64b3-b2bd-469d-643f-08dc87010255
X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2024 14:49:18.5078
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: xPRHPqhnIwPmzm7B/mIDFFQUwIJ3SHRm4xlQWSEV+/kNWxEAfVKvDr9UAqubn8lC
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4078

On Fri, Jun 07, 2024 at 12:36:46AM +0000, Tian, Kevin wrote:
> > From: Jason Gunthorpe <jgg@nvidia.com>
> > Sent: Friday, June 7, 2024 8:27 AM
> > 
> > On Thu, Jun 06, 2024 at 11:44:58AM -0700, Nicolin Chen wrote:
> > > On Thu, Jun 06, 2024 at 03:24:23PM -0300, Jason Gunthorpe wrote:
> > > > On Sun, Jun 02, 2024 at 08:25:34PM -0700, Nicolin Chen wrote:
> > > >
> > > > > > I understand the appeal of doing this has been to minimize qemu
> > > > > > changes in its ACPI parts if we tackle that instead maybe we should
> > > > > > just not implement viommu to multiple piommu. It is somewhat
> > > > > > complicated.
> > > > >
> > > > > Would you please clarify that suggestion "not implement viommu
> > > > > to multiple piommu"?
> > > > >
> > > > > For regular nesting (SMMU), we are still doing one vSMMU in the
> > > > > VMM, though VCMDQ case would be an exception....
> > > >
> > > > This is what I mean, always do multiple vSMMU if there are multiple
> > > > physical pSMMUs. Don't replicate any virtual commands across pSMMUs.
> > >
> > > Thanks for clarifying. That also means you'd prefer putting the
> > > command dispatcher in VMM, which is what we have at this moment.
> > 
> > Unless someone knows a reason why we should strive hard to have only a
> > single vSMMU and accept some invalidation inefficiency?
> > 
> 
> migration? a single vSMMU provides better compatibility between
> src/dest...

Maybe, though I think we can safely split a single pSMMU into multiple
vSMMUs using the IOMMUFD vIOMMU interface. So if your machine model
has two vSMMUs and your physical HW has less we can still make that
work.

IOTLB efficiency will suffer though when splitting 1p -> 2v while
invalidation performance will suffer when joining 2p -> 1v.

Jason