Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   "Tian, Kevin" <kevin.tian@intel.com>
To:     Nicolin Chen <nicolinc@nvidia.com>,
        Jason Gunthorpe <jgg@nvidia.com>
CC:     "Liu, Yi L" <yi.l.liu@intel.com>,
        "joro@8bytes.org" <joro@8bytes.org>,
        "alex.williamson@redhat.com" <alex.williamson@redhat.com>,
        "robin.murphy@arm.com" <robin.murphy@arm.com>,
        "baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>,
        "cohuck@redhat.com" <cohuck@redhat.com>,
        "eric.auger@redhat.com" <eric.auger@redhat.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "mjrosato@linux.ibm.com" <mjrosato@linux.ibm.com>,
        "chao.p.peng@linux.intel.com" <chao.p.peng@linux.intel.com>,
        "yi.y.sun@linux.intel.com" <yi.y.sun@linux.intel.com>,
        "peterx@redhat.com" <peterx@redhat.com>,
        "jasowang@redhat.com" <jasowang@redhat.com>,
        "shameerali.kolothum.thodi@huawei.com" 
        <shameerali.kolothum.thodi@huawei.com>,
        "lulu@redhat.com" <lulu@redhat.com>,
        "suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-kselftest@vger.kernel.org" <linux-kselftest@vger.kernel.org>,
        "Duan, Zhenzhong" <zhenzhong.duan@intel.com>
Subject: RE: [PATCH v4 09/12] iommu/vt-d: Add iotlb flush for nested domain
Thread-Topic: [PATCH v4 09/12] iommu/vt-d: Add iotlb flush for nested domain
Thread-Index: AQHZvh/uFyWMLiVXhUCSOYKFk2h4da/WraUQgAhYJoCAAMpSgIAAnN2AgABV8ACAAPMnQIAACsKAgAAAQaCAAArdAIAAc+AAgAAuy4CAAAHYgIAAEEsAgABtjjCAANwPAIAAFauAgAAk/wCAABq7AIAAbchg
Date:   Fri, 11 Aug 2023 03:52:52 +0000
Message-ID: <BN9PR11MB5276D0B3E0106C73C498B8018C10A@BN9PR11MB5276.namprd11.prod.outlook.com>
References: <BN9PR11MB5276912120F662498910A1D48C12A@BN9PR11MB5276.namprd11.prod.outlook.com>
 <DS0PR11MB7529C310FAEA61B6E7988629C312A@DS0PR11MB7529.namprd11.prod.outlook.com>
 <ZNO92PIx2IQ70+DY@nvidia.com> <ZNPlGd4/72dahSs4@Asurada-Nvidia>
 <ZNPmpW3/zDnjqxyU@nvidia.com> <ZNP0UKGU6id5wfc6@Asurada-Nvidia>
 <BN9PR11MB527683351B687B97AB84B51B8C13A@BN9PR11MB5276.namprd11.prod.outlook.com>
 <ZNUI0D7ZMvLWlBNx@nvidia.com> <ZNUa/VmeiIo0YA0v@Asurada-Nvidia>
 <ZNU6BnTgNEWlwNYQ@nvidia.com> <ZNVQcmYp27ap7h30@Asurada-Nvidia>
In-Reply-To: <ZNVQcmYp27ap7h30@Asurada-Nvidia>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?rB+YHD/ROANefPhVNLU0Yn7YIpdE0TRS44Uqg6VBEejhEST0VRpNuaD3Ueb2?=
 =?us-ascii?Q?OK1XvTsMNZlefZJccnWkSZX2xPGiaI9DGN3NVldQRxYI5uGqBuATqWKlfogy?=
 =?us-ascii?Q?AgxQ921SpGTa0SCbgYBefvhfvkx/KzyHuV1VPGKemIF078GrEwfTeGGErh+W?=
 =?us-ascii?Q?/rfoeL1kX778ReXulW3SmQQ6V4dioH4P2mfRtr9K9J4BnhbSeprjgmGN8Pe5?=
 =?us-ascii?Q?OfyJYewLMRb/fm+bHF0+VKNIPYFKByoF1x9Otus7apdiSUjrzwrvOWvvzY5h?=
 =?us-ascii?Q?9jXZQ022Q6IKXQ5Tdb0WAvqHRARpMYLHDiUPR8ooL9hYRaRjIpaiC5bLZapf?=
 =?us-ascii?Q?y1WxfwnrsOfkuFnbZE8/F2OmycpgXN7BQbIlaefGWEwgQjejRRjSSpLBuA53?=
 =?us-ascii?Q?GSjPlLxSwGdRDhxwx9ZmqUMIzD7PHofclBLvvPTE8/Yrt+w451A/YKXYI5jo?=
 =?us-ascii?Q?UuXV+Yx/wG8nHTjQ3m9ZCqd7csT7jEo8e6nqSY5i+tYdKVhnYUyenAWl+eEE?=
 =?us-ascii?Q?gJR73emVkM/31D2V4pBhRIZQKG8gToXDokI66ulIGk0H+5GMsjhZK08mqgUo?=
 =?us-ascii?Q?J2+/kHJdI1K/qGm+a5Jt1Y3uw6OgX57xmm2IsgasgLJ0TvxB7+ZMX6JXoeCv?=
 =?us-ascii?Q?lt9YzLJeiJfkzX42+jPJh3l4BJ8b4xBubFKvaXRzXs0uHQBqsk/T8PdZXKaW?=
 =?us-ascii?Q?f3/zUP3OLUfnt6kFmpwY1aBLoccCt4XC9QSaZYgW2PEF1Jozm4318zEHtjoI?=
 =?us-ascii?Q?cpXEKL0GnCohic5zKPi/J73ojffSkzr0wAUnSffqy1X2kucBt7ZyKfFCo3GT?=
 =?us-ascii?Q?f4d2dN4TypiWcQwCBh4l7j1DblCAuynSVcfc2h3ynQrF80u7tFoAyGg7YtCK?=
 =?us-ascii?Q?CfVzbnBCiRV2r1ntCy00FnaStBGndVjK4YqDVFEfJ9ksZ+TgYzzjSdyj3zo7?=
 =?us-ascii?Q?N7ztb8sYoTTifxuXDu7M6QWFLylU4Lq4K/vU0Q3qIvJQCSZXSJ+y2Q1QU28V?=
 =?us-ascii?Q?4P9gTKNuJcDQ7rM3Hdi4CbDbpAhzbmcrp5Q+fGbCBhrwLBLdlOKPTlLOcA/A?=
 =?us-ascii?Q?d77PXogWWT+4N5mf23ighqnL8DLF+g+HeREviyNub/D6FZaCeKl1O5i3tsg+?=
 =?us-ascii?Q?g+5231f2WinOqP4zjYK3x6rgefr7nV2JrE6KR65v2pOfBNHSiz8Nb7rYMK25?=
 =?us-ascii?Q?XV3G+Vse6kGTSg/9UuJ59Do38uBdOsLvXqse2BgtZpYOoIEOgPp1slM0uUby?=
 =?us-ascii?Q?zkXr+SeMKEvbrtMaSbRS3BQsL75K9fxINDT01B6wedluhR3cmFJKpVjqoX+5?=
 =?us-ascii?Q?sZZ3EQt1EUmj4sWHQUsDSDSURGvaFf/xdgvURRD7QxGr4gYLNkmkcI7tdSzy?=
 =?us-ascii?Q?UCwX1yOkDFCXFFcKoYrlpKho9jZpTxFRVu9tLR4xanxRFfT/j5kYADmgW5s0?=
 =?us-ascii?Q?+Gmmf3XowC5O8w+GcU+dTjiKWaqxI5rQb1COQ57bC30zKEx2yMcaPGYDj36G?=
 =?us-ascii?Q?igs4MZpMqVXULMBH2b0gcmDuQ8egRrhIEo2SHRuwxqEVgpt2nA4jsbTSamNz?=
 =?us-ascii?Q?cv3XO1UFiilnC1GZDosrPgPc37zXWrrLN8JJLkHU?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8cb2def5-f4fa-434b-37f3-08db9a1e7044
X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Aug 2023 03:52:52.5865
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: f7Lp6wbw0Peg0C8JsKAONhUFKvXsidaWy2g+6LpEU7/UplhIRcwyGJhLA+LEcqnUZLUNyVFtBbH8lG/6pS7zcA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR11MB6510
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        SPF_HELO_NONE,SPF_NONE,URIBL_BLOCKED autolearn=ham autolearn_force=no
        version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> From: Nicolin Chen <nicolinc@nvidia.com>
> Sent: Friday, August 11, 2023 5:03 AM
>=20
> > > > Is there a use case for invaliation only SW emulated rings, and do =
we
> > > > care about optimizing for the wrap around case?
> > >
> > > Hmm, why a SW emulated ring?
> >
> > That is what you are building. The VMM catches the write of the
> > producer pointer and the VMM SW bundles it up to call into the kernel.
>=20
> Still not fully getting it. Do you mean a ring that is prepared
> by the VMM? I think the only case that we need to handle a ring
> is what I did by forwarding the guest CMDQ (a ring) to the host
> directly. Not sure why VMM would need another ring for those
> linearized invalidation commands. Or maybe I misunderstood..
>=20

iiuc the point of a ring-based native format is to maximum code reuse
when later in-kernel fast invalidation path (from kvm to smmu driver)
is enabled. Then both slow (via vmm) and fast (in-kernel) path use
the same logic to handle guest invalidation queue.

But if stepping back a bit supporting an array-based non-native format
could simplify the uAPI design and allows code sharing for array among
vendor drivers. You can still keep the entry as native format then the
only difference with future in-kernel fast path is just on walking an array
vs. walking a ring. And VMM doesn't need to expose non-invalidate
cmds to the kernel and then be skipped.

Thanks
Kevin