Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   "Tian, Kevin" <kevin.tian@intel.com>
To:     Nicolin Chen <nicolinc@nvidia.com>,
        Jason Gunthorpe <jgg@nvidia.com>
CC:     Robin Murphy <robin.murphy@arm.com>,
        "will@kernel.org" <will@kernel.org>,
        "eric.auger@redhat.com" <eric.auger@redhat.com>,
        "baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>,
        "joro@8bytes.org" <joro@8bytes.org>,
        "shameerali.kolothum.thodi@huawei.com" 
        <shameerali.kolothum.thodi@huawei.com>,
        "jean-philippe@linaro.org" <jean-philippe@linaro.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: RE: [PATCH v1 14/14] iommu/arm-smmu-v3: Add
 arm_smmu_cache_invalidate_user
Thread-Topic: [PATCH v1 14/14] iommu/arm-smmu-v3: Add
 arm_smmu_cache_invalidate_user
Thread-Index: AQHZUnWe1bQRwtfhAUSI/T7BqHBa167yh7YAgAALsACAANbagIAAyRsAgAqMnjCABPKxgIAANdgAgAAeawCAAPG3IIAAOI6AgAE8z4CAA0qgoA==
Date:   Fri, 24 Mar 2023 09:02:34 +0000
Message-ID: <BN9PR11MB527622D86AE048D3394C12A28C849@BN9PR11MB5276.namprd11.prod.outlook.com>
References: <1467e666-1b6c-c285-3f79-f8e8b088718b@arm.com>
 <ZAn7uC9UweiNdGkJ@nvidia.com> <ZAqv87fjbdynVaHA@Asurada-Nvidia>
 <ZAtYphmOuEqQ1BiC@nvidia.com>
 <BN9PR11MB52768F4D3E21C5231C1A04D38CBD9@BN9PR11MB5276.namprd11.prod.outlook.com>
 <ZBhYq8o4gqFZVb6k@nvidia.com> <ZBiF1hZVrp/aJRM6@Asurada-Nvidia>
 <ZBifWkY2Lt/U1Z7R@nvidia.com>
 <BN9PR11MB5276E6E3FCA90582AA61BDDE8C819@BN9PR11MB5276.namprd11.prod.outlook.com>
 <ZBmZj2pscX0hx2kQ@nvidia.com> <ZBqjUeEjjtCclgLN@Asurada-Nvidia>
In-Reply-To: <ZBqjUeEjjtCclgLN@Asurada-Nvidia>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?VlFUz1uGUrKlqC+LWX1E1pt7w7RKBQnB8AsaH5kmEeOd0zZEUmlIq5DV7avm?=
 =?us-ascii?Q?kZjt1LQasWBZzRFiuRNz/gVyr/Gj1R4fxObxqh1KU6JRuFLWd4ecTA31Xr8c?=
 =?us-ascii?Q?+vJEbMbrhgcY4H1Z1bSQ6WZ42ieE4w00XWJS7Dl6upHr74P5PcOlT0NU0Sem?=
 =?us-ascii?Q?AXrU68aUqeFVqT+ikV5CZWMLU9m7ogYFt+g3EBVYYCh9KUNrptMV+qQBoBuE?=
 =?us-ascii?Q?3fuV9YOXjWoKu110upgb9YOPNpf0m12U24W4+1cyvLmd05wXusVlM24fYG40?=
 =?us-ascii?Q?v1iVNtpiepog5Zg9HiGg5eTN0OfOSOHg0z/uoSuHBflxzrpmoFeHlTqYCyK4?=
 =?us-ascii?Q?ew+eMMKnxjGDGib3YZHSMgEjywMEtLH/+IM+OT4Rpu6rf1QPA9kPkY4D2SLp?=
 =?us-ascii?Q?0TlZTL3w2pCzP5P2ucRWxi5l3dAHLBfNLHdxL0Hw/Kt4IwVghiZQxFIZOAkS?=
 =?us-ascii?Q?JNne/BTIwonNH0SgPZRg/hTyc1tD2TNZND9uNRG9wcSWISuzHasEed1kuxgT?=
 =?us-ascii?Q?CMA8+1Wqn6HVycJA/Ws3PD8fGiRXGbjCitM8Gc1bXIlHV9Bp8F16OXGivRjk?=
 =?us-ascii?Q?GNiI10aerM3F6GO2eayL/CdI8cQYFLGssA9aNgpRsn3qCZifn17Q7m0azO+w?=
 =?us-ascii?Q?rg7PhtvuTObHdTPZE1qkfFbSJ5G0wDGesa/rw/3OBpR9nQbvIGpOssXJIMUT?=
 =?us-ascii?Q?qs3mM2+Hax21aRErrrpwGOnwetgOHjtWf7CSv6zGg26HOxPz9rNUDHwvyHKT?=
 =?us-ascii?Q?zyAc5eCNQHMEBjul/jVNnyqVelC0Xrcf439vzAl4lYuQeyUcVuJqulDd2AaS?=
 =?us-ascii?Q?6le0/hm0F+UonVrItYCR10WqKS+soiTJiaVDXpXNRUl5dPHpyszmzOl0/BKH?=
 =?us-ascii?Q?gY+1jWtv9MXK94o9XwTe8CboKEIEvBlDAQem4DMll12TGu6CVVX/84VV2wP4?=
 =?us-ascii?Q?6i8IM8QzOKci3WDRE+QK2xKpNIsVKJBW8DE1gD8+d1yumFZogzRW3Gipr6Hs?=
 =?us-ascii?Q?XM3D/2QiERbj8NAEIjP0W9pv5WNeUIID7JZJod0qEjRdIaPpQY2DklsAtBQ6?=
 =?us-ascii?Q?JsysuGOMBt6TPTmlTlPWFuqg+bTH/BiM1bgaA95XnPZuhGi6fkJFrXISSGG6?=
 =?us-ascii?Q?QZL0nPsmZQVuQb4ltnIJRB26AYoEbeLI/2jgzlhxgLZU2ZKkT8IikYIz3bAN?=
 =?us-ascii?Q?oFTcBSZsDiV+6InAFe1gWDaf05N5ltIDeF5uViFJoUMrY1xOrnT/N9p1/7W8?=
 =?us-ascii?Q?Bt0zoK2d1X3Nv3J2VR5g+PrKST/pOk62HH6FQ0wvRAor9RR+oF9OW/5o+Joa?=
 =?us-ascii?Q?yNauCTIO6DpcF4avLno3gZBzJtSLdjw+DSX6DvnQvfEafYTS5TqEkqXteQa2?=
 =?us-ascii?Q?mcjyqm7cxwIv5FJ0WMN5Ilr55rXmF7gpixH4jwkuNGVmnUcx3v2Hjhhmupe3?=
 =?us-ascii?Q?p6Q///LZ7dKIS8YyaDvgg1uModsPJMQ7HMWuuxcgYME5+Njjx1fSWAQ7/yNy?=
 =?us-ascii?Q?hZRZc7tjzLkjNPmHAdLzGbQ4Y0jmNa3QgFmrI3G/i9quzGj1rvktC34FVoRM?=
 =?us-ascii?Q?vHttaEIgNtJy+cWR8APO1Bxq5vnCWMQrIM0jyffh?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5a95fa1b-f6ff-4600-cb19-08db2c46820a
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Mar 2023 09:02:34.3510
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: C9ScXR/oa9zRPZDFJRlqjrxEYDXl5zZnsUAbfjA91L2AUF0BROQU4gDesOlj4Ed8d6xEUKve4SvVF0FQ2NCKkQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR11MB6739
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,
        SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> From: Nicolin Chen <nicolinc@nvidia.com>
> Sent: Wednesday, March 22, 2023 2:42 PM
>=20
> On Tue, Mar 21, 2023 at 08:48:31AM -0300, Jason Gunthorpe wrote:
> > On Tue, Mar 21, 2023 at 08:34:00AM +0000, Tian, Kevin wrote:
> >
> > > > > Rephrasing that to put into a design: the IOCTL would pass a
> > > > > user pointer to the queue, the size of the queue, then a head
> > > > > pointer and a tail pointer? Then the kernel reads out all the
> > > > > commands between the head and the tail and handles all those
> > > > > invalidation commands only?
> > > >
> > > > Yes, that is one possible design
> > >
> > > If we cannot have the short path in the kernel then I'm not sure the
> > > value of using native format and queue in the uAPI. Batching can
> > > be enabled over any format.
> >
> > SMMUv3 will have a hardware short path where the HW itself runs the
> > VM's command queue and does this logic.
> >
> > So I like the symmetry of the SW path being close to that.
>=20
> A tricky thing here that I just realized:
>=20
> With VCMDQ, the guest will have two CMDQs. One is the vSMMU's
> CMDQ handling all non-TLBI commands like CMD_CFGI_STE via the
> invalidation IOCTL, and the other hardware accelerated VCMDQ
> handling all TLBI commands by the HW. In this setup, we will
> need a VCMDQ kernel driver to dispatch commands into the two
> different queues.
>=20

why doesn't hw generate a vm-exit for unsupported CMDs in VCMDQ
and then let them emulated by vSMMU? such events should be rare
once map/unmap are being conducted...