Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37;
From:   "Tian, Kevin" <kevin.tian@intel.com>
To:     "Liu, Yi L" <yi.l.liu@intel.com>,
        "alex.williamson@redhat.com" <alex.williamson@redhat.com>,
        "jgg@nvidia.com" <jgg@nvidia.com>,
        "robin.murphy@arm.com" <robin.murphy@arm.com>,
        "baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>
CC:     "joro@8bytes.org" <joro@8bytes.org>,
        "cohuck@redhat.com" <cohuck@redhat.com>,
        "eric.auger@redhat.com" <eric.auger@redhat.com>,
        "nicolinc@nvidia.com" <nicolinc@nvidia.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "mjrosato@linux.ibm.com" <mjrosato@linux.ibm.com>,
        "chao.p.peng@linux.intel.com" <chao.p.peng@linux.intel.com>,
        "yi.y.sun@linux.intel.com" <yi.y.sun@linux.intel.com>,
        "peterx@redhat.com" <peterx@redhat.com>,
        "jasowang@redhat.com" <jasowang@redhat.com>,
        "shameerali.kolothum.thodi@huawei.com" 
        <shameerali.kolothum.thodi@huawei.com>,
        "lulu@redhat.com" <lulu@redhat.com>,
        "suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-kselftest@vger.kernel.org" <linux-kselftest@vger.kernel.org>,
        "Duan, Zhenzhong" <zhenzhong.duan@intel.com>,
        "Martins, Joao" <joao.m.martins@oracle.com>
Subject: RE: [RFC 3/3] vfio/pci: Expose PCIe PASID capability to userspace
Thread-Topic: [RFC 3/3] vfio/pci: Expose PCIe PASID capability to userspace
Thread-Index: AQHZ8FxBOJ6q7SodUkevUoT08ZF/YrAuTq0g
Date:   Wed, 27 Sep 2023 08:07:54 +0000
Message-ID: <BN9PR11MB5276375C4BE33AC0632321A68CC2A@BN9PR11MB5276.namprd11.prod.outlook.com>
References: <20230926093121.18676-1-yi.l.liu@intel.com>
 <20230926093121.18676-4-yi.l.liu@intel.com>
In-Reply-To: <20230926093121.18676-4-yi.l.liu@intel.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?Dho1NIsZ4hPazxZNzxDxlky3M5zPzyuTX6al1/aGgcAlmyMHhOnwNwq9kwwR?=
 =?us-ascii?Q?/3umITh7ZH2xwRG6MCTcmZOvB1uQpfU8LkwpTOf3U+jYPeqJNtUlGA86YFME?=
 =?us-ascii?Q?slG23MwjrZYTlvrYn8wa92/aLF7siDzqGjjvV2Ty9ty3Xd79uoK3ZVY7MkdW?=
 =?us-ascii?Q?ThLFRAlZ5HMilfxX2nnSBzlHMmWQitrJ2TLXylI6kx3aBc3Za9PnvLhW8IV4?=
 =?us-ascii?Q?ZVDaPyiLMT5OfuX5ZMVbHOM0dOIFQI3/Ezb8XKENx/9k07cQrSldx4/JEJyP?=
 =?us-ascii?Q?PRaj+M1Qguy84CL/S23FqZAgEQM+wD512aMhlG2lOLS7bTEef4G7M41IvSMr?=
 =?us-ascii?Q?sNzCgn5t3vchS4/jjzgAgTLomYGVLBsUx2NfgA/pLY2di0CkcUU9V4Ci5QN5?=
 =?us-ascii?Q?0Lw0yeupctckJejFbAEmE8TL3o1RFZwRcN/1/i67zuENlOrIudGe7kHn0IZm?=
 =?us-ascii?Q?MqWHeGnuWRUv67/udeQ1/DkEBzN7CD2i4B+yzEc4Cy/suP6CR8mk6lp2WFRp?=
 =?us-ascii?Q?uffXRDOzCCkwXLb0KanXVT65+wAALR/SJSesTTmop3f+DP6fKTCNkKfiC8tK?=
 =?us-ascii?Q?tzG2Wm3c+M4F4AM8avFZR81fbnhQxp886EkdY+NleZk/u+0+Nn/V8SUmKNV6?=
 =?us-ascii?Q?W6T5DzJZxsfpLE5G5ixO5Geeq9O4CAGTsSh9stXTZVtlrpXQQsxWsOLby1hm?=
 =?us-ascii?Q?sMHobHVhYITijSs2SqDADiXpjIWvuJ4AW9cDUOiAGXo1oNtSnuwI/9LUOMFJ?=
 =?us-ascii?Q?ukLR72RV6IzTlzcTPoVyv0I6XoQ0VdPQEl56m9yV9FD0XmmaP/vbFFacxy7e?=
 =?us-ascii?Q?dytQk1dsCpYkUbWQOahIrzBSPCJrMdnSfCwf8XxtuhDMUm4HyPC4NkvRKUMO?=
 =?us-ascii?Q?FY9D1UnalfCxAK/gBqPxgkNu+SYwyOmJJNQAjvqgSsPX/2/hqtufXAgKHe1x?=
 =?us-ascii?Q?O1uCOiZvIlmROFKJyGjil9Pgc/EO4gKGvI8rr89LoivLY4sCVx92ZEIPul1j?=
 =?us-ascii?Q?IZ3S7UyZcBAgdNvRE0D8/NFKzxHeZGSsVeN+Wst2I0DKlHzSF/QBLY/H9K0v?=
 =?us-ascii?Q?a4Hq5v3L/9ojBoQsotmzmfSrjhN3wxqQoQu3ic3gH4t464n3IMwjilVQg4GC?=
 =?us-ascii?Q?//B+MdFn4myNfcBGYdPIhqFJ9wcbUy4tFI8fJAugfz6ugz6iTmBQaIn62+sI?=
 =?us-ascii?Q?QQLNLEAK8Iz9oU4o3KYGef1V4fW9712FhidRi9m/VFmITD1SE9pkRadWnl1A?=
 =?us-ascii?Q?u69Fqv2drDhW+ZfQzDqwou8GLtefKxiphFAi3ltCH/rM07lu8h0Mx6QRpevj?=
 =?us-ascii?Q?f/ClrBSvqgSanumVr7Cso6sZzwvTBm6BtMto72VxK8sNM7No3AmHIwrHiiv3?=
 =?us-ascii?Q?d/j1MITg7x2W32cPylxceLe9THQrEYr4Z6a3gDOlQRCedJuM/FMQW5QMvRVh?=
 =?us-ascii?Q?gz4zcMIYlOdht0SeJ+Vl+MJWET6b0JVcYpDWpbtcmhOH5H6x6ogEuSslIK1F?=
 =?us-ascii?Q?UqycnIGwMCYTLDzf67VuHSrCGT+eX6Kdw7zen8qMVHIqrDAxKJBgI02vFoue?=
 =?us-ascii?Q?Iy4cptq+Qt+uS3U82+UuewSC6Nk0yGtn6hYpxGMQ?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 68febf0f-db79-48b4-7b29-08dbbf30da8b
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Sep 2023 08:07:54.8411
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: yJ6ES+5pzmIuCW9yKAx79nijgnU/NYq/p4udPCScdXKWHXevTkpORB0QIiniaiwmJJZ8D2dwOO5BLDdvLUp7XA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB4518
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,
        RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Wed, 27 Sep 2023 01:08:09 -0700 (PDT)

> From: Liu, Yi L <yi.l.liu@intel.com>
> Sent: Tuesday, September 26, 2023 5:31 PM
>=20
> This exposes PCIe PASID capability to userspace and where to emulate this
> capability if wants to further expose it to VM.
>=20
> And this only exposes PASID capability for devices which has PCIe PASID
> extended struture in its configuration space. While for VFs, userspace
> is still unable to see this capability as SR-IOV spec forbides VF to
> implement PASID capability extended structure. It is a TODO in future.
> Related discussion can be found in below links:
>=20
> https://lore.kernel.org/kvm/20200407095801.648b1371@w520.home/
> https://lore.kernel.org/kvm/BL1PR11MB5271A60035EF591A5BE8AC878C01A
> @BL1PR11MB5271.namprd11.prod.outlook.com/
>=20

Yes, we need a decision for VF case.

If the consensus is to continue exposing the PASID capability in vfio-pci
config space by developing a kernel quirk mechanism to find offset for
VF, then this patch for PF is orthogonal to that VF work and can go as it i=
s.

But if the decision is to have a device feature for the user to enumerate
the vPASID capability and let the VMM take care of finding the vPASID
cap offset, then better we start doing that for PF too since it's not good
to have two enumeration interfaces for PF/VF respectively.

My preference is via device feature given Qemu already includes lots of
quirks for vfio-pci devices. Another reason is that when supporting vPASID
with SIOV there are some arch constraints which the driver needs to
report to the user to follow (e.g.  don't assign ENQCMD-capable sibling
vdev's to a same guest, etc.). A device feature interface can better
encapsulate everything related to vPASID in one place.

Thanks
Kevin