Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) client-ip=2620:137:e000::3:1;
From:   "Duan, Zhenzhong" <zhenzhong.duan@intel.com>
To:     "Liu, Yi L" <yi.l.liu@intel.com>,
        "joro@8bytes.org" <joro@8bytes.org>,
        "alex.williamson@redhat.com" <alex.williamson@redhat.com>,
        "jgg@nvidia.com" <jgg@nvidia.com>,
        "Tian, Kevin" <kevin.tian@intel.com>,
        "robin.murphy@arm.com" <robin.murphy@arm.com>,
        "baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>
CC:     "cohuck@redhat.com" <cohuck@redhat.com>,
        "eric.auger@redhat.com" <eric.auger@redhat.com>,
        "nicolinc@nvidia.com" <nicolinc@nvidia.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "mjrosato@linux.ibm.com" <mjrosato@linux.ibm.com>,
        "chao.p.peng@linux.intel.com" <chao.p.peng@linux.intel.com>,
        "yi.y.sun@linux.intel.com" <yi.y.sun@linux.intel.com>,
        "peterx@redhat.com" <peterx@redhat.com>,
        "jasowang@redhat.com" <jasowang@redhat.com>,
        "shameerali.kolothum.thodi@huawei.com" 
        <shameerali.kolothum.thodi@huawei.com>,
        "lulu@redhat.com" <lulu@redhat.com>,
        "suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-kselftest@vger.kernel.org" <linux-kselftest@vger.kernel.org>,
        "joao.m.martins@oracle.com" <joao.m.martins@oracle.com>,
        "Zeng, Xin" <xin.zeng@intel.com>,
        "Zhao, Yan Y" <yan.y.zhao@intel.com>
Subject: RE: [PATCH 3/3] vfio: Report PASID capability via VFIO_DEVICE_FEATURE
 ioctl
Thread-Topic: [PATCH 3/3] vfio: Report PASID capability via
 VFIO_DEVICE_FEATURE ioctl
Thread-Index: AQHaIPx2oNLTxoUOTUKh4MiWO0yFiLCNv+Ug
Date:   Mon, 27 Nov 2023 07:28:21 +0000
Message-ID: <SJ0PR11MB6744DC9B7C7D0E4122F224FD92BDA@SJ0PR11MB6744.namprd11.prod.outlook.com>
References: <20231127063909.129153-1-yi.l.liu@intel.com>
 <20231127063909.129153-4-yi.l.liu@intel.com>
In-Reply-To: <20231127063909.129153-4-yi.l.liu@intel.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?IOt9lOh0cAJ4tQSdrB7Yr43JXK5dIGdaeCEeeupUxvyrjA5YKCCsdhZnu36m?=
 =?us-ascii?Q?zuLnkVaSheon3l627PyhqcOMjAPMdabtHz/esqvOKGdXTmWqNARuafUh8xIc?=
 =?us-ascii?Q?yZrVgkAOcOZXOBIFxVMWREJDqAjeRUdiHpQaFT1SoRSXsrWwYqHqLD1hSR+g?=
 =?us-ascii?Q?oZvr1FK7YCGdyMTK0uQ2+ec5yWqUW5H32N2iWoWWjDxqMLj0lFrCSSnwAIq5?=
 =?us-ascii?Q?4UsnGdbPJAHKpAmxhrYvMNsTazeVMVC3vWZE+coUW5tpATUZf0H/7kUi5fDe?=
 =?us-ascii?Q?iCRvmEinB95fEGlQD5Lrq8Emopt+ufsVYnjlDryzAD6UkddTv7o/26Jj3XvE?=
 =?us-ascii?Q?Po8lyWGa9J3FJf/VnBk5sunF/rjGxGiJpQJimCQ9xhZZoQe8A4JpfLnPQhya?=
 =?us-ascii?Q?eHwk9fd63h8YXn1IGdojVEf/Y6vzcUkzukYemx6eA2fQbkWroxJfMhzILj1u?=
 =?us-ascii?Q?jKlkhPwK3G0WMhNfimK1FX4H6Dp7IHWW5Q+j0ERVWNMOz9AM5M0S81+Q6WWk?=
 =?us-ascii?Q?Mv+y9CMrV4A1v+qlpm45CwFjf5G1+Z2NnctsOXzotAYbB3kfTsLaSfm+9vPQ?=
 =?us-ascii?Q?+ASa1oja9QjIp3Mqd9+XaUBzwMvobDvNm68+4Va0kcJOf89/AiHXUvfSneHc?=
 =?us-ascii?Q?8lJXRdFthSOCyHSHuJ+Grc+lzPR3BaOy2pw3+Hawdb8ch9jr3IhIj27Ul9xy?=
 =?us-ascii?Q?TpZ3aoMqmVSMQhHp+QIH73m3bdB0YyWQILD6A3kVoo+sC6kE5g4Qm8Ftgsgw?=
 =?us-ascii?Q?9DqkO7/SfQyiEDdR2Qa1w18FxcXwnYV6M+Kdt/29xJQweokhoCqT2cB5YFBL?=
 =?us-ascii?Q?bZWpRkICOo7pA7ajobDoIu4zEJ53jcQDWfeAu+Wkue7nvS8jZgnVlHObYcIw?=
 =?us-ascii?Q?dRTkpDGdiA9IahErhpo1LLxSTm6dtcFVDUOKrmlh6KYwed6N9hnbEhxzolNB?=
 =?us-ascii?Q?vQaep+RtiPW2eZHtuDIVNC5rwTn/wD+pHEy0eq1edJuDNGkQRSUoP37k4kNR?=
 =?us-ascii?Q?Jqsp0h/dfXa94accz/Wp515pRl6f61O3/Q+mGQPdB2E3xLizXSp66cMZzTZ2?=
 =?us-ascii?Q?MRXKShdpyv1+ANjdQxq9Mc/WJk6UIbmvz0IWMlDMznaYRazMvWbcrW4NOvI3?=
 =?us-ascii?Q?3+hJry9/qNX8jukwkdtj0YoxA0a+Hsx72OXko3gPf3W03kRwCVP6dt1aiw+J?=
 =?us-ascii?Q?3KtwMO0FEvQgVPi0mFshXDRdXWkF8bQ2UFVP1zQyw2SFuJsmrbqYmEZMhf21?=
 =?us-ascii?Q?lUY3BSu94rRx7sZh4nPt/QqoQw1Ge66jy0eDzDvRTGLJRC5ZT+titZa+1fG/?=
 =?us-ascii?Q?ZE43CY+eVkn5BFLpiALQYKqmKsRBbqnSKEXKy+OJ77d2mlGuIX2JIyEaaiT7?=
 =?us-ascii?Q?kSwdvqxIcZApGjXYHK15BKwcVeT1BFoZ0fsuXE7WfsdJJZTrks+Fu4LEFgnM?=
 =?us-ascii?Q?K/G/12h6ypmEaYm9eow4xmNdKcwz6bRsu2rB0XOaTzbnENU9S6GFIvciXPJA?=
 =?us-ascii?Q?nMtR9+DKvzGn+Tkj5U8bOWHcI2fnuZEL9CXoQktd659ylDeTJeb3S416dns8?=
 =?us-ascii?Q?A/Pf1MBxUgm7LRFzr9CzlJTPlSOTHUxyDEUoI5ec?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SJ0PR11MB6744.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e751eee5-7041-4ffd-6e4e-08dbef1a6f2d
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Nov 2023 07:28:21.6418
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: joeprRnvbD68T5gppCcPkuo68I8eBV9rtsXGx6U0lXjopPZYDDlFvNUtNIZcs5yQUJZR7CfZocmX1WQEcILNjKnuhrCsitRQGYuSuyg98ng=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR11MB6992
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Sun, 26 Nov 2023 23:28:39 -0800 (PST)


>-----Original Message-----
>From: Liu, Yi L <yi.l.liu@intel.com>
>Sent: Monday, November 27, 2023 2:39 PM
>Subject: [PATCH 3/3] vfio: Report PASID capability via VFIO_DEVICE_FEATURE
>ioctl
>
>This reports the PASID capability data to userspace via VFIO_DEVICE_FEATUR=
E,
>hence userspace could probe PASID capability by it. This is a bit differen=
t
>with other capabilities which are reported to userspace when the user read=
s
>the device's PCI configuration space. There are two reasons for this.
>
> - First, Qemu by default exposes all available PCI capabilities in vfio-p=
ci
>   config space to the guest as read-only, so adding PASID capability in t=
he
>   vfio-pci config space will make it exposed to the guest automatically w=
hile
>   an old Qemu doesn't really support it.
>
> - Second, PASID capability does not exit on VFs (instead shares the cap o=
f
>   the PF). Creating a virtual PASID capability in vfio-pci config space n=
eeds
>   to find a hole to place it, but doing so may require device specific
>   knowledge to avoid potential conflict with device specific registers li=
ke
>   hiden bits in VF config space. It's simpler by moving this burden to th=
e
>   VMM instead of maintaining a quirk system in the kernel.
>
>Suggested-by: Alex Williamson <alex.williamson@redhat.com>
>Signed-off-by: Yi Liu <yi.l.liu@intel.com>
>---
> drivers/vfio/pci/vfio_pci_core.c | 47 ++++++++++++++++++++++++++++++++
> include/uapi/linux/vfio.h        | 13 +++++++++
> 2 files changed, 60 insertions(+)
>
>diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_=
core.c
>index 1929103ee59a..8038aa45500e 100644
>--- a/drivers/vfio/pci/vfio_pci_core.c
>+++ b/drivers/vfio/pci/vfio_pci_core.c
>@@ -1495,6 +1495,51 @@ static int vfio_pci_core_feature_token(struct
>vfio_device *device, u32 flags,
> 	return 0;
> }
>
>+static int vfio_pci_core_feature_pasid(struct vfio_device *device, u32 fl=
ags,
>+				       struct vfio_device_feature_pasid __user
>*arg,
>+				       size_t argsz)
>+{
>+	struct vfio_pci_core_device *vdev =3D
>+		container_of(device, struct vfio_pci_core_device, vdev);
>+	struct vfio_device_feature_pasid pasid =3D { 0 };
>+	struct pci_dev *pdev =3D vdev->pdev;
>+	u32 capabilities =3D 0;
>+	int ret;
>+
>+	/* We do not support SET of the PASID capability */
>+	ret =3D vfio_check_feature(flags, argsz, VFIO_DEVICE_FEATURE_GET,
>+				 sizeof(pasid));
>+	if (ret !=3D 1)
>+		return ret;
>+
>+	/*
>+	 * Needs go to PF if the device is VF as VF shares its PF's
>+	 * PASID Capability.
>+	 */
>+	if (pdev->is_virtfn)
>+		pdev =3D pci_physfn(pdev);
>+
>+	if (!pdev->pasid_enabled)
>+		goto out;

Does a PF bound to VFIO have pasid enabled by default?
Isn't the guest kernel's responsibility to enable pasid cap of an assigned =
PF?

Thanks
Zhenzhong

>+
>+#ifdef CONFIG_PCI_PASID
>+	pci_read_config_dword(pdev, pdev->pasid_cap + PCI_PASID_CAP,
>+			      &capabilities);
>+#endif
>+
>+	if (capabilities & PCI_PASID_CAP_EXEC)
>+		pasid.capabilities |=3D VFIO_DEVICE_PASID_CAP_EXEC;
>+	if (capabilities & PCI_PASID_CAP_PRIV)
>+		pasid.capabilities |=3D VFIO_DEVICE_PASID_CAP_PRIV;
>+
>+	pasid.width =3D (capabilities >> 8) & 0x1f;
>+
>+out:
>+	if (copy_to_user(arg, &pasid, sizeof(pasid)))
>+		return -EFAULT;
>+	return 0;
>+}
>+
> int vfio_pci_core_ioctl_feature(struct vfio_device *device, u32 flags,
> 				void __user *arg, size_t argsz)
> {
>@@ -1508,6 +1553,8 @@ int vfio_pci_core_ioctl_feature(struct vfio_device
>*device, u32 flags,
> 		return vfio_pci_core_pm_exit(device, flags, arg, argsz);
> 	case VFIO_DEVICE_FEATURE_PCI_VF_TOKEN:
> 		return vfio_pci_core_feature_token(device, flags, arg, argsz);
>+	case VFIO_DEVICE_FEATURE_PASID:
>+		return vfio_pci_core_feature_pasid(device, flags, arg, argsz);
> 	default:
> 		return -ENOTTY;
> 	}
>diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
>index 495193629029..8326faf8622b 100644
>--- a/include/uapi/linux/vfio.h
>+++ b/include/uapi/linux/vfio.h
>@@ -1512,6 +1512,19 @@ struct vfio_device_feature_bus_master {
> };
> #define VFIO_DEVICE_FEATURE_BUS_MASTER 10
>
>+/**
>+ * Upon VFIO_DEVICE_FEATURE_GET, return the PASID capability for the devi=
ce.
>+ * Zero width means no support for PASID.
>+ */
>+struct vfio_device_feature_pasid {
>+	__u16 capabilities;
>+#define VFIO_DEVICE_PASID_CAP_EXEC	(1 << 0)
>+#define VFIO_DEVICE_PASID_CAP_PRIV	(1 << 1)
>+	__u8 width;
>+	__u8 __reserved;
>+};
>+#define VFIO_DEVICE_FEATURE_PASID 11
>+
> /* -------- API for Type1 VFIO IOMMU -------- */
>
> /**
>--
>2.34.1