From:   "Tian, Kevin" <kevin.tian@intel.com>
To:     "Liu, Yi L" <yi.l.liu@intel.com>,
        "alex.williamson@redhat.com" <alex.williamson@redhat.com>,
        "jgg@nvidia.com" <jgg@nvidia.com>
CC:     "cohuck@redhat.com" <cohuck@redhat.com>,
        "eric.auger@redhat.com" <eric.auger@redhat.com>,
        "nicolinc@nvidia.com" <nicolinc@nvidia.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "mjrosato@linux.ibm.com" <mjrosato@linux.ibm.com>,
        "chao.p.peng@linux.intel.com" <chao.p.peng@linux.intel.com>,
        "yi.y.sun@linux.intel.com" <yi.y.sun@linux.intel.com>,
        "peterx@redhat.com" <peterx@redhat.com>,
        "jasowang@redhat.com" <jasowang@redhat.com>,
        "shameerali.kolothum.thodi@huawei.com" 
        <shameerali.kolothum.thodi@huawei.com>,
        "lulu@redhat.com" <lulu@redhat.com>,
        "suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com>,
        "intel-gvt-dev@lists.freedesktop.org" 
        <intel-gvt-dev@lists.freedesktop.org>,
        "intel-gfx@lists.freedesktop.org" <intel-gfx@lists.freedesktop.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-s390@vger.kernel.org" <linux-s390@vger.kernel.org>
Subject: RE: [PATCH v2 05/14] kvm/vfio: Accept vfio device file from userspace
Thread-Topic: [PATCH v2 05/14] kvm/vfio: Accept vfio device file from
 userspace
Thread-Index: AQHZOgo2enquzi+bhkCRgt8XFHLg9q7C1IrA
Date:   Tue, 7 Feb 2023 03:36:44 +0000
Message-ID: <BN9PR11MB5276CC3B1773CF183E147D6B8CDB9@BN9PR11MB5276.namprd11.prod.outlook.com>
References: <20230206090532.95598-1-yi.l.liu@intel.com>
 <20230206090532.95598-6-yi.l.liu@intel.com>
In-Reply-To: <20230206090532.95598-6-yi.l.liu@intel.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?aoJLJNfHnaDMWS9a3XJ64Puciweh7OMWDlqzyHFxiYdmq/q1+A4Azn+2RG2F?=
 =?us-ascii?Q?VKBDA2nGzhzmFVEyO9QK1VyGcLpCGfUP6A6idXfeWdIxKtvmxGUFIgBZHbWU?=
 =?us-ascii?Q?vfTw9l8sBSPdPnA8LdD/A93bUavcpadQfVO01jBQpKe9N4gS/GlOw9ttFo67?=
 =?us-ascii?Q?JKJ2jO/h45I3tirFMuzMVLAaQw0FkOMY54al7sGXMQRti8sQ/8OH9p0LNEb0?=
 =?us-ascii?Q?rvDi6Y+gb3OTXQqs9HTuDHHBdfp9fP3Zi45ArLbXOLiHe5DhFS4EBnc9WxUP?=
 =?us-ascii?Q?HfJZoTU2FEe01GUWjEQwLHhQAl9aQcSnKJ5d9Aa9eUru/REubSMElpP3J7tn?=
 =?us-ascii?Q?Mn5VDDU1mSxjqBSwwxzRpkOMdTwhCOZazEMSi8Ux1fxaTCv8i4fXucI6rEum?=
 =?us-ascii?Q?fcR1kadbY3Hg3RsMNVO9UD9GZ90cbdcqMLtfvJ7QWwWT8fi8Lf6f6dN7lV0z?=
 =?us-ascii?Q?wYsPtu7Um9OnpWqgEbQFI1n0XnY01PEhnJqglgjaK7veuoQbTRdMWtBckMJ9?=
 =?us-ascii?Q?qgln998p9S+ky75AkXZW/N/Msm35rOcKAdslMHJ+FYcFavuW6D7fDID2acXU?=
 =?us-ascii?Q?TU1PyIvkRSwALepW5FTixjz0JdWnwBVpUnFS7p98znCqoV4BllrACzNv82Iu?=
 =?us-ascii?Q?7U7jE1da+5ZevCg7CGLRC2AHGnnyviVwbdch19eIqMKJY+aMMvoq7gF+p2WB?=
 =?us-ascii?Q?BNfPi3K5tHbMIWrL23F84A3eYqyz7wkG6bXaoIXtut2lCfb4uDguxGP4TXLf?=
 =?us-ascii?Q?opTA9Hg88pyI/3xwJwmKmb3KZ7hec2r3/xwt6PCBijN6u/vyMwfmeUdxUF/n?=
 =?us-ascii?Q?FprSiq063DMzHpu2ArdGaMsIcDZx08T2kSNCgktKhqcEEvpEbFzYydet/n4S?=
 =?us-ascii?Q?QpWNA9rvpZmTlRJ3Y1XJBEiJAXcX58YmNbFn1gAvtObaTyksUJ8S5LIgu66y?=
 =?us-ascii?Q?Qs3BX8SUjNKmzgl62yN5BukPWQC8m8fmujI7FMvAFSNTx6QYmRaGDkPAY4Lw?=
 =?us-ascii?Q?grjfAHjLyDBRKDD2ASgvHn6BtMWsr8qHqb59CysAhhHDyBPviN9oEAaZNSIP?=
 =?us-ascii?Q?eaF3KzN0YKy78MmEX6RNUKFKAzKxms3QDyp8F3eGivoYdXejA3pcN2rj9q7+?=
 =?us-ascii?Q?srsuPRDFAns++Qy1ro64FqETAEerV9+5gloEvo3EEyX3VZJ+zGabtLr6WjjC?=
 =?us-ascii?Q?ngoLChsnVpB593O5Ld9Cxe1eDf5bJM77hQn8ke0F7Zta+P5Fvbl867a0tFmP?=
 =?us-ascii?Q?G+DFokJ3pQ1d6jdNTLvqEM/sHDFd9Sub/Dq8XAIXlIX+Z646rH8Z+m2Kycpe?=
 =?us-ascii?Q?Nn+iG1lhoIb513Ye3yftPHn+6MfR4mT32OQAPLEjPAN6tBgyEQeOqcj4VMGF?=
 =?us-ascii?Q?CgA8k+DGE43kFWFph6Wp4fvaUEET2F+mhKKGzw1zInBQZkips8CNUVPi693H?=
 =?us-ascii?Q?IviED+V1z1iNehyDkRZxlrhb6ihcNt6/I6g8GnfGrgI7ggq5vZ2vvpf/hFxU?=
 =?us-ascii?Q?ZlldQHZx+51TJUxf0dX+IzSHpQCmxRPZaEh4LcwrOjzEyhILoh/vtJPDmCjA?=
 =?us-ascii?Q?9NrqEqLac2QsUIMd0Z7QBZBEW0i6dNY3ls4ZqNhI?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d220efcf-1bdd-4c60-7dce-08db08bc88df
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Feb 2023 03:36:44.5966
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: KCjFwrdTRB2OvvoyqpCoOVzyctlOgqXn0Z1hcI5aAWk/A3XK67u4KQj/NQxSkzt3cc5IJ0/nmDiVjV5Iwx2IIg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR11MB7194
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> From: Liu, Yi L <yi.l.liu@intel.com>
> Sent: Monday, February 6, 2023 5:05 PM
>=20
> This defines KVM_DEV_VFIO_FILE* and make alias with
> KVM_DEV_VFIO_GROUP*.
> Old userspace uses KVM_DEV_VFIO_GROUP* works as well.
>=20
> Signed-off-by: Yi Liu <yi.l.liu@intel.com>
> ---
>  Documentation/virt/kvm/devices/vfio.rst | 40 ++++++++++++++-----------
>  include/uapi/linux/kvm.h                | 16 +++++++---
>  virt/kvm/vfio.c                         | 16 +++++-----
>  3 files changed, 42 insertions(+), 30 deletions(-)
>=20
> diff --git a/Documentation/virt/kvm/devices/vfio.rst
> b/Documentation/virt/kvm/devices/vfio.rst
> index 2d20dc561069..7f84ec26ca4a 100644
> --- a/Documentation/virt/kvm/devices/vfio.rst
> +++ b/Documentation/virt/kvm/devices/vfio.rst
> @@ -9,23 +9,26 @@ Device types supported:
>    - KVM_DEV_TYPE_VFIO
>=20
>  Only one VFIO instance may be created per VM.  The created device
> -tracks VFIO groups in use by the VM and features of those groups
> -important to the correctness and acceleration of the VM.  As groups
> -are enabled and disabled for use by the VM, KVM should be updated
> -about their presence.  When registered with KVM, a reference to the
> -VFIO-group is held by KVM.
> +tracks VFIO files (group or device) in use by the VM and features
> +of those groups/devices important to the correctness and acceleration
> +of the VM. As groups/devices are enabled and disabled for use by the
> +VM, KVM should be updated about their presence.  When registered with
> +KVM, a reference to the VFIO file is held by KVM.
>=20
>  Groups:

"Files"

> -  KVM_DEV_VFIO_GROUP
> -
> -KVM_DEV_VFIO_GROUP attributes:
> -  KVM_DEV_VFIO_GROUP_ADD: Add a VFIO group to VFIO-KVM device
> tracking
> -	kvm_device_attr.addr points to an int32_t file descriptor
> -	for the VFIO group.
> -  KVM_DEV_VFIO_GROUP_DEL: Remove a VFIO group from VFIO-KVM
> device tracking
> -	kvm_device_attr.addr points to an int32_t file descriptor
> -	for the VFIO group.
> -  KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE: attaches a guest visible TCE
> table
> +  KVM_DEV_VFIO_FILE
> +  - alias: KVM_DEV_VFIO_GROUP
> +
> +KVM_DEV_VFIO_FILE attributes:
> +  KVM_DEV_VFIO_FILE_ADD: Add a VFIO file (group/device) to VFIO-KVM
> device
> +	tracking kvm_device_attr.addr points to an int32_t file descriptor

"... device tracking" and "kvm_device.attr.addr points to..." are two
sentences. They are deliberately arranged to be in different lines.

> +	for the VFIO file.
> +	- alias: KVM_DEV_VFIO_GROUP_ADD
> +  KVM_DEV_VFIO_FILE_DEL: Remove a VFIO file (group/device) from VFIO-
> KVM
> +	device tracking kvm_device_attr.addr points to an int32_t file
> +	descriptor for the VFIO file.

ditto

> +	- alias: KVM_DEV_VFIO_GROUP_DEL
> +  KVM_DEV_VFIO_FILE_SET_SPAPR_TCE: attaches a guest visible TCE table
>  	allocated by sPAPR KVM.

somehow here it should emphasize that the file can only be group

>  	kvm_device_attr.addr points to a struct::
>=20
> @@ -36,6 +39,7 @@ KVM_DEV_VFIO_GROUP attributes:
>=20
>  	where:
>=20
> -	- @groupfd is a file descriptor for a VFIO group;
> -	- @tablefd is a file descriptor for a TCE table allocated via
> -	  KVM_CREATE_SPAPR_TCE.
> +	*) @groupfd is a file descriptor for a VFIO group;
> +	*) @tablefd is a file descriptor for a TCE table allocated via

why changing above two lines?

> +	   KVM_CREATE_SPAPR_TCE.
> +	- alias: KVM_DEV_VFIO_FILE_SET_SPAPR_TCE

GROUP