Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   "Tian, Kevin" <kevin.tian@intel.com>
To:     Jason Gunthorpe <jgg@nvidia.com>
CC:     "Liu, Yi L" <yi.l.liu@intel.com>,
        "alex.williamson@redhat.com" <alex.williamson@redhat.com>,
        "hch@lst.de" <hch@lst.de>,
        "jasowang@redhat.com" <jasowang@redhat.com>,
        "joro@8bytes.org" <joro@8bytes.org>,
        "jean-philippe@linaro.org" <jean-philippe@linaro.org>,
        "parav@mellanox.com" <parav@mellanox.com>,
        "lkml@metux.net" <lkml@metux.net>,
        "pbonzini@redhat.com" <pbonzini@redhat.com>,
        "lushenming@huawei.com" <lushenming@huawei.com>,
        "eric.auger@redhat.com" <eric.auger@redhat.com>,
        "corbet@lwn.net" <corbet@lwn.net>,
        "Raj, Ashok" <ashok.raj@intel.com>,
        "yi.l.liu@linux.intel.com" <yi.l.liu@linux.intel.com>,
        "Tian, Jun J" <jun.j.tian@intel.com>, "Wu, Hao" <hao.wu@intel.com>,
        "Jiang, Dave" <dave.jiang@intel.com>,
        "jacob.jun.pan@linux.intel.com" <jacob.jun.pan@linux.intel.com>,
        "kwankhede@nvidia.com" <kwankhede@nvidia.com>,
        "robin.murphy@arm.com" <robin.murphy@arm.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
        "dwmw2@infradead.org" <dwmw2@infradead.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>,
        "david@gibson.dropbear.id.au" <david@gibson.dropbear.id.au>,
        "nicolinc@nvidia.com" <nicolinc@nvidia.com>
Subject: RE: [RFC 03/20] vfio: Add vfio_[un]register_device()
Thread-Topic: [RFC 03/20] vfio: Add vfio_[un]register_device()
Thread-Index: AQHXrSFwNP3/pIcrZ0iM9X/cSoy0lKuuqYQAgAB3MFCAAB2WgIAAAOgA
Date:   Wed, 22 Sep 2021 00:59:27 +0000
Message-ID: <BN9PR11MB54330FF701EAB6BAB55C53078CA29@BN9PR11MB5433.namprd11.prod.outlook.com>
References: <20210919063848.1476776-1-yi.l.liu@intel.com>
 <20210919063848.1476776-4-yi.l.liu@intel.com>
 <20210921160108.GO327412@nvidia.com>
 <BN9PR11MB5433D4590BA725C79196E0248CA19@BN9PR11MB5433.namprd11.prod.outlook.com>
 <20210922005337.GC327412@nvidia.com>
In-Reply-To: <20210922005337.GC327412@nvidia.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?pXvlUjsEsFvPIB6Y24/7MraEpkKWk7Y9lbcd8/8ftl0TXJ6CwURapOB8GjM1?=
 =?us-ascii?Q?xSfb0D/+f9LqC0PEEic1ZOiTTbIkLhF7TY/7SbgiI3I4Q3jVULFN9dD1YuLE?=
 =?us-ascii?Q?7m8tTgP2dezOy8ELo/G6dBuRTH8+yx9PjJ3a2p51syVVfUUyeEMkTPeHO5Cw?=
 =?us-ascii?Q?2VLF/6CqlBYjVHVol91Xx27DKNfiZfGWRs5gcFFYCIXzSb6Nlo/NLiWpSAQh?=
 =?us-ascii?Q?KO4q9o4AesPdaGppK9uZcQlpKU79ymdV1oCAf6jDfeFVIJukblig1STLAU3k?=
 =?us-ascii?Q?dU+Hstr/DH3aYTxyZGBFKLNGWqFUJISTrwC0EgqvHaceMWZWvn/VtCU/nyIr?=
 =?us-ascii?Q?2ksgnHOexzAJ/RIoV0AYiOAwwy2fydbhLRD5MLhZtLFwL/mcTbP1E7FOdJNI?=
 =?us-ascii?Q?bdoPAF/eFiVAiok4V8bx6MVB9SLCwWeu5Gp65FBUlahpUeZCNUCqFWrfZkJM?=
 =?us-ascii?Q?aX5rubYZSqb68VFCjDScu5zl8SnfkyL6YIW2YImdLTnAya4EY4seVq96/8oh?=
 =?us-ascii?Q?thLG5aqP+PUD1JZuXSlvRMDc/U5cgtLJPjMUsGXyYBiRRyV8lv0zp4lg5fnV?=
 =?us-ascii?Q?xG+zX9V9ME4z15+cfi5G6wmSE+MEIFWIV2OzQZzsqUx0NK3kBbjuZriOsrN5?=
 =?us-ascii?Q?HZpILgBjsdNHgLw2TYKGo51vp56AnlZXyBsTIDSdXE2e4FGsDA/b9a3ZgJkQ?=
 =?us-ascii?Q?T9+YVeUSUJRVK1/yuJ/5eaUoyaCkeCC7qnf9J551jp0EXszGrzx9hmgRBxRY?=
 =?us-ascii?Q?jKyph+VaqUfunPQ4yLhTed3/Opm4/S7zZ+0H9sYn5unKXLGq4e/ji+gE9oK6?=
 =?us-ascii?Q?0H0poZoQoNAI7NM7Lc11MsdQE1aDwa1V7/4lvGhrbl8E4jXz0L1gsjoJcz/R?=
 =?us-ascii?Q?G2L1Te9iFUuR/jbm+EdCQKuOrpcPaFp+gYtvTr4fT/iX4HbFLRrjsec/epJX?=
 =?us-ascii?Q?Ijs6h+NXXgtFcgxYgtDbLW1EivVsudfawDoPA9TMyvTnQnimhY5xxW6VotD6?=
 =?us-ascii?Q?KijPg9BCJw9f8/t6G+zaO4RVFq1z/wWBtxMZ0zS6kB7IdQ+Ni7jQ/hM8AUaO?=
 =?us-ascii?Q?WksVa7UUAjPNACoNCK+anqtH+ohBzQpSa70gvrF6kwhLlHC9yMgufjOmcC2Q?=
 =?us-ascii?Q?7X6g5Jt1uePZ0fMuDbufjZqxQ26DONNPrtyhOQRj2MhCH+ZIWpbPbYsViHhb?=
 =?us-ascii?Q?wNEc7xtEKTA3ttZ+aOXRkitsjjHeUl5z5f5iVOJdRKhGiePg7OruQ0BAR3HL?=
 =?us-ascii?Q?9XomiMvYtsDqm9/X/cuxZOZ/PFj66tIWlCFDIPWJYDdmZ35/+GDslNXx2iRM?=
 =?us-ascii?Q?e3c=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5433.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c513d2b7-3be1-47a5-4521-08d97d643a27
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Sep 2021 00:59:27.3946
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 7bBVvfA1TvFoR4ocahxs0WmhXuUHFa3CKhCwus3aejvorpmzt5nfFlL7YxWFUmUt5itRAxQNxToLtL+qXn9hXw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR11MB0049
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Wednesday, September 22, 2021 8:54 AM
>=20
> On Tue, Sep 21, 2021 at 11:10:15PM +0000, Tian, Kevin wrote:
> > > From: Jason Gunthorpe <jgg@nvidia.com>
> > > Sent: Wednesday, September 22, 2021 12:01 AM
> > >
> > > On Sun, Sep 19, 2021 at 02:38:31PM +0800, Liu Yi L wrote:
> > > > With /dev/vfio/devices introduced, now a vfio device driver has thr=
ee
> > > > options to expose its device to userspace:
> > > >
> > > > a)  only legacy group interface, for devices which haven't been mov=
ed
> to
> > > >     iommufd (e.g. platform devices, sw mdev, etc.);
> > > >
> > > > b)  both legacy group interface and new device-centric interface, f=
or
> > > >     devices which supports iommufd but also wants to keep backward
> > > >     compatibility (e.g. pci devices in this RFC);
> > > >
> > > > c)  only new device-centric interface, for new devices which don't =
carry
> > > >     backward compatibility burden (e.g. hw mdev/subdev with pasid);
> > >
> > > We shouldn't have 'b'? Where does it come from?
> >
> > a vfio-pci device can be opened via the existing group interface. if no=
 b) it
> > means legacy vfio userspace can never use vfio-pci device any more
> > once the latter is moved to iommufd.
>=20
> Sorry, I think I ment a, which I guess you will say is SW mdev devices

We listed a) here in case we don't want to move all vfio device types to
use iommufd in one breath. It's supposed to be a type valid only in this
transition phase. In the end only b) and c) are valid.

>=20
> But even so, I think the way forward here is to still always expose
> the device /dev/vfio/devices/X and some devices may not allow iommufd
> usage initially.
>=20
> Providing an ioctl to bind to a normal VFIO container or group might
> allow a reasonable fallback in userspace..
>=20

but doesn't a new ioctl still imply breaking existing vfio userspace?

Thanks
Kevin