Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   "Tian, Kevin" <kevin.tian@intel.com>
To:     Alex Williamson <alex.williamson@redhat.com>,
        Jason Gunthorpe <jgg@nvidia.com>
CC:     "Liu, Yi L" <yi.l.liu@intel.com>, "hch@lst.de" <hch@lst.de>,
        "jasowang@redhat.com" <jasowang@redhat.com>,
        "joro@8bytes.org" <joro@8bytes.org>,
        "jean-philippe@linaro.org" <jean-philippe@linaro.org>,
        "parav@mellanox.com" <parav@mellanox.com>,
        "lkml@metux.net" <lkml@metux.net>,
        "pbonzini@redhat.com" <pbonzini@redhat.com>,
        "lushenming@huawei.com" <lushenming@huawei.com>,
        "eric.auger@redhat.com" <eric.auger@redhat.com>,
        "corbet@lwn.net" <corbet@lwn.net>,
        "Raj, Ashok" <ashok.raj@intel.com>,
        "yi.l.liu@linux.intel.com" <yi.l.liu@linux.intel.com>,
        "Tian, Jun J" <jun.j.tian@intel.com>, "Wu, Hao" <hao.wu@intel.com>,
        "Jiang, Dave" <dave.jiang@intel.com>,
        "jacob.jun.pan@linux.intel.com" <jacob.jun.pan@linux.intel.com>,
        "kwankhede@nvidia.com" <kwankhede@nvidia.com>,
        "robin.murphy@arm.com" <robin.murphy@arm.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
        "dwmw2@infradead.org" <dwmw2@infradead.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>,
        "david@gibson.dropbear.id.au" <david@gibson.dropbear.id.au>,
        "nicolinc@nvidia.com" <nicolinc@nvidia.com>
Subject: RE: [RFC 03/20] vfio: Add vfio_[un]register_device()
Thread-Topic: [RFC 03/20] vfio: Add vfio_[un]register_device()
Thread-Index: AQHXrSFwNP3/pIcrZ0iM9X/cSoy0lKuuqYQAgAB3MFCAAB2WgIAAjDNwgAA0YACAAIKvAIAAJpDg
Date:   Wed, 22 Sep 2021 22:34:42 +0000
Message-ID: <BN9PR11MB543366158EA87572902EFF5E8CA29@BN9PR11MB5433.namprd11.prod.outlook.com>
References: <20210919063848.1476776-1-yi.l.liu@intel.com>
        <20210919063848.1476776-4-yi.l.liu@intel.com>
        <20210921160108.GO327412@nvidia.com>
        <BN9PR11MB5433D4590BA725C79196E0248CA19@BN9PR11MB5433.namprd11.prod.outlook.com>
        <20210922005337.GC327412@nvidia.com>
        <BN9PR11MB54338D108AF5A87614717EF98CA29@BN9PR11MB5433.namprd11.prod.outlook.com>
        <20210922122252.GG327412@nvidia.com>
 <20210922141036.5cd46b2b.alex.williamson@redhat.com>
In-Reply-To: <20210922141036.5cd46b2b.alex.williamson@redhat.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?RsaHJw3uLolBUtUj4Fd9MAnpcxnOkK9qREJL/E/F/3JeJ9OGNZQeshldR/Mx?=
 =?us-ascii?Q?QZ332jvnDq8elS7YusR6c8iHvjbYQYSdl448F4JSXZz0tdAdF5IhY2PkIKuY?=
 =?us-ascii?Q?blYyjWpD0rrtLpQWseym8D5Mu79WMc0t7KGf7u+0BbwAk66XEVeNwHBGsTyP?=
 =?us-ascii?Q?UjlTdpDZRUCAJNhJ+/cMcKn9NeSgbSeQf6dGC9WIILHadXdxemUjlZ6QscYT?=
 =?us-ascii?Q?yN8g5IT3iH0/EmIXonpaXZvd1RrBXS71ho3hLezxfMrOxf8v07NY4WndSRMK?=
 =?us-ascii?Q?++QCQxC8EA8T1BaSNUUv4v+i4NV3MNcSegCrqH0YzbM9Q4/KsYyETEfc3ts+?=
 =?us-ascii?Q?H2j1ovub0HMR4bYJkE9uElqOLv7T9kgJBZLETkqkCtf6ukhFSIwefW7dTuYN?=
 =?us-ascii?Q?eJi01PM2EqJun8NS/NHj4Q7txcKgaSxMwCIyLcRmUVCAhK2qAsZDBy2YVZQ/?=
 =?us-ascii?Q?mLwqowvQvrgQT8iOiBfX/xoOd0zYBCkuAxBmygrQdEi9KBHjTPm5UfzAMpVG?=
 =?us-ascii?Q?efgJ4FpL3c3hEtCHoeYWV5/FHrEiFo+aHTt0QsZ4kqyzr7pBJ3fPML4tfe/h?=
 =?us-ascii?Q?Bz09RIar2XMd040k5TEsuXdBQfsZW+k6JMjX0ydocxVITZ+OskvI5XE9o2aX?=
 =?us-ascii?Q?Zjfedak2mn01GPItjfRR3btCf0aB0EthowRIsjHFA00GB7yDROsTzwSq4vEC?=
 =?us-ascii?Q?n9MzTFaqHq9LaCPBIn+i3soGmUA6ayFw3r4OzqGCPjsgeamLqeFZCs0oc0zM?=
 =?us-ascii?Q?t6VlUJZTP8V2R3x1kmnRMwrGfxNqjJWSI03Pcnen/280QDeZTpcbnOyPQ9Bo?=
 =?us-ascii?Q?y1hbbXVP/NmnoY4/jlBhTgLVQrWQho3T2bKGTMgnqDMr0+cyoF5PlpR5H5gO?=
 =?us-ascii?Q?QL9MmmYIAM23OrCpf/tV7rDqIxeReC6gtW9pPhAka6W3zfybJhB4nJj/5vud?=
 =?us-ascii?Q?TQODLSnw37nFBrUOy9VOTLOP/QSEIg9ZkdlmTMZwkzN5XFf4y9O4sCtGv57h?=
 =?us-ascii?Q?XV38tAoGmbKzJUuTn2LSBm7ANxEUaCjnw+uRJElKx7xQk1R+va6UgbPESPq/?=
 =?us-ascii?Q?cEpFUL5coBSxCiDyeOG58PQ153dgY04IADu0600Y4QJDulQi0jMQ4Pt/DoIW?=
 =?us-ascii?Q?1e0R+I1du9QWBHguDoPrHoRpIQxuWLqiT1mHTgDEzqSNBz2EH2faEPe50qmH?=
 =?us-ascii?Q?aGVe91oPrOehtheo5RUK3L9yfPehBTI4gd4bdOD+UUuDGRhVenJ+i066uw4a?=
 =?us-ascii?Q?J5CRavRI69k7bnDXdJlpmisdnc/fq0BKQK7URraWblRpRqYCCrk06pvKQs3w?=
 =?us-ascii?Q?7aUkiakAWhoWjaXu0ZoRTzUq?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5433.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ec6faf97-c456-40c7-0cdc-08d97e192bf0
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Sep 2021 22:34:42.4238
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 6dOSzTg2AJQ1SV3lMf0Ib/3pJVcwipJyuPExwrqs55h//77gun8ZfP+r3w2UwedTpWtOjwczQd9jO+hF8oMHRg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR11MB1537
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> From: Alex Williamson <alex.williamson@redhat.com>
> Sent: Thursday, September 23, 2021 4:11 AM
>=20
> On Wed, 22 Sep 2021 09:22:52 -0300
> Jason Gunthorpe <jgg@nvidia.com> wrote:
>=20
> > On Wed, Sep 22, 2021 at 09:23:34AM +0000, Tian, Kevin wrote:
> >
> > > > Providing an ioctl to bind to a normal VFIO container or group migh=
t
> > > > allow a reasonable fallback in userspace..
> > >
> > > I didn't get this point though. An error in binding already allows th=
e
> > > user to fall back to the group path. Why do we need introduce another
> > > ioctl to explicitly bind to container via the nongroup interface?
> >
> > New userspace still needs a fallback path if it hits the 'try and
> > fail'. Keeping the device FD open and just using a different ioctl to
> > bind to a container/group FD, which new userspace can then obtain as a
> > fallback, might be OK.
> >
> > Hard to see without going through the qemu parts, so maybe just keep
> > it in mind
>=20
> If we assume that the container/group/device interface is essentially
> deprecated once we have iommufd, it doesn't make a lot of sense to me
> to tack on a container/device interface just so userspace can avoid
> reverting to the fully legacy interface.
>=20
> But why would we create vfio device interface files at all if they
> can't work?  I'm not really on board with creating a try-and-fail
> interface for a mechanism that cannot work for a given device.  The
> existence of the device interface should indicate that it's supported.
> Thanks,
>=20

Now it's a try-and-fail model even for devices which support iommufd.
Per Jason's suggestion, a device is always opened with a parked fops
which supports only bind. Binding serves as the contract for handling
exclusive ownership on a device and switching to normal fops if
succeed. So the user has to try-and-fail in case multiple threads attempt=20
to open a same device. Device which doesn't support iommufd is not
different, except binding request 100% fails (due to missing .bind_iommufd
in kernel driver).

Thanks
Kevin