Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   "Tian, Kevin" <kevin.tian@intel.com>
To:     Jason Gunthorpe <jgg@nvidia.com>
CC:     "Liu, Yi L" <yi.l.liu@intel.com>,
        "alex.williamson@redhat.com" <alex.williamson@redhat.com>,
        "hch@lst.de" <hch@lst.de>,
        "jasowang@redhat.com" <jasowang@redhat.com>,
        "joro@8bytes.org" <joro@8bytes.org>,
        "jean-philippe@linaro.org" <jean-philippe@linaro.org>,
        "parav@mellanox.com" <parav@mellanox.com>,
        "lkml@metux.net" <lkml@metux.net>,
        "pbonzini@redhat.com" <pbonzini@redhat.com>,
        "lushenming@huawei.com" <lushenming@huawei.com>,
        "eric.auger@redhat.com" <eric.auger@redhat.com>,
        "corbet@lwn.net" <corbet@lwn.net>,
        "Raj, Ashok" <ashok.raj@intel.com>,
        "yi.l.liu@linux.intel.com" <yi.l.liu@linux.intel.com>,
        "Tian, Jun J" <jun.j.tian@intel.com>, "Wu, Hao" <hao.wu@intel.com>,
        "Jiang, Dave" <dave.jiang@intel.com>,
        "jacob.jun.pan@linux.intel.com" <jacob.jun.pan@linux.intel.com>,
        "kwankhede@nvidia.com" <kwankhede@nvidia.com>,
        "robin.murphy@arm.com" <robin.murphy@arm.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
        "dwmw2@infradead.org" <dwmw2@infradead.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>,
        "david@gibson.dropbear.id.au" <david@gibson.dropbear.id.au>,
        "nicolinc@nvidia.com" <nicolinc@nvidia.com>
Subject: RE: [RFC 02/20] vfio: Add device class for /dev/vfio/devices
Thread-Topic: [RFC 02/20] vfio: Add device class for /dev/vfio/devices
Thread-Index: AQHXrSFvz6zvm2xHBUaW/Kl+BWVvQauuqGKAgACCWyCAABPxgIAAAavw
Date:   Wed, 22 Sep 2021 01:07:11 +0000
Message-ID: <BN9PR11MB543322943B84EFAC88AD25438CA29@BN9PR11MB5433.namprd11.prod.outlook.com>
References: <20210919063848.1476776-1-yi.l.liu@intel.com>
 <20210919063848.1476776-3-yi.l.liu@intel.com>
 <20210921155705.GN327412@nvidia.com>
 <BN9PR11MB5433A709ED352FD81DDBF5A68CA19@BN9PR11MB5433.namprd11.prod.outlook.com>
 <20210922005501.GD327412@nvidia.com>
In-Reply-To: <20210922005501.GD327412@nvidia.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?y2nRsRqREUuu/UdODhZYrOAh/4XELuhyX1OI6QyssbaWXXIl2G4MUbxuwdLZ?=
 =?us-ascii?Q?dhoPfPUjgEkMPVHq/c2MtOBEsben6ZcOoZTI8hBdSMlvpf2WIJlIdlsKOiPm?=
 =?us-ascii?Q?q492tIMkEOrutT9rKZrMfBsQERrIXMMfNvHodb2VGLZeWwjTvemQNfsv428l?=
 =?us-ascii?Q?rbFHWclvKlr05amdnDXOQpWIcK5/Uvtpzz6joTZrGrZ+IP8AxVl/EpOSmeaG?=
 =?us-ascii?Q?KC9SkHxo7K7d4BQ142ctdreXY2vyOfZAavuLyVHYfD2cSRreEkMkhTONu1RC?=
 =?us-ascii?Q?c0GUTeNmLThhbLGFofpxNV8oqHXgsFzDQd5jbLI2xgdeUBamHAxQBny57HUu?=
 =?us-ascii?Q?vyE6dudlIiGBrghFQfB0Fk1CQpzBZecCpIrkN2nrwPIQgro5Axq2xgD/+T3Q?=
 =?us-ascii?Q?Uc07eNa14W88ZPmivljQbQvl31V+2GSccrRpXa39sUtJe6kt0cwpyxdXDXPD?=
 =?us-ascii?Q?qosWsFdaYZkIfAMduGRtsWGhZvwhABl+qJgULadZbBYVOlENT5rr5tmhxMoK?=
 =?us-ascii?Q?eCG/50EoNI2fd3/R6/BhbOpU+5CLfF/JGUSkmW9qFYq1+06BwnWTRL5aWxxA?=
 =?us-ascii?Q?kObU2TfUey1GQaVLqgL5RyfwetPxh+7AL4LWJiWp6HScxW4wqoxJET1FvTbH?=
 =?us-ascii?Q?sFsgK0AMh68v/CUs2gMAa4Li37jwEtWj5VqO59M5c8WrfIKy+UHodvM63StX?=
 =?us-ascii?Q?9sBO6M5FZsn48i/F1LUhOS5cWpbcoqsK68KGuxXtwcByWGDSzJTHbLBVKpbX?=
 =?us-ascii?Q?KAaqTXbH3Isi239BG2SWxcE7rqBIJH5COMXFJSfTwKHAknMRBT3FU/oEUvep?=
 =?us-ascii?Q?H4bsOcPi2lkN2uR8J/lB4FEN1qJwLPDnEy5n0+KC7iw1PjB2Z37GrffvO+Y0?=
 =?us-ascii?Q?U2kyddK0qMLjP0RZVmgQreTGskHhioWtT6hWXVCCmt0QQEV2TgrNxy/AO/Yr?=
 =?us-ascii?Q?NWZJ3/9sAnXuW12CgdJ0sE/duJtSdDVBSJm6OGz8qf83QxFx1C+FsyI4xq+j?=
 =?us-ascii?Q?7eSKuX5wYFLYHtNzxiEaJRwoQ6Hq8/8ySKsaKUY7SjQ7speEScCDbpku9LpV?=
 =?us-ascii?Q?KaJOfk7n0dZBM1rYlHsGHymZZu+Wqwx9UVkszNYDPqw1/wdoxsfGp7hxMXrq?=
 =?us-ascii?Q?CMH3xyggkdW1TV7QItgkxLbhfHxwyAH3PW997mq9Lv8AF85tOZb4A8Q4/Qm7?=
 =?us-ascii?Q?oUCKVosPtN9StvoecxXVlCe05PcdWyO/nT4eLR3Jtxbve6RT4EGe/Dz85mir?=
 =?us-ascii?Q?8KNIZ0uUJx2aZBC0Vypa2+vQ3reGrEqBJf4OOLzBo31L7+bI6XmMBK549rnD?=
 =?us-ascii?Q?H/c=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5433.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9b4ee719-bcfe-4047-bfe2-08d97d654edf
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Sep 2021 01:07:11.6269
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Raf3uSqVBfaura95SDrxKuY1GxwEVLxIvo5F/nU6ulvtDfJ9/4vsdl+fkzI2pUykbHYuo2RmwEogrUWQxwuRFw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR11MB0051
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Wednesday, September 22, 2021 8:55 AM
>=20
> On Tue, Sep 21, 2021 at 11:56:06PM +0000, Tian, Kevin wrote:
> > > The opened atomic is aweful. A newly created fd should start in a
> > > state where it has a disabled fops
> > >
> > > The only thing the disabled fops can do is register the device to the
> > > iommu fd. When successfully registered the device gets the normal fop=
s.
> > >
> > > The registration steps should be done under a normal lock inside the
> > > vfio_device. If a vfio_device is already registered then further
> > > registration should fail.
> > >
> > > Getting the device fd via the group fd triggers the same sequence as
> > > above.
> > >
> >
> > Above works if the group interface is also connected to iommufd, i.e.
> > making vfio type1 as a shim. In this case we can use the registration
> > status as the exclusive switch. But if we keep vfio type1 separate as
> > today, then a new atomic is still necessary. This all depends on how
> > we want to deal with vfio type1 and iommufd, and possibly what's
> > discussed here just adds another pound to the shim option...
>=20
> No, it works the same either way, the group FD path is identical to
> the normal FD path, it just triggers some of the state transitions
> automatically internally instead of requiring external ioctls.
>=20
> The device FDs starts disabled, an internal API binds it to the iommu
> via open coding with the group API, and then the rest of the APIs can
> be enabled. Same as today.
>=20

Still a bit confused. if vfio type1 also connects to iommufd, whether=20
the device is registered can be centrally checked based on whether
an iommu_ctx is recorded. But if type1 doesn't talk to iommufd at
all, don't we still need introduce a new state (calling it 'opened' or
'registered') to protect the two interfaces? In this case what is the
point of keeping device FD disabled even for the group path?