Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   "Tian, Kevin" <kevin.tian@intel.com>
To:     Alex Williamson <alex.williamson@redhat.com>,
        Jason Gunthorpe <jgg@nvidia.com>
CC:     "Liu, Yi L" <yi.l.liu@intel.com>, "hch@lst.de" <hch@lst.de>,
        "jasowang@redhat.com" <jasowang@redhat.com>,
        "joro@8bytes.org" <joro@8bytes.org>,
        "jean-philippe@linaro.org" <jean-philippe@linaro.org>,
        "parav@mellanox.com" <parav@mellanox.com>,
        "lkml@metux.net" <lkml@metux.net>,
        "pbonzini@redhat.com" <pbonzini@redhat.com>,
        "lushenming@huawei.com" <lushenming@huawei.com>,
        "eric.auger@redhat.com" <eric.auger@redhat.com>,
        "corbet@lwn.net" <corbet@lwn.net>,
        "Raj, Ashok" <ashok.raj@intel.com>,
        "yi.l.liu@linux.intel.com" <yi.l.liu@linux.intel.com>,
        "Tian, Jun J" <jun.j.tian@intel.com>, "Wu, Hao" <hao.wu@intel.com>,
        "Jiang, Dave" <dave.jiang@intel.com>,
        "jacob.jun.pan@linux.intel.com" <jacob.jun.pan@linux.intel.com>,
        "kwankhede@nvidia.com" <kwankhede@nvidia.com>,
        "robin.murphy@arm.com" <robin.murphy@arm.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
        "dwmw2@infradead.org" <dwmw2@infradead.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>,
        "david@gibson.dropbear.id.au" <david@gibson.dropbear.id.au>,
        "nicolinc@nvidia.com" <nicolinc@nvidia.com>
Subject: RE: [RFC 05/20] vfio/pci: Register device to /dev/vfio/devices
Thread-Topic: [RFC 05/20] vfio/pci: Register device to /dev/vfio/devices
Thread-Index: AQHXrSF8PdwE3l0WRkaTif5FzzsyqKuutGGAgABLSoCAAENA4A==
Date:   Wed, 22 Sep 2021 01:19:08 +0000
Message-ID: <BN9PR11MB5433D909662D484EFE9C82E28CA29@BN9PR11MB5433.namprd11.prod.outlook.com>
References: <20210919063848.1476776-1-yi.l.liu@intel.com>
        <20210919063848.1476776-6-yi.l.liu@intel.com>
        <20210921164001.GR327412@nvidia.com>
 <20210921150929.5977702c.alex.williamson@redhat.com>
In-Reply-To: <20210921150929.5977702c.alex.williamson@redhat.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?JbSlH8V1xkMnkd2Uh+BLFGaVeGjU3gySoPoRFQsdAlb0e2feDDJYzhxVp9/M?=
 =?us-ascii?Q?ECYNMxFxWHLVLtEVxDGAniVpLZurcd+lAGLXyksIoQmpgVksM++jztysWVvO?=
 =?us-ascii?Q?yxY3HgUhCinrtbed3uECged5IH+SnGG1h8+e3zGbNXLmsSx+wuCOOmpnp+VH?=
 =?us-ascii?Q?BXAQqxqTiKyeZ/2bbjFy7zNRz/hiSJ9fzCZFVb5x4Pq0WkQagGZWV/3XZ1eb?=
 =?us-ascii?Q?aBACppupkxeWr1iPCFlmGPAPTCNtFTT4oa9yqB5O4wb5Cu72IfEWp6gH2liz?=
 =?us-ascii?Q?psafhuP9CqMolBp0BKoJyBHcP1QFzGiuA04RnOLKkyCEpTKa6/+yXEUIL6rK?=
 =?us-ascii?Q?e93Ykj85gdjOCzadj+QQumZnCmWjQKw98eciilx75sUw+s7CG5ndV7bcH3YM?=
 =?us-ascii?Q?ch+AYYCde7ii64UUojqJeZJ6rVHYDSRuR72S25JKkHKNKq4dZMsHYdtiOP/8?=
 =?us-ascii?Q?wfuUs5Pl+SrikaaDD2k7YJs+5yF+vxsDWwGpHHqIS3aY7RAylE1sTysBmJik?=
 =?us-ascii?Q?uJ5EBdfnMCgwJjvYFqJYlDIPRfs/V8Qi4KdLSV4gZwQ48jZY9NML1m4CtXUe?=
 =?us-ascii?Q?a4qi4jRy3VlBUoXwutjLttpe2+trA8dXvtI2a8ASq1d5m4O886hagBeWD9UQ?=
 =?us-ascii?Q?MtejXp4tAYspiApOJPUD4VxrijM9BCJa/dYwpaiATQtvdYIO9l/f6P7ew+5C?=
 =?us-ascii?Q?3vlpuqRmphHEVAPN27MnuDUziKQC20WgfHo3DTTLMRU4PE7+oIaz17nR8Bfv?=
 =?us-ascii?Q?P6CW/PExkbzWgVfJQWNMpPDzodVQ5nY/Ob/ADl73zOmXxovErk6W0tFKCV7Q?=
 =?us-ascii?Q?0LrPYj75Yqa4VXzsbym8EyabjlT2mfyJ2RolqGbtEj2Qa+xo2oePzLNym6LH?=
 =?us-ascii?Q?8lbWtNxK5R6BBcBZ9jBYIaV8JV4aDYqKTuD+x8jTgO4FrVNPPZaWxfPdJgSS?=
 =?us-ascii?Q?4zA+uAWmFOlcU6sByctpN0+J18M0hnLFffOvp/KSyFYjQyV7P4Ma7yBzXSgs?=
 =?us-ascii?Q?2F7sFbpHYdrS0gFctyBfO1fQ7rdel9MFwRCwQGB44rcld+WXi/Pv4e2Fk8Fh?=
 =?us-ascii?Q?rCsphUWcpqIbes9zGkbXq3AXkYGaR4R8wFBPUwAezpZy4XZ1HIQwhuOs187+?=
 =?us-ascii?Q?Yiebx6EqTH6tyU+fXabeZv6NX5VmTQz0zz+uqJclJu5V0bEXhwf+qTlpCPWN?=
 =?us-ascii?Q?2728O3p/ZwVnp2yLuraMsuyhjkj74skuJpu2bJrW0hjo9qg0Jy+JTEo8Iwbs?=
 =?us-ascii?Q?5y6zd7IP85Tz0jalhT3fEXoHJl2niOsgHToPCepRVXpF8tmeEiQ2ZcrcNP7w?=
 =?us-ascii?Q?UhE=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5433.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e390139b-0f86-4f16-448f-08d97d66f9fd
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Sep 2021 01:19:08.1855
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: J+BIDJH9FQZ+p1VqZw4e9TvnHnfL75/klROHwPkeqgrvZSsJxJuSYnB/zPNwbYC5bTRUETgXhcFRhe2I7ZjSDg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR1101MB2339
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> From: Alex Williamson <alex.williamson@redhat.com>
> Sent: Wednesday, September 22, 2021 5:09 AM
>=20
> On Tue, 21 Sep 2021 13:40:01 -0300
> Jason Gunthorpe <jgg@nvidia.com> wrote:
>=20
> > On Sun, Sep 19, 2021 at 02:38:33PM +0800, Liu Yi L wrote:
> > > This patch exposes the device-centric interface for vfio-pci devices.=
 To
> > > be compatiable with existing users, vfio-pci exposes both legacy grou=
p
> > > interface and device-centric interface.
> > >
> > > As explained in last patch, this change doesn't apply to devices whic=
h
> > > cannot be forced to snoop cache by their upstream iommu. Such devices
> > > are still expected to be opened via the legacy group interface.
>=20
> This doesn't make much sense to me.  The previous patch indicates
> there's work to be done in updating the kvm-vfio contract to understand
> DMA coherency, so you're trying to limit use cases to those where the
> IOMMU enforces coherency, but there's QEMU work to be done to support
> the iommufd uAPI at all.  Isn't part of that work to understand how KVM
> will be told about non-coherent devices rather than "meh, skip it in the
> kernel"?  Also let's not forget that vfio is not only for KVM.

The policy here is that VFIO will not expose such devices (no enforce-snoop=
)
in the new device hierarchy at all. In this case QEMU will fall back to the
group interface automatically and then rely on the existing contract to con=
nect=20
vfio and QEMU. It doesn't need to care about the whatever new contract
until such devices are exposed in the new interface.

yes, vfio is not only for KVM. But here it's more a task split based on sta=
ging
consideration. imo it's not necessary to further split task into supporting
non-snoop device for userspace driver and then for kvm.