Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   "Tian, Kevin" <kevin.tian@intel.com>
To:     Jason Gunthorpe <jgg@nvidia.com>
CC:     Alex Williamson <alex.williamson@redhat.com>,
        "Liu, Yi L" <yi.l.liu@intel.com>, "hch@lst.de" <hch@lst.de>,
        "jasowang@redhat.com" <jasowang@redhat.com>,
        "joro@8bytes.org" <joro@8bytes.org>,
        "jean-philippe@linaro.org" <jean-philippe@linaro.org>,
        "parav@mellanox.com" <parav@mellanox.com>,
        "lkml@metux.net" <lkml@metux.net>,
        "pbonzini@redhat.com" <pbonzini@redhat.com>,
        "lushenming@huawei.com" <lushenming@huawei.com>,
        "eric.auger@redhat.com" <eric.auger@redhat.com>,
        "corbet@lwn.net" <corbet@lwn.net>,
        "Raj, Ashok" <ashok.raj@intel.com>,
        "yi.l.liu@linux.intel.com" <yi.l.liu@linux.intel.com>,
        "Tian, Jun J" <jun.j.tian@intel.com>, "Wu, Hao" <hao.wu@intel.com>,
        "Jiang, Dave" <dave.jiang@intel.com>,
        "jacob.jun.pan@linux.intel.com" <jacob.jun.pan@linux.intel.com>,
        "kwankhede@nvidia.com" <kwankhede@nvidia.com>,
        "robin.murphy@arm.com" <robin.murphy@arm.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
        "dwmw2@infradead.org" <dwmw2@infradead.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>,
        "david@gibson.dropbear.id.au" <david@gibson.dropbear.id.au>,
        "nicolinc@nvidia.com" <nicolinc@nvidia.com>
Subject: RE: [RFC 10/20] iommu/iommufd: Add IOMMU_DEVICE_GET_INFO
Thread-Topic: [RFC 10/20] iommu/iommufd: Add IOMMU_DEVICE_GET_INFO
Thread-Index: AQHXrSGNbNtRgavabUSKJjvt8l12BauwlhaAgAAouwCAACufAIAAEL4QgACKr4CACtdfoIAA3DaAgBUX8GCAAHivgIAAl4FggAl6ZNCAAXETAIAAHLdQgAYt1oCAA0rdUA==
Date:   Thu, 28 Oct 2021 02:07:46 +0000
Message-ID: <BN9PR11MB5433E2A78648049A41C484B78C869@BN9PR11MB5433.namprd11.prod.outlook.com>
References: <BN9PR11MB54333BDB1E58387FD9999DF18CA39@BN9PR11MB5433.namprd11.prod.outlook.com>
 <20210923114219.GG964074@nvidia.com>
 <BN9PR11MB5433519229319BA951CA97638CAA9@BN9PR11MB5433.namprd11.prod.outlook.com>
 <20210930222355.GH964074@nvidia.com>
 <BN9PR11MB5433530032DC8400B71FCB788CB89@BN9PR11MB5433.namprd11.prod.outlook.com>
 <20211014154259.GT2744544@nvidia.com>
 <BN9PR11MB543327BB6D58AEF91AD2C9D18CB99@BN9PR11MB5433.namprd11.prod.outlook.com>
 <BL1PR11MB5429973588E4FBCEC8F519A88CBF9@BL1PR11MB5429.namprd11.prod.outlook.com>
 <20211021233036.GN2744544@nvidia.com>
 <BN9PR11MB5433482C3754A8A383C3B6298C809@BN9PR11MB5433.namprd11.prod.outlook.com>
 <20211025233459.GM2744544@nvidia.com>
In-Reply-To: <20211025233459.GM2744544@nvidia.com>
Accept-Language: en-US
Content-Language: en-US
dlp-version: 11.6.200.16
dlp-product: dlpe-windows
dlp-reaction: no-action
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?xKecstFExDz/ccZ2yuEGrzZvJxRNBPStwOxdKdkoY3csPgM9ehsKfpr9CxPZ?=
 =?us-ascii?Q?Au96+e0MnRh+8oL3bUKFyWS7I9wWLQDo0uW9IlPuSY2u2qqky8zw8EoHfn/n?=
 =?us-ascii?Q?VhcpxMEJMErJF+AiGc9VUGfhViq6UeNzIGMlFqqvkkHVG35SNHai3tHWLqCU?=
 =?us-ascii?Q?Z4421x4xG31WsTQvFLGdTfLo0JHS8bu7H7gk578Mn8O8ZIKXpuHSKpcLyKY2?=
 =?us-ascii?Q?QVv52NEIPTUpW7SKTdpUMGjEsYAoxVICihdmQeRMJ1Hki2NjBI2QYcGy/QF1?=
 =?us-ascii?Q?eWMzMrMPSM1AIx2u3y7KkZk+VJ+pjKH4NsD1f2EfbA9TsxXn/DUz1QlM30LT?=
 =?us-ascii?Q?LrmZDpM5BcBYEJUjRVXPhLbUaSRgl+x2G9OwFZ3bN5RF+0iNbEu3ztH/9+ov?=
 =?us-ascii?Q?EtfZNPtM0zkGiNhz8WXT1kKRz5zCMn00vpk1xVIkAGnjBZYcjgbb3HYfpzGZ?=
 =?us-ascii?Q?xm2yQp+nS+C+m9eZlY9acmNNx04OVzNoWrmYV7rEHe6egV6bzX4T1s+eK2NP?=
 =?us-ascii?Q?/9Q6Op7AwWul8bz4OvHoHrVvXSdkvzZqsjs9pvP4upQWNVtPcRJICQO8NOuC?=
 =?us-ascii?Q?er48tAEDcRvqNqSNife1LYb7ABmxOKPP4AbO6A+CHi4UT2XLxTVDGrm+NYnx?=
 =?us-ascii?Q?Lf4bpvUg9P0fDUG49A1JHhS0lUVol8EjZTyKZmYKxzNaE4ejJ95mpXvLN0WA?=
 =?us-ascii?Q?eE8HLvshFBcDwBnM25x88leaHOYGmxFDO6/d0de0hh/AuduOPqB1EmNQHg3e?=
 =?us-ascii?Q?gGoCFDRAH9swPnnvrrTb+y4cuThNbNaPHbaiinhsEoy8HDx5a6yH7Q46oT7e?=
 =?us-ascii?Q?IRpc1tp0k36K711iEvkslJFtYuWFujhEDh7L8Kv16h8I2IbfactVjy4GuMxT?=
 =?us-ascii?Q?tLe7aXHQI1spGpgSRxZExwxmxqTdQjszYkvHu9NizP04EB2++xue9OAP7bO8?=
 =?us-ascii?Q?WxK8cTaZD7yvoHBNMz15jvO3sEDmnTq6JS4S3iCfJlHAP//KbfEaPpXKR7q7?=
 =?us-ascii?Q?t7RP8zALve+umVuBslLsQD8qXeRi08fzgaUFxORNcqywwXYEHnTDgS/cccUx?=
 =?us-ascii?Q?DlLQ80CGZl5NleAfISNOukTzSw8PC8ARd9jI8u5QCybp79k15qFc798clTVY?=
 =?us-ascii?Q?H6z+Kr8E3Pk9gxVMq/VpK14EGXuqJ2CMR9xftNX9+xZy/QpqtlB4U9kM1W+n?=
 =?us-ascii?Q?RsapTXGgQtL8s035j/nobKq2sZEE0UVafU0DmJzbHCFYiUtp2D0mYluVv+fp?=
 =?us-ascii?Q?MeWIhr0O0uso2QFz74zXo9rr/Hq/K230pa9xNiWzf1DlZmhHT60b5IxJaEa/?=
 =?us-ascii?Q?ZtnAdnmeHJpFN2RS489HooFCOrTTJIfvQwLujtR2NmcloyF+I5iIgI0OBpF5?=
 =?us-ascii?Q?RWHTy3kOgU7/uBVkuwvUnHxmBiwLdKLwDGdbrOtMTeVME7y3xXDKuXo2ovkF?=
 =?us-ascii?Q?L7A07jF+EOaItlMtUS+5dZfPJbI8XnruI8l7QLvGC2r6aJWExacArtHthgJi?=
 =?us-ascii?Q?zWCOgfgFhNlr8U9sdaTfkZRpjimOMHITSwrYN5PWIs+WIQCYFAb5xJ4YiFgp?=
 =?us-ascii?Q?KkplVmScMaYiK2nuhaKuGwXlvrZ9GzVCPEP+iDqQFiQsJOPBsaNvxJM4gQDR?=
 =?us-ascii?Q?zWnv6nvkom1RohJ0fg/Y0lk=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5433.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e1b8675b-9cf1-4667-615f-08d999b7bc11
X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Oct 2021 02:07:46.0407
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: ACeEfeGiDUy4o6UmIbFfByDaW71eo6jyvNxQLbtwVRupXuwFvDB1uZQjIPMfGpke24keXNnjI1kuUw8XusdfLw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR11MB5420
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Tuesday, October 26, 2021 7:35 AM
>=20
> On Fri, Oct 22, 2021 at 03:08:06AM +0000, Tian, Kevin wrote:
>=20
> > > I have no idea what security model makes sense for wbinvd, that is th=
e
> > > major question you have to answer.
> >
> > wbinvd flushes the entire cache in local cpu. It's more a performance
> > isolation problem but nothing can prevent it once the user is allowed
> > to call this ioctl. This is the main reason why wbinvd is a privileged
> > instruction and is emulated by kvm as a nop unless an assigned device
> > has no-snoop requirement. alternatively the user may call clflush
> > which is unprivileged and can invalidate a specific cache line, though
> > not efficient for flushing a big buffer.
> >
> > One tricky thing is that the process might be scheduled to different
> > cpus between writing buffers and calling wbinvd ioctl. Since wbvind
> > only has local behavior, it requires the ioctl to call wbinvd on all
> > cpus that this process has previously been scheduled on.
>=20
> That is such a hassle, you may want to re-open this with the kvm
> people as it seems ARM also has different behavior between VM and
> process here.
>=20
> The ideal is already not being met, so maybe we can keep special
> casing cache ops?
>=20

Now Paolo confirmed wbinvd ioctl is just a thought experiment.=20

Then Jason, want to have a clarification on 'keep special casing' here.

Did you mean adopting the vfio model which neither allows the user
to decide no-snoop format nor provides a wbinvd ioctl for the user
to manage buffers used for no-snoop traffic, or still wanting the user=20
to decide no-snoop format but not implementing a wbinvd ioctl?

The latter option sounds a bit incomplete from uAPI p.o.v. but it
allows us to stay with one-format-one-ioas policy. And anyway the
userspace can still call clflush to do cacheline-based invalidation,
if necessary.

The former option would force us to support multi-formats-one-ioas.

either case it's iommufd which decides and tells kvm whether wbinvd=20
is allowed for a process.=20

Thanks
Kevin