Received-SPF: pass (google.com: domain of linux-kernel+bounces-183366-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
From: "Tian, Kevin" <kevin.tian@intel.com>
To: Jason Gunthorpe <jgg@nvidia.com>, Alex Williamson
	<alex.williamson@redhat.com>, "Vetter, Daniel" <daniel.vetter@intel.com>
CC: "Zhao, Yan Y" <yan.y.zhao@intel.com>, "kvm@vger.kernel.org"
	<kvm@vger.kernel.org>, "linux-kernel@vger.kernel.org"
	<linux-kernel@vger.kernel.org>, "x86@kernel.org" <x86@kernel.org>,
	"iommu@lists.linux.dev" <iommu@lists.linux.dev>, "pbonzini@redhat.com"
	<pbonzini@redhat.com>, "seanjc@google.com" <seanjc@google.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "tglx@linutronix.de" <tglx@linutronix.de>,
	"mingo@redhat.com" <mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>,
	"hpa@zytor.com" <hpa@zytor.com>, "corbet@lwn.net" <corbet@lwn.net>,
	"joro@8bytes.org" <joro@8bytes.org>, "will@kernel.org" <will@kernel.org>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>, "baolu.lu@linux.intel.com"
	<baolu.lu@linux.intel.com>, "Liu, Yi L" <yi.l.liu@intel.com>
Subject: RE: [PATCH 4/5] vfio/type1: Flush CPU caches on DMA pages in
 non-coherent domains
Thread-Topic: [PATCH 4/5] vfio/type1: Flush CPU caches on DMA pages in
 non-coherent domains
Thread-Index: AQHaoEb1yTw+2+R5T02uRFeJZuQMNbGPN+OAgAER64CAAGvrAIAEE0QAgATCszCAANP1gIABWkKAgAPDRsA=
Date: Mon, 20 May 2024 02:52:43 +0000
Message-ID: <BN9PR11MB5276250B2CF376D15D16FF928CE92@BN9PR11MB5276.namprd11.prod.outlook.com>
References: <20240507061802.20184-1-yan.y.zhao@intel.com>
 <20240507062138.20465-1-yan.y.zhao@intel.com>
 <20240509121049.58238a6f.alex.williamson@redhat.com>
 <Zj33cUe7HYOIfj5N@yzhao56-desk.sh.intel.com>
 <20240510105728.76d97bbb.alex.williamson@redhat.com>
 <ZkG9IEQwi7HG3YBk@yzhao56-desk.sh.intel.com>
 <BN9PR11MB52766D78684F6206121590B98CED2@BN9PR11MB5276.namprd11.prod.outlook.com>
 <20240516143159.0416d6c7.alex.williamson@redhat.com>
 <20240517171117.GB20229@nvidia.com>
In-Reply-To: <20240517171117.GB20229@nvidia.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?l+3D+aaZxnYH8bak+DpOunf6rl12hQ+XzbEUflMmuzgysFDU0Fj2Q7CfNL/t?=
 =?us-ascii?Q?MDimM9UvL4l01u8MstImU7ZrH434LdIpnwXyBu2njLtd/uZMXzl57wDO8YLg?=
 =?us-ascii?Q?xNJV/KBRJUDKKAcuNw0AWscDsMiVLTMKwUHZhgF8Yyzjbk9cXv+fKsKZfkkm?=
 =?us-ascii?Q?Ix5rU8PS25dWQvhTcZilaEGMU8eWoI0MrQkQ5vZ4FQAKH8x7OZHz7GPZ/znD?=
 =?us-ascii?Q?lZHnhSriTH6OCS6oJ6drvyRpqxz1N3ceJifu3wIz1NadoQq9aeV8wWXSc7zM?=
 =?us-ascii?Q?iv1+WViX4a7yCu4/lqpqli46QZq0pVUU9hU6ZQUy6xF0yoYH20ffCmZiDXpO?=
 =?us-ascii?Q?uG376/Vb/6ZtQtZTwWmYbe6kLzYMVhMZqco0u/q2B9tqLIyM2v1/hxrPxy49?=
 =?us-ascii?Q?54dXKRWdNADuTa8R1LiRH9lPL7iDFb+jzOX27HIvtg+VKWYqDnrei/Kcamli?=
 =?us-ascii?Q?qaOaSpNoCvSgwyuDqPlf+p7KSQQeVXCM0XKsmx2gWa2MHea0+mc4OtEQQFNG?=
 =?us-ascii?Q?Cyctx3HeT0r9eITweZdqa10Tm7enQixr6sEv7uYGlS6NF5FbFCgvtznilTD8?=
 =?us-ascii?Q?xx56Jrbsrit5PwVpLmP/hAjPQnKEZ0upvNHzEl9iPKTIrGdOr4XozVpzFp2m?=
 =?us-ascii?Q?95yb+rdGW0hNJOxdE2NaA39uIunoV0Ym1GNU8u2cVq8FoAjDON053I8NCTMa?=
 =?us-ascii?Q?066CbvEretkJaX37aBVIIcLU4TJe5XZHKA+af31h60mgDTBGDXXAS9BjHhp3?=
 =?us-ascii?Q?q6oRb8TfhjSzb8e+3reTSO8EGe5gCAq+hIu6H+0NvoAmz/mKERDbw6jAcO7e?=
 =?us-ascii?Q?T+PLxCVvoqdsLmb49EgyCVSgVYUey+3H+oH3vGWFyHnwHzf0oenOXtNMIe19?=
 =?us-ascii?Q?eGEa2kBTOJdEroSUBvy3mIWKLEeE0NVeXBrCERi9ULdiSF/sN/YuomCo4YhC?=
 =?us-ascii?Q?9so7qmVtNWspAnvWi51Z2ayoQUISs2ahRH3eN1Pd440+KNrkkCHESEqTrsOD?=
 =?us-ascii?Q?a9rKBWcdIpvPCNsSmi33sGe0UQipVpogD2kZkzTy/xnz4pHRb2WcTa2HtJuL?=
 =?us-ascii?Q?DwvGEYDe4+oOpFjJHnTPxSKMe7ZV29+1yUVsUt3Q3VTTENikpFgEg65uck8L?=
 =?us-ascii?Q?CZPOdBTMXKdwUTPVLvtn3ZrKeBGp12ueig9AJM+73BZjLrdRORkTWueCmmGy?=
 =?us-ascii?Q?drAmtgUy8dGf+GkqJqThM3KPnATZL3JX27QlhznuNBeMj+obWQTnvTOO+WXY?=
 =?us-ascii?Q?w2hbMMQ4vfMQkXc9MIOdrMiF318OQIFXt4fc3mXtK3KMygaqvT38WPGa6Pwv?=
 =?us-ascii?Q?gaWFU6XcEO/tiBFFR9V/Hy0/0/HWGwLB8dTxt+ww0y3Ywook1BAHfsTOh5FP?=
 =?us-ascii?Q?J8JQlh2ucHkDlLBLxunbVEC9W430J5m032Er0UaAvQyyTCxTxrU+SdDa4LnA?=
 =?us-ascii?Q?6Y5Xk4mvcUn71IxRYqT85ErmUCvMY0XfwtZYdDCDMNvyqwXjXJBGNHF2YhQ5?=
 =?us-ascii?Q?1/1v13lhW/oiGbI4agjYJkuM2uCdqV1y5atAJ/G8T32OttvQ1pUrMgpdACbm?=
 =?us-ascii?Q?Kz9DSh8R7RNYHT1UyAZdQx2VnSsfBNY4syXUjzZu?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
List-Id: <linux-kernel.vger.kernel.org>
List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org>
List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org>
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 643aeebf-e460-4baf-abb8-08dc7877ec03
X-MS-Exchange-CrossTenant-originalarrivaltime: 20 May 2024 02:52:43.5729
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: E2nGHaBvyHdRF0ya/1EiyYxIDIT2BX5hlqZQ+4/6PSmDNmuxf5qFqMDCJ8oWG++DqVaHzou/R8YpWwpsADcHag==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR11MB8149
X-OriginatorOrg: intel.com

+Daniel

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Saturday, May 18, 2024 1:11 AM
>=20
> On Thu, May 16, 2024 at 02:31:59PM -0600, Alex Williamson wrote:
>=20
> > Yes, exactly.  Zero'ing the page would obviously reestablish the
> > coherency, but the page could be reallocated without being zero'd and a=
s
> > you describe the owner of that page could then get inconsistent
> > results.
>=20
> I think if we care about the performance of this stuff enough to try
> and remove flushes we'd be better off figuring out how to disable no
> snoop in PCI config space and trust the device not to use it and avoid
> these flushes.
>=20
> iommu enforcement is nice, but at least ARM has been assuming that the
> PCI config space bit is sufficient.
>=20
> Intel/AMD are probably fine here as they will only flush for weird GPU
> cases, but I expect ARM is going to be unhappy.
>=20

My impression was that Intel GPU is not usable w/o non-coherent DMA,
but I don't remember whether it's unusable being a functional breakage
or a user experience breakage. e.g. I vaguely recalled that the display
engine cannot afford high resolution/high refresh rate using the snoop
way so the IOMMU dedicated for the GPU doesn't implement the force
snoop capability.

Daniel, can you help explain the behavior of Intel GPU in case nosnoop
is disabled in the PCI config space?

Overall it sounds that we are talking about different requirements. For
Intel GPU nosnoop is a must but it is not currently done securely so we
need add proper flush to fix it, while for ARM looks you don't have a
case which relies on nosnoop so finding a way to disable it is more
straightforward?