Received-SPF: pass (google.com: domain of linux-kernel+bounces-186831-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
From: "Tian, Kevin" <kevin.tian@intel.com>
To: Jason Gunthorpe <jgg@nvidia.com>
CC: Alex Williamson <alex.williamson@redhat.com>, "Vetter, Daniel"
	<daniel.vetter@intel.com>, "Zhao, Yan Y" <yan.y.zhao@intel.com>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>, "linux-kernel@vger.kernel.org"
	<linux-kernel@vger.kernel.org>, "x86@kernel.org" <x86@kernel.org>,
	"iommu@lists.linux.dev" <iommu@lists.linux.dev>, "pbonzini@redhat.com"
	<pbonzini@redhat.com>, "seanjc@google.com" <seanjc@google.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "tglx@linutronix.de" <tglx@linutronix.de>,
	"mingo@redhat.com" <mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>,
	"hpa@zytor.com" <hpa@zytor.com>, "corbet@lwn.net" <corbet@lwn.net>,
	"joro@8bytes.org" <joro@8bytes.org>, "will@kernel.org" <will@kernel.org>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>, "baolu.lu@linux.intel.com"
	<baolu.lu@linux.intel.com>, "Liu, Yi L" <yi.l.liu@intel.com>
Subject: RE: [PATCH 4/5] vfio/type1: Flush CPU caches on DMA pages in
 non-coherent domains
Thread-Topic: [PATCH 4/5] vfio/type1: Flush CPU caches on DMA pages in
 non-coherent domains
Thread-Index: AQHaoEb1yTw+2+R5T02uRFeJZuQMNbGPN+OAgAER64CAAGvrAIAEE0QAgATCszCAANP1gIABWkKAgAPDRsCAAnQoAIAAA/SAgAADhwCAAB2LgIAABQiAgAC9HuCAAG5egIAAtd9QgAADQICAAADyYA==
Date: Wed, 22 May 2024 23:40:58 +0000
Message-ID: <BN9PR11MB5276C2DD3F924ED2EB6AD3988CEB2@BN9PR11MB5276.namprd11.prod.outlook.com>
References: <20240517171117.GB20229@nvidia.com>
 <BN9PR11MB5276250B2CF376D15D16FF928CE92@BN9PR11MB5276.namprd11.prod.outlook.com>
 <20240521160714.GJ20229@nvidia.com>
 <20240521102123.7baaf85a.alex.williamson@redhat.com>
 <20240521163400.GK20229@nvidia.com>
 <20240521121945.7f144230.alex.williamson@redhat.com>
 <20240521183745.GP20229@nvidia.com>
 <BN9PR11MB52769E209C5B978C7094A5C08CEB2@BN9PR11MB5276.namprd11.prod.outlook.com>
 <20240522122939.GT20229@nvidia.com>
 <BN9PR11MB527604CDF2E7FA49176200028CEB2@BN9PR11MB5276.namprd11.prod.outlook.com>
 <20240522233213.GI20229@nvidia.com>
In-Reply-To: <20240522233213.GI20229@nvidia.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?VFe7WvT6MpVmvdzdW1yWWbI0lscYJUJwttdNvU5juhvAl+50b4js6Yqhof7+?=
 =?us-ascii?Q?k7PoPxUQzf+Tp/gzwEiqcqmTGjY6b3zgHHeSi9cUgS1qLHvUtkdnTLGod8jZ?=
 =?us-ascii?Q?LM4aUnYNAWJqOiP4UBmVp2SBycUYpHTlLcBUccagm9lpqQ4ZNYoXNWdoZ1kV?=
 =?us-ascii?Q?Yz37++BOtG51P4pKPVe5ex9fxOBNcaZEF2fW0xntI9M/PdrhgrfJo2vL5AuY?=
 =?us-ascii?Q?ODnleo9RLOnHmaQPNW7xzrdRISuFZe4MxX8itN4WFfEJz8nbp0pp2ptyZ8fz?=
 =?us-ascii?Q?HD8oq4g19sgMpF3IJVuHEcmw7KICgYWEwKE997mYlD8+R3p6gIJlPdmKOfUl?=
 =?us-ascii?Q?xJX6mGBlaOMkHb082kSbqxAslsfGl8fndD9HeVUUY1p6mTMGxW5UXPBUT5pV?=
 =?us-ascii?Q?hVpXZE1NJ5A6SkdGVm0zqTVJmcjgxnDH20BNQp4eaw5kktxTf6Jg9vBCmDiL?=
 =?us-ascii?Q?kwB7hSH3Bt5M6KDcGSIF+Czu2ex6+wwsWrM0jQKkuEQI5qOSK5LzDEB3t3XK?=
 =?us-ascii?Q?S+kGcYFmoHYkN+X8D7X5uiSNT+BkuhV+CU41D4jeHHyq5xK/IY18WpDLB0Hd?=
 =?us-ascii?Q?Fwgs00X01Ahf5RUt7mAXnTi9S/A7vC+NWNE2VL/HFGSKA2d1YNlMedaHA0pg?=
 =?us-ascii?Q?2mPTskMoEWH1NpDFylzibTOfSd+L95DA6Z2DXFeszTZoz7un1mnkteHFjmcZ?=
 =?us-ascii?Q?Sf3f1FNr78rbO1KWnHeJlCjgusjM2x3gidEWsaf94Kuzm6cAp7sbRgL06QPI?=
 =?us-ascii?Q?cjFZdUvMrUp2hWRxUIxnVJZjwDXdfCnrpT6Fm7U4OPH4PCjldSK0FHuT84+G?=
 =?us-ascii?Q?YeHPL3zFJ2v/z/4r0TyyBtgepx/vxR1wrNWCLFZBbA0CtKayU/o81LTwJoE7?=
 =?us-ascii?Q?Vt8SUwlcriroK4hulgEZWLVOQvuc5oU3KJ3kXB00D67R3/tGEF7KH+A3BVVb?=
 =?us-ascii?Q?heaatDxuV1C6E3rAPEvMtCXX0FAMaVrHYcghy4l9r1/ZKjRCJpP9bFoTI8/H?=
 =?us-ascii?Q?czED6LsIiPvraMagex5a2off/XdwBn7qLdLK8bEpiv7hkGJ3Kpj/SIR990Ix?=
 =?us-ascii?Q?NUwF9R50SifmE26Nvlqq2Nkvjt/5go86DZJH4SoegHjyu3KQ4HD3JPFF60Sr?=
 =?us-ascii?Q?Ko4TupBVACuTwN+ohVWZF5d/Bmmcp69Jl7Gw8P0cyth7LggnPestiueSsPgV?=
 =?us-ascii?Q?Y1H+57rwyJ70kcrbRE6tOVaamViJRD9rlqy1uu9VVD3w/U8qGcikb4K0C3je?=
 =?us-ascii?Q?QzFq4MLNa7eEDiItTj8K0qdPcpAKEfjF0XGlc0EMQovxfD4cR7yPD1T3z09m?=
 =?us-ascii?Q?Ke83iubuKFFaQZ12ZocOaIOCxMYCwdy+nou/rk4iShu1SNhSh1DQbeyO7Xct?=
 =?us-ascii?Q?zhGfPkAxixQjsx/u5dZkucnevXSNrHB/6NLXU18LSXR5f+GNNCaX5atyyQPw?=
 =?us-ascii?Q?LyQa3YMWTrXGrmOqgYLNNWpECSfTtkxtnp4RHb15zy69FzaJCgtc/y92rN5+?=
 =?us-ascii?Q?A61EgJ5UBeZL+sr0efLoi6h9O3cy3raSKzOrcwFChetzNbgJRbFtq2WPBs2C?=
 =?us-ascii?Q?NzeP+w3wTfyzVqErHWbhIXt9rADQVnqmDB+spboZ?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
List-Id: <linux-kernel.vger.kernel.org>
List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org>
List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org>
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f2b18265-5a2c-45c7-0af0-08dc7ab8a1fd
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 May 2024 23:40:58.9939
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: d18nMh7MqBHs3jluvMmzqAtvidNXdWCZZP3UNb+QXiEfYpOxbrnVyerkIYetmdQ+2dPhZCw1THpx4BNWwNXubQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR11MB8727
X-OriginatorOrg: intel.com

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Thursday, May 23, 2024 7:32 AM
>=20
> On Wed, May 22, 2024 at 11:26:21PM +0000, Tian, Kevin wrote:
> > > From: Jason Gunthorpe <jgg@nvidia.com>
> > > Sent: Wednesday, May 22, 2024 8:30 PM
> > >
> > > On Wed, May 22, 2024 at 06:24:14AM +0000, Tian, Kevin wrote:
> > > > I'm fine to do a special check in the attach path to enable the flu=
sh
> > > > only for Intel GPU.
> > >
> > > We already effectively do this already by checking the domain
> > > capabilities. Only the Intel GPU will have a non-coherent domain.
> > >
> >
> > I'm confused. In earlier discussions you wanted to find a way to not
> > publish others due to the check of non-coherent domain, e.g. some
> > ARM SMMU cannot force snoop.
> >
> > Then you and Alex discussed the possibility of reducing pessimistic
> > flushes by virtualizing the PCI NOSNOOP bit.
> >
> > With that in mind I was thinking whether we explicitly enable this
> > flush only for Intel GPU instead of checking non-coherent domain
> > in the attach path, since it's the only device with such requirement.
>=20
> I am suggesting to do both checks:
>  - If the iommu domain indicates it has force coherency then leave PCI
>    no-snoop alone and no flush
>  - If the PCI NOSNOOP bit is or can be 0 then no flush
>  - Otherwise flush

How to judge whether PCI NOSNOOP can be 0? If following PCI spec
it can always be set to 0 but then we break the requirement for Intel
GPU. If we explicitly exempt Intel GPU in 2nd check  then what'd be
the value of doing that generic check?