Received-SPF: pass (google.com: domain of linux-kernel+bounces-180721-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
From: "Tian, Kevin" <kevin.tian@intel.com>
To: "Zhao, Yan Y" <yan.y.zhao@intel.com>, Alex Williamson
	<alex.williamson@redhat.com>
CC: "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"x86@kernel.org" <x86@kernel.org>, "jgg@nvidia.com" <jgg@nvidia.com>,
	"iommu@lists.linux.dev" <iommu@lists.linux.dev>, "pbonzini@redhat.com"
	<pbonzini@redhat.com>, "seanjc@google.com" <seanjc@google.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>, "peterz@infradead.org"
	<peterz@infradead.org>, "tglx@linutronix.de" <tglx@linutronix.de>,
	"mingo@redhat.com" <mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>,
	"hpa@zytor.com" <hpa@zytor.com>, "corbet@lwn.net" <corbet@lwn.net>,
	"joro@8bytes.org" <joro@8bytes.org>, "will@kernel.org" <will@kernel.org>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>, "baolu.lu@linux.intel.com"
	<baolu.lu@linux.intel.com>, "Liu, Yi L" <yi.l.liu@intel.com>
Subject: RE: [PATCH 4/5] vfio/type1: Flush CPU caches on DMA pages in
 non-coherent domains
Thread-Topic: [PATCH 4/5] vfio/type1: Flush CPU caches on DMA pages in
 non-coherent domains
Thread-Index: AQHaoEb1yTw+2+R5T02uRFeJZuQMNbGPN+OAgAER64CAAGvrAIAEE0QAgATA9BA=
Date: Thu, 16 May 2024 07:53:03 +0000
Message-ID: <BN9PR11MB527606B1387E3CF37F5442F68CED2@BN9PR11MB5276.namprd11.prod.outlook.com>
References: <20240507061802.20184-1-yan.y.zhao@intel.com>
 <20240507062138.20465-1-yan.y.zhao@intel.com>
 <20240509121049.58238a6f.alex.williamson@redhat.com>
 <Zj33cUe7HYOIfj5N@yzhao56-desk.sh.intel.com>
 <20240510105728.76d97bbb.alex.williamson@redhat.com>
 <ZkG9IEQwi7HG3YBk@yzhao56-desk.sh.intel.com>
In-Reply-To: <ZkG9IEQwi7HG3YBk@yzhao56-desk.sh.intel.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?CSky3Wa7YdxnUTbilivQz84oWRYt2I8qq6AhhQQpRVu6MgxtE6sBk2V8IW9R?=
 =?us-ascii?Q?dUeNq55alPEMMP90gbnFad/nL9mmiTvwbeJDOvADhbjjodZ1owY2axJ/IZ35?=
 =?us-ascii?Q?UyHu3PuSoEemXY3nByrnF5PuCtsW3cnFdHwuIypi0TZeQqCERxjywb1FFApS?=
 =?us-ascii?Q?0VXDdDFWgaxDbGLl/DHGLIb5N4S4/3sBPss8FBOuKeXiUqGYEt/nkFlOmjXW?=
 =?us-ascii?Q?XnfuMqGjMCAKDXG/MDUMbo+eXQhmlPajatsFDjnNcJguRaB1TsFgFIqGxxdU?=
 =?us-ascii?Q?sv1gERM643O0W+8bi70Fq1N2vKQsU8UtLhsJtQfGrUDF5brkRQNhJ2nlhsYQ?=
 =?us-ascii?Q?BGSjzoc01sNiYWrbM9XibYaiSIGiCxwSxCXROeVh2bpz8x/5nCqtq84Aw8Pc?=
 =?us-ascii?Q?fhyw0ZnMcC9accSqQVavp7jQtqb8/fHNUQ1hDptVNFlDCNDL+JLMZtuqGx20?=
 =?us-ascii?Q?5UfTlIVxAub+zFCBFVZm1hSNhnVlZQ20hq6GVN7seT2nt6dv7V+Sp38Cls68?=
 =?us-ascii?Q?mG97utowlY8t2KCVCHJowv+U0RJ28NuJcEC1O3guaI8ASVFfc8dsVZ1ik5SM?=
 =?us-ascii?Q?DizSibLbzKNu8oy+dIEjhu8Wuk7jnx72orErZ3o0VRE3GxVH3/JQCsPfzhOP?=
 =?us-ascii?Q?Mvb2+6qbCPxDxaNVw+sD1GBRV7lbcqt3RnoB7qDORqI5N/IM+xlNp26VKojb?=
 =?us-ascii?Q?wIPAkNqPIdCsWHrTUxdIAsxG7EAxbkJ/p6XWV3QavgYRKTX4gWT3k6erfPZQ?=
 =?us-ascii?Q?omZvo1DSyvceldESpLYHZQn8EVYanmbSczUAqBp3UIEQoBEQOQMuY2IT7d2Y?=
 =?us-ascii?Q?7gqCa3D7XMvfNaddO0JEfKe0FE443bls6p/sSrZMa6wbioKcXWWIW8PVdLJ4?=
 =?us-ascii?Q?NXWMKh+OduWxujmfNR+Kx3c7skfab8QFqIZ2y3qY6q6gNN4BmZTAy2AT2PN6?=
 =?us-ascii?Q?P94rft9Xb79ojdStUBinG+rMclKSMsAa6sOH0Bpa42AnC1+fIUUDFlojQVV9?=
 =?us-ascii?Q?w8QA0ZrIZ2IB2fUJ1YcT3OSAGUigKgm5McSRZXTNLcfHwATd4+HcO8kF6E2n?=
 =?us-ascii?Q?g6bjEk5J6mMh2fR2247NKHeTw9h/nUGq2Qoa+epSi1qqUq4osRHSNu2Yyp/j?=
 =?us-ascii?Q?j9hcb9fJ+K/y1eByL73w8oWDUx0IVjSwGldKj2shbwFp6yWyA3fVDcYkU+v2?=
 =?us-ascii?Q?kJvRNWmZK+Gi/OqDJ0ODFLslCiv6O0NuRZkggdQiqChQmCSYInn+2VJqzKqw?=
 =?us-ascii?Q?Pu7gW8cSxczQbB+1HXI/i5Q3CWx8ZhPJB2iCLOYxDg4KQxqEbX9V/DSel79f?=
 =?us-ascii?Q?47MB54/qN4q3iIUKb8I5YpqP0DMUD7V5SC/oTOzZSw8SDqiFEHC1LGyM5V+3?=
 =?us-ascii?Q?VPaFzqYZxrvUGnSCIu0bo1VsCsgGlS4qXStdaVQeatdu8Hch2xp+tuCq1gIu?=
 =?us-ascii?Q?M3VOauz3z/ZXaKbOxfxqSv7oS7BaA6W4bKW5a1bAw/urtC3fDRB4U2gd9eEn?=
 =?us-ascii?Q?qzoEbrJctmSBWEk0naqxm5arxH3VHPh2X1iBbTr5BiRk1+gpGqjw4BajrujT?=
 =?us-ascii?Q?6jwux9KNTSVx9UB9TIhB42B7sIHG4na8XOOIzCfU?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
List-Id: <linux-kernel.vger.kernel.org>
List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org>
List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org>
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b17a2be3-763d-4e5d-951e-08dc757d371f
X-MS-Exchange-CrossTenant-originalarrivaltime: 16 May 2024 07:53:03.5669
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Cg9BRET59dQb6SzI0HvB6EIV9bFNepwtGWRHOQbcediMYFogPZ1GgQ7x0LIuX89yYlIigmni6sipI7VxtJ30Ng==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR11MB5102
X-OriginatorOrg: intel.com

> From: Zhao, Yan Y <yan.y.zhao@intel.com>
> Sent: Monday, May 13, 2024 3:11 PM
> On Fri, May 10, 2024 at 10:57:28AM -0600, Alex Williamson wrote:
> > On Fri, 10 May 2024 18:31:13 +0800
> > Yan Zhao <yan.y.zhao@intel.com> wrote:
> > >
> > > The dma->cache_flush_required is to mark whether pages in a vfio_dma
> requires
> > > cache flush in the subsequence mapping into the first non-coherent
> domain
> > > and page unpinning.
> >
> > How do we arrive at a sequence where we have dma-
> >cache_flush_required
> > that isn't the result of being mapped into a domain with
> > !domain->enforce_cache_coherency?
> Hmm, dma->cache_flush_required IS the result of being mapped into a
> domain with
> !domain->enforce_cache_coherency.
> My concern only arrives from the actual code sequence, i.e.
> dma->cache_flush_required is set to true before the actual mapping.
>=20
> If we rename it to dma->mapped_noncoherent and only set it to true after
> the
> actual successful mapping, it would lead to more code to handle flushing =
for
> the
> unwind case.
> Currently, flush for unwind is handled centrally in vfio_unpin_pages_remo=
te()
> by checking dma->cache_flush_required, which is true even before a full
> successful mapping, so we won't miss flush on any pages that are mapped
> into a
> non-coherent domain in a short window.
>=20

What about storing a vfio_iommu pointer in vfio_dma? Or pass an extra
parameter to vfio_unpin_pages_remote()...