Received-SPF: pass (google.com: domain of linux-kernel+bounces-185811-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Date: Wed, 22 May 2024 14:29:19 +0800
From: Yan Zhao <yan.y.zhao@intel.com>
To: Jason Gunthorpe <jgg@nvidia.com>, <kvm@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>, <x86@kernel.org>,
	<alex.williamson@redhat.com>, <kevin.tian@intel.com>,
	<iommu@lists.linux.dev>, <pbonzini@redhat.com>, <seanjc@google.com>,
	<dave.hansen@linux.intel.com>, <luto@kernel.org>, <peterz@infradead.org>,
	<tglx@linutronix.de>, <mingo@redhat.com>, <bp@alien8.de>, <hpa@zytor.com>,
	<corbet@lwn.net>, <joro@8bytes.org>, <will@kernel.org>,
	<robin.murphy@arm.com>, <baolu.lu@linux.intel.com>, <yi.l.liu@intel.com>
Subject: Re: [PATCH 5/5] iommufd: Flush CPU caches on DMA pages in
 non-coherent domains
Message-ID: <Zk2Qv4pnSKZBsLYv@yzhao56-desk.sh.intel.com>
Reply-To: Yan Zhao <yan.y.zhao@intel.com>
References: <20240510132928.GS4650@nvidia.com>
 <ZkHEsfaGAXuOFMkq@yzhao56-desk.sh.intel.com>
 <ZkN/F3dGKfGSdf/6@nvidia.com>
 <ZkRe/HeAIgscsYZw@yzhao56-desk.sh.intel.com>
 <ZkUeWAjHuvIhLcFH@nvidia.com>
 <ZkVwS8n7ARzKAbyW@yzhao56-desk.sh.intel.com>
 <20240517170418.GA20229@nvidia.com>
 <Zkq5ZL+saJbEkfBQ@yzhao56-desk.sh.intel.com>
 <20240521160442.GI20229@nvidia.com>
 <Zk1jrI8bOR5vYKlc@yzhao56-desk.sh.intel.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <Zk1jrI8bOR5vYKlc@yzhao56-desk.sh.intel.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?if19Y8x2Uj8iSAi85ylC8988Z8iVuKt7ssSeyI1OhKMl6RkbiU1Nthp1CcHU?=
 =?us-ascii?Q?OOSZHANOkpB+7EGgY1ZAhJw6IcxXP4BfImrviUmquTKE9SPSx2SJC8HOYXjy?=
 =?us-ascii?Q?9ZVGBhRvq6GDHSZLjhGjyQxrVzTNU5fW3oYKYaM5H0gNkWuk1n13ElVJFKxw?=
 =?us-ascii?Q?LzNOwg46F6Si+byzD5qbhZJv99JxfoidvR7Qj0PWu83ur4qdqxMUuu8YiHJy?=
 =?us-ascii?Q?NQRtoUjAyYMa+8VH5XSkWkS6Gc+GGR1+Ud8gzeQooysodbJRTQ64df8D3UV5?=
 =?us-ascii?Q?ozRRQOmFBrNdSlqhU+uMBu1lX0ylGa9BPaCFl9FGg0ptDD2oCMxTi5wUfiO0?=
 =?us-ascii?Q?6V1eOKPhlLq81RP3PsaQ48zQ+5/jsq78T7b7amVXTJte5QpARS65B7Kqiqqf?=
 =?us-ascii?Q?Hx8tR/gxU3K2Tgm3beKsbU9vRMgXoM0Or3UJjTQhyJxEAjLZFud6Iqc4eMhK?=
 =?us-ascii?Q?iQh6FSXPqi04C5MWwu5rxKH5E/Ek3FXpEzEj54JRcG7U5hb07qeTIOTVwh9W?=
 =?us-ascii?Q?oZ3A5s0HUmIqVdCs9Ox/EX7/cjYM3krsRv4/mSUJV6GcWsdt3zwGuC8Iqs3n?=
 =?us-ascii?Q?3OrdsX0rir5C8UOKrzIrmqi326tz3UTIJ5PH7d37He0n81RsO6+Pl/Fr15gF?=
 =?us-ascii?Q?KDN69N707okf4sN1O0WETinn7sSQWVat42T25N+zvtVKrxPFIwognp79gMGp?=
 =?us-ascii?Q?5YNa6KluQ1VjCH9unwbi3/BYs7jlGLuWwDysN4FmIy+zylSStNerKdRSnVsa?=
 =?us-ascii?Q?Pg8ajCjHyUFfdu+Bfbe5T79bLCHrXuy47FWBQZ6r6kKU4/q4Ix4E2YZG2bHt?=
 =?us-ascii?Q?ErHKlCytkzf/gXNkRTvgZEvLS8Piqwo6BhQMlrZP5RwM7UYoaOKU4/XIGVfF?=
 =?us-ascii?Q?qhxbFPtRs//wspPhd67zplr15YcE8gOeIe9fKLwASHE0c0CgRzaw9djkX6cy?=
 =?us-ascii?Q?7ZerEXBL5u/liv3LaBvhdTII9YoAwYq20UcjDCytMzxYJIlss5HgUDug1LNV?=
 =?us-ascii?Q?hO6dQDM6pCAKbeWlV+slX/P8gqukItUogTJnL8hcLWkWS2N/JFujO99a2oV0?=
 =?us-ascii?Q?ox6v9ZFO7BZoP0dtYYgCeLk/ZQbdqd2sPOdcEty7MusQX3k0Zny3Sxnm0bOe?=
 =?us-ascii?Q?1FqLOva0P37iLUQohD4GjTlwnCGp7lswuNK77dhjjF79HjfEPvEBsyfkoMbG?=
 =?us-ascii?Q?i0SbIWeqVh4+feeo3Anl12s6ZmMeDEJdrBR9iWsmo4aKpJUlBFGRBJX4QguA?=
 =?us-ascii?Q?d6/EoySxMcgVyS/Pr162qryEQntzP9BcOaajpOwuQ41+GRdiUReWPA/dsO72?=
 =?us-ascii?Q?Nz0ZsAPxse25IpvM02RHdYxJE77DzGkaDoi3KVlDJrrvZeZbGQ7Ful/uw2gQ?=
 =?us-ascii?Q?fwN1zg9AJ7RQsyeMZWN6FzdCMWG1V2Fkr/nqWn4bn5DgW9v6MsjkdHquD0Ev?=
 =?us-ascii?Q?Uuq1kM2FwIBYFXUFT6MgYYz3WIc/Gllv7GDI9E+9/lf3mPPJ41fwOrfMYs4B?=
 =?us-ascii?Q?7tDTmUjEad5RzLjSyQnh+R0yUMA6QWZPXxcV5qIUyoRKox8Kcu1id+C5iMFw?=
 =?us-ascii?Q?YWeC2pfvYfn4dxY970Z4ncQ/6b76utcd1Uhqiiim?=
X-MS-Exchange-CrossTenant-Network-Message-Id: ccb60d34-e300-46d5-d705-08dc7a28a1fd
X-MS-Exchange-CrossTenant-AuthSource: DS7PR11MB5966.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 06:30:11.5884
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: TF2EgO8QcathqSzDuMsZBx31kiFo2IY8Av3hYt/BPqaV27cDf4bfGIp90yoh23KuUqqu1UhzdD0TVpSAJc8Tag==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB8089
X-OriginatorOrg: intel.com

> > If you want to minimize flushes then you can't store flush
> > minimization information in the pages because it isn't global to the
> > pages and will not be accurate enough.
> > 
> > > > If pfn_reader_fill_span() does batch_from_domain() and
> > > > the source domain's storage_domain is non-coherent then you can skip
> > > > the flush. This is not pedantically perfect in skipping all flushes, but
> > > > in practice it is probably good enough.
> > 
> > > We don't know whether the source storage_domain is non-coherent since
> > > area->storage_domain is of "struct iommu_domain".
> >  
> > > Do you want to add a flag in "area", e.g. area->storage_domain_is_noncoherent,
> > > and set this flag along side setting storage_domain?
> > 
> > Sure, that could work.
> When the storage_domain is set in iopt_area_fill_domains(),
>     "area->storage_domain = xa_load(&area->iopt->domains, 0);"
> is there a convenient way to know the storage_domain is non-coherent?
Also asking for when storage_domain is switching to an arbitrary remaining domain
in iopt_unfill_domain().

And in iopt_area_unfill_domains(), after iopt_area_unmap_domain_range()
of a non-coherent domain which is not the storage domain, how can we know that
the domain is non-coherent?