Received-SPF: pass (google.com: domain of linux-kernel+bounces-185732-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Date: Wed, 22 May 2024 11:41:42 +0800
From: Yan Zhao <yan.y.zhao@intel.com>
To: Jason Gunthorpe <jgg@nvidia.com>
CC: Christoph Hellwig <hch@infradead.org>, <kvm@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>, <x86@kernel.org>,
	<alex.williamson@redhat.com>, <kevin.tian@intel.com>,
	<iommu@lists.linux.dev>, <pbonzini@redhat.com>, <seanjc@google.com>,
	<dave.hansen@linux.intel.com>, <luto@kernel.org>, <peterz@infradead.org>,
	<tglx@linutronix.de>, <mingo@redhat.com>, <bp@alien8.de>, <hpa@zytor.com>,
	<corbet@lwn.net>, <joro@8bytes.org>, <will@kernel.org>,
	<robin.murphy@arm.com>, <baolu.lu@linux.intel.com>, <yi.l.liu@intel.com>,
	Russell King <linux@armlinux.org.uk>
Subject: Re: [PATCH 3/5] x86/mm: Introduce and export interface
 arch_clean_nonsnoop_dma()
Message-ID: <Zk1pdkQj/CuBahqx@yzhao56-desk.sh.intel.com>
Reply-To: Yan Zhao <yan.y.zhao@intel.com>
References: <20240507061802.20184-1-yan.y.zhao@intel.com>
 <20240507062044.20399-1-yan.y.zhao@intel.com>
 <ZktZDmcNnsHhp4Tm@infradead.org>
 <20240521154939.GH20229@nvidia.com>
 <20240521160016.GA2513156@nvidia.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20240521160016.GA2513156@nvidia.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?XUSMOeOLOKNMTXrl0k5fnXh6FmqglDzzv52nekpxNBpZPKgB4nb4QygQTLmt?=
 =?us-ascii?Q?g9KRHXz0ShOaZEi4ziPBTdzd3kgNHSrxsLlwZxrMg2FSxp7mgpFZkEga2ega?=
 =?us-ascii?Q?cgA+fsLXxNB++1JVoQoeeY0KpS6dxgSKkqrHeE9JG2zyMZnPdT806H/rFQSl?=
 =?us-ascii?Q?V2oJzqcKMWaHtbiqQiPQewSaca7Z5j3xgTVpVzx4O0mKaPHGPOWcrUdepwKe?=
 =?us-ascii?Q?tggxx2iZ2jwY3rqXAFzl4eUPWxeKV8f3ambfPulASOBAByB1Gl/It67H9fgW?=
 =?us-ascii?Q?Mb8B1BMqvUUrAb3fzzAbd/qPBalsSM/5nqFFcxXsN4BDWH1aSGGOZEtDi/e1?=
 =?us-ascii?Q?Lb6f1f/WLtocFiEzOkTKwJ7tvN3pFq50SDmyuHjBQFalGjVkaiQI3QJKihUP?=
 =?us-ascii?Q?QSnCKBRTWzxdAT4LhS4y5ngKB/SAFbN8VwHXUPCsu+2Q0gBIQhfvzOMIAwZ9?=
 =?us-ascii?Q?9xAP03dSymVchN4KYptupjQcxJocIJxRcuPPK3N/1KDnDC1DqT1Vk2MbKvZc?=
 =?us-ascii?Q?B+9lsT139M4Gj9aRPva8DXoqmo4G0gFG1qaKKPuqjkJpn2226ptABqRh5InL?=
 =?us-ascii?Q?bvYPdQtNKNbrb9eUGVQX9Xkkx2kldVqBMVi4fOWadBvDO7rTCFJxBhe+gOd9?=
 =?us-ascii?Q?7PF/mf7pK/azlh6kqZ28gqc3NU/FJEW0KROCz70ql0F7Dzh7k+7AVSY9yLdq?=
 =?us-ascii?Q?RyZS6Sts7olmmpPzqSnD6PPmElINWELMbcaIzftJ0FgxlExx6Wpxxl9RczjD?=
 =?us-ascii?Q?hvlE3RXIX0i2cvmI7bQIRVycvBI0XqUTMmotnwrg2jGupdxC0D3gbJEC9IvR?=
 =?us-ascii?Q?N1V6PHincQt88oiqtOSRdr13cwMolC1kF4j8Ru3YRr+2qb+66pJnS+SdvhWV?=
 =?us-ascii?Q?u1Z8EhUyeQgDoW8dtlyId6fGDTYPAZVSPDxqSQViE3yUj70kmYwU5pMtxR3c?=
 =?us-ascii?Q?PRbMXifHOXbn3c8sn2xd2s/TEHget1j2RSF3KlHhThpi3NZe2Fn0zCN66gut?=
 =?us-ascii?Q?X6rPhOJWw3HHcvzwls3PeJBjoU1KTM/Id2r6LBLAvmiFK9GvOPFuznAWWh+/?=
 =?us-ascii?Q?X//e+14q7tg24PraIbG5wSDoD13q+xQiX18CusfC1Mcvt/MOGLArRXLWCDuL?=
 =?us-ascii?Q?VkARwpggBtT6Cd+sQ3L5TuqNQBvDhRaioVB/cEC5cPa70SsWZre/NQ415K/3?=
 =?us-ascii?Q?hB9uzS+RzhYrHxwfQSP5U5kskYOwcVNVlm1yxAIMpDCc1M3I9zWqw9TocWdQ?=
 =?us-ascii?Q?JTSwq75WbpIrnznf1T3WEtroqppBHORWkq7FnZjrncc9odm+syFb55Re3UMH?=
 =?us-ascii?Q?gChmB/g3cm49aPaDIfFbvOSP4gK2cx3itGs5rrzu+YxQVryQc/+qu/Ai9tpZ?=
 =?us-ascii?Q?0plLsO6Mksbpxo1IHbAwW7z8kw0qIOiVXusRdnQ1emD4B49zku+69OUj9c9a?=
 =?us-ascii?Q?v5GJ/ePu7Of1Riy0MeO7itY6V9e3iqCD7/CvVBY7LUyDX0l5RRWN1Odd27zZ?=
 =?us-ascii?Q?4/4SBrtfe5XNCyYtwYuOFNvdFQal6FhsUPvlqv2ySmiCg1tiS01difdv2f6d?=
 =?us-ascii?Q?3svDHsoKFK9df8Hb04vI6U6it/fClnZZZPvvbpz9?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 03c260c7-1c8f-4a4e-64fa-08dc7a11384c
X-MS-Exchange-CrossTenant-AuthSource: DS7PR11MB5966.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 03:42:35.8647
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: QCCgfdet1xZKoqsEnlx5wnxdbw2Yr34/lJDai4FRd3PhgqvaJTo/it2lm83SrVVVsj2CuNqXebu7cDC0Y0SSwQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR11MB6508
X-OriginatorOrg: intel.com

On Tue, May 21, 2024 at 01:00:16PM -0300, Jason Gunthorpe wrote:
> On Tue, May 21, 2024 at 12:49:39PM -0300, Jason Gunthorpe wrote:
> > On Mon, May 20, 2024 at 07:07:10AM -0700, Christoph Hellwig wrote:
> > > On Tue, May 07, 2024 at 02:20:44PM +0800, Yan Zhao wrote:
> > > > Introduce and export interface arch_clean_nonsnoop_dma() to flush CPU
> > > > caches for memory involved in non-coherent DMAs (DMAs that lack CPU cache
> > > > snooping).
> > > 
> > > Err, no.  There should really be no exported cache manipulation macros,
> > > as drivers are almost guaranteed to get this wrong.  I've added
> > > Russell to the Cc list who has been extremtly vocal about this at least
> > > for arm.
> > 
> > We could possibly move this under some IOMMU core API (ie flush and
> > map, unmap and flush), the iommu APIs are non-modular so this could
> > avoid the exported symbol.
> 
> Though this would be pretty difficult for unmap as we don't have the
> pfns in the core code to flush. I don't think we have alot of good
> options but to make iommufd & VFIO handle this directly as they have
> the list of pages to flush on the unmap side. Use a namespace?
Given we'll rename this function to arch_flush_cache_phys() which takes physical
address as input, and there're already clflush_cache_range() and
arch_invalidate_pmem() exported with vaddr as input, is this export still good?