Received-SPF: pass (google.com: domain of linux-kernel+bounces-185713-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Date: Wed, 22 May 2024 11:17:00 +0800
From: Yan Zhao <yan.y.zhao@intel.com>
To: Jason Gunthorpe <jgg@nvidia.com>
CC: <kvm@vger.kernel.org>, <linux-kernel@vger.kernel.org>, <x86@kernel.org>,
	<alex.williamson@redhat.com>, <kevin.tian@intel.com>,
	<iommu@lists.linux.dev>, <pbonzini@redhat.com>, <seanjc@google.com>,
	<dave.hansen@linux.intel.com>, <luto@kernel.org>, <peterz@infradead.org>,
	<tglx@linutronix.de>, <mingo@redhat.com>, <bp@alien8.de>, <hpa@zytor.com>,
	<corbet@lwn.net>, <joro@8bytes.org>, <will@kernel.org>,
	<robin.murphy@arm.com>, <baolu.lu@linux.intel.com>, <yi.l.liu@intel.com>
Subject: Re: [PATCH 5/5] iommufd: Flush CPU caches on DMA pages in
 non-coherent domains
Message-ID: <Zk1jrI8bOR5vYKlc@yzhao56-desk.sh.intel.com>
Reply-To: Yan Zhao <yan.y.zhao@intel.com>
References: <Zj3UuHQe4XgdDmDs@yzhao56-desk.sh.intel.com>
 <20240510132928.GS4650@nvidia.com>
 <ZkHEsfaGAXuOFMkq@yzhao56-desk.sh.intel.com>
 <ZkN/F3dGKfGSdf/6@nvidia.com>
 <ZkRe/HeAIgscsYZw@yzhao56-desk.sh.intel.com>
 <ZkUeWAjHuvIhLcFH@nvidia.com>
 <ZkVwS8n7ARzKAbyW@yzhao56-desk.sh.intel.com>
 <20240517170418.GA20229@nvidia.com>
 <Zkq5ZL+saJbEkfBQ@yzhao56-desk.sh.intel.com>
 <20240521160442.GI20229@nvidia.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20240521160442.GI20229@nvidia.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?JxVPqcynWksoOSvBtRvx17f22xTw7ugX5a0O7X7YHRYjAZX/DbV1kuDxcVE2?=
 =?us-ascii?Q?oU/YZSQrt2J0nTSoDeY4L2NiT08nTdK3k+QY57szxBrSrv555vOT/7SNgRmf?=
 =?us-ascii?Q?OlghlLGET5YIXoQRpngp2tjPIsSHb2wfNHXcxEWWQbScwXXihTvbA5rvTr2i?=
 =?us-ascii?Q?WFW4eaA8micFIIdv58JEwWDSTAwLmbw9WP9f1Qvri0HMp5ej9Snd8s+qvutK?=
 =?us-ascii?Q?CFjx/oq5tEfXngCKhja4NMiYiLl1CJKWTfaUechQ9eHbghkTNrP5mMBUjNLM?=
 =?us-ascii?Q?T2eq/O5/rfkKsFKYfrYjmTvDIJr5yBTpFPaQt1w79UsyUHp7QYdLBN/6J3dQ?=
 =?us-ascii?Q?FKUxyNc57VhZQp7ozDPZjA+mns1fOo8vBWPmcyrGEr9Lp8YxnolJv1ykkzv9?=
 =?us-ascii?Q?C7avYNodTBNyQlJxY83SI2fXzQdDYPbKwVQulHg6z2DQQi8QXPitOBnwRwuW?=
 =?us-ascii?Q?dqceqtjMgtujime+Eoo0s7Edmi/GWW/qrlD0B+hDvfIfA3YuqkfLlLd0zM3C?=
 =?us-ascii?Q?FJv0HW3et4bx/MtvWjlDkaLpX+OcJ04ZIyNZ35FRyVTKEXPyxL/cmdj0wNtX?=
 =?us-ascii?Q?HONUF6chERFIqm8PE0sI9i/xwLNTXHV7MOmAusagVkjtQONdHBrW/1TRkNeI?=
 =?us-ascii?Q?MxYn8aIlfv4whRqEz68Ip4gMwnVOJ1T8RVslXLmhqnlyzTSBJrG5Nr5lc6mK?=
 =?us-ascii?Q?Oktei7nfAw6Pj9X+PETTgxLxemdETKASB74IZ/UktMsY18DN7Vk7ag8+yKou?=
 =?us-ascii?Q?Hj3hrIGZNsd9P/9U0dbl6kCytq1VmFXbf6Xw9GIgSPY0torbpJB/XaNnlfnM?=
 =?us-ascii?Q?+Toa/rma1Zc/fxLBJ5kYV85ON9nEXQdJD1CYktbJ2eF1o7kfRZu7N7CViVWT?=
 =?us-ascii?Q?7Savkb+r37M/y6sXmOeRxKfD1z5F1bxUT4NiaiG1Qiqkyk8gnEUThJl/aMeo?=
 =?us-ascii?Q?UBIlup4vqGDbgarprLE2eyLXpX/i9pLWSKz1ZhJLWW3yDWvT27WItOFgVMhV?=
 =?us-ascii?Q?VK0MtWGQKfte1j2RrfVAaBna5q3SNqMx1ijcnmU9rerp7xPGHtXQ0XOMSFA+?=
 =?us-ascii?Q?HlHWUWens+MPvUZF3FpE+Db2gGWfjwFXVhzGwx4sPLrqs7Rx/+H3zlEZQ3KP?=
 =?us-ascii?Q?woPxWY8FBYEtF8dpduRQ503AxjpzpiNiO+zpy905OGf/kxpZyLr5yzx+nGNr?=
 =?us-ascii?Q?Kvlhd9B+GLpCQZIclRMkJleXMsRND5Ymar+JL74QrNfo/oHCTSk0SEYUjJ6L?=
 =?us-ascii?Q?qQV8+WQao46G0FjDTKOtANIT+SNtQBf+OGY4wQpde3xLrgVrzDUCLqPykv6z?=
 =?us-ascii?Q?DPb+C3HdmZzFkITnj64fpqT8Ez39SjxoPeytaxB2PDw0NKGsYvqiYge9+2WM?=
 =?us-ascii?Q?N2OYOtanN0EXb8C+5Blr0axxngLFwKYrbhNyfDOlwEkBJUvPvP/ZsD9mgTqT?=
 =?us-ascii?Q?oo3SsQ0kTu6iGOR63hI5V5OJH7Aemow2S0z7rYRtXWNdE/uDXea6ft3VMLFZ?=
 =?us-ascii?Q?5krV9k1Z/FT+hTS8oyn2B4ziFppwd9K43XBygBockNo6L6zVqr8w6PfgJuZ3?=
 =?us-ascii?Q?EbuR0nOUWeLhbvPRRjqjglKN26R3OpxQiM6KYIM7?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 4d50afea-c962-4107-a956-08dc7a0dc53f
X-MS-Exchange-CrossTenant-AuthSource: DS7PR11MB5966.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 03:17:54.3548
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Pnia+XNU3caPX9tL+lXoBQf26689zMavxtiGL6gvRxQ3mskzUd7c0sApDZXQIp2iS9IgUdb4TpIgB7CdYOSTlQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR11MB6903
X-OriginatorOrg: intel.com

On Tue, May 21, 2024 at 01:04:42PM -0300, Jason Gunthorpe wrote:
> On Mon, May 20, 2024 at 10:45:56AM +0800, Yan Zhao wrote:
> > On Fri, May 17, 2024 at 02:04:18PM -0300, Jason Gunthorpe wrote:
> > > On Thu, May 16, 2024 at 10:32:43AM +0800, Yan Zhao wrote:
> > > > On Wed, May 15, 2024 at 05:43:04PM -0300, Jason Gunthorpe wrote:
> > > > > On Wed, May 15, 2024 at 03:06:36PM +0800, Yan Zhao wrote:
> > > > > 
> > > > > > > So it has to be calculated on closer to a page by page basis (really a
> > > > > > > span by span basis) if flushing of that span is needed based on where
> > > > > > > the pages came from. Only pages that came from a hwpt that is
> > > > > > > non-coherent can skip the flushing.
> > > > > > Is area by area basis also good?
> > > > > > Isn't an area either not mapped to any domain or mapped into all domains?
> > > > > 
> > > > > Yes, this is what the span iterator turns into in the background, it
> > > > > goes area by area to cover things.
> > > > > 
> > > > > > But, yes, considering the limited number of non-coherent domains, it appears
> > > > > > more robust and clean to always flush for non-coherent domain in
> > > > > > iopt_area_fill_domain().
> > > > > > It eliminates the need to decide whether to retain the area flag during a split.
> > > > > 
> > > > > And flush for pin user pages, so you basically always flush because
> > > > > you can't tell where the pages came from.
> > > > As a summary, do you think it's good to flush in below way?
> > > > 
> > > > 1. in iopt_area_fill_domains(), flush before mapping a page into domains when
> > > >    iopt->noncoherent_domain_cnt > 0, no matter where the page is from.
> > > >    Record cache_flush_required in pages for unpin.
> > > > 2. in iopt_area_fill_domain(), pass in hwpt to check domain non-coherency.
> > > >    flush before mapping a page into a non-coherent domain, no matter where the
> > > >    page is from.
> > > >    Record cache_flush_required in pages for unpin.
> > > > 3. in batch_unpin(), flush if pages->cache_flush_required before
> > > >    unpin_user_pages.
> > > 
> > > It does not quite sound right, there should be no tracking in the
> > > pages of this stuff.
> > What's the downside of having tracking in the pages?
> 
> Well, a counter doesn't make sense. You could have a single sticky bit
> that indicates that all PFNs are coherency dirty and overflush them on
> every map and unmap operation.
cache_flush_required is a sticky bit actually. It's set if any PFN in the
iopt_pages is mapped into a noncoherent domain.
batch_unpin() checks this sticky bit for flush.

@@ -198,6 +198,11 @@ struct iopt_pages {
        void __user *uptr;
        bool writable:1;
        u8 account_mode;
+       /*
+        * CPU cache flush is required before mapping the pages to or after
+        * unmapping it from a noncoherent domain
+        */
+       bool cache_flush_required:1;

(Please ignore the confusing comment).

iopt->noncoherent_domain_cnt is a counter. It's increased/decreased on
non-coherent hwpt attach/detach.

@@ -53,6 +53,7 @@ struct io_pagetable {
        struct rb_root_cached reserved_itree;
        u8 disable_large_pages;
        unsigned long iova_alignment;
+       unsigned int noncoherent_domain_cnt;
 };

Since iopt->domains contains no coherency info, this counter helps
iopt_area_fill_domains() to decide whether to flush pages and set sticky bit
cache_flush_required in iopt_pages.
Though it's not that useful to iopt_area_fill_domain(), after your suggestion
to pass in hwpt.

> This is certainly the simplest option, but gives the maximal flushes.

Why does this give the maximal flushes?
Considering the flush after unmap,
- With a sticky bit in iopt_pages, once a iopt_pages has been mapped into a
  non-coherent domain, the PFNs in the iopt_pages will be flushed for only once
  right before the page is unpinned.

- But if we do the flush after each iopt_area_unmap_domain_range() for each
  non-coherent domain, then the flush count for each PFN is the count of
  non-coherent domains.

> 
> If you want to minimize flushes then you can't store flush
> minimization information in the pages because it isn't global to the
> pages and will not be accurate enough.
> 
> > > If pfn_reader_fill_span() does batch_from_domain() and
> > > the source domain's storage_domain is non-coherent then you can skip
> > > the flush. This is not pedantically perfect in skipping all flushes, but
> > > in practice it is probably good enough.
> 
> > We don't know whether the source storage_domain is non-coherent since
> > area->storage_domain is of "struct iommu_domain".
>  
> > Do you want to add a flag in "area", e.g. area->storage_domain_is_noncoherent,
> > and set this flag along side setting storage_domain?
> 
> Sure, that could work.
When the storage_domain is set in iopt_area_fill_domains(),
    "area->storage_domain = xa_load(&area->iopt->domains, 0);"
is there a convenient way to know the storage_domain is non-coherent?

> 
> > > __iopt_area_unfill_domain() (and children) must flush after
> > > iopt_area_unmap_domain_range() if the area's domain is
> > > non-coherent. This is also not perfect, but probably good enough.
> > Do you mean flush after each iopt_area_unmap_domain_range() if the domain is
> > non-coherent?
> > The problem is that iopt_area_unmap_domain_range() knows only IOVA, the
> > IOVA->PFN relationship is not available without iommu_iova_to_phys() and
> > iommu_domain contains no coherency info.
> 
> Yes, you'd have to read back the PFNs on this path which it doesn't do
> right now.. Given this pain it would be simpler to have one bit in the
> pages that marks it permanently non-coherent and all pfns will be
> flushed before put_page is called.
> 
> The trouble with a counter is that the count going to zero doesn't
> really mean we flushed the PFN if it is being held someplace else.
Not sure if you are confused between iopt->noncoherent_domain_cnt and
pages->cache_flush_required.

iopt->noncoherent_domain_cnt is increased/decreased on non-coherent hwpt
attach/detach.

Once iopt->noncoherent_domain_cnt is non-zero, sticky bit cache_flush_required
in iopt_pages will be set during filling domain, PFNs in the iopt_pages will be
flushed right before unpinning even though iopt->noncoherent_domain_cnt might
have gone to 0 at that time.