Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 16 Aug 2022 22:45:37 -0400
From:   Peter Xu <peterx@redhat.com>
To:     Alistair Popple <apopple@nvidia.com>
Cc:     huang ying <huang.ying.caritas@gmail.com>, linux-mm@kvack.org,
        akpm@linux-foundation.org, linux-kernel@vger.kernel.org,
        "Sierra Guiza, Alejandro (Alex)" <alex.sierra@amd.com>,
        Felix Kuehling <Felix.Kuehling@amd.com>,
        Jason Gunthorpe <jgg@nvidia.com>,
        John Hubbard <jhubbard@nvidia.com>,
        David Hildenbrand <david@redhat.com>,
        Ralph Campbell <rcampbell@nvidia.com>,
        Matthew Wilcox <willy@infradead.org>,
        Karol Herbst <kherbst@redhat.com>,
        Lyude Paul <lyude@redhat.com>, Ben Skeggs <bskeggs@redhat.com>,
        Logan Gunthorpe <logang@deltatee.com>, paulus@ozlabs.org,
        linuxppc-dev@lists.ozlabs.org, Huang Ying <ying.huang@intel.com>,
        stable@vger.kernel.org
Subject: Re: [PATCH v2 1/2] mm/migrate_device.c: Copy pte dirty bit to page
Message-ID: <YvxWUY9eafFJ27ef@xz-m1.local>
References: <6e77914685ede036c419fa65b6adc27f25a6c3e9.1660635033.git-series.apopple@nvidia.com>
 <CAC=cRTPGiXWjk=CYnCrhJnLx3mdkGDXZpvApo6yTbeW7+ZGajA@mail.gmail.com>
 <Yvv/eGfi3LW8WxPZ@xz-m1.local>
 <871qtfvdlw.fsf@nvdebian.thelocal>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <871qtfvdlw.fsf@nvdebian.thelocal>
Precedence: bulk

On Wed, Aug 17, 2022 at 11:49:03AM +1000, Alistair Popple wrote:
> 
> Peter Xu <peterx@redhat.com> writes:
> 
> > On Tue, Aug 16, 2022 at 04:10:29PM +0800, huang ying wrote:
> >> > @@ -193,11 +194,10 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp,
> >> >                         bool anon_exclusive;
> >> >                         pte_t swp_pte;
> >> >
> >> > +                       flush_cache_page(vma, addr, pte_pfn(*ptep));
> >> > +                       pte = ptep_clear_flush(vma, addr, ptep);
> >>
> >> Although I think it's possible to batch the TLB flushing just before
> >> unlocking PTL.  The current code looks correct.
> >
> > If we're with unconditionally ptep_clear_flush(), does it mean we should
> > probably drop the "unmapped" and the last flush_tlb_range() already since
> > they'll be redundant?
> 
> This patch does that, unless I missed something?

Yes it does.  Somehow I didn't read into the real v2 patch, sorry!

> 
> > If that'll need to be dropped, it looks indeed better to still keep the
> > batch to me but just move it earlier (before unlock iiuc then it'll be
> > safe), then we can keep using ptep_get_and_clear() afaiu but keep "pte"
> > updated.
> 
> I think we would also need to check should_defer_flush(). Looking at
> try_to_unmap_one() there is this comment:
> 
> 			if (should_defer_flush(mm, flags) && !anon_exclusive) {
> 				/*
> 				 * We clear the PTE but do not flush so potentially
> 				 * a remote CPU could still be writing to the folio.
> 				 * If the entry was previously clean then the
> 				 * architecture must guarantee that a clear->dirty
> 				 * transition on a cached TLB entry is written through
> 				 * and traps if the PTE is unmapped.
> 				 */
> 
> And as I understand it we'd need the same guarantee here. Given
> try_to_migrate_one() doesn't do batched TLB flushes either I'd rather
> keep the code as consistent as possible between
> migrate_vma_collect_pmd() and try_to_migrate_one(). I could look at
> introducing TLB flushing for both in some future patch series.

should_defer_flush() is TTU-specific code?

IIUC the caller sets TTU_BATCH_FLUSH showing that tlb can be omitted since
the caller will be responsible for doing it.  In migrate_vma_collect_pmd()
iiuc we don't need that hint because it'll be flushed within the same
function but just only after the loop of modifying the ptes.  Also it'll be
with the pgtable lock held.

Indeed try_to_migrate_one() doesn't do batching either, but IMHO it's just
harder to do due to using the vma walker (e.g., the lock is released in
not_found() implicitly so iiuc it's hard to do tlb flush batching safely in
the loop of page_vma_mapped_walk).  Also that's less a concern since the
loop will only operate upon >1 ptes only if it's a thp page mapped in ptes.
OTOH migrate_vma_collect_pmd() operates on all ptes on a pmd always.

No strong opinion anyway, it's just a bit of a pity because fundamentally
this patch is removing the batching tlb flush.  I also don't know whether
there'll be observe-able perf degrade for migrate_vma_collect_pmd(),
especially on large machines.

Thanks,

-- 
Peter Xu