Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 31 Aug 2022 17:09:17 -0400
From:   Peter Xu <peterx@redhat.com>
To:     Yang Shi <shy828301@gmail.com>
Cc:     David Hildenbrand <david@redhat.com>, linux-kernel@vger.kernel.org,
        linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>,
        Jason Gunthorpe <jgg@nvidia.com>,
        John Hubbard <jhubbard@nvidia.com>,
        Andrea Arcangeli <aarcange@redhat.com>,
        Hugh Dickins <hughd@google.com>,
        "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Subject: Re: [PATCH v1] mm/ksm: update stale comment in write_protect_page()
Message-ID: <Yw/N/RuCdcjlOZPS@xz-m1.local>
References: <20220831083024.37138-1-david@redhat.com>
 <CAHbLzkqeDAnCdt3q4E2RZw64QEzVaO_pseR3VaoHUhB+rZFcZQ@mail.gmail.com>
 <Yw+uASvgWU5+9PvI@xz-m1.local>
 <CAHbLzkpH=a-gbwZYfJx6qD_Rvz6AEP+xwCKDbwJRPNt9W3Anxg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <CAHbLzkpH=a-gbwZYfJx6qD_Rvz6AEP+xwCKDbwJRPNt9W3Anxg@mail.gmail.com>
Precedence: bulk

On Wed, Aug 31, 2022 at 01:38:21PM -0700, Yang Shi wrote:
> On Wed, Aug 31, 2022 at 11:52 AM Peter Xu <peterx@redhat.com> wrote:
> >
> > On Wed, Aug 31, 2022 at 10:55:43AM -0700, Yang Shi wrote:
> > > On Wed, Aug 31, 2022 at 1:30 AM David Hildenbrand <david@redhat.com> wrote:
> > > >
> > > > The comment is stale, because a TLB flush is no longer sufficient and
> > > > required to synchronize against concurrent GUP-fast. This used to be true
> > > > in the past, whereby a TLB flush would have implied an IPI on architectures
> > > > that support GUP-fast, resulting in GUP-fast that disables local interrupts
> > > > from completing before completing the flush.
> > >
> > > Hmm... it seems there might be problem for THP collapse IIUC. THP
> > > collapse clears and flushes pmd before doing anything on pte and
> > > relies on interrupt disable of fast GUP to serialize against fast GUP.
> > > But if TLB flush is no longer sufficient, then we may run into the
> > > below race IIUC:
> > >
> > >          CPU A                                                CPU B
> > > THP collapse                                             fast GUP
> > >
> > > gup_pmd_range() <-- see valid pmd
> > >
> > > gup_pte_range() <-- work on pte
> > > clear pmd and flush TLB
> > > __collapse_huge_page_isolate()
> > >     isolate page <-- before GUP bump refcount
> > >
> > >    pin the page
> > > __collapse_huge_page_copy()
> > >     copy data to huge page
> > >     clear pte (don't flush TLB)
> > > Install huge pmd for huge page
> > >
> > > return the obsolete page
> >
> > Maybe the pmd level tlb flush is still needed, but on pte level it's
> > optional (where we can rely on fast-gup rechecking on the pte change)?
> 
> Do you mean in khugepaged?

What I wanted to say before was that the immediate tlb flush (after pgtable
entry cleared) seems to be only needed by pmd level to guarantee safety
with concurrent fast-gup, since fast-gup can detect pte change after
pinning, and that should already guarantee safe concurrent fast-gup to me.

After reading the other emails, afaiu we're on the same page.

> It does TLB flush, but some arches may not use IPI.

Yeah, I see that ppc book3s code has customized pmdp_collapse_flush() to
explicit do the IPIs besides tlb flush using smp calls.

I assume pmdp_collapse_flush() should always be properly implemented to
guarantee safety against fast-gup, or I also agree it could be a bug.

-- 
Peter Xu