Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Wed, 22 Sep 2021 16:23:21 -0400
From:   Peter Xu <peterx@redhat.com>
To:     Yang Shi <shy828301@gmail.com>
Cc:     David Hildenbrand <david@redhat.com>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Linux MM <linux-mm@kvack.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        Andrea Arcangeli <aarcange@redhat.com>,
        Axel Rasmussen <axelrasmussen@google.com>,
        Hugh Dickins <hughd@google.com>,
        Nadav Amit <nadav.amit@gmail.com>
Subject: Re: [PATCH] mm/khugepaged: Detecting uffd-wp vma more efficiently
Message-ID: <YUuQuRwKj66lBwMv@t490s>
References: <20210922175156.130228-1-peterx@redhat.com>
 <6bbb8e29-9e21-dfbe-d23d-61de7e3cc6db@redhat.com>
 <YUt833H6eaYFyHXD@t490s>
 <CAHbLzkq7kTdEC6ADJRAxXBEOYhsjR8E9uiEFCYFmtA5cD5fMhA@mail.gmail.com>
 <YUuMXCFbj1mmOKec@t490s>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <YUuMXCFbj1mmOKec@t490s>
Precedence: bulk

On Wed, Sep 22, 2021 at 04:04:44PM -0400, Peter Xu wrote:
> On Wed, Sep 22, 2021 at 12:29:35PM -0700, Yang Shi wrote:
> > khugepqged does remove the pgtables. Please check out
> > retract_page_tables(). The pmd will be cleared and the ptes will be
> > freed otherwise the collapsed THP won't get PMD mapped by later
> > access.
> 
> Indeed.
> 
> I should probably still properly disable khugepaged for at least VM_SHARED &&
> VM_UFFD_WP, then I'd keep the anonymous && minor mode behavior untouched.
> 
> The other problem is even if current mm/vma doesn't have UFFD_WP registered,
> some other mm/vma could have UFFD_WP enabled there that mapped the same file.
> Checking that up within retract_page_tables() on all VMAs seems to be a bit too
> late.
> 
> Checking it early may not trivially work too - I can walk the vma interval tree
> at the entry of khugepaged_scan_file(), making sure no vma has UFFD_WP set.
> However I don't see how it'll stop some of the vma from having UFFD_WP
> registered later after that point but before retract_page_tables().
> 
> I'll need to think about it, but thanks for the input, Yang.  That's a very
> important point.

Perhaps I need something like this:

---8<---
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index 045cc579f724..c63e957336d1 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -1583,6 +1583,15 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff)
                pmd = mm_find_pmd(mm, addr);
                if (!pmd)
                        continue;
+               /*
+                * When a vma is registered with uffd-wp, we can't recycle the
+                * pmd pgtable because there can be pte markers installed.
+                * Skip it only, so the rest mm/vma can still have the same
+                * file mapped hugely, however it'll always mapped in small
+                * page size for uffd-wp registered ranges.
+                */
+               if (userfaultfd_wp(vma))
+                       continue;
                /*
                 * We need exclusive mmap_lock to retract page table.
                 *
---8<---

I won't post a v2 because then that patch will be shmem-only and uffd-wp-only.
I'll keep it with the upcoming series I'm going to post to support shmem.

Thanks,

-- 
Peter Xu