Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 27 Jul 2023 09:15:37 -0400
From:   Peter Xu <peterx@redhat.com>
To:     David Hildenbrand <david@redhat.com>
Cc:     liubo <liubo254@huawei.com>, akpm@linux-foundation.org,
        linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org,
        hughd@google.com, willy@infradead.org
Subject: Re: [PATCH] smaps: Fix the abnormal memory statistics obtained
 through /proc/pid/smaps
Message-ID: <ZMJt+VWzIG4GAjeb@x1n>
References: <20230726073409.631838-1-liubo254@huawei.com>
 <CADFyXm5nkgZjVMj3iJhqQnyA1AOmqZ-AKdaWyUD=UvZsOEOcPg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <CADFyXm5nkgZjVMj3iJhqQnyA1AOmqZ-AKdaWyUD=UvZsOEOcPg@mail.gmail.com>
Precedence: bulk

On Thu, Jul 27, 2023 at 01:37:06PM +0200, David Hildenbrand wrote:
> On Wed, Jul 26, 2023 at 9:40 AM liubo <liubo254@huawei.com> wrote:
> >
> > In commit 474098edac26 ("mm/gup: replace FOLL_NUMA by
> > gup_can_follow_protnone()"), FOLL_NUMA was removed and replaced by
> > the gup_can_follow_protnone interface.
> >
> > However, for the case where the user-mode process uses transparent
> > huge pages, when analyzing the memory usage through
> > /proc/pid/smaps_rollup, the obtained memory usage is not consistent
> > with the RSS in /proc/pid/status.
> >
> > Related examples are as follows:
> > cat /proc/15427/status
> > VmRSS:  20973024 kB
> > RssAnon:        20971616 kB
> > RssFile:            1408 kB
> > RssShmem:              0 kB
> >
> > cat /proc/15427/smaps_rollup
> > 00400000-7ffcc372d000 ---p 00000000 00:00 0 [rollup]
> > Rss:            14419432 kB
> > Pss:            14418079 kB
> > Pss_Dirty:      14418016 kB
> > Pss_Anon:       14418016 kB
> > Pss_File:             63 kB
> > Pss_Shmem:             0 kB
> > Anonymous:      14418016 kB
> > LazyFree:              0 kB
> > AnonHugePages:  14417920 kB
> >
> > The root cause is that the traversal In the page table, the number of
> > pages obtained by smaps_pmd_entry does not include the pages
> > corresponding to PROTNONE,resulting in a different situation.
> >
> 
> Thanks for reporting and debugging!
> 
> > Therefore, when obtaining pages through the follow_trans_huge_pmd
> > interface, add the FOLL_FORCE flag to count the pages corresponding to
> > PROTNONE to solve the above problem.
> >
> 
> We really want to avoid the usage of FOLL_FORCE, and ideally limit it
> to ptrace only.

Fundamentally when removing FOLL_NUMA we did already assumed !FORCE is
FOLL_NUMA.  It means to me after the removal it's not possible to say in a
gup walker that "it's not FORCEd, but I don't want to trigger NUMA but just
get the page".

Is that what we want?  Shall we document that in FOLL_FORCE if we intended
to enforce numa balancing as long as !FORCE?

> 
> > Signed-off-by: liubo <liubo254@huawei.com>
> > Fixes: 474098edac26 ("mm/gup: replace FOLL_NUMA by gup_can_follow_protnone()")
> > ---
> >  fs/proc/task_mmu.c | 6 ++++--
> >  1 file changed, 4 insertions(+), 2 deletions(-)
> >
> > diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
> > index c1e6531cb02a..ed08f9b869e2 100644
> > --- a/fs/proc/task_mmu.c
> > +++ b/fs/proc/task_mmu.c
> > @@ -571,8 +571,10 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr,
> >         bool migration = false;
> >
> >         if (pmd_present(*pmd)) {
> > -               /* FOLL_DUMP will return -EFAULT on huge zero page */
> > -               page = follow_trans_huge_pmd(vma, addr, pmd, FOLL_DUMP);
> > +               /* FOLL_DUMP will return -EFAULT on huge zero page
> > +                * FOLL_FORCE follow a PROT_NONE mapped page
> > +                */
> > +               page = follow_trans_huge_pmd(vma, addr, pmd, FOLL_DUMP | FOLL_FORCE);
> >         } else if (unlikely(thp_migration_supported() && is_swap_pmd(*pmd))) {
> >                 swp_entry_t entry = pmd_to_swp_entry(*pmd);
> 
> Might do as an easy fix. But we really should get rid of that
> absolutely disgusting usage of follow_trans_huge_pmd().
> 
> We don't need 99% of what follow_trans_huge_pmd() does here.
> 
> Would the following also fix your issue?
> 
> diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
> index 507cd4e59d07..fc744964816e 100644
> --- a/fs/proc/task_mmu.c
> +++ b/fs/proc/task_mmu.c
> @@ -587,8 +587,7 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr,
>         bool migration = false;
> 
>         if (pmd_present(*pmd)) {
> -               /* FOLL_DUMP will return -EFAULT on huge zero page */
> -               page = follow_trans_huge_pmd(vma, addr, pmd, FOLL_DUMP);
> +               page = vm_normal_page_pmd(vma, addr, *pmd);
>         } else if (unlikely(thp_migration_supported() && is_swap_pmd(*pmd))) {
>                 swp_entry_t entry = pmd_to_swp_entry(*pmd);
> 
> It also skips the shared zeropage and pmd_devmap(),
> 
> Otherwise, a simple pmd_page(*pmd) + is_huge_zero_pmd(*pmd) check will do, but I
> suspect vm_normal_page_pmd() might be what we actually want to have here.
> 
> Because smaps_pte_entry() properly checks for vm_normal_page().

There're indeed some very trivial detail in vm_normal_page_pmd() that's
different, but maybe not so relevant.  E.g.,

	if (WARN_ON_ONCE(folio_ref_count(folio) <= 0))
		return -ENOMEM;

	if (unlikely(!(flags & FOLL_PCI_P2PDMA) && is_pci_p2pdma_page(page)))
		return -EREMOTEIO;

I'm not sure whether the p2pdma page would matter in any form here.  E.g.,
whether it can be mapped privately.

-- 
Peter Xu