Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Thu, 23 Sep 2021 17:57:11 -0700 (PDT)
From:   Hugh Dickins <hughd@google.com>
To:     Zi Yan <ziy@nvidia.com>
cc:     Hugh Dickins <hughd@google.com>, Yang Shi <shy828301@gmail.com>,
        "Kirill A. Shutemov" <kirill@shutemov.name>,
        Matthew Wilcox <willy@infradead.org>,
        Kent Overstreet <kent.overstreet@gmail.com>,
        Linux FS-devel Mailing List <linux-fsdevel@vger.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Linux MM <linux-mm@kvack.org>,
        Johannes Weiner <hannes@cmpxchg.org>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        "Darrick J. Wong" <djwong@kernel.org>,
        Christoph Hellwig <hch@infradead.org>,
        David Howells <dhowells@redhat.com>,
        Mike Kravetz <mike.kravetz@oracle.com>
Subject: Re: Mapcount of subpages
In-Reply-To: <24B432CB-5CBB-4309-A9D0-6E1C4395A013@nvidia.com>
Message-ID: <63e1cfcc-b7dd-ca55-39b2-7a9d2f6ff7eb@google.com>
References: <YUvWm6G16+ib+Wnb@moria.home.lan> <YUvzINep9m7G0ust@casper.infradead.org> <YUwNZFPGDj4Pkspx@moria.home.lan> <YUxnnq7uFBAtJ3rT@casper.infradead.org> <20210923124502.nxfdaoiov4sysed4@box.shutemov.name> <72cc2691-5ebe-8b56-1fe8-eeb4eb4a4c74@google.com>
 <CAHbLzkrELUKR2saOkA9_EeAyZwdboSq0HN6rhmCg2qxwSjdzbg@mail.gmail.com> <2A311B26-8B33-458E-B2C1-8BA2CF3484AA@nvidia.com> <77b59314-5593-1a2e-293c-b66e8235ad@google.com> <24B432CB-5CBB-4309-A9D0-6E1C4395A013@nvidia.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Precedence: bulk

On Thu, 23 Sep 2021, Zi Yan wrote:
> On 23 Sep 2021, at 19:48, Hugh Dickins wrote:
> > On Thu, 23 Sep 2021, Zi Yan wrote:
> >> On 23 Sep 2021, at 17:54, Yang Shi wrote:
> >>> On Thu, Sep 23, 2021 at 2:10 PM Hugh Dickins <hughd@google.com> wrote:
> >>>>
> >>>> NR_FILE_MAPPED being used for /proc/meminfo's "Mapped:" and a couple
> >>>> of other such stats files, and for a reclaim heuristic in mm/vmscan.c.
> >>>>
> >>>> Allow ourselves more slack in NR_FILE_MAPPED accounting (either count
> >>>> each pte as if it mapped the whole THP, or don't count a THP's ptes
> >>>> at all - you opted for the latter in the "Mlocked:" accounting),
> >>>> and I suspect subpage _mapcount could be abandoned.
> >>>
> >>> AFAIK, partial THP unmap may need the _mapcount information of every
> >>> subpage otherwise the deferred split can't know what subpages could be
> >>> freed.
> >
> > I believe Yang Shi is right insofar as the decision on whether it's worth
> > queuing for deferred split is being done based on those subpage _mapcounts.
> > That is a use I had not considered, and I've given no thought to how
> > important or not it is.
> >
> >>
> >> Could we just scan page tables of a THP during deferred split process
> >> instead? Deferred split is a slow path already, so maybe it can afford
> >> the extra work.
> >
> > But unless I misunderstand, actually carrying out the deferred split
> > already unmaps, uses migration entries, and remaps the remaining ptes:
> > needing no help from subpage _mapcounts to do those, and free the rest.
> 
> You are right. unmap_page() during THP split is scanning the page tables
> already.
> 
> For deciding whether to queue a THP for deferred split, we probably can
> keep PageDoubleMap bit to indicate if any subpage is PTE mapped.

Maybe, maybe not.

> 
> But without subpage _mapcount, detecting extra pins to a THP before split
> might be not as easy as with it. This means every THP split will need to
> perform unmap_page(), then check the remaining page_count to see if
> THP split is possible. That would also introduce extra system-wide overheads
> from unmapping pages. Am I missing anything?

I did not explain clearly enough: a subpage's ptes must still be counted
in total_mapcount(); but I'm suggesting that perhaps they can be counted
all together (either in the head page's _mapcount, or in a separate field
if that works better), instead of being distributed amongst the separate
subpages' _mapcounts.

And this would lower the system-wide overheads inside total_mapcount()
and page_mapped() (and maybe others).

Hugh