Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 5 Aug 2022 20:24:25 +0100
From:   Matthew Wilcox <willy@infradead.org>
To:     Rik van Riel <riel@fb.com>
Cc:     "Alex Zhu (Kernel)" <alexlzhu@fb.com>,
        Kernel Team <Kernel-team@fb.com>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "akpm@linux-foundation.org" <akpm@linux-foundation.org>
Subject: Re: [PATCH v3] mm: add thp_utilization metrics to
 /proc/thp_utilization
Message-ID: <Yu1uabedm+NYnnAj@casper.infradead.org>
References: <20220805184016.2926168-1-alexlzhu@fb.com>
 <Yu1mcD6Jp4fCVEMi@casper.infradead.org>
 <0b16dbac6444bfcdfbeb4df4280354839bfe1a8f.camel@fb.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <0b16dbac6444bfcdfbeb4df4280354839bfe1a8f.camel@fb.com>
Precedence: bulk

On Fri, Aug 05, 2022 at 07:04:30PM +0000, Rik van Riel wrote:
> On Fri, 2022-08-05 at 19:50 +0100, Matthew Wilcox wrote:
> > > 
> > > This change introduces a tool that scans through all of physical
> > > memory for anonymous THPs and groups them into buckets based
> > > on utilization. It also includes an interface under
> > > /proc/thp_utilization.
> > 
> > OK, so I understand why we want to do the scanning, but why do we
> > want to
> > report it to userspace at all?? And if we do, why do we want to do it
> > in
> > this format?? AFAIK, there's nothing userspace can do with the
> > information
> > "93% of your THPs are underutilised".? If there was something
> > userspace
> > could do about it, wouldn't it need to know which ones?
> > 
> > Isn't the real solution here entirely in-kernel?? This scanning
> > thread
> > you've created should be the one splitting the THPs.? And anyway,
> > isn't
> > this exactly the kind of thing that DAMON was invented for?
> 
> Alex does have an (in kernel) shrinker that can reclaim
> underutilized THPs in order to free memory.

Ah!  So when that exists, this interface tells us "how well" we're doing.

> This is a regular shrinker called from kswapd. I am not
> sure a shrinker going through the DAMON infrastructure
> would be any smaller, faster, or better. OTOH, DAMON
> does allow for more flexible policy...
> 
> Getting some info on the effectiveness of the shrinker
> seems useful, though. Could debugfs be a better place?
> Should this be resubmitted together with the shrinker
> code that makes this more useful?

Yeah, debugfs seems like a better place.  And I'd love to see the shrinker
code.  Before you mentioned that I was having all kinds of peculiar
feelings about this code.  For example, suppose you have incredibly hot
256kB of data, but the other 1792kB of data are never touched ... that
could cause us to do entirely the wrong thing and break up this THP.
Having it as a shrinker makes sense because the hot 256kB will keep the
THP from reaching the end of the list and being reclaimed.