Sender: Minchan Kim <minchan.kim@gmail.com>
Date:   Thu, 26 Jan 2023 09:10:46 -0800
From:   Minchan Kim <minchan@kernel.org>
To:     Michal Hocko <mhocko@suse.com>
Cc:     Andrew Morton <akpm@linux-foundation.org>,
        Suren Baghdasaryan <surenb@google.com>,
        Matthew Wilcox <willy@infradead.org>,
        linux-mm <linux-mm@kvack.org>,
        LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v2] mm/madvise: add vmstat statistics for
 madvise_[cold|pageout]
Message-ID: <Y9K0Fn9MFPAgJjlM@google.com>
References: <20230125005457.4139289-1-minchan@kernel.org>
 <Y9DigKf0w712t0OO@dhcp22.suse.cz>
 <Y9FacrcUIaLZq4DL@google.com>
 <Y9FhtBbnlNxAZAS4@dhcp22.suse.cz>
 <Y9Fv9YnNn7bHvLkN@google.com>
 <Y9GhNxqfjTEAFr5V@dhcp22.suse.cz>
 <Y9Grb2rggptkCu+n@google.com>
 <Y9I+3ZkWRdXdPBxg@dhcp22.suse.cz>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Y9I+3ZkWRdXdPBxg@dhcp22.suse.cz>
Precedence: bulk

On Thu, Jan 26, 2023 at 09:50:37AM +0100, Michal Hocko wrote:
> On Wed 25-01-23 14:21:35, Minchan Kim wrote:
> > On Wed, Jan 25, 2023 at 10:37:59PM +0100, Michal Hocko wrote:
> > > On Wed 25-01-23 10:07:49, Minchan Kim wrote:
> > > > On Wed, Jan 25, 2023 at 06:07:00PM +0100, Michal Hocko wrote:
> > > > > On Wed 25-01-23 08:36:02, Minchan Kim wrote:
> > > > > > On Wed, Jan 25, 2023 at 09:04:16AM +0100, Michal Hocko wrote:
> > > > > > > On Tue 24-01-23 16:54:57, Minchan Kim wrote:
> > > > > > > > madvise LRU manipulation APIs need to scan address ranges to find
> > > > > > > > present pages at page table and provides advice hints for them.
> > > > > > > > 
> > > > > > > > Likewise pg[scan/steal] count on vmstat, madvise_pg[scanned/hinted]
> > > > > > > > shows the proactive reclaim efficiency so this patch adds those
> > > > > > > > two statistics in vmstat.
> > > > > > > > 
> > > > > > > > 	madvise_pgscanned, madvise_pghinted
> > > > > > > > 
> > > > > > > > Since proactive reclaim using process_madvise(2) as userland
> > > > > > > > memory policy is popular(e.g,. Android ActivityManagerService),
> > > > > > > > those stats are helpful to know how efficiently the policy works
> > > > > > > > well.
> > > > > > > 
> > > > > > > The usecase description is still too vague. What are those values useful
> > > > > > > for? Is there anything actionable based on those numbers? How do you
> > > > > > > deal with multiple parties using madvise resp. process_madvise so that
> > > > > > > their stats are combined?
> > > > > > 
> > > > > > The metric helps monitoing system MM health under fleet and experimental
> > > > > > tuning with diffrent policies from the centralized userland memory daemon.
> > > > > 
> > > > > That is just too vague for me to imagine anything more specific then, we
> > > > > have numbers and we can show them in a report. What does it actually
> > > > > mean that madvise_pgscanned is high. Or that pghinted / pgscanned is
> > > > > low (that you tend to manually reclaim sparse mappings)?
> > > > 
> > > > If that's low, it means the userspace daemon's current tune/policy are
> > > > inefficient or too aggressive since it is working on address spacess
> > > > of processes which don't have enough memory the hint can work(e.g.,
> > > > shared addresses, cold address ranges or some special address ranges like
> > > > VM_PFNMAP) so sometime, we can detect regression to find culprit or
> > > > have a chance to look into better ideas to improve.
> > > 
> > > Are you sure this is really meaningful metric? Just consider a large and
> > > sparsely populated mapping. This can be a perfect candidate for user
> > > space reclaim target (e.g. consider a mapping covering a large matrix
> > > or other similar data structure). pghinted/pgscanned would be really
> > > small while the reclaim efficiency could be quite high in that case,
> > > wouldn't it?
> > 
> > Why do you think it's efficient? It need to spend quite CPU cycle to
> > scan a few of pages to evict. I don't see it's efficient if it happens
> > quite a lot.
> 
> Because it doesn't really matter how many page tables you have to scan
> but how easily you can reclaim the memory behind that. Because it is the

I really don't follow your claim here. Efficiency is input vs output.
For the those advices, input is number of scanned ptes vs. number of
hinted pages.
If you keep need to scan the sparsed huge address range to reclaim just
a few of pages, that's really inefficient.
If you keep hinting to the non-populated-page, already swapped-out and
hint-cannot-work address ranges, that's really inefficient.

What do you see the problem here? What exactly do you mean "how easily"
from your context?

> memory that matters. Just consider THP vs. 4k backed address ranges. You
> are going to scan much more for latter by design. That doesn't really
> mean that this is a worse candidate for reclaim and you should be only
> focusing on THP backed mappings. See?

I don't understand your point here. The stat doesn't aim to make such
decision. If THP page shows the good yield from the efficienty above,
that's good. If 4K page shows the bad yield, should we focus on THP
pages? How could you conclude such decision from the stat?

The stat just sees the current health of the system and find something
regressed/improved compared to old to intitiate further investigation.

> 
> I suspect you try to mimic pgscan/pgsteal effectivness metric on the
> address space but that is a fundamentally different thing.

I don't see anything different, fundamentally.