Received-SPF: pass (google.com: domain of linux-kernel+bounces-122124-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Precedence: bulk
MIME-Version: 1.0
References: <20240327213108.2384666-1-yuanchu@google.com> <ZgSTNCP5f+T5VtBI@memverge.com>
In-Reply-To: <ZgSTNCP5f+T5VtBI@memverge.com>
From: Yuanchu Xie <yuanchu@google.com>
Date: Wed, 27 Mar 2024 15:53:39 -0700
Message-ID: <CAJj2-QEg3+Ztg3rK6FpVVCxSG4DaDPWsO_bha5v5GrJazc5DVQ@mail.gmail.com>
Subject: Re: [RFC PATCH v3 0/8] mm: workingset reporting
To: Gregory Price <gregory.price@memverge.com>
Cc: David Hildenbrand <david@redhat.com>, "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>, 
	Khalid Aziz <khalid.aziz@oracle.com>, Henry Huang <henry.hj@antgroup.com>, 
	Yu Zhao <yuzhao@google.com>, Dan Williams <dan.j.williams@intel.com>, 
	Huang Ying <ying.huang@intel.com>, Wei Xu <weixugc@google.com>, 
	David Rientjes <rientjes@google.com>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, 
	"Rafael J. Wysocki" <rafael@kernel.org>, Andrew Morton <akpm@linux-foundation.org>, 
	Johannes Weiner <hannes@cmpxchg.org>, Michal Hocko <mhocko@kernel.org>, 
	Roman Gushchin <roman.gushchin@linux.dev>, Muchun Song <muchun.song@linux.dev>, 
	Shuah Khan <shuah@kernel.org>, Yosry Ahmed <yosryahmed@google.com>, 
	Matthew Wilcox <willy@infradead.org>, Sudarshan Rajagopalan <quic_sudaraja@quicinc.com>, 
	Kairui Song <kasong@tencent.com>, "Michael S. Tsirkin" <mst@redhat.com>, 
	Vasily Averin <vasily.averin@linux.dev>, Nhat Pham <nphamcs@gmail.com>, 
	Miaohe Lin <linmiaohe@huawei.com>, Qi Zheng <zhengqi.arch@bytedance.com>, 
	Abel Wu <wuyun.abel@bytedance.com>, "Vishal Moola (Oracle)" <vishal.moola@gmail.com>, 
	Kefeng Wang <wangkefeng.wang@huawei.com>, linux-kernel@vger.kernel.org, 
	linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kselftest@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, Mar 27, 2024 at 2:44=E2=80=AFPM Gregory Price
<gregory.price@memverge.com> wrote:
>
> On Wed, Mar 27, 2024 at 02:30:59PM -0700, Yuanchu Xie wrote:
> > I realize this does not generalize well to hotness information, but I
> > lack the intuition for an abstraction that presents hotness in a useful
> > way. Based on a recent proposal for move_phys_pages[2], it seems like
> > userspace tiering software would like to move specific physical pages,
> > instead of informing the kernel "move x number of hot pages to y
> > device". Please advise.
> >
> > [2]
> > https://lore.kernel.org/lkml/20240319172609.332900-1-gregory.price@memv=
erge.com/
> >
>
> Please note that this proposed interface (move_phys_pages) is very
> unlikely to be received upstream due to side channel concerns. Instead,
> it's more likely that the tiering component will expose a "promote X
> pages from tier A to tier B", and the kernel component would then
> use/consume hotness information to determine which pages to promote.

I see that mm/memory-tiers.c only has support for demotion. What kind
of hotness information do devices typically provide? The OCP proposal
is not very specific about this.
A list of hot pages with configurable threshold?
Access frequency for all pages at configured granularity?
Is there a way to tell which NUMA node is accessing them, for page promotio=
n?
>
> (Just as one example, there are many more realistic designs)
>
> So if there is a way to expose workingset data to the mm/memory_tiers.c
> component instead of via sysfs/cgroup - that is preferable.

Appreciate the feedback. The data in its current form might be useful
to inform demotion decisions, but for promotion, are you aware of any
recent developments? I would like to encode hotness as workingset data
as well.
>
> The 'move_phys_pages' interface is more of an experimental interface to
> test the effectiveness of this approach without having to plumb out the
> entire system.  Definitely anything userland interface should not be
> designed to generate physical address information for consumption unless
> it is hard-locked behind admin caps.
>
> Regards,
> Gregory