Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <64e3e50508b0bf27ed5d6957161e2b3631c1164b.camel@linux.intel.com>
Subject: Re: [RFC PATCH 0/3] Cgroup accounting of memory tier usage
From:   Tim Chen <tim.c.chen@linux.intel.com>
To:     Ying Huang <ying.huang@intel.com>, linux-mm@kvack.org,
        akpm@linux-foundation.org
Cc:     Wei Xu <weixugc@google.com>, Greg Thelen <gthelen@google.com>,
        Yang Shi <shy828301@gmail.com>,
        Davidlohr Bueso <dave@stgolabs.net>,
        Brice Goglin <brice.goglin@gmail.com>,
        Michal Hocko <mhocko@kernel.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Hesham Almatary <hesham.almatary@huawei.com>,
        Dave Hansen <dave.hansen@intel.com>,
        Jonathan Cameron <Jonathan.Cameron@huawei.com>,
        Alistair Popple <apopple@nvidia.com>,
        Dan Williams <dan.j.williams@intel.com>,
        Feng Tang <feng.tang@intel.com>,
        Jagdish Gediya <jvgediya@linux.ibm.com>,
        Baolin Wang <baolin.wang@linux.alibaba.com>,
        David Rientjes <rientjes@google.com>,
        "Aneesh Kumar K . V" <aneesh.kumar@linux.ibm.com>,
        Shakeel Butt <shakeelb@google.com>
Date:   Wed, 15 Jun 2022 10:47:11 -0700
In-Reply-To: <68b6a7e92d48a3285a5707378459bb9ae805f333.camel@intel.com>
References: <cover.1655242024.git.tim.c.chen@linux.intel.com>
         <68b6a7e92d48a3285a5707378459bb9ae805f333.camel@intel.com>
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.34.4 (3.34.4-1.fc31) 
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Precedence: bulk

On Wed, 2022-06-15 at 12:58 +0800, Ying Huang wrote:
> On Tue, 2022-06-14 at 15:25 -0700, Tim Chen wrote:
> > For controlling usage of a top tiered memory by a cgroup, accounting
> > of top tier memory usage is needed.  This patch set implements the
> > following:
> > 
> > Patch 1 introduces interface and simple implementation to retrieve
> > 	cgroup tiered memory usage
> > Patch 2 introduces more efficient accounting with top tier memory page counter 
> > Patch 3 provides a sysfs interface to repot the the top tiered memory
> > 	usage.
> > 
> > The patchset works with Aneesh's v6 memory-tiering implementation [1].
> > It is a preparatory patch set before introducing features to
> > control top tiered memory in cgroups.
> > 
> > I'll like to first get feedback to see if 
> > (1) Controllng the topmost tiered memory is enough 
> > or
> > (2) Multiple tiers at the top levels need to be grouped into "toptier"
> > or
> 
> If we combine top-N tiers, I think the better name could be "fast-tier",
> in contrast to "slow-tier".
> 

I can see use cases for grouping tiers. For example, it makese sense for HBM and 
DRAM tiers be grouped together into a "fast-tier-group".

To make things simple, we can define any tiers above or equal
to the rank of DRAM will belong to this fast-tier-group.

An implication for page promotion/demotion is it needs
to take tier grouping into consideration.  You want to demote
pages away from current tier-group.  For example,
you want to demote HBM (fast-tier-group) into PMEM (slow-tier-group)
instead of into DRAM (fast-tier-group).  

The question is whether fast/slow tier groups are sufficient.
Or you need fast/slow/slower groups?

> > (3) There are use cases not covered by (1) and (2). 
> 
> Is it necessary to control memory usage of each tier (except the
> lowest/slowest)?  I am not the right person to answer the question, but
> I want to ask it.
> 

I have the same question.

Tim