Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
Message-ID: <c5d55131-76ff-7354-2954-7cfac365a9a5@linux.ibm.com>
Date:   Thu, 12 May 2022 12:42:02 +0530
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.8.0
Subject: Re: RFC: Memory Tiering Kernel Interfaces (v2)
Content-Language: en-US
To:     "ying.huang@intel.com" <ying.huang@intel.com>,
        Wei Xu <weixugc@google.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Greg Thelen <gthelen@google.com>,
        Yang Shi <shy828301@gmail.com>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Jagdish Gediya <jvgediya@linux.ibm.com>,
        Michal Hocko <mhocko@kernel.org>,
        Tim C Chen <tim.c.chen@intel.com>,
        Dave Hansen <dave.hansen@intel.com>,
        Alistair Popple <apopple@nvidia.com>,
        Baolin Wang <baolin.wang@linux.alibaba.com>,
        Feng Tang <feng.tang@intel.com>,
        Jonathan Cameron <Jonathan.Cameron@huawei.com>,
        Davidlohr Bueso <dave@stgolabs.net>,
        Dan Williams <dan.j.williams@intel.com>,
        David Rientjes <rientjes@google.com>,
        Linux MM <linux-mm@kvack.org>,
        Brice Goglin <brice.goglin@gmail.com>,
        Hesham Almatary <hesham.almatary@huawei.com>
References: <CAAPL-u-DGLcKRVDnChN9ZhxPkfxQvz9Sb93kVoX_4J2oiJSkUw@mail.gmail.com>
 <56b41ce6922ed5f640d9bd46a603fa27576532a9.camel@intel.com>
From:   Aneesh Kumar K V <aneesh.kumar@linux.ibm.com>
In-Reply-To: <56b41ce6922ed5f640d9bd46a603fa27576532a9.camel@intel.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk

On 5/12/22 12:33 PM, ying.huang@intel.com wrote:
> On Wed, 2022-05-11 at 23:22 -0700, Wei Xu wrote:
>> Sysfs Interfaces
>> ================
>>
>> * /sys/devices/system/memtier/memtierN/nodelist
>>
>>    where N = 0, 1, 2 (the kernel supports only 3 tiers for now).
>>
>>    Format: node_list
>>
>>    Read-only.  When read, list the memory nodes in the specified tier.
>>
>>    Tier 0 is the highest tier, while tier 2 is the lowest tier.
>>
>>    The absolute value of a tier id number has no specific meaning.
>>    What matters is the relative order of the tier id numbers.
>>
>>    When a memory tier has no nodes, the kernel can hide its memtier
>>    sysfs files.
>>
>> * /sys/devices/system/node/nodeN/memtier
>>
>>    where N = 0, 1, ...
>>
>>    Format: int or empty
>>
>>    When read, list the memory tier that the node belongs to.  Its value
>>    is empty for a CPU-only NUMA node.
>>
>>    When written, the kernel moves the node into the specified memory
>>    tier if the move is allowed.  The tier assignment of all other nodes
>>    are not affected.
>>
>>    Initially, we can make this interface read-only.
> 
> It seems that "/sys/devices/system/node/nodeN/memtier" has all
> information we needed.  Do we really need
> "/sys/devices/system/memtier/memtierN/nodelist"?
> 
> That can be gotten via a simple shell command line,
> 
> $ grep . /sys/devices/system/node/nodeN/memtier | sort -n -k 2 -t ':'
> 

It will be really useful to fetch the memory tier node list in an easy 
fashion rather than reading multiple sysfs directories. If we don't have 
other attributes for memorytier, we could keep
"/sys/devices/system/memtier/memtierN" a NUMA node list there by 
avoiding /sys/devices/system/memtier/memtierN/nodelist

-aneesh