Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18;
Message-ID: <f9a26536-05f6-5d12-5c61-cdd35ab33a40@linux.ibm.com>
Date:   Mon, 6 Jun 2022 09:22:07 +0530
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.10.0
Subject: Re: [PATCH v5 9/9] mm/demotion: Update node_is_toptier to work with
 memory tiers
Content-Language: en-US
To:     Ying Huang <ying.huang@intel.com>, linux-mm@kvack.org,
        akpm@linux-foundation.org
Cc:     Wei Xu <weixugc@google.com>, Greg Thelen <gthelen@google.com>,
        Yang Shi <shy828301@gmail.com>,
        Davidlohr Bueso <dave@stgolabs.net>,
        Tim C Chen <tim.c.chen@intel.com>,
        Brice Goglin <brice.goglin@gmail.com>,
        Michal Hocko <mhocko@kernel.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Hesham Almatary <hesham.almatary@huawei.com>,
        Dave Hansen <dave.hansen@intel.com>,
        Jonathan Cameron <Jonathan.Cameron@huawei.com>,
        Alistair Popple <apopple@nvidia.com>,
        Dan Williams <dan.j.williams@intel.com>,
        Feng Tang <feng.tang@intel.com>,
        Jagdish Gediya <jvgediya@linux.ibm.com>,
        Baolin Wang <baolin.wang@linux.alibaba.com>,
        David Rientjes <rientjes@google.com>
References: <20220603134237.131362-1-aneesh.kumar@linux.ibm.com>
 <20220603134237.131362-10-aneesh.kumar@linux.ibm.com>
 <6e94b7e2a6192e4cacba1db3676b5b5cf9b98eac.camel@intel.com>
From:   Aneesh Kumar K V <aneesh.kumar@linux.ibm.com>
In-Reply-To: <6e94b7e2a6192e4cacba1db3676b5b5cf9b98eac.camel@intel.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 6/6/22 8:41 AM, Ying Huang wrote:
> On Fri, 2022-06-03 at 19:12 +0530, Aneesh Kumar K.V wrote:
>> With memory tiers support we can have memory on NUMA nodes
>> in the top tier from which we want to avoid promotion tracking NUMA
>> faults. Update node_is_toptier to work with memory tiers. To
>> avoid taking locks, a nodemask is maintained for all demotion
>> targets. All NUMA nodes are by default top tier nodes and as
>> we add new lower memory tiers NUMA nodes get added to the
>> demotion targets thereby moving them out of the top tier.
> 
> Check the usage of node_is_toptier(),
> 
> - migrate_misplaced_page()
>    node_is_toptier() is used to check whether migration is a promotion.
> We can avoid to use it.  Just compare the rank of the nodes.
> 
> - change_pte_range() and change_huge_pmd()
>    node_is_toptier() is used to avoid scanning fast memory (DRAM) pages
> for promotion.  So I think we should change the name to node_is_fast()
> as follows,
> 
> static inline bool node_is_fast(int node)
> {
> 	return NODE_DATA(node)->mt_rank >= MEMORY_RANK_DRAM;
> }
> 

But that gives special meaning to MEMORY_RANK_DRAM. As detailed in other 
patches, absolute value of rank doesn't carry any meaning. It is only
the relative value w.r.t other memory tiers that decide whether it is 
fast or not. Agreed by default memory tiers get built with 
MEMORY_RANK_DRAM. But userspace can change the rank value of 'memtier1' 
Hence to determine a node is consisting of fast memory is essentially 
figuring out whether node is the top most tier in memory hierarchy and 
not just the memory tier rank value is >= MEMORY_RANK_DRAM?

-aneesh