Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
Message-ID: <bf23f05830db51bab3b06bac6e54d4743d37e955.camel@intel.com>
Subject: Re: [PATCH 1/3] memory tiering: hot page selection with hint page
 fault latency
From:   "ying.huang@intel.com" <ying.huang@intel.com>
To:     Jagdish Gediya <jvgediya@linux.ibm.com>
Cc:     Peter Zijlstra <peterz@infradead.org>,
        Mel Gorman <mgorman@techsingularity.net>,
        Andrew Morton <akpm@linux-foundation.org>, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org, Michal Hocko <mhocko@suse.com>,
        Rik van Riel <riel@surriel.com>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        Yang Shi <shy828301@gmail.com>, Zi Yan <ziy@nvidia.com>,
        Wei Xu <weixugc@google.com>, osalvador <osalvador@suse.de>,
        Shakeel Butt <shakeelb@google.com>,
        Zhong Jiang <zhongjiang-ali@linux.alibaba.com>
Date:   Fri, 15 Apr 2022 10:42:31 +0800
In-Reply-To: <YlggP5Tub19gjF4g@li-6e1fa1cc-351b-11b2-a85c-b897023bb5f3.ibm.com>
References: <20220408071222.219689-1-ying.huang@intel.com>
         <20220408071222.219689-2-ying.huang@intel.com>
         <YlggP5Tub19gjF4g@li-6e1fa1cc-351b-11b2-a85c-b897023bb5f3.ibm.com>
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.38.3-1 
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Precedence: bulk

Hi, Jagdish,

On Thu, 2022-04-14 at 18:53 +0530, Jagdish Gediya wrote:
> On Fri, Apr 08, 2022 at 03:12:20PM +0800, Huang Ying wrote:
[snip]
> > +
> > +static int numa_hint_fault_latency(struct page *page)
> > +{
> > +	int last_time, time;
> > +
> > +	time = jiffies_to_msecs(jiffies);
> > +	last_time = xchg_page_access_time(page, time);
> > +
> > +	return (time - last_time) & PAGE_ACCESS_TIME_MASK;
> 
> This code can possibly consider cold page as hot,
> 
> Assume,
> 
> LAST_CPUPID_SHIFT = 12
> PAGE_ACCESS_TIME_BUCKETS = 0
> sysctl_numa_balancing_hot_threshold = 1000
> 
> Assume while changing pte,
> jiffies_to_msecs(jiffies) = 0xAABB0100
> 
> So value saved in page->flags will be lowest 12 bits of 0xAABB0100
> which is 0x100.
> 
> Assume when numa_hint_fault_latency() gets called,
> time = jiffies_to_msecs(jiffies) = 0xAACC0100
> 
> So, time = 0xAACC0100, and last_time = 0x100,
> time - last_time = 0xAACC0100 - 0x100 = 0xAACC0000
> 0xAACC0000 & PAGE_ACCESS_TIME_MASK = 0xAACC0000 & ((1 << 12) - 1) = 0
> 
> so the return value of this function is 0, the code will consider it as
> hot page but it is cold page because actual difference is
> 0xAACC0100 - 0xAABB0100 = 110000 ms
> 

Yes.  This is possible.

> There may be more such scenarios. What do you think?

The algorithm just works statistically correct.  That is, for really hot
pages, their hint page fault latency will be short and we can promote it
when they are accessed.  For cold pages, it's still possible for them to
be identified as hot pages.  But the possibility is much lower than that
of the hot pages.

We can try to improve further here.  But as the first step, I want to
keep the algorithm as simple as possible.  Then we can try improve it
step by step and show benefit in each step to justify the further
optimization.

> > +}
> > +

Best Regards,
Huang, Ying