From:   "Huang, Ying" <ying.huang@intel.com>
To:     Baolin Wang <baolin.wang@linux.alibaba.com>, <sj@kernel.org>
Cc:     <akpm@linux-foundation.org>, <dave.hansen@linux.intel.com>,
        <ziy@nvidia.com>, <shy828301@gmail.com>,
        <zhongjiang-ali@linux.alibaba.com>, <xlpang@linux.alibaba.com>,
        <linux-mm@kvack.org>, <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v2 0/2] Add a new scheme to support demotion on tiered
 memory system
References: <cover.1640171137.git.baolin.wang@linux.alibaba.com>
        <87a6gsceo6.fsf@yhuang6-desk2.ccr.corp.intel.com>
        <90c5f31f-9e0a-df6d-8639-8a46ee02f9fa@linux.alibaba.com>
Date:   Thu, 23 Dec 2021 11:22:35 +0800
In-Reply-To: <90c5f31f-9e0a-df6d-8639-8a46ee02f9fa@linux.alibaba.com> (Baolin
        Wang's message of "Thu, 23 Dec 2021 09:21:45 +0800")
Message-ID: <875yrgc8ec.fsf@yhuang6-desk2.ccr.corp.intel.com>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=ascii
Precedence: bulk

Baolin Wang <baolin.wang@linux.alibaba.com> writes:

> On 12/23/2021 9:07 AM, Huang, Ying wrote:
>> Baolin Wang <baolin.wang@linux.alibaba.com> writes:
>> 
>>> Hi,
>>>
>>> Now on tiered memory system with different memory types, the reclaim path in
>>> shrink_page_list() already support demoting pages to slow memory node instead
>>> of discarding the pages. However, at that time the fast memory node memory
>>> wartermark is already tense, which will increase the memory allocation latency
>>> during page demotion. So a new method from user space demoting cold pages
>>> proactively will be more helpful.
>>>
>>> We can rely on the DAMON in user space to help to monitor the cold memory on
>>> fast memory node, and demote the cold pages to slow memory node proactively to
>>> keep the fast memory node in a healthy state.
>>>
>>> This patch set introduces a new scheme named DAMOS_DEMOTE to support this feature,
>>> and works well from my testing. Any comments are welcome. Thanks.
>> As a performance optimization patch, it's better to provide some
>> test
>> results.
>
> Actually this is a functional patch, which adds a new scheme for
> DAMON. And I think it is too early to measure the performance for the
> real workload, and more work need to do for DAMON used on tiered
> memory system (like supporting promotion scheme later).

I don't think you provide any new functionality except the performance
influence.

And I think proactive demotion itself can show some performance benefit
already.  Just like we can find the performance benefit in the proactive
reclaim patchset as below.

https://lore.kernel.org/lkml/20211019150731.16699-1-sj@kernel.org/

>> Another question is why we shouldn't do this in user space?  With DAMON,
>> it's possible to export cold memory regions information to the user
>> space, then we can use move_pages() to migrate them from DRAM to PMEM.
>> What's the problem of that?
>
> IMO this is the purpose of introducing scheme for DAMON, and you can
> check more in the Documentation/admin-guide/mm/damon/usage.rst.
>
> "
> Schemes
> -------
>
> For usual DAMON-based data access aware memory management
> optimizations, users
> would simply want the system to apply a memory management action to a memory
> region of a specific access pattern.  DAMON receives such formalized
> operation
> schemes from the user and applies those to the target processes.
> "

For proactive reclaim, we haven't a user space ABI to reclaim a page of
a process from memory to disk.  So it appears necessary to add a kernel
module to do that.

But for proactive demotion, we already have a user space ABI
(move_pages()) to demote a page of a process from DRAM to PMEM.  What
prevents you to do all these in the user space?

And, I found there are MADV_XXX schemes too.  Where the user space ABIs
are available already.  TBH, I don't know why we need these given there
are already user space ABIs.  Maybe this is a question for SeongJae too.

Best Regards,
Huang, Ying