Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
From:   "Huang\, Ying" <ying.huang@intel.com>
To:     Dave Hansen <dave.hansen@intel.com>
Cc:     Jonathan Adams <jwadams@google.com>, Linux-MM <linux-mm@kvack.org>,
        LKML <linux-kernel@vger.kernel.org>,
        "Williams\, Dan J" <dan.j.williams@intel.com>,
        "Verma\, Vishal L" <vishal.l.verma@intel.com>,
        Wu Fengguang <fengguang.wu@intel.com>
Subject: Re: [RFC] Memory Tiering
References: <c3d6de4d-f7c3-b505-2e64-8ee5f70b2118@intel.com>
        <CA+VK+GMAqMVXKQqjGzSj9P+-TKr_Jn6qQ1cHSyxhDsoChorm_w@mail.gmail.com>
        <bab0848c-3229-bcb5-8921-d150939a7ce2@intel.com>
Date:   Fri, 25 Oct 2019 11:30:46 +0800
In-Reply-To: <bab0848c-3229-bcb5-8921-d150939a7ce2@intel.com> (Dave Hansen's
        message of "Thu, 24 Oct 2019 09:33:01 -0700")
Message-ID: <87o8y5h57d.fsf@yhuang-dev.intel.com>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=ascii
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Dave Hansen <dave.hansen@intel.com> writes:

> On 10/23/19 4:11 PM, Jonathan Adams wrote:
>> we would have a bidirectional attachment:
>> 
>> A is marked "move cold pages to" B
>> B is marked "move hot pages to" A
>> C is marked "move cold pages to" D
>> D is marked "move hot pages to" C
>> 
>> By using autonuma for moving PMEM pages back to DRAM, you avoid
>> needing the B->A  & D->C links, at the cost of migrating the pages
>> back synchronously at pagefault time (assuming my understanding of how
>> autonuma works is accurate).
>> 
>> Our approach still lets you have multiple levels of hierarchy for a
>> given socket (you could imaging an "E" node with the same relation to
>> "B" as "B" has to "A"), but doesn't make it easy to represent (say) an
>> "E" which was equally close to all sockets (which I could imagine for
>> something like remote memory on GenZ or what-have-you), since there
>> wouldn't be a single back link; there would need to be something like
>> your autonuma support to achieve that.
>> 
>> Does that make sense?
>
> Yes, it does.  We've actually tried a few other approaches separate from
> autonuma-based ones for promotion.  For some of those, we have a
> promotion path which is separate from the demotion path.
>
> That said, I took a quick look to see what the autonuma behavior was and
> couldn't find anything obvious.  Ying, when moving a slow page due to
> autonuma, do we move it close to the CPU that did the access, or do we
> promote it to the DRAM close to the slow memory where it is now?

Now in autonuma, the slow page will be moved to the CPU that did the
access.  So I think Jonathan's requirement has been covered already.

Best Regards,
Huang, Ying