Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37;
Received-SPF: Pass (protection.outlook.com: domain of micron.com designates
 137.201.242.130 as permitted sender) receiver=protection.outlook.com;
 client-ip=137.201.242.130; helo=mail.micron.com; pr=C
From:   Ravi Jonnalagadda <ravis.opensrc@micron.com>
To:     <mhocko@suse.com>
CC:     <Jonathan.Cameron@huawei.com>, <Ravis.OpenSrc@micron.com>,
        <aneesh.kumar@linux.ibm.com>, <dan.j.williams@intel.com>,
        <emirakhur@micron.com>, <fvdl@google.com>,
        <gregory.price@memverge.com>, <hannes@cmpxchg.org>,
        <haowang3@fb.com>, <hasanalmaruf@fb.com>,
        <hezhongkun.hzk@bytedance.com>, <john@jagalactic.com>,
        <linux-api@vger.kernel.org>, <linux-cxl@vger.kernel.org>,
        <linux-kernel@vger.kernel.org>, <linux-mm@kvack.org>,
        <sthanneeru.opensrc@micron.com>, <tj@kernel.org>,
        <vtavarespetr@micron.com>, <ying.huang@intel.com>
Subject: Re: [RFC PATCH 0/2] Node migration between memory tiers
Date:   Tue, 5 Dec 2023 14:49:58 +0530
Message-ID: <20231205091958.55-1-ravis.opensrc@micron.com>
In-Reply-To: <ZW7km-SED5oIGGnZ@tiehlicka>
References: <ZW7km-SED5oIGGnZ@tiehlicka>
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Dec 2023 09:20:07.8465
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 48b3770a-24d5-49f9-38d9-08dbf5735fbe
X-MS-Exchange-CrossTenant-Id: f38a5ecd-2813-4862-b11b-ac1d563c806f
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f38a5ecd-2813-4862-b11b-ac1d563c806f;Ip=[137.201.242.130];Helo=[mail.micron.com]
X-MS-Exchange-CrossTenant-AuthSource: DS2PEPF00003441.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR08MB8732
X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Tue, 05 Dec 2023 01:20:22 -0800 (PST)

>On Tue 05-12-23 14:12:17, Srinivasulu Thanneeru wrote:=0D
>> =0D
>> =0D
>> On 12/5/2023 2:05 PM, Michal Hocko wrote:=0D
>> > CAUTION: EXTERNAL EMAIL. Do not click links or open attachments unless=
 you recognize the sender and were expecting this message.=0D
>> > =0D
>> > =0D
>> > On Tue 05-12-23 01:26:07, Srinivasulu Thanneeru wrote:=0D
>> > > =0D
>> > > =0D
>> > > On 12/4/2023 9:13 PM, Michal Hocko wrote:=0D
>> > > > CAUTION: EXTERNAL EMAIL. Do not click links or open attachments un=
less you recognize the sender and were expecting this message.=0D
>> > > > =0D
>> > > > =0D
>> > > > On Fri 01-12-23 03:34:20, sthanneeru.opensrc@micron.com wrote:=0D
>> > > > > From: Srinivasulu Thanneeru <sthanneeru.opensrc@micron.com>=0D
>> > > > > =0D
>> > > > > The memory tiers feature allows nodes with similar memory types=
=0D
>> > > > > or performance characteristics to be grouped together in a=0D
>> > > > > memory tier. However, there is currently no provision for=0D
>> > > > > moving a node from one tier to another on demand.=0D
>> > > > =0D
>> > > > Could you expand on why this is really needed/necessary? What is t=
he=0D
>> > > > actual usecase?=0D
>> > > =0D
>> > > Hi Michal Hock,=0D
>> > > =0D
>> > > Following two use-cases we have observed.=0D
>> > > 1. It is not accurate to group similar memory types in the same tier=
,=0D
>> > >     because even similar memory types may have different speed grade=
s.=0D
>> > =0D
>> > Presumably they are grouped based on a HW configuration. Does that mea=
n=0D
>> > that the configuration is wrong? Are you trying to workaround that by=
=0D
>> > this interface?=0D
>> > =0D
>> > > 2. Some systems boots up with CXL devices and DRAM on the same memor=
y-tier,=0D
>> > > we need a way to move the CXL nodes to the correct tier from the use=
r space.=0D
>> > =0D
>> > Again, could you expand a bit more and explain why this cannot be=0D
>> > configured automatically?=0D
>> =0D
>> Yes, in both cases above, if hardware not automatically populated proper=
ly,=0D
>> in that case this interface would help to correct it from user space.=0D
>> =0D
>> We had observed case-2 in our setups.=0D
>=0D
>How hard it is to address this at the HW level?=0D
>=0D
>Btw. this is really important piece of context that should be part of=0D
>the changelog. Quite honestly introducing user interfaces solely to=0D
>workaround HW issues seems a rather weak justification. Are there any=0D
>usecases you can think of where this would be useful?=0D
>=0D
>-- =0D
>Michal Hocko=0D
>SUSE Labs=0D
=0D
Hello Michal Hocko,=0D
=0D
    It will be useful if we want interleave weights to be applied on memory=
 tiers=0D
instead of nodes.=0D
    Also, for near memory processing use cases where some accelerator would=
 like=0D
to have hot pages migrated to a different node with HBM, pmem or CXL instea=
d of=0D
CPU attached memory for performing it's operations quicker.=0D
=0D
There was a prior discussion on this functionality in a previous thread, wh=
ere=0D
Huang Ying thought this might be a useful feature to overcome limitations o=
f=0D
systems where nodes with different bandwidth characteristics are grouped in=
 =0D
a single tier.=0D
=0D
https://lore.kernel.org/lkml/87a5rw1wu8.fsf@yhuang6-desk2.ccr.corp.intel.co=
m/=0D
=0D
--=0D
Best Regards,=0D
Ravi Jonnalagadda=