Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32;
Received-SPF: Pass (protection.outlook.com: domain of micron.com designates
 137.201.242.130 as permitted sender) receiver=protection.outlook.com;
 client-ip=137.201.242.130; helo=mail.micron.com; pr=C
Message-ID: <8eae7d95-fc98-4ac7-8f83-d0caee00bc87@micron.com>
Date:   Tue, 5 Dec 2023 14:32:20 +0530
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [EXT] Re: [RFC PATCH 0/2] Node migration between memory tiers
Content-Language: en-US
To:     Michal Hocko <mhocko@suse.com>
CC:     <aneesh.kumar@linux.ibm.com>, <linux-cxl@vger.kernel.org>,
        <linux-mm@kvack.org>, <dan.j.williams@intel.com>,
        <hannes@cmpxchg.org>, <hasanalmaruf@fb.com>, <haowang3@fb.com>,
        <ying.huang@intel.com>, <gregory.price@memverge.com>,
        <tj@kernel.org>, <hezhongkun.hzk@bytedance.com>, <fvdl@google.com>,
        <john@jagalactic.com>, <emirakhur@micron.com>,
        <vtavarespetr@micron.com>, <Ravis.OpenSrc@micron.com>,
        <Jonathan.Cameron@huawei.com>, <linux-kernel@vger.kernel.org>,
        <linux-api@vger.kernel.org>
References: <20231130220422.2033-1-sthanneeru.opensrc@micron.com>
 <ZW3zl2Fke5FtQCv3@tiehlicka>
 <1db561a9-6984-418d-9305-a2a5ece93696@micron.com>
 <ZW7g4ExYF79gMEBU@tiehlicka>
 <2552828e-6865-4fa8-a9c4-8ed76dd85257@micron.com>
 <ZW7km-SED5oIGGnZ@tiehlicka>
From:   Srinivasulu Thanneeru <sthanneeru.opensrc@micron.com>
In-Reply-To: <ZW7km-SED5oIGGnZ@tiehlicka>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Dec 2023 09:02:32.4101
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 1e08090c-9afb-4ec5-04cf-08dbf570eaa5
X-MS-Exchange-CrossTenant-Id: f38a5ecd-2813-4862-b11b-ac1d563c806f
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f38a5ecd-2813-4862-b11b-ac1d563c806f;Ip=[137.201.242.130];Helo=[mail.micron.com]
X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002BA50.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYXPR08MB9261
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Tue, 05 Dec 2023 01:02:54 -0800 (PST)


On 12/5/2023 2:21 PM, Michal Hocko wrote:
> CAUTION: EXTERNAL EMAIL. Do not click links or open attachments unless you recognize the sender and were expecting this message.
> 
> 
> On Tue 05-12-23 14:12:17, Srinivasulu Thanneeru wrote:
>>
>>
>> On 12/5/2023 2:05 PM, Michal Hocko wrote:
>>> CAUTION: EXTERNAL EMAIL. Do not click links or open attachments unless you recognize the sender and were expecting this message.
>>>
>>>
>>> On Tue 05-12-23 01:26:07, Srinivasulu Thanneeru wrote:
>>>>
>>>>
>>>> On 12/4/2023 9:13 PM, Michal Hocko wrote:
>>>>> CAUTION: EXTERNAL EMAIL. Do not click links or open attachments unless you recognize the sender and were expecting this message.
>>>>>
>>>>>
>>>>> On Fri 01-12-23 03:34:20, sthanneeru.opensrc@micron.com wrote:
>>>>>> From: Srinivasulu Thanneeru <sthanneeru.opensrc@micron.com>
>>>>>>
>>>>>> The memory tiers feature allows nodes with similar memory types
>>>>>> or performance characteristics to be grouped together in a
>>>>>> memory tier. However, there is currently no provision for
>>>>>> moving a node from one tier to another on demand.
>>>>>
>>>>> Could you expand on why this is really needed/necessary? What is the
>>>>> actual usecase?
>>>>
>>>> Hi Michal Hock,
>>>>
>>>> Following two use-cases we have observed.
>>>> 1. It is not accurate to group similar memory types in the same tier,
>>>>      because even similar memory types may have different speed grades.
>>>
>>> Presumably they are grouped based on a HW configuration. Does that mean
>>> that the configuration is wrong? Are you trying to workaround that by
>>> this interface?
>>>
>>>> 2. Some systems boots up with CXL devices and DRAM on the same memory-tier,
>>>> we need a way to move the CXL nodes to the correct tier from the user space.
>>>
>>> Again, could you expand a bit more and explain why this cannot be
>>> configured automatically?
>>
>> Yes, in both cases above, if hardware not automatically populated properly,
>> in that case this interface would help to correct it from user space.
>>
>> We had observed case-2 in our setups.
> 
> How hard it is to address this at the HW level?
> 
> Btw. this is really important piece of context that should be part of
> the changelog. Quite honestly introducing user interfaces solely to
> workaround HW issues seems a rather weak justification. Are there any
> usecases you can think of where this would be useful?

I'm not sure how difficult to fix it in the hardware.

Sure, i will capture the use-cases in the change log, will be sending V2 
by changing interface from adistance_offset to memtier_overwrite to 
avoid complicated math for finding offset at user-level.

Thank you Michal Hocko for the feedback.

> --
> Michal Hocko
> SUSE Labs