Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18;
Message-ID: <eaf22a62-73c7-d96b-fcd8-4437eb3d8559@amd.com>
Date:   Mon, 6 Jun 2022 17:39:56 +0530
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.8.1
Subject: Re: [RFC PATCH v4 4/7] mm/demotion/dax/kmem: Set node's memory tier
 to MEMORY_TIER_PMEM
Content-Language: en-US
To:     "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>,
        linux-mm@kvack.org, akpm@linux-foundation.org
Cc:     Huang Ying <ying.huang@intel.com>,
        Greg Thelen <gthelen@google.com>,
        Yang Shi <shy828301@gmail.com>,
        Davidlohr Bueso <dave@stgolabs.net>,
        Tim C Chen <tim.c.chen@intel.com>,
        Brice Goglin <brice.goglin@gmail.com>,
        Michal Hocko <mhocko@kernel.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Hesham Almatary <hesham.almatary@huawei.com>,
        Dave Hansen <dave.hansen@intel.com>,
        Jonathan Cameron <Jonathan.Cameron@huawei.com>,
        Alistair Popple <apopple@nvidia.com>,
        Dan Williams <dan.j.williams@intel.com>,
        Feng Tang <feng.tang@intel.com>,
        Jagdish Gediya <jvgediya@linux.ibm.com>,
        Baolin Wang <baolin.wang@linux.alibaba.com>,
        David Rientjes <rientjes@google.com>
References: <CAAPL-u-dFp7PwPH6DfbYdnY8xaGsHz3tRQ0CPGVkiqURvdN8=A@mail.gmail.com>
 <20220527122528.129445-1-aneesh.kumar@linux.ibm.com>
 <20220527122528.129445-5-aneesh.kumar@linux.ibm.com>
 <5706f5e9-0609-98c9-a0cd-7d96336d73dd@amd.com>
 <8e651a1e-d189-3e8a-438f-298f21402bd2@linux.ibm.com>
 <d45374fa-6e51-36cb-9a2c-96f85d9de528@amd.com>
 <c98eb873-a5bb-edcc-743d-89cfffe52cd9@linux.ibm.com>
 <d6c94fd5-053b-7e6f-dc4e-83184ecf131a@amd.com>
 <a844c8c9-e1e1-2ccb-d58c-a5a608afabc0@linux.ibm.com>
 <87fski80sx.fsf@linux.ibm.com>
From:   Bharata B Rao <bharata@amd.com>
In-Reply-To: <87fski80sx.fsf@linux.ibm.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?eWdTV2I0RWxKSVM4azBoY0lTZmIyUTV4ak1wVFVTYm5KekUwZm1MeHYydVBo?=
 =?utf-8?B?cVBpUWJVTUZzcFdkVVM0U0dIL05tdnFneC9Sb1ZreFlJYWdCa3lmUDQ4ZHY5?=
 =?utf-8?B?d2RJNm1wT1NJMEJDMURiek5GY2NYTGRSVndhUTZnTWRHQkp2NEg3blg2THZv?=
 =?utf-8?B?V1UrcXB4cDd5S0Vpd2R0TWp3a0kzc3poSEZka1lhT2ZFRVo4ZzkrWFdnQm9Y?=
 =?utf-8?B?ZFNmMHg2MkRFNXlvcnpOZFovWU9LQ0lMdExlZEdJUFo1NnVITFlYWnBYSnht?=
 =?utf-8?B?L2Z4bktITE1vOXpOWFZrbEcxUDdGUEd2K29qajBhNjZFRzNpTDM3MlBrSXFJ?=
 =?utf-8?B?eUErOE0rcTNGOGhWOTJMcDBSVzRvSnJ2UFg5TU4rVlNyZVpkTE01TmdCZ1FE?=
 =?utf-8?B?VHMycnZ0eHFJSE9tTllGV2FMdnVBSFNzRzFGN0hScHFOMklFaTcrRjJneHE4?=
 =?utf-8?B?eVpLdzB4NDNnK1VMN3RhYk5LdFh3WEZ5ZGhlbFFZOWlLUkxnOTNPVFB2YW5E?=
 =?utf-8?B?UlFEMXJtRVBYMGYzWTkyV1M1M2YwUm1rSzdKL0JRVy9BeXI1dWQ2WUZsbFhH?=
 =?utf-8?B?S2ZvSTM4MzU5eTZMamtMdE9wMFhLMVdyQ2dJam9YMG9HZjBhcjNWWXlDSmQz?=
 =?utf-8?B?aVppaU9MUmQzZGN6SHVkL29zOTl5bVBVS1lhWFZoNE50ZnVmZ3FVRnk5R2Vm?=
 =?utf-8?B?QndVdlVhYTloVUpxdU9DanhFZUxBZWJXelR2Y2d3cFM2UDhwQ0kxbm5oSWNw?=
 =?utf-8?B?eTdGczh3cVJNRzJuckhyQTdWUmFjeUdTSnkxZWhwdG1HZmJvUVIwcjAzLzFE?=
 =?utf-8?B?RURnTmNsVk5BY3JRT3BxM2kveXFUNVNLeTZDOXoxclpYRXd5RDMxZmVEeFBK?=
 =?utf-8?B?SG56dm9WakExMDJEaXBjWWdVcVFBTDMzZENIVUxUa25iQzFMc3lQZ1Y2U0tG?=
 =?utf-8?B?dU85czlFZGhETVRyTXNvRDRiUXY5amxFZC9IUTQvWnRMZ0RwMHk3YW00aytp?=
 =?utf-8?B?QlUxSEdXZ3RUd0k5VHhCT0diQzZ1QmhyY0taMzMvYmNlbWJ4ekxnbUk1QjFN?=
 =?utf-8?B?K0g5ZldJRXd3S0lGY1pmdGlmSldvWkRVQmV1cit4amdSTXljZjlNcW9kMGkw?=
 =?utf-8?B?a1NERWhjcXBjOTF2LzFkSERhNmNzTENyM2xpUmdTUFNTVU5wOFA3M1RNa2NW?=
 =?utf-8?B?RVJEVEE5Vk9na3U4V0QwL2lMdXYya1c3cTRIZFIxOGJLNE1GcnpaeXZTcmRj?=
 =?utf-8?B?RDZ1S1hiYk45dVdlQ0ltbVpBZXdCTzFiOU0yKzZ3UUUyRzR2aHB3UjQ2S3pP?=
 =?utf-8?B?WTI3QVpmOHdCZW9EL1doRnpPeTk1UFd1MDBzbEZsVXcvOUliVmNTdDNBaHN0?=
 =?utf-8?B?Wk1DNGFlR0IwbGY1T0J6MjhFdHhPaVNObnMwK1V2aFN1OU82cFZ1aHd3VTNN?=
 =?utf-8?B?d1hQeVlBams1NElHMEtVMnNDbTdOM3NiVk5xbEU1ODJ5dXZLZGFGbFZEUHl2?=
 =?utf-8?B?ZnozQWJWQklYTFBuY0Y4NXQwSmRyK3dhT25zSmxGUlNCZWFxY1BUMm00MWxV?=
 =?utf-8?B?aVJNRkN5dHJoamNIUXdTclBvM0s2cVRGUWt5ckoxMnczblc0bmlpYzFGbFhh?=
 =?utf-8?B?M3hyQmZScmJlcUdYVkJVU1FRUjZ0M2VkUkFxUitveGtlSVc5bytTbEMvUU55?=
 =?utf-8?B?RkZ6dWpKRE5zM2Q4NVN3ZjRnK1ozVnNDcXFSNzN5SGFQRVllVlFqQzhOS25O?=
 =?utf-8?B?VFdDcjdWN3dnTWdSbFRLTEczRXEraCs5SFNIWE9yZ3BRTHZISTFPUUJScFBY?=
 =?utf-8?B?OHBCOHVRanlvUDBmWTdKN1lOKzdTRktKQ25iSlc1VjVnN05rRk9LU3VyZ0hV?=
 =?utf-8?B?Q2xEWWlyT0lFa0Y0aG9oR0NCak0wbEJkZEdBOFhVcHBPbzNiREpoZVhVTVNQ?=
 =?utf-8?B?VU0zcStWUE1jbHNoaTVuZ3NkR3lkSTkxTWtKbkMvZ2pGTmRUMlZGYzExcXlj?=
 =?utf-8?B?L0laZlNlZmhkR1M2NkRVNE9KTzE4Q3M1MG5uYWptWEduaHo0ZVJOa3VQeE1G?=
 =?utf-8?B?d2kxY2pqUCtRRFkrYzZKNWh1SC95aVVlVkUvRHlBWThpL3oyS2dPdXYzV2FB?=
 =?utf-8?B?bGZoWTBpSFF2dWFsSS9oL0ViZ2FveGY0T0VOUGFsZWFQLzFxOWRJbE9DZ25T?=
 =?utf-8?B?OVUrK0FZL2c4VEx3Y3ZOSG5TVHZ2MTJub0VldndFVmdJVlp1eXRFUG1wcllr?=
 =?utf-8?B?aVE0YmpmVjhKbmVoTnM5QXNFMkxvTDlkVklxQi8wN0hXRUZGd2JqWnJpdEVY?=
 =?utf-8?B?OTM2NS9vZEJRM25YR1BvVTZWUWRjUlJNWGJvc2xHOStENmY5aGRjQT09?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bdb47771-ebd4-412a-ebba-08da47b583a9
X-MS-Exchange-CrossTenant-AuthSource: IA1PR12MB6434.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jun 2022 12:10:15.2606
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: QCWaJDdoUePOUcktbuxfyNdFdex56ReyhO/cVIJxE9Xkc9EAnoWerpcjb8ZomGTiS5CUhWOck1mpS6AuD24npg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR12MB1934
X-Spam-Status: No, score=-4.9 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,NICE_REPLY_A,RDNS_NONE,SPF_HELO_NONE,
	T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no
	version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 6/6/2022 5:24 PM, Aneesh Kumar K.V wrote:
> Aneesh Kumar K V <aneesh.kumar@linux.ibm.com> writes:
>>
> Can you try this change?
> 
> diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c
> index 7a11c387fbbc..905609260dda 100644
> --- a/drivers/dax/kmem.c
> +++ b/drivers/dax/kmem.c
> @@ -94,6 +94,17 @@ static int dev_dax_kmem_probe(struct dev_dax *dev_dax)
>  		goto err_reg_mgid;
>  	data->mgid = rc;
>  
> +	/*
> +	 * This get called before the node is brought online. That
> +	 * is because depending on the value of mhp_default_online_type
> +	 * the kernel will online the memory along with hotplug
> +	 * operation. Add the new memory tier before we try to bring
> +	 * memory blocks online. Otherwise new node will get added to
> +	 * the default memory tier via hotplug callbacks.
> +	 */
> +#ifdef CONFIG_TIERED_MEMORY
> +	node_set_memory_tier(numa_node, MEMORY_TIER_PMEM);
> +#endif
>  	for (i = 0; i < dev_dax->nr_range; i++) {
>  		struct resource *res;
>  		struct range range;
> @@ -148,9 +159,6 @@ static int dev_dax_kmem_probe(struct dev_dax *dev_dax)
>  
>  	dev_set_drvdata(dev, data);
>  
> -#ifdef CONFIG_TIERED_MEMORY
> -	node_set_memory_tier(numa_node, MEMORY_TIER_PMEM);
> -#endif
>  	return 0;
>  
>  err_request_mem:

Yes, this fixes the issue for me. Thanks.

Regards,
Bharata.