Received: by 2002:a05:7412:8d10:b0:f3:1519:9f41 with SMTP id bj16csp6452410rdb; Thu, 14 Dec 2023 21:05:26 -0800 (PST) X-Google-Smtp-Source: AGHT+IHbdmxXzDCA2WNjtXLz1lIej6iAdRtKWhJVeB2xA5hkSeckZCHzLg6zzs2ajxt1jiSbTtCY X-Received: by 2002:a05:622a:c2:b0:425:9d92:821c with SMTP id p2-20020a05622a00c200b004259d92821cmr14561587qtw.131.1702616726171; Thu, 14 Dec 2023 21:05:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702616726; cv=none; d=google.com; s=arc-20160816; b=V32ELeLigViwdL56zNTi4cncRndqE6VvF85pwsECceMRs1VzSFx6MGjiPo45/fbLKC G26qWk8+7FAglNiFLt5R0InDJ1H+OYihPQagalEBqFGsHfCIsvjaJAKam0lOaE015B8j A0FYpZ8xWXDpZM6JAW1LKPgzOST9NUz9GS0d7cVfHSuLSK30eOfnsG8OZQ1+/6YmynTG VnEbDHq4P8/amSTNEqfwwGlfC8mzpZ+5flbTmC38eTCw1KRePxZd2zEeMJ6dOJIUIzJj 9MN2hM7fLcyZoOHRP+53m4CaKbaVhX31fXEjVkRIWC43Z7wF8j4V70WFNJBuWeqHu2Na i22g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:dkim-signature; bh=82tVI8yC7TSTADa68x5itJ1R7onWWe4C0HEZQDQqKoQ=; fh=bbHxXucMOVLTDh4fTEjnIcXhUMGqTKifR41gZEPp22g=; b=kom7IgQbGLpXyW1G/Yt+YBTB06xopBgzkMbj3XmlG7xan+NFt1PlTc8NXzMIzbxRGN 7OexM7E8KceG9f7k/jrKrHELxcPiQlY9WW330dR4UsycNw33mO4JijAB43WC4w4iC+Id DEjLgioTVc619tlxHDthI+XxZa0bwoHPQPglHUIfLx6FJEUOc4m4QrEkUON4Z17AAMFc of+UTHqMfwAYsKE76U96qsgW3CUc/0P5RI+u6lyjgThN/NueASbQqmNrCKTeQiQdEzkq yvvUZKPHHtqh0kDh40niseSxph79fXBvMTPCCoYNjhwkQs677Q51oQak85JAV8g4xUGC Qtww== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=F21HQ4Ba; spf=pass (google.com: domain of linux-kernel+bounces-437-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-437-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id y24-20020ac85258000000b0042579ad34d1si15879968qtn.775.2023.12.14.21.05.26 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Dec 2023 21:05:26 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-437-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=F21HQ4Ba; spf=pass (google.com: domain of linux-kernel+bounces-437-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-437-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id D234D1C22383 for ; Fri, 15 Dec 2023 05:05:25 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9D665C8C7; Fri, 15 Dec 2023 05:05:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="F21HQ4Ba" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 160CA846C; Fri, 15 Dec 2023 05:05:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1702616704; x=1734152704; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version:content-transfer-encoding; bh=zVDTobLFJhMgqVsvxBgPsU6QiHyyeU7rAJxmclOn3CI=; b=F21HQ4BaQcYH3+7/Fryu98kelSJuwRLRI/KIDq6xa0SE+V2GUwoOg1A1 gL/DXWVKPHwv8bkJLc7BUfGSP20cyOjyq8GbJj/LICBdleQlYQKy3U1NI WSBlZYkUbH1X9P2f271vHTRKqFDT5BMq1KbhXHOoL1zbbffq+855tl/zR W1F78eZTmNlpKcldF6k3rMuFr+HZ9F+s5qRfneqYUIOQOJOYkwU9UkWmU 8rPXEgYxQfBdYNVbrLHEHj7cOE8Y5U3jSXx38Lm15Kmd9Qce/4KSGoYYE 3458Zl4m8ei0UwW73orfhiV6oIYLQ9XmWBDiGX3gCuu7ICYna5S6HHAk9 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10924"; a="2398142" X-IronPort-AV: E=Sophos;i="6.04,277,1695711600"; d="scan'208";a="2398142" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orvoesa104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Dec 2023 21:05:04 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10924"; a="803585229" X-IronPort-AV: E=Sophos;i="6.04,277,1695711600"; d="scan'208";a="803585229" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Dec 2023 21:04:59 -0800 From: "Huang, Ying" To: Cc: , , , , , , , , , , , , , Subject: Re: [RFC PATCH v2 0/2] Node migration between memory tiers In-Reply-To: <20231213175329.594-1-sthanneeru.opensrc@micron.com> (sthanneeru opensrc's message of "Wed, 13 Dec 2023 23:23:27 +0530") References: <20231213175329.594-1-sthanneeru.opensrc@micron.com> Date: Fri, 15 Dec 2023 13:02:59 +0800 Message-ID: <87cyv8qcqk.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable writes: > From: Srinivasulu Thanneeru > > The memory tiers feature allows nodes with similar memory types > or performance characteristics to be grouped together in a > memory tier. However, there is currently no provision for > moving a node from one tier to another on demand. > > This patch series aims to support node migration between tiers > on demand by sysadmin/root user using the provided sysfs for > node migration. > > To migrate a node to a tier, the corresponding node=E2=80=99s sysfs > memtier_override is written with target tier id. > > Example: Move node2 to memory tier2 from its default tier(i.e 4) > > 1. To check current memtier of node2 > $cat /sys/devices/system/node/node2/memtier_override > memory_tier4 > > 2. To migrate node2 to memory_tier2 > $echo 2 > /sys/devices/system/node/node2/memtier_override > $cat /sys/devices/system/node/node2/memtier_override > memory_tier2 > > Usecases: > > 1. Useful to move cxl nodes to the right tiers from userspace, when > the hardware fails to assign the tiers correctly based on > memorytypes. > > On some platforms we have observed cxl memory being assigned to > the same tier as DDR memory. This is arguably a system firmware > bug, but it is true that tiers represent *ranges* of performance > and we believe it's important for the system operator to have > the ability to override bad firmware or OS decisions about tier > assignment as a fail-safe against potential bad outcomes. > > 2. Useful if we want interleave weights to be applied on memory tiers > instead of nodes. > In a previous thread, Huang Ying thought > this feature might be useful to overcome limitations of systems > where nodes with different bandwidth characteristics are grouped > in a single tier. > https://lore.kernel.org/lkml/87a5rw1wu8.fsf@yhuang6-desk2.ccr.corp.intel.= com/ > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > Version Notes: > > V2 : Changed interface to memtier_override from adistance_offset. > memtier_override was recommended by > 1. John Groves > 2. Ravi Shankar > 3. Brice Goglin It appears that you ignored my comments for V1 as follows ... https://lore.kernel.org/lkml/87o7f62vur.fsf@yhuang6-desk2.ccr.corp.intel.co= m/ https://lore.kernel.org/lkml/87jzpt2ft5.fsf@yhuang6-desk2.ccr.corp.intel.co= m/ https://lore.kernel.org/lkml/87a5qp2et0.fsf@yhuang6-desk2.ccr.corp.intel.co= m/ -- Best Regards, Huang, Ying > V1 : Introduced adistance_offset sysfs. > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > Srinivasulu Thanneeru (2): > base/node: Add sysfs for memtier_override > memory tier: Support node migration between tiers > > Documentation/ABI/stable/sysfs-devices-node | 7 ++ > drivers/base/node.c | 47 ++++++++++++ > include/linux/memory-tiers.h | 11 +++ > include/linux/node.h | 11 +++ > mm/memory-tiers.c | 85 ++++++++++++--------- > 5 files changed, 125 insertions(+), 36 deletions(-)