Received: by 2002:a05:6a10:1287:0:0:0:0 with SMTP id d7csp62167pxv; Wed, 14 Jul 2021 22:54:08 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxYttIYHLr8fOwf4JHxIwehmdCHJon5wgkXvtXfhH67pdxkY+TXKUs6yboEGtEs9IoUrqiY X-Received: by 2002:a5d:914a:: with SMTP id y10mr1849405ioq.140.1626328448035; Wed, 14 Jul 2021 22:54:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1626328448; cv=none; d=google.com; s=arc-20160816; b=lZ6p99NWJmnUwgbyaeGTRsWt7BPslYIOy9AfGErVrL7/kAdmMeSFwXAzjpxz0rRX4b /SxGmZEGMbrGiws6/7TvWfURnp+181MlpLJjY/XzjA3yJbeO69HxAiv6x8BKcCihgKoF qvoAMVPm+/bulqXE1K0Ilz7MPcest+/0Ds3r8Jl/nxLsPa/0iznuVi411QzH3udpr1gt cZnyNgstSoUasIBCCsRQ7fxUuZazb4Bu7Ikn4/nMsIoDiiqYeWDP+RbSjfXrY7t/IVJJ iJgduQASSkf2DnPZelEzjj1uRSI0J60gsj4pWUgx1R6xqNKB4DMLFoIlkJnetGbNiHCv //Ew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=nJMBE7NQbNE12HiVnqW8Ux9aIWnkPkp1gNmzFVf3Ek0=; b=INf8TqCPwfaagcKY4pF3DbJbOV9a7suasOLZiqUpuQUn15O672TbGoScPkt0uGqWvM HpIj1pVOOQ4fUvi/AsY8tiDlzpGcaTzbeL2t1u+xuJrHbpEWVN5SAxPxu9PKqZHYWPVj D7+11/8Mrt38TIKkQ66E+DdkSDyNF8qPtN5YOMtpxfqq9xgwLT+bOxED5ASvQxSgbDkb QfNOc8layuSleGlrrbElimFK/JFT2L/rc5hx+VMzBj4xuzXBOyC7cybtzQ1jhdIhtDVm 3nKXaFbnSMpeBvWvCVGUbdIzhXmnftKQ2LAVsY/Pr9jhbG1Dyy2SgUCXcfP9RTyupwsT e7gA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id p31si6136348jac.95.2021.07.14.22.53.56; Wed, 14 Jul 2021 22:54:08 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239747AbhGOFzN (ORCPT + 99 others); Thu, 15 Jul 2021 01:55:13 -0400 Received: from mga02.intel.com ([134.134.136.20]:22887 "EHLO mga02.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239479AbhGOFzN (ORCPT ); Thu, 15 Jul 2021 01:55:13 -0400 X-IronPort-AV: E=McAfee;i="6200,9189,10045"; a="197661944" X-IronPort-AV: E=Sophos;i="5.84,240,1620716400"; d="scan'208";a="197661944" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2021 22:52:20 -0700 X-IronPort-AV: E=Sophos;i="5.84,240,1620716400"; d="scan'208";a="505591586" Received: from yhuang6-mobl1.sh.intel.com ([10.238.6.138]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2021 22:52:17 -0700 From: Huang Ying To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Dave Hansen , "Huang, Ying" , Yang Shi , Michal Hocko , Wei Xu , Zi Yan , osalvador , David Rientjes , Dan Williams , David Hildenbrand Subject: [PATCH -V10 2/9] mm/migrate: update node demotion order on hotplug events Date: Thu, 15 Jul 2021 13:51:38 +0800 Message-Id: <20210715055145.195411-3-ying.huang@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210715055145.195411-1-ying.huang@intel.com> References: <20210715055145.195411-1-ying.huang@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Dave Hansen Reclaim-based migration is attempting to optimize data placement in memory based on the system topology. If the system changes, so must the migration ordering. The implementation is conceptually simple and entirely unoptimized. On any memory or CPU hotplug events, assume that a node was added or removed and recalculate all migration targets. This ensures that the node_demotion[] array is always ready to be used in case the new reclaim mode is enabled. This recalculation is far from optimal, most glaringly that it does not even attempt to figure out the hotplug event would have some *actual* effect on the demotion order. But, given the expected paucity of hotplug events, this should be fine. Signed-off-by: Dave Hansen Signed-off-by: "Huang, Ying" Reviewed-by: Yang Shi Cc: Michal Hocko Cc: Wei Xu Cc: Zi Yan Cc: osalvador Cc: David Rientjes Cc: Dan Williams Cc: David Hildenbrand -- Changes since 20210618: * moved RCU part to the prev patch in series. Changes since 20210302: * remove duplicate synchronize_rcu() --- mm/migrate.c | 90 +++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 89 insertions(+), 1 deletion(-) diff --git a/mm/migrate.c b/mm/migrate.c index b7a40ab47648..a40c391f9ca7 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -49,6 +49,7 @@ #include #include #include +#include #include @@ -3057,6 +3058,7 @@ void migrate_vma_finalize(struct migrate_vma *migrate) EXPORT_SYMBOL(migrate_vma_finalize); #endif /* CONFIG_DEVICE_PRIVATE */ +#if defined(CONFIG_MEMORY_HOTPLUG) /* Disable reclaim-based migration. */ static void __disable_all_migrate_targets(void) { @@ -3191,10 +3193,96 @@ static void __set_migration_target_nodes(void) /* * For callers that do not hold get_online_mems() already. */ -__maybe_unused // <- temporay to prevent warnings during bisects static void set_migration_target_nodes(void) { get_online_mems(); __set_migration_target_nodes(); put_online_mems(); } + +/* + * React to hotplug events that might affect the migration targets + * like events that online or offline NUMA nodes. + * + * The ordering is also currently dependent on which nodes have + * CPUs. That means we need CPU on/offline notification too. + */ +static int migration_online_cpu(unsigned int cpu) +{ + set_migration_target_nodes(); + return 0; +} + +static int migration_offline_cpu(unsigned int cpu) +{ + set_migration_target_nodes(); + return 0; +} + +/* + * This leaves migrate-on-reclaim transiently disabled between + * the MEM_GOING_OFFLINE and MEM_OFFLINE events. This runs + * whether reclaim-based migration is enabled or not, which + * ensures that the user can turn reclaim-based migration at + * any time without needing to recalculate migration targets. + * + * These callbacks already hold get_online_mems(). That is why + * __set_migration_target_nodes() can be used as opposed to + * set_migration_target_nodes(). + */ +static int __meminit migrate_on_reclaim_callback(struct notifier_block *self, + unsigned long action, void *arg) +{ + switch (action) { + case MEM_GOING_OFFLINE: + /* + * Make sure there are not transient states where + * an offline node is a migration target. This + * will leave migration disabled until the offline + * completes and the MEM_OFFLINE case below runs. + */ + disable_all_migrate_targets(); + break; + case MEM_OFFLINE: + case MEM_ONLINE: + /* + * Recalculate the target nodes once the node + * reaches its final state (online or offline). + */ + __set_migration_target_nodes(); + break; + case MEM_CANCEL_OFFLINE: + /* + * MEM_GOING_OFFLINE disabled all the migration + * targets. Reenable them. + */ + __set_migration_target_nodes(); + break; + case MEM_GOING_ONLINE: + case MEM_CANCEL_ONLINE: + break; + } + + return notifier_from_errno(0); +} + +static int __init migrate_on_reclaim_init(void) +{ + int ret; + + ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "migrate on reclaim", + migration_online_cpu, + migration_offline_cpu); + /* + * In the unlikely case that this fails, the automatic + * migration targets may become suboptimal for nodes + * where N_CPU changes. With such a small impact in a + * rare case, do not bother trying to do anything special. + */ + WARN_ON(ret < 0); + + hotplug_memory_notifier(migrate_on_reclaim_callback, 100); + return 0; +} +late_initcall(migrate_on_reclaim_init); +#endif /* CONFIG_MEMORY_HOTPLUG */ -- 2.30.2