Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34;
From:   Sumanth Korikkar <sumanthk@linux.ibm.com>
To:     linux-mm <linux-mm@kvack.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        David Hildenbrand <david@redhat.com>
Cc:     Oscar Salvador <osalvador@suse.de>, Michal Hocko <mhocko@suse.com>,
        "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>,
        Anshuman Khandual <anshuman.khandual@arm.com>,
        Gerald Schaefer <gerald.schaefer@linux.ibm.com>,
        Sumanth Korikkar <sumanthk@linux.ibm.com>,
        Alexander Gordeev <agordeev@linux.ibm.com>,
        Heiko Carstens <hca@linux.ibm.com>,
        Vasily Gorbik <gor@linux.ibm.com>,
        linux-s390 <linux-s390@vger.kernel.org>,
        LKML <linux-kernel@vger.kernel.org>
Subject: [PATCH v2 1/7] mm/memory_hotplug: introduce mhp_flag MHP_OFFLINE_INACCESSIBLE
Date:   Thu, 23 Nov 2023 10:23:37 +0100
Message-Id: <20231123092343.1703707-2-sumanthk@linux.ibm.com>
In-Reply-To: <20231123092343.1703707-1-sumanthk@linux.ibm.com>
References: <20231123092343.1703707-1-sumanthk@linux.ibm.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk

Introduce MHP_OFFLINE_INACCESSIBLE mhp_flag to mark the hotplugged
memory block as inaccessible during the memory hotplug addition phase.
With support for "memmap on memory", the altmap is prepared at this
stage. Architectures like s390 anticipate that memmap should not be
accessed until memory is physically accessible and is accessible only
when it enters the memory hotplug onlining phase using the memory
notifier.  Introduce the flag to inform the memory hotplug
infrastructure that the memory remains inaccessible until the memory
hotplug onlining phase begins.

Implementation considerations:
mhp inaccessible flag is initially set in altmap. This is useful in
arch_add_memory(). When the memory block device is added, the mhp
inaccessible information is passed to memory_block. The flag is used in
subsequent patch to avoid accessing memmap during memory hotplug
addition phase.

Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
---
 drivers/base/memory.c          |  2 ++
 include/linux/memory.h         |  1 +
 include/linux/memory_hotplug.h | 10 ++++++++++
 include/linux/memremap.h       |  1 +
 mm/memory_hotplug.c            |  3 ++-
 5 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/drivers/base/memory.c b/drivers/base/memory.c
index 8a13babd826c..51915d5c3f88 100644
--- a/drivers/base/memory.c
+++ b/drivers/base/memory.c
@@ -774,6 +774,8 @@ static int add_memory_block(unsigned long block_id, unsigned long state,
 	mem->state = state;
 	mem->nid = NUMA_NO_NODE;
 	mem->altmap = altmap;
+	if (altmap)
+		mem->inaccessible = altmap->inaccessible;
 	INIT_LIST_HEAD(&mem->group_next);
 
 #ifndef CONFIG_NUMA
diff --git a/include/linux/memory.h b/include/linux/memory.h
index f53cfdaaaa41..655714d4e65a 100644
--- a/include/linux/memory.h
+++ b/include/linux/memory.h
@@ -67,6 +67,7 @@ struct memory_group {
 struct memory_block {
 	unsigned long start_section_nr;
 	unsigned long state;		/* serialized by the dev->lock */
+	bool inaccessible;		/* during memory addition phase */
 	int online_type;		/* for passing data to online routine */
 	int nid;			/* NID for this memory block */
 	/*
diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h
index 7d2076583494..8988cd5ad55d 100644
--- a/include/linux/memory_hotplug.h
+++ b/include/linux/memory_hotplug.h
@@ -106,6 +106,16 @@ typedef int __bitwise mhp_t;
  * implies the node id (nid).
  */
 #define MHP_NID_IS_MGID		((__force mhp_t)BIT(2))
+/*
+ * Mark the hotplugged memory block as inaccessible during the memory hotplug
+ * addition phase. With support for "memmap on memory," the altmap is prepared
+ * at this stage. Architectures like s390 anticipate that memmap should not be
+ * accessed until memory is physically accessible and is accessible only when
+ * it enters the memory hotplug onlining phase using the memory notifier.
+ * Utilize this flag to inform the memory hotplug infrastructure that the
+ * memory remains inaccessible until the memory hotplug onlining phase begins.
+ */
+#define MHP_OFFLINE_INACCESSIBLE	((__force mhp_t)BIT(3))
 
 /*
  * Extended parameters for memory hotplug:
diff --git a/include/linux/memremap.h b/include/linux/memremap.h
index 744c830f4b13..9837f3e6fb95 100644
--- a/include/linux/memremap.h
+++ b/include/linux/memremap.h
@@ -25,6 +25,7 @@ struct vmem_altmap {
 	unsigned long free;
 	unsigned long align;
 	unsigned long alloc;
+	bool inaccessible;
 };
 
 /*
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 7a5fc89a8652..1e02eba166b0 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1092,7 +1092,6 @@ int mhp_init_memmap_on_memory(unsigned long pfn, unsigned long nr_pages,
 	ret = kasan_add_zero_shadow(__va(PFN_PHYS(pfn)), PFN_PHYS(nr_pages));
 	if (ret)
 		return ret;
-
 	move_pfn_range_to_zone(zone, pfn, nr_pages, NULL, MIGRATE_UNMOVABLE);
 
 	for (i = 0; i < nr_pages; i++)
@@ -1439,6 +1438,8 @@ int __ref add_memory_resource(int nid, struct resource *res, mhp_t mhp_flags)
 	if (mhp_flags & MHP_MEMMAP_ON_MEMORY) {
 		if (mhp_supports_memmap_on_memory(size)) {
 			mhp_altmap.free = memory_block_memmap_on_memory_pages();
+			if (mhp_flags & MHP_OFFLINE_INACCESSIBLE)
+				mhp_altmap.inaccessible = true;
 			params.altmap = kmalloc(sizeof(struct vmem_altmap), GFP_KERNEL);
 			if (!params.altmap) {
 				ret = -ENOMEM;
-- 
2.39.2