Introduce MHP_OFFLINE_INACCESSIBLE mhp_flag to mark the hotplugged
memory block as inaccessible during the memory hotplug addition phase.
With support for "memmap on memory", the altmap is prepared at this
stage. Architectures like s390 anticipate that memmap should not be
accessed until memory is physically accessible and is accessible only
when it enters the memory hotplug onlining phase using the memory
notifier. Introduce the flag to inform the memory hotplug
infrastructure that the memory remains inaccessible until the memory
hotplug onlining phase begins.
Implementation considerations:
mhp inaccessible flag is initially set in altmap. This is useful in
arch_add_memory(). When the memory block device is added, the mhp
inaccessible information is passed to memory_block. The flag is used in
subsequent patch to avoid accessing memmap during memory hotplug
addition phase.
Signed-off-by: Sumanth Korikkar <[email protected]>
---
drivers/base/memory.c | 2 ++
include/linux/memory.h | 1 +
include/linux/memory_hotplug.h | 10 ++++++++++
include/linux/memremap.h | 1 +
mm/memory_hotplug.c | 3 ++-
5 files changed, 16 insertions(+), 1 deletion(-)
diff --git a/drivers/base/memory.c b/drivers/base/memory.c
index 8a13babd826c..51915d5c3f88 100644
--- a/drivers/base/memory.c
+++ b/drivers/base/memory.c
@@ -774,6 +774,8 @@ static int add_memory_block(unsigned long block_id, unsigned long state,
mem->state = state;
mem->nid = NUMA_NO_NODE;
mem->altmap = altmap;
+ if (altmap)
+ mem->inaccessible = altmap->inaccessible;
INIT_LIST_HEAD(&mem->group_next);
#ifndef CONFIG_NUMA
diff --git a/include/linux/memory.h b/include/linux/memory.h
index f53cfdaaaa41..655714d4e65a 100644
--- a/include/linux/memory.h
+++ b/include/linux/memory.h
@@ -67,6 +67,7 @@ struct memory_group {
struct memory_block {
unsigned long start_section_nr;
unsigned long state; /* serialized by the dev->lock */
+ bool inaccessible; /* during memory addition phase */
int online_type; /* for passing data to online routine */
int nid; /* NID for this memory block */
/*
diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h
index 7d2076583494..8988cd5ad55d 100644
--- a/include/linux/memory_hotplug.h
+++ b/include/linux/memory_hotplug.h
@@ -106,6 +106,16 @@ typedef int __bitwise mhp_t;
* implies the node id (nid).
*/
#define MHP_NID_IS_MGID ((__force mhp_t)BIT(2))
+/*
+ * Mark the hotplugged memory block as inaccessible during the memory hotplug
+ * addition phase. With support for "memmap on memory," the altmap is prepared
+ * at this stage. Architectures like s390 anticipate that memmap should not be
+ * accessed until memory is physically accessible and is accessible only when
+ * it enters the memory hotplug onlining phase using the memory notifier.
+ * Utilize this flag to inform the memory hotplug infrastructure that the
+ * memory remains inaccessible until the memory hotplug onlining phase begins.
+ */
+#define MHP_OFFLINE_INACCESSIBLE ((__force mhp_t)BIT(3))
/*
* Extended parameters for memory hotplug:
diff --git a/include/linux/memremap.h b/include/linux/memremap.h
index 744c830f4b13..9837f3e6fb95 100644
--- a/include/linux/memremap.h
+++ b/include/linux/memremap.h
@@ -25,6 +25,7 @@ struct vmem_altmap {
unsigned long free;
unsigned long align;
unsigned long alloc;
+ bool inaccessible;
};
/*
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 7a5fc89a8652..1e02eba166b0 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1092,7 +1092,6 @@ int mhp_init_memmap_on_memory(unsigned long pfn, unsigned long nr_pages,
ret = kasan_add_zero_shadow(__va(PFN_PHYS(pfn)), PFN_PHYS(nr_pages));
if (ret)
return ret;
-
move_pfn_range_to_zone(zone, pfn, nr_pages, NULL, MIGRATE_UNMOVABLE);
for (i = 0; i < nr_pages; i++)
@@ -1439,6 +1438,8 @@ int __ref add_memory_resource(int nid, struct resource *res, mhp_t mhp_flags)
if (mhp_flags & MHP_MEMMAP_ON_MEMORY) {
if (mhp_supports_memmap_on_memory(size)) {
mhp_altmap.free = memory_block_memmap_on_memory_pages();
+ if (mhp_flags & MHP_OFFLINE_INACCESSIBLE)
+ mhp_altmap.inaccessible = true;
params.altmap = kmalloc(sizeof(struct vmem_altmap), GFP_KERNEL);
if (!params.altmap) {
ret = -ENOMEM;
--
2.39.2
On 23.11.23 10:23, Sumanth Korikkar wrote:
> Introduce MHP_OFFLINE_INACCESSIBLE mhp_flag to mark the hotplugged
> memory block as inaccessible during the memory hotplug addition phase.
> With support for "memmap on memory", the altmap is prepared at this
> stage. Architectures like s390 anticipate that memmap should not be
> accessed until memory is physically accessible and is accessible only
> when it enters the memory hotplug onlining phase using the memory
> notifier. Introduce the flag to inform the memory hotplug
> infrastructure that the memory remains inaccessible until the memory
> hotplug onlining phase begins.
>
> Implementation considerations:
> mhp inaccessible flag is initially set in altmap. This is useful in
> arch_add_memory(). When the memory block device is added, the mhp
> inaccessible information is passed to memory_block. The flag is used in
> subsequent patch to avoid accessing memmap during memory hotplug
> addition phase.
>
> Signed-off-by: Sumanth Korikkar <[email protected]>
> ---
> drivers/base/memory.c | 2 ++
> include/linux/memory.h | 1 +
> include/linux/memory_hotplug.h | 10 ++++++++++
> include/linux/memremap.h | 1 +
> mm/memory_hotplug.c | 3 ++-
> 5 files changed, 16 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/base/memory.c b/drivers/base/memory.c
> index 8a13babd826c..51915d5c3f88 100644
> --- a/drivers/base/memory.c
> +++ b/drivers/base/memory.c
> @@ -774,6 +774,8 @@ static int add_memory_block(unsigned long block_id, unsigned long state,
> mem->state = state;
> mem->nid = NUMA_NO_NODE;
> mem->altmap = altmap;
> + if (altmap)
> + mem->inaccessible = altmap->inaccessible;
> INIT_LIST_HEAD(&mem->group_next);
>
> #ifndef CONFIG_NUMA
> diff --git a/include/linux/memory.h b/include/linux/memory.h
> index f53cfdaaaa41..655714d4e65a 100644
> --- a/include/linux/memory.h
> +++ b/include/linux/memory.h
> @@ -67,6 +67,7 @@ struct memory_group {
> struct memory_block {
> unsigned long start_section_nr;
> unsigned long state; /* serialized by the dev->lock */
> + bool inaccessible; /* during memory addition phase */
Is that really required? After all, the altmap is stored in the memory
block and accessible there.
> int online_type; /* for passing data to online routine */
> int nid; /* NID for this memory block */
> /*
> diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h
> index 7d2076583494..8988cd5ad55d 100644
> --- a/include/linux/memory_hotplug.h
> +++ b/include/linux/memory_hotplug.h
> @@ -106,6 +106,16 @@ typedef int __bitwise mhp_t;
> * implies the node id (nid).
> */
> #define MHP_NID_IS_MGID ((__force mhp_t)BIT(2))
> +/*
> + * Mark the hotplugged memory block as inaccessible during the memory hotplug
> + * addition phase. With support for "memmap on memory," the altmap is prepared
> + * at this stage. Architectures like s390 anticipate that memmap should not be
> + * accessed until memory is physically accessible and is accessible only when
> + * it enters the memory hotplug onlining phase using the memory notifier.
> + * Utilize this flag to inform the memory hotplug infrastructure that the
> + * memory remains inaccessible until the memory hotplug onlining phase begins.
> + */
> +#define MHP_OFFLINE_INACCESSIBLE ((__force mhp_t)BIT(3))
I'd suggest to squash all 3 patches. Then we can properly document here:
/*
* The hotplugged memory is completely inaccessible while the memory is
* offline. The memory provider will handle MEM_PREPARE_ONLINE /
* MEM_FINISH_OFFLINE notifications and make the memory accessible.
*
* This flag is only relevant when used along with MHP_MEMMAP_ON_MEMORY,
* because the altmap cannot be written (e.g., poisoned) when adding
* memory -- before it is set online.
*
* This allows for adding memory with an altmap that is not currently
* made available by a hypervisor. When onlining that memory, the
* hypervisor can be instructed to make that memory available, and
* the onlining phase will not require any memory allocations, which is
* helpful in low-memory situations.
*/
Cheers,
David / dhildenb