Hi All,
The patch series implements "memmap on memory" feature on s390.
Patch 1 introduces MEM_PREPARE_ONLINE/MEM_FINISH_OFFLINE memory
notifiers to prepare the transition of memory to and from a physically
accessible state. New mhp_flag MHP_OFFLINE_INACCESSIBLE is introduced to
ensure altmap cannot be written when addidng memory - before it is set
online. This enhancement is crucial for implementing the "memmap on
memory" feature for s390 in a subsequent patch.
Patches 2 allocates vmemmap pages from self-contained memory range for
s390. It allocates memory map (struct pages array) from the hotplugged
memory range, rather than using system memory by passing altmap to
vmemmap functions.
Patch 3 removes unhandled memory notifier types on s390.
Patch 4 implements MEM_PREPARE_ONLINE/MEM_FINISH_OFFLINE memory
notifiers on s390. MEM_PREPARE_ONLINE memory notifier makes memory block
physical accessible via sclp assign command. The notifier ensures
self-contained memory maps are accessible and hence enabling the "memmap
on memory" on s390. MEM_FINISH_OFFLINE memory notifier shifts the memory
block to an inaccessible state via sclp unassign command
Patch 5 finally enables MHP_MEMMAP_ON_MEMORY on s390
Note:
These patches are rebased on top of three fixes:
mm: use vmem_altmap code without CONFIG_ZONE_DEVICE
mm/memory_hotplug: fix error handling in add_memory_resource()
mm/memory_hotplug: add missing mem_hotplug_lock
v5:
* Added reviewed-by
* Removed variables altmap_start, altmap_size in sclp_cmd.c
* Used PFN_PHYS macro.
Thanks for the valualble feedback.
v4:
* Introduced two new fields, altmap_start_pfn and altmap_nr_pages, in
the memory_notify structure and document it that it is used only in
MEM_PREPARE_ONLINE/MEM_FINISH_OFFLINE callbacks.
* Incorporated the newly added fields into s390's
MEM_PREPARE_ONLINE/MEM_FINISH_OFFLINE notifier callbacks.
* Prevent access to memblock->altmap->free in the s390 notifier callback.
* page_init_poison() could be performed similar to when adding new
memory in sparse_add_section(). Perform it without cond_resched().
v3:
* added comments to MHP_OFFLINE_ACCESSIBLE as suggested by David.
* Squashed three commits related to new memory notifier.
v2:
* Fixes are integrated and hence removed from this patch series
Suggestions from David:
* Add new flag MHP_OFFLINE_INACCESSIBLE to avoid accessing memory
during memory hotplug addition phase.
* Avoid page_init_poison() on memmap during mhp addition phase, when
MHP_OFFLINE_INACCESSIBLE mhp_flag is passed in add_memory().
* Do not skip add_pages() in arch_add_memory(). Similarly, remove
similar hacks in arch_remove_memory().
* Use MHP_PREPARE_ONLINE/MHP_FINISH_OFFLINE naming convention for
new memory notifiers.
* Rearrange removal of unused s390 memory notifier.
* Necessary commit messages changes.
Thank you
Sumanth Korikkar (5):
mm/memory_hotplug: introduce MEM_PREPARE_ONLINE/MEM_FINISH_OFFLINE
notifiers
s390/mm: allocate vmemmap pages from self-contained memory range
s390/sclp: remove unhandled memory notifier type
s390/mm: implement MEM_PREPARE_ONLINE/MEM_FINISH_OFFLINE notifiers
s390: enable MHP_MEMMAP_ON_MEMORY
arch/s390/Kconfig | 1 +
arch/s390/mm/init.c | 3 --
arch/s390/mm/vmem.c | 62 +++++++++++++++++++---------------
drivers/base/memory.c | 23 ++++++++++++-
drivers/s390/char/sclp_cmd.c | 44 +++++++++++++++++++-----
include/linux/memory.h | 9 +++++
include/linux/memory_hotplug.h | 18 +++++++++-
include/linux/memremap.h | 1 +
mm/memory_hotplug.c | 13 ++++++-
mm/sparse.c | 3 +-
10 files changed, 134 insertions(+), 43 deletions(-)
--
2.40.1
Enable MHP_MEMMAP_ON_MEMORY to support "memmap on memory".
memory_hotplug.memmap_on_memory=true kernel parameter should be set in
kernel boot option to enable the feature.
Reviewed-by: Gerald Schaefer <[email protected]>
Signed-off-by: Sumanth Korikkar <[email protected]>
---
arch/s390/Kconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index 3bec98d20283..4b9b0f947ddb 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -113,6 +113,7 @@ config S390
select ARCH_INLINE_WRITE_UNLOCK_BH
select ARCH_INLINE_WRITE_UNLOCK_IRQ
select ARCH_INLINE_WRITE_UNLOCK_IRQRESTORE
+ select ARCH_MHP_MEMMAP_ON_MEMORY_ENABLE
select ARCH_STACKWALK
select ARCH_SUPPORTS_ATOMIC_RMW
select ARCH_SUPPORTS_DEBUG_PAGEALLOC
--
2.40.1
On Tue, 28 Nov 2023 16:52:22 +0100 Sumanth Korikkar <[email protected]> wrote:
> The patch series implements "memmap on memory" feature on s390.
The cover letter doesn't acutally have a description of what "memmap on
memory" *is*. A nice overview to help readers understand what they're
about to look at. A description of what value this feature brings to
our users. Use-cases. That sort of thing.
I guess the [1/N] changelog covers it, but it's hard to tell. It isn't
exactly broad-sweep overview.
Probably something short would suffice. There are plenty of examples
on the mailing list, please take a look and send us something?
On Tue, Nov 28, 2023 at 03:56:34PM -0800, Andrew Morton wrote:
Hi Andrew,
> The cover letter doesn't acutally have a description of what "memmap on
> memory" *is*. A nice overview to help readers understand what they're
> about to look at. A description of what value this feature brings to
> our users. Use-cases. That sort of thing.
>
> I guess the [1/N] changelog covers it, but it's hard to tell. It isn't
> exactly broad-sweep overview.
>
> Probably something short would suffice. There are plenty of examples
> on the mailing list, please take a look and send us something?
Sumanth is on vacation, I will try to answer.
This series brings "memmap on memory" support to s390 platform.
That is - allocate 'struct pages' array describing a memory block
being onlined not from available system memory, but from the
memory block itself.
This improves the current situation on s390, where struct pages
for all memory that potentially can be added to the system, is
preallocated, so that memory online will always succeed but at
the cost of significant memory consumption.
Unlike other architectures, the challenge s390 faces is the memory
block being onlined is not accessible at the time of hotplug event.
To make it physically accessible two new MEM_PREPARE_ONLINE and
MEM_FINISH_OFFLINE memory notifiers are added. That allows to succeed
"memmap on memory" initialization.
For the existing architectures the two new notifiers are unknown
and they are not affected in any way.
Thanks!