Hello guys, here comes the v5 version. Any comments are welcome!
The v5 version is based on today's linus tree (3.12-rc2)
HEAD is:
commit 4a10c2ac2f368583138b774ca41fac4207911983
Author: Linus Torvalds <[email protected]>
Date: Mon Sep 23 15:41:09 2013 -0700
Linux 3.12-rc2
[Problem]
The current Linux cannot migrate pages used by the kerenl because
of the kernel direct mapping. In Linux kernel space, va = pa + PAGE_OFFSET.
When the pa is changed, we cannot simply update the pagetable and
keep the va unmodified. So the kernel pages are not migratable.
There are also some other issues will cause the kernel pages not migratable.
For example, the physical address may be cached somewhere and will be used.
It is not to update all the caches.
When doing memory hotplug in Linux, we first migrate all the pages in one
memory device somewhere else, and then remove the device. But if pages are
used by the kernel, they are not migratable. As a result, memory used by
the kernel cannot be hot-removed.
Modifying the kernel direct mapping mechanism is too difficult to do. And
it may cause the kernel performance down and unstable. So we use the following
way to do memory hotplug.
[What we are doing]
In Linux, memory in one numa node is divided into several zones. One of the
zones is ZONE_MOVABLE, which the kernel won't use.
In order to implement memory hotplug in Linux, we are going to arrange all
hotpluggable memory in ZONE_MOVABLE so that the kernel won't use these memory.
To do this, we need ACPI's help.
In ACPI, SRAT(System Resource Affinity Table) contains NUMA info. The memory
affinities in SRAT record every memory range in the system, and also, flags
specifying if the memory range is hotpluggable.
(Please refer to ACPI spec 5.0 5.2.16)
With the help of SRAT, we have to do the following two things to achieve our
goal:
1. When doing memory hot-add, allow the users arranging hotpluggable as
ZONE_MOVABLE.
(This has been done by the MOVABLE_NODE functionality in Linux.)
2. when the system is booting, prevent bootmem allocator from allocating
hotpluggable memory for the kernel before the memory initialization
finishes.
The problem 2 is the key problem we are going to solve. But before solving it,
we need some preparation. Please see below.
[Preparation]
Bootloader has to load the kernel image into memory. And this memory must be
unhotpluggable. We cannot prevent this anyway. So in a memory hotplug system,
we can assume any node the kernel resides in is not hotpluggable.
Before SRAT is parsed, we don't know which memory ranges are hotpluggable. But
memblock has already started to work. In the current kernel, memblock allocates
the following memory before SRAT is parsed:
setup_arch()
|->memblock_x86_fill() /* memblock is ready */
|......
|->early_reserve_e820_mpc_new() /* allocate memory under 1MB */
|->reserve_real_mode() /* allocate memory under 1MB */
|->init_mem_mapping() /* allocate page tables, about 2MB to map 1GB memory */
|->dma_contiguous_reserve() /* specified by user, should be low */
|->setup_log_buf() /* specified by user, several mega bytes */
|->relocate_initrd() /* could be large, but will be freed after boot, should reorder */
|->acpi_initrd_override() /* several mega bytes */
|->reserve_crashkernel() /* could be large, should reorder */
|......
|->initmem_init() /* Parse SRAT */
According to Tejun's advice, before SRAT is parsed, we should try our best to
allocate memory near the kernel image. Since the whole node the kernel resides
in won't be hotpluggable, and for a modern server, a node may have at least 16GB
memory, allocating several mega bytes memory around the kernel image won't cross
to hotpluggable memory.
[About this patch-set]
So this patch-set does the following:
1. Make memblock be able to allocate memory bottom up.
1) Keep all the memblock APIs' prototype unmodified.
2) When the direction is bottom up, keep the start address greater than the
end of kernel image.
2. Improve init_mem_mapping() to support allocate page tables in bottom up direction.
3. Introduce "movablenode" boot option to enable and disable this functionality.
PS: Reordering of relocate_initrd() has not been done yet. acpi_initrd_override()
needs to access initrd with virtual address. So relocate_initrd() must be done
before acpi_initrd_override().
Change log v4 -> v5:
1. Change memblock.current_direction to a boolean memblock.bottom_up. And remove
the direction enum.
2. Update and add some comments to explain things clearer.
3. Misc fixes, such as removing unnecessary #ifdef
Change log v3 -> v4:
1. Use bottom-up/top-down to unify things. Thanks tj.
2. Factor out of current top-down implementation and then introduce bottom-up mode,
not mixing them in one patch. Thanks tj.
3. Changes function naming: memblock_direction_bottom_up -> memblock_bottom_up
4. Use memblock_set_bottom_up to replace memblock_set_current_direction, which makes
the code simpler. Thanks tj.
5. Define two implementions of function memblock_bottom_up and memblock_set_bottom_up
in order not to use #ifdef in the boot code. Thanks tj.
6. Add comments to explain why retry top-down allocation when bottom_up allocation
failed. Thanks tj and toshi!
Change log v2 -> v3:
1. According to Toshi's suggestion, move the direction checking logic into memblock.
And simply the code more.
Change log v1 -> v2:
1. According to tj's suggestion, implemented a new function memblock_alloc_bottom_up()
to allocate memory from bottom upwards, whihc can simplify the code.
Tang Chen (6):
memblock: Factor out of top-down allocation
memblock: Introduce bottom-up allocation mode
x86/mm: Factor out of top-down direct mapping setup
x86/mem-hotplug: Support initialize page tables in bottom-up
x86, acpi, crash, kdump: Do reserve_crashkernel() after SRAT is
parsed.
mem-hotplug: Introduce movablenode boot option
Documentation/kernel-parameters.txt | 15 ++++
arch/x86/kernel/setup.c | 15 ++++-
arch/x86/mm/init.c | 118 ++++++++++++++++++++++++++++------
include/linux/memblock.h | 16 +++++
mm/memblock.c | 122 +++++++++++++++++++++++++++++++----
mm/memory_hotplug.c | 31 +++++++++
6 files changed, 283 insertions(+), 34 deletions(-)
From: Tang Chen <[email protected]>
This patch creates a new function __memblock_find_range_rev
to factor out of top-down allocation from memblock_find_in_range_node.
This is a preparation because we will introduce a new bottom-up
allocation mode in the following patch.
Acked-by: Tejun Heo <[email protected]>
Signed-off-by: Tang Chen <[email protected]>
Signed-off-by: Zhang Yanfei <[email protected]>
---
mm/memblock.c | 47 ++++++++++++++++++++++++++++++++++-------------
1 files changed, 34 insertions(+), 13 deletions(-)
diff --git a/mm/memblock.c b/mm/memblock.c
index 0ac412a..3d80c74 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -83,33 +83,25 @@ static long __init_memblock memblock_overlaps_region(struct memblock_type *type,
}
/**
- * memblock_find_in_range_node - find free area in given range and node
+ * __memblock_find_range_rev - find free area utility, in reverse order
* @start: start of candidate range
* @end: end of candidate range, can be %MEMBLOCK_ALLOC_{ANYWHERE|ACCESSIBLE}
* @size: size of free area to find
* @align: alignment of free area to find
* @nid: nid of the free area to find, %MAX_NUMNODES for any node
*
- * Find @size free area aligned to @align in the specified range and node.
+ * Utility called from memblock_find_in_range_node(), find free area top-down.
*
* RETURNS:
* Found address on success, %0 on failure.
*/
-phys_addr_t __init_memblock memblock_find_in_range_node(phys_addr_t start,
- phys_addr_t end, phys_addr_t size,
- phys_addr_t align, int nid)
+static phys_addr_t __init_memblock
+__memblock_find_range_rev(phys_addr_t start, phys_addr_t end,
+ phys_addr_t size, phys_addr_t align, int nid)
{
phys_addr_t this_start, this_end, cand;
u64 i;
- /* pump up @end */
- if (end == MEMBLOCK_ALLOC_ACCESSIBLE)
- end = memblock.current_limit;
-
- /* avoid allocating the first page */
- start = max_t(phys_addr_t, start, PAGE_SIZE);
- end = max(start, end);
-
for_each_free_mem_range_reverse(i, nid, &this_start, &this_end, NULL) {
this_start = clamp(this_start, start, end);
this_end = clamp(this_end, start, end);
@@ -121,10 +113,39 @@ phys_addr_t __init_memblock memblock_find_in_range_node(phys_addr_t start,
if (cand >= this_start)
return cand;
}
+
return 0;
}
/**
+ * memblock_find_in_range_node - find free area in given range and node
+ * @start: start of candidate range
+ * @end: end of candidate range, can be %MEMBLOCK_ALLOC_{ANYWHERE|ACCESSIBLE}
+ * @size: size of free area to find
+ * @align: alignment of free area to find
+ * @nid: nid of the free area to find, %MAX_NUMNODES for any node
+ *
+ * Find @size free area aligned to @align in the specified range and node.
+ *
+ * RETURNS:
+ * Found address on success, %0 on failure.
+ */
+phys_addr_t __init_memblock memblock_find_in_range_node(phys_addr_t start,
+ phys_addr_t end, phys_addr_t size,
+ phys_addr_t align, int nid)
+{
+ /* pump up @end */
+ if (end == MEMBLOCK_ALLOC_ACCESSIBLE)
+ end = memblock.current_limit;
+
+ /* avoid allocating the first page */
+ start = max_t(phys_addr_t, start, PAGE_SIZE);
+ end = max(start, end);
+
+ return __memblock_find_range_rev(start, end, size, align, nid);
+}
+
+/**
* memblock_find_in_range - find free area in given range
* @start: start of candidate range
* @end: end of candidate range, can be %MEMBLOCK_ALLOC_{ANYWHERE|ACCESSIBLE}
--
1.7.1
From: Tang Chen <[email protected]>
The Linux kernel cannot migrate pages used by the kernel. As a result, kernel
pages cannot be hot-removed. So we cannot allocate hotpluggable memory for
the kernel.
ACPI SRAT (System Resource Affinity Table) contains the memory hotplug info.
But before SRAT is parsed, memblock has already started to allocate memory
for the kernel. So we need to prevent memblock from doing this.
In a memory hotplug system, any numa node the kernel resides in should
be unhotpluggable. And for a modern server, each node could have at least
16GB memory. So memory around the kernel image is highly likely unhotpluggable.
So the basic idea is: Allocate memory from the end of the kernel image and
to the higher memory. Since memory allocation before SRAT is parsed won't
be too much, it could highly likely be in the same node with kernel image.
The current memblock can only allocate memory top-down. So this patch introduces
a new bottom-up allocation mode to allocate memory bottom-up. And later
when we use this allocation direction to allocate memory, we will limit
the start address above the kernel.
Signed-off-by: Tang Chen <[email protected]>
Signed-off-by: Zhang Yanfei <[email protected]>
---
include/linux/memblock.h | 16 +++++++++
mm/memblock.c | 81 ++++++++++++++++++++++++++++++++++++++++++++-
2 files changed, 95 insertions(+), 2 deletions(-)
diff --git a/include/linux/memblock.h b/include/linux/memblock.h
index 31e95ac..c1e2633 100644
--- a/include/linux/memblock.h
+++ b/include/linux/memblock.h
@@ -35,6 +35,7 @@ struct memblock_type {
};
struct memblock {
+ bool bottom_up; /* is bottom up direction? */
phys_addr_t current_limit;
struct memblock_type memory;
struct memblock_type reserved;
@@ -148,6 +149,21 @@ phys_addr_t memblock_alloc_try_nid(phys_addr_t size, phys_addr_t align, int nid)
phys_addr_t memblock_alloc(phys_addr_t size, phys_addr_t align);
+#ifdef CONFIG_MOVABLE_NODE
+static inline void memblock_set_bottom_up(bool enable)
+{
+ memblock.bottom_up = enable;
+}
+
+static inline bool memblock_bottom_up(void)
+{
+ return memblock.bottom_up;
+}
+#else
+static inline void memblock_set_bottom_up(bool enable) {}
+static inline bool memblock_bottom_up(void) { return false; }
+#endif
+
/* Flags for memblock_alloc_base() amd __memblock_alloc_base() */
#define MEMBLOCK_ALLOC_ANYWHERE (~(phys_addr_t)0)
#define MEMBLOCK_ALLOC_ACCESSIBLE 0
diff --git a/mm/memblock.c b/mm/memblock.c
index 3d80c74..a8e81c3 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -20,6 +20,8 @@
#include <linux/seq_file.h>
#include <linux/memblock.h>
+#include <asm-generic/sections.h>
+
static struct memblock_region memblock_memory_init_regions[INIT_MEMBLOCK_REGIONS] __initdata_memblock;
static struct memblock_region memblock_reserved_init_regions[INIT_MEMBLOCK_REGIONS] __initdata_memblock;
@@ -32,6 +34,7 @@ struct memblock memblock __initdata_memblock = {
.reserved.cnt = 1, /* empty dummy entry */
.reserved.max = INIT_MEMBLOCK_REGIONS,
+ .bottom_up = false,
.current_limit = MEMBLOCK_ALLOC_ANYWHERE,
};
@@ -83,6 +86,38 @@ static long __init_memblock memblock_overlaps_region(struct memblock_type *type,
}
/**
+ * __memblock_find_range - find free area utility
+ * @start: start of candidate range
+ * @end: end of candidate range, can be %MEMBLOCK_ALLOC_{ANYWHERE|ACCESSIBLE}
+ * @size: size of free area to find
+ * @align: alignment of free area to find
+ * @nid: nid of the free area to find, %MAX_NUMNODES for any node
+ *
+ * Utility called from memblock_find_in_range_node(), find free area bottom-up.
+ *
+ * RETURNS:
+ * Found address on success, 0 on failure.
+ */
+static phys_addr_t __init_memblock
+__memblock_find_range(phys_addr_t start, phys_addr_t end, phys_addr_t size,
+ phys_addr_t align, int nid)
+{
+ phys_addr_t this_start, this_end, cand;
+ u64 i;
+
+ for_each_free_mem_range(i, nid, &this_start, &this_end, NULL) {
+ this_start = clamp(this_start, start, end);
+ this_end = clamp(this_end, start, end);
+
+ cand = round_up(this_start, align);
+ if (cand < this_end && this_end - cand >= size)
+ return cand;
+ }
+
+ return 0;
+}
+
+/**
* __memblock_find_range_rev - find free area utility, in reverse order
* @start: start of candidate range
* @end: end of candidate range, can be %MEMBLOCK_ALLOC_{ANYWHERE|ACCESSIBLE}
@@ -93,7 +128,7 @@ static long __init_memblock memblock_overlaps_region(struct memblock_type *type,
* Utility called from memblock_find_in_range_node(), find free area top-down.
*
* RETURNS:
- * Found address on success, %0 on failure.
+ * Found address on success, 0 on failure.
*/
static phys_addr_t __init_memblock
__memblock_find_range_rev(phys_addr_t start, phys_addr_t end,
@@ -127,13 +162,24 @@ __memblock_find_range_rev(phys_addr_t start, phys_addr_t end,
*
* Find @size free area aligned to @align in the specified range and node.
*
+ * When allocation direction is bottom-up, the @start should be greater
+ * than the end of the kernel image. Otherwise, it will be trimmed. The
+ * reason is that we want the bottom-up allocation just near the kernel
+ * image so it is highly likely that the allocated memory and the kernel
+ * will reside in the same node.
+ *
+ * If bottom-up allocation failed, will try to allocate memory top-down.
+ *
* RETURNS:
- * Found address on success, %0 on failure.
+ * Found address on success, 0 on failure.
*/
phys_addr_t __init_memblock memblock_find_in_range_node(phys_addr_t start,
phys_addr_t end, phys_addr_t size,
phys_addr_t align, int nid)
{
+ int ret;
+ phys_addr_t kernel_end;
+
/* pump up @end */
if (end == MEMBLOCK_ALLOC_ACCESSIBLE)
end = memblock.current_limit;
@@ -141,6 +187,37 @@ phys_addr_t __init_memblock memblock_find_in_range_node(phys_addr_t start,
/* avoid allocating the first page */
start = max_t(phys_addr_t, start, PAGE_SIZE);
end = max(start, end);
+ kernel_end = __pa_symbol(_end);
+
+ /*
+ * try bottom-up allocation only when bottom-up mode
+ * is set and @end is above the kernel image.
+ */
+ if (memblock_bottom_up() && end > kernel_end) {
+ phys_addr_t bottom_up_start;
+
+ /* make sure we will allocate above the kernel */
+ bottom_up_start = max(start, kernel_end);
+
+ /* ok, try bottom-up allocation first */
+ ret = __memblock_find_range(bottom_up_start, end,
+ size, align, nid);
+ if (ret)
+ return ret;
+
+ /*
+ * we always limit bottom-up allocation above the kernel,
+ * but top-down allocation doesn't have the limit, so
+ * retrying top-down allocation may succeed when bottom-up
+ * allocation failed.
+ *
+ * bottom-up allocation is expected to be fail very rarely,
+ * so we use WARN_ONCE() here to see the stack trace if
+ * fail happens.
+ */
+ WARN_ONCE(1, "memblock: Failed to allocate memory in bottom up "
+ "direction. Now try top down direction.\n");
+ }
return __memblock_find_range_rev(start, end, size, align, nid);
}
--
1.7.1
From: Tang Chen <[email protected]>
This patch creates a new function memory_map_top_down to
factor out of the top-down direct memory mapping pagetable
setup. This is also a preparation for the following patch,
which will introduce the bottom-up memory mapping. That said,
we will put the two ways of pagetable setup into separate
functions, and choose to use which way in init_mem_mapping,
which makes the code more clear.
Signed-off-by: Tang Chen <[email protected]>
Signed-off-by: Zhang Yanfei <[email protected]>
---
arch/x86/mm/init.c | 58 ++++++++++++++++++++++++++++++++++------------------
1 files changed, 38 insertions(+), 20 deletions(-)
diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c
index 04664cd..dbe57e5 100644
--- a/arch/x86/mm/init.c
+++ b/arch/x86/mm/init.c
@@ -401,27 +401,26 @@ static unsigned long __init init_range_memory_mapping(
/* (PUD_SHIFT-PMD_SHIFT)/2 */
#define STEP_SIZE_SHIFT 5
-void __init init_mem_mapping(void)
+
+/**
+ * memory_map_top_down - Map [map_start, map_end) top down
+ * @map_start: start address of the target memory range
+ * @map_end: end address of the target memory range
+ *
+ * This function will setup direct mapping for memory range
+ * [map_start, map_end) in top-down.
+ */
+static void __init memory_map_top_down(unsigned long map_start,
+ unsigned long map_end)
{
- unsigned long end, real_end, start, last_start;
+ unsigned long real_end, start, last_start;
unsigned long step_size;
unsigned long addr;
unsigned long mapped_ram_size = 0;
unsigned long new_mapped_ram_size;
- probe_page_size_mask();
-
-#ifdef CONFIG_X86_64
- end = max_pfn << PAGE_SHIFT;
-#else
- end = max_low_pfn << PAGE_SHIFT;
-#endif
-
- /* the ISA range is always mapped regardless of memory holes */
- init_memory_mapping(0, ISA_END_ADDRESS);
-
/* xen has big range in reserved near end of ram, skip it at first.*/
- addr = memblock_find_in_range(ISA_END_ADDRESS, end, PMD_SIZE, PMD_SIZE);
+ addr = memblock_find_in_range(map_start, map_end, PMD_SIZE, PMD_SIZE);
real_end = addr + PMD_SIZE;
/* step_size need to be small so pgt_buf from BRK could cover it */
@@ -436,13 +435,13 @@ void __init init_mem_mapping(void)
* end of RAM in [min_pfn_mapped, max_pfn_mapped) used as new pages
* for page table.
*/
- while (last_start > ISA_END_ADDRESS) {
+ while (last_start > map_start) {
if (last_start > step_size) {
start = round_down(last_start - 1, step_size);
- if (start < ISA_END_ADDRESS)
- start = ISA_END_ADDRESS;
+ if (start < map_start)
+ start = map_start;
} else
- start = ISA_END_ADDRESS;
+ start = map_start;
new_mapped_ram_size = init_range_memory_mapping(start,
last_start);
last_start = start;
@@ -453,8 +452,27 @@ void __init init_mem_mapping(void)
mapped_ram_size += new_mapped_ram_size;
}
- if (real_end < end)
- init_range_memory_mapping(real_end, end);
+ if (real_end < map_end)
+ init_range_memory_mapping(real_end, map_end);
+}
+
+void __init init_mem_mapping(void)
+{
+ unsigned long end;
+
+ probe_page_size_mask();
+
+#ifdef CONFIG_X86_64
+ end = max_pfn << PAGE_SHIFT;
+#else
+ end = max_low_pfn << PAGE_SHIFT;
+#endif
+
+ /* the ISA range is always mapped regardless of memory holes */
+ init_memory_mapping(0, ISA_END_ADDRESS);
+
+ /* setup direct mapping for range [ISA_END_ADDRESS, end) in top-down*/
+ memory_map_top_down(ISA_END_ADDRESS, end);
#ifdef CONFIG_X86_64
if (max_pfn > max_low_pfn) {
--
1.7.1
From: Tang Chen <[email protected]>
The Linux kernel cannot migrate pages used by the kernel. As a
result, kernel pages cannot be hot-removed. So we cannot allocate
hotpluggable memory for the kernel.
In a memory hotplug system, any numa node the kernel resides in
should be unhotpluggable. And for a modern server, each node could
have at least 16GB memory. So memory around the kernel image is
highly likely unhotpluggable.
ACPI SRAT (System Resource Affinity Table) contains the memory
hotplug info. But before SRAT is parsed, memblock has already
started to allocate memory for the kernel. So we need to prevent
memblock from doing this.
So direct memory mapping page tables setup is the case. init_mem_mapping()
is called before SRAT is parsed. To prevent page tables being allocated
within hotpluggable memory, we will use bottom-up direction to allocate
page tables from the end of kernel image to the higher memory.
Signed-off-by: Tang Chen <[email protected]>
Signed-off-by: Zhang Yanfei <[email protected]>
---
arch/x86/mm/init.c | 64 ++++++++++++++++++++++++++++++++++++++++++++++++++-
1 files changed, 62 insertions(+), 2 deletions(-)
diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c
index dbe57e5..d35363e 100644
--- a/arch/x86/mm/init.c
+++ b/arch/x86/mm/init.c
@@ -456,6 +456,48 @@ static void __init memory_map_top_down(unsigned long map_start,
init_range_memory_mapping(real_end, map_end);
}
+/**
+ * memory_map_bottom_up - Map [map_start, map_end) bottom up
+ * @map_start: start address of the target memory range
+ * @map_end: end address of the target memory range
+ *
+ * This function will setup direct mapping for memory range
+ * [map_start, map_end) in bottom-up.
+ */
+static void __init memory_map_bottom_up(unsigned long map_start,
+ unsigned long map_end)
+{
+ unsigned long next, new_mapped_ram_size, start;
+ unsigned long mapped_ram_size = 0;
+ /* step_size need to be small so pgt_buf from BRK could cover it */
+ unsigned long step_size = PMD_SIZE;
+
+ start = map_start;
+ min_pfn_mapped = start >> PAGE_SHIFT;
+
+ /*
+ * We start from the bottom (@map_start) and go to the top (@map_end).
+ * The memblock_find_in_range() gets us a block of RAM from the
+ * end of RAM in [min_pfn_mapped, max_pfn_mapped) used as new pages
+ * for page table.
+ */
+ while (start < map_end) {
+ if (map_end - start > step_size) {
+ next = round_up(start + 1, step_size);
+ if (next > map_end)
+ next = map_end;
+ } else
+ next = map_end;
+
+ new_mapped_ram_size = init_range_memory_mapping(start, next);
+ start = next;
+
+ if (new_mapped_ram_size > mapped_ram_size)
+ step_size <<= STEP_SIZE_SHIFT;
+ mapped_ram_size += new_mapped_ram_size;
+ }
+}
+
void __init init_mem_mapping(void)
{
unsigned long end;
@@ -471,8 +513,26 @@ void __init init_mem_mapping(void)
/* the ISA range is always mapped regardless of memory holes */
init_memory_mapping(0, ISA_END_ADDRESS);
- /* setup direct mapping for range [ISA_END_ADDRESS, end) in top-down*/
- memory_map_top_down(ISA_END_ADDRESS, end);
+ /*
+ * If the allocation is in bottom-up direction, we setup direct mapping
+ * in bottom-up, otherwise we setup direct mapping in top-down.
+ */
+ if (memblock_bottom_up()) {
+ unsigned long kernel_end;
+
+ kernel_end = __pa_symbol(_end);
+ /*
+ * we need two separate calls here. This is because we want to
+ * allocate page tables above the kernel. So we first map
+ * [kernel_end, end) to make memory above the kernel be mapped
+ * as soon as possible. And then use page tables allocated above
+ * the kernel to map [ISA_END_ADDRESS, kernel_end).
+ */
+ memory_map_bottom_up(kernel_end, end);
+ memory_map_bottom_up(ISA_END_ADDRESS, kernel_end);
+ } else {
+ memory_map_top_down(ISA_END_ADDRESS, end);
+ }
#ifdef CONFIG_X86_64
if (max_pfn > max_low_pfn) {
--
1.7.1
From: Tang Chen <[email protected]>
Memory reserved for crashkernel could be large. So we should not allocate
this memory bottom up from the end of kernel image.
When SRAT is parsed, we will be able to know whihc memory is hotpluggable,
and we can avoid allocating this memory for the kernel. So reorder
reserve_crashkernel() after SRAT is parsed.
Acked-by: Tejun Heo <[email protected]>
Signed-off-by: Tang Chen <[email protected]>
Reviewed-by: Zhang Yanfei <[email protected]>
---
arch/x86/kernel/setup.c | 8 ++++++--
1 files changed, 6 insertions(+), 2 deletions(-)
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index f0de629..36cfce3 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -1120,8 +1120,6 @@ void __init setup_arch(char **cmdline_p)
acpi_initrd_override((void *)initrd_start, initrd_end - initrd_start);
#endif
- reserve_crashkernel();
-
vsmp_init();
io_delay_init();
@@ -1136,6 +1134,12 @@ void __init setup_arch(char **cmdline_p)
initmem_init();
memblock_find_dma_reserve();
+ /*
+ * Reserve memory for crash kernel after SRAT is parsed so that it
+ * won't consume hotpluggable memory.
+ */
+ reserve_crashkernel();
+
#ifdef CONFIG_KVM_GUEST
kvmclock_init();
#endif
--
1.7.1
From: Tang Chen <[email protected]>
The hot-Pluggable field in SRAT specifies which memory is hotpluggable.
As we mentioned before, if hotpluggable memory is used by the kernel,
it cannot be hot-removed. So memory hotplug users may want to set all
hotpluggable memory in ZONE_MOVABLE so that the kernel won't use it.
Memory hotplug users may also set a node as movable node, which has
ZONE_MOVABLE only, so that the whole node can be hot-removed.
But the kernel cannot use memory in ZONE_MOVABLE. By doing this, the
kernel cannot use memory in movable nodes. This will cause NUMA
performance down. And other users may be unhappy.
So we need a way to allow users to enable and disable this functionality.
In this patch, we introduce movablenode boot option to allow users to
choose to not to consume hotpluggable memory at early boot time and
later we can set it as ZONE_MOVABLE.
To achieve this, the movablenode boot option will control the memblock
allocation direction. That said, after memblock is ready, before SRAT is
parsed, we should allocate memory near the kernel image as we explained
in the previous patches. So if movablenode boot option is set, the kernel
does the following:
1. After memblock is ready, make memblock allocate memory bottom up.
2. After SRAT is parsed, make memblock behave as default, allocate memory
top down.
Users can specify "movablenode" in kernel commandline to enable this
functionality. For those who don't use memory hotplug or who don't want
to lose their NUMA performance, just don't specify anything. The kernel
will work as before.
Suggested-by: Kamezawa Hiroyuki <[email protected]>
Signed-off-by: Tang Chen <[email protected]>
Signed-off-by: Zhang Yanfei <[email protected]>
---
Documentation/kernel-parameters.txt | 15 +++++++++++++++
arch/x86/kernel/setup.c | 7 +++++++
mm/memory_hotplug.c | 31 +++++++++++++++++++++++++++++++
3 files changed, 53 insertions(+), 0 deletions(-)
diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt
index 1a036cd..8c056c4 100644
--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
@@ -1769,6 +1769,21 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
that the amount of memory usable for all allocations
is not too small.
+ movablenode [KNL,X86] This parameter enables/disables the
+ kernel to arrange hotpluggable memory ranges recorded
+ in ACPI SRAT(System Resource Affinity Table) as
+ ZONE_MOVABLE. And these memory can be hot-removed when
+ the system is up.
+ By specifying this option, all the hotpluggable memory
+ will be in ZONE_MOVABLE, which the kernel cannot use.
+ This will cause NUMA performance down. For users who
+ care about NUMA performance, just don't use it.
+ If all the memory ranges in the system are hotpluggable,
+ then the ones used by the kernel at early time, such as
+ kernel code and data segments, initrd file and so on,
+ won't be set as ZONE_MOVABLE, and won't be hotpluggable.
+ Otherwise the kernel won't have enough memory to boot.
+
MTD_Partition= [MTD]
Format: <name>,<region-number>,<size>,<offset>
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 36cfce3..b8fefb7 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -1132,6 +1132,13 @@ void __init setup_arch(char **cmdline_p)
early_acpi_boot_init();
initmem_init();
+
+ /*
+ * When ACPI SRAT is parsed, which is done in initmem_init(),
+ * set memblock back to the top-down direction.
+ */
+ memblock_set_bottom_up(false);
+
memblock_find_dma_reserve();
/*
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index ed85fe3..dcd819a 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -31,6 +31,7 @@
#include <linux/firmware-map.h>
#include <linux/stop_machine.h>
#include <linux/hugetlb.h>
+#include <linux/memblock.h>
#include <asm/tlbflush.h>
@@ -1412,6 +1413,36 @@ static bool can_offline_normal(struct zone *zone, unsigned long nr_pages)
}
#endif /* CONFIG_MOVABLE_NODE */
+static int __init cmdline_parse_movablenode(char *p)
+{
+#ifdef CONFIG_MOVABLE_NODE
+ /*
+ * Memory used by the kernel cannot be hot-removed because Linux
+ * cannot migrate the kernel pages. When memory hotplug is
+ * enabled, we should prevent memblock from allocating memory
+ * for the kernel.
+ *
+ * ACPI SRAT records all hotpluggable memory ranges. But before
+ * SRAT is parsed, we don't know about it.
+ *
+ * The kernel image is loaded into memory at very early time. We
+ * cannot prevent this anyway. So on NUMA system, we set any
+ * node the kernel resides in as un-hotpluggable.
+ *
+ * Since on modern servers, one node could have double-digit
+ * gigabytes memory, we can assume the memory around the kernel
+ * image is also un-hotpluggable. So before SRAT is parsed, just
+ * allocate memory near the kernel image to try the best to keep
+ * the kernel away from hotpluggable memory.
+ */
+ memblock_set_bottom_up(true);
+#else
+ pr_warn("movablenode option not supported");
+#endif
+ return 0;
+}
+early_param("movablenode", cmdline_parse_movablenode);
+
/* check which state of node_states will be changed when offline memory */
static void node_states_check_changes_offline(unsigned long nr_pages,
struct zone *zone, struct memory_notify *arg)
--
1.7.1
Hello,
On Wed, Sep 25, 2013 at 02:27:48AM +0800, Zhang Yanfei wrote:
> +#ifdef CONFIG_MOVABLE_NODE
> +static inline void memblock_set_bottom_up(bool enable)
> +{
> + memblock.bottom_up = enable;
> +}
> +
> +static inline bool memblock_bottom_up(void)
> +{
> + return memblock.bottom_up;
> +}
Can you please explain what this is for here?
> + /*
> + * we always limit bottom-up allocation above the kernel,
> + * but top-down allocation doesn't have the limit, so
> + * retrying top-down allocation may succeed when bottom-up
> + * allocation failed.
> + *
> + * bottom-up allocation is expected to be fail very rarely,
> + * so we use WARN_ONCE() here to see the stack trace if
> + * fail happens.
> + */
> + WARN_ONCE(1, "memblock: Failed to allocate memory in bottom up "
> + "direction. Now try top down direction.\n");
> + }
You and I would know what was going on and what the consequence of the
failure may be but the above warning message is kinda useless to a
user / admin, right? It doesn't really say anything meaningful.
Thanks.
--
tejun
On Wed, Sep 25, 2013 at 02:29:06AM +0800, Zhang Yanfei wrote:
> +/**
> + * memory_map_top_down - Map [map_start, map_end) top down
> + * @map_start: start address of the target memory range
> + * @map_end: end address of the target memory range
> + *
> + * This function will setup direct mapping for memory range
> + * [map_start, map_end) in top-down.
Can you please put a bit more effort into the function description?
Other than that,
Acked-by: Tejun Heo <[email protected]>
Thanks.
--
tejun
Hello,
On Wed, Sep 25, 2013 at 02:30:51AM +0800, Zhang Yanfei wrote:
> +/**
> + * memory_map_bottom_up - Map [map_start, map_end) bottom up
> + * @map_start: start address of the target memory range
> + * @map_end: end address of the target memory range
> + *
> + * This function will setup direct mapping for memory range
> + * [map_start, map_end) in bottom-up.
Ditto about the comment.
> + */
> +static void __init memory_map_bottom_up(unsigned long map_start,
> + unsigned long map_end)
> +{
> + unsigned long next, new_mapped_ram_size, start;
> + unsigned long mapped_ram_size = 0;
> + /* step_size need to be small so pgt_buf from BRK could cover it */
> + unsigned long step_size = PMD_SIZE;
> +
> + start = map_start;
> + min_pfn_mapped = start >> PAGE_SHIFT;
> +
> + /*
> + * We start from the bottom (@map_start) and go to the top (@map_end).
> + * The memblock_find_in_range() gets us a block of RAM from the
> + * end of RAM in [min_pfn_mapped, max_pfn_mapped) used as new pages
> + * for page table.
> + */
> + while (start < map_end) {
> + if (map_end - start > step_size) {
> + next = round_up(start + 1, step_size);
> + if (next > map_end)
> + next = map_end;
> + } else
> + next = map_end;
> +
> + new_mapped_ram_size = init_range_memory_mapping(start, next);
> + start = next;
> +
> + if (new_mapped_ram_size > mapped_ram_size)
> + step_size <<= STEP_SIZE_SHIFT;
> + mapped_ram_size += new_mapped_ram_size;
> + }
> +}
As Yinghai pointed out in another thread, do we need to worry about
falling back to top-down?
Thanks.
--
tejun
On Wed, Sep 25, 2013 at 02:34:34AM +0800, Zhang Yanfei wrote:
> From: Tang Chen <[email protected]>
>
> Memory reserved for crashkernel could be large. So we should not allocate
> this memory bottom up from the end of kernel image.
>
> When SRAT is parsed, we will be able to know whihc memory is hotpluggable,
> and we can avoid allocating this memory for the kernel. So reorder
> reserve_crashkernel() after SRAT is parsed.
>
> Acked-by: Tejun Heo <[email protected]>
So, I was hoping to hear from you on how you tested it when I wrote
the previous comment - the "provided..." part.
--
tejun
On Wed, Sep 25, 2013 at 02:35:14AM +0800, Zhang Yanfei wrote:
> From: Tang Chen <[email protected]>
>
> The hot-Pluggable field in SRAT specifies which memory is hotpluggable.
> As we mentioned before, if hotpluggable memory is used by the kernel,
> it cannot be hot-removed. So memory hotplug users may want to set all
> hotpluggable memory in ZONE_MOVABLE so that the kernel won't use it.
>
> Memory hotplug users may also set a node as movable node, which has
> ZONE_MOVABLE only, so that the whole node can be hot-removed.
>
> But the kernel cannot use memory in ZONE_MOVABLE. By doing this, the
> kernel cannot use memory in movable nodes. This will cause NUMA
> performance down. And other users may be unhappy.
>
> So we need a way to allow users to enable and disable this functionality.
> In this patch, we introduce movablenode boot option to allow users to
> choose to not to consume hotpluggable memory at early boot time and
> later we can set it as ZONE_MOVABLE.
>
> To achieve this, the movablenode boot option will control the memblock
> allocation direction. That said, after memblock is ready, before SRAT is
> parsed, we should allocate memory near the kernel image as we explained
> in the previous patches. So if movablenode boot option is set, the kernel
> does the following:
>
> 1. After memblock is ready, make memblock allocate memory bottom up.
> 2. After SRAT is parsed, make memblock behave as default, allocate memory
> top down.
>
> Users can specify "movablenode" in kernel commandline to enable this
> functionality. For those who don't use memory hotplug or who don't want
> to lose their NUMA performance, just don't specify anything. The kernel
> will work as before.
>
> Suggested-by: Kamezawa Hiroyuki <[email protected]>
> Signed-off-by: Tang Chen <[email protected]>
> Signed-off-by: Zhang Yanfei <[email protected]>
I hope the param description and comment were better. Not necessarily
longer, but clearer, so it'd be great if you can polish them a bit
more. Other than that,
Acked-by: Tejun Heo <[email protected]>
Thanks.
--
tejun
Hello tejun,
Thanks for your quick comments first:)
On 09/26/2013 10:45 PM, Tejun Heo wrote:
> Hello,
>
> On Wed, Sep 25, 2013 at 02:27:48AM +0800, Zhang Yanfei wrote:
>> +#ifdef CONFIG_MOVABLE_NODE
>> +static inline void memblock_set_bottom_up(bool enable)
>> +{
>> + memblock.bottom_up = enable;
>> +}
>> +
>> +static inline bool memblock_bottom_up(void)
>> +{
>> + return memblock.bottom_up;
>> +}
>
> Can you please explain what this is for here?
OK, will do.
>
>> + /*
>> + * we always limit bottom-up allocation above the kernel,
>> + * but top-down allocation doesn't have the limit, so
>> + * retrying top-down allocation may succeed when bottom-up
>> + * allocation failed.
>> + *
>> + * bottom-up allocation is expected to be fail very rarely,
>> + * so we use WARN_ONCE() here to see the stack trace if
>> + * fail happens.
>> + */
>> + WARN_ONCE(1, "memblock: Failed to allocate memory in bottom up "
>> + "direction. Now try top down direction.\n");
>> + }
>
> You and I would know what was going on and what the consequence of the
> failure may be but the above warning message is kinda useless to a
> user / admin, right? It doesn't really say anything meaningful.
>
Hmmmm.. May be something like this:
WARN_ONCE(1, "Failed to allocated memory above the kernel in bottom-up,"
"so try to allocate memory below the kernel.");
Thanks
--
Thanks.
Zhang Yanfei
On 09/26/2013 10:46 PM, Tejun Heo wrote:
> On Wed, Sep 25, 2013 at 02:29:06AM +0800, Zhang Yanfei wrote:
>> +/**
>> + * memory_map_top_down - Map [map_start, map_end) top down
>> + * @map_start: start address of the target memory range
>> + * @map_end: end address of the target memory range
>> + *
>> + * This function will setup direct mapping for memory range
>> + * [map_start, map_end) in top-down.
>
> Can you please put a bit more effort into the function description?
Sorry.... I will try to make a more detailed description.
>
> Other than that,
>
> Acked-by: Tejun Heo <[email protected]>
Thanks.
--
Thanks.
Zhang Yanfei
Hello tejun,
On 09/26/2013 10:48 PM, Tejun Heo wrote:
> Hello,
>
> On Wed, Sep 25, 2013 at 02:30:51AM +0800, Zhang Yanfei wrote:
>> +/**
>> + * memory_map_bottom_up - Map [map_start, map_end) bottom up
>> + * @map_start: start address of the target memory range
>> + * @map_end: end address of the target memory range
>> + *
>> + * This function will setup direct mapping for memory range
>> + * [map_start, map_end) in bottom-up.
>
> Ditto about the comment.
OK, will do.
>
>> + */
>> +static void __init memory_map_bottom_up(unsigned long map_start,
>> + unsigned long map_end)
>> +{
>> + unsigned long next, new_mapped_ram_size, start;
>> + unsigned long mapped_ram_size = 0;
>> + /* step_size need to be small so pgt_buf from BRK could cover it */
>> + unsigned long step_size = PMD_SIZE;
>> +
>> + start = map_start;
>> + min_pfn_mapped = start >> PAGE_SHIFT;
>> +
>> + /*
>> + * We start from the bottom (@map_start) and go to the top (@map_end).
>> + * The memblock_find_in_range() gets us a block of RAM from the
>> + * end of RAM in [min_pfn_mapped, max_pfn_mapped) used as new pages
>> + * for page table.
>> + */
>> + while (start < map_end) {
>> + if (map_end - start > step_size) {
>> + next = round_up(start + 1, step_size);
>> + if (next > map_end)
>> + next = map_end;
>> + } else
>> + next = map_end;
>> +
>> + new_mapped_ram_size = init_range_memory_mapping(start, next);
>> + start = next;
>> +
>> + if (new_mapped_ram_size > mapped_ram_size)
>> + step_size <<= STEP_SIZE_SHIFT;
>> + mapped_ram_size += new_mapped_ram_size;
>> + }
>> +}
>
> As Yinghai pointed out in another thread, do we need to worry about
> falling back to top-down?
I've explained to him. Nop, we don't need to worry about that. Because even
the min_pfn_mapped becomes ISA_END_ADDRESS in the second call below, we won't
allocate memory below the kernel because we have limited the allocation above
the kernel.
Thanks.
--
Thanks.
Zhang Yanfei
On 09/26/2013 10:49 PM, Tejun Heo wrote:
> On Wed, Sep 25, 2013 at 02:34:34AM +0800, Zhang Yanfei wrote:
>> From: Tang Chen <[email protected]>
>>
>> Memory reserved for crashkernel could be large. So we should not allocate
>> this memory bottom up from the end of kernel image.
>>
>> When SRAT is parsed, we will be able to know whihc memory is hotpluggable,
>> and we can avoid allocating this memory for the kernel. So reorder
>> reserve_crashkernel() after SRAT is parsed.
>>
>> Acked-by: Tejun Heo <[email protected]>
>
> So, I was hoping to hear from you on how you tested it when I wrote
> the previous comment - the "provided..." part.
>
This function is actually used for kexec/kdump. So After applying
this patch, booting the kernel, this reservation is successful and
the kdump service starts successfully.
Thanks.
--
Thanks.
Zhang Yanfei
On Thu, Sep 26, 2013 at 11:43:02PM +0800, Zhang Yanfei wrote:
> > As Yinghai pointed out in another thread, do we need to worry about
> > falling back to top-down?
>
> I've explained to him. Nop, we don't need to worry about that. Because even
> the min_pfn_mapped becomes ISA_END_ADDRESS in the second call below, we won't
> allocate memory below the kernel because we have limited the allocation above
> the kernel.
Maybe I misunderstood but wasn't he worrying about there not being
enough space above kernel? In that case, it'd automatically fall back
to top-down allocation anyway, right?
Thanks.
--
tejun
On Thu, Sep 26, 2013 at 11:37:34PM +0800, Zhang Yanfei wrote:
> >> + WARN_ONCE(1, "memblock: Failed to allocate memory in bottom up "
> >> + "direction. Now try top down direction.\n");
> >> + }
> >
> > You and I would know what was going on and what the consequence of the
> > failure may be but the above warning message is kinda useless to a
> > user / admin, right? It doesn't really say anything meaningful.
> >
>
> Hmmmm.. May be something like this:
>
> WARN_ONCE(1, "Failed to allocated memory above the kernel in bottom-up,"
> "so try to allocate memory below the kernel.");
How about something like "memblock: bottom-up allocation failed,
memory hotunplug may be affected\n".
Thanks.
--
tejun
On 09/26/2013 11:50 PM, Tejun Heo wrote:
> On Thu, Sep 26, 2013 at 11:37:34PM +0800, Zhang Yanfei wrote:
>>>> + WARN_ONCE(1, "memblock: Failed to allocate memory in bottom up "
>>>> + "direction. Now try top down direction.\n");
>>>> + }
>>>
>>> You and I would know what was going on and what the consequence of the
>>> failure may be but the above warning message is kinda useless to a
>>> user / admin, right? It doesn't really say anything meaningful.
>>>
>>
>> Hmmmm.. May be something like this:
>>
>> WARN_ONCE(1, "Failed to allocated memory above the kernel in bottom-up,"
>> "so try to allocate memory below the kernel.");
>
> How about something like "memblock: bottom-up allocation failed,
> memory hotunplug may be affected\n".
>
Ok, I understand what you want. Explicitly telling the user the functionality
may be invalid due to some failure. Yeah, this is really meaningful, i will
take yours, thanks.
--
Thanks.
Zhang Yanfei
On 09/26/2013 11:48 PM, Tejun Heo wrote:
> On Thu, Sep 26, 2013 at 11:43:02PM +0800, Zhang Yanfei wrote:
>>> As Yinghai pointed out in another thread, do we need to worry about
>>> falling back to top-down?
>>
>> I've explained to him. Nop, we don't need to worry about that. Because even
>> the min_pfn_mapped becomes ISA_END_ADDRESS in the second call below, we won't
>> allocate memory below the kernel because we have limited the allocation above
>> the kernel.
>
> Maybe I misunderstood but wasn't he worrying about there not being
> enough space above kernel? In that case, it'd automatically fall back
> to top-down allocation anyway, right?
Ah, I see. You are saying another issue. He is worrying that if we use
kexec to load the kernel high, say we have 16GB, we put the kernel in
15.99GB (just an example), so we only have less than 100MB above the kernel.
But as I've explained to him, in almost all the cases, if we want our
memory hotplug work, we don't do that. And yeah, assume we have this
problem, it'd fall back to top down and that return backs to patch 2,
we will trigger the WARN_ONCE, and the admin will know what has happened.
Thanks.
--
Thanks.
Zhang Yanfei
On Fri, Sep 27, 2013 at 12:03:01AM +0800, Zhang Yanfei wrote:
> Ah, I see. You are saying another issue. He is worrying that if we use
> kexec to load the kernel high, say we have 16GB, we put the kernel in
> 15.99GB (just an example), so we only have less than 100MB above the kernel.
>
> But as I've explained to him, in almost all the cases, if we want our
> memory hotplug work, we don't do that. And yeah, assume we have this
> problem, it'd fall back to top down and that return backs to patch 2,
> we will trigger the WARN_ONCE, and the admin will know what has happened.
Alright,
Acked-by: Tejun Heo <[email protected]>
Thanks.
--
tejun
On 09/26/2013 10:53 PM, Tejun Heo wrote:
> On Wed, Sep 25, 2013 at 02:35:14AM +0800, Zhang Yanfei wrote:
>> From: Tang Chen <[email protected]>
>>
>> The hot-Pluggable field in SRAT specifies which memory is hotpluggable.
>> As we mentioned before, if hotpluggable memory is used by the kernel,
>> it cannot be hot-removed. So memory hotplug users may want to set all
>> hotpluggable memory in ZONE_MOVABLE so that the kernel won't use it.
>>
>> Memory hotplug users may also set a node as movable node, which has
>> ZONE_MOVABLE only, so that the whole node can be hot-removed.
>>
>> But the kernel cannot use memory in ZONE_MOVABLE. By doing this, the
>> kernel cannot use memory in movable nodes. This will cause NUMA
>> performance down. And other users may be unhappy.
>>
>> So we need a way to allow users to enable and disable this functionality.
>> In this patch, we introduce movablenode boot option to allow users to
>> choose to not to consume hotpluggable memory at early boot time and
>> later we can set it as ZONE_MOVABLE.
>>
>> To achieve this, the movablenode boot option will control the memblock
>> allocation direction. That said, after memblock is ready, before SRAT is
>> parsed, we should allocate memory near the kernel image as we explained
>> in the previous patches. So if movablenode boot option is set, the kernel
>> does the following:
>>
>> 1. After memblock is ready, make memblock allocate memory bottom up.
>> 2. After SRAT is parsed, make memblock behave as default, allocate memory
>> top down.
>>
>> Users can specify "movablenode" in kernel commandline to enable this
>> functionality. For those who don't use memory hotplug or who don't want
>> to lose their NUMA performance, just don't specify anything. The kernel
>> will work as before.
>>
>> Suggested-by: Kamezawa Hiroyuki <[email protected]>
>> Signed-off-by: Tang Chen <[email protected]>
>> Signed-off-by: Zhang Yanfei <[email protected]>
>
> I hope the param description and comment were better. Not necessarily
> longer, but clearer, so it'd be great if you can polish them a bit
OK. Trying below:
movablenode [KNL,X86] This option enables the kernel to arrange
hotpluggable memory into ZONE_MOVABLE zone. If memory
in a node is all hotpluggable, the option may make
the whole node has only one ZONE_MOVABLE zone, so that
the whole node can be hot-removed after system is up.
Note that this option may cause NUMA performance down.
As for the comment in cmdline_parse_movablenode():
/*
* ACPI SRAT records all hotpluggable memory ranges. But before
* SRAT is parsed, we don't know about it. So by specifying this
* option, we will use the bottom-up mode to try allocating memory
* near the kernel image before SRAT is parsed.
*
* Bottom-up mode prevents memblock allocating hotpluggable memory
* for the kernel so that the kernel will arrange hotpluggable
* memory into ZONE_MOVABLE zone when possible.
*/
Thanks.
> more. Other than that,
>
> Acked-by: Tejun Heo <[email protected]>
>
> Thanks.
>
--
Thanks.
Zhang Yanfei
On 09/26/2013 11:37 PM, Zhang Yanfei wrote:
> Hello tejun,
>
> Thanks for your quick comments first:)
>
> On 09/26/2013 10:45 PM, Tejun Heo wrote:
>> Hello,
>>
>> On Wed, Sep 25, 2013 at 02:27:48AM +0800, Zhang Yanfei wrote:
>>> +#ifdef CONFIG_MOVABLE_NODE
>>> +static inline void memblock_set_bottom_up(bool enable)
>>> +{
>>> + memblock.bottom_up = enable;
>>> +}
>>> +
>>> +static inline bool memblock_bottom_up(void)
>>> +{
>>> + return memblock.bottom_up;
>>> +}
>>
>> Can you please explain what this is for here?
>
> OK, will do.
I write the function description here so you could give your
comments still in this version.
/*
* Set the allocation direction to bottom-up or top-down.
*/
static inline void memblock_set_bottom_up(bool enable)
{
memblock.bottom_up = enable;
}
/*
* Check if the allocation direction is bottom-up or not.
* if this is true, that said, the boot option "movablenode"
* has been specified, and memblock will allocate memory
* just near the kernel image.
*/
static inline bool memblock_bottom_up(void)
{
return memblock.bottom_up;
}
Thanks.
--
Thanks.
Zhang Yanfei
Hello tejun,
On 09/26/2013 11:39 PM, Zhang Yanfei wrote:
> On 09/26/2013 10:46 PM, Tejun Heo wrote:
>> On Wed, Sep 25, 2013 at 02:29:06AM +0800, Zhang Yanfei wrote:
>>> +/**
>>> + * memory_map_top_down - Map [map_start, map_end) top down
>>> + * @map_start: start address of the target memory range
>>> + * @map_end: end address of the target memory range
>>> + *
>>> + * This function will setup direct mapping for memory range
>>> + * [map_start, map_end) in top-down.
>>
>> Can you please put a bit more effort into the function description?
>
> Sorry.... I will try to make a more detailed description.
Trying below:
/**
* memory_map_top_down - Map [map_start, map_end) top down
* @map_start: start address of the target memory range
* @map_end: end address of the target memory range
*
* This function will setup direct mapping for memory range
* [map_start, map_end) in top-down. That said, the page tables
* will be allocated at the end of the memory, and we map the
* memory top-down.
*/
static void __init memory_map_top_down(unsigned long map_start,
unsigned long map_end)
{
Thanks.
>
>>
>> Other than that,
>>
>> Acked-by: Tejun Heo <[email protected]>
>
> Thanks.
>
--
Thanks.
Zhang Yanfei
On 09/26/2013 10:48 PM, Tejun Heo wrote:
> Hello,
>
> On Wed, Sep 25, 2013 at 02:30:51AM +0800, Zhang Yanfei wrote:
>> +/**
>> + * memory_map_bottom_up - Map [map_start, map_end) bottom up
>> + * @map_start: start address of the target memory range
>> + * @map_end: end address of the target memory range
>> + *
>> + * This function will setup direct mapping for memory range
>> + * [map_start, map_end) in bottom-up.
>
> Ditto about the comment.
Trying below:
/**
* memory_map_bottom_up - Map [map_start, map_end) bottom up
* @map_start: start address of the target memory range
* @map_end: end address of the target memory range
*
* This function will setup direct mapping for memory range
* [map_start, map_end) in bottom-up. Since we have limited the
* bottom-up allocation above the kernel, the page tables will
* be allocated just above the kernel and we map the memory
* in [map_start, map_end) in bottom-up.
*/
static void __init memory_map_bottom_up(unsigned long map_start,
unsigned long map_end)
{
Thanks.
--
Thanks.
Zhang Yanfei
On Wed, 25 Sep 2013 02:30:51 +0800 Zhang Yanfei <[email protected]> wrote:
> From: Tang Chen <[email protected]>
>
> The Linux kernel cannot migrate pages used by the kernel. As a
> result, kernel pages cannot be hot-removed. So we cannot allocate
> hotpluggable memory for the kernel.
>
> In a memory hotplug system, any numa node the kernel resides in
> should be unhotpluggable. And for a modern server, each node could
> have at least 16GB memory. So memory around the kernel image is
> highly likely unhotpluggable.
>
> ACPI SRAT (System Resource Affinity Table) contains the memory
> hotplug info. But before SRAT is parsed, memblock has already
> started to allocate memory for the kernel. So we need to prevent
> memblock from doing this.
>
> So direct memory mapping page tables setup is the case. init_mem_mapping()
> is called before SRAT is parsed. To prevent page tables being allocated
> within hotpluggable memory, we will use bottom-up direction to allocate
> page tables from the end of kernel image to the higher memory.
>
> ...
>
> + kernel_end = __pa_symbol(_end);
__pa_symbol() is implemented only on mips and x86.
I stole the mips implementation like this:
--- a/mm/memblock.c~a
+++ a/mm/memblock.c
@@ -187,8 +187,11 @@ phys_addr_t __init_memblock memblock_fin
/* avoid allocating the first page */
start = max_t(phys_addr_t, start, PAGE_SIZE);
end = max(start, end);
+#ifdef CONFIG_X86
kernel_end = __pa_symbol(_end);
-
+#else
+ kernel_end = __pa(RELOC_HIDE((unsigned long)(_end), 0));
+#endif
/*
* try bottom-up allocation only when bottom-up mode
* is set and @end is above the kernel image.
just so I can get a -mm release out the door.
Can we put this in a common header somewhere?
Andrew Morton <[email protected]> wrote:
>On Wed, 25 Sep 2013 02:30:51 +0800 Zhang Yanfei
><[email protected]> wrote:
>
>> From: Tang Chen <[email protected]>
>>
>> The Linux kernel cannot migrate pages used by the kernel. As a
>> result, kernel pages cannot be hot-removed. So we cannot allocate
>> hotpluggable memory for the kernel.
>>
>> In a memory hotplug system, any numa node the kernel resides in
>> should be unhotpluggable. And for a modern server, each node could
>> have at least 16GB memory. So memory around the kernel image is
>> highly likely unhotpluggable.
>>
>> ACPI SRAT (System Resource Affinity Table) contains the memory
>> hotplug info. But before SRAT is parsed, memblock has already
>> started to allocate memory for the kernel. So we need to prevent
>> memblock from doing this.
>>
>> So direct memory mapping page tables setup is the case.
>init_mem_mapping()
>> is called before SRAT is parsed. To prevent page tables being
>allocated
>> within hotpluggable memory, we will use bottom-up direction to
>allocate
>> page tables from the end of kernel image to the higher memory.
>>
>> ...
>>
>> + kernel_end = __pa_symbol(_end);
>
>__pa_symbol() is implemented only on mips and x86.
>
>I stole the mips implementation like this:
>
>--- a/mm/memblock.c~a
>+++ a/mm/memblock.c
>@@ -187,8 +187,11 @@ phys_addr_t __init_memblock memblock_fin
> /* avoid allocating the first page */
> start = max_t(phys_addr_t, start, PAGE_SIZE);
> end = max(start, end);
>+#ifdef CONFIG_X86
> kernel_end = __pa_symbol(_end);
>-
>+#else
>+ kernel_end = __pa(RELOC_HIDE((unsigned long)(_end), 0));
>+#endif
> /*
> * try bottom-up allocation only when bottom-up mode
> * is set and @end is above the kernel image.
>
>just so I can get a -mm release out the door.
--
Sent from my mobile phone. Please pardon brevity and lack of formatting.
On Wed, 2013-09-25 at 02:25 +0800, Zhang Yanfei wrote:
> From: Tang Chen <[email protected]>
>
> This patch creates a new function __memblock_find_range_rev
> to factor out of top-down allocation from memblock_find_in_range_node.
> This is a preparation because we will introduce a new bottom-up
> allocation mode in the following patch.
>
> Acked-by: Tejun Heo <[email protected]>
> Signed-off-by: Tang Chen <[email protected]>
> Signed-off-by: Zhang Yanfei <[email protected]>
Acked-by: Toshi Kani <[email protected]>
A minor comment below...
> ---
> mm/memblock.c | 47 ++++++++++++++++++++++++++++++++++-------------
> 1 files changed, 34 insertions(+), 13 deletions(-)
>
> diff --git a/mm/memblock.c b/mm/memblock.c
> index 0ac412a..3d80c74 100644
> --- a/mm/memblock.c
> +++ b/mm/memblock.c
> @@ -83,33 +83,25 @@ static long __init_memblock memblock_overlaps_region(struct memblock_type *type,
> }
>
> /**
> - * memblock_find_in_range_node - find free area in given range and node
> + * __memblock_find_range_rev - find free area utility, in reverse order
> * @start: start of candidate range
> * @end: end of candidate range, can be %MEMBLOCK_ALLOC_{ANYWHERE|ACCESSIBLE}
> * @size: size of free area to find
> * @align: alignment of free area to find
> * @nid: nid of the free area to find, %MAX_NUMNODES for any node
> *
> - * Find @size free area aligned to @align in the specified range and node.
> + * Utility called from memblock_find_in_range_node(), find free area top-down.
> *
> * RETURNS:
> * Found address on success, %0 on failure.
> */
> -phys_addr_t __init_memblock memblock_find_in_range_node(phys_addr_t start,
> - phys_addr_t end, phys_addr_t size,
> - phys_addr_t align, int nid)
> +static phys_addr_t __init_memblock
> +__memblock_find_range_rev(phys_addr_t start, phys_addr_t end,
Since we are now using the terms "top down" and "bottom up"
consistently, how about name this function as
__memblock_find_range_top_down()?
Thanks,
-Toshi
On Wed, 2013-09-25 at 02:27 +0800, Zhang Yanfei wrote:
> From: Tang Chen <[email protected]>
>
> The Linux kernel cannot migrate pages used by the kernel. As a result, kernel
> pages cannot be hot-removed. So we cannot allocate hotpluggable memory for
> the kernel.
>
> ACPI SRAT (System Resource Affinity Table) contains the memory hotplug info.
> But before SRAT is parsed, memblock has already started to allocate memory
> for the kernel. So we need to prevent memblock from doing this.
>
> In a memory hotplug system, any numa node the kernel resides in should
> be unhotpluggable. And for a modern server, each node could have at least
> 16GB memory. So memory around the kernel image is highly likely unhotpluggable.
>
> So the basic idea is: Allocate memory from the end of the kernel image and
> to the higher memory. Since memory allocation before SRAT is parsed won't
> be too much, it could highly likely be in the same node with kernel image.
>
> The current memblock can only allocate memory top-down. So this patch introduces
> a new bottom-up allocation mode to allocate memory bottom-up. And later
> when we use this allocation direction to allocate memory, we will limit
> the start address above the kernel.
>
> Signed-off-by: Tang Chen <[email protected]>
> Signed-off-by: Zhang Yanfei <[email protected]>
:
> /**
> + * __memblock_find_range - find free area utility
> + * @start: start of candidate range
> + * @end: end of candidate range, can be %MEMBLOCK_ALLOC_{ANYWHERE|ACCESSIBLE}
> + * @size: size of free area to find
> + * @align: alignment of free area to find
> + * @nid: nid of the free area to find, %MAX_NUMNODES for any node
> + *
> + * Utility called from memblock_find_in_range_node(), find free area bottom-up.
> + *
> + * RETURNS:
> + * Found address on success, 0 on failure.
> + */
> +static phys_addr_t __init_memblock
> +__memblock_find_range(phys_addr_t start, phys_addr_t end, phys_addr_t size,
Similarly, how about name this function as
__memblock_find_range_bottom_up()?
> + phys_addr_t align, int nid)
> +{
> + phys_addr_t this_start, this_end, cand;
> + u64 i;
> +
> + for_each_free_mem_range(i, nid, &this_start, &this_end, NULL) {
> + this_start = clamp(this_start, start, end);
> + this_end = clamp(this_end, start, end);
> +
> + cand = round_up(this_start, align);
> + if (cand < this_end && this_end - cand >= size)
> + return cand;
> + }
> +
> + return 0;
> +}
> +
> +/**
> * __memblock_find_range_rev - find free area utility, in reverse order
> * @start: start of candidate range
> * @end: end of candidate range, can be %MEMBLOCK_ALLOC_{ANYWHERE|ACCESSIBLE}
> @@ -93,7 +128,7 @@ static long __init_memblock memblock_overlaps_region(struct memblock_type *type,
> * Utility called from memblock_find_in_range_node(), find free area top-down.
> *
> * RETURNS:
> - * Found address on success, %0 on failure.
> + * Found address on success, 0 on failure.
> */
> static phys_addr_t __init_memblock
> __memblock_find_range_rev(phys_addr_t start, phys_addr_t end,
> @@ -127,13 +162,24 @@ __memblock_find_range_rev(phys_addr_t start, phys_addr_t end,
> *
> * Find @size free area aligned to @align in the specified range and node.
> *
> + * When allocation direction is bottom-up, the @start should be greater
> + * than the end of the kernel image. Otherwise, it will be trimmed. The
> + * reason is that we want the bottom-up allocation just near the kernel
> + * image so it is highly likely that the allocated memory and the kernel
> + * will reside in the same node.
> + *
> + * If bottom-up allocation failed, will try to allocate memory top-down.
> + *
> * RETURNS:
> - * Found address on success, %0 on failure.
> + * Found address on success, 0 on failure.
> */
> phys_addr_t __init_memblock memblock_find_in_range_node(phys_addr_t start,
> phys_addr_t end, phys_addr_t size,
> phys_addr_t align, int nid)
> {
> + int ret;
> + phys_addr_t kernel_end;
> +
> /* pump up @end */
> if (end == MEMBLOCK_ALLOC_ACCESSIBLE)
> end = memblock.current_limit;
> @@ -141,6 +187,37 @@ phys_addr_t __init_memblock memblock_find_in_range_node(phys_addr_t start,
> /* avoid allocating the first page */
> start = max_t(phys_addr_t, start, PAGE_SIZE);
> end = max(start, end);
> + kernel_end = __pa_symbol(_end);
Please address the issue in __pa_symbol() that Andrew pointed out.
Thanks,
-Toshi
On Wed, 2013-09-25 at 02:29 +0800, Zhang Yanfei wrote:
> From: Tang Chen <[email protected]>
>
> This patch creates a new function memory_map_top_down to
> factor out of the top-down direct memory mapping pagetable
> setup. This is also a preparation for the following patch,
> which will introduce the bottom-up memory mapping. That said,
> we will put the two ways of pagetable setup into separate
> functions, and choose to use which way in init_mem_mapping,
> which makes the code more clear.
>
> Signed-off-by: Tang Chen <[email protected]>
> Signed-off-by: Zhang Yanfei <[email protected]>
Acked-by: Toshi Kani <[email protected]>
Thanks,
-Toshi
On Wed, 2013-09-25 at 02:34 +0800, Zhang Yanfei wrote:
> From: Tang Chen <[email protected]>
>
> Memory reserved for crashkernel could be large. So we should not allocate
> this memory bottom up from the end of kernel image.
>
> When SRAT is parsed, we will be able to know whihc memory is hotpluggable,
> and we can avoid allocating this memory for the kernel. So reorder
> reserve_crashkernel() after SRAT is parsed.
>
> Acked-by: Tejun Heo <[email protected]>
> Signed-off-by: Tang Chen <[email protected]>
> Reviewed-by: Zhang Yanfei <[email protected]>
> ---
> arch/x86/kernel/setup.c | 8 ++++++--
> 1 files changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> index f0de629..36cfce3 100644
> --- a/arch/x86/kernel/setup.c
> +++ b/arch/x86/kernel/setup.c
> @@ -1120,8 +1120,6 @@ void __init setup_arch(char **cmdline_p)
> acpi_initrd_override((void *)initrd_start, initrd_end - initrd_start);
> #endif
>
> - reserve_crashkernel();
> -
> vsmp_init();
>
> io_delay_init();
> @@ -1136,6 +1134,12 @@ void __init setup_arch(char **cmdline_p)
> initmem_init();
> memblock_find_dma_reserve();
>
> + /*
> + * Reserve memory for crash kernel after SRAT is parsed so that it
> + * won't consume hotpluggable memory.
> + */
> + reserve_crashkernel();
Out of curiosity, is there any particular reason why it is moved after
memblock_find_dma_reserve(), not initmem_init()?
Thanks,
-Toshi