2014-01-13 21:56:34

by Prarit Bhargava

[permalink] [raw]
Subject: [PATCH] x86, acpi memory hotplug, add parameter to disable memory hotplug

When booting a kexec/kdump kernel on a system that has specific memory hotplug
regions the boot will fail with warnings like:

[ 2.939467] swapper/0: page allocation failure: order:9, mode:0x84d0
[ 2.946564] CPU: 0 PID: 1 Comm: swapper/0 Not tainted
3.10.0-65.el7.x86_64 #1
[ 2.954532] Hardware name: QCI QSSC-S4R/QSSC-S4R, BIOS
QSSC-S4R.QCI.01.00.S013.032920111005 03/29/2011
[ 2.964926] 0000000000000000 ffff8800341bd8c8 ffffffff815bcc67
ffff8800341bd950
[ 2.973224] ffffffff8113b1a0 ffff880036339b00 0000000000000009
00000000000084d0
[ 2.981523] ffff8800341bd950 ffffffff815b87ee 0000000000000000
0000000000000200
[ 2.989821] Call Trace:
[ 2.992560] [<ffffffff815bcc67>] dump_stack+0x19/0x1b
[ 2.998300] [<ffffffff8113b1a0>] warn_alloc_failed+0xf0/0x160
[ 3.004817] [<ffffffff815b87ee>] ?
__alloc_pages_direct_compact+0xac/0x196
[ 3.012594] [<ffffffff8113f14f>] __alloc_pages_nodemask+0x7ff/0xa00
[ 3.019692] [<ffffffff815b417c>] vmemmap_alloc_block+0x62/0xba
[ 3.026303] [<ffffffff815b41e9>] vmemmap_alloc_block_buf+0x15/0x3b
[ 3.033302] [<ffffffff815b1ff6>] vmemmap_populate+0xb4/0x21b
[ 3.039718] [<ffffffff815b461d>] sparse_mem_map_populate+0x27/0x35
[ 3.046717] [<ffffffff815b400f>] sparse_add_one_section+0x7a/0x185
[ 3.053720] [<ffffffff815a1e9f>] __add_pages+0xaf/0x240
[ 3.059656] [<ffffffff81047359>] arch_add_memory+0x59/0xd0
[ 3.065877] [<ffffffff815a21d9>] add_memory+0xb9/0x1b0
[ 3.071713] [<ffffffff81333b9c>] acpi_memory_device_add+0x18d/0x26d
[ 3.078813] [<ffffffff81309a01>] acpi_bus_device_attach+0x7d/0xcd
[ 3.085719] [<ffffffff8132379d>] acpi_ns_walk_namespace+0xc8/0x17f
[ 3.092716] [<ffffffff81309984>] ? acpi_bus_type_and_status+0x90/0x90
[ 3.100004] [<ffffffff81309984>] ? acpi_bus_type_and_status+0x90/0x90
[ 3.107293] [<ffffffff81323c8c>] acpi_walk_namespace+0x95/0xc5
[ 3.113904] [<ffffffff8130a6d6>] acpi_bus_scan+0x8b/0x9d
[ 3.119933] [<ffffffff81a2019a>] acpi_scan_init+0x63/0x160
[ 3.126153] [<ffffffff81a1ffb5>] acpi_init+0x25d/0x2a6
[ 3.131987] [<ffffffff81a1fd58>] ? acpi_sleep_proc_init+0x2a/0x2a
[ 3.138889] [<ffffffff810020e2>] do_one_initcall+0xe2/0x190
[ 3.145210] [<ffffffff819e20c4>] kernel_init_freeable+0x17c/0x207
[ 3.152111] [<ffffffff819e18d0>] ? do_early_param+0x88/0x88
[ 3.158430] [<ffffffff8159fea0>] ? rest_init+0x80/0x80
[ 3.164264] [<ffffffff8159feae>] kernel_init+0xe/0x180
[ 3.170097] [<ffffffff815cca2c>] ret_from_fork+0x7c/0xb0
[ 3.176123] [<ffffffff8159fea0>] ? rest_init+0x80/0x80
[ 3.181956] Mem-Info:
[ 3.184490] Node 0 DMA per-cpu:
[ 3.188007] CPU 0: hi: 0, btch: 1 usd: 0
[ 3.193353] Node 0 DMA32 per-cpu:
[ 3.197060] CPU 0: hi: 42, btch: 7 usd: 0
[ 3.202410] active_anon:0 inactive_anon:0 isolated_anon:0
[ 3.202410] active_file:0 inactive_file:0 isolated_file:0
[ 3.202410] unevictable:0 dirty:0 writeback:0 unstable:0
[ 3.202410] free:872 slab_reclaimable:13 slab_unreclaimable:1880
[ 3.202410] mapped:0 shmem:0 pagetables:0 bounce:0
[ 3.202410] free_cma:0

because the system has run out of memory at boot time. This occurs
because of the following sequence in the boot:

Main kernel boots and sets E820 map. The second kernel is booted with a
map generated by the kdump service using memmap= and memmap=exactmap.
These parameters are added to the kernel parameters of the kexec/kdump
kernel. The kexec/kdump kernel has limited memory resources so as not
to severely impact the main kernel.

The system then panics and the kdump/kexec kernel boots (which is a
completely new kernel boot). During this boot ACPI is initialized and the
kernel (as can be seen above) traverses the ACPI namespace and finds an
entry for a memory device to be hotadded.

ie)

[ 3.053720] [<ffffffff815a1e9f>] __add_pages+0xaf/0x240
[ 3.059656] [<ffffffff81047359>] arch_add_memory+0x59/0xd0
[ 3.065877] [<ffffffff815a21d9>] add_memory+0xb9/0x1b0
[ 3.071713] [<ffffffff81333b9c>] acpi_memory_device_add+0x18d/0x26d
[ 3.078813] [<ffffffff81309a01>] acpi_bus_device_attach+0x7d/0xcd
[ 3.085719] [<ffffffff8132379d>] acpi_ns_walk_namespace+0xc8/0x17f
[ 3.092716] [<ffffffff81309984>] ? acpi_bus_type_and_status+0x90/0x90
[ 3.100004] [<ffffffff81309984>] ? acpi_bus_type_and_status+0x90/0x90
[ 3.107293] [<ffffffff81323c8c>] acpi_walk_namespace+0x95/0xc5
[ 3.113904] [<ffffffff8130a6d6>] acpi_bus_scan+0x8b/0x9d
[ 3.119933] [<ffffffff81a2019a>] acpi_scan_init+0x63/0x160
[ 3.126153] [<ffffffff81a1ffb5>] acpi_init+0x25d/0x2a6

At this point the kernel adds page table information and the the kexec/kdump
kernel runs out of memory.

This can also be reproduced with a "regular" kernel by using the
memmap=exactmap and mem=X parameters on the main kernel and booting.

This patchset resolves the problem by adding a kernel parameter,
acpi_no_memhotplug, to disable ACPI memory hotplug. ACPI memory hotplug
should also be disabled by default when a user specified a memory mapping with
"memmap=exactmap" or "mem=X".

Signed-off-by: Prarit Bhargava <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: "H. Peter Anvin" <[email protected]>
Cc: [email protected]
Cc: Len Brown <[email protected]>
Cc: "Rafael J. Wysocki" <[email protected]>
Cc: Linn Crosetto <[email protected]>
Cc: Pekka Enberg <[email protected]>
Cc: Yinghai Lu <[email protected]>
Cc: Andrew Morton <[email protected]>
Cc: Toshi Kani <[email protected]>
Cc: Tang Chen <[email protected]>
Cc: Wen Congyang <[email protected]>
Cc: Vivek Goyal <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: Toshi Kani <[email protected]>
Cc: [email protected]
Cc: [email protected]
---
Documentation/kernel-parameters.txt | 3 +++
arch/x86/kernel/e820.c | 4 ++++
drivers/acpi/acpi_memhotplug.c | 18 ++++++++++++++++++
include/linux/memory_hotplug.h | 9 +++++++++
4 files changed, 34 insertions(+)

diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt
index b9e9bd8..ea93f75 100644
--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
@@ -343,6 +343,9 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
no: ACPI OperationRegions are not marked as reserved,
no further checks are performed.

+ acpi_no_memhotplug [ACPI] Disable memory hotplug. Useful for kexec
+ and kdump kernels.
+
add_efi_memmap [EFI; X86] Include EFI memory map in
kernel's map of available physical RAM.

diff --git a/arch/x86/kernel/e820.c b/arch/x86/kernel/e820.c
index 174da5f..3c431fe 100644
--- a/arch/x86/kernel/e820.c
+++ b/arch/x86/kernel/e820.c
@@ -20,6 +20,7 @@
#include <linux/firmware-map.h>
#include <linux/memblock.h>
#include <linux/sort.h>
+#include <linux/memory_hotplug.h>

#include <asm/e820.h>
#include <asm/proto.h>
@@ -834,6 +835,8 @@ static int __init parse_memopt(char *p)
return -EINVAL;
e820_remove_range(mem_size, ULLONG_MAX - mem_size, E820_RAM, 1);

+ set_acpi_no_memhotplug();
+
return 0;
}
early_param("mem", parse_memopt);
@@ -857,6 +860,7 @@ static int __init parse_memmap_one(char *p)
#endif
e820.nr_map = 0;
userdef = 1;
+ set_acpi_no_memhotplug();
return 0;
}

diff --git a/drivers/acpi/acpi_memhotplug.c b/drivers/acpi/acpi_memhotplug.c
index 551dad7..d104a7d 100644
--- a/drivers/acpi/acpi_memhotplug.c
+++ b/drivers/acpi/acpi_memhotplug.c
@@ -361,7 +361,25 @@ static void acpi_memory_device_remove(struct acpi_device *device)
acpi_memory_device_free(mem_device);
}

+static bool acpi_no_memhotplug;
+
+void set_acpi_no_memhotplug(void)
+{
+ acpi_no_memhotplug = true;
+ pr_info_once("ACPI: Memory Hotplug Disabled\n");
+}
+
void __init acpi_memory_hotplug_init(void)
{
+ if (acpi_no_memhotplug)
+ return;
+
acpi_scan_add_handler_with_hotplug(&memory_device_handler, "memory");
}
+
+static int __init disable_acpi_memory_hotplug(char *str)
+{
+ set_acpi_no_memhotplug();
+ return 1;
+}
+__setup("acpi_no_memhotplug", disable_acpi_memory_hotplug);
diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h
index 4ca3d95..3cdb6e0 100644
--- a/include/linux/memory_hotplug.h
+++ b/include/linux/memory_hotplug.h
@@ -12,6 +12,15 @@ struct pglist_data;
struct mem_section;
struct memory_block;

+#ifdef CONFIG_ACPI_HOTPLUG_MEMORY
+/* set flag to disable ACPI memory hotplug */
+extern void set_acpi_no_memhotplug(void);
+#else
+static inline void set_acpi_no_memhotplug(void)
+{
+}
+#endif
+
#ifdef CONFIG_MEMORY_HOTPLUG

/*
--
1.7.9.3


2014-01-13 22:17:26

by KOSAKI Motohiro

[permalink] [raw]
Subject: Re: [PATCH] x86, acpi memory hotplug, add parameter to disable memory hotplug

On Mon, Jan 13, 2014 at 4:56 PM, Prarit Bhargava <[email protected]> wrote:
> When booting a kexec/kdump kernel on a system that has specific memory hotplug
> regions the boot will fail with warnings like:
>
> [ 2.939467] swapper/0: page allocation failure: order:9, mode:0x84d0
> [ 2.946564] CPU: 0 PID: 1 Comm: swapper/0 Not tainted
> 3.10.0-65.el7.x86_64 #1
> [ 2.954532] Hardware name: QCI QSSC-S4R/QSSC-S4R, BIOS
> QSSC-S4R.QCI.01.00.S013.032920111005 03/29/2011
> [ 2.964926] 0000000000000000 ffff8800341bd8c8 ffffffff815bcc67
> ffff8800341bd950
> [ 2.973224] ffffffff8113b1a0 ffff880036339b00 0000000000000009
> 00000000000084d0
> [ 2.981523] ffff8800341bd950 ffffffff815b87ee 0000000000000000
> 0000000000000200
> [ 2.989821] Call Trace:
> [ 2.992560] [<ffffffff815bcc67>] dump_stack+0x19/0x1b
> [ 2.998300] [<ffffffff8113b1a0>] warn_alloc_failed+0xf0/0x160
> [ 3.004817] [<ffffffff815b87ee>] ?
> __alloc_pages_direct_compact+0xac/0x196
> [ 3.012594] [<ffffffff8113f14f>] __alloc_pages_nodemask+0x7ff/0xa00
> [ 3.019692] [<ffffffff815b417c>] vmemmap_alloc_block+0x62/0xba
> [ 3.026303] [<ffffffff815b41e9>] vmemmap_alloc_block_buf+0x15/0x3b
> [ 3.033302] [<ffffffff815b1ff6>] vmemmap_populate+0xb4/0x21b
> [ 3.039718] [<ffffffff815b461d>] sparse_mem_map_populate+0x27/0x35
> [ 3.046717] [<ffffffff815b400f>] sparse_add_one_section+0x7a/0x185
> [ 3.053720] [<ffffffff815a1e9f>] __add_pages+0xaf/0x240
> [ 3.059656] [<ffffffff81047359>] arch_add_memory+0x59/0xd0
> [ 3.065877] [<ffffffff815a21d9>] add_memory+0xb9/0x1b0
> [ 3.071713] [<ffffffff81333b9c>] acpi_memory_device_add+0x18d/0x26d
> [ 3.078813] [<ffffffff81309a01>] acpi_bus_device_attach+0x7d/0xcd
> [ 3.085719] [<ffffffff8132379d>] acpi_ns_walk_namespace+0xc8/0x17f
> [ 3.092716] [<ffffffff81309984>] ? acpi_bus_type_and_status+0x90/0x90
> [ 3.100004] [<ffffffff81309984>] ? acpi_bus_type_and_status+0x90/0x90
> [ 3.107293] [<ffffffff81323c8c>] acpi_walk_namespace+0x95/0xc5
> [ 3.113904] [<ffffffff8130a6d6>] acpi_bus_scan+0x8b/0x9d
> [ 3.119933] [<ffffffff81a2019a>] acpi_scan_init+0x63/0x160
> [ 3.126153] [<ffffffff81a1ffb5>] acpi_init+0x25d/0x2a6
> [ 3.131987] [<ffffffff81a1fd58>] ? acpi_sleep_proc_init+0x2a/0x2a
> [ 3.138889] [<ffffffff810020e2>] do_one_initcall+0xe2/0x190
> [ 3.145210] [<ffffffff819e20c4>] kernel_init_freeable+0x17c/0x207
> [ 3.152111] [<ffffffff819e18d0>] ? do_early_param+0x88/0x88
> [ 3.158430] [<ffffffff8159fea0>] ? rest_init+0x80/0x80
> [ 3.164264] [<ffffffff8159feae>] kernel_init+0xe/0x180
> [ 3.170097] [<ffffffff815cca2c>] ret_from_fork+0x7c/0xb0
> [ 3.176123] [<ffffffff8159fea0>] ? rest_init+0x80/0x80
> [ 3.181956] Mem-Info:
> [ 3.184490] Node 0 DMA per-cpu:
> [ 3.188007] CPU 0: hi: 0, btch: 1 usd: 0
> [ 3.193353] Node 0 DMA32 per-cpu:
> [ 3.197060] CPU 0: hi: 42, btch: 7 usd: 0
> [ 3.202410] active_anon:0 inactive_anon:0 isolated_anon:0
> [ 3.202410] active_file:0 inactive_file:0 isolated_file:0
> [ 3.202410] unevictable:0 dirty:0 writeback:0 unstable:0
> [ 3.202410] free:872 slab_reclaimable:13 slab_unreclaimable:1880
> [ 3.202410] mapped:0 shmem:0 pagetables:0 bounce:0
> [ 3.202410] free_cma:0
>
> because the system has run out of memory at boot time. This occurs
> because of the following sequence in the boot:
>
> Main kernel boots and sets E820 map. The second kernel is booted with a
> map generated by the kdump service using memmap= and memmap=exactmap.
> These parameters are added to the kernel parameters of the kexec/kdump
> kernel. The kexec/kdump kernel has limited memory resources so as not
> to severely impact the main kernel.
>
> The system then panics and the kdump/kexec kernel boots (which is a
> completely new kernel boot). During this boot ACPI is initialized and the
> kernel (as can be seen above) traverses the ACPI namespace and finds an
> entry for a memory device to be hotadded.
>
> ie)
>
> [ 3.053720] [<ffffffff815a1e9f>] __add_pages+0xaf/0x240
> [ 3.059656] [<ffffffff81047359>] arch_add_memory+0x59/0xd0
> [ 3.065877] [<ffffffff815a21d9>] add_memory+0xb9/0x1b0
> [ 3.071713] [<ffffffff81333b9c>] acpi_memory_device_add+0x18d/0x26d
> [ 3.078813] [<ffffffff81309a01>] acpi_bus_device_attach+0x7d/0xcd
> [ 3.085719] [<ffffffff8132379d>] acpi_ns_walk_namespace+0xc8/0x17f
> [ 3.092716] [<ffffffff81309984>] ? acpi_bus_type_and_status+0x90/0x90
> [ 3.100004] [<ffffffff81309984>] ? acpi_bus_type_and_status+0x90/0x90
> [ 3.107293] [<ffffffff81323c8c>] acpi_walk_namespace+0x95/0xc5
> [ 3.113904] [<ffffffff8130a6d6>] acpi_bus_scan+0x8b/0x9d
> [ 3.119933] [<ffffffff81a2019a>] acpi_scan_init+0x63/0x160
> [ 3.126153] [<ffffffff81a1ffb5>] acpi_init+0x25d/0x2a6
>
> At this point the kernel adds page table information and the the kexec/kdump
> kernel runs out of memory.
>
> This can also be reproduced with a "regular" kernel by using the
> memmap=exactmap and mem=X parameters on the main kernel and booting.
>
> This patchset resolves the problem by adding a kernel parameter,
> acpi_no_memhotplug, to disable ACPI memory hotplug. ACPI memory hotplug
> should also be disabled by default when a user specified a memory mapping with
> "memmap=exactmap" or "mem=X".
>
> Signed-off-by: Prarit Bhargava <[email protected]>
> Cc: Thomas Gleixner <[email protected]>
> Cc: Ingo Molnar <[email protected]>
> Cc: "H. Peter Anvin" <[email protected]>
> Cc: [email protected]
> Cc: Len Brown <[email protected]>
> Cc: "Rafael J. Wysocki" <[email protected]>
> Cc: Linn Crosetto <[email protected]>
> Cc: Pekka Enberg <[email protected]>
> Cc: Yinghai Lu <[email protected]>
> Cc: Andrew Morton <[email protected]>
> Cc: Toshi Kani <[email protected]>
> Cc: Tang Chen <[email protected]>
> Cc: Wen Congyang <[email protected]>
> Cc: Vivek Goyal <[email protected]>
> Cc: [email protected]
> Cc: [email protected]
> Cc: Toshi Kani <[email protected]>
> Cc: [email protected]
> Cc: [email protected]

I think we need a knob manually enable mem-hotplug when specify memmap. But
it is another story.

Acked-by: KOSAKI Motohiro <[email protected]>

2014-01-13 23:41:35

by Prarit Bhargava

[permalink] [raw]
Subject: Re: [PATCH] x86, acpi memory hotplug, add parameter to disable memory hotplug



On 01/13/2014 05:17 PM, KOSAKI Motohiro wrote:
> On Mon, Jan 13, 2014 at 4:56 PM, Prarit Bhargava <[email protected]> wrote:
>> When booting a kexec/kdump kernel on a system that has specific memory hotplug
>> regions the boot will fail with warnings like:
>>
>> [ 2.939467] swapper/0: page allocation failure: order:9, mode:0x84d0
>> [ 2.946564] CPU: 0 PID: 1 Comm: swapper/0 Not tainted
>> 3.10.0-65.el7.x86_64 #1
>> [ 2.954532] Hardware name: QCI QSSC-S4R/QSSC-S4R, BIOS
>> QSSC-S4R.QCI.01.00.S013.032920111005 03/29/2011
>> [ 2.964926] 0000000000000000 ffff8800341bd8c8 ffffffff815bcc67
>> ffff8800341bd950
>> [ 2.973224] ffffffff8113b1a0 ffff880036339b00 0000000000000009
>> 00000000000084d0
>> [ 2.981523] ffff8800341bd950 ffffffff815b87ee 0000000000000000
>> 0000000000000200
>> [ 2.989821] Call Trace:
>> [ 2.992560] [<ffffffff815bcc67>] dump_stack+0x19/0x1b
>> [ 2.998300] [<ffffffff8113b1a0>] warn_alloc_failed+0xf0/0x160
>> [ 3.004817] [<ffffffff815b87ee>] ?
>> __alloc_pages_direct_compact+0xac/0x196
>> [ 3.012594] [<ffffffff8113f14f>] __alloc_pages_nodemask+0x7ff/0xa00
>> [ 3.019692] [<ffffffff815b417c>] vmemmap_alloc_block+0x62/0xba
>> [ 3.026303] [<ffffffff815b41e9>] vmemmap_alloc_block_buf+0x15/0x3b
>> [ 3.033302] [<ffffffff815b1ff6>] vmemmap_populate+0xb4/0x21b
>> [ 3.039718] [<ffffffff815b461d>] sparse_mem_map_populate+0x27/0x35
>> [ 3.046717] [<ffffffff815b400f>] sparse_add_one_section+0x7a/0x185
>> [ 3.053720] [<ffffffff815a1e9f>] __add_pages+0xaf/0x240
>> [ 3.059656] [<ffffffff81047359>] arch_add_memory+0x59/0xd0
>> [ 3.065877] [<ffffffff815a21d9>] add_memory+0xb9/0x1b0
>> [ 3.071713] [<ffffffff81333b9c>] acpi_memory_device_add+0x18d/0x26d
>> [ 3.078813] [<ffffffff81309a01>] acpi_bus_device_attach+0x7d/0xcd
>> [ 3.085719] [<ffffffff8132379d>] acpi_ns_walk_namespace+0xc8/0x17f
>> [ 3.092716] [<ffffffff81309984>] ? acpi_bus_type_and_status+0x90/0x90
>> [ 3.100004] [<ffffffff81309984>] ? acpi_bus_type_and_status+0x90/0x90
>> [ 3.107293] [<ffffffff81323c8c>] acpi_walk_namespace+0x95/0xc5
>> [ 3.113904] [<ffffffff8130a6d6>] acpi_bus_scan+0x8b/0x9d
>> [ 3.119933] [<ffffffff81a2019a>] acpi_scan_init+0x63/0x160
>> [ 3.126153] [<ffffffff81a1ffb5>] acpi_init+0x25d/0x2a6
>> [ 3.131987] [<ffffffff81a1fd58>] ? acpi_sleep_proc_init+0x2a/0x2a
>> [ 3.138889] [<ffffffff810020e2>] do_one_initcall+0xe2/0x190
>> [ 3.145210] [<ffffffff819e20c4>] kernel_init_freeable+0x17c/0x207
>> [ 3.152111] [<ffffffff819e18d0>] ? do_early_param+0x88/0x88
>> [ 3.158430] [<ffffffff8159fea0>] ? rest_init+0x80/0x80
>> [ 3.164264] [<ffffffff8159feae>] kernel_init+0xe/0x180
>> [ 3.170097] [<ffffffff815cca2c>] ret_from_fork+0x7c/0xb0
>> [ 3.176123] [<ffffffff8159fea0>] ? rest_init+0x80/0x80
>> [ 3.181956] Mem-Info:
>> [ 3.184490] Node 0 DMA per-cpu:
>> [ 3.188007] CPU 0: hi: 0, btch: 1 usd: 0
>> [ 3.193353] Node 0 DMA32 per-cpu:
>> [ 3.197060] CPU 0: hi: 42, btch: 7 usd: 0
>> [ 3.202410] active_anon:0 inactive_anon:0 isolated_anon:0
>> [ 3.202410] active_file:0 inactive_file:0 isolated_file:0
>> [ 3.202410] unevictable:0 dirty:0 writeback:0 unstable:0
>> [ 3.202410] free:872 slab_reclaimable:13 slab_unreclaimable:1880
>> [ 3.202410] mapped:0 shmem:0 pagetables:0 bounce:0
>> [ 3.202410] free_cma:0
>>
>> because the system has run out of memory at boot time. This occurs
>> because of the following sequence in the boot:
>>
>> Main kernel boots and sets E820 map. The second kernel is booted with a
>> map generated by the kdump service using memmap= and memmap=exactmap.
>> These parameters are added to the kernel parameters of the kexec/kdump
>> kernel. The kexec/kdump kernel has limited memory resources so as not
>> to severely impact the main kernel.
>>
>> The system then panics and the kdump/kexec kernel boots (which is a
>> completely new kernel boot). During this boot ACPI is initialized and the
>> kernel (as can be seen above) traverses the ACPI namespace and finds an
>> entry for a memory device to be hotadded.
>>
>> ie)
>>
>> [ 3.053720] [<ffffffff815a1e9f>] __add_pages+0xaf/0x240
>> [ 3.059656] [<ffffffff81047359>] arch_add_memory+0x59/0xd0
>> [ 3.065877] [<ffffffff815a21d9>] add_memory+0xb9/0x1b0
>> [ 3.071713] [<ffffffff81333b9c>] acpi_memory_device_add+0x18d/0x26d
>> [ 3.078813] [<ffffffff81309a01>] acpi_bus_device_attach+0x7d/0xcd
>> [ 3.085719] [<ffffffff8132379d>] acpi_ns_walk_namespace+0xc8/0x17f
>> [ 3.092716] [<ffffffff81309984>] ? acpi_bus_type_and_status+0x90/0x90
>> [ 3.100004] [<ffffffff81309984>] ? acpi_bus_type_and_status+0x90/0x90
>> [ 3.107293] [<ffffffff81323c8c>] acpi_walk_namespace+0x95/0xc5
>> [ 3.113904] [<ffffffff8130a6d6>] acpi_bus_scan+0x8b/0x9d
>> [ 3.119933] [<ffffffff81a2019a>] acpi_scan_init+0x63/0x160
>> [ 3.126153] [<ffffffff81a1ffb5>] acpi_init+0x25d/0x2a6
>>
>> At this point the kernel adds page table information and the the kexec/kdump
>> kernel runs out of memory.
>>
>> This can also be reproduced with a "regular" kernel by using the
>> memmap=exactmap and mem=X parameters on the main kernel and booting.
>>
>> This patchset resolves the problem by adding a kernel parameter,
>> acpi_no_memhotplug, to disable ACPI memory hotplug. ACPI memory hotplug
>> should also be disabled by default when a user specified a memory mapping with
>> "memmap=exactmap" or "mem=X".
>>
>> Signed-off-by: Prarit Bhargava <[email protected]>
>> Cc: Thomas Gleixner <[email protected]>
>> Cc: Ingo Molnar <[email protected]>
>> Cc: "H. Peter Anvin" <[email protected]>
>> Cc: [email protected]
>> Cc: Len Brown <[email protected]>
>> Cc: "Rafael J. Wysocki" <[email protected]>
>> Cc: Linn Crosetto <[email protected]>
>> Cc: Pekka Enberg <[email protected]>
>> Cc: Yinghai Lu <[email protected]>
>> Cc: Andrew Morton <[email protected]>
>> Cc: Toshi Kani <[email protected]>
>> Cc: Tang Chen <[email protected]>
>> Cc: Wen Congyang <[email protected]>
>> Cc: Vivek Goyal <[email protected]>
>> Cc: [email protected]
>> Cc: [email protected]
>> Cc: Toshi Kani <[email protected]>
>> Cc: [email protected]
>> Cc: [email protected]
>
> I think we need a knob manually enable mem-hotplug when specify memmap. But
> it is another story.
>
> Acked-by: KOSAKI Motohiro <[email protected]>

As mentioned, self-NAK. I have seen a system that I needed to specify
memmap=exactmap & had hotplug memory. I will only keep the acpi_no_memhotplug
option in the next version of the patch.

P.

2014-01-14 01:12:37

by Yasuaki Ishimatsu

[permalink] [raw]
Subject: Re: [PATCH] x86, acpi memory hotplug, add parameter to disable memory hotplug

(2014/01/14 8:41), Prarit Bhargava wrote:
>
>
> On 01/13/2014 05:17 PM, KOSAKI Motohiro wrote:
>> On Mon, Jan 13, 2014 at 4:56 PM, Prarit Bhargava <[email protected]> wrote:
>>> When booting a kexec/kdump kernel on a system that has specific memory hotplug
>>> regions the boot will fail with warnings like:
>>>
>>> [ 2.939467] swapper/0: page allocation failure: order:9, mode:0x84d0
>>> [ 2.946564] CPU: 0 PID: 1 Comm: swapper/0 Not tainted
>>> 3.10.0-65.el7.x86_64 #1
>>> [ 2.954532] Hardware name: QCI QSSC-S4R/QSSC-S4R, BIOS
>>> QSSC-S4R.QCI.01.00.S013.032920111005 03/29/2011
>>> [ 2.964926] 0000000000000000 ffff8800341bd8c8 ffffffff815bcc67
>>> ffff8800341bd950
>>> [ 2.973224] ffffffff8113b1a0 ffff880036339b00 0000000000000009
>>> 00000000000084d0
>>> [ 2.981523] ffff8800341bd950 ffffffff815b87ee 0000000000000000
>>> 0000000000000200
>>> [ 2.989821] Call Trace:
>>> [ 2.992560] [<ffffffff815bcc67>] dump_stack+0x19/0x1b
>>> [ 2.998300] [<ffffffff8113b1a0>] warn_alloc_failed+0xf0/0x160
>>> [ 3.004817] [<ffffffff815b87ee>] ?
>>> __alloc_pages_direct_compact+0xac/0x196
>>> [ 3.012594] [<ffffffff8113f14f>] __alloc_pages_nodemask+0x7ff/0xa00
>>> [ 3.019692] [<ffffffff815b417c>] vmemmap_alloc_block+0x62/0xba
>>> [ 3.026303] [<ffffffff815b41e9>] vmemmap_alloc_block_buf+0x15/0x3b
>>> [ 3.033302] [<ffffffff815b1ff6>] vmemmap_populate+0xb4/0x21b
>>> [ 3.039718] [<ffffffff815b461d>] sparse_mem_map_populate+0x27/0x35
>>> [ 3.046717] [<ffffffff815b400f>] sparse_add_one_section+0x7a/0x185
>>> [ 3.053720] [<ffffffff815a1e9f>] __add_pages+0xaf/0x240
>>> [ 3.059656] [<ffffffff81047359>] arch_add_memory+0x59/0xd0
>>> [ 3.065877] [<ffffffff815a21d9>] add_memory+0xb9/0x1b0
>>> [ 3.071713] [<ffffffff81333b9c>] acpi_memory_device_add+0x18d/0x26d
>>> [ 3.078813] [<ffffffff81309a01>] acpi_bus_device_attach+0x7d/0xcd
>>> [ 3.085719] [<ffffffff8132379d>] acpi_ns_walk_namespace+0xc8/0x17f
>>> [ 3.092716] [<ffffffff81309984>] ? acpi_bus_type_and_status+0x90/0x90
>>> [ 3.100004] [<ffffffff81309984>] ? acpi_bus_type_and_status+0x90/0x90
>>> [ 3.107293] [<ffffffff81323c8c>] acpi_walk_namespace+0x95/0xc5
>>> [ 3.113904] [<ffffffff8130a6d6>] acpi_bus_scan+0x8b/0x9d
>>> [ 3.119933] [<ffffffff81a2019a>] acpi_scan_init+0x63/0x160
>>> [ 3.126153] [<ffffffff81a1ffb5>] acpi_init+0x25d/0x2a6
>>> [ 3.131987] [<ffffffff81a1fd58>] ? acpi_sleep_proc_init+0x2a/0x2a
>>> [ 3.138889] [<ffffffff810020e2>] do_one_initcall+0xe2/0x190
>>> [ 3.145210] [<ffffffff819e20c4>] kernel_init_freeable+0x17c/0x207
>>> [ 3.152111] [<ffffffff819e18d0>] ? do_early_param+0x88/0x88
>>> [ 3.158430] [<ffffffff8159fea0>] ? rest_init+0x80/0x80
>>> [ 3.164264] [<ffffffff8159feae>] kernel_init+0xe/0x180
>>> [ 3.170097] [<ffffffff815cca2c>] ret_from_fork+0x7c/0xb0
>>> [ 3.176123] [<ffffffff8159fea0>] ? rest_init+0x80/0x80
>>> [ 3.181956] Mem-Info:
>>> [ 3.184490] Node 0 DMA per-cpu:
>>> [ 3.188007] CPU 0: hi: 0, btch: 1 usd: 0
>>> [ 3.193353] Node 0 DMA32 per-cpu:
>>> [ 3.197060] CPU 0: hi: 42, btch: 7 usd: 0
>>> [ 3.202410] active_anon:0 inactive_anon:0 isolated_anon:0
>>> [ 3.202410] active_file:0 inactive_file:0 isolated_file:0
>>> [ 3.202410] unevictable:0 dirty:0 writeback:0 unstable:0
>>> [ 3.202410] free:872 slab_reclaimable:13 slab_unreclaimable:1880
>>> [ 3.202410] mapped:0 shmem:0 pagetables:0 bounce:0
>>> [ 3.202410] free_cma:0
>>>
>>> because the system has run out of memory at boot time. This occurs
>>> because of the following sequence in the boot:
>>>
>>> Main kernel boots and sets E820 map. The second kernel is booted with a
>>> map generated by the kdump service using memmap= and memmap=exactmap.
>>> These parameters are added to the kernel parameters of the kexec/kdump
>>> kernel. The kexec/kdump kernel has limited memory resources so as not
>>> to severely impact the main kernel.
>>>
>>> The system then panics and the kdump/kexec kernel boots (which is a
>>> completely new kernel boot). During this boot ACPI is initialized and the
>>> kernel (as can be seen above) traverses the ACPI namespace and finds an
>>> entry for a memory device to be hotadded.
>>>
>>> ie)
>>>
>>> [ 3.053720] [<ffffffff815a1e9f>] __add_pages+0xaf/0x240
>>> [ 3.059656] [<ffffffff81047359>] arch_add_memory+0x59/0xd0
>>> [ 3.065877] [<ffffffff815a21d9>] add_memory+0xb9/0x1b0
>>> [ 3.071713] [<ffffffff81333b9c>] acpi_memory_device_add+0x18d/0x26d
>>> [ 3.078813] [<ffffffff81309a01>] acpi_bus_device_attach+0x7d/0xcd
>>> [ 3.085719] [<ffffffff8132379d>] acpi_ns_walk_namespace+0xc8/0x17f
>>> [ 3.092716] [<ffffffff81309984>] ? acpi_bus_type_and_status+0x90/0x90
>>> [ 3.100004] [<ffffffff81309984>] ? acpi_bus_type_and_status+0x90/0x90
>>> [ 3.107293] [<ffffffff81323c8c>] acpi_walk_namespace+0x95/0xc5
>>> [ 3.113904] [<ffffffff8130a6d6>] acpi_bus_scan+0x8b/0x9d
>>> [ 3.119933] [<ffffffff81a2019a>] acpi_scan_init+0x63/0x160
>>> [ 3.126153] [<ffffffff81a1ffb5>] acpi_init+0x25d/0x2a6
>>>
>>> At this point the kernel adds page table information and the the kexec/kdump
>>> kernel runs out of memory.
>>>
>>> This can also be reproduced with a "regular" kernel by using the
>>> memmap=exactmap and mem=X parameters on the main kernel and booting.
>>>
>>> This patchset resolves the problem by adding a kernel parameter,
>>> acpi_no_memhotplug, to disable ACPI memory hotplug. ACPI memory hotplug
>>> should also be disabled by default when a user specified a memory mapping with
>>> "memmap=exactmap" or "mem=X".
>>>
>>> Signed-off-by: Prarit Bhargava <[email protected]>
>>> Cc: Thomas Gleixner <[email protected]>
>>> Cc: Ingo Molnar <[email protected]>
>>> Cc: "H. Peter Anvin" <[email protected]>
>>> Cc: [email protected]
>>> Cc: Len Brown <[email protected]>
>>> Cc: "Rafael J. Wysocki" <[email protected]>
>>> Cc: Linn Crosetto <[email protected]>
>>> Cc: Pekka Enberg <[email protected]>
>>> Cc: Yinghai Lu <[email protected]>
>>> Cc: Andrew Morton <[email protected]>
>>> Cc: Toshi Kani <[email protected]>
>>> Cc: Tang Chen <[email protected]>
>>> Cc: Wen Congyang <[email protected]>
>>> Cc: Vivek Goyal <[email protected]>
>>> Cc: [email protected]
>>> Cc: [email protected]
>>> Cc: Toshi Kani <[email protected]>
>>> Cc: [email protected]
>>> Cc: [email protected]
>>
>> I think we need a knob manually enable mem-hotplug when specify memmap. But
>> it is another story.
>>
>> Acked-by: KOSAKI Motohiro <[email protected]>
>
> As mentioned, self-NAK. I have seen a system that I needed to specify
> memmap=exactmap & had hotplug memory. I will only keep the acpi_no_memhotplug
> option in the next version of the patch.


Your following first patch is simply and makes sense.

http://marc.info/?l=linux-acpi&m=138922019607796&w=2

Thanks,
Yasuaki Ishiamtsu

>
> P.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>

2014-01-14 01:45:12

by Dave Young

[permalink] [raw]
Subject: Re: [PATCH] x86, acpi memory hotplug, add parameter to disable memory hotplug

On 01/13/14 at 04:56pm, Prarit Bhargava wrote:
> When booting a kexec/kdump kernel on a system that has specific memory hotplug
> regions the boot will fail with warnings like:
>
> [ 2.939467] swapper/0: page allocation failure: order:9, mode:0x84d0
> [ 2.946564] CPU: 0 PID: 1 Comm: swapper/0 Not tainted
> 3.10.0-65.el7.x86_64 #1
> [ 2.954532] Hardware name: QCI QSSC-S4R/QSSC-S4R, BIOS
> QSSC-S4R.QCI.01.00.S013.032920111005 03/29/2011
> [ 2.964926] 0000000000000000 ffff8800341bd8c8 ffffffff815bcc67
> ffff8800341bd950
> [ 2.973224] ffffffff8113b1a0 ffff880036339b00 0000000000000009
> 00000000000084d0
> [ 2.981523] ffff8800341bd950 ffffffff815b87ee 0000000000000000
> 0000000000000200
> [ 2.989821] Call Trace:
> [ 2.992560] [<ffffffff815bcc67>] dump_stack+0x19/0x1b
> [ 2.998300] [<ffffffff8113b1a0>] warn_alloc_failed+0xf0/0x160
> [ 3.004817] [<ffffffff815b87ee>] ?
> __alloc_pages_direct_compact+0xac/0x196
> [ 3.012594] [<ffffffff8113f14f>] __alloc_pages_nodemask+0x7ff/0xa00
> [ 3.019692] [<ffffffff815b417c>] vmemmap_alloc_block+0x62/0xba
> [ 3.026303] [<ffffffff815b41e9>] vmemmap_alloc_block_buf+0x15/0x3b
> [ 3.033302] [<ffffffff815b1ff6>] vmemmap_populate+0xb4/0x21b
> [ 3.039718] [<ffffffff815b461d>] sparse_mem_map_populate+0x27/0x35
> [ 3.046717] [<ffffffff815b400f>] sparse_add_one_section+0x7a/0x185
> [ 3.053720] [<ffffffff815a1e9f>] __add_pages+0xaf/0x240
> [ 3.059656] [<ffffffff81047359>] arch_add_memory+0x59/0xd0
> [ 3.065877] [<ffffffff815a21d9>] add_memory+0xb9/0x1b0
> [ 3.071713] [<ffffffff81333b9c>] acpi_memory_device_add+0x18d/0x26d
> [ 3.078813] [<ffffffff81309a01>] acpi_bus_device_attach+0x7d/0xcd
> [ 3.085719] [<ffffffff8132379d>] acpi_ns_walk_namespace+0xc8/0x17f
> [ 3.092716] [<ffffffff81309984>] ? acpi_bus_type_and_status+0x90/0x90
> [ 3.100004] [<ffffffff81309984>] ? acpi_bus_type_and_status+0x90/0x90
> [ 3.107293] [<ffffffff81323c8c>] acpi_walk_namespace+0x95/0xc5
> [ 3.113904] [<ffffffff8130a6d6>] acpi_bus_scan+0x8b/0x9d
> [ 3.119933] [<ffffffff81a2019a>] acpi_scan_init+0x63/0x160
> [ 3.126153] [<ffffffff81a1ffb5>] acpi_init+0x25d/0x2a6
> [ 3.131987] [<ffffffff81a1fd58>] ? acpi_sleep_proc_init+0x2a/0x2a
> [ 3.138889] [<ffffffff810020e2>] do_one_initcall+0xe2/0x190
> [ 3.145210] [<ffffffff819e20c4>] kernel_init_freeable+0x17c/0x207
> [ 3.152111] [<ffffffff819e18d0>] ? do_early_param+0x88/0x88
> [ 3.158430] [<ffffffff8159fea0>] ? rest_init+0x80/0x80
> [ 3.164264] [<ffffffff8159feae>] kernel_init+0xe/0x180
> [ 3.170097] [<ffffffff815cca2c>] ret_from_fork+0x7c/0xb0
> [ 3.176123] [<ffffffff8159fea0>] ? rest_init+0x80/0x80
> [ 3.181956] Mem-Info:
> [ 3.184490] Node 0 DMA per-cpu:
> [ 3.188007] CPU 0: hi: 0, btch: 1 usd: 0
> [ 3.193353] Node 0 DMA32 per-cpu:
> [ 3.197060] CPU 0: hi: 42, btch: 7 usd: 0
> [ 3.202410] active_anon:0 inactive_anon:0 isolated_anon:0
> [ 3.202410] active_file:0 inactive_file:0 isolated_file:0
> [ 3.202410] unevictable:0 dirty:0 writeback:0 unstable:0
> [ 3.202410] free:872 slab_reclaimable:13 slab_unreclaimable:1880
> [ 3.202410] mapped:0 shmem:0 pagetables:0 bounce:0
> [ 3.202410] free_cma:0
>
> because the system has run out of memory at boot time. This occurs
> because of the following sequence in the boot:
>
> Main kernel boots and sets E820 map. The second kernel is booted with a
> map generated by the kdump service using memmap= and memmap=exactmap.
> These parameters are added to the kernel parameters of the kexec/kdump
> kernel. The kexec/kdump kernel has limited memory resources so as not
> to severely impact the main kernel.
>
> The system then panics and the kdump/kexec kernel boots (which is a
> completely new kernel boot). During this boot ACPI is initialized and the
> kernel (as can be seen above) traverses the ACPI namespace and finds an
> entry for a memory device to be hotadded.
>
> ie)
>
> [ 3.053720] [<ffffffff815a1e9f>] __add_pages+0xaf/0x240
> [ 3.059656] [<ffffffff81047359>] arch_add_memory+0x59/0xd0
> [ 3.065877] [<ffffffff815a21d9>] add_memory+0xb9/0x1b0
> [ 3.071713] [<ffffffff81333b9c>] acpi_memory_device_add+0x18d/0x26d
> [ 3.078813] [<ffffffff81309a01>] acpi_bus_device_attach+0x7d/0xcd
> [ 3.085719] [<ffffffff8132379d>] acpi_ns_walk_namespace+0xc8/0x17f
> [ 3.092716] [<ffffffff81309984>] ? acpi_bus_type_and_status+0x90/0x90
> [ 3.100004] [<ffffffff81309984>] ? acpi_bus_type_and_status+0x90/0x90
> [ 3.107293] [<ffffffff81323c8c>] acpi_walk_namespace+0x95/0xc5
> [ 3.113904] [<ffffffff8130a6d6>] acpi_bus_scan+0x8b/0x9d
> [ 3.119933] [<ffffffff81a2019a>] acpi_scan_init+0x63/0x160
> [ 3.126153] [<ffffffff81a1ffb5>] acpi_init+0x25d/0x2a6
>
> At this point the kernel adds page table information and the the kexec/kdump
> kernel runs out of memory.
>
> This can also be reproduced with a "regular" kernel by using the
> memmap=exactmap and mem=X parameters on the main kernel and booting.
>
> This patchset resolves the problem by adding a kernel parameter,
> acpi_no_memhotplug, to disable ACPI memory hotplug. ACPI memory hotplug
> should also be disabled by default when a user specified a memory mapping with
> "memmap=exactmap" or "mem=X".
>
> Signed-off-by: Prarit Bhargava <[email protected]>
> Cc: Thomas Gleixner <[email protected]>
> Cc: Ingo Molnar <[email protected]>
> Cc: "H. Peter Anvin" <[email protected]>
> Cc: [email protected]
> Cc: Len Brown <[email protected]>
> Cc: "Rafael J. Wysocki" <[email protected]>
> Cc: Linn Crosetto <[email protected]>
> Cc: Pekka Enberg <[email protected]>
> Cc: Yinghai Lu <[email protected]>
> Cc: Andrew Morton <[email protected]>
> Cc: Toshi Kani <[email protected]>
> Cc: Tang Chen <[email protected]>
> Cc: Wen Congyang <[email protected]>
> Cc: Vivek Goyal <[email protected]>
> Cc: [email protected]
> Cc: [email protected]
> Cc: Toshi Kani <[email protected]>
> Cc: [email protected]
> Cc: [email protected]
> ---
> Documentation/kernel-parameters.txt | 3 +++
> arch/x86/kernel/e820.c | 4 ++++
> drivers/acpi/acpi_memhotplug.c | 18 ++++++++++++++++++
> include/linux/memory_hotplug.h | 9 +++++++++
> 4 files changed, 34 insertions(+)
>
> diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt
> index b9e9bd8..ea93f75 100644
> --- a/Documentation/kernel-parameters.txt
> +++ b/Documentation/kernel-parameters.txt
> @@ -343,6 +343,9 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
> no: ACPI OperationRegions are not marked as reserved,
> no further checks are performed.
>
> + acpi_no_memhotplug [ACPI] Disable memory hotplug. Useful for kexec
> + and kdump kernels.
> +
> add_efi_memmap [EFI; X86] Include EFI memory map in
> kernel's map of available physical RAM.
>
> diff --git a/arch/x86/kernel/e820.c b/arch/x86/kernel/e820.c
> index 174da5f..3c431fe 100644
> --- a/arch/x86/kernel/e820.c
> +++ b/arch/x86/kernel/e820.c
> @@ -20,6 +20,7 @@
> #include <linux/firmware-map.h>
> #include <linux/memblock.h>
> #include <linux/sort.h>
> +#include <linux/memory_hotplug.h>
>
> #include <asm/e820.h>
> #include <asm/proto.h>
> @@ -834,6 +835,8 @@ static int __init parse_memopt(char *p)
> return -EINVAL;
> e820_remove_range(mem_size, ULLONG_MAX - mem_size, E820_RAM, 1);
>
> + set_acpi_no_memhotplug();
> +
> return 0;
> }
> early_param("mem", parse_memopt);
> @@ -857,6 +860,7 @@ static int __init parse_memmap_one(char *p)
> #endif
> e820.nr_map = 0;
> userdef = 1;
> + set_acpi_no_memhotplug();
> return 0;
> }
>
> diff --git a/drivers/acpi/acpi_memhotplug.c b/drivers/acpi/acpi_memhotplug.c
> index 551dad7..d104a7d 100644
> --- a/drivers/acpi/acpi_memhotplug.c
> +++ b/drivers/acpi/acpi_memhotplug.c
> @@ -361,7 +361,25 @@ static void acpi_memory_device_remove(struct acpi_device *device)
> acpi_memory_device_free(mem_device);
> }
>
> +static bool acpi_no_memhotplug;
> +
> +void set_acpi_no_memhotplug(void)
> +{
> + acpi_no_memhotplug = true;
> + pr_info_once("ACPI: Memory Hotplug Disabled\n");
> +}
> +
> void __init acpi_memory_hotplug_init(void)
> {
> + if (acpi_no_memhotplug)
> + return;
> +
> acpi_scan_add_handler_with_hotplug(&memory_device_handler, "memory");
> }
> +
> +static int __init disable_acpi_memory_hotplug(char *str)
> +{
> + set_acpi_no_memhotplug();
> + return 1;
> +}
> +__setup("acpi_no_memhotplug", disable_acpi_memory_hotplug);
> diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h
> index 4ca3d95..3cdb6e0 100644
> --- a/include/linux/memory_hotplug.h
> +++ b/include/linux/memory_hotplug.h
> @@ -12,6 +12,15 @@ struct pglist_data;
> struct mem_section;
> struct memory_block;
>
> +#ifdef CONFIG_ACPI_HOTPLUG_MEMORY
> +/* set flag to disable ACPI memory hotplug */
> +extern void set_acpi_no_memhotplug(void);
> +#else
> +static inline void set_acpi_no_memhotplug(void)
> +{
> +}
> +#endif
> +
> #ifdef CONFIG_MEMORY_HOTPLUG
>
> /*
> --
> 1.7.9.3
>

Acked-by: Dave Young <[email protected]>

2014-01-14 01:47:32

by Toshi Kani

[permalink] [raw]
Subject: Re: [PATCH] x86, acpi memory hotplug, add parameter to disable memory hotplug

On Tue, 2014-01-14 at 10:11 +0900, Yasuaki Ishimatsu wrote:
:
> >> I think we need a knob manually enable mem-hotplug when specify memmap. But
> >> it is another story.
> >>
> >> Acked-by: KOSAKI Motohiro <[email protected]>
> >
> > As mentioned, self-NAK. I have seen a system that I needed to specify
> > memmap=exactmap & had hotplug memory. I will only keep the acpi_no_memhotplug
> > option in the next version of the patch.
>
>
> Your following first patch is simply and makes sense.
>
> http://marc.info/?l=linux-acpi&m=138922019607796&w=2
>

In this option, it also requires changing kexec-tools to specify the new
option for kdump. It won't be simpler.

Thanks,
-Toshi

2014-01-14 02:44:53

by Yasuaki Ishimatsu

[permalink] [raw]
Subject: Re: [PATCH] x86, acpi memory hotplug, add parameter to disable memory hotplug

(2014/01/14 10:41), Toshi Kani wrote:
> On Tue, 2014-01-14 at 10:11 +0900, Yasuaki Ishimatsu wrote:
> :
>>>> I think we need a knob manually enable mem-hotplug when specify memmap. But
>>>> it is another story.
>>>>
>>>> Acked-by: KOSAKI Motohiro <[email protected]>
>>>
>>> As mentioned, self-NAK. I have seen a system that I needed to specify
>>> memmap=exactmap & had hotplug memory. I will only keep the acpi_no_memhotplug
>>> option in the next version of the patch.
>>
>>
>> Your following first patch is simply and makes sense.
>>
>> http://marc.info/?l=linux-acpi&m=138922019607796&w=2
>>
>
> In this option, it also requires changing kexec-tools to specify the new
> option for kdump. It won't be simpler.

Hmm.
I use memm= boot option and hotplug memory for memory hot-remove.
At least, the patch cannot be accepted.

Thanks,
Yasuaki Ishimatsu

>
> Thanks,
> -Toshi
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>

2014-01-14 10:58:41

by Prarit Bhargava

[permalink] [raw]
Subject: Re: [PATCH] x86, acpi memory hotplug, add parameter to disable memory hotplug



On 01/13/2014 09:43 PM, Yasuaki Ishimatsu wrote:
> (2014/01/14 10:41), Toshi Kani wrote:
>> On Tue, 2014-01-14 at 10:11 +0900, Yasuaki Ishimatsu wrote:
>> :
>>>>> I think we need a knob manually enable mem-hotplug when specify memmap. But
>>>>> it is another story.
>>>>>
>>>>> Acked-by: KOSAKI Motohiro <[email protected]>
>>>>
>>>> As mentioned, self-NAK. I have seen a system that I needed to specify
>>>> memmap=exactmap & had hotplug memory. I will only keep the acpi_no_memhotplug
>>>> option in the next version of the patch.
>>>
>>>
>>> Your following first patch is simply and makes sense.
>>>
>>> http://marc.info/?l=linux-acpi&m=138922019607796&w=2
>>>
>>
>> In this option, it also requires changing kexec-tools to specify the new
>> option for kdump. It won't be simpler.
>
> Hmm.
> I use memm= boot option and hotplug memory for memory hot-remove.
> At least, the patch cannot be accepted.

Thanks for the information Yasuaki. I will resubmit my first patch that only
adds the kernel parameter.

P.

>
> Thanks,
> Yasuaki Ishimatsu
>
>>
>> Thanks,
>> -Toshi
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
>> the body of a message to [email protected]
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>
>

2014-01-14 11:05:37

by Prarit Bhargava

[permalink] [raw]
Subject: Re: [PATCH] x86, acpi memory hotplug, add parameter to disable memory hotplug



On 01/13/2014 08:41 PM, Toshi Kani wrote:
> On Tue, 2014-01-14 at 10:11 +0900, Yasuaki Ishimatsu wrote:
> :
>>>> I think we need a knob manually enable mem-hotplug when specify memmap. But
>>>> it is another story.
>>>>
>>>> Acked-by: KOSAKI Motohiro <[email protected]>
>>>
>>> As mentioned, self-NAK. I have seen a system that I needed to specify
>>> memmap=exactmap & had hotplug memory. I will only keep the acpi_no_memhotplug
>>> option in the next version of the patch.
>>
>>
>> Your following first patch is simply and makes sense.
>>
>> http://marc.info/?l=linux-acpi&m=138922019607796&w=2
>>
>
> In this option, it also requires changing kexec-tools to specify the new
> option for kdump. It won't be simpler.

It will be simpler for the kernel and those of us who have to debug busted e820
maps ;)

Unfortunately I may not be able to give you the automatic disable. I did
contemplate adding a !is_kdump_kernel() to the ACPI memory hotplug init call,
but it seems like that is unacceptable as well.

P.

>
> Thanks,
> -Toshi
>

2014-01-14 14:38:13

by Vivek Goyal

[permalink] [raw]
Subject: Re: [PATCH] x86, acpi memory hotplug, add parameter to disable memory hotplug

On Tue, Jan 14, 2014 at 06:05:15AM -0500, Prarit Bhargava wrote:
>
>
> On 01/13/2014 08:41 PM, Toshi Kani wrote:
> > On Tue, 2014-01-14 at 10:11 +0900, Yasuaki Ishimatsu wrote:
> > :
> >>>> I think we need a knob manually enable mem-hotplug when specify memmap. But
> >>>> it is another story.
> >>>>
> >>>> Acked-by: KOSAKI Motohiro <[email protected]>
> >>>
> >>> As mentioned, self-NAK. I have seen a system that I needed to specify
> >>> memmap=exactmap & had hotplug memory. I will only keep the acpi_no_memhotplug
> >>> option in the next version of the patch.
> >>
> >>
> >> Your following first patch is simply and makes sense.
> >>
> >> http://marc.info/?l=linux-acpi&m=138922019607796&w=2
> >>
> >
> > In this option, it also requires changing kexec-tools to specify the new
> > option for kdump. It won't be simpler.
>
> It will be simpler for the kernel and those of us who have to debug busted e820
> maps ;)
>
> Unfortunately I may not be able to give you the automatic disable. I did
> contemplate adding a !is_kdump_kernel() to the ACPI memory hotplug init call,
> but it seems like that is unacceptable as well.

Yep, I don't think hotplug feature should be tied to it being a kdump
kernel.

Thanks
Vivek

2014-01-14 15:23:15

by Toshi Kani

[permalink] [raw]
Subject: Re: [PATCH] x86, acpi memory hotplug, add parameter to disable memory hotplug

On Tue, 2014-01-14 at 11:43 +0900, Yasuaki Ishimatsu wrote:
> (2014/01/14 10:41), Toshi Kani wrote:
> > On Tue, 2014-01-14 at 10:11 +0900, Yasuaki Ishimatsu wrote:
> > :
> >>>> I think we need a knob manually enable mem-hotplug when specify memmap. But
> >>>> it is another story.
> >>>>
> >>>> Acked-by: KOSAKI Motohiro <[email protected]>
> >>>
> >>> As mentioned, self-NAK. I have seen a system that I needed to specify
> >>> memmap=exactmap & had hotplug memory. I will only keep the acpi_no_memhotplug
> >>> option in the next version of the patch.
> >>
> >>
> >> Your following first patch is simply and makes sense.
> >>
> >> http://marc.info/?l=linux-acpi&m=138922019607796&w=2
> >>
> >
> > In this option, it also requires changing kexec-tools to specify the new
> > option for kdump. It won't be simpler.
>
> Hmm.
> I use memm= boot option and hotplug memory for memory hot-remove.
> At least, the patch cannot be accepted.

Do you mean mem=nG option? I am curious to know how memory hot-remove
works in this case. Don't acpi_memhotplug add all the memory ranges at
boot, and defeat the mem=nG option?

Thanks,
-Toshi

2014-01-14 15:26:51

by Vivek Goyal

[permalink] [raw]
Subject: Re: [PATCH] x86, acpi memory hotplug, add parameter to disable memory hotplug

On Tue, Jan 14, 2014 at 06:05:15AM -0500, Prarit Bhargava wrote:

[..]
> >>> As mentioned, self-NAK. I have seen a system that I needed to specify
> >>> memmap=exactmap & had hotplug memory. I will only keep the acpi_no_memhotplug
> >>> option in the next version of the patch.
> >>
> >>
> >> Your following first patch is simply and makes sense.
> >>
> >> http://marc.info/?l=linux-acpi&m=138922019607796&w=2
> >>
> >
> > In this option, it also requires changing kexec-tools to specify the new
> > option for kdump. It won't be simpler.
>
> It will be simpler for the kernel and those of us who have to debug busted e820
> maps ;)
>
> Unfortunately I may not be able to give you the automatic disable. I did
> contemplate adding a !is_kdump_kernel() to the ACPI memory hotplug init call,
> but it seems like that is unacceptable as well.

I think everybody agrees that there has to be a stand alone command line
option to disable memory hotplug.

Whether to tie it into memmap=exactmap and mem=X is the contentious bit.
So I would suggest that just post a patch to disable memory hotplut using
a command line and later more patches can go in if people strongly feel
the need to tie it into memmap=exactmap.

In the mean time, we will modify /etc/sysconfig/kdump to pass
acpi_no_memhotplug so that user does not have to worry about passing this
parameter and kexec-tools will not have to be modified either.

Thanks
Vivek

2014-01-14 15:37:29

by Toshi Kani

[permalink] [raw]
Subject: Re: [PATCH] x86, acpi memory hotplug, add parameter to disable memory hotplug

On Tue, 2014-01-14 at 10:26 -0500, Vivek Goyal wrote:
> On Tue, Jan 14, 2014 at 06:05:15AM -0500, Prarit Bhargava wrote:
>
> [..]
> > >>> As mentioned, self-NAK. I have seen a system that I needed to specify
> > >>> memmap=exactmap & had hotplug memory. I will only keep the acpi_no_memhotplug
> > >>> option in the next version of the patch.
> > >>
> > >>
> > >> Your following first patch is simply and makes sense.
> > >>
> > >> http://marc.info/?l=linux-acpi&m=138922019607796&w=2
> > >>
> > >
> > > In this option, it also requires changing kexec-tools to specify the new
> > > option for kdump. It won't be simpler.
> >
> > It will be simpler for the kernel and those of us who have to debug busted e820
> > maps ;)
> >
> > Unfortunately I may not be able to give you the automatic disable. I did
> > contemplate adding a !is_kdump_kernel() to the ACPI memory hotplug init call,
> > but it seems like that is unacceptable as well.
>
> I think everybody agrees that there has to be a stand alone command line
> option to disable memory hotplug.
>
> Whether to tie it into memmap=exactmap and mem=X is the contentious bit.
> So I would suggest that just post a patch to disable memory hotplut using
> a command line and later more patches can go in if people strongly feel
> the need to tie it into memmap=exactmap.
>
> In the mean time, we will modify /etc/sysconfig/kdump to pass
> acpi_no_memhotplug so that user does not have to worry about passing this
> parameter and kexec-tools will not have to be modified either.

Fine by me. Thanks for modifying /etc/sysconfig/kdump file.
-Toshi