The commit below adds hot-added memory range to memblock, after
creating pgdat for new node.
commit f9126ab9241f66562debf69c2c9d8fee32ddcc53
Author: Xishi Qiu <[email protected]>
Date: Fri Aug 14 15:35:16 2015 -0700
memory-hotplug: fix wrong edge when hot add a new node
But there is a problem:
add_memory()
|--> hotadd_new_pgdat()
|--> free_area_init_node()
|--> get_pfn_range_for_nid()
|--> find start_pfn and end_pfn in memblock
|--> ......
|--> memblock_add_node(start, size, nid) -------- Here, just too late.
get_pfn_range_for_nid() will find that start_pfn and end_pfn are both 0.
As a result, when adding memory, dmesg will give the following wrong message.
[ 2007.577000] Initmem setup node 5 [mem 0x0000000000000000-0xffffffffffffffff]
[ 2007.584000] On node 5 totalpages: 0
[ 2007.585000] Built 5 zonelists in Node order, mobility grouping on. Total pages: 32588823
[ 2007.594000] Policy zone: Normal
[ 2007.598000] init_memory_mapping: [mem 0x60000000000-0x607ffffffff]
The solution is simple, just add the memory range to memblock a little earlier,
before hotadd_new_pgdat().
Signed-off-by: Tang Chen <[email protected]>
---
mm/memory_hotplug.c | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 6da82bc..9b78aff 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1248,6 +1248,14 @@ int __ref add_memory(int nid, u64 start, u64 size)
mem_hotplug_begin();
+ /*
+ * Add new range to memblock so that when hotadd_new_pgdat() is called to
+ * allocate new pgdat, get_pfn_range_for_nid() will be able to find this
+ * new range and calculate total pages correctly. The range will be remove
+ * at hot-remove time.
+ */
+ memblock_add_node(start, size, nid);
+
new_node = !node_online(nid);
if (new_node) {
pgdat = hotadd_new_pgdat(nid, start);
@@ -1277,7 +1285,6 @@ int __ref add_memory(int nid, u64 start, u64 size)
/* create new memmap entry */
firmware_map_add_hotplug(start, start + size, "System RAM");
- memblock_add_node(start, size, nid);
goto out;
@@ -1286,6 +1293,7 @@ error:
if (new_pgdat)
rollback_node_hotadd(nid, pgdat);
release_memory_resource(res);
+ memblock_remove(start, size);
out:
mem_hotplug_done();
--
1.8.3.1
On 2015/8/24 1:06, Tang Chen wrote:
> The commit below adds hot-added memory range to memblock, after
> creating pgdat for new node.
>
> commit f9126ab9241f66562debf69c2c9d8fee32ddcc53
> Author: Xishi Qiu <[email protected]>
> Date: Fri Aug 14 15:35:16 2015 -0700
>
> memory-hotplug: fix wrong edge when hot add a new node
>
> But there is a problem:
>
> add_memory()
> |--> hotadd_new_pgdat()
> |--> free_area_init_node()
> |--> get_pfn_range_for_nid()
> |--> find start_pfn and end_pfn in memblock
> |--> ......
> |--> memblock_add_node(start, size, nid) -------- Here, just too late.
>
> get_pfn_range_for_nid() will find that start_pfn and end_pfn are both 0.
> As a result, when adding memory, dmesg will give the following wrong message.
>
> [ 2007.577000] Initmem setup node 5 [mem 0x0000000000000000-0xffffffffffffffff]
> [ 2007.584000] On node 5 totalpages: 0
> [ 2007.585000] Built 5 zonelists in Node order, mobility grouping on. Total pages: 32588823
> [ 2007.594000] Policy zone: Normal
> [ 2007.598000] init_memory_mapping: [mem 0x60000000000-0x607ffffffff]
>
> The solution is simple, just add the memory range to memblock a little earlier,
> before hotadd_new_pgdat().
>
> Signed-off-by: Tang Chen <[email protected]>
> ---
> mm/memory_hotplug.c | 10 +++++++++-
> 1 file changed, 9 insertions(+), 1 deletion(-)
>
> diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
> index 6da82bc..9b78aff 100644
> --- a/mm/memory_hotplug.c
> +++ b/mm/memory_hotplug.c
> @@ -1248,6 +1248,14 @@ int __ref add_memory(int nid, u64 start, u64 size)
>
> mem_hotplug_begin();
>
> + /*
> + * Add new range to memblock so that when hotadd_new_pgdat() is called to
> + * allocate new pgdat, get_pfn_range_for_nid() will be able to find this
> + * new range and calculate total pages correctly. The range will be remove
> + * at hot-remove time.
> + */
> + memblock_add_node(start, size, nid);
> +
Hi Tang,
Looks fine to me.
If we add memblock_add_node() here, we should reset the managed pages and present pages,
so please revert my patch which Andrew has already merged into mm-tree.
"[PATCH 2/2] memory-hotplug: remove reset_node_managed_pages() and reset_node_managed_pages() in hotadd_new_pgdat()"
Thanks,
Xishi Qiu
> new_node = !node_online(nid);
> if (new_node) {
> pgdat = hotadd_new_pgdat(nid, start);
> @@ -1277,7 +1285,6 @@ int __ref add_memory(int nid, u64 start, u64 size)
>
> /* create new memmap entry */
> firmware_map_add_hotplug(start, start + size, "System RAM");
> - memblock_add_node(start, size, nid);
>
> goto out;
>
> @@ -1286,6 +1293,7 @@ error:
> if (new_pgdat)
> rollback_node_hotadd(nid, pgdat);
> release_memory_resource(res);
> + memblock_remove(start, size);
>
> out:
> mem_hotplug_done();
On 2015/8/24 17:22, Xishi Qiu wrote:
> On 2015/8/24 1:06, Tang Chen wrote:
>
>> The commit below adds hot-added memory range to memblock, after
>> creating pgdat for new node.
>>
>> commit f9126ab9241f66562debf69c2c9d8fee32ddcc53
>> Author: Xishi Qiu <[email protected]>
>> Date: Fri Aug 14 15:35:16 2015 -0700
>>
>> memory-hotplug: fix wrong edge when hot add a new node
>>
>> But there is a problem:
>>
>> add_memory()
>> |--> hotadd_new_pgdat()
>> |--> free_area_init_node()
>> |--> get_pfn_range_for_nid()
>> |--> find start_pfn and end_pfn in memblock
>> |--> ......
>> |--> memblock_add_node(start, size, nid) -------- Here, just too late.
>>
>> get_pfn_range_for_nid() will find that start_pfn and end_pfn are both 0.
>> As a result, when adding memory, dmesg will give the following wrong message.
>>
Hi Tang,
Another question, if we add cpu first, there will be print error too.
cpu_up()
try_online_node()
hotadd_new_pgdat()
So how about just skip the print if the size is empty or just print
"node xx is empty now, will update when online memory"?
Thanks,
Xishi Qiu
>> [ 2007.577000] Initmem setup node 5 [mem 0x0000000000000000-0xffffffffffffffff]
>> [ 2007.584000] On node 5 totalpages: 0
>> [ 2007.585000] Built 5 zonelists in Node order, mobility grouping on. Total pages: 32588823
>> [ 2007.594000] Policy zone: Normal
>> [ 2007.598000] init_memory_mapping: [mem 0x60000000000-0x607ffffffff]
>>
On 08/25/2015 09:09 AM, Xishi Qiu wrote:
> On 2015/8/24 17:22, Xishi Qiu wrote:
>
>> On 2015/8/24 1:06, Tang Chen wrote:
>>
>>> The commit below adds hot-added memory range to memblock, after
>>> creating pgdat for new node.
>>>
>>> commit f9126ab9241f66562debf69c2c9d8fee32ddcc53
>>> Author: Xishi Qiu <[email protected]>
>>> Date: Fri Aug 14 15:35:16 2015 -0700
>>>
>>> memory-hotplug: fix wrong edge when hot add a new node
>>>
>>> But there is a problem:
>>>
>>> add_memory()
>>> |--> hotadd_new_pgdat()
>>> |--> free_area_init_node()
>>> |--> get_pfn_range_for_nid()
>>> |--> find start_pfn and end_pfn in memblock
>>> |--> ......
>>> |--> memblock_add_node(start, size, nid) -------- Here, just too late.
>>>
>>> get_pfn_range_for_nid() will find that start_pfn and end_pfn are both 0.
>>> As a result, when adding memory, dmesg will give the following wrong message.
>>>
> Hi Tang,
>
> Another question, if we add cpu first, there will be print error too.
>
> cpu_up()
> try_online_node()
> hotadd_new_pgdat()
>
> So how about just skip the print if the size is empty or just print
> "node xx is empty now, will update when online memory"?
As Liu Jiang said, memory-less node is not supported on x86 now.
And he is working on it.
Please refer to https://lkml.org/lkml/2015/8/16/130.
About your question, now, node could only be onlined when it has some
memory.
So the printed message is also about memory, and sis put in
hotadd_new_pgdat() .
I think the author of the code didn't think about online a node when a
CPU is up.
But now, memory-less will be supported. So, I think, as you said, the
message should
be modified.
But how it will go, I think we should refer to Liu Jiang's patch, and
make a decision.
Thanks.
>
> Thanks,
> Xishi Qiu
>
>>> [ 2007.577000] Initmem setup node 5 [mem 0x0000000000000000-0xffffffffffffffff]
>>> [ 2007.584000] On node 5 totalpages: 0
>>> [ 2007.585000] Built 5 zonelists in Node order, mobility grouping on. Total pages: 32588823
>>> [ 2007.594000] Policy zone: Normal
>>> [ 2007.598000] init_memory_mapping: [mem 0x60000000000-0x607ffffffff]
>>>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
> .
>