This fixies a few issues when I tried to enable pmem as RAM device on arm64.
To use memory_add_physaddr_to_nid as a fallback nid, it would be better
implement a general version (__weak) in mm/memory_hotplug. After that, arm64/
sh/s390 can simply use the general version, and PowerPC/ia64/x86 will use
arch specific version.
Tested on ThunderX2 host/qemu "-M virt" guest with a nvdimm device. The
memblocks from the dax pmem device can be either hot-added or hot-removed
on arm64 guest. Also passed the compilation test on x86.
Changes:
v4: - remove "device-dax: use fallback nid when numa_node is invalid", wait
for Dan Williams' phys_addr_to_target_node() patch
- folder v3 patch1-4 into single one, no functional changes
v3: https://lkml.org/lkml/2020/7/8/1541
- introduce general version memory_add_physaddr_to_nid, refine the arch
specific one
- fix an uninitialization bug in v2 device-dax patch
v2: https://lkml.org/lkml/2020/7/7/71
- Drop unnecessary patch to harden try_offline_node
- Use new solution(by David) to fix dev->target_node=-1 during probing
- Refine the mem_hotplug_begin/done patch
v1: https://lkml.org/lkml/2020/7/5/381
Jia He (2):
mm/memory_hotplug: introduce default dummy
memory_add_physaddr_to_nid()
mm/memory_hotplug: fix unpaired mem_hotplug_begin/done
arch/arm64/mm/numa.c | 10 ----------
arch/ia64/mm/numa.c | 2 --
arch/sh/mm/init.c | 9 ---------
arch/x86/mm/numa.c | 1 -
mm/memory_hotplug.c | 15 ++++++++++++---
5 files changed, 12 insertions(+), 25 deletions(-)
--
2.17.1
This is to introduce a general dummy helper. memory_add_physaddr_to_nid()
is a fallback option to get the nid in case NUMA_NO_NID is detected.
After this patch, arm64/sh/s390 can simply use the general dummy version.
PowerPC/x86/ia64 will still use their specific version.
This is the preparation to set a fallback value for dev_dax->target_node.
Reviewed-by: David Hildenbrand <[email protected]>
Signed-off-by: Jia He <[email protected]>
---
arch/arm64/mm/numa.c | 10 ----------
arch/ia64/mm/numa.c | 2 --
arch/sh/mm/init.c | 9 ---------
arch/x86/mm/numa.c | 1 -
mm/memory_hotplug.c | 10 ++++++++++
5 files changed, 10 insertions(+), 22 deletions(-)
diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
index aafcee3e3f7e..73f8b49d485c 100644
--- a/arch/arm64/mm/numa.c
+++ b/arch/arm64/mm/numa.c
@@ -461,13 +461,3 @@ void __init arm64_numa_init(void)
numa_init(dummy_numa_init);
}
-
-/*
- * We hope that we will be hotplugging memory on nodes we already know about,
- * such that acpi_get_node() succeeds and we never fall back to this...
- */
-int memory_add_physaddr_to_nid(u64 addr)
-{
- pr_warn("Unknown node for memory at 0x%llx, assuming node 0\n", addr);
- return 0;
-}
diff --git a/arch/ia64/mm/numa.c b/arch/ia64/mm/numa.c
index 5e1015eb6d0d..f34964271101 100644
--- a/arch/ia64/mm/numa.c
+++ b/arch/ia64/mm/numa.c
@@ -106,7 +106,5 @@ int memory_add_physaddr_to_nid(u64 addr)
return 0;
return nid;
}
-
-EXPORT_SYMBOL_GPL(memory_add_physaddr_to_nid);
#endif
#endif
diff --git a/arch/sh/mm/init.c b/arch/sh/mm/init.c
index a70ba0fdd0b3..f75932ba87a6 100644
--- a/arch/sh/mm/init.c
+++ b/arch/sh/mm/init.c
@@ -430,15 +430,6 @@ int arch_add_memory(int nid, u64 start, u64 size,
return ret;
}
-#ifdef CONFIG_NUMA
-int memory_add_physaddr_to_nid(u64 addr)
-{
- /* Node 0 for now.. */
- return 0;
-}
-EXPORT_SYMBOL_GPL(memory_add_physaddr_to_nid);
-#endif
-
void arch_remove_memory(int nid, u64 start, u64 size,
struct vmem_altmap *altmap)
{
diff --git a/arch/x86/mm/numa.c b/arch/x86/mm/numa.c
index 8ee952038c80..2a6e62af4636 100644
--- a/arch/x86/mm/numa.c
+++ b/arch/x86/mm/numa.c
@@ -929,5 +929,4 @@ int memory_add_physaddr_to_nid(u64 start)
nid = numa_meminfo.blk[0].nid;
return nid;
}
-EXPORT_SYMBOL_GPL(memory_add_physaddr_to_nid);
#endif
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index da374cd3d45b..b49ab743d914 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -350,6 +350,16 @@ int __ref __add_pages(int nid, unsigned long pfn, unsigned long nr_pages,
return err;
}
+#ifdef CONFIG_NUMA
+int __weak memory_add_physaddr_to_nid(u64 start)
+{
+ pr_info_once("Unknown target node for memory at 0x%llx, assuming node 0\n",
+ start);
+ return 0;
+}
+EXPORT_SYMBOL_GPL(memory_add_physaddr_to_nid);
+#endif
+
/* find the smallest valid pfn in the range [start_pfn, end_pfn) */
static unsigned long find_smallest_section_pfn(int nid, struct zone *zone,
unsigned long start_pfn,
--
2.17.1
When check_memblock_offlined_cb() returns failed rc(e.g. the memblock is
online at that time), mem_hotplug_begin/done is unpaired in such case.
Therefore a warning:
Call Trace:
percpu_up_write+0x33/0x40
try_remove_memory+0x66/0x120
? _cond_resched+0x19/0x30
remove_memory+0x2b/0x40
dev_dax_kmem_remove+0x36/0x72 [kmem]
device_release_driver_internal+0xf0/0x1c0
device_release_driver+0x12/0x20
bus_remove_device+0xe1/0x150
device_del+0x17b/0x3e0
unregister_dev_dax+0x29/0x60
devm_action_release+0x15/0x20
release_nodes+0x19a/0x1e0
devres_release_all+0x3f/0x50
device_release_driver_internal+0x100/0x1c0
driver_detach+0x4c/0x8f
bus_remove_driver+0x5c/0xd0
driver_unregister+0x31/0x50
dax_pmem_exit+0x10/0xfe0 [dax_pmem]
Fixes: f1037ec0cc8a ("mm/memory_hotplug: fix remove_memory() lockdep splat")
Cc: [email protected] # v5.6+
Signed-off-by: Jia He <[email protected]>
Reviewed-by: David Hildenbrand <[email protected]>
Acked-by: Michal Hocko <[email protected]>
Acked-by: Dan Williams <[email protected]>
---
mm/memory_hotplug.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index b49ab743d914..3e0645387daf 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1752,7 +1752,7 @@ static int __ref try_remove_memory(int nid, u64 start, u64 size)
*/
rc = walk_memory_blocks(start, size, NULL, check_memblock_offlined_cb);
if (rc)
- goto done;
+ return rc;
/* remove memmap entry */
firmware_map_remove(start, start + size, "System RAM");
@@ -1776,9 +1776,8 @@ static int __ref try_remove_memory(int nid, u64 start, u64 size)
try_offline_node(nid);
-done:
mem_hotplug_done();
- return rc;
+ return 0;
}
/**
--
2.17.1
On 10.07.20 05:16, Jia He wrote:
> This fixies a few issues when I tried to enable pmem as RAM device on arm64.
>
> To use memory_add_physaddr_to_nid as a fallback nid, it would be better
> implement a general version (__weak) in mm/memory_hotplug. After that, arm64/
> sh/s390 can simply use the general version, and PowerPC/ia64/x86 will use
> arch specific version.
>
> Tested on ThunderX2 host/qemu "-M virt" guest with a nvdimm device. The
> memblocks from the dax pmem device can be either hot-added or hot-removed
> on arm64 guest. Also passed the compilation test on x86.
>
> Changes:
> v4: - remove "device-dax: use fallback nid when numa_node is invalid", wait
> for Dan Williams' phys_addr_to_target_node() patch
So, this series no longer does what it promises? "Fix and enable pmem as
RAM device on arm64"
--
Thanks,
David / dhildenb
Hi David
> -----Original Message-----
> From: David Hildenbrand <[email protected]>
> Sent: Friday, July 10, 2020 4:30 PM
> To: Justin He <[email protected]>; Catalin Marinas
> <[email protected]>; Will Deacon <[email protected]>; Tony Luck
> <[email protected]>; Fenghua Yu <[email protected]>; Yoshinori Sato
> <[email protected]>; Rich Felker <[email protected]>; Dave Hansen
> <[email protected]>; Andy Lutomirski <[email protected]>; Peter
> Zijlstra <[email protected]>; Thomas Gleixner <[email protected]>;
> Ingo Molnar <[email protected]>; Borislav Petkov <[email protected]>
> Cc: [email protected]; H. Peter Anvin <[email protected]>; Dan Williams
> <[email protected]>; Vishal Verma <[email protected]>; Dave
> Jiang <[email protected]>; Andrew Morton <[email protected]>;
> Baoquan He <[email protected]>; Chuhong Yuan <[email protected]>; Mike
> Rapoport <[email protected]>; Logan Gunthorpe <[email protected]>;
> Masahiro Yamada <[email protected]>; Michal Hocko <[email protected]>;
> [email protected]; [email protected]; linux-
> [email protected]; [email protected]; [email protected];
> [email protected]; Jonathan Cameron <[email protected]>; Kaly
> Xin <[email protected]>
> Subject: Re: [PATCH v4 0/2] Fix and enable pmem as RAM device on arm64
>
> On 10.07.20 05:16, Jia He wrote:
> > This fixies a few issues when I tried to enable pmem as RAM device on
> arm64.
> >
> > To use memory_add_physaddr_to_nid as a fallback nid, it would be better
> > implement a general version (__weak) in mm/memory_hotplug. After that,
> arm64/
> > sh/s390 can simply use the general version, and PowerPC/ia64/x86 will
> use
> > arch specific version.
> >
> > Tested on ThunderX2 host/qemu "-M virt" guest with a nvdimm device. The
> > memblocks from the dax pmem device can be either hot-added or hot-
> removed
> > on arm64 guest. Also passed the compilation test on x86.
> >
> > Changes:
> > v4: - remove "device-dax: use fallback nid when numa_node is invalid",
> wait
> > for Dan Williams' phys_addr_to_target_node() patch
>
> So, this series no longer does what it promises? "Fix and enable pmem as
> RAM device on arm64"
>
Hmm, a little bit awkward but seems no long what it promises. How about
sending patch1 patch2 individually without this cover-letter?
--
Cheers,
Justin (Jia He)
On Thu, Jul 9, 2020 at 8:17 PM Jia He <[email protected]> wrote:
>
> This fixies a few issues when I tried to enable pmem as RAM device on arm64.
>
> To use memory_add_physaddr_to_nid as a fallback nid, it would be better
> implement a general version (__weak) in mm/memory_hotplug. After that, arm64/
> sh/s390 can simply use the general version, and PowerPC/ia64/x86 will use
> arch specific version.
>
> Tested on ThunderX2 host/qemu "-M virt" guest with a nvdimm device. The
> memblocks from the dax pmem device can be either hot-added or hot-removed
> on arm64 guest. Also passed the compilation test on x86.
>
> Changes:
> v4: - remove "device-dax: use fallback nid when numa_node is invalid", wait
> for Dan Williams' phys_addr_to_target_node() patch
FWIW, I put these patches through a 0-day run overnight and will be
posting them today.