This is v3 of the series.
I based it off of Linus's latest tree.
I did not include David Rientjes's "mm: slab allocate memory section nodemask
for large systems" patch in my series, since it's not necessarily related.
Please consider for inclusion for the next merge window (v2.6.33).
Thanks,
/ac
v2 -> v3:
- rebased to Linus's latest tree (799dd75b)
- Added David Rientjes's Acked-by: flags
- dropped S390 cc's, since they are unaffected by this series
v1 -> v2: http://thread.gmane.org/gmane.linux.kernel.mm/40084/
Address David Rientjes's comments
- check return value of sysfs_create_link in register_cpu_under_node
- do /not/ convert [un]register_cpu_under_node to return void, since
sparse starts whinging if you ignore sysfs_create_link()'s return
value and working around sparse makes the code ugly
- adjust documentation
Added S390 maintainers to cc: for patch [1/5] as per Kame-san's
suggestion. S390 may map a memory section to more than one node,
causing this series to break.
---
Alex Chiang (5):
mm: add numa node symlink for memory section in sysfs
mm: refactor register_cpu_under_node()
mm: refactor unregister_cpu_under_node()
mm: add numa node symlink for cpu devices in sysfs
Documentation: ABI: /sys/devices/system/cpu/cpu#/node
Documentation/ABI/testing/sysfs-devices-memory | 14 ++++-
Documentation/ABI/testing/sysfs-devices-system-cpu | 14 +++++
Documentation/memory-hotplug.txt | 11 ++--
drivers/base/node.c | 58 ++++++++++++++------
4 files changed, 76 insertions(+), 21 deletions(-)
Commit c04fc586c (mm: show node to memory section relationship with
symlinks in sysfs) created symlinks from nodes to memory sections, e.g.
/sys/devices/system/node/node1/memory135 -> ../../memory/memory135
If you're examining the memory section though and are wondering what
node it might belong to, you can find it by grovelling around in
sysfs, but it's a little cumbersome.
Add a reverse symlink for each memory section that points back to the
node to which it belongs.
Cc: Gary Hade <[email protected]>
Cc: Badari Pulavarty <[email protected]>
Cc: Ingo Molnar <[email protected]>
Acked-by: David Rientjes <[email protected]>
Signed-off-by: Alex Chiang <[email protected]>
---
Documentation/ABI/testing/sysfs-devices-memory | 14 +++++++++++++-
Documentation/memory-hotplug.txt | 11 +++++++----
drivers/base/node.c | 11 ++++++++++-
3 files changed, 30 insertions(+), 6 deletions(-)
diff --git a/Documentation/ABI/testing/sysfs-devices-memory b/Documentation/ABI/testing/sysfs-devices-memory
index 9fe91c0..bf1627b 100644
--- a/Documentation/ABI/testing/sysfs-devices-memory
+++ b/Documentation/ABI/testing/sysfs-devices-memory
@@ -60,6 +60,19 @@ Description:
Users: hotplug memory remove tools
https://w3.opensource.ibm.com/projects/powerpc-utils/
+
+What: /sys/devices/system/memoryX/nodeY
+Date: October 2009
+Contact: Linux Memory Management list <[email protected]>
+Description:
+ When CONFIG_NUMA is enabled, a symbolic link that
+ points to the corresponding NUMA node directory.
+
+ For example, the following symbolic link is created for
+ memory section 9 on node0:
+ /sys/devices/system/memory/memory9/node0 -> ../../node/node0
+
+
What: /sys/devices/system/node/nodeX/memoryY
Date: September 2008
Contact: Gary Hade <[email protected]>
@@ -70,4 +83,3 @@ Description:
memory section directory. For example, the following symbolic
link is created for memory section 9 on node0.
/sys/devices/system/node/node0/memory9 -> ../../memory/memory9
-
diff --git a/Documentation/memory-hotplug.txt b/Documentation/memory-hotplug.txt
index bbc8a6a..57e7e9c 100644
--- a/Documentation/memory-hotplug.txt
+++ b/Documentation/memory-hotplug.txt
@@ -160,12 +160,15 @@ Under each section, you can see 4 files.
NOTE:
These directories/files appear after physical memory hotplug phase.
-If CONFIG_NUMA is enabled the
-/sys/devices/system/memory/memoryXXX memory section
-directories can also be accessed via symbolic links located in
-the /sys/devices/system/node/node* directories. For example:
+If CONFIG_NUMA is enabled the memoryXXX/ directories can also be accessed
+via symbolic links located in the /sys/devices/system/node/node* directories.
+
+For example:
/sys/devices/system/node/node0/memory9 -> ../../memory/memory9
+A backlink will also be created:
+/sys/devices/system/memory/memory9/node0 -> ../../node/node0
+
--------------------------------
4. Physical memory hot-add phase
--------------------------------
diff --git a/drivers/base/node.c b/drivers/base/node.c
index 1fe5536..3108b21 100644
--- a/drivers/base/node.c
+++ b/drivers/base/node.c
@@ -268,6 +268,7 @@ static int get_nid_for_pfn(unsigned long pfn)
/* register memory section under specified node if it spans that node */
int register_mem_sect_under_node(struct memory_block *mem_blk, int nid)
{
+ int ret;
unsigned long pfn, sect_start_pfn, sect_end_pfn;
if (!mem_blk)
@@ -284,9 +285,15 @@ int register_mem_sect_under_node(struct memory_block *mem_blk, int nid)
continue;
if (page_nid != nid)
continue;
- return sysfs_create_link_nowarn(&node_devices[nid].sysdev.kobj,
+ ret = sysfs_create_link_nowarn(&node_devices[nid].sysdev.kobj,
&mem_blk->sysdev.kobj,
kobject_name(&mem_blk->sysdev.kobj));
+ if (ret)
+ return ret;
+
+ return sysfs_create_link_nowarn(&mem_blk->sysdev.kobj,
+ &node_devices[nid].sysdev.kobj,
+ kobject_name(&node_devices[nid].sysdev.kobj));
}
/* mem section does not span the specified node */
return 0;
@@ -315,6 +322,8 @@ int unregister_mem_sect_under_nodes(struct memory_block *mem_blk)
continue;
sysfs_remove_link(&node_devices[nid].sysdev.kobj,
kobject_name(&mem_blk->sysdev.kobj));
+ sysfs_remove_link(&mem_blk->sysdev.kobj,
+ kobject_name(&node_devices[nid].sysdev.kobj));
}
return 0;
}
By returning early if the node is not online, we can unindent the
interesting code by one level.
No functional change.
Signed-off-by: Alex Chiang <[email protected]>
---
drivers/base/node.c | 20 +++++++++++---------
1 files changed, 11 insertions(+), 9 deletions(-)
diff --git a/drivers/base/node.c b/drivers/base/node.c
index 3108b21..ef7dd22 100644
--- a/drivers/base/node.c
+++ b/drivers/base/node.c
@@ -227,16 +227,18 @@ struct node node_devices[MAX_NUMNODES];
*/
int register_cpu_under_node(unsigned int cpu, unsigned int nid)
{
- if (node_online(nid)) {
- struct sys_device *obj = get_cpu_sysdev(cpu);
- if (!obj)
- return 0;
- return sysfs_create_link(&node_devices[nid].sysdev.kobj,
- &obj->kobj,
- kobject_name(&obj->kobj));
- }
+ struct sys_device *obj;
- return 0;
+ if (!node_online(nid))
+ return 0;
+
+ obj = get_cpu_sysdev(cpu);
+ if (!obj)
+ return 0;
+
+ return sysfs_create_link(&node_devices[nid].sysdev.kobj,
+ &obj->kobj,
+ kobject_name(&obj->kobj));
}
int unregister_cpu_under_node(unsigned int cpu, unsigned int nid)
By returning early if the node is not online, we can unindent the
interesting code by two levels.
No functional change.
Signed-off-by: Alex Chiang <[email protected]>
---
drivers/base/node.c | 18 ++++++++++++------
1 files changed, 12 insertions(+), 6 deletions(-)
diff --git a/drivers/base/node.c b/drivers/base/node.c
index ef7dd22..ffda067 100644
--- a/drivers/base/node.c
+++ b/drivers/base/node.c
@@ -243,12 +243,18 @@ int register_cpu_under_node(unsigned int cpu, unsigned int nid)
int unregister_cpu_under_node(unsigned int cpu, unsigned int nid)
{
- if (node_online(nid)) {
- struct sys_device *obj = get_cpu_sysdev(cpu);
- if (obj)
- sysfs_remove_link(&node_devices[nid].sysdev.kobj,
- kobject_name(&obj->kobj));
- }
+ struct sys_device *obj;
+
+ if (!node_online(nid))
+ return 0;
+
+ obj = get_cpu_sysdev(cpu);
+ if (!obj)
+ return 0;
+
+ sysfs_remove_link(&node_devices[nid].sysdev.kobj,
+ kobject_name(&obj->kobj));
+
return 0;
}
You can discover which CPUs belong to a NUMA node by examining
/sys/devices/system/node/node#/
However, it's not convenient to go in the other direction, when looking at
/sys/devices/system/cpu/cpu#/
Yes, you can muck about in sysfs, but adding these symlinks makes
life a lot more convenient.
Acked-by: David Rientjes <[email protected]>
Signed-off-by: Alex Chiang <[email protected]>
---
drivers/base/node.c | 11 ++++++++++-
1 files changed, 10 insertions(+), 1 deletions(-)
diff --git a/drivers/base/node.c b/drivers/base/node.c
index ffda067..24fa962 100644
--- a/drivers/base/node.c
+++ b/drivers/base/node.c
@@ -227,6 +227,7 @@ struct node node_devices[MAX_NUMNODES];
*/
int register_cpu_under_node(unsigned int cpu, unsigned int nid)
{
+ int ret;
struct sys_device *obj;
if (!node_online(nid))
@@ -236,9 +237,15 @@ int register_cpu_under_node(unsigned int cpu, unsigned int nid)
if (!obj)
return 0;
- return sysfs_create_link(&node_devices[nid].sysdev.kobj,
+ ret = sysfs_create_link(&node_devices[nid].sysdev.kobj,
&obj->kobj,
kobject_name(&obj->kobj));
+ if (ret)
+ return ret;
+
+ return sysfs_create_link(&obj->kobj,
+ &node_devices[nid].sysdev.kobj,
+ kobject_name(&node_devices[nid].sysdev.kobj));
}
int unregister_cpu_under_node(unsigned int cpu, unsigned int nid)
@@ -254,6 +261,8 @@ int unregister_cpu_under_node(unsigned int cpu, unsigned int nid)
sysfs_remove_link(&node_devices[nid].sysdev.kobj,
kobject_name(&obj->kobj));
+ sysfs_remove_link(&obj->kobj,
+ kobject_name(&node_devices[nid].sysdev.kobj));
return 0;
}
Describe NUMA node symlink created for CPUs when CONFIG_NUMA is set.
Cc: Greg KH <[email protected]>
Cc: Randy Dunlap <[email protected]>
Signed-off-by: Alex Chiang <[email protected]>
---
Documentation/ABI/testing/sysfs-devices-system-cpu | 14 ++++++++++++++
1 files changed, 14 insertions(+), 0 deletions(-)
diff --git a/Documentation/ABI/testing/sysfs-devices-system-cpu b/Documentation/ABI/testing/sysfs-devices-system-cpu
index 5aace16..1671634 100644
--- a/Documentation/ABI/testing/sysfs-devices-system-cpu
+++ b/Documentation/ABI/testing/sysfs-devices-system-cpu
@@ -94,6 +94,20 @@ Description: Discover and change the online state of a CPU.
For more information, please read Documentation/cpu-hotplug.txt
+What: /sys/devices/system/cpu/cpu#/node
+Date: October 2009
+Contact: Linux memory management mailing list <[email protected]>
+Description: Discover NUMA node a CPU belongs to
+
+ When CONFIG_NUMA is enabled, a symbolic link that points
+ to the corresponding NUMA node directory.
+
+ For example, the following symlink is created for cpu42
+ in NUMA node 2:
+
+ /sys/devices/system/cpu/cpu42/node2 -> ../../node/node2
+
+
What: /sys/devices/system/cpu/cpu#/topology/core_id
/sys/devices/system/cpu/cpu#/topology/core_siblings
/sys/devices/system/cpu/cpu#/topology/core_siblings_list
On Tue, Nov 10, 2009 at 03:36:44PM -0700, Alex Chiang wrote:
> Commit c04fc586c (mm: show node to memory section relationship with
> symlinks in sysfs) created symlinks from nodes to memory sections, e.g.
>
> /sys/devices/system/node/node1/memory135 -> ../../memory/memory135
>
> If you're examining the memory section though and are wondering what
> node it might belong to, you can find it by grovelling around in
> sysfs, but it's a little cumbersome.
>
> Add a reverse symlink for each memory section that points back to the
> node to which it belongs.
Hi Alex,
I'm kinda late to the party but I finally had a chance to review
and try it out on one of our systems today. Looks good to me.
Tested-by: Gary Hade <[email protected]>
Acked-by: Gary Hade <[email protected]>
Gary
--
Gary Hade
System x Enablement
IBM Linux Technology Center
503-578-4503 IBM T/L: 775-4503
[email protected]
http://www.ibm.com/linux/ltc