Sub-NUMA clustering (SNC) invalidates previous assumptions of a 1:1
relationship between blades, sockets, and nodes. Make changes
necessary to support this, including some general improvements along the
way.
Steve Wahl (8):
x86/platform/uv: Add platform resolving #defines for misc
GAM_MMIOH_REDIRECT*
x86/platform/uv: Introduce helper function uv_pnode_to_socket.
x86/platform/uv: Fix incorrect mask define use in calc_mmioh_map
x86/platform/uv: When searching for minimums, start at INT_MAX not
99999
x86/platform/uv: helper functions for allocating and freeing
conversion tables
x86/platform/uv: UV support for sub-NUMA clustering
x86/platform/uv: Remove remaining BUG_ON() and BUG() calls
x86/platform/uv: Update UV[23] platform code for SNC
arch/x86/include/asm/uv/uv_hub.h | 32 ++-
arch/x86/include/asm/uv/uv_mmrs.h | 18 +-
arch/x86/kernel/apic/x2apic_uv_x.c | 318 ++++++++++++++++++-----------
3 files changed, 232 insertions(+), 136 deletions(-)
--
v2: Include the subsystem name (x86/platform/uv:) on the first line of
the commit (subject line).
v3: Use return from WARN_ON_ONCE() to simplify the code, remove memory
leaks from memory allocation failure conditions, and remove an
unnecessary set of curly braces, per comments from Ingo Molnar.
v4: Testing found that some configurations required adding a check for
"uv_blade_to_node(bid) == SOCK_EMPTY" in uv_system_init_hub().
Testing also found that some uv3 configurations didn't work correctly
with the new blade number handling. The range of blade numbers is now
needed, not just the count, so calc_mmioh_map() and
boot_init_possible_blades() needed some adjustment. And a reviewer
suggested using INT_MAX instead of 999999.
v5: Break up a single patch into smaller patches and re-factor a bit to
make the changes more understandable. Unfortunately, the fundamental
change of the relationship between nodes, pnodes, and hubs touches a
lot of places that need to be changed at the same time, so patch 6 is
still rather large.
2.26.2
On Fri, May 19, 2023 at 02:07:44PM -0500, Steve Wahl wrote:
> Sub-NUMA clustering (SNC) invalidates previous assumptions of a 1:1
> relationship between blades, sockets, and nodes. Make changes
> necessary to support this, including some general improvements along the
> way.
>
> Steve Wahl (8):
> x86/platform/uv: Add platform resolving #defines for misc
> GAM_MMIOH_REDIRECT*
> x86/platform/uv: Introduce helper function uv_pnode_to_socket.
> x86/platform/uv: Fix incorrect mask define use in calc_mmioh_map
> x86/platform/uv: When searching for minimums, start at INT_MAX not
> 99999
> x86/platform/uv: helper functions for allocating and freeing
> conversion tables
> x86/platform/uv: UV support for sub-NUMA clustering
> x86/platform/uv: Remove remaining BUG_ON() and BUG() calls
> x86/platform/uv: Update UV[23] platform code for SNC
>
> arch/x86/include/asm/uv/uv_hub.h | 32 ++-
> arch/x86/include/asm/uv/uv_mmrs.h | 18 +-
> arch/x86/kernel/apic/x2apic_uv_x.c | 318 ++++++++++++++++++-----------
> 3 files changed, 232 insertions(+), 136 deletions(-)
>
> --
> v2: Include the subsystem name (x86/platform/uv:) on the first line of
> the commit (subject line).
>
> v3: Use return from WARN_ON_ONCE() to simplify the code, remove memory
> leaks from memory allocation failure conditions, and remove an
> unnecessary set of curly braces, per comments from Ingo Molnar.
>
> v4: Testing found that some configurations required adding a check for
> "uv_blade_to_node(bid) == SOCK_EMPTY" in uv_system_init_hub().
> Testing also found that some uv3 configurations didn't work correctly
> with the new blade number handling. The range of blade numbers is now
> needed, not just the count, so calc_mmioh_map() and
> boot_init_possible_blades() needed some adjustment. And a reviewer
> suggested using INT_MAX instead of 999999.
>
> v5: Break up a single patch into smaller patches and re-factor a bit to
> make the changes more understandable. Unfortunately, the fundamental
> change of the relationship between nodes, pnodes, and hubs touches a
> lot of places that need to be changed at the same time, so patch 6 is
> still rather large.
Gentle ping. We would like to get this in before the hardware it
supports (Sapphire Rapids based) ships. If there's some other avenue
I should be taking, please let me know!
Thanks,
--> Steve Wahl, [email protected]