Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 31 May 2023 11:25:43 -0500
From:   Steve Wahl <steve.wahl@hpe.com>
To:     Steve Wahl <steve.wahl@hpe.com>,
        Dimitri Sivanich <dimitri.sivanich@hpe.com>,
        Russ Anderson <russ.anderson@hpe.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H. Peter Anvin" <hpa@zytor.com>, linux-kernel@vger.kernel.org
Subject: Re: [PATCH RESEND v5 0/8] Support sub-NUMA clustering on UV
Message-ID: <20230531162543.GA1008147@dog.eag.rdlabs.hpecorp.net>
References: <20230519190752.3297140-1-steve.wahl@hpe.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20230519190752.3297140-1-steve.wahl@hpe.com>
User-Agent: Mutt/1.10.1 (2018-07-13)
Precedence: bulk

On Fri, May 19, 2023 at 02:07:44PM -0500, Steve Wahl wrote:
> Sub-NUMA clustering (SNC) invalidates previous assumptions of a 1:1
> relationship between blades, sockets, and nodes.  Make changes
> necessary to support this, including some general improvements along the
> way.
> 
> Steve Wahl (8):
>   x86/platform/uv: Add platform resolving #defines for misc
>     GAM_MMIOH_REDIRECT*
>   x86/platform/uv: Introduce helper function uv_pnode_to_socket.
>   x86/platform/uv: Fix incorrect mask define use in calc_mmioh_map
>   x86/platform/uv: When searching for minimums, start at INT_MAX not
>     99999
>   x86/platform/uv: helper functions for allocating and freeing
>     conversion tables
>   x86/platform/uv: UV support for sub-NUMA clustering
>   x86/platform/uv: Remove remaining BUG_ON() and BUG() calls
>   x86/platform/uv: Update UV[23] platform code for SNC
> 
>  arch/x86/include/asm/uv/uv_hub.h   |  32 ++-
>  arch/x86/include/asm/uv/uv_mmrs.h  |  18 +-
>  arch/x86/kernel/apic/x2apic_uv_x.c | 318 ++++++++++++++++++-----------
>  3 files changed, 232 insertions(+), 136 deletions(-)
> 
> -- 
> v2: Include the subsystem name (x86/platform/uv:) on the first line of
> the commit (subject line).
> 
> v3: Use return from WARN_ON_ONCE() to simplify the code, remove memory
> leaks from memory allocation failure conditions, and remove an
> unnecessary set of curly braces, per comments from Ingo Molnar.
> 
> v4: Testing found that some configurations required adding a check for
> "uv_blade_to_node(bid) == SOCK_EMPTY" in uv_system_init_hub().
> Testing also found that some uv3 configurations didn't work correctly
> with the new blade number handling.  The range of blade numbers is now
> needed, not just the count, so calc_mmioh_map() and
> boot_init_possible_blades() needed some adjustment.  And a reviewer
> suggested using INT_MAX instead of 999999.
> 
> v5: Break up a single patch into smaller patches and re-factor a bit to
> make the changes more understandable.  Unfortunately, the fundamental
> change of the relationship between nodes, pnodes, and hubs touches a
> lot of places that need to be changed at the same time, so patch 6 is
> still rather large.

Gentle ping.  We would like to get this in before the hardware it
supports (Sapphire Rapids based) ships.  If there's some other avenue
I should be taking, please let me know!

Thanks,

--> Steve Wahl,  steve.wahl@hpe.com