Received-SPF: pass (google.com: domain of linux-kernel+bounces-166722-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Date: Thu, 2 May 2024 17:27:42 +0100
From: Jonathan Cameron <Jonathan.Cameron@huawei.com>
To: Robert Richter <rrichter@amd.com>
CC: "Rafael J. Wysocki" <rafael@kernel.org>, Thomas Gleixner
	<tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, Borislav Petkov
	<bp@alien8.de>, Dave Hansen <dave.hansen@linux.intel.com>, <x86@kernel.org>,
	Andy Lutomirski <luto@kernel.org>, Peter Zijlstra <peterz@infradead.org>,
	Alison Schofield <alison.schofield@intel.com>, "Dan Williams"
	<dan.j.williams@intel.com>, <linux-acpi@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>, <linux-cxl@vger.kernel.org>, Derick Marks
	<derick.w.marks@intel.com>, "H. Peter Anvin" <hpa@zytor.com>, Len Brown
	<lenb@kernel.org>
Subject: Re: [PATCH v6 1/7] x86/numa: Fix SRAT lookup of CFMWS ranges with
 numa_fill_memblks()
Message-ID: <20240502172742.00002823@huawei.com>
In-Reply-To: <ZjOAOGInZDO65b_T@rric.localdomain>
References: <20240430092200.2335887-1-rrichter@amd.com>
	<20240430092200.2335887-2-rrichter@amd.com>
	<20240430154856.00006d15@Huawei.com>
	<ZjOAOGInZDO65b_T@rric.localdomain>
Organization: Huawei Technologies R&D (UK) Ltd.
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset="US-ASCII"
Content-Transfer-Encoding: 7bit

On Thu, 2 May 2024 13:59:52 +0200
Robert Richter <rrichter@amd.com> wrote:

> On 30.04.24 15:48:56, Jonathan Cameron wrote:
> > On Tue, 30 Apr 2024 11:21:54 +0200
> > Robert Richter <rrichter@amd.com> wrote:
> >   
> > > For configurations that have the kconfig option NUMA_KEEP_MEMINFO
> > > disabled numa_fill_memblks() only returns with NUMA_NO_MEMBLK (-1).
> > > SRAT lookup fails then because an existing SRAT memory range cannot be
> > > found for a CFMWS address range. This causes the addition of a
> > > duplicate numa_memblk with a different node id and a subsequent page
> > > fault and kernel crash during boot.
> > > 
> > > Fix this by making numa_fill_memblks() always available regardless of
> > > NUMA_KEEP_MEMINFO.
> > > 
> > > The fix also removes numa_fill_memblks() from sparsemem.h using
> > > __weak.
> > > 
> > > From Dan:
> > > 
> > > """
> > > It just feels like numa_fill_memblks() has absolutely no business being
> > > defined in arch/x86/include/asm/sparsemem.h.
> > > 
> > > The only use for numa_fill_memblks() is to arrange for NUMA nodes to be
> > > applied to memory ranges hot-onlined by the CXL driver.
> > > 
> > > It belongs right next to numa_add_memblk(), and I suspect
> > > arch/x86/include/asm/sparsemem.h was only chosen to avoid figuring out
> > > what to do about the fact that linux/numa.h does not include asm/numa.h
> > > and that all implementations either provide numa_add_memblk() or select
> > > the generic implementation.
> > > 
> > > So I would prefer that this do the proper fix and get
> > > numa_fill_memblks() completely out of the sparsemem.h path.
> > > 
> > > Something like the following which boots for me.
> > > """
> > > 
> > > Note that the issue was initially introduced with [1]. But since
> > > phys_to_target_node() was originally used that returned the valid node
> > > 0, an additional numa_memblk was not added. Though, the node id was
> > > wrong too, a message is seen then in the logs:
> > > 
> > >  kernel/numa.c:  pr_info_once("Unknown target node for memory at 0x%llx, assuming node 0\n",
> > > 
> > > [1] commit fd49f99c1809 ("ACPI: NUMA: Add a node and memblk for each
> > >     CFMWS not in SRAT")
> > > 
> > > Suggested-by: Dan Williams <dan.j.williams@intel.com>
> > > Link: https://lore.kernel.org/all/66271b0072317_69102944c@dwillia2-xfh.jf.intel.com.notmuch/
> > > Fixes: 8f1004679987 ("ACPI/NUMA: Apply SRAT proximity domain to entire CFMWS window")
> > > Cc: Derick Marks <derick.w.marks@intel.com>
> > > Cc: Dan Williams <dan.j.williams@intel.com>
> > > Cc: Alison Schofield <alison.schofield@intel.com>
> > > Signed-off-by: Robert Richter <rrichter@amd.com>  
> > 
> > Whilst I'm not particularly keen on an arch specific solution for this
> > and the stub is effectively pointless beyond making the build work, I guess
> > this works well enough for now.
> > 
> > Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> > 
> > I was aiming to post the ARM64 handling this cycle but it hasn't quite happened yet :(
> > Maybe we can look at whether there is a better level share at than
> > the whole function once that is done.  
> 
> Thanks for review.
> 
> It seems better to change x86 to use the generic implementation of
> numa_add_memblk() in drivers/base/arch_numa.c. That already contains
> code to deal with and merge overlapping blocks, it also checks memory
> attributes. But that is not scope of this patch.
> 
There is some history that Dan pointed me at a while back... Maybe this one is fine
but in general people have tried and given up on unifying x86 memblock handling
with the generic version :(
https://lore.kernel.org/linux-mm/159457121480.754248.17292511837648775358.stgit@dwillia2-desk3.amr.corp.intel.com/

> -Robert