Received-SPF: pass (google.com: domain of linux-kernel+bounces-166418-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Date: Thu, 2 May 2024 13:59:52 +0200
From: Robert Richter <rrichter@amd.com>
To: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Cc: "Rafael J. Wysocki" <rafael@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
	Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
	Andy Lutomirski <luto@kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Alison Schofield <alison.schofield@intel.com>,
	Dan Williams <dan.j.williams@intel.com>, linux-acpi@vger.kernel.org,
	linux-kernel@vger.kernel.org, linux-cxl@vger.kernel.org,
	Derick Marks <derick.w.marks@intel.com>,
	"H. Peter Anvin" <hpa@zytor.com>, Len Brown <lenb@kernel.org>
Subject: Re: [PATCH v6 1/7] x86/numa: Fix SRAT lookup of CFMWS ranges with
 numa_fill_memblks()
Message-ID: <ZjOAOGInZDO65b_T@rric.localdomain>
References: <20240430092200.2335887-1-rrichter@amd.com>
 <20240430092200.2335887-2-rrichter@amd.com>
 <20240430154856.00006d15@Huawei.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240430154856.00006d15@Huawei.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?1Vrh4rn0exeAhm4lYZTajKIurXEn8S8vxmwraKDEpWTMzR/fEZ7Jt1mF7+BV?=
 =?us-ascii?Q?Nmbk0q8rA8ov1T588piJ4uWGu0YFNowPjgecNSDJW1Zfyz6HHsKOtna8elBp?=
 =?us-ascii?Q?x6eNYybV3ARNJ3tOHBcL0IuGpMdQkeeoapiCDF5cTvtH/hHnNw90NUcIgfP1?=
 =?us-ascii?Q?csyREaVXHFtp/a8ZSME51+tEm4RtaluVs9E8D7ACsyI0zAF53ALhUhyWga0d?=
 =?us-ascii?Q?a3qEFE4ZkEi56Lu9AqDcM5wnzlCRCxFsgxzIb6cwATJPwPRg7hcpeGajENSa?=
 =?us-ascii?Q?n+2s+LSmhpLmXmprYe3rmCex3qaHKlzyEC9tYdBYeTNycYbPvXMuI9yNqFXm?=
 =?us-ascii?Q?2/LhYdm9bWjvneKzZma1SHij9udQXHaD40XDJ96pVWhgAds2LaKO9MtNrn0i?=
 =?us-ascii?Q?j++oR8Y0gqO+kDHZny1tx6A2tWSzFlNPSUdsPLpewCqvJCM6YOUlogRZYU5Z?=
 =?us-ascii?Q?aLhyjz0EXQEJyJ+rXH5OQGOvr7M9u1sUOJx12RA9bCBsKA8neI1c8JG/ziqm?=
 =?us-ascii?Q?cYN7CqYPfj4o7bzRlYC0sPTx1wDqRnrhB+ap0LGNxnn2U7pfB3mQsnmzOxw4?=
 =?us-ascii?Q?/UX6awOSV4Tv/nWO6zJQUTlqKbay6AlA308az4CNFMUl+LbhWu0erHY3YEd+?=
 =?us-ascii?Q?VNXtzfZXForNxRehF7UrRkItimgFylnk1b2greIyNsvf3NFJuS3TziDbuwHV?=
 =?us-ascii?Q?X5ywmVp8I4W4zhiUP6G0X0mTP3Sb4M3P8JIibfO/t8ud+dfCbgJzDMVBGknu?=
 =?us-ascii?Q?VnpLVKLyn8B8ESKiCZVfpno9GwMX9D+AJIJW0y4tbqno8U9KMrA/cwihHRIF?=
 =?us-ascii?Q?eqvhlaQ16qMmFC2M9htImWoXjYJWyodaMgz39i11jW7IM+rA0LX3oCh9gEy9?=
 =?us-ascii?Q?eWjLDmtdqyhEJvUyRqaiqfqmxK3SiM2QGH6VF5ZkwivHK2Y/DG7EBUJ4J92Z?=
 =?us-ascii?Q?IQrO1XiXQanFij1tbZkQ//r5FZzWLFYmIXswLK8ePx2SqiT8uRHGx0kOZkvq?=
 =?us-ascii?Q?YqRkwkM0QrhbpoUX9hR9/LZkh0pkaXxiaNRvWkA8F4B2XQTVL4zqdnd+6hj8?=
 =?us-ascii?Q?9YHhYbUK2wTVA6jwTUyf1SQaYAeUBuFP3bb+XD4wUdRXw3sFvSC/Oq56eq5h?=
 =?us-ascii?Q?xaeKzFCdy995U1J83C3NXYn6EboAvSKV2zXDEawA93+a7hXRYqK2ZsaufM8i?=
 =?us-ascii?Q?EmiSKfpVS9hXhlW5QdiF+73S4EuqVR5RdZIR+a+B2Ta7Qd9ixPLjMNYaSx4J?=
 =?us-ascii?Q?GbzLlhpwADVADiZIqBzpObpMAMd82qu0eIMc5qEmbtfEym328S2a2VxG5it9?=
 =?us-ascii?Q?PQodnPY8L9Qa3HIt+V3IR61TPZtRCAZ7lVFsvf3J3g0TRdHZPmt6Q+Gh8hK0?=
 =?us-ascii?Q?muDnnRRBbbIUUY2lcF/lwf32eh4I/oya9EI8/Kc8Zigt7UlyZiLBTUUN7lPu?=
 =?us-ascii?Q?mwe4oGl6U5gAWEoEVP9dGCEjaLbrzWCFjlGpNvnMCnPMQNRtVbG+oLHgNxf+?=
 =?us-ascii?Q?UdyIAiKtewezVPXMhhMwjUsKFzVXnhetJoSr2lkf3u1go3gZagXpFONLlZAl?=
 =?us-ascii?Q?DtUk46rPmrykNHsGWCJzn5J/M3yzUgXSzoAwpD0F?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3d63c0a3-3984-4770-c664-08dc6a9f64f1
X-MS-Exchange-CrossTenant-AuthSource: CYYPR12MB8750.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2024 12:00:00.7145
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: c2kDDO24ksAHCKju1MdL3HNDGaedRRJs1/ceo4lhQt6iQJNvwhNgu34QBXbszJ7es9+5y+LlbLAeywdwixjbEw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB9077

On 30.04.24 15:48:56, Jonathan Cameron wrote:
> On Tue, 30 Apr 2024 11:21:54 +0200
> Robert Richter <rrichter@amd.com> wrote:
> 
> > For configurations that have the kconfig option NUMA_KEEP_MEMINFO
> > disabled numa_fill_memblks() only returns with NUMA_NO_MEMBLK (-1).
> > SRAT lookup fails then because an existing SRAT memory range cannot be
> > found for a CFMWS address range. This causes the addition of a
> > duplicate numa_memblk with a different node id and a subsequent page
> > fault and kernel crash during boot.
> > 
> > Fix this by making numa_fill_memblks() always available regardless of
> > NUMA_KEEP_MEMINFO.
> > 
> > The fix also removes numa_fill_memblks() from sparsemem.h using
> > __weak.
> > 
> > From Dan:
> > 
> > """
> > It just feels like numa_fill_memblks() has absolutely no business being
> > defined in arch/x86/include/asm/sparsemem.h.
> > 
> > The only use for numa_fill_memblks() is to arrange for NUMA nodes to be
> > applied to memory ranges hot-onlined by the CXL driver.
> > 
> > It belongs right next to numa_add_memblk(), and I suspect
> > arch/x86/include/asm/sparsemem.h was only chosen to avoid figuring out
> > what to do about the fact that linux/numa.h does not include asm/numa.h
> > and that all implementations either provide numa_add_memblk() or select
> > the generic implementation.
> > 
> > So I would prefer that this do the proper fix and get
> > numa_fill_memblks() completely out of the sparsemem.h path.
> > 
> > Something like the following which boots for me.
> > """
> > 
> > Note that the issue was initially introduced with [1]. But since
> > phys_to_target_node() was originally used that returned the valid node
> > 0, an additional numa_memblk was not added. Though, the node id was
> > wrong too, a message is seen then in the logs:
> > 
> >  kernel/numa.c:  pr_info_once("Unknown target node for memory at 0x%llx, assuming node 0\n",
> > 
> > [1] commit fd49f99c1809 ("ACPI: NUMA: Add a node and memblk for each
> >     CFMWS not in SRAT")
> > 
> > Suggested-by: Dan Williams <dan.j.williams@intel.com>
> > Link: https://lore.kernel.org/all/66271b0072317_69102944c@dwillia2-xfh.jf.intel.com.notmuch/
> > Fixes: 8f1004679987 ("ACPI/NUMA: Apply SRAT proximity domain to entire CFMWS window")
> > Cc: Derick Marks <derick.w.marks@intel.com>
> > Cc: Dan Williams <dan.j.williams@intel.com>
> > Cc: Alison Schofield <alison.schofield@intel.com>
> > Signed-off-by: Robert Richter <rrichter@amd.com>
> 
> Whilst I'm not particularly keen on an arch specific solution for this
> and the stub is effectively pointless beyond making the build work, I guess
> this works well enough for now.
> 
> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> 
> I was aiming to post the ARM64 handling this cycle but it hasn't quite happened yet :(
> Maybe we can look at whether there is a better level share at than
> the whole function once that is done.

Thanks for review.

It seems better to change x86 to use the generic implementation of
numa_add_memblk() in drivers/base/arch_numa.c. That already contains
code to deal with and merge overlapping blocks, it also checks memory
attributes. But that is not scope of this patch.

-Robert