Received-SPF: pass (google.com: domain of linux-kernel+bounces-110453-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Date: Thu, 21 Mar 2024 17:55:57 +0100
From: Robert Richter <rrichter@amd.com>
To: Alison Schofield <alison.schofield@intel.com>
Cc: "Rafael J. Wysocki" <rafael@kernel.org>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	Andy Lutomirski <luto@kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
	x86@kernel.org, Dan Williams <dan.j.williams@intel.com>,
	linux-acpi@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-cxl@vger.kernel.org, Derick Marks <derick.w.marks@intel.com>,
	"H. Peter Anvin" <hpa@zytor.com>
Subject: Re: [PATCH v2 1/3] x86/numa: Fix SRAT lookup of CFMWS ranges with
 numa_fill_memblks()
Message-ID: <Zfxmnfj1K0OTk89U@rric.localdomain>
References: <20240319120026.2246389-1-rrichter@amd.com>
 <20240319120026.2246389-2-rrichter@amd.com>
 <Zfsg3wZpSFVT+Zv2@aschofie-mobl2>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Zfsg3wZpSFVT+Zv2@aschofie-mobl2>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?1W1gZ6nqFi+G7hRGmlpPZA4ot8aMnzSR86TVtGP4BFlhbShD4GyFU+BLsrhH?=
 =?us-ascii?Q?wU/zGF8VfObCRJfe9NFjYCNrajMb4j4i7XezfFHEsvxWwF2SbqEb1UNhGswd?=
 =?us-ascii?Q?1aT/vldGw9EXxkKaNdNkbRAkLHJ38L9uyWn8gZsAc6cNYIjalzK6ZBL6VmK3?=
 =?us-ascii?Q?OUETPBQ0rboSdJ0p85rA01ga9YvuiownrdslX8e4EcbI2Qa0PmMfL10zMOUD?=
 =?us-ascii?Q?lJR6WLAwBS9X6qVrYv2wPDtOaNw4CDzNCnGZJf0j2oZKZO3rUMXQxmTy3Btm?=
 =?us-ascii?Q?jA4LysTpodiN2WfTJaTm6B0AQTyeRr0HssYpAyTe/NAMj7cPjuzcosg1A2W8?=
 =?us-ascii?Q?ocyUlkP350O7kkgYAFGXooSsD2w7CRdGtqv4aMhNmd5V8DTV3sPdSXLbOGqg?=
 =?us-ascii?Q?1vBm/vo97W06OsWe0nGUl327A3xjX8yREptOdQKw1kx8s8GXEiXuOsE8c1gD?=
 =?us-ascii?Q?A4+Sl13QTOlygIKVV4NEOY5P9vZj+Gut/st/VPp4lg5FZBKi34fgTmLdA0ej?=
 =?us-ascii?Q?phCiAIw6MIELg62yRNqWitIbCGe9IrPCB/2CK/w2FVuV9s1c/Dp2Qtkb5tTv?=
 =?us-ascii?Q?iF5jVcFbazu/2JgQEmq+t64OGWwUk/tNonPBJelw3P7YcQSm/QZSHoKzzOLc?=
 =?us-ascii?Q?TVQtld62E/8Pc5XMYz1StSyIvBbycjN6xkSxopjccuN08XHu+l2rviG7QtV+?=
 =?us-ascii?Q?tvHvOR0Ug+yyT7ldP4i8DjNJE3//fjCgCklcMKt3Rw+Pw0WnS0KcvLNkKjpM?=
 =?us-ascii?Q?06rZQcLsSMgfMXs5KvMhkLzlOiYvVfhqubflG8jWXg8YsEhZgmjP2Ydy37pP?=
 =?us-ascii?Q?6SP8+tjfpDq/pIqhSv0EtytSFRNrwmE3JY4jpo+4SGXNmX3hPuFFLmN/Fpin?=
 =?us-ascii?Q?WCnDyIL6BHbR8o5M9fwVOTkpX+ntiEdIA5AmsNteZaniZ7xuh4wMmNaFH8E8?=
 =?us-ascii?Q?34u3hH+vSc8l1K2R/3hzjMaQ7GSZa7SETyd4tV61GwT9OIjBmZ7b4gMXiUb6?=
 =?us-ascii?Q?MHnRCpiqos3ONz+KmgRrAuJuD5oIpPLsHHcH7a7e72BOvDOEBZhX9phkZkRN?=
 =?us-ascii?Q?hP61rIVbSDBAFvJbYl66jcdl5DN3mE9ix8MLeAEKSG+uGlNxwEIDFaHoP74b?=
 =?us-ascii?Q?pQbQhzzvfZ/ksjDNNiZd0bqLiQzBjmTIvReOyr/UzacEpzCw9uKOkpGfTifQ?=
 =?us-ascii?Q?mzRwr0PPgvgkFM5wvodmPx04k2ND8O+m9lcfvNKBIvi7BHko2xOrAFFX+W24?=
 =?us-ascii?Q?iu4LWKZGiM4nVK7CwfVOy4RCfORuhlDKVGaiVlx/C+6w7NFv0pn/u/9qhgjv?=
 =?us-ascii?Q?K8VJd6AxwkOeXc1zEPQl/2m+GESOn8iVb4u33XrrJPSH155V5pKaGZZtZB1B?=
 =?us-ascii?Q?uez2sotlVNUHGAfb4BRQSNpEKpLzIi+mZkI0W8lhU+9rA6w9GevZ3yIhW88x?=
 =?us-ascii?Q?R8rxjHmOBvEAZxAB+epHXAwpJTH8ovxN7kaW9Ewt2tADxptTqFPMUEUbMuef?=
 =?us-ascii?Q?IojMUpD+/kpuX+9iqv6vsSPntQfZ9AH53ZD6x8pYzolLo1DFJH2p5qEm68ja?=
 =?us-ascii?Q?g3s8+gE5l2HbzDNZlGGEpJFNtGlx402U9OB1M7ry?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 964fe297-a558-4596-da9d-08dc49c7cb66
X-MS-Exchange-CrossTenant-AuthSource: CYYPR12MB8750.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Mar 2024 16:56:04.0998
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: svC0xFrqzsBXuRDPbp4lLalJOzbzTK0ygt42vmGlUq40e7XwCEJLX+vCIEdoUtGf/4EdKTnTAKrYbOno/0bkyw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4133

Alison,

On 20.03.24 10:46:07, Alison Schofield wrote:
> On Tue, Mar 19, 2024 at 01:00:23PM +0100, Robert Richter wrote:
> > For configurations that have the kconfig option NUMA_KEEP_MEMINFO
> > disabled, the SRAT lookup done with numa_fill_memblks() fails
> > returning NUMA_NO_MEMBLK (-1). An existing SRAT memory range cannot be
> > found for a CFMWS address range. This causes the addition of a
> > duplicate numa_memblk with a different node id and a subsequent page
> > fault and kernel crash during boot.
> > 
> > numa_fill_memblks() is implemented and used in the init section only.
> > The option NUMA_KEEP_MEMINFO is only for the case when NUMA data will
> > be used outside of init. So fix the SRAT lookup by moving
> > numa_fill_memblks() out of the NUMA_KEEP_MEMINFO block to make it
> > always available in the init section.
> > 
> > Note that the issue was initially introduced with [1]. But since
> > phys_to_target_node() was originally used that returned the valid node
> > 0, an additional numa_memblk was not added. Though, the node id was
> > wrong too.
> 
> Hi Richard,
> 
> I recall a bit of wrangling w #defines to make ARM64 and LOONGARCH build.
> I'm seeing an x86 build error today:
> 
> >> arch/x86/mm/numa.c:957:12: error: redefinition of 'numa_fill_memblks'
>      957 | int __init numa_fill_memblks(u64 start, u64 end)
> 
> include/linux/numa.h:40:26: note: previous definition of 'numa_fill_memblks' with type
> +'int(u64,  u64)' {aka 'int(long long unsigned int,  long long unsigned int)'}
>       40 | static inline int __init numa_fill_memblks(u64 start, u64 end)
>          |                          ^~~~~~~~~~~~~~~~~
> 
> In addition to what you suggest, would something like this diff below be
> a useful safety measure to distinguish num_fill_memblks() success (rc:0)
> and possible non-existence (rc:-1). I don't think it hurts to take a
> second look using phys_to_target_node() (totall untested)
> 
> diff --git a/drivers/acpi/numa/srat.c b/drivers/acpi/numa/srat.c
> index 070a52e4daa8..0c48fe32ced4 100644
> --- a/drivers/acpi/numa/srat.c
> +++ b/drivers/acpi/numa/srat.c
> @@ -437,9 +437,16 @@ static int __init acpi_parse_cfmws(union acpi_subtable_headers *header,
>          * found for any portion of the window to cover the entire
>          * window.
>          */
> -       if (!numa_fill_memblks(start, end))
> +       rc = numa_fill_memblks(start, end);
> +       if (!rc)
>                 return 0;
>  
> +       if (rc == NUMA_NO_MEMBLK) {
> +               node = phys_to_target_node(start);
> +               if (node != NUMA_NO_NODE)
> +                       return 0;
> +       }
> +

for non-x86 the numa_add_memblk() function looks good in a way that it
is able to handle presumable overlapping regions. numa_fill_memblks()
would just fail then and numa_add_memblk() being called. For x86 we
need numa_fill_memblks() since x86 specific numa_add_memblk() cannot
handle the overlapping case.

That said, we do not need the 2nd check. It looks to me that it
actually breaks non-x86 as the whole block may not be registered (if
it is larger than anything existing).

For x86 the 2nd check may never happen if numa_fill_memblks() is
always enabled (which is this patch for).

So we should be good without your change.

Thanks,

-Robert

>         /* No SRAT description. Create a new node. */
> 
> --Alison
> 
> > 
> > [1] fd49f99c1809 ("ACPI: NUMA: Add a node and memblk for each CFMWS not in SRAT")
> > 
> > Fixes: 8f1004679987 ("ACPI/NUMA: Apply SRAT proximity domain to entire CFMWS window")
> > Cc: Derick Marks <derick.w.marks@intel.com>
> > Cc: Dan Williams <dan.j.williams@intel.com>
> > Cc: Alison Schofield <alison.schofield@intel.com>
> > Signed-off-by: Robert Richter <rrichter@amd.com>
> > ---
> >  arch/x86/mm/numa.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/arch/x86/mm/numa.c b/arch/x86/mm/numa.c
> > index 65e9a6e391c0..ce84ba86e69e 100644
> > --- a/arch/x86/mm/numa.c
> > +++ b/arch/x86/mm/numa.c
> > @@ -929,6 +929,8 @@ int memory_add_physaddr_to_nid(u64 start)
> >  }
> >  EXPORT_SYMBOL_GPL(memory_add_physaddr_to_nid);
> >  
> > +#endif
> > +
> >  static int __init cmp_memblk(const void *a, const void *b)
> >  {
> >  	const struct numa_memblk *ma = *(const struct numa_memblk **)a;
> > @@ -1001,5 +1003,3 @@ int __init numa_fill_memblks(u64 start, u64 end)
> >  	}
> >  	return 0;
> >  }
> > -
> > -#endif
> > -- 
> > 2.39.2
> >