From: Conor Dooley <[email protected]>
RISC-V has no sane defaults to fall back on where there is no cpu-map
in the devicetree.
Without sane defaults, the package, core and thread IDs are all set to
-1. This causes user-visible inaccuracies for tools like hwloc/lstopo
which rely on the sysfs cpu topology files to detect a system's
topology.
On a PolarFire SoC, which should have 4 harts with a thread each,
lstopo currently reports:
Machine (793MB total)
Package L#0
NUMANode L#0 (P#0 793MB)
Core L#0
L1d L#0 (32KB) + L1i L#0 (32KB) + PU L#0 (P#0)
L1d L#1 (32KB) + L1i L#1 (32KB) + PU L#1 (P#1)
L1d L#2 (32KB) + L1i L#2 (32KB) + PU L#2 (P#2)
L1d L#3 (32KB) + L1i L#3 (32KB) + PU L#3 (P#3)
Adding calls to store_cpu_topology() in {boot,smp} hart bringup code
results in the correct topolgy being reported:
Machine (793MB total)
Package L#0
NUMANode L#0 (P#0 793MB)
L1d L#0 (32KB) + L1i L#0 (32KB) + Core L#0 + PU L#0 (P#0)
L1d L#1 (32KB) + L1i L#1 (32KB) + Core L#1 + PU L#1 (P#1)
L1d L#2 (32KB) + L1i L#2 (32KB) + Core L#2 + PU L#2 (P#2)
L1d L#3 (32KB) + L1i L#3 (32KB) + Core L#3 + PU L#3 (P#3)
CC: [email protected]
Fixes: 03f11f03dbfe ("RISC-V: Parse cpu topology during boot.")
Reported-by: Brice Goglin <[email protected]>
Link: https://github.com/open-mpi/hwloc/issues/536
Signed-off-by: Conor Dooley <[email protected]>
---
---
arch/riscv/Kconfig | 2 +-
arch/riscv/kernel/smpboot.c | 4 +++-
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 2af0701b7518..4b6c2fdbb57c 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -52,7 +52,7 @@ config RISCV
select COMMON_CLK
select CPU_PM if CPU_IDLE
select EDAC_SUPPORT
- select GENERIC_ARCH_TOPOLOGY if SMP
+ select GENERIC_ARCH_TOPOLOGY
select GENERIC_ATOMIC64 if !64BIT
select GENERIC_CLOCKEVENTS_BROADCAST if SMP
select GENERIC_EARLY_IOREMAP
diff --git a/arch/riscv/kernel/smpboot.c b/arch/riscv/kernel/smpboot.c
index f1e4948a4b52..a1c861f84fe2 100644
--- a/arch/riscv/kernel/smpboot.c
+++ b/arch/riscv/kernel/smpboot.c
@@ -40,6 +40,8 @@ static DECLARE_COMPLETION(cpu_running);
void __init smp_prepare_boot_cpu(void)
{
init_cpu_topology();
+
+ store_cpu_topology(smp_processor_id());
}
void __init smp_prepare_cpus(unsigned int max_cpus)
@@ -161,9 +163,9 @@ asmlinkage __visible void smp_callin(void)
mmgrab(mm);
current->active_mm = mm;
+ store_cpu_topology(curr_cpuid);
notify_cpu_starting(curr_cpuid);
numa_add_cpu(curr_cpuid);
- update_siblings_masks(curr_cpuid);
set_cpu_online(curr_cpuid, 1);
/*
--
2.37.0
On Sat, Jul 09, 2022 at 04:23:55PM +0100, Conor Dooley wrote:
> From: Conor Dooley <[email protected]>
>
> RISC-V has no sane defaults to fall back on where there is no cpu-map
> in the devicetree.
> Without sane defaults, the package, core and thread IDs are all set to
> -1. This causes user-visible inaccuracies for tools like hwloc/lstopo
> which rely on the sysfs cpu topology files to detect a system's
> topology.
>
> On a PolarFire SoC, which should have 4 harts with a thread each,
> lstopo currently reports:
>
> Machine (793MB total)
> Package L#0
> NUMANode L#0 (P#0 793MB)
> Core L#0
> L1d L#0 (32KB) + L1i L#0 (32KB) + PU L#0 (P#0)
> L1d L#1 (32KB) + L1i L#1 (32KB) + PU L#1 (P#1)
> L1d L#2 (32KB) + L1i L#2 (32KB) + PU L#2 (P#2)
> L1d L#3 (32KB) + L1i L#3 (32KB) + PU L#3 (P#3)
>
> Adding calls to store_cpu_topology() in {boot,smp} hart bringup code
> results in the correct topolgy being reported:
>
> Machine (793MB total)
> Package L#0
> NUMANode L#0 (P#0 793MB)
> L1d L#0 (32KB) + L1i L#0 (32KB) + Core L#0 + PU L#0 (P#0)
> L1d L#1 (32KB) + L1i L#1 (32KB) + Core L#1 + PU L#1 (P#1)
> L1d L#2 (32KB) + L1i L#2 (32KB) + Core L#2 + PU L#2 (P#2)
> L1d L#3 (32KB) + L1i L#3 (32KB) + Core L#3 + PU L#3 (P#3)
>
> CC: [email protected]
> Fixes: 03f11f03dbfe ("RISC-V: Parse cpu topology during boot.")
> Reported-by: Brice Goglin <[email protected]>
> Link: https://github.com/open-mpi/hwloc/issues/536
> Signed-off-by: Conor Dooley <[email protected]>
> ---
> ---
> arch/riscv/Kconfig | 2 +-
> arch/riscv/kernel/smpboot.c | 4 +++-
> 2 files changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 2af0701b7518..4b6c2fdbb57c 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -52,7 +52,7 @@ config RISCV
> select COMMON_CLK
> select CPU_PM if CPU_IDLE
> select EDAC_SUPPORT
> - select GENERIC_ARCH_TOPOLOGY if SMP
> + select GENERIC_ARCH_TOPOLOGY
I am not sure of !SMP as ARM64 is default SMP only. I have never reviewed
the arch topology code with !SMP considered. I will leave that part to
RISC-V developers.
> select GENERIC_ATOMIC64 if !64BIT
> select GENERIC_CLOCKEVENTS_BROADCAST if SMP
> select GENERIC_EARLY_IOREMAP
> diff --git a/arch/riscv/kernel/smpboot.c b/arch/riscv/kernel/smpboot.c
> index f1e4948a4b52..a1c861f84fe2 100644
> --- a/arch/riscv/kernel/smpboot.c
> +++ b/arch/riscv/kernel/smpboot.c
> @@ -40,6 +40,8 @@ static DECLARE_COMPLETION(cpu_running);
> void __init smp_prepare_boot_cpu(void)
> {
> init_cpu_topology();
> +
> + store_cpu_topology(smp_processor_id());
> }
>
> void __init smp_prepare_cpus(unsigned int max_cpus)
> @@ -161,9 +163,9 @@ asmlinkage __visible void smp_callin(void)
> mmgrab(mm);
> current->active_mm = mm;
>
> + store_cpu_topology(curr_cpuid);
> notify_cpu_starting(curr_cpuid);
> numa_add_cpu(curr_cpuid);
> - update_siblings_masks(curr_cpuid);
> set_cpu_online(curr_cpuid, 1);
>
> /*
Other than that, this looks good. FWIW:
Reviewed-by: Sudeep Holla <[email protected]>
--
Regards,
Sudeep
On 11/07/2022 15:59, Sudeep Holla wrote:
> EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
>
> On Sat, Jul 09, 2022 at 04:23:55PM +0100, Conor Dooley wrote:
>> From: Conor Dooley <[email protected]>
>>
>> RISC-V has no sane defaults to fall back on where there is no cpu-map
>> in the devicetree.
>> Without sane defaults, the package, core and thread IDs are all set to
>> -1. This causes user-visible inaccuracies for tools like hwloc/lstopo
>> which rely on the sysfs cpu topology files to detect a system's
>> topology.
>>
>> On a PolarFire SoC, which should have 4 harts with a thread each,
>> lstopo currently reports:
>>
>> Machine (793MB total)
>> Package L#0
>> NUMANode L#0 (P#0 793MB)
>> Core L#0
>> L1d L#0 (32KB) + L1i L#0 (32KB) + PU L#0 (P#0)
>> L1d L#1 (32KB) + L1i L#1 (32KB) + PU L#1 (P#1)
>> L1d L#2 (32KB) + L1i L#2 (32KB) + PU L#2 (P#2)
>> L1d L#3 (32KB) + L1i L#3 (32KB) + PU L#3 (P#3)
>>
>> Adding calls to store_cpu_topology() in {boot,smp} hart bringup code
>> results in the correct topolgy being reported:
>>
>> Machine (793MB total)
>> Package L#0
>> NUMANode L#0 (P#0 793MB)
>> L1d L#0 (32KB) + L1i L#0 (32KB) + Core L#0 + PU L#0 (P#0)
>> L1d L#1 (32KB) + L1i L#1 (32KB) + Core L#1 + PU L#1 (P#1)
>> L1d L#2 (32KB) + L1i L#2 (32KB) + Core L#2 + PU L#2 (P#2)
>> L1d L#3 (32KB) + L1i L#3 (32KB) + Core L#3 + PU L#3 (P#3)
>>
>> CC: [email protected]
>> Fixes: 03f11f03dbfe ("RISC-V: Parse cpu topology during boot.")
>> Reported-by: Brice Goglin <[email protected]>
>> Link: https://github.com/open-mpi/hwloc/issues/536
>> Signed-off-by: Conor Dooley <[email protected]>
>> ---
>> ---
>> arch/riscv/Kconfig | 2 +-
>> arch/riscv/kernel/smpboot.c | 4 +++-
>> 2 files changed, 4 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
>> index 2af0701b7518..4b6c2fdbb57c 100644
>> --- a/arch/riscv/Kconfig
>> +++ b/arch/riscv/Kconfig
>> @@ -52,7 +52,7 @@ config RISCV
>> select COMMON_CLK
>> select CPU_PM if CPU_IDLE
>> select EDAC_SUPPORT
>> - select GENERIC_ARCH_TOPOLOGY if SMP
>> + select GENERIC_ARCH_TOPOLOGY
>
> I am not sure of !SMP as ARM64 is default SMP only. I have never reviewed
> the arch topology code with !SMP considered. I will leave that part to
> RISC-V developers.
>
I checked it on a D1 which is !SMP - no trouble booting and
the topology reporting seemed fine.
Thanks for the reviews,
Conor.