Currently, the range and default value of NR_CPUS is too restrictive
for high-end RISC-V systems with large number of HARTs. The latest
QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is
restrictive for QEMU as well. Other major architectures (such as
ARM64, x86_64, MIPS, etc) have a much higher range and default
value of NR_CPUS.
This patch increases NR_CPUS range to 2-256 and default value to
XLEN (i.e. 32 for RV32 and 64 for RV64).
Signed-off-by: Anup Patel <[email protected]>
---
arch/riscv/Kconfig | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 5adcbd9b5e88..54f125eb10f6 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -274,10 +274,11 @@ config SMP
If you don't know what to do here, say N.
config NR_CPUS
- int "Maximum number of CPUs (2-32)"
- range 2 32
+ int "Maximum number of CPUs (2-256)"
+ range 2 256
depends on SMP
- default "8"
+ default "32" if 32BIT
+ default "64" if 64BIT
config HOTPLUG_CPU
bool "Support for hot-pluggable CPUs"
--
2.25.1
On Fri, Mar 18, 2022 at 3:46 PM Ben Dooks <[email protected]> wrote:
>
> On 17/03/2022 03:55, Anup Patel wrote:
> > Currently, the range and default value of NR_CPUS is too restrictive
> > for high-end RISC-V systems with large number of HARTs. The latest
> > QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is
> > restrictive for QEMU as well.
If qemu allows 512, what is the reason for limiting the kernel to 256?
> > Other major architectures (such as
> > ARM64, x86_64, MIPS, etc) have a much higher range and default
> > value of NR_CPUS.
>
> What's the memory overhead for increasing this?
It's supposed to be very small, I would expect three main sources of overhead:
- cpumask_t variables, those grow once you go beyond the size of
an unsigned long (32 or 64 bits), so with the default just on the limit, this
makes no difference.
Note that you can run out of stack space with NR_CPUS values if
CONFIG_CPUMASK_OFFSTACK is disabled. Should not be
a problem for 512 or below.
- percpu variables: these are dynamically allocated based on the number of
CPUs at boot time, so they should not have any real impact.
- NR_CPUS sized arrays, these are sometimes used in place of
percpu data. This is only a problem if the array members individually
are more than a few bytes. There are not too many of these in the kernel,
as using those is discouraged.
Arnd
On 17/03/2022 03:55, Anup Patel wrote:
> Currently, the range and default value of NR_CPUS is too restrictive
> for high-end RISC-V systems with large number of HARTs. The latest
> QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is
> restrictive for QEMU as well. Other major architectures (such as
> ARM64, x86_64, MIPS, etc) have a much higher range and default
> value of NR_CPUS.
What's the memory overhead for increasing this?
--
Ben Dooks http://www.codethink.co.uk/
Senior Engineer Codethink - Providing Genius
https://www.codethink.co.uk/privacy.html
On Fri, Mar 18, 2022 at 8:44 PM Arnd Bergmann <[email protected]> wrote:
>
> On Fri, Mar 18, 2022 at 3:46 PM Ben Dooks <[email protected]> wrote:
> >
> > On 17/03/2022 03:55, Anup Patel wrote:
> > > Currently, the range and default value of NR_CPUS is too restrictive
> > > for high-end RISC-V systems with large number of HARTs. The latest
> > > QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is
> > > restrictive for QEMU as well.
>
> If qemu allows 512, what is the reason for limiting the kernel to 256?
There is no particular reason. I will change this to 2-512 range to match
the maximum number of CPUs supported by the QEMU virt machine.
>
> > > Other major architectures (such as
> > > ARM64, x86_64, MIPS, etc) have a much higher range and default
> > > value of NR_CPUS.
> >
> > What's the memory overhead for increasing this?
>
> It's supposed to be very small, I would expect three main sources of overhead:
>
> - cpumask_t variables, those grow once you go beyond the size of
> an unsigned long (32 or 64 bits), so with the default just on the limit, this
> makes no difference.
> Note that you can run out of stack space with NR_CPUS values if
> CONFIG_CPUMASK_OFFSTACK is disabled. Should not be
> a problem for 512 or below.
Yes, the rationale is to have default NR_CPUS such that size of
cpumask_t does not change for RV32 and RV64.
>
> - percpu variables: these are dynamically allocated based on the number of
> CPUs at boot time, so they should not have any real impact.
>
> - NR_CPUS sized arrays, these are sometimes used in place of
> percpu data. This is only a problem if the array members individually
> are more than a few bytes. There are not too many of these in the kernel,
> as using those is discouraged.
Separately, we are also trying to replace NR_CPUS sized arrays.
Regards,
Anup