Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20220317035542.272547-1-apatel@ventanamicro.com>
 <f1079093-54db-bd20-3342-b06d069c5d1f@codethink.co.uk> <CAK8P3a0Gt7TOT2zpNw7YNR=MNLBWVAADL5dBgY15ZL8sDW20HQ@mail.gmail.com>
In-Reply-To: <CAK8P3a0Gt7TOT2zpNw7YNR=MNLBWVAADL5dBgY15ZL8sDW20HQ@mail.gmail.com>
From:   Anup Patel <apatel@ventanamicro.com>
Date:   Sat, 19 Mar 2022 17:09:31 +0530
Message-ID: <CAK9=C2XGgfEZ890DgKmPXKw7=1BjnP-2L+_bpx+efX_rnw=_Nw@mail.gmail.com>
Subject: Re: [PATCH] RISC-V: Increase range and default value of NR_CPUS
To:     Arnd Bergmann <arnd@arndb.de>
Cc:     Ben Dooks <ben.dooks@codethink.co.uk>,
        Palmer Dabbelt <palmer@dabbelt.com>,
        Paul Walmsley <paul.walmsley@sifive.com>,
        Atish Patra <atishp@atishpatra.org>,
        Alistair Francis <Alistair.Francis@wdc.com>,
        Anup Patel <anup@brainfault.org>,
        linux-riscv <linux-riscv@lists.infradead.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Fri, Mar 18, 2022 at 8:44 PM Arnd Bergmann <arnd@arndb.de> wrote:
>
> On Fri, Mar 18, 2022 at 3:46 PM Ben Dooks <ben.dooks@codethink.co.uk> wrote:
> >
> > On 17/03/2022 03:55, Anup Patel wrote:
> > > Currently, the range and default value of NR_CPUS is too restrictive
> > > for high-end RISC-V systems with large number of HARTs. The latest
> > > QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is
> > > restrictive for QEMU as well.
>
> If qemu allows 512, what is the reason for limiting the kernel to 256?

There is no particular reason. I will change this to 2-512 range to match
the maximum number of CPUs supported by the QEMU virt machine.

>
> > > Other major architectures (such as
> > > ARM64, x86_64, MIPS, etc) have a much higher range and default
> > > value of NR_CPUS.
> >
> > What's the memory overhead for increasing this?
>
> It's supposed to be very small, I would expect three main sources of overhead:
>
> - cpumask_t variables, those grow once you go beyond the size of
>    an unsigned long (32 or 64 bits), so with the default just on the limit, this
>    makes no difference.
>    Note that you can run out of stack space with NR_CPUS values if
>    CONFIG_CPUMASK_OFFSTACK is disabled. Should not be
>    a problem for 512 or below.

Yes, the rationale is to have default NR_CPUS such that size of
cpumask_t does not change for RV32 and RV64.

>
> - percpu variables: these are dynamically allocated based on the number of
>   CPUs at boot time, so they should not have any real impact.
>
> - NR_CPUS sized arrays, these are sometimes used in place of
>   percpu data. This is only a problem if the array members individually
>   are more than a few bytes. There are not too many of these in the kernel,
>   as using those is discouraged.

Separately, we are also trying to replace NR_CPUS sized arrays.

Regards,
Anup