Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Uladzislau Rezki <urezki@gmail.com>
Date:   Fri, 3 Jul 2020 23:20:47 +0200
To:     Linus Torvalds <torvalds@linux-foundation.org>
Cc:     Uladzislau Rezki <urezki@gmail.com>,
        peter enderborg <peter.enderborg@sony.com>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Linux-MM <linux-mm@kvack.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        GregKroah-Hartmangregkh@linuxfoundation.org,
        Peter Zijlstra <peterz@infradead.org>
Subject: Re: nr_cpu_ids vs AMD 3970x(32 physical CPUs)
Message-ID: <20200703212047.GA6856@pc636>
References: <20200703155749.GA6255@pc636>
 <8a2a55e6-6087-e4bf-3d35-ed4b4c216369@sony.com>
 <20200703192807.GB5207@pc636>
 <CAHk-=wiagrzJs9OBe_6bHK+Cc6RdoCOV85CiJAd3MhYnc8idfw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAHk-=wiagrzJs9OBe_6bHK+Cc6RdoCOV85CiJAd3MhYnc8idfw@mail.gmail.com>
User-Agent: Mutt/1.10.1 (2018-07-13)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

> On Fri, Jul 3, 2020 at 12:28 PM Uladzislau Rezki <urezki@gmail.com> wrote:
> >
> > I have MSI TRX40 with latest BIOS.
> 
> I think it's just that the BIOS is set for the max possible, in case
> you'd have a 3990X.
> 
3990x is the top one in this series, so indeed it can be a case and
explanation why nr_cpu_ids is set to 128.

>
> I compile my kernel with CONFIG_NR_CPUS's set to 64. That works around
> the issue.
> 
> Lots of distros seem to set CONFIG_MAXSMP to true, which I guess is
> the most generic thing to do, but the problem with that is not just
> the silly problem with the BIOS, but it also means that the kernel
> does dynamic allocation for cpumasks even if you _don't_ have that
> problem, because at compile-time you don't know how big the cpumask
> will be.
> 
> With CONFIG_NR_CPUS's set to 64, the kernel will just use a "unsigned
> long" on the stack (and in various data structures) and be done with
> it, and not do unnecessary dynamic allocations.
> 
Thanks for proposed workaround! I will update the CONFIG_NR_CPUS with
proper value in my .config

Some background:
Actually i have been thinking about making vmalloc address space to
be per-CPU, i.e. divide it to per-CPU address space making an allocation
lock-less. It will eliminate a high lock contention. When i have done
a prototype i noticed and realized that there is a silly issue with
nr_cpu_ids on some systems.

Therefore i reported about it.

Thanks, Linus!

--
Vlad Rezki