Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Uladzislau Rezki <urezki@gmail.com>
Date:   Fri, 3 Jul 2020 19:24:18 +0200
To:     Gabriel C <nix.or.die@googlemail.com>
Cc:     Uladzislau Rezki <urezki@gmail.com>,
        LKML <linux-kernel@vger.kernel.org>, linux-mm@kvack.org,
        Andrew Morton <akpm@linux-foundation.org>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        GregKroah-Hartmangregkh@linuxfoundation.org,
        "Peter Zijlstra (Intel)" <peterz@infradead.org>
Subject: Re: nr_cpu_ids vs AMD 3970x(32 physical CPUs)
Message-ID: <20200703172418.GA4500@pc636>
References: <20200703155749.GA6255@pc636>
 <CAEJqkgiFFh8CvXkM4ZzXxNQmOJLL7WcgDL6rM83safNgEewZ9w@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAEJqkgiFFh8CvXkM4ZzXxNQmOJLL7WcgDL6rM83safNgEewZ9w@mail.gmail.com>
User-Agent: Mutt/1.10.1 (2018-07-13)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

> >
> > I have a system based on AMD 3970x CPUs. It has 32 physical cores
> > and 64 threads. It seems that "nr_cpu_ids" variable is not correctly
> > set on latest 5.8-rc3 kernel. Please have a look below on dmesg output:
> >
> > <snip>
> > urezki@pc638:~$ sudo dmesg | grep 128
> > [    0.000000] IOAPIC[0]: apic_id 128, version 33, address 0xfec00000, GSI 0-23
> > [    0.000000] smpboot: Allowing 128 CPUs, 64 hotplug CPUs
> > [    0.000000] setup_percpu: NR_CPUS:512 nr_cpumask_bits:512 nr_cpu_ids:128 nr_node_ids:1
> > ...
> > [    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=128, Nodes=1
> > [    0.000000] rcu:     RCU restricting CPUs from NR_CPUS=512 to nr_cpu_ids=128.
> > [    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=128
> > urezki@pc638:~$
> > <snip>
> >
> > For example SLUB thinks that it deals with 128 CPUs in the system what is
> > wrong if i do not miss something. Since nr_cpu_ids is broken(?), thus the
> > "cpu_possible_mask" does not correspond to reality as well.
> >
> > Any thoughts?
> 
> This is not a 5.8-rc3 problem. Almost all AMD CPUs and APUs are
> looking like this.
> The only CPUs I own are getting that right is a dual EPYC box,
> everything else is broken
> regarding the right C/T & socket(s) count, and that probably bc is
> using NUAM code
> to have the info.
> 
> I reported that a while back and no-one ever cared.
> 
> There is even a comment in the hotplug code saying setting the wrong CPU count
> is a waste of resources.
> 
> I have a 2200G is reporting 48Cores.
> 
> AMD Ryzen 7 3750H reporting twice the cores and twice the socket.
> 
> ...
> 
> [    0.040578] smpboot: Allowing 16 CPUs, 8 hotplug CPUs
> ...
> [    0.382122] smpboot: Max logical packages: 2
> ..
> 
> I boot all the boxes restricting the cores to the correct count on the
> command line.
> 
> Wasted resource or not, this is still a bug IMO.
> 
I suspect that DEFINE_PER_CPU variables can be twice as big,
but i have not checked it actually. So, if the code needs to
identify real number of CPUs it can be a challenge :)

Thanks.

--
Vlad Rezki