Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Mon, 3 Dec 2018 12:13:59 +0100
From:   Borislav Petkov <bp@alien8.de>
To:     Andrew Cooper <andrew.cooper3@citrix.com>
Cc:     Xen-devel List <xen-devel@lists.xen.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Jan Beulich <JBeulich@suse.com>,
        Brian Woods <brian.woods@amd.com>,
        SuraveeSuthikulpanit <suravee.suthikulpanit@amd.com>,
        Boris Ostrovsky <boris.ostrovsky@oracle.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>,
        "H. Peter Anvin" <hpa@zytor.com>,
        Tom Lendacky <thomas.lendacky@amd.com>,
        Brijesh Singh <brijeshkumar.singh@amd.com>
Subject: Re: AMD EPYC Topology problems
Message-ID: <20181203111359.GA31670@zn.tnic>
References: <7369db0c-4917-0807-eef4-cba5e5ae0d4e@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <7369db0c-4917-0807-eef4-cba5e5ae0d4e@citrix.com>
User-Agent: Mutt/1.10.1 (2018-07-13)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Sun, Dec 02, 2018 at 08:23:05PM +0000, Andrew Cooper wrote:
> Hello,
> 
> I have dual socket server with the following processor:
> 
> [root@xrtmia-09-01 ~]# head /proc/cpuinfo 
> processor	: 0
> vendor_id	: AuthenticAMD
> cpu family	: 23
> model		: 1
> model name	: AMD EPYC 7281 16-Core Processor
> stepping	: 2
> 
> Which has highlighted a issue in the topology derivation logic. 
> (Actually, it was discovered with Xen, but we share the same topology
> infrastructure and the issue is also present with Linux).
> 
> There are a total of 64 threads in the system, made of two 32-thread
> sockets.  The APIC IDs for this system are sparse - they are 0x0-0x3,
> 0x8-0xb, 0x10-0x13 etc, all the way up to 0x7b.
> 
> This is because the socket is made of 4 nodes with 4 cores each, but
> space has been left in the layout for the maximum possible number of
> APIC IDs.
> 
> In particular, CPUID 0x80000008:ecx reports 0x0000601f.  That is, an
> APIC ID shift of 6 (reporting a maximum of 64 threads per socket), and
> NC as 31 (reporting 32 threads per socket in the current configuration).
> 
> c->x86_max_cores is derived from NC and shifted once to exclude threads,
> giving it a final value of 16 cores per socket.

So far so good.

> Given the sparseness of the APIC IDs, it is unsafe to allocate an array

Do we do this somewhere or is this a hypothetical thing?

> of c->x86_max_cores entries, then index it with c->cpu_core_id, as half
> the cores in the system have a cpu_core_id greater than x86_max_cores. 

You lost me here. ->cpu_core_id comes from CPUID_Fn8000001E_EBX[7:0].
Are you saying, those core IDs on your box are sparse like the APIC IDs
you mention above?

> There is no logical core ID derived during boot which might be a safe to
> use as an index.
> 
> Furthermore, the documentation indicates that these values are expected
> to be per-package, while they are all actually per-socket (with up to 4
> nodes per socket) in the EPYC case.

From Documentation/x86/topology.txt:
"
  - cpuinfo_x86.x86_max_cores:

    The number of cores in a package. This information is retrieved via CPUID."

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.