Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754082AbdLDNNe (ORCPT ); Mon, 4 Dec 2017 08:13:34 -0500 Received: from mx1.redhat.com ([209.132.183.28]:37584 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752197AbdLDNNa (ORCPT ); Mon, 4 Dec 2017 08:13:30 -0500 Subject: Re: [bisected] x86 boot still broken on -rc2 To: Jakub Kicinski , LKML References: <20171201163954.2e356787@cakuba.netronome.com> <20171203172824.3d9b7ede@cakuba.netronome.com> <58b4f89a-63c2-b8bd-4414-fbc312c52697@redhat.com> Cc: "netdev@vger.kernel.org" , Thomas Gleixner From: Prarit Bhargava Message-ID: Date: Mon, 4 Dec 2017 08:13:29 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: <58b4f89a-63c2-b8bd-4414-fbc312c52697@redhat.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Mon, 04 Dec 2017 13:13:30 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3823 Lines: 94 On 12/04/2017 07:28 AM, Prarit Bhargava wrote: > > > On 12/03/2017 08:28 PM, Jakub Kicinski wrote: >> Same thing on rc2, bisected down to: >> >> commit b4c0a7326f5dc0ef7a64128b0ae7d081f4b2cbd1 (refs/bisect/bad) >> Author: Prarit Bhargava >> Date: Tue Nov 14 07:42:57 2017 -0500 >> >> x86/smpboot: Fix __max_logical_packages estimate >> >> A system booted with a small number of cores enabled per package >> panics because the estimate of __max_logical_packages is too low. >> >> This occurs when the total number of active cores across all packages is >> less than the maximum core count for a single package. e.g.: >> >> On a 4 package system with 20 cores/package where only 4 cores are >> enabled on each package, the value of __max_logical_packages is >> calculated as DIV_ROUND_UP(16 / 20) = 1 and not 4. >> >> Calculate __max_logical_packages after the cpu enumeration has completed. >> Use the boot cpu's data to extrapolate the number of packages. >> >> Signed-off-by: Prarit Bhargava >> Signed-off-by: Thomas Gleixner >> Cc: Tom Lendacky >> Cc: Andi Kleen >> Cc: Christian Borntraeger >> Cc: Peter Zijlstra >> Cc: Kan Liang >> Cc: He Chen >> Cc: Stephane Eranian >> Cc: Dave Hansen >> Cc: Piotr Luc >> Cc: Andy Lutomirski >> Cc: Arvind Yadav >> Cc: Vitaly Kuznetsov >> Cc: Borislav Petkov >> Cc: Tim Chen >> Cc: Mathias Krause >> Cc: "Kirill A. Shutemov" >> Link: https://lkml.kernel.org/r/20171114124257.22013-4-prarit@redhat.com >> >> >> On Fri, 1 Dec 2017 16:39:54 -0800, Jakub Kicinski wrote: >>> Hi! >>> >>> I'm hitting these after DaveM pulled rc1 into net-next on my Xeon >>> E5-2630 v4 box. It also happens on linux-next. Did anyone else >>> experience it? (.config attached) >>> >>> [ 5.003771] WARNING: CPU: 14 PID: 1 at ../arch/x86/events/intel/uncore.c:936 uncore_pci_probe+0x285/0x2b0 >>> [ 5.007544] Modules linked in: >>> [ 5.007544] CPU: 14 PID: 1 Comm: swapper/0 Not tainted 4.15.0-rc1-perf-00225-gb2a4e0a76b1d #782 >>> [ 5.007544] Hardware name: Dell Inc. PowerEdge R730/072T6D, BIOS 2.3.4 11/08/2016 > > I have a Dell R730 available for use. OOC are you booting with the default > BIOS options? > Jakub, I was able to reproduce this on a similar system by DISABLING hyperthreading in the BIOS. Doing this on other systems seems to have no impact. What is odd about this system when booting is that the kernel claims that hyperthreading is ENABLED: x86: Booting SMP configuration: .... node #0, CPUs: #1 #2 #3 #4 .... node #1, CPUs: #5 #6 #7 #8 #9 .... node #0, CPUs: #10 #11 #12 #13 #14 .... node #1, CPUs: #15 #16 #17 #18 #19 smp: Brought up 2 nodes, 20 CPUs smpboot: Max logical packages: 1 which means that the calculation of logical packages is wrong because ncpus = cpu_data(0).booted_cores * smp_num_siblings; ncpus = 10 * 2; ncpus = 20; smp_num_siblings is defined as "The number of threads in a core" which should be 1 if HT/SMT is disabled. It looks like my patch has exposed a bug in the smp_num_siblings calculation. I'm still debugging ... FWIW, I did test this code on systems by disabling HT/SMT in BIOS on several systems. I have tested those systems again and don't see a problem. It is something peculiar to a few systems. P.