Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758849AbcJYMWM (ORCPT ); Tue, 25 Oct 2016 08:22:12 -0400 Received: from Galois.linutronix.de ([146.0.238.70]:33008 "EHLO Galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754121AbcJYMWI (ORCPT ); Tue, 25 Oct 2016 08:22:08 -0400 Date: Tue, 25 Oct 2016 14:22:05 +0200 From: Sebastian Andrzej Siewior To: "Charles (Chas) Williams" Cc: linux-kernel@vger.kernel.org, rt@linutronix.de Subject: Re: [PREEMPT-RT] Oops in rapl_cpu_prepare() Message-ID: <20161025122205.cw5xyejcg7xegnmq@linutronix.de> References: <20161021105630.y2iym7smtdpyo54z@linutronix.de> <4e56a576-1a9f-e195-5ed9-2bc7169c4d94@brocade.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <4e56a576-1a9f-e195-5ed9-2bc7169c4d94@brocade.com> User-Agent: NeoMutt/20161014 (1.7.1) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2753 Lines: 69 On 2016-10-21 17:03:56 [-0400], Charles (Chas) Williams wrote: > I can't get dedicated access to the specific bare metal since it is > running as a dedicated hypervisor. I haven't seen this issue anywhere > else though with the 4.8 kernel. That is something :) > > If a callback (such as CPUHP_PERF_X86_RAPL_PREP) fail then we rollback > > to the starting point (in case of CPU up it would be CPUHP_OFFLINE. > > You'll like this, I just did a little printk debugging because it was > easier than trying to get a debugger running: > > [ 3.107126] init_rapl_pmus: maxpkg 4 there! vmware bug. It probably worked by chance. > [ 3.107263] rapl_cpu_prepare: pmu ffff880234faa540 cpu 0 pkgid 0 > [ 3.107400] rapl_cpu_prepare: pmu ffff880234faa600 cpu 1 pkgid 2 > [ 3.107537] rapl_cpu_prepare: pmu ffff880234faa6c0 cpu 2 pkgid 65535 > [ 3.107662] rapl_cpu_online: pmu ffff880234faa540 cpu 0 pkgid 0 > [ 3.107907] rapl_cpu_online: pmu ffff880234faa600 cpu 1 pkgid 2 > [ 3.108133] rapl_cpu_online: pmu ffff880234faa6c0 cpu 2 pkgid 65535 > [ 3.108333] rapl_cpu_online: pmu ffff880234faa6c0 cpu 3 pkgid 65535 > > where pkgid is topology_logical_package_id(cpu). > > I can't understand why I don't see a cpu 3 during cpu prepare, when I > see one later. because cpu 2 and 3 share the same package and if your printk is at the bottom of the function, it will return early. > The 65535 is a -1 from topology_phys_to_logical_pkg() > getting assigned to the logical_proc_id apparently. yes. The topology field is u16. > So this is pretty puzzling. Since this is a guest running under VMWare, I > don't know that there is any particular CPU pinning or emulation of RAPL. I assume "init_rapl_pmus: maxpkg 4" is from init_rapl_pmus() returning topology_max_packages(). So it says 4 but then returns 65535 for CPU 2 and 3. That -1 comes probably from topology_update_package_map(). Could you please send a complete boot log and try the following patch? This one should fix your boot problem and disable RAPL if the info is invalid. diff --git a/arch/x86/events/intel/rapl.c b/arch/x86/events/intel/rapl.c index 0a535cea8ff3..f5d85f2853d7 100644 --- a/arch/x86/events/intel/rapl.c +++ b/arch/x86/events/intel/rapl.c @@ -682,6 +682,15 @@ static int __init init_rapl_pmus(void) { int maxpkg = topology_max_packages(); size_t size; + unsigned int cpu; + + for_each_possible_cpu(cpu) { + if (topology_logical_package_id(cpu) >= maxpkg) { + pr_err("rapl pmu error: max package: %u but CPU%d belongs to %u\n", + maxpkg, cpu, topology_logical_package_id(cpu)); + return -EINVAL; + } + } size = sizeof(*rapl_pmus) + maxpkg * sizeof(struct rapl_pmu *); rapl_pmus = kzalloc(size, GFP_KERNEL); Sebastian