Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755234AbbHLMd3 (ORCPT ); Wed, 12 Aug 2015 08:33:29 -0400 Received: from terminus.zytor.com ([198.137.202.10]:42540 "EHLO terminus.zytor.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752719AbbHLMdZ (ORCPT ); Wed, 12 Aug 2015 08:33:25 -0400 Date: Wed, 12 Aug 2015 05:32:51 -0700 From: tip-bot for Matt Fleming Message-ID: Cc: vikas.shivappa@intel.com, hpa@zytor.com, matt.fleming@intel.com, mingo@kernel.org, tony.luck@intel.com, tglx@linutronix.de, peterz@infradead.org, linux-kernel@vger.kernel.org, kanaka.d.juvva@intel.com, torvalds@linux-foundation.org, ashok.raj@intel.com Reply-To: mingo@kernel.org, vikas.shivappa@intel.com, matt.fleming@intel.com, hpa@zytor.com, peterz@infradead.org, tglx@linutronix.de, tony.luck@intel.com, torvalds@linux-foundation.org, kanaka.d.juvva@intel.com, linux-kernel@vger.kernel.org, ashok.raj@intel.com In-Reply-To: <1438863163-14083-1-git-send-email-matt@codeblueprint.co.uk> References: <1438863163-14083-1-git-send-email-matt@codeblueprint.co.uk> To: linux-tip-commits@vger.kernel.org Subject: [tip:perf/core] perf/x86/intel/cqm: Do not access cpu_data() from CPU_UP_PREPARE handler Git-Commit-ID: d7a702f0b1033cf402fef65bd6395072738f0844 X-Mailer: tip-git-log-daemon Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Content-Disposition: inline Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 4980 Lines: 109 Commit-ID: d7a702f0b1033cf402fef65bd6395072738f0844 Gitweb: http://git.kernel.org/tip/d7a702f0b1033cf402fef65bd6395072738f0844 Author: Matt Fleming AuthorDate: Thu, 6 Aug 2015 13:12:43 +0100 Committer: Ingo Molnar CommitDate: Wed, 12 Aug 2015 11:37:23 +0200 perf/x86/intel/cqm: Do not access cpu_data() from CPU_UP_PREPARE handler Tony reports that booting his 144-cpu machine with maxcpus=10 triggers the following WARN_ON(): [ 21.045727] WARNING: CPU: 8 PID: 647 at arch/x86/kernel/cpu/perf_event_intel_cqm.c:1267 intel_cqm_cpu_prepare+0x75/0x90() [ 21.045744] CPU: 8 PID: 647 Comm: systemd-udevd Not tainted 4.2.0-rc4 #1 [ 21.045745] Hardware name: Intel Corporation BRICKLAND/BRICKLAND, BIOS BRHSXSD1.86B.0066.R00.1506021730 06/02/2015 [ 21.045747] 0000000000000000 0000000082771b09 ffff880856333ba8 ffffffff81669b67 [ 21.045748] 0000000000000000 0000000000000000 ffff880856333be8 ffffffff8107b02a [ 21.045750] ffff88085b789800 ffff88085f68a020 ffffffff819e2470 000000000000000a [ 21.045750] Call Trace: [ 21.045757] [] dump_stack+0x45/0x57 [ 21.045759] [] warn_slowpath_common+0x8a/0xc0 [ 21.045761] [] warn_slowpath_null+0x1a/0x20 [ 21.045762] [] intel_cqm_cpu_prepare+0x75/0x90 [ 21.045764] [] intel_cqm_cpu_notifier+0x42/0x160 [ 21.045767] [] notifier_call_chain+0x4d/0x80 [ 21.045769] [] __raw_notifier_call_chain+0xe/0x10 [ 21.045770] [] _cpu_up+0xe8/0x190 [ 21.045771] [] cpu_up+0x7a/0xa0 [ 21.045774] [] cpu_subsys_online+0x40/0x90 [ 21.045777] [] device_online+0x67/0x90 [ 21.045778] [] online_store+0x8a/0xa0 [ 21.045782] [] dev_attr_store+0x18/0x30 [ 21.045785] [] sysfs_kf_write+0x3a/0x50 [ 21.045786] [] kernfs_fop_write+0x120/0x170 [ 21.045789] [] __vfs_write+0x37/0x100 [ 21.045791] [] ? __sb_start_write+0x58/0x110 [ 21.045795] [] ? security_file_permission+0x3d/0xc0 [ 21.045796] [] vfs_write+0xa9/0x190 [ 21.045797] [] SyS_write+0x55/0xc0 [ 21.045800] [] ? do_page_fault+0x30/0x80 [ 21.045804] [] entry_SYSCALL_64_fastpath+0x12/0x71 [ 21.045805] ---[ end trace fe228b836d8af405 ]--- The root cause is that CPU_UP_PREPARE is completely the wrong notifier action from which to access cpu_data(), because smp_store_cpu_info() won't have been executed by the target CPU at that point, which in turn means that ->x86_cache_max_rmid and ->x86_cache_occ_scale haven't been filled out. Instead let's invoke our handler from CPU_STARTING and rename it appropriately. Reported-by: Tony Luck Signed-off-by: Matt Fleming Signed-off-by: Peter Zijlstra (Intel) Cc: Ashok Raj Cc: Kanaka Juvva Cc: Linus Torvalds Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Vikas Shivappa Link: http://lkml.kernel.org/r/1438863163-14083-1-git-send-email-matt@codeblueprint.co.uk Signed-off-by: Ingo Molnar --- arch/x86/kernel/cpu/perf_event_intel_cqm.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/arch/x86/kernel/cpu/perf_event_intel_cqm.c b/arch/x86/kernel/cpu/perf_event_intel_cqm.c index 63eb68b..377e8f8 100644 --- a/arch/x86/kernel/cpu/perf_event_intel_cqm.c +++ b/arch/x86/kernel/cpu/perf_event_intel_cqm.c @@ -1255,7 +1255,7 @@ static inline void cqm_pick_event_reader(int cpu) cpumask_set_cpu(cpu, &cqm_cpumask); } -static void intel_cqm_cpu_prepare(unsigned int cpu) +static void intel_cqm_cpu_starting(unsigned int cpu) { struct intel_pqr_state *state = &per_cpu(pqr_state, cpu); struct cpuinfo_x86 *c = &cpu_data(cpu); @@ -1296,13 +1296,11 @@ static int intel_cqm_cpu_notifier(struct notifier_block *nb, unsigned int cpu = (unsigned long)hcpu; switch (action & ~CPU_TASKS_FROZEN) { - case CPU_UP_PREPARE: - intel_cqm_cpu_prepare(cpu); - break; case CPU_DOWN_PREPARE: intel_cqm_cpu_exit(cpu); break; case CPU_STARTING: + intel_cqm_cpu_starting(cpu); cqm_pick_event_reader(cpu); break; } @@ -1373,7 +1371,7 @@ static int __init intel_cqm_init(void) goto out; for_each_online_cpu(i) { - intel_cqm_cpu_prepare(i); + intel_cqm_cpu_starting(i); cqm_pick_event_reader(i); } -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/