Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753319AbdDMLcu (ORCPT ); Thu, 13 Apr 2017 07:32:50 -0400 Received: from mx2.suse.de ([195.135.220.15]:58248 "EHLO mx1.suse.de" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751467AbdDMLcc (ORCPT ); Thu, 13 Apr 2017 07:32:32 -0400 Date: Thu, 13 Apr 2017 13:31:59 +0200 From: Borislav Petkov To: "Luck, Tony" Cc: Thomas Gleixner , Dan Williams , "Verma, Vishal L" , "linux-kernel@vger.kernel.org" , "linux-nvdimm@lists.01.org" , "ross.zwisler@linux.intel.com" , "x86@kernel.org" Subject: Re: [RFC PATCH] x86, mce: change the mce notifier to 'blocking' from 'atomic' Message-ID: <20170413113159.rc32ebiswn64nzrr@pd.tnic> References: <20170412202238.5d327vmwjqvbzzop@pd.tnic> <1492028744.2738.14.camel@intel.com> <20170412205229.GA13659@intel.com> <20170412211931.GA15771@intel.com> <20170412214749.jyt7cmyhovivtb2m@pd.tnic> <20170412221639.5klmqk4mjbvy6btx@pd.tnic> <20170412222619.GA17839@intel.com> <20170412222925.r3izasv3yuyjy62e@pd.tnic> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20170412222925.r3izasv3yuyjy62e@pd.tnic> User-Agent: NeoMutt/20170113 (1.7.2) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 6093 Lines: 169 On Thu, Apr 13, 2017 at 12:29:25AM +0200, Borislav Petkov wrote: > On Wed, Apr 12, 2017 at 03:26:19PM -0700, Luck, Tony wrote: > > We can futz with that and have them specify which chain (or both) > > that they want to be added to. > > Well, I didn't want the atomic chain to be a notifier because we can > keep it simple and non-blocking. Only the process context one will be. > > So the question is, do we even have a use case for outside consumers > hanging on the atomic chain? Because if not, we're good to go. Ok, new day, new patch. Below is what we could do: we don't call the notifier at all on the atomic path but only print the MCEs. We do log them and if the machine survives, we process them accordingly. This is only a fix for upstream so that the current issue at hand is addressed. For later, we'd need to split the paths in: critical_print_mce() or somesuch which immediately dumps the MCE to dmesg, and mce_log() which does the slow path of logging MCEs and calling the blocking notifier. Now, I'd want to have decoding of the MCE on the critical path too so I have to think about how to do that nicely. Maybe move the decoding bits which are the same between Intel and AMD in mce.c and have some vendor-specific, fast calls. We'll see. Btw, this is something Ingo has been mentioning for a while. Anyway, here's just the urgent fix for now. Thanks. --- From: Vishal Verma Date: Tue, 11 Apr 2017 16:44:57 -0600 Subject: [PATCH] x86/mce: Make the MCE notifier a blocking one The NFIT MCE handler callback (for handling media errors on NVDIMMs) takes a mutex to add the location of a memory error to a list. But since the notifier call chain for machine checks (x86_mce_decoder_chain) is atomic, we get a lockdep splat like: BUG: sleeping function called from invalid context at kernel/locking/mutex.c:620 in_atomic(): 1, irqs_disabled(): 0, pid: 4, name: kworker/0:0 [..] Call Trace: dump_stack ___might_sleep __might_sleep mutex_lock_nested ? __lock_acquire nfit_handle_mce notifier_call_chain atomic_notifier_call_chain ? atomic_notifier_call_chain mce_gen_pool_process Convert the notifier to a blocking one which gets to run only in process context. Boris: remove the notifier call in atomic context in print_mce(). For now, let's print the MCE on the atomic path so that we can make sure it goes out. We still log it for process context later. Reported-by: Ross Zwisler Signed-off-by: Vishal Verma Cc: Tony Luck Cc: Dan Williams Cc: linux-edac Cc: x86-ml Cc: Link: http://lkml.kernel.org/r/20170411224457.24777-1-vishal.l.verma@intel.com Fixes: 6839a6d96f4e ("nfit: do an ARS scrub on hitting a latent media error") Signed-off-by: Borislav Petkov --- arch/x86/kernel/cpu/mcheck/mce-genpool.c | 2 +- arch/x86/kernel/cpu/mcheck/mce-internal.h | 2 +- arch/x86/kernel/cpu/mcheck/mce.c | 18 ++++-------------- 3 files changed, 6 insertions(+), 16 deletions(-) diff --git a/arch/x86/kernel/cpu/mcheck/mce-genpool.c b/arch/x86/kernel/cpu/mcheck/mce-genpool.c index 1e5a50c11d3c..217cd4449bc9 100644 --- a/arch/x86/kernel/cpu/mcheck/mce-genpool.c +++ b/arch/x86/kernel/cpu/mcheck/mce-genpool.c @@ -85,7 +85,7 @@ void mce_gen_pool_process(struct work_struct *__unused) head = llist_reverse_order(head); llist_for_each_entry_safe(node, tmp, head, llnode) { mce = &node->mce; - atomic_notifier_call_chain(&x86_mce_decoder_chain, 0, mce); + blocking_notifier_call_chain(&x86_mce_decoder_chain, 0, mce); gen_pool_free(mce_evt_pool, (unsigned long)node, sizeof(*node)); } } diff --git a/arch/x86/kernel/cpu/mcheck/mce-internal.h b/arch/x86/kernel/cpu/mcheck/mce-internal.h index 903043e6a62b..19592ba1a320 100644 --- a/arch/x86/kernel/cpu/mcheck/mce-internal.h +++ b/arch/x86/kernel/cpu/mcheck/mce-internal.h @@ -13,7 +13,7 @@ enum severity_level { MCE_PANIC_SEVERITY, }; -extern struct atomic_notifier_head x86_mce_decoder_chain; +extern struct blocking_notifier_head x86_mce_decoder_chain; #define ATTR_LEN 16 #define INITIAL_CHECK_INTERVAL 5 * 60 /* 5 minutes */ diff --git a/arch/x86/kernel/cpu/mcheck/mce.c b/arch/x86/kernel/cpu/mcheck/mce.c index 5accfbdee3f0..8e470735b16b 100644 --- a/arch/x86/kernel/cpu/mcheck/mce.c +++ b/arch/x86/kernel/cpu/mcheck/mce.c @@ -123,7 +123,7 @@ static void (*quirk_no_way_out)(int bank, struct mce *m, struct pt_regs *regs); * CPU/chipset specific EDAC code can register a notifier call here to print * MCE errors in a human-readable form. */ -ATOMIC_NOTIFIER_HEAD(x86_mce_decoder_chain); +BLOCKING_NOTIFIER_HEAD(x86_mce_decoder_chain); /* Do initial initialization of a struct mce */ void mce_setup(struct mce *m) @@ -220,7 +220,7 @@ void mce_register_decode_chain(struct notifier_block *nb) WARN_ON(nb->priority > MCE_PRIO_LOWEST && nb->priority < MCE_PRIO_EDAC); - atomic_notifier_chain_register(&x86_mce_decoder_chain, nb); + blocking_notifier_chain_register(&x86_mce_decoder_chain, nb); } EXPORT_SYMBOL_GPL(mce_register_decode_chain); @@ -228,7 +228,7 @@ void mce_unregister_decode_chain(struct notifier_block *nb) { atomic_dec(&num_notifiers); - atomic_notifier_chain_unregister(&x86_mce_decoder_chain, nb); + blocking_notifier_chain_unregister(&x86_mce_decoder_chain, nb); } EXPORT_SYMBOL_GPL(mce_unregister_decode_chain); @@ -321,18 +321,8 @@ static void __print_mce(struct mce *m) static void print_mce(struct mce *m) { - int ret = 0; - __print_mce(m); - - /* - * Print out human-readable details about the MCE error, - * (if the CPU has an implementation for that) - */ - ret = atomic_notifier_call_chain(&x86_mce_decoder_chain, 0, m); - if (ret == NOTIFY_STOP) - return; - + mce_log(m); pr_emerg_ratelimited(HW_ERR "Run the above through 'mcelog --ascii'\n"); } -- 2.11.0 SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) --