Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1759260Ab3JON1e (ORCPT ); Tue, 15 Oct 2013 09:27:34 -0400 Received: from e8.ny.us.ibm.com ([32.97.182.138]:55362 "EHLO e8.ny.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1759224Ab3JON1d (ORCPT ); Tue, 15 Oct 2013 09:27:33 -0400 Date: Tue, 15 Oct 2013 06:27:18 -0700 From: "Paul E. McKenney" To: Frederic Weisbecker Cc: Peter Zijlstra , "Liu, Chuansheng" , Steven Rostedt , "Ingo Molnar (mingo@kernel.org)" , "hpa@zytor.com" , "akpm@linux-foundation.org" , "x86@kernel.org" , "'linux-kernel@vger.kernel.org' (linux-kernel@vger.kernel.org)" , "Wang, Xiaoming" , "Li, Zhuangzhi" Subject: Re: Panic and page fault in loop during handling NMI backtrace handler Message-ID: <20131015132718.GF9150@linux.vnet.ibm.com> Reply-To: paulmck@linux.vnet.ibm.com References: <27240C0AC20F114CBF8149A2696CBE4A01B60835@SHSMSX101.ccr.corp.intel.com> <20131015121849.GA29884@localhost.localdomain> <20131015123717.GJ10651@twins.programming.kicks-ass.net> <20131015124837.GC9150@linux.vnet.ibm.com> <20131015125914.GC29884@localhost.localdomain> <20131015130659.GD9150@linux.vnet.ibm.com> <20131015131757.GD29884@localhost.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20131015131757.GD29884@localhost.localdomain> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-MML: No X-Content-Scanned: Fidelis XPS MAILER x-cbid: 13101513-0320-0000-0000-0000015C9086 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2093 Lines: 40 On Tue, Oct 15, 2013 at 03:17:59PM +0200, Frederic Weisbecker wrote: > On Tue, Oct 15, 2013 at 06:06:59AM -0700, Paul E. McKenney wrote: > > On Tue, Oct 15, 2013 at 02:59:15PM +0200, Frederic Weisbecker wrote: > > > On Tue, Oct 15, 2013 at 05:48:37AM -0700, Paul E. McKenney wrote: > > > > On Tue, Oct 15, 2013 at 02:37:17PM +0200, Peter Zijlstra wrote: > > > > > On Tue, Oct 15, 2013 at 02:18:53PM +0200, Frederic Weisbecker wrote: > > > > > > On Tue, Oct 15, 2013 at 02:01:04AM +0000, Liu, Chuansheng wrote: > > > > > > > We meet one issue that during trigger all CPU backtrace, but during in the NMI handler arch_trigger_all_cpu_backtrace_handler, > > > > > > > It hit the PAGE fault, then PAGE fault is in loop, at last the thread stack overflow, and system panic. > > > > > > > > > > > > > > Anyone can give some help? Thanks. > > > > > > > > > > > > Looks like we re-enter the fault several times. On x86-32, NMIs can > > > > > > fault if they dereference vmalloc'ed area. I wonder if the module thing > > > > > > we lookup in the NMI is stored on some vmalloc'ed area. > > > > > > > > > > IIRC modules are indeed allocated using vmalloc. See module_alloc() > > > > > using vmalloc_exec() > > > > > > > > This might then be a module that uses call_rcu(), but which does not have > > > > the needed rcu_barrier() in the module-exit function. > > > > > > I rather believe it's due to the lazy paging of vmalloc area in x86-32. We had issues > > > like that in the past. For example that's the reason why we do an ad-hoc per-cpu > > > allocation on callchain buffers in perf rather than using alloc_percpu() which might > > > use vmalloc. > > > > I must defer to your greater experience with this type of bug. > > With some chances I'll be proved wrong. I hope, because that issue is not easily fixed. Fair enough! Thanx, Paul -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/