Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752181AbaLGO60 (ORCPT ); Sun, 7 Dec 2014 09:58:26 -0500 Received: from aserp1040.oracle.com ([141.146.126.69]:26169 "EHLO aserp1040.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750912AbaLGO6Z (ORCPT ); Sun, 7 Dec 2014 09:58:25 -0500 Message-ID: <54846B06.8050906@oracle.com> Date: Sun, 07 Dec 2014 09:58:14 -0500 From: Sasha Levin User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: Linus Torvalds CC: Dave Jones , Chris Mason , =?UTF-8?B?RMOibmllbCBGcmFnYQ==?= , "Paul E. McKenney" , Linux Kernel Mailing List Subject: Re: frequent lockups in 3.18rc4 References: <20141127225637.GA24019@redhat.com> <547b8a45.6e608c0a.20f9.1002@mx.google.com> <547bbe36.48548c0a.105c.779c@mx.google.com> <20141201191431.GA17385@linux.vnet.ibm.com> <547ccf74.a5198c0a.25de.26d9@mx.google.com> <20141201230339.GA20487@ret.masoncoding.com> <20141202193252.GB17595@redhat.com> <547E4C14.6040509@oracle.com> <54813C03.8040009@oracle.com> <5481C92E.6020805@oracle.com> In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Source-IP: ucsinet22.oracle.com [156.151.31.94] Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 12/05/2014 01:15 PM, Linus Torvalds wrote: > On Fri, Dec 5, 2014 at 7:03 AM, Sasha Levin wrote: >> >> Yes, it's going to a serial line, but it's only about 100 lines/second on >> average. I wouldn't expect it to cause anything to hang! > > A regular 16650 serial chip? Running at 115kbps, I assume? So that's > about 11kB/s. > > And the serial console is polling, since it can't sleep or depend on interrupts. > > At a average line length of what, 40 characters? At less than 300 > lines/s, you'd be using up 100% of one CPU. And since the printouts > are serialized, that would be all other CPU's too.. > > 100 lines/s _average_ means that I can easily see it be 300lines/s for a while. > > So yeah. The serial console is simply not designed to handle > continuous output. It's for the "occasional" stuff. > > The fact that your rcu lockups go away when you make the fault > injection be quiet makes me really suspect this is related. The lockups themselves "go away", but looking closer at the log without those extra prints, I'm seeing: [ 1458.700070] INFO: rcu_preempt detected stalls on CPUs/tasks: [ 1458.700133] (detected by 19, t=30502 jiffies, g=12293, c=12292, q=0) [ 1458.702764] INFO: Stall ended before state dump start Quite often. Maybe the extra prints were just a catalyst? Thanks, Sasha -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/