Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756142AbaFRUno (ORCPT ); Wed, 18 Jun 2014 16:43:44 -0400 Received: from cantor2.suse.de ([195.135.220.15]:51779 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754778AbaFRUnh (ORCPT ); Wed, 18 Jun 2014 16:43:37 -0400 Date: Wed, 18 Jun 2014 22:43:35 +0200 From: "Luis R. Rodriguez" To: Petr =?iso-8859-1?Q?Ml=E1dek?= Cc: "Luis R. Rodriguez" , hpa@linux.intel.com, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, Andrew Lunn , Stephen Warren , Michal Hocko , Joe Perches , Arun KS , Kees Cook , Davidlohr Bueso , Chris Metcalf Subject: Re: [RFT 2/2] printk: allow increasing the ring buffer depending on the number of CPUs Message-ID: <20140618204335.GQ4841@wotan.suse.de> References: <1403090065-13879-1-git-send-email-mcgrof@do-not-panic.com> <1403090065-13879-2-git-send-email-mcgrof@do-not-panic.com> <20140618154255.GJ634@pathway.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20140618154255.GJ634@pathway.suse.cz> User-Agent: Mutt/1.5.17 (2007-11-01) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Jun 18, 2014 at 05:42:55PM +0200, Petr Ml?dek wrote: > On Wed 2014-06-18 04:14:25, Luis R. Rodriguez wrote: > > From: "Luis R. Rodriguez" > > > > The default size of the ring buffer is too small for machines > > with a large amount of CPUs under heavy load. What ends up > > happening when debugging is the ring buffer overlaps and chews > > up old messages making debugging impossible unless the size is > > passed as a kernel parameter. An idle system upon boot up will > > on average spew out only about one or two extra lines but where > > this really matters is on heavy load and that will vary widely > > depending on the system and environment. > > > > There are mechanisms to help increase the kernel ring buffer > > for tracing through debugfs, and those interfaces even allow growing > > the kernel ring buffer per CPU. We also have a static value which > > can be passed upon boot. Relying on debugfs however is not ideal > > for production, and relying on the value passed upon bootup is > > can only used *after* an issue has creeped up. Instead of being > > reactive this adds a proactive measure which lets you scale the > > amount of contributions you'd expect to the kernel ring buffer > > under load by each CPU in the worst case scenario. > > > > We use num_possible_cpus() to avoid complexities which could be > > introduced by dynamically changing the ring buffer size at run > > time, num_possible_cpus() lets us use the upper limit on possible > > number of CPUs therefore avoiding having to deal with hotplugging > > CPUs on and off. This introduces the kernel configuration option > > LOG_CPU_MIN_BUF_SHIFT which is used to specify the maximum amount > > of contributions to the kernel ring buffer in the worst case before > > the kernel ring buffer flips over, the size is specified as a power > > of 2. The total amount of contributions made by each CPU must be > > greater than half of the default kernel ring buffer size > > (1 << LOG_BUF_SHIFT bytes) in order to trigger an increase upon > > bootup. The kernel ring buffer is increased to the next power of > > two that would fit the required minimum kernel ring buffer size > > plus the additional CPU contribution. For example if LOG_BUF_SHIFT > > is 18 (256 KB) you'd require at least 128 KB contributions by > > other CPUs in order to trigger an increase of the kernel ring buffer. > > With a LOG_CPU_BUF_SHIFT of 12 (4 KB) you'd require at least > > anything over > 64 possible CPUs to trigger an increase. If you > > had 128 possible CPUs the amount of minimum required kernel ring > > buffer bumps to: > > > > ((1 << 18) + ((128 - 1) * (1 << 12))) / 1024 = 764 KB > > > > Since we require the ring buffer to be a power of two the new > > required size would be 1024 KB. > > > > This CPU contributions are ignored when the "log_buf_len" kernel parameter > > is used as it forces the exact size of the ring buffer to an expected power > > of two value. > > > > Cc: Andrew Lunn > > Cc: Stephen Warren > > Cc: Michal Hocko > > Cc: Petr Mladek > > Cc: Andrew Morton > > Cc: Joe Perches > > Cc: Arun KS > > Cc: Kees Cook > > Cc: Davidlohr Bueso > > Cc: Chris Metcalf > > Cc: linux-kernel@vger.kernel.org > > Signed-off-by: Luis R. Rodriguez > > It is where we ended after several versions. I am happy with this > state. > > Signed-off-by: Petr Mladek > Tested-by: Petr Mladek Thanks, I did some minor spell fixes, and also renamed min to max for the kconfig variable. Will send as final patches now. Luis -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/