Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750716AbWJJNFI (ORCPT ); Tue, 10 Oct 2006 09:05:08 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1750705AbWJJNFF (ORCPT ); Tue, 10 Oct 2006 09:05:05 -0400 Received: from ms-smtp-02.nyroc.rr.com ([24.24.2.56]:61687 "EHLO ms-smtp-02.nyroc.rr.com") by vger.kernel.org with ESMTP id S1750700AbWJJNFD (ORCPT ); Tue, 10 Oct 2006 09:05:03 -0400 Subject: Re: [PATCH] Fix WARN_ON / WARN_ON_ONCE regression From: Steven Rostedt To: tim.c.chen@linux.intel.com Cc: Andrew Morton , Jeremy Fitzhardinge , herbert@gondor.apana.org.au, linux-kernel@vger.kernel.org, leonid.i.ananiev@intel.com In-Reply-To: <1160442541.4548.15.camel@localhost.localdomain> References: <1159916644.8035.35.camel@localhost.localdomain> <4522FB04.1080001@goop.org> <1159919263.8035.65.camel@localhost.localdomain> <45233B1E.3010100@goop.org> <1159968095.8035.76.camel@localhost.localdomain> <20061004093025.ab235eaa.akpm@osdl.org> <1159978929.8035.109.camel@localhost.localdomain> <20061004103408.1a38b8ad.akpm@osdl.org> <1160442541.4548.15.camel@localhost.localdomain> Content-Type: text/plain Date: Tue, 10 Oct 2006 09:04:34 -0400 Message-Id: <1160485474.22525.6.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.6.2 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1456 Lines: 49 Tim, Great work on finding this! On Mon, 2006-10-09 at 18:09 -0700, Tim Chen wrote: > > > The original "C" code looks very innocent: > > if (WARN_ON(__ret_warn_once)); > __warn_once = 0; > > The equivalent asm code generated by gcc looks like: > > temp = 0; > if (!WARN_ON(__ret_warn_once)) > temp = __warn_once; > __warn_once = temp; > > > As a result, a global variable is being written from all CPUs > everywhere and caused excessive cache line bouncing on SMP. > We measured that HITM event increased by 75% and > read-for-ownership event increased by 50%. Adding a > __read_mostly directive to __warn_once didn't help > because gcc still generate assembly code that write to > that global variable. Holy crap! I wonder where else in the kernel gcc is doing this. (of course I'm using gcc4 so I don't know). Is there another gcc attribute to actually tell gcc that a variable is really mostly read only (besides placing it in a mostly read only elf section)? What was wrong with the original WARN_ON_ONCE with if (unlikely(condition) && __warn_once) This didn't have the cache crash problem too, did it? I don't have a gcc3 around to test. -- Steve - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/