Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752152AbZIQKgm (ORCPT ); Thu, 17 Sep 2009 06:36:42 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1751495AbZIQKgm (ORCPT ); Thu, 17 Sep 2009 06:36:42 -0400 Received: from mk-filter-2-a-1.mail.uk.tiscali.com ([212.74.100.53]:15835 "EHLO mk-filter-2-a-1.mail.uk.tiscali.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750883AbZIQKgl (ORCPT ); Thu, 17 Sep 2009 06:36:41 -0400 X-Trace: 262678829/mk-filter-2.mail.uk.tiscali.com/B2C/$b2c-THROTTLED-DYNAMIC/b2c-CUSTOMER-DYNAMIC-IP/79.69.8.160/None/hugh.dickins@tiscali.co.uk X-SBRS: None X-RemoteIP: 79.69.8.160 X-IP-MAIL-FROM: hugh.dickins@tiscali.co.uk X-SMTP-AUTH: X-MUA: X-IP-BHB: Once X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Aq8EAO+tsUpPRQig/2dsb2JhbACBU9xlhBgFgVg X-IronPort-AV: E=Sophos;i="4.44,403,1249254000"; d="scan'208";a="262678829" Date: Thu, 17 Sep 2009 11:35:54 +0100 (BST) From: Hugh Dickins X-X-Sender: hugh@sister.anvils To: Peter Zijlstra cc: "Zhang, Yanmin" , LKML , Ingo Molnar , Arjan van de Ven , Andrew Morton , Andi Kleen , "lee.schermerhorn@hp.com" Subject: Re: aim7 scalability issue on 4 socket machine In-Reply-To: <1253180411.8497.1.camel@twins> Message-ID: References: <1253179879.2606.37.camel@ymzhang> <1253180411.8497.1.camel@twins> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2154 Lines: 48 On Thu, 17 Sep 2009, Peter Zijlstra wrote: > On Thu, 2009-09-17 at 17:31 +0800, Zhang, Yanmin wrote: > > Aim7 result is bad on my new Nehalem machines (4*8*2 logical cpu). Perf counter > > shows spinlock consumes 70% cpu time on the machine. Lock_stat shows > > anon_vma->lock causes most of the spinlock contention. Function tracer shows > > below call chain creates the spinlock. > > > > do_brk => vma_merge =>vma_adjust > > > > Aim7 consists of lots of subtests. One test is to fork lots of processes and > > every process calls sbrk for 1000 times to grow/shrink the heap. All the vma of > > the heap of all sub-processes point to the same anon_vma and use the same > > anon_vma->lock. When sbrk is called, kernel calls do_brk => vma_merge =>vma_adjust > > and lock anon_vma->lock to create spinlock contentions. > > > > There is a comment section in front of spin_lock(&anon_vma->lock. It says > > anon_vma lock can be optimized when just changing vma->vm_end. As a matter > > of fact, anon_vma->lock is used to protect anon_vma->list when an entry is > > deleted/inserted or the list is accessed. There is no such deletion/insertion > > if only vma->end is changed in function vma_adjust. > > > > Below patch fixes it. > > > > Test results with kernel 2.6.31-rc8. The improvement on the machine is about 150%. > > Did you see Lee's patch?: > > http://lkml.org/lkml/2009/9/9/290 > > Added Lee and Hugh to CC, retained the below patch for them. Thanks a lot for the CC, Peter. See my reply to that mail for the slightly corrected version. Yes, Yanmin and Lee appear to be fixing exactly the same issue. I haven't thought through Yanmin's version for correctness, but it lacks the vm_start check I added to Lee's, and I do prefer Lee's style - hey, nothing personal! So, Yanmin, please retest with http://lkml.org/lkml/2009/9/13/25 and let us know if that works as well for you - thanks. Hugh -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/