Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1161089AbXALVqJ (ORCPT ); Fri, 12 Jan 2007 16:46:09 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1161093AbXALVqJ (ORCPT ); Fri, 12 Jan 2007 16:46:09 -0500 Received: from omx1-ext.sgi.com ([192.48.179.11]:58858 "EHLO omx1.sgi.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1161089AbXALVqG (ORCPT ); Fri, 12 Jan 2007 16:46:06 -0500 Date: Fri, 12 Jan 2007 13:45:43 -0800 (PST) From: Christoph Lameter To: Ravikiran G Thirumalai cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andi Kleen , Andrew Morton , "Shai Fultheim (Shai@scalex86.org)" , pravin b shelar , a.p.zijlstra@chello.nl Subject: Re: High lock spin time for zone->lru_lock under extreme conditions In-Reply-To: <20070112214021.GA4300@localhost.localdomain> Message-ID: References: <20070112160104.GA5766@localhost.localdomain> <20070112214021.GA4300@localhost.localdomain> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1154 Lines: 24 On Fri, 12 Jan 2007, Ravikiran G Thirumalai wrote: > > Does the system scale the right way if you stay within the bounds of node > > memory? I.e. allocate 1.5GB from each process? > > Yes. We see problems only when we oversubscribe memory. Ok in that case we can have more than 2 processors trying to acquire the same zone lock. If they have all exhausted their node local memory and are all going off node then all processor may be hitting the last node that has some memory left which will cause a very high degree of contention. Moreover mostatomic operations are to remote memory which is also increasing the problem by making the atomic ops take longer. Typically mature NUMA system have implemented hardware provisions that can deal with such high degrees of contention. If this is simply a SMP system that was turned into a NUMA box then this is a new hardware scenario for the engineers. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/