Date: Mon, 6 Feb 2006 08:48:25 -0800 (PST)
From: Christoph Lameter <clameter@engr.sgi.com>
To: Paul Jackson <pj@sgi.com>
cc: Andi Kleen <ak@suse.de>, mingo@elte.hu, akpm@osdl.org, dgc@sgi.com,
       steiner@sgi.com, Simon.Derr@bull.net, linux-kernel@vger.kernel.org
Subject: Re: [PATCH 1/5] cpuset memory spread basic implementation
In-Reply-To: <20060206063549.d155c619.pj@sgi.com>
Message-ID: <Pine.LNX.4.62.0602060839440.16337@schroedinger.engr.sgi.com>
References: <20060204071910.10021.8437.sendpatchset@jackhammer.engr.sgi.com>
 <200602061109.45788.ak@suse.de> <20060206101156.GA1761@elte.hu>
 <200602061116.44040.ak@suse.de> <20060206063549.d155c619.pj@sgi.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1694
Lines: 30

Just some words of clarification here:

Memory spreading is useful if multiple processes are running on 
multiple nodes that access the same set of files (and therefore use the 
same dentries inodes etc). This is only true for very large applications.
Users typically segment a machine using cpusets to give these apps a 
range of nodes and processors to use. The rest of the system may be used
for other needs. The spreading should therefore be restricted to the set 
of nodes in use by that application.

This is very different from the typical case of a single threaded process 
roaming across some data and then terminating. In that case we always want 
placement of memory as near to the process as possible. In cases were we 
are not sure about future application behavior it is best to assume that 
node local is best. Spreading memory allocations for storage that is only 
accessed from one processor will reduce the performance of an application.

So the default operating mode needs to be node local.

There is one exception from this during system bootup. In that case we are 
allocating structures that will be finally be used by processes running on 
all nodes on the system. It is therefore advantageous to spread these 
allocations out. That is currently accomplished by setting the default 
memory allocation policy to MPOL_INTERLEAVE while a kernel boots. Memory 
allocation policies revert to default (node local) when init starts.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/