Return-Path: <linux-kernel-owner+w=401wt.eu-S1753134AbYANSBA@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S1753134AbYANSBA (ORCPT <rfc822;w@1wt.eu>);
	Mon, 14 Jan 2008 13:01:00 -0500
Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1750851AbYANSAv
	(ORCPT <rfc822;linux-kernel-outgoing>);
	Mon, 14 Jan 2008 13:00:51 -0500
Received: from netops-testserver-3-out.sgi.com ([192.48.171.28]:48676 "EHLO
	relay.sgi.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org
	with ESMTP id S1751356AbYANSAv (ORCPT
	<rfc822;linux-kernel@vger.kernel.org>);
	Mon, 14 Jan 2008 13:00:51 -0500
Message-ID: <478BA351.2000501@sgi.com>
Date: Mon, 14 Jan 2008 10:00:49 -0800
From: Mike Travis <travis@sgi.com>
User-Agent: Thunderbird 2.0.0.6 (X11/20070801)
MIME-Version: 1.0
To: Ingo Molnar <mingo@elte.hu>
CC: Andi Kleen <ak@suse.de>, Andrew Morton <akpm@linux-foundation.org>,
       Christoph Lameter <clameter@sgi.com>, Jack Steiner <steiner@sgi.com>,
       linux-mm@kvack.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH 00/10] x86: Reduce memory and intra-node effects with
 large count NR_CPUs
References: <20080113183453.973425000@sgi.com> <20080114081418.GB18296@elte.hu> <200801141104.18789.ak@suse.de> <20080114101133.GA23238@elte.hu>
In-Reply-To: <20080114101133.GA23238@elte.hu>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
Content-Length: 1913
Lines: 43


Ingo Molnar wrote:
> * Andi Kleen <ak@suse.de> wrote:
> 
>>> i.e. we've got ~22K bloat per CPU - which is not bad, but because 
>>> it's a static component, it hurts smaller boxes. For distributors to 
>>> enable CONFIG_NR_CPU=1024 by default i guess that bloat has to drop 
>>> below 1-2K per CPU :-/ [that would still mean 1-2MB total bloat but 
>>> that's much more acceptable than 23MB]
>> Even 1-2MB overhead would be too much for distributors I think. 
>> Ideally there must be near zero overhead for possible CPUs (and I see 
>> no principle reason why this is not possible) Worst case a low few 
>> hundred KBs, but even that would be much.
> 
> i think this patchset already gives a net win, by moving stuff from 
> NR_CPUS arrays into per_cpu area. (Travis please confirm that this is 
> indeed what the numbers show)
> 
> The (total-)size of the per-cpu area(s) grows linearly with the number 
> of CPUs, so we'll have the expected near-zero overhead on 4-8-16-32 CPUs 
> and the expected larger total overhead on 1024 CPUs.
> 
> 	Ingo

Yes, and it's just the first step.  Ideally, there is *no* extra memory
used by specifying NR_CPUS = <whatever> and all the extra memory only
comes into play when they are "possible/probable".  This means that almost
all of the data needs to be in the percpu area (and compact that as much
as possible) or in the initdata section and discarded after use.

And Andi is right, the distributors will not default the NR_CPUS to a large
value unless there is zero or very little overhead.  And since so much
depends on using standard configurations (certifications, etc.) we cannot
depend on a special build.

Thanks,
Mike

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/