From: ebiederm@xmission.com (Eric W. Biederman)
To: Christoph Lameter <cl@linux-foundation.org>
Cc: "H. Peter Anvin" <hpa@zytor.com>, Jeremy Fitzhardinge <jeremy@goop.org>,
       Ingo Molnar <mingo@elte.hu>, Mike Travis <travis@sgi.com>,
       Andrew Morton <akpm@linux-foundation.org>,
       Jack Steiner <steiner@sgi.com>, linux-kernel@vger.kernel.org,
       Arjan van de Ven <arjan@infradead.org>
References: <20080709165129.292635000@polaris-admin.engr.sgi.com>
	<m1abgqzv3g.fsf@frodo.ebiederm.org> <20080709200757.GD14009@elte.hu>
	<48751B57.8030605@goop.org> <48751CF9.4020901@linux-foundation.org>
	<4875209D.8010603@goop.org> <48752CCD.30507@linux-foundation.org>
	<m1wsjupuiz.fsf@frodo.ebiederm.org> <48753C99.5050408@goop.org>
	<m1ej62iq9z.fsf@frodo.ebiederm.org> <487555A8.2050007@zytor.com>
	<487556A5.5090907@goop.org> <4876194E.4080205@linux-foundation.org>
	<48761C06.3020003@zytor.com> <48762A3B.5050104@linux-foundation.org>
	<48762DD2.5090802@zytor.com> <487637A1.4080403@linux-foundation.org>
Date: Thu, 10 Jul 2008 10:26:30 -0700
In-Reply-To: <487637A1.4080403@linux-foundation.org> (Christoph Lameter's
	message of "Thu, 10 Jul 2008 11:24:01 -0500")
Message-ID: <m1prpl4pmh.fsf@frodo.ebiederm.org>
User-Agent: Gnus/5.110006 (No Gnus v0.6) Emacs/21.4 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Subject: Re: [RFC 00/15] x86_64: Optimize percpu accesses
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1991
Lines: 49

Christoph Lameter <cl@linux-foundation.org> writes:

> H. Peter Anvin wrote:
>
>> but there is a distinct lack of wiggle room, which can be resolved
>> either by using negative offsets, or by moving the kernel text area up a
>> bit from -2 GB.
>
> Lets say we reserve 256MB of cpu alloc space per processor.

First off right now reserving more than about 64KB is ridiculous.  We rightly
don't have that many per cpu variables.

> On a system with 4k processors this will result in the need for 1TB virtual
> address space for per cpu areas (note that there may be more processors in the
> future). Preferably we would calculate the address of the per cpu area by
>
> 	PERCPU_START_ADDRESS + PERCPU_SIZE * smp_processor_id()
>
> instead of looking it up in a table because that will save a memory access on
> per_cpu().

???? Optimizing per_cpu seems to be the wrong path.  If you want to go fast you
access the data on the cpu you start out on.

> The first percpu area would ideally be the per cpu segment generated by the
> linker.
>
> How would that fit into the address map? In particular the 2G distance between
> code and the first per cpu area must not be violated unless we go to a zero
> based approach.
>
> Maybe there is another way of arranging things that would allow for this?

Yes.  Start with a patch that doesn't have freaky failures that can't be understood
or bisected because the patch is too big.  The only reason we are having a conversation
about alternative implementations is because the current implementation has weird
random incomprehensible failures.  The most likely culprit is playing with
the linker.  It could be something else.

So please REFACTOR the patch that changes things to DO ONE THING PER PATCH.

Eric

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/