Date: Tue, 27 May 2003 11:11:04 +0200
From: Andi Kleen <ak@suse.de>
To: Erich Focht <efocht@hpce.nec.com>
Cc: LSE <lse-tech@lists.sourceforge.net>,
       linux-kernel <linux-kernel@vger.kernel.org>
Subject: Re: [Lse-tech] Node affine NUMA scheduler extension
Message-ID: <20030527091104.GB31510@wotan.suse.de>
References: <200305271031.55554.efocht@hpce.nec.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <200305271031.55554.efocht@hpce.nec.com>
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1383
Lines: 27

On Tue, May 27, 2003 at 10:31:55AM +0200, Erich Focht wrote:
> This patch is an adaptation of the earlier work on the node affine
> NUMA scheduler to the NUMA features meanwhile integrated into
> 2.5. Compared to the patch posted for 2.5.39 this one is much simpler
> and easier to understand.

Yes, it is also much simpler than my implementation (I did a similar homenode
scheduler for an 2.4 kernel). The basic principles are the same.

But the main problems I have is that the tuning for threads is very 
difficult. On AMD64 where Node equals CPU it is important
to home node balance threads too. After some experiments I settled on 
homenode assignment on the first load balance (called "lazy homenode")  
When a thread clones it initially executes on the CPU of the parent, but 
there is a window until the first load balance tick where it can allocate
memory on the wrong node.  I found a lot of code runs very badly until the 
cache decay parameter is set to 0 (no special cache affinity) to allow
quick initial migration. Migration directly on fork/clone requires a lot 
of changes and also breaks down on some benchmarks.

-Andi

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/