Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1765427AbYCGFT6 (ORCPT ); Fri, 7 Mar 2008 00:19:58 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1753162AbYCGFTr (ORCPT ); Fri, 7 Mar 2008 00:19:47 -0500 Received: from ns.suse.de ([195.135.220.2]:43188 "EHLO mx1.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752364AbYCGFTq (ORCPT ); Fri, 7 Mar 2008 00:19:46 -0500 Date: Fri, 7 Mar 2008 06:19:44 +0100 From: Nick Piggin To: Christoph Lameter Cc: netdev@vger.kernel.org, Linux Kernel Mailing List , yanmin_zhang@linux.intel.com, David Miller , Eric Dumazet Subject: Re: [rfc][patch 2/3] slab: introduce SMP alignment Message-ID: <20080307051944.GJ21185@wotan.suse.de> References: <20080303093529.GB15091@wotan.suse.de> <20080303200355.GB8974@wotan.suse.de> <20080303201211.GE8974@wotan.suse.de> <20080303202411.GH8974@wotan.suse.de> <20080307043700.GI21185@wotan.suse.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.9i Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1908 Lines: 36 On Thu, Mar 06, 2008 at 09:11:39PM -0800, Christoph Lameter wrote: > On Fri, 7 Mar 2008, Nick Piggin wrote: > > > > No the slab allocators were optimized for VSMP so that the > > > internode_alignment is not necessary there. That was actually one of the > > > requirements that triggered the slab numa work. > > > > BTW. can you explain this incredible claim that the slab allocators > > do not need internode_alignment to avoid false sharing of allocated > > objects on VSMP? I don't understand how that would work at all. > > The queues are per node which means that pages from which we allocate are > distinct per node. As long as processes allocate and free object on a > single node all is fine. Problems can arise though if objects are used > across node. This does not solve the problem. Say if you have 2 objects allocated from a given slab and you want to avoid cacheline contention between them, then you have to always ensure they don't overlap on the same cacheline. The fact this happens naturally when you allocate 2 objects from 2 different nodes doesn't really help: the same single CPU can allocate objects that you want to avoid false sharing with. Consider a task struct for example: if you start up a whole lot of worker threads from a single CPU, then you will allocate them all from that one CPU. Then they are spread over all CPUs and you get cacheline contention. Which is why VSMP would legitimately want to use internode alignment on some structures. And internode alignment is total overkill on any other type of system, so you can't go around annotating all your structures with it and hope KMEM_CACHE does the right thing. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/