Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753245Ab2JMPK0 (ORCPT ); Sat, 13 Oct 2012 11:10:26 -0400 Received: from mail-wg0-f44.google.com ([74.125.82.44]:61234 "EHLO mail-wg0-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751898Ab2JMPKZ (ORCPT ); Sat, 13 Oct 2012 11:10:25 -0400 Subject: Re: [Q] Default SLAB allocator From: Eric Dumazet To: David Rientjes Cc: Andi Kleen , Ezequiel Garcia , Linux Kernel Mailing List , linux-mm@kvack.org, Tim Bird , celinux-dev@lists.celinuxforum.org In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Date: Sat, 13 Oct 2012 17:10:21 +0200 Message-ID: <1350141021.21172.14949.camel@edumazet-glaptop> Mime-Version: 1.0 X-Mailer: Evolution 2.28.3 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1612 Lines: 39 On Sat, 2012-10-13 at 02:51 -0700, David Rientjes wrote: > On Thu, 11 Oct 2012, Andi Kleen wrote: > > > When did you last test? Our regressions had disappeared a few kernels > > ago. > > > > This was in August when preparing for LinuxCon, I tested netperf TCP_RR on > two 64GB machines (one client, one server), four nodes each, with thread > counts in multiples of the number of cores. SLUB does a comparable job, > but once we have the the number of threads equal to three times the number > of cores, it degrades almost linearly. I'll run it again next week and > get some numbers on 3.6. In latest kernels, skb->head no longer use kmalloc()/kfree(), so SLAB vs SLUB is less a concern for network loads. In 3.7, (commit 69b08f62e17) we use fragments of order-3 pages to populate skb->head. SLUB was really bad in the common workload you describe (allocations done by one cpu, freeing done by other cpus), because all kfree() hit the slow path and cpus contend in __slab_free() in the loop guarded by cmpxchg_double_slab(). SLAB has a cache for this, while SLUB directly hit the main "struct page" to add the freed object to freelist. I played some months ago adding a percpu associative cache to SLUB, then just moved on other strategy. (Idea for this per cpu cache was to build a temporary free list of objects to batch accesses to struct page) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/