Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755879Ab2JPStV (ORCPT ); Tue, 16 Oct 2012 14:49:21 -0400 Received: from mail-ia0-f174.google.com ([209.85.210.174]:47547 "EHLO mail-ia0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755827Ab2JPStT (ORCPT ); Tue, 16 Oct 2012 14:49:19 -0400 MIME-Version: 1.0 In-Reply-To: <507DAB0F.30000@am.sony.com> References: <1350392160.3954.986.camel@edumazet-glaptop> <507DA245.9050709@am.sony.com> <507DAB0F.30000@am.sony.com> Date: Tue, 16 Oct 2012 15:49:18 -0300 Message-ID: Subject: Re: [Q] Default SLAB allocator From: Ezequiel Garcia To: Tim Bird Cc: Eric Dumazet , David Rientjes , Andi Kleen , Linux Kernel Mailing List , "linux-mm@kvack.org" , "celinux-dev@lists.celinuxforum.org" Content-Type: text/plain; charset=ISO-8859-1 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2751 Lines: 69 On Tue, Oct 16, 2012 at 3:44 PM, Tim Bird wrote: > On 10/16/2012 11:27 AM, Ezequiel Garcia wrote: >> On Tue, Oct 16, 2012 at 3:07 PM, Tim Bird wrote: >>> On 10/16/2012 05:56 AM, Eric Dumazet wrote: >>>> On Tue, 2012-10-16 at 09:35 -0300, Ezequiel Garcia wrote: >>>> >>>>> Now, returning to the fragmentation. The problem with SLAB is that >>>>> its smaller cache available for kmalloced objects is 32 bytes; >>>>> while SLUB allows 8, 16, 24 ... >>>>> >>>>> Perhaps adding smaller caches to SLAB might make sense? >>>>> Is there any strong reason for NOT doing this? >>>> >>>> I would remove small kmalloc-XX caches, as sharing a cache line >>>> is sometime dangerous for performance, because of false sharing. >>>> >>>> They make sense only for very small hosts. >>> >>> That's interesting... >>> >>> It would be good to measure the performance/size tradeoff here. >>> I'm interested in very small systems, and it might be worth >>> the tradeoff, depending on how bad the performance is. Maybe >>> a new config option would be useful (I can hear the groans now... :-) >>> >>> Ezequiel - do you have any measurements of how much memory >>> is wasted by 32-byte kmalloc allocations for smaller objects, >>> in the tests you've been doing? >> >> Yes, we have some numbers: >> >> http://elinux.org/Kernel_dynamic_memory_analysis#Kmalloc_objects >> >> Are they too informal? I can add some details... > > >> They've been measured on a **very** minimal setup, almost every option >> is stripped out, except from initramfs, sysfs, and trace. >> >> On this scenario, strings allocated for file names and directories >> created by sysfs >> are quite noticeable, being 4-16 bytes, and produce a lot of fragmentation from >> that 32 byte cache at SLAB. > > The detail I'm interested in is the amount of wastage for a > "common" workload, for each of the SLxB systems. Are we talking a > few K, or 10's or 100's of K? It sounds like it's all from short strings. > Are there other things using the 32-byte kmalloc cache, that waste > a lot of memory (in aggregate) as well? > A more "Common" workload is one of the next items on my queue. > Does your tool indicate a specific callsite (or small set of callsites) > where these small allocations are made? It sounds like it's in the filesystem > and would be content-driven (by the length of filenames)? > That's right. And, IMHO, the problem is precisely that the allocation size is content-driven. Ezequiel -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/