Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756286AbcDLHYp (ORCPT ); Tue, 12 Apr 2016 03:24:45 -0400 Received: from mx1.redhat.com ([209.132.183.28]:55360 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751403AbcDLHYl (ORCPT ); Tue, 12 Apr 2016 03:24:41 -0400 Date: Tue, 12 Apr 2016 09:24:34 +0200 From: Jesper Dangaard Brouer To: js1304@gmail.com Cc: Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Joonsoo Kim , brouer@redhat.com Subject: Re: [PATCH v2 11/11] mm/slab: lockless decision to grow cache Message-ID: <20160412092434.0929a04c@redhat.com> In-Reply-To: <1460436666-20462-12-git-send-email-iamjoonsoo.kim@lge.com> References: <1460436666-20462-1-git-send-email-iamjoonsoo.kim@lge.com> <1460436666-20462-12-git-send-email-iamjoonsoo.kim@lge.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3234 Lines: 79 On Tue, 12 Apr 2016 13:51:06 +0900 js1304@gmail.com wrote: > From: Joonsoo Kim > > To check whther free objects exist or not precisely, we need to grab a ^^^^^^ (spelling) > lock. But, accuracy isn't that important because race window would be > even small and if there is too much free object, cache reaper would reap > it. So, this patch makes the check for free object exisistence not to ^^^^^^^^^^^ (spelling) > hold a lock. This will reduce lock contention in heavily allocation case. > > Note that until now, n->shared can be freed during the processing by > writing slabinfo, but, with some trick in this patch, we can access it > freely within interrupt disabled period. > > Below is the result of concurrent allocation/free in slab allocation > benchmark made by Christoph a long time ago. I make the output simpler. > The number shows cycle count during alloc/free respectively so less is > better. I cannot figure out which if Christoph's tests you are using. And I even have a copy of his test here: https://github.com/netoptimizer/prototype-kernel/blob/master/kernel/mm/slab_test.c I think you need to describe the test a bit better... Looking a long time at the output on my own system, I guess you are showing results from the "Concurrent allocs". Then it would be relevant how many CPUs your system have. It would also be relevant to mention that N=10000. And perhaps mention that it means, e.g all CPUs do N=10000 alloc concurrently, synchronize before doing N free concurrently. > * Before > Kmalloc N*alloc N*free(32): Average=248/966 > Kmalloc N*alloc N*free(64): Average=261/949 > Kmalloc N*alloc N*free(128): Average=314/1016 > Kmalloc N*alloc N*free(256): Average=741/1061 > Kmalloc N*alloc N*free(512): Average=1246/1152 > Kmalloc N*alloc N*free(1024): Average=2437/1259 > Kmalloc N*alloc N*free(2048): Average=4980/1800 > Kmalloc N*alloc N*free(4096): Average=9000/2078 > > * After > Kmalloc N*alloc N*free(32): Average=344/792 > Kmalloc N*alloc N*free(64): Average=347/882 > Kmalloc N*alloc N*free(128): Average=390/959 > Kmalloc N*alloc N*free(256): Average=393/1067 > Kmalloc N*alloc N*free(512): Average=683/1229 > Kmalloc N*alloc N*free(1024): Average=1295/1325 > Kmalloc N*alloc N*free(2048): Average=2513/1664 > Kmalloc N*alloc N*free(4096): Average=4742/2172 > > It shows that allocation performance decreases for the object size up to > 128 and it may be due to extra checks in cache_alloc_refill(). But, with > considering improvement of free performance, net result looks the same. > Result for other size class looks very promising, roughly, 50% performance > improvement. Super nice performance boost. The numbers on my system are significantly smaller, but this is a before/after test and the absolute numbers are not that important. Oh, maybe this was because I ran the test with SLUB... recompiling with SLAB... and the results are comparable to your numbers (on my 8 core i7-4790K CPU @ 4.00GHz) -- Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat Author of http://www.iptv-analyzer.org LinkedIn: http://www.linkedin.com/in/brouer