Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752032AbbFXIq6 (ORCPT ); Wed, 24 Jun 2015 04:46:58 -0400 Received: from mail-wg0-f52.google.com ([74.125.82.52]:34590 "EHLO mail-wg0-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751448AbbFXIqx (ORCPT ); Wed, 24 Jun 2015 04:46:53 -0400 Date: Wed, 24 Jun 2015 10:46:48 +0200 From: Ingo Molnar To: Peter Zijlstra Cc: Daniel Wagner , oleg@redhat.com, paulmck@linux.vnet.ibm.com, tj@kernel.org, mingo@redhat.com, linux-kernel@vger.kernel.org, der.herr@hofr.at, dave@stgolabs.net, riel@redhat.com, viro@ZenIV.linux.org.uk, torvalds@linux-foundation.org, jlayton@poochiereds.net Subject: Re: [RFC][PATCH 00/13] percpu rwsem -v2 Message-ID: <20150624084648.GB27873@gmail.com> References: <20150622121623.291363374@infradead.org> <55884FC2.6030607@bmw-carit.de> <20150622190553.GZ3644@twins.programming.kicks-ass.net> <5589285C.2010100@bmw-carit.de> <20150623143411.GA25159@twins.programming.kicks-ass.net> <558973A7.6010407@bmw-carit.de> <20150623175012.GD3644@twins.programming.kicks-ass.net> <20150623193624.GH18673@twins.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150623193624.GH18673@twins.programming.kicks-ass.net> User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3365 Lines: 79 * Peter Zijlstra wrote: > On Tue, Jun 23, 2015 at 07:50:12PM +0200, Peter Zijlstra wrote: > > On Tue, Jun 23, 2015 at 04:56:39PM +0200, Daniel Wagner wrote: > > > flock02 > > > mean variance sigma max min > > > tip-1 11.8994 0.5874 0.7664 13.2022 8.6324 > > > tip-2 11.7394 0.5252 0.7247 13.2540 9.7513 > > > tip-3 11.8155 0.5288 0.7272 13.2700 9.9480 > > > tip+percpu-rswem-1 15.3601 0.8981 0.9477 16.8116 12.6910 > > > tip+percpu-rswem-2 15.2558 0.8442 0.9188 17.0199 12.9586 > > > tip+percpu-rswem-3 15.5297 0.6386 0.7991 17.4392 12.7992 > > > > I did indeed manage to get flock02 down to a usable level and found: > > Aside from the flock_lock_file function moving up, we also get an > increase in _raw_spin_lock. > > Before: > > 5.17% 5.17% flock02 [kernel.vmlinux] [k] _raw_spin_lock > | > ---_raw_spin_lock > | > |--99.75%-- flock_lock_file_wait > | sys_flock > | entry_SYSCALL_64_fastpath > | flock > --0.25%-- [...] > > > After: > > 7.20% 7.20% flock02 [kernel.vmlinux] [k] _raw_spin_lock > | > ---_raw_spin_lock > | > |--52.23%-- flock_lock_file_wait > | sys_flock > | entry_SYSCALL_64_fastpath > | flock > | > |--25.92%-- flock_lock_file > | flock_lock_file_wait > | sys_flock > | entry_SYSCALL_64_fastpath > | flock > | > |--21.42%-- locks_delete_lock_ctx > | flock_lock_file > | flock_lock_file_wait > | sys_flock > | entry_SYSCALL_64_fastpath > | flock > --0.43%-- [...] > > > And its not at all clear to me why this would be. It looks like > FILE_LOCK_DEFERRED is happening, but I've not yet figured out why that > would be. So I'd suggest to first compare preemption behavior: does the workload context-switch heavily, and is it the exact same context switching rate and are the points of preemption the same as well between the two kernels? [ Such high variance is often caused by (dynamically) unstable load balancing and the workload never finding a good equilibrium. Any observable locking overhead is usually just a second order concern or a symptom. Assuming the workload context switches heavily. ] Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/