Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753963AbbGTFxR (ORCPT ); Mon, 20 Jul 2015 01:53:17 -0400 Received: from hotel311.server4you.de ([85.25.146.15]:49457 "EHLO hotel311.server4you.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751960AbbGTFxP (ORCPT ); Mon, 20 Jul 2015 01:53:15 -0400 Subject: Re: [RFC][PATCH 00/13] percpu rwsem -v2 To: Peter Zijlstra , Linus Torvalds References: <20150622190553.GZ3644@twins.programming.kicks-ass.net> <5589285C.2010100@bmw-carit.de> <20150623143411.GA25159@twins.programming.kicks-ass.net> <558973A7.6010407@bmw-carit.de> <20150623175012.GD3644@twins.programming.kicks-ass.net> <20150623193624.GH18673@twins.programming.kicks-ass.net> <20150624084648.GB27873@gmail.com> <558A75EA.40905@bmw-carit.de> <55938163.7020606@monom.org> <20150702094155.GW19282@twins.programming.kicks-ass.net> Cc: Ingo Molnar , Oleg Nesterov , Paul McKenney , Tejun Heo , Ingo Molnar , Linux Kernel Mailing List , der.herr@hofr.at, Davidlohr Bueso , Rik van Riel , Al Viro , Jeff Layton From: Daniel Wagner X-Enigmail-Draft-Status: N1110 Message-ID: <55AC8CC4.1020802@monom.org> Date: Mon, 20 Jul 2015 07:53:08 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 In-Reply-To: <20150702094155.GW19282@twins.programming.kicks-ass.net> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2435 Lines: 62 On 07/02/2015 11:41 AM, Peter Zijlstra wrote: > On Wed, Jul 01, 2015 at 02:54:59PM -0700, Linus Torvalds wrote: >> On Tue, Jun 30, 2015 at 10:57 PM, Daniel Wagner wrote: >>> >>> And an attempt at visualization: >>> >>> http://monom.org/posix01/sweep-4.1.0-02756-ge3d06bd.png >>> http://monom.org/posix01/sweep-4.1.0-02769-g6ce2591.png >> >> Ugh. The old numbers look (mostly) fairly tight, and then the new ones >> are all over the map, and usually much worse. >> >> We've seen this behavior before when switching from a non-sleeping >> lock to a sleeping one. The sleeping locks have absolutely horrible >> behavior when they get contended, and spend tons of CPU time on the >> sleep/wakeup management, > > Right, I'm just not seeing how any of that would happen here :/ The read > side would only ever block on reading /proc/$something and I'm fairly > sure that benchmark doesn't actually touch that file. > > In any case, I will look into this, I've just not had time yet.. I did some more testing and found out that the slow path of percpu_down_read() is never taken (as expected). The only change left is the exchange from a percpu arch_spinlock_t spinlocks to percpu spinlock_t spinlocks. Turning them back into arch_spinlock_t gives almost the same numbers as with spinlock_t. Then Peter suggested to change the code to preempt_disable(); spin_unlock(); preempt_enable_no_resched(); to verify if arch_spin_lock() is buggy and does not disable preemption and we see a lock holder preemption on non virt setups. Here all the numbers and plots: - base line http://monom.org/posix01-4/tip-4.1.0-02756-ge3d06bd.png http://monom.org/posix01-4/tip-4.1.0-02756-ge3d06bd.txt - arch_spinlock_t http://monom.org/posix01-4/arch_spintlock_t-4.1.0-02769-g6ce2591-dirty.png http://monom.org/posix01-4/arch_spintlock_t-4.1.0-02769-g6ce2591-dirty.txt http://monom.org/posix01-4/arch_spintlock_t-4.1.0-02769-g6ce2591-dirty.patch - no resched http://monom.org/posix01-4/no_resched-4.1.0-02770-g4d518cf.png http://monom.org/posix01-4/no_resched-4.1.0-02770-g4d518cf.txt http://monom.org/posix01-4/no_resched-4.1.0-02770-g4d518cf.patch cheers, daniel -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/