Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754027AbbFWJfm (ORCPT ); Tue, 23 Jun 2015 05:35:42 -0400 Received: from mail.bmw-carit.de ([62.245.222.98]:53677 "EHLO mail.bmw-carit.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751450AbbFWJfc (ORCPT ); Tue, 23 Jun 2015 05:35:32 -0400 X-CTCH-RefID: str=0001.0A0C0204.5589285D.01B9,ss=1,re=0.000,recu=0.000,reip=0.000,cl=1,cld=1,fgs=0 Message-ID: <5589285C.2010100@bmw-carit.de> Date: Tue, 23 Jun 2015 11:35:24 +0200 From: Daniel Wagner User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: Peter Zijlstra , Daniel Wagner CC: , , , , , , , , , , Subject: Re: [RFC][PATCH 00/13] percpu rwsem -v2 References: <20150622121623.291363374@infradead.org> <55884FC2.6030607@bmw-carit.de> <20150622190553.GZ3644@twins.programming.kicks-ass.net> In-Reply-To: <20150622190553.GZ3644@twins.programming.kicks-ass.net> Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 6615 Lines: 167 On 06/22/2015 09:05 PM, Peter Zijlstra wrote: > On Mon, Jun 22, 2015 at 08:11:14PM +0200, Daniel Wagner wrote: >> On 06/22/2015 02:16 PM, Peter Zijlstra wrote: >>> Also, since Linus thinks lglocks is a failed locking primitive (which I whole >>> heartedly agree with, its preempt-disable latencies are an abomination), it >>> also converts the global part of fs/locks's usage of lglock over to a >>> percpu-rwsem and uses a per-cpu spinlock for the local part. This both provides >>> another (4th) percpu-rwsem users and removes an lglock user. >> >> I did a quick lockperf run with these patches on a 4 socket E5-4610 machine. >> These microbenches execercise the fs' locks a bit. >> >> I suspect I got the wrong tree. The patches did not apply cleanly. The resulting >> kernel boots fine and doesn't explode... so far... > > Its against tip/master, although I expect the locking/core bits that > were sent to Linus earlier today to be the biggest missing piece. > > All I really did was build a kernel with lockdep enabled and boot + > build a kernel to see it didn't go belly up. > >> The results aren't looking too bad. Though building a kernel with 'make -j200' >> was extreme slow. I'll look into it tomorrow. So this turns out to be false alarm. I had icecream installed/actived and that interfered with gcc. Stupid me. The machine has 0.5TB memory and doesn't seem to be really concerned about 'make -j200' make clean && time make -j200 mainline 4.1.0 2nd run real 1m7.595s user 28m43.125s sys 3m48.189s tip v4.1-2756-ge3d06bd 2nd run real 1m6.871s user 28m50.803s sys 3m50.223s 3rd run real 1m6.974s user 28m52.093s sys 3m50.259s tip v4.1-2769-g6ce2591 (percpu-rwsem) 2nd run real 1m7.847s user 29m0.439s sys 3m51.181s 3rd run real 1m7.113s user 29m3.127s sys 3m51.516s Compared to 'make -j64' on tip v4.1-2756-ge3d06bd 2nd run real 1m7.605s user 28m3.121s sys 3m52.541s >> https://git.samba.org/jlayton/linux.git/?p=jlayton/lockperf.git;a=summary > > Sweet, I wasn't aware these existed. I'll go have a play. > >> posix01 >> mean variance sigma max min >> 4.1.0 121.9020 27882.5260 166.9806 603.5509 0.0063 >> percpu-rwsem 185.3981 38474.3836 196.1489 580.6532 0.0073 >> >> >> posix02 >> mean variance sigma max min >> 4.1.0 12.7461 3.1802 1.7833 15.5411 8.1018 >> percpu-rwsem 16.2341 4.3038 2.0746 19.3271 11.1751 >> > > These two seem to hurt, lemme go look at what they do. Now here the same tests with tip and tip+percpu-rwsem. The patches applied cleanly :) I put all the raw data here[1] in case someone is interested. Some of the test behave a bit strange, running extremely fast compared to the other runs. That is probably the result of me trying to reduce the run time to the min. flock01 mean variance sigma max min 4.1.0 11.7075 816.3341 28.5716 125.6552 0.0021 4.1.0+percpu-rwsem 11.4614 760.1345 27.5705 132.5030 0.0026 tip 6.8390 329.3037 18.1467 81.0373 0.0021 tip+percpu-rwsem 10.0870 546.7435 23.3825 106.2396 0.0026 flock02 mean variance sigma max min 4.1.0 7.0197 1.1812 1.0868 10.6188 5.1706 4.1.0+percpu-rwsem 9.3194 1.3443 1.1594 11.5902 6.6138 tip 7.1057 1.6719 1.2930 11.2362 5.1434 tip+percpu-rwsem 9.0357 1.9874 1.4097 14.0254 6.4380 lease01 mean variance sigma max min 4.1.0 41.8361 23.8462 4.8833 51.3493 28.5859 4.1.0+percpu-rwsem 40.2738 20.8323 4.5642 49.6037 28.0704 tip 30.2617 13.0900 3.6180 36.6398 20.2085 tip+percpu-rwsem 31.2730 17.9787 4.2401 37.8981 19.2944 lease02 mean variance sigma max min 4.1.0 71.2159 12.7763 3.5744 77.8432 58.0390 4.1.0+percpu-rwsem 71.4312 14.7688 3.8430 76.5036 57.8615 tip 20.2019 5.2042 2.2813 23.1071 13.4647 tip+percpu-rwsem 20.8305 6.6631 2.5813 23.8034 11.2815 posix01 mean variance sigma max min 4.1.0 121.9020 27882.5260 166.9806 603.5509 0.0063 4.1.0+percpu-rwsem 185.3981 38474.3836 196.1489 580.6532 0.0073 tip 129.2736 23752.7122 154.1191 474.0604 0.0063 tip+percpu-rwsem 142.6474 24732.1571 157.2646 468.7478 0.0072 posix02 mean variance sigma max min 4.1.0 12.7461 3.1802 1.7833 15.5411 8.1018 4.1.0+percpu-rwsem 16.2341 4.3038 2.0746 19.3271 11.1751 tip 13.2810 5.3958 2.3229 20.1243 8.9361 tip+percpu-rwsem 15.6802 4.7514 2.1798 21.5704 9.4074 posix03 mean variance sigma max min 4.1.0 0.9121 0.0000 0.0000 0.9121 0.9121 4.1.0+percpu-rwsem 0.9379 0.0000 0.0000 0.9379 0.9379 tip 0.8647 0.0009 0.0297 0.9274 0.7995 tip+percpu-rwsem 0.8147 0.0003 0.0161 0.8530 0.7824 posix04 mean variance sigma max min 4.1.0 0.0703 0.0044 0.0664 0.6764 0.0437 4.1.0+percpu-rwsem 0.0675 0.0007 0.0267 0.3236 0.0491 tip 0.0618 0.0027 0.0521 0.5642 0.0453 tip+percpu-rwsem 0.0658 0.0003 0.0175 0.1793 0.0493 cheers, daniel [1] http://monom.org/percpu-rwsem/ -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/