Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758369AbbGGRHL (ORCPT ); Tue, 7 Jul 2015 13:07:11 -0400 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:34323 "EHLO mx0b-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757747AbbGGRGq (ORCPT ); Tue, 7 Jul 2015 13:06:46 -0400 Message-ID: <559C0700.6090009@fb.com> Date: Tue, 7 Jul 2015 13:06:08 -0400 From: Josef Bacik User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: Mike Galbraith CC: Peter Zijlstra , , , , , kernel-team Subject: Re: [patch] Re: [PATCH RESEND] sched: prefer an idle cpu vs an idle sibling for BALANCE_WAKE References: <1432761736-22093-1-git-send-email-jbacik@fb.com> <20150528102127.GD3644@twins.programming.kicks-ass.net> <20150528110514.GR18673@twins.programming.kicks-ass.net> <1434087305.3674.26.camel@gmail.com> <5581B70D.2000800@fb.com> <1434588939.3444.25.camel@gmail.com> <55823F33.7040005@fb.com> <1434600765.3393.9.camel@gmail.com> <55957871.7080906@fb.com> <1435905658.6418.52.camel@gmail.com> <1436025462.17152.37.camel@gmail.com> <1436080661.22930.22.camel@gmail.com> <1436159590.5850.27.camel@gmail.com> <559A91F4.7000903@fb.com> <1436207790.2940.30.camel@gmail.com> <559AD9CE.4090309@fb.com> <1436241678.1836.29.camel@gmail.com> <1436262224.1836.74.camel@gmail.com> In-Reply-To: <1436262224.1836.74.camel@gmail.com> Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [192.168.52.123] X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.14.151,1.0.33,0.0.0000 definitions=2015-07-07_06:2015-07-07,2015-07-07,1970-01-01 signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3711 Lines: 91 On 07/07/2015 05:43 AM, Mike Galbraith wrote: > On Tue, 2015-07-07 at 06:01 +0200, Mike Galbraith wrote: >> On Mon, 2015-07-06 at 15:41 -0400, Josef Bacik wrote: >> >>> So the NO_WAKE_WIDE_IDLE results are very good, almost the same as the >>> baseline with a slight regression at lower RPS and a slight improvement >>> at high RPS. >> >> Good. I can likely drop the rest then (I like dinky, so do CPUs;). I'm >> not real keen on the feature unless your numbers are really good, and >> odds are that ain't gonna happen. > > More extensive testing in pedantic-man mode increased my confidence of > that enough to sign off and ship the dirt simple version. Any further > twiddles should grow their own wings if they want to fly anyway, the > simplest form helps your real world load, as well as the not so real > pgbench, my numbers for that below. > > virgin master, 2 socket box > postgres@nessler:~> pgbench.sh > clients 12 tps = 96233.854271 1.000 > clients 24 tps = 142234.686166 1.000 > clients 36 tps = 148433.534531 1.000 > clients 48 tps = 133105.634302 1.000 > clients 60 tps = 128903.080371 1.000 > clients 72 tps = 128591.821782 1.000 > clients 84 tps = 114445.967116 1.000 > clients 96 tps = 109803.557524 1.000 avg 125219.017 1.000 > > V3 (KISS, below) > postgres@nessler:~> pgbench.sh > clients 12 tps = 120793.023637 1.255 > clients 24 tps = 144668.961468 1.017 > clients 36 tps = 156705.239251 1.055 > clients 48 tps = 152004.886893 1.141 > clients 60 tps = 138582.113864 1.075 > clients 72 tps = 136286.891104 1.059 > clients 84 tps = 137420.986043 1.200 > clients 96 tps = 135199.060242 1.231 avg 140207.645 1.119 1.000 > > V2 NO_WAKE_WIDE_IDLE > postgres@nessler:~> pgbench.sh > clients 12 tps = 121821.966162 1.265 > clients 24 tps = 146446.388366 1.029 > clients 36 tps = 151373.362190 1.019 > clients 48 tps = 156806.730746 1.178 > clients 60 tps = 133933.491567 1.039 > clients 72 tps = 131460.489424 1.022 > clients 84 tps = 130859.340261 1.143 > clients 96 tps = 130787.476584 1.191 avg 137936.155 1.101 0.983 > > V2 WAKE_WIDE_IDLE (crawl in a hole feature, you're dead) > postgres@nessler:~> pgbench.sh > clients 12 tps = 121297.791570 > clients 24 tps = 145939.488312 > clients 36 tps = 155336.090263 > clients 48 tps = 149018.245323 > clients 60 tps = 136730.079391 > clients 72 tps = 134886.116831 > clients 84 tps = 130493.283398 > clients 96 tps = 126043.336074 > > > sched: beef up wake_wide() > > Josef Bacik reported that Facebook sees better performance with their > 1:N load (1 dispatch/node, N workers/node) when carrying an old patch > to try very hard to wake to an idle CPU. While looking at wake_wide(), > I noticed that it doesn't pay attention to wakeup of the 1:N waker, > returning 1 only when the 1:N waker is waking one of its minions. > > Correct that, and don't bother doing domain traversal when we know > that all we need to do is check for an idle cpu. > > Signed-off-by: Mike Galbraith Ok you can add Tested-by: Josef Bacik The new patch is the best across the board, there are no regressions and about a 5% improvement for the bulk of the run (25 percentile and up). Thanks for your help on this! Josef -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/