Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1759805AbbFBROw (ORCPT ); Tue, 2 Jun 2015 13:14:52 -0400 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:51226 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1759397AbbFBRNI (ORCPT ); Tue, 2 Jun 2015 13:13:08 -0400 Message-ID: <556DE3FB.9020400@fb.com> Date: Tue, 2 Jun 2015 13:12:27 -0400 From: Josef Bacik User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: Peter Zijlstra CC: , , , , , kernel-team Subject: Re: [PATCH RESEND] sched: prefer an idle cpu vs an idle sibling for BALANCE_WAKE References: <1432761736-22093-1-git-send-email-jbacik@fb.com> <20150528102127.GD3644@twins.programming.kicks-ass.net> <20150528110514.GR18673@twins.programming.kicks-ass.net> <5568D43D.20703@fb.com> <556CB4A8.1050509@fb.com> <1433191354.11346.22.camel@twins> In-Reply-To: <1433191354.11346.22.camel@twins> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [192.168.52.123] X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.14.151,1.0.33,0.0.0000 definitions=2015-06-02_14:2015-06-02,2015-06-02,1970-01-01 signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2530 Lines: 76 On 06/01/2015 04:42 PM, Peter Zijlstra wrote: > On Mon, 2015-06-01 at 15:38 -0400, Josef Bacik wrote: > >> Ok I got this patch to give me the same performance as all our other >> crap, just need to apply this incremental >> >> >> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c >> index b71eb2b..e11cfec 100644 >> --- a/kernel/sched/fair.c >> +++ b/kernel/sched/fair.c >> @@ -4761,13 +4761,10 @@ select_task_rq_fair(struct task_struct *p, int >> prev_cpu, int sd_flag, int wake_f >> >> if (tmp->flags & sd_flag) >> sd = tmp; >> - else if (!want_affine || (want_affine && affine_sd)) >> - break; >> } > > That bit worries me a bit, because that causes us to have a weird > definition for what sd is. > > Without WAKE_AFFINE, sd is the biggest domain with BALANCE_WAKE (or any > other sd_flag) set. > > But with WAKE_AFFINE, its the first domain that satisfies the wake > affine constraint of covering both the previous and waking cpu. It > basically reduces sd to affine_sd. > Ok I took Rik's idea and came out with this diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index b71eb2b..75073d3 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -4761,13 +4761,14 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int sd_flag, int wake_f if (tmp->flags & sd_flag) sd = tmp; - else if (!want_affine || (want_affine && affine_sd)) + else if (want_affine && affine_sd) { + sd = affine_sd; break; + } } if (affine_sd && cpu != prev_cpu && wake_affine(affine_sd, p, sync)) { prev_cpu = cpu; - sd = NULL; /* WAKE_AFFINE trumps BALANCE_WAKE */ } if (sd_flag & SD_BALANCE_WAKE) { But now that I re-read your response I think this is even more what you were worried about than less. Basically it comes down to if sd isn't set then we get shit performance. I realize that this magic to find an idle cpu when sd is set is pretty heavy handed, but it's obviously helpful in our case. So let me ask this question. When do we want to do the heavy handed search and when do we not? With WAKE_AFFINE what is our ultimate goal vs the other SD's? If we don't have an sd that matches our sd_flags what should we be doing, should we just go with whatever cpu we're on and carry on? Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/