Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 85AE2C43383
Date:   Thu, 27 Feb 2020 09:06:08 +0530
From:   Pavan Kondeti <pkondeti@codeaurora.org>
To:     Qais Yousef <qais.yousef@arm.com>
Cc:     Ingo Molnar <mingo@kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Steven Rostedt <rostedt@goodmis.org>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Juri Lelli <juri.lelli@redhat.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
        LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v2 5/6] sched/rt: Better manage pushing unfit tasks on
 wakeup
Message-ID: <20200227033608.GN28029@codeaurora.org>
References: <20200223184001.14248-1-qais.yousef@arm.com>
 <20200223184001.14248-6-qais.yousef@arm.com>
 <20200224061004.GH28029@codeaurora.org>
 <20200224121139.cbz2dt5heiouknif@e107158-lin.cambridge.arm.com>
 <CAEU1=PncyV=-vqjkDHSJ4hUhhTfYUgVN-HAe4zXMHtFx1oc5XA@mail.gmail.com>
 <20200224174138.n6pmoeffqg7eqiy2@e107158-lin.cambridge.arm.com>
 <20200225035505.GI28029@codeaurora.org>
 <20200226160247.iqvdakiqbakk2llz@e107158-lin.cambridge.arm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20200226160247.iqvdakiqbakk2llz@e107158-lin.cambridge.arm.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, Feb 26, 2020 at 04:02:48PM +0000, Qais Yousef wrote:
> On 02/25/20 09:25, Pavan Kondeti wrote:
> > > I haven't been staring at this code for as long as you, but since we have
> > > logic at wakeup to do a push, I think we need something here anyway for unfit
> > > tasks.
> > > 
> > > Fixing select_task_rq_rt() to better balance tasks will help a lot in general,
> > > but if that was enough already then why do we need to consider a push at the
> > > wakeup at all then?
> > > 
> > > AFAIU, in SMP the whole push-pull mechanism is racy and we introduce redundancy
> > > at taking the decision on various points to ensure we minimize this racy nature
> > > of SMP systems. Anything could have happened between the time we called
> > > select_task_rq_rt() and the wakeup, so we double check again before we finally
> > > go and run. That's how I interpret it.
> > > 
> > > I am open to hear about other alternatives first anyway. Your help has been
> > > much appreciated so far.
> > > 
> > 
> > The search inside find_lowest_rq() happens without any locks so I believe it
> > is expected to have races like this. In fact there is a comment in the code
> > saying "This test is optimistic, if we get it wrong the load-balancer
> > will have to sort it out" in select_task_rq_rt(). However, the push logic
> > as of today works only for overloaded case. In that sense, your patch fixes
> > this race for b.L systems. At the same time, I feel like tracking nonfit tasks
> > just to fix this race seems to be too much. I will leave this to Steve and
> > others to take a decision.
> 
> I do think without this tasks can end up on the wrong CPU longer than they
> should. Keep in mind that if a task is boosted to run on a big core, it still
> have to compete with non-boosted tasks who can run on a any cpu. So this
> opportunistic push might be necessary.
> 
> For 5.6 though, I'll send an updated series that removes the fitness check from
> task_woken_rt() && switched_to_rt() and carry on with this discussion for 5.7.
> 
> > 
> > I thought of suggesting to remove the below check from select_task_rq_rt()
> > 
> > p->prio < cpu_rq(target)->rt.highest_prio.curr
> > 
> > which would then make the target CPU overloaded and the push logic would
> > spread the tasks. That works for a b.L system too. However there seems to
> > be a very good reason for doing this. see
> > https://lore.kernel.org/patchwork/patch/539137/
> > 
> > The fact that a CPU is part of lowest_mask but running a higher prio RT
> > task means there is a race. Should we retry one more time to see if we find
> > another CPU?
> 
> Isn't this what I did in v1?
> 
> https://lore.kernel.org/lkml/20200214163949.27850-4-qais.yousef@arm.com/
> 

Yes, that patch allows overloading the CPU When the priorities are same.
I think, We should also consider when a low prio task and high prio task
are waking at the same time and high prio task winning the race.

Thanks,
Pavan

-- 
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project.