Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Fri, 8 Nov 2019 21:49:40 +0100
From:   Peter Zijlstra <peterz@infradead.org>
To:     Valentin Schneider <valentin.schneider@arm.com>
Cc:     mingo@kernel.org, vincent.guittot@linaro.org,
        dietmar.eggemann@arm.com, juri.lelli@redhat.com,
        rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de,
        linux-kernel@vger.kernel.org, qperret@google.com,
        qais.yousef@arm.com, ktkhai@virtuozzo.com
Subject: Re: [PATCH 1/7] sched: Fix pick_next_task() vs change pattern race
Message-ID: <20191108204940.GL3079@worktop.programming.kicks-ass.net>
References: <20191108131553.027892369@infradead.org>
 <20191108131909.428842459@infradead.org>
 <e19c566b-dc14-a5aa-de4f-c67cdb17620c@arm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <e19c566b-dc14-a5aa-de4f-c67cdb17620c@arm.com>
User-Agent: Mutt/1.10.1 (2018-07-13)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Fri, Nov 08, 2019 at 04:05:25PM +0000, Valentin Schneider wrote:
> On 08/11/2019 13:15, Peter Zijlstra wrote:
> > +static int balance_rt(struct rq *rq, struct task_struct *p, struct rq_flags *rf)
> > +{
> > +	if (!on_rt_rq(&p->rt) && need_pull_rt_task(rq, p)) {
> > +		/*
> > +		 * This is OK, because current is on_cpu, which avoids it being
> > +		 * picked for load-balance and preemption/IRQs are still
> > +		 * disabled avoiding further scheduler activity on it and we've
> > +		 * not yet started the picking loop.
> > +		 */
> > +		rq_unpin_lock(rq, rf);
> > +		pull_rt_task(rq);
> > +		rq_repin_lock(rq, rf);
> > +	}
> > +
> > +	return sched_stop_runnable(rq) || sched_dl_runnable(rq) || sched_rt_runnable(rq);
> 
> So we already have some dependencies on the class ordering (e.g. fair->idle),
> but I'm wondering if would it make sense to have these runnable functions be
> defined as sched_class callbacks?
> 
> e.g.
> 
>   rt_sched_class.runnable = rt_runnable
> 
> w/ rt_runnable() just being a non-inlined sched_rt_runnable() you define
> further down the patch (or a wrapper to it). The balance return pattern could
> then become:
> 
>   for_class_range(class, sched_class_highest, rt_sched_class->next)
>   	if (class->runnable(rq))
> 		return true;
>   return false;
> 
> (and replace rt_sched_class by whatever class' balance callback this is)
> 
> It's a bit neater, but I'm pretty sure it's going to run worse :/
> The only unaffected one would be fair, since newidle_balance() already does
> that "for free".

Yeah, it'll be pretty terrible :/

That said, I might have some clues on how to get rid of all the indirect
calls, but I need to play around a bit. It'll be invasive though :/
(like that ever stopped me).