Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Fri, 8 Nov 2019 11:02:12 +0000
From:   Quentin Perret <qperret@google.com>
To:     Peter Zijlstra <peterz@infradead.org>
Cc:     Kirill Tkhai <ktkhai@virtuozzo.com>, linux-kernel@vger.kernel.org,
        aaron.lwe@gmail.com, valentin.schneider@arm.com, mingo@kernel.org,
        pauld@redhat.com, jdesfossez@digitalocean.com,
        naravamudan@digitalocean.com, vincent.guittot@linaro.org,
        dietmar.eggemann@arm.com, juri.lelli@redhat.com,
        rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de,
        kernel-team@android.com, john.stultz@linaro.org
Subject: Re: NULL pointer dereference in pick_next_task_fair
Message-ID: <20191108110212.GA204618@google.com>
References: <20191028174603.GA246917@google.com>
 <20191106120525.GX4131@hirez.programming.kicks-ass.net>
 <33643a5b-1b83-8605-2347-acd1aea04f93@virtuozzo.com>
 <20191106165437.GX4114@hirez.programming.kicks-ass.net>
 <20191106172737.GM5671@hirez.programming.kicks-ass.net>
 <831c2cd4-40a4-31b2-c0aa-b5f579e770d6@virtuozzo.com>
 <20191107132628.GZ4114@hirez.programming.kicks-ass.net>
 <20191107153848.GA31774@google.com>
 <20191107184356.GF4114@hirez.programming.kicks-ass.net>
 <20191107192907.GA30258@worktop.programming.kicks-ass.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20191107192907.GA30258@worktop.programming.kicks-ass.net>
User-Agent: Mutt/1.10.1 (2018-07-13)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Thursday 07 Nov 2019 at 20:29:07 (+0100), Peter Zijlstra wrote:
> I still havne't had food, but this here compiles...

And it seems to work, too :)

> @@ -3929,13 +3929,17 @@ pick_next_task(struct rq *rq, struct task_struct *prev, struct rq_flags *rf)
>  	}
> 
>  restart:
> -	/*
> -	 * Ensure that we put DL/RT tasks before the pick loop, such that they
> -	 * can PULL higher prio tasks when we lower the RQ 'priority'.
> -	 */
> -	prev->sched_class->put_prev_task(rq, prev, rf);
> -	if (!rq->nr_running)
> -		newidle_balance(rq, rf);
> +#ifdef CONFIG_SMP
> +	for (class = prev->sched_class;
> +	     class != &idle_sched_class;
> +	     class = class->next) {
> +
> +		if (class->balance(rq, prev, rf))
> +			break;
> +	}
> +#endif
> +
> +	put_prev_task(rq, prev);

Right, that looks much cleaner IMO. I'm thinking if we killed the
special case for CFS above we could do with a single loop to iterate the
classes, and you could fold ->balance() in ->pick_next_task() ...
That would remove one call site to newidle_balance() too, which I think
is good. Hackbench probably won't like that, though.

>  	for_each_class(class) {
>  		p = class->pick_next_task(rq, NULL, NULL);
> @@ -6201,7 +6205,7 @@ static struct task_struct *__pick_migrate_task(struct rq *rq)
>  	for_each_class(class) {
>  		next = class->pick_next_task(rq, NULL, NULL);
>  		if (next) {
> -			next->sched_class->put_prev_task(rq, next, NULL);
> +			next->sched_class->put_prev_task(rq, next);
>  			return next;
>  		}
>  	}
> diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
> index 2dc48720f189..b6c3fb10cf57 100644
> --- a/kernel/sched/deadline.c
> +++ b/kernel/sched/deadline.c
> @@ -1778,15 +1778,9 @@ pick_next_task_dl(struct rq *rq, struct task_struct *prev, struct rq_flags *rf)
>  	return p;
>  }
> 
> -static void put_prev_task_dl(struct rq *rq, struct task_struct *p, struct rq_flags *rf)
> +static int balance_dl(struct rq *rq, struct task_struct *p, struct rq_flags *rf)
>  {
> -	update_curr_dl(rq);
> -
> -	update_dl_rq_load_avg(rq_clock_pelt(rq), rq, 1);
> -	if (on_dl_rq(&p->dl) && p->nr_cpus_allowed > 1)
> -		enqueue_pushable_dl_task(rq, p);
> -

Ah, and this can actually be done after the pull because the two are in
fact mutually exclusive. And same thing for RT. Good :)

> -	if (rf && !on_dl_rq(&p->dl) && need_pull_dl_task(rq, p)) {
> +	if (!on_dl_rq(&p->dl) && need_pull_dl_task(rq, p)) {
>  		/*
>  		 * This is OK, because current is on_cpu, which avoids it being
>  		 * picked for load-balance and preemption/IRQs are still
> @@ -1797,6 +1791,18 @@ static void put_prev_task_dl(struct rq *rq, struct task_struct *p, struct rq_fla
>  		pull_dl_task(rq);
>  		rq_repin_lock(rq, rf);
>  	}
> +
> +	return rq->dl.dl_nr_running > 0;
> +}
> +
> +
> +static void put_prev_task_dl(struct rq *rq, struct task_struct *p)
> +{
> +	update_curr_dl(rq);
> +
> +	update_dl_rq_load_avg(rq_clock_pelt(rq), rq, 1);
> +	if (on_dl_rq(&p->dl) && p->nr_cpus_allowed > 1)
> +		enqueue_pushable_dl_task(rq, p);
>  }
> 
>  /*
> @@ -2438,6 +2444,7 @@ const struct sched_class dl_sched_class = {
>  	.check_preempt_curr	= check_preempt_curr_dl,
> 
>  	.pick_next_task		= pick_next_task_dl,
> +	.balance		= balance_dl,
>  	.put_prev_task		= put_prev_task_dl,
>  	.set_next_task		= set_next_task_dl,
> 
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index a14487462b6c..6b983214e00f 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -6746,10 +6746,18 @@ done: __maybe_unused;
>  	return NULL;
>  }
> 
> +static int balance_fair(struct rq *rq, struct task_struct *prev, struct rq_flags *rf)
> +{
> +	if (rq->cfs.nr_running)
> +		return 1;
> +
> +	return newidle_balance(rq, rf) != 0;

And you can ignore the RETRY_TASK case here under the assumption that
we must have tried to pull from RT/DL before ending up here ?

Thanks,
Quentin