Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756795AbcCBCFn (ORCPT ); Tue, 1 Mar 2016 21:05:43 -0500 Received: from mail177-1.suw61.mandrillapp.com ([198.2.177.1]:18360 "EHLO mail177-1.suw61.mandrillapp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754413AbcCAXvX (ORCPT ); Tue, 1 Mar 2016 18:51:23 -0500 From: Greg Kroah-Hartman Subject: [PATCH 3.14 044/130] sched: Allow balance callbacks for check_class_changed() X-Mailer: git-send-email 2.7.2 To: Cc: Greg Kroah-Hartman , , Mike Galbraith , "Peter Zijlstra (Intel)" , , , , , , , Thomas Gleixner , Byungchul Park Message-Id: <20160301234501.304823725@linuxfoundation.org> In-Reply-To: <20160301234459.768886030@linuxfoundation.org> References: <20160301234459.768886030@linuxfoundation.org> X-Report-Abuse: Please forward a copy of this message, including all headers, to abuse@mandrill.com X-Report-Abuse: You can also report abuse here: http://mandrillapp.com/contact/abuse?id=30481620.3ee8ff3839704fdfb0b233a951d93e2f X-Mandrill-User: md_30481620 Date: Tue, 01 Mar 2016 23:51:04 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3226 Lines: 107 3.14-stable review patch. If anyone has any objections, please let me know. ------------------ From: Peter Zijlstra commit 4c9a4bc89a9cca8128bce67d6bc8870d6b7ee0b2 upstream. In order to remove dropping rq->lock from the switched_{to,from}()/prio_changed() sched_class methods, run the balance callbacks after it. We need to remove dropping rq->lock because its buggy, suppose using sched_setattr()/sched_setscheduler() to change a running task from FIFO to OTHER. By the time we get to switched_from_rt() the task is already enqueued on the cfs runqueues. If switched_from_rt() does pull_rt_task() and drops rq->lock, load-balancing can come in and move our task @p to another rq. The subsequent switched_to_fair() still assumes @p is on @rq and bad things will happen. By using balance callbacks we delay the load-balancing operations {rt,dl}x{push,pull} until we've done all the important work and the task is fully set up. Furthermore, the balance callbacks do not know about @p, therefore they cannot get confused like this. Reported-by: Mike Galbraith Signed-off-by: Peter Zijlstra (Intel) Cc: ktkhai@parallels.com Cc: rostedt@goodmis.org Cc: juri.lelli@gmail.com Cc: pang.xunlei@linaro.org Cc: oleg@redhat.com Cc: wanpeng.li@linux.intel.com Link: http://lkml.kernel.org/r/20150611124742.615343911@infradead.org Signed-off-by: Thomas Gleixner Signed-off-by: Byungchul Park Signed-off-by: Greg Kroah-Hartman --- kernel/sched/core.c | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -937,6 +937,13 @@ inline int task_curr(const struct task_s return cpu_curr(task_cpu(p)) == p; } +/* + * switched_from, switched_to and prio_changed must _NOT_ drop rq->lock, + * use the balance_callback list if you want balancing. + * + * this means any call to check_class_changed() must be followed by a call to + * balance_callback(). + */ static inline void check_class_changed(struct rq *rq, struct task_struct *p, const struct sched_class *prev_class, int oldprio) @@ -1423,8 +1430,12 @@ ttwu_do_wakeup(struct rq *rq, struct tas p->state = TASK_RUNNING; #ifdef CONFIG_SMP - if (p->sched_class->task_woken) + if (p->sched_class->task_woken) { + /* + * XXX can drop rq->lock; most likely ok. + */ p->sched_class->task_woken(rq, p); + } if (rq->idle_stamp) { u64 delta = rq_clock(rq) - rq->idle_stamp; @@ -3006,7 +3017,11 @@ void rt_mutex_setprio(struct task_struct check_class_changed(rq, p, prev_class, oldprio); out_unlock: + preempt_disable(); /* avoid rq from going away on us */ __task_rq_unlock(rq); + + balance_callback(rq); + preempt_enable(); } #endif @@ -3512,10 +3527,17 @@ change: enqueue_task(rq, p, 0); check_class_changed(rq, p, prev_class, oldprio); + preempt_disable(); /* avoid rq from going away on us */ task_rq_unlock(rq, p, &flags); rt_mutex_adjust_pi(p); + /* + * Run balance callbacks after we've adjusted the PI chain. + */ + balance_callback(rq); + preempt_enable(); + return 0; }