Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757895Ab1EBKza (ORCPT ); Mon, 2 May 2011 06:55:30 -0400 Received: from fgwmail6.fujitsu.co.jp ([192.51.44.36]:39121 "EHLO fgwmail6.fujitsu.co.jp" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755389Ab1EBKz0 (ORCPT ); Mon, 2 May 2011 06:55:26 -0400 X-SecurityPolicyCheck-FJ: OK by FujitsuOutboundMailChecker v1.3.1 From: KOSAKI Motohiro To: KOSAKI Motohiro Subject: [PATCH 1/2] cpuset: fix cpuset_cpus_allowed_fallback() don't update tsk->rt.nr_cpus_allowed Cc: kosaki.motohiro@jp.fujitsu.com, Oleg Nesterov , LKML , Andrew Morton , Peter Zijlstra , Ingo Molnar , Li Zefan , Miao Xie In-Reply-To: <20110502194416.2D61.A69D9226@jp.fujitsu.com> References: <20110428161149.GA15658@redhat.com> <20110502194416.2D61.A69D9226@jp.fujitsu.com> Message-Id: <20110502195657.2D68.A69D9226@jp.fujitsu.com> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Mailer: Becky! ver. 2.56.05 [ja] Date: Mon, 2 May 2011 19:55:23 +0900 (JST) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2714 Lines: 80 The rule is, we have to update tsk->rt.nr_cpus_allowed too if we change tsk->cpus_allowed. Otherwise RT scheduler may confuse. This patch fixes it. btw, system_state checking is very important. current boot sequence is (1) smp_init (ie secondary cpus up and created cpu bound kthreads). (2) sched_init_smp(). Then following bad scenario can be happen, (1) cpuup call notifier(CPU_UP_PREPARE) (2) A cpu notifier consumer create FIFO kthread (3) It call kthread_bind() ... but, now secondary cpu haven't ONLINE (3) schedule() makes fallback and cpuset_cpus_allowed_fallback change task->cpus_allowed (4) find_lowest_rq() touch local_cpu_mask if task->rt.nr_cpus_allowed != 1, but it haven't been initialized. RCU folks plan to introduce such FIFO kthread and our testing hitted the above issue. Then this patch also protect it. Signed-off-by: KOSAKI Motohiro Cc: Oleg Nesterov Cc: Peter Zijlstra Cc: Ingo Molnar --- include/linux/cpuset.h | 1 + kernel/cpuset.c | 1 + kernel/sched.c | 4 ++++ 3 files changed, 6 insertions(+), 0 deletions(-) diff --git a/include/linux/cpuset.h b/include/linux/cpuset.h index f20eb8f..42dcbdc 100644 --- a/include/linux/cpuset.h +++ b/include/linux/cpuset.h @@ -147,6 +147,7 @@ static inline void cpuset_cpus_allowed(struct task_struct *p, static inline int cpuset_cpus_allowed_fallback(struct task_struct *p) { cpumask_copy(&p->cpus_allowed, cpu_possible_mask); + p->rt.nr_cpus_allowed = cpumask_weight(&p->cpus_allowed); return cpumask_any(cpu_active_mask); } diff --git a/kernel/cpuset.c b/kernel/cpuset.c index 1ceeb04..6e5bbe8 100644 --- a/kernel/cpuset.c +++ b/kernel/cpuset.c @@ -2220,6 +2220,7 @@ int cpuset_cpus_allowed_fallback(struct task_struct *tsk) cpumask_copy(&tsk->cpus_allowed, cpu_possible_mask); cpu = cpumask_any(cpu_active_mask); } + tsk->rt.nr_cpus_allowed = cpumask_weight(&tsk->cpus_allowed); return cpu; } diff --git a/kernel/sched.c b/kernel/sched.c index fd4625f..bfcd219 100644 --- a/kernel/sched.c +++ b/kernel/sched.c @@ -2352,6 +2352,10 @@ static int select_fallback_rq(int cpu, struct task_struct *p) if (dest_cpu < nr_cpu_ids) return dest_cpu; + /* Don't worry. It's temporary mismatch. */ + if (system_state < SYSTEM_RUNNING) + return cpu; + /* No more Mr. Nice Guy. */ dest_cpu = cpuset_cpus_allowed_fallback(p); /* -- 1.7.3.1 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/