Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752836Ab0LUT3G (ORCPT ); Tue, 21 Dec 2010 14:29:06 -0500 Received: from e4.ny.us.ibm.com ([32.97.182.144]:45131 "EHLO e4.ny.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751240Ab0LUT3B (ORCPT ); Tue, 21 Dec 2010 14:29:01 -0500 Date: Tue, 21 Dec 2010 11:28:49 -0800 From: "Paul E. McKenney" To: Frederic Weisbecker Cc: LKML , Thomas Gleixner , Peter Zijlstra , Ingo Molnar , Steven Rostedt , Lai Jiangshan , Andrew Morton , Anton Blanchard , Tim Pepper Subject: Re: [RFC PATCH 10/15] nohz_task: Enter in extended quiescent state when in userspace Message-ID: <20101221192849.GP2143@linux.vnet.ibm.com> Reply-To: paulmck@linux.vnet.ibm.com References: <1292858662-5650-1-git-send-email-fweisbec@gmail.com> <1292858662-5650-11-git-send-email-fweisbec@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1292858662-5650-11-git-send-email-fweisbec@gmail.com> User-Agent: Mutt/1.5.20 (2009-06-14) X-Content-Scanned: Fidelis XPS MAILER Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 6811 Lines: 214 On Mon, Dec 20, 2010 at 04:24:17PM +0100, Frederic Weisbecker wrote: > A nohz task can safely enter into extended quiescent state when > it goes into userspace, this avoids a remote cpu to force the > nohz task to be interrupted in order to notify quiescent states. > > We enter into an extended quiescent state when: > > - A nohz task resumes to userspace and is alone running on the > CPU (we check if the local cpu is in nohz mode, which means > no other task compete on that CPU). If the tick is still running > then entering into extended QS will be done later from the second > case: > > - When the tick stops and verify the current task is a nohz one, > is alone running on the CPU and runs in userspace. > > We exit the extended quiescent state when: > > - A nohz task enters the kernel and is alone running on the CPU. > Again we check if the local cpu is in nohz mode for that. If > the tick is still running then it means we are not in an extended > QS and we don't do anything. > > - The tick restarts because a new task is enqueued. > > Whether the nohz task is in userspace or not is tracked by the > per cpu nohz_task_ext_qs variable. > > Architectures need to provide some backend to notify userspace > exit/entry in order to support this mode. > It needs to implement the TIF_NOHZ flag that switches to slow > path syscall mode and to notify exceptions entry/exit. > > We don't need to handle irqs or nmis as those are already handled > by RCU through rcu_enter_irq/nmi helpers. One question below... > Signed-off-by: Frederic Weisbecker > Cc: Thomas Gleixner > Cc: Peter Zijlstra > Cc: Paul E. McKenney > Cc: Ingo Molnar > Cc: Steven Rostedt > Cc: Lai Jiangshan > Cc: Andrew Morton > Cc: Anton Blanchard > Cc: Tim Pepper > --- > arch/Kconfig | 4 +++ > include/linux/tick.h | 16 ++++++++++- > kernel/sched.c | 3 ++ > kernel/time/tick-sched.c | 61 +++++++++++++++++++++++++++++++++++++++++++++- > 4 files changed, 81 insertions(+), 3 deletions(-) > > diff --git a/arch/Kconfig b/arch/Kconfig > index e631791..d1ebea3 100644 > --- a/arch/Kconfig > +++ b/arch/Kconfig > @@ -177,5 +177,9 @@ config HAVE_ARCH_JUMP_LABEL > > config HAVE_NO_HZ_TASK > bool > + help > + Features necessary hooks for a task wanting to enter nohz > + while running alone on a CPU: thread flag for syscall hooks > + and exceptions entry/exit hooks. > > source "kernel/gcov/Kconfig" > diff --git a/include/linux/tick.h b/include/linux/tick.h > index 7465a47..a704bb7 100644 > --- a/include/linux/tick.h > +++ b/include/linux/tick.h > @@ -8,6 +8,7 @@ > > #include > #include > +#include > > #ifdef CONFIG_GENERIC_CLOCKEVENTS > > @@ -130,10 +131,21 @@ extern u64 get_cpu_iowait_time_us(int cpu, u64 *last_update_time); > > #ifdef CONFIG_NO_HZ_TASK > DECLARE_PER_CPU(int, task_nohz_mode); > +DECLARE_PER_CPU(int, nohz_task_ext_qs); > + > +extern void tick_nohz_task_enter_kernel(void); > +extern void tick_nohz_task_exit_kernel(void); > +extern void tick_nohz_task_enter_exception(struct pt_regs *regs); > +extern void tick_nohz_task_exit_exception(struct pt_regs *regs); > extern int tick_nohz_task_mode(void); > -#else > + > +#else /* !NO_HZ_TASK */ > +static inline void tick_nohz_task_enter_kernel(void) { } > +static inline void tick_nohz_task_exit_kernel(void) { } > +static inline void tick_nohz_task_enter_exception(struct pt_regs *regs) { } > +static inline void tick_nohz_task_exit_exception(struct pt_regs *regs) { } > static inline int tick_nohz_task_mode(void) { return 0; } > -#endif > +#endif /* !NO_HZ_TASK */ > > # else /* !NO_HZ */ > static inline void tick_nohz_stop_sched_tick(int inidle) { } > diff --git a/kernel/sched.c b/kernel/sched.c > index b99f192..4412493 100644 > --- a/kernel/sched.c > +++ b/kernel/sched.c > @@ -2464,6 +2464,9 @@ static void nohz_task_cpu_update(void *unused) > if (rq->nr_running > 1 || rcu_pending(cpu) || rcu_needs_cpu(cpu)) { If the task enters a system call in nohz mode, and then that system call enqueues an RCU callback, this code path will pull that CPU out of nohz mode, right? Thanx, Paul > __get_cpu_var(task_nohz_mode) = 0; > tick_nohz_restart_sched_tick(); > + > + if (__get_cpu_var(nohz_task_ext_qs)) > + rcu_exit_nohz(); > } > } > > diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c > index 88011b9..9a4aa39 100644 > --- a/kernel/time/tick-sched.c > +++ b/kernel/time/tick-sched.c > @@ -720,6 +720,62 @@ void tick_check_idle(int cpu) > } > > #ifdef CONFIG_NO_HZ_TASK > +DEFINE_PER_CPU(int, nohz_task_ext_qs); > + > +void tick_nohz_task_exit_kernel(void) > +{ > + unsigned long flags; > + > + if (!test_thread_flag(TIF_NOHZ)) > + return; > + > + local_irq_save(flags); > + > + __get_cpu_var(nohz_task_ext_qs) = 1; > + /* > + * Only enter extended QS if the tick is not running. > + * Otherwise the tick will handle that later when it > + * will decide to stop. > + */ > + if (__get_cpu_var(task_nohz_mode)) > + rcu_enter_nohz(); > + > + local_irq_restore(flags); > +} > + > +void tick_nohz_task_enter_kernel(void) > +{ > + unsigned long flags; > + > + if (!test_thread_flag(TIF_NOHZ)) > + return; > + > + local_irq_save(flags); > + > + __get_cpu_var(nohz_task_ext_qs) = 0; > + /* > + * If the tick was running, then we weren't in > + * rcu extended period. Only exit extended QS > + * if we were in such state. > + */ > + if (__get_cpu_var(task_nohz_mode)) > + rcu_exit_nohz(); > + > + local_irq_restore(flags); > +} > + > +void tick_nohz_task_enter_exception(struct pt_regs *regs) > +{ > + if (user_mode(regs)) > + tick_nohz_task_enter_kernel(); > +} > + > +void tick_nohz_task_exit_exception(struct pt_regs *regs) > +{ > + if (user_mode(regs)) > + tick_nohz_task_exit_kernel(); > +} > + > int tick_nohz_task_mode(void) > { > return __get_cpu_var(task_nohz_mode); > @@ -730,8 +786,11 @@ static void tick_nohz_task_stop_tick(void) > if (!test_thread_flag(TIF_NOHZ) || __get_cpu_var(task_nohz_mode)) > return; > > - if (nohz_task_can_stop_tick()) > + if (nohz_task_can_stop_tick()) { > __get_cpu_var(task_nohz_mode) = 1; > + if (__get_cpu_var(nohz_task_ext_qs)) > + rcu_enter_nohz(); > + } > } > #else > static void tick_nohz_task_stop_tick(void) { } > -- > 1.7.3.2 > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/