Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932203AbaGSAov (ORCPT ); Fri, 18 Jul 2014 20:44:51 -0400 Received: from mail-wg0-f50.google.com ([74.125.82.50]:65389 "EHLO mail-wg0-f50.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758111AbaGSAok (ORCPT ); Fri, 18 Jul 2014 20:44:40 -0400 From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Ingo Molnar , "Paul E. McKenney" , Peter Zijlstra , Steven Rostedt , Thomas Gleixner , Viresh Kumar Subject: [PATCH 08/10] nohz: Fetch timekeeping max deferment only for timekeeper Date: Sat, 19 Jul 2014 02:44:19 +0200 Message-Id: <1405730661-9355-9-git-send-email-fweisbec@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1405730661-9355-1-git-send-email-fweisbec@gmail.com> References: <1405730661-9355-1-git-send-email-fweisbec@gmail.com> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org We fetch it unconditionally from the tick stop code whereas only the timekeeper, or the CPU that carried that duty last, needs it. Fetching the timekeeping max deferment should be lightweight but it still involves a few read side barriers and a seqcount that may well be cache cold for non-timekeepers. So lets spare it when possible by inverting the way we handle timekeeper deferment state machine. Cc: Ingo Molnar Cc: Paul E. McKenney Cc: Peter Zijlstra Cc: Steven Rostedt Cc: Thomas Gleixner Cc: Viresh Kumar Signed-off-by: Frederic Weisbecker --- kernel/time/tick-sched.c | 57 ++++++++++++++++++++++++++---------------------- 1 file changed, 31 insertions(+), 26 deletions(-) diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 2ea2143..bcba79d 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -529,6 +529,36 @@ u64 get_cpu_iowait_time_us(int cpu, u64 *last_update_time) } EXPORT_SYMBOL_GPL(get_cpu_iowait_time_us); +/* + * If this cpu is the one which updates jiffies, then + * give up the assignment and let it be taken by the + * cpu which runs the tick timer next, which might be + * this cpu as well. If we don't drop this here the + * jiffies might be stale and do_timer() never + * invoked. Keep track of the fact that it was the one + * which had the do_timer() duty last. If this cpu is + * the one which had the do_timer() duty last, we + * limit the sleep time to the timekeeping max deferement + * value. Otherwise we can sleep as long as we want. + */ +static u64 timekeeping_deferment(struct tick_sched *ts, int cpu) +{ + u64 time_delta = KTIME_MAX; + + if (tick_do_timer_cpu == cpu) { + time_delta = timekeeping_max_deferment(); + tick_do_timer_cpu = TICK_DO_TIMER_NONE; + ts->do_timer_last = 1; + } else if (ts->do_timer_last) { + if (tick_do_timer_cpu == TICK_DO_TIMER_NONE) + time_delta = timekeeping_max_deferment(); + else + ts->do_timer_last = 0; + } + + return time_delta; +} + static ktime_t tick_nohz_stop_sched_tick(struct tick_sched *ts, ktime_t now, int cpu) { @@ -536,9 +566,6 @@ static ktime_t tick_nohz_stop_sched_tick(struct tick_sched *ts, ktime_t last_update, expires, ret = { .tv64 = 0 }; unsigned long rcu_delta_jiffies; struct clock_event_device *dev = __get_cpu_var(tick_cpu_device).evtdev; - u64 time_delta; - - time_delta = timekeeping_max_deferment(); /* Read jiffies and the time when jiffies were updated last */ do { @@ -570,29 +597,7 @@ static ktime_t tick_nohz_stop_sched_tick(struct tick_sched *ts, /* Schedule the tick, if we are at least one jiffie off */ if ((long)delta_jiffies >= 1) { - - /* - * If this cpu is the one which updates jiffies, then - * give up the assignment and let it be taken by the - * cpu which runs the tick timer next, which might be - * this cpu as well. If we don't drop this here the - * jiffies might be stale and do_timer() never - * invoked. Keep track of the fact that it was the one - * which had the do_timer() duty last. If this cpu is - * the one which had the do_timer() duty last, we - * limit the sleep time to the timekeeping - * max_deferement value which we retrieved - * above. Otherwise we can sleep as long as we want. - */ - if (cpu == tick_do_timer_cpu) { - tick_do_timer_cpu = TICK_DO_TIMER_NONE; - ts->do_timer_last = 1; - } else if (tick_do_timer_cpu != TICK_DO_TIMER_NONE) { - time_delta = KTIME_MAX; - ts->do_timer_last = 0; - } else if (!ts->do_timer_last) { - time_delta = KTIME_MAX; - } + u64 time_delta = timekeeping_deferment(ts, cpu); #ifdef CONFIG_NO_HZ_FULL if (!ts->inidle) { -- 1.8.3.1 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/