Ingo,
Please pull the timers/core branch that can be found at:
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
timers/core
---
* The patch from Viresh removes some unecessary scheduler IPIs that wake
up the CPUs when deferrable timers are enqueued on remote targets.
In practice I have seen on boot some of these IPIs from various
sources: MCE, vmstat/SLAB, cpufreq. They happen either on initcall
or cpu hotplug. Since these timers are enqueued on all CPUs, there
are some potential big rounds of IPIs that are spared with this patch.
But it's just what I've seen on my own machine on boot. I expect some
more scenarios where a few IPIs will be avoided depending on configs
and usecases because we have some more users of deferrable timers.
* Kconfig text made clearer for full dynticks by Paul Gortmaker.
Thanks,
Frederic
---
Paul Gortmaker (1):
nohz: ensure users are aware boot CPU is not NO_HZ_FULL
Viresh Kumar (1):
timer: Spare IPI when deferrable timer is queued on idle remote targets
kernel/time/Kconfig | 2 +-
kernel/timer.c | 9 ++++++++-
2 files changed, 9 insertions(+), 2 deletions(-)
From: Paul Gortmaker <[email protected]>
This bit of information is in the Kconfig help text:
"Note the boot CPU will still be kept outside the range to
handle the timekeeping duty."
However neither the variable NO_HZ_FULL_ALL, or the prompt
convey this important detail, so lets add it to the prompt
to make it more explicitly obvious to the average user.
Acked-by: Paul E. McKenney <[email protected]>
Signed-off-by: Paul Gortmaker <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Paul Gortmaker <[email protected]>
Cc: Paul E. McKenney <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Steven Rostedt <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Frederic Weisbecker <[email protected]>
---
kernel/time/Kconfig | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/time/Kconfig b/kernel/time/Kconfig
index 3ce6e8c..f448513 100644
--- a/kernel/time/Kconfig
+++ b/kernel/time/Kconfig
@@ -124,7 +124,7 @@ config NO_HZ_FULL
endchoice
config NO_HZ_FULL_ALL
- bool "Full dynticks system on all CPUs by default"
+ bool "Full dynticks system on all CPUs by default (except CPU 0)"
depends on NO_HZ_FULL
help
If the user doesn't pass the nohz_full boot option to
--
1.8.3.1
From: Viresh Kumar <[email protected]>
When a timer is enqueued or modified on a remote target, the latter is
expected to see and handle this timer on its next tick. However if the
target is idle and CONFIG_NO_HZ_IDLE=y, the CPU may be sleeping tickless
and the timer may be ignored.
wake_up_nohz_cpu() takes care of that by setting TIF_NEED_RESCHED and
sending an IPI to idle targets so that the tick is reevaluated on the
idle loop through the tick_nohz_idle_*() APIs.
Now this is all performed regardless of the power properties of the
timer. If the timer is deferrable, idle targets don't need to be woken
up. Only the next buzy tick needs to care about it, and no IPI kick
is needed for that to happen.
So lets spare the IPI on idle targets when the timer is deferrable.
Meanwhile we keep the current behaviour on full dynticks targets. We can
spare IPIs on idle full dynticks targets as well but some tricky races
against idle_cpu() must be dealt all along to make sure that the timer
is well handled after idle exit. We can deal with that later since
NO_HZ_FULL already has more important powersaving issues.
Reported-by: Thomas Gleixner <[email protected]>
Signed-off-by: Viresh Kumar <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Paul Gortmaker <[email protected]>
Cc: Paul E. McKenney <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Steven Rostedt <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Link: http://lkml.kernel.org/r/CAKohpomMZ0TAN2e6N76_g4ZRzxd5vZ1XfuZfxrP7GMxfTNiLVw@mail.gmail.com
Signed-off-by: Frederic Weisbecker <[email protected]>
---
kernel/timer.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/kernel/timer.c b/kernel/timer.c
index accfd24..b75e789 100644
--- a/kernel/timer.c
+++ b/kernel/timer.c
@@ -939,8 +939,15 @@ void add_timer_on(struct timer_list *timer, int cpu)
* with the timer by holding the timer base lock. This also
* makes sure that a CPU on the way to stop its tick can not
* evaluate the timer wheel.
+ *
+ * Spare the IPI for deferrable timers on idle targets though.
+ * The next busy ticks will take care of it. Except full dynticks
+ * require special care against races with idle_cpu(), lets deal
+ * with that later.
*/
- wake_up_nohz_cpu(cpu);
+ if (!tbase_get_deferrable(timer->base) || tick_nohz_full_cpu(cpu))
+ wake_up_nohz_cpu(cpu);
+
spin_unlock_irqrestore(&base->lock, flags);
}
EXPORT_SYMBOL_GPL(add_timer_on);
--
1.8.3.1
* Frederic Weisbecker <[email protected]> wrote:
> Ingo,
>
> Please pull the timers/core branch that can be found at:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
> timers/core
> ---
>
> * The patch from Viresh removes some unecessary scheduler IPIs that wake
> up the CPUs when deferrable timers are enqueued on remote targets.
>
> In practice I have seen on boot some of these IPIs from various
> sources: MCE, vmstat/SLAB, cpufreq. They happen either on initcall
> or cpu hotplug. Since these timers are enqueued on all CPUs, there
> are some potential big rounds of IPIs that are spared with this patch.
>
> But it's just what I've seen on my own machine on boot. I expect some
> more scenarios where a few IPIs will be avoided depending on configs
> and usecases because we have some more users of deferrable timers.
>
> * Kconfig text made clearer for full dynticks by Paul Gortmaker.
>
> Thanks,
> Frederic
> ---
>
> Paul Gortmaker (1):
> nohz: ensure users are aware boot CPU is not NO_HZ_FULL
>
> Viresh Kumar (1):
> timer: Spare IPI when deferrable timer is queued on idle remote targets
>
>
> kernel/time/Kconfig | 2 +-
> kernel/timer.c | 9 ++++++++-
> 2 files changed, 9 insertions(+), 2 deletions(-)
So I wanted to pull this fine set of commits, just to discover that
Thomas already pulled it into timers/core :-)
Ingo
On Thu, Feb 27, 2014 at 01:17:07PM +0100, Ingo Molnar wrote:
>
> * Frederic Weisbecker <[email protected]> wrote:
>
> > Ingo,
> >
> > Please pull the timers/core branch that can be found at:
> >
> > git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
> > timers/core
> > ---
> >
> > * The patch from Viresh removes some unecessary scheduler IPIs that wake
> > up the CPUs when deferrable timers are enqueued on remote targets.
> >
> > In practice I have seen on boot some of these IPIs from various
> > sources: MCE, vmstat/SLAB, cpufreq. They happen either on initcall
> > or cpu hotplug. Since these timers are enqueued on all CPUs, there
> > are some potential big rounds of IPIs that are spared with this patch.
> >
> > But it's just what I've seen on my own machine on boot. I expect some
> > more scenarios where a few IPIs will be avoided depending on configs
> > and usecases because we have some more users of deferrable timers.
> >
> > * Kconfig text made clearer for full dynticks by Paul Gortmaker.
> >
> > Thanks,
> > Frederic
> > ---
> >
> > Paul Gortmaker (1):
> > nohz: ensure users are aware boot CPU is not NO_HZ_FULL
> >
> > Viresh Kumar (1):
> > timer: Spare IPI when deferrable timer is queued on idle remote targets
> >
> >
> > kernel/time/Kconfig | 2 +-
> > kernel/timer.c | 9 ++++++++-
> > 2 files changed, 9 insertions(+), 2 deletions(-)
>
> So I wanted to pull this fine set of commits, just to discover that
> Thomas already pulled it into timers/core :-)
Ah! Now I understand why it was pulled silently in the evening ;)