Hi,
the following patch is a win for power management on x86....
... but since this touches generic code.. are there any
other architectures that would be negatively affected by this?
Subject: [patch] Remove the per cpu tick skew
Historically, Linux has tried to make the regular timer tick on the various
CPUs not happen at the same time, to avoid contention on xtime_lock.
Nowadays, with the tickless kernel, this contention no longer happens
since time keeping and updating are done differently. In addition,
this skew is actually hurting power consumption in a measurable
way on many-core systems.
Signed-off-by: Arjan van de Ven <[email protected]>
--- linux.trees.git/kernel/time/tick-sched.c~ 2010-07-16 09:40:50.000000000 -0400
+++ linux.trees.git/kernel/time/tick-sched.c 2010-07-26 11:18:51.138003329 -0400
@@ -780,7 +780,6 @@
{
struct tick_sched *ts = &__get_cpu_var(tick_cpu_sched);
ktime_t now = ktime_get();
- u64 offset;
/*
* Emulate tick processing via per-CPU hrtimers:
@@ -790,10 +789,6 @@
/* Get the next period (per cpu) */
hrtimer_set_expires(&ts->sched_timer, tick_init_jiffy_update());
- offset = ktime_to_ns(tick_period) >> 1;
- do_div(offset, num_possible_cpus());
- offset *= smp_processor_id();
- hrtimer_add_expires_ns(&ts->sched_timer, offset);
for (;;) {
hrtimer_forward(&ts->sched_timer, now, tick_period);
--
Arjan van de Ven Intel Open Source Technology Centre
For development, discussion and tips for power savings,
visit http://www.lesswatts.org
On Tue, Jul 27, 2010 at 9:02 PM, Arjan van de Ven <[email protected]> wrote:
> the following patch is a win for power management on x86....
> ... but since this touches generic code.. are there any
> other architectures that would be negatively affected by this?
It was added to avoid contention when all the cpus grabbed the
xtime_lock (causing latency spikes of timer overhead * number of
cpus). We don't grab the xtime lock everywhere anymore, so that
shouldn't bite us, but I'm curious if there are not other global locks
taken that may see extra contention without this change.
> Subject: [patch] Remove the per cpu tick skew
>
> Historically, Linux has tried to make the regular timer tick on the various
> CPUs not happen at the same time, to avoid contention on xtime_lock.
>
> Nowadays, with the tickless kernel, this contention no longer happens
> since time keeping and updating are done differently. In addition,
> this skew is actually hurting power consumption in a measurable
> way on many-core systems.
>
> Signed-off-by: Arjan van de Ven <[email protected]>
I'll give it a spin against -rt and see if we show any latency jumps.
thanks
-john
On Wed, Jul 28, 2010 at 1:26 PM, john stultz <[email protected]> wrote:
> On Tue, Jul 27, 2010 at 9:02 PM, Arjan van de Ven <[email protected]> wrote:
>> Historically, Linux has tried to make the regular timer tick on the various
>> CPUs not happen at the same time, to avoid contention on xtime_lock.
>>
>> Nowadays, with the tickless kernel, this contention no longer happens
>> since time keeping and updating are done differently. In addition,
>> this skew is actually hurting power consumption in a measurable
>> way on many-core systems.
>>
>
> I'll give it a spin against -rt and see if we show any latency jumps.
In my testing on a 8way box, I didn't see any concerning latencies
with this patch running cyclictest or a FIFO99 gtod loop.
thanks
-john
On Tue, Jul 27, 2010 at 09:02:10PM -0700, Arjan van de Ven wrote:
> Hi,
>
> the following patch is a win for power management on x86....
> ... but since this touches generic code.. are there any
> other architectures that would be negatively affected by this?
>
>
>
> Subject: [patch] Remove the per cpu tick skew
>
> Historically, Linux has tried to make the regular timer tick on the various
> CPUs not happen at the same time, to avoid contention on xtime_lock.
>
> Nowadays, with the tickless kernel, this contention no longer happens
> since time keeping and updating are done differently. In addition,
> this skew is actually hurting power consumption in a measurable
> way on many-core systems.
Question, how much of a win is it? What does it do that tickless
idle does not, can you explain?
>
> Signed-off-by: Arjan van de Ven <[email protected]>
>
> --- linux.trees.git/kernel/time/tick-sched.c~ 2010-07-16 09:40:50.000000000 -0400
> +++ linux.trees.git/kernel/time/tick-sched.c 2010-07-26 11:18:51.138003329 -0400
> @@ -780,7 +780,6 @@
> {
> struct tick_sched *ts = &__get_cpu_var(tick_cpu_sched);
> ktime_t now = ktime_get();
> - u64 offset;
>
> /*
> * Emulate tick processing via per-CPU hrtimers:
> @@ -790,10 +789,6 @@
>
> /* Get the next period (per cpu) */
> hrtimer_set_expires(&ts->sched_timer, tick_init_jiffy_update());
> - offset = ktime_to_ns(tick_period) >> 1;
> - do_div(offset, num_possible_cpus());
> - offset *= smp_processor_id();
> - hrtimer_add_expires_ns(&ts->sched_timer, offset);
>
> for (;;) {
> hrtimer_forward(&ts->sched_timer, now, tick_period);
>
> --
> Arjan van de Ven Intel Open Source Technology Centre
> For development, discussion and tips for power savings,
> visit http://www.lesswatts.org
> --
> To unsubscribe from this list: send the line "unsubscribe linux-arch" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, 30 Jul 2010 17:27:04 +1000
Nick Piggin <[email protected]> wrote:
> > Nowadays, with the tickless kernel, this contention no longer
> > happens since time keeping and updating are done differently. In
> > addition, this skew is actually hurting power consumption in a
> > measurable way on many-core systems.
>
> Question, how much of a win is it? What does it do that tickless
> idle does not, can you explain?
tickless idle works great if you're really almost idle
if there's "some work but not fully busy" this still matters
this is not about 'a few milliwatts', but on a server in our labs
(sorry, no hardware details in public) this effect is in the "several
dozen Watts" range.
--
Arjan van de Ven Intel Open Source Technology Centre
For development, discussion and tips for power savings,
visit http://www.lesswatts.org
Commit-ID: af5ab277ded04bd9bc6b048c5a2f0e7d70ef0867
Gitweb: http://git.kernel.org/tip/af5ab277ded04bd9bc6b048c5a2f0e7d70ef0867
Author: Arjan van de Ven <[email protected]>
AuthorDate: Tue, 27 Jul 2010 21:02:10 -0700
Committer: Thomas Gleixner <[email protected]>
CommitDate: Mon, 2 Aug 2010 21:45:58 +0200
clockevents: Remove the per cpu tick skew
Historically, Linux has tried to make the regular timer tick on the
various CPUs not happen at the same time, to avoid contention on
xtime_lock.
Nowadays, with the tickless kernel, this contention no longer happens
since time keeping and updating are done differently. In addition,
this skew is actually hurting power consumption in a measurable way on
many-core systems.
Signed-off-by: Arjan van de Ven <[email protected]>
LKML-Reference: <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
---
kernel/time/tick-sched.c | 5 -----
1 files changed, 0 insertions(+), 5 deletions(-)
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 813993b..74644cc 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -780,7 +780,6 @@ void tick_setup_sched_timer(void)
{
struct tick_sched *ts = &__get_cpu_var(tick_cpu_sched);
ktime_t now = ktime_get();
- u64 offset;
/*
* Emulate tick processing via per-CPU hrtimers:
@@ -790,10 +789,6 @@ void tick_setup_sched_timer(void)
/* Get the next period (per cpu) */
hrtimer_set_expires(&ts->sched_timer, tick_init_jiffy_update());
- offset = ktime_to_ns(tick_period) >> 1;
- do_div(offset, num_possible_cpus());
- offset *= smp_processor_id();
- hrtimer_add_expires_ns(&ts->sched_timer, offset);
for (;;) {
hrtimer_forward(&ts->sched_timer, now, tick_period);