2023-04-13 18:07:22

by Aaron Thompson

Subject: [RESEND PATCH] sched/clock: Fix local_clock() before sched_clock_init()

Have local_clock() return sched_clock() if sched_clock_init() has not
yet run. sched_clock_cpu() has this check but it was not included in the
new noinstr implementation of local_clock().

The effect can be seen on x86 with CONFIG_PRINTK_TIME enabled, for
instance. scd->clock quickly reaches the value of TICK_NSEC and that
value is returned until sched_clock_init() runs.

dmesg without this patch:

[ 0.000000] kvm-clock: ...
[ 0.000002] kvm-clock: ...
[ 0.000672] clocksource: ...
[ 0.001000] tsc: ...
[ 0.001000] e820: ...
[ 0.001000] e820: ...
...
[ 0.001000] ..TIMER: ...
[ 0.001000] clocksource: ...
[ 0.378956] Calibrating delay loop ...
[ 0.379955] pid_max: ...

dmesg with this patch:

[ 0.000000] kvm-clock: ...
[ 0.000001] kvm-clock: ...
[ 0.000675] clocksource: ...
[ 0.002685] tsc: ...
[ 0.003331] e820: ...
[ 0.004190] e820: ...
...
[ 0.421939] ..TIMER: ...
[ 0.422842] clocksource: ...
[ 0.424582] Calibrating delay loop ...
[ 0.425580] pid_max: ...

Fixes: 776f22913b8e ("sched/clock: Make local_clock() noinstr")
Signed-off-by: Aaron Thompson <[email protected]>
---
kernel/sched/clock.c | 3 +++
1 file changed, 3 insertions(+)

diff --git a/kernel/sched/clock.c b/kernel/sched/clock.c
index 5732fa75ebab..b5cc2b53464d 100644
--- a/kernel/sched/clock.c
+++ b/kernel/sched/clock.c
@@ -300,6 +300,9 @@ noinstr u64 local_clock(void)
if (static_branch_likely(&__sched_clock_stable))
return sched_clock() + __sched_clock_offset;

+ if (!static_branch_likely(&sched_clock_running))
+ return sched_clock();
+
preempt_disable_notrace();
clock = sched_clock_local(this_scd());
preempt_enable_notrace();
--
2.39.2

2023-04-14 08:28:28

by Peter Zijlstra

Subject: Re: [RESEND PATCH] sched/clock: Fix local_clock() before sched_clock_init()

On Thu, Apr 13, 2023 at 05:50:12PM +0000, Aaron Thompson wrote:
> Have local_clock() return sched_clock() if sched_clock_init() has not
> yet run. sched_clock_cpu() has this check but it was not included in the
> new noinstr implementation of local_clock().
>
> The effect can be seen on x86 with CONFIG_PRINTK_TIME enabled, for
> instance. scd->clock quickly reaches the value of TICK_NSEC and that
> value is returned until sched_clock_init() runs.
>
> dmesg without this patch:
>
> [ 0.000000] kvm-clock: ...
> [ 0.000002] kvm-clock: ...
> [ 0.000672] clocksource: ...
> [ 0.001000] tsc: ...
> [ 0.001000] e820: ...
> [ 0.001000] e820: ...
> ...
> [ 0.001000] ..TIMER: ...
> [ 0.001000] clocksource: ...
> [ 0.378956] Calibrating delay loop ...
> [ 0.379955] pid_max: ...
>
> dmesg with this patch:
>
> [ 0.000000] kvm-clock: ...
> [ 0.000001] kvm-clock: ...
> [ 0.000675] clocksource: ...
> [ 0.002685] tsc: ...
> [ 0.003331] e820: ...
> [ 0.004190] e820: ...
> ...
> [ 0.421939] ..TIMER: ...
> [ 0.422842] clocksource: ...
> [ 0.424582] Calibrating delay loop ...
> [ 0.425580] pid_max: ...
>
> Fixes: 776f22913b8e ("sched/clock: Make local_clock() noinstr")
> Signed-off-by: Aaron Thompson <[email protected]>

Thanks!

2023-04-22 07:45:07

by tip-bot2 for Jacob Pan

Subject: [tip: sched/core] sched/clock: Fix local_clock() before sched_clock_init()

The following commit has been merged into the sched/core branch of tip:

Commit-ID: f31dcb152a3d0816e2f1deab4e64572336da197d
Gitweb: https://git.kernel.org/tip/f31dcb152a3d0816e2f1deab4e64572336da197d
Author: Aaron Thompson <[email protected]>
AuthorDate: Thu, 13 Apr 2023 17:50:12
Committer: Peter Zijlstra <[email protected]>
CommitterDate: Fri, 21 Apr 2023 13:24:21 +02:00

sched/clock: Fix local_clock() before sched_clock_init()

Have local_clock() return sched_clock() if sched_clock_init() has not
yet run. sched_clock_cpu() has this check but it was not included in the
new noinstr implementation of local_clock().

The effect can be seen on x86 with CONFIG_PRINTK_TIME enabled, for
instance. scd->clock quickly reaches the value of TICK_NSEC and that
value is returned until sched_clock_init() runs.

dmesg without this patch:

[ 0.000000] kvm-clock: ...
[ 0.000002] kvm-clock: ...
[ 0.000672] clocksource: ...
[ 0.001000] tsc: ...
[ 0.001000] e820: ...
[ 0.001000] e820: ...
...
[ 0.001000] ..TIMER: ...
[ 0.001000] clocksource: ...
[ 0.378956] Calibrating delay loop ...
[ 0.379955] pid_max: ...

dmesg with this patch:

[ 0.000000] kvm-clock: ...
[ 0.000001] kvm-clock: ...
[ 0.000675] clocksource: ...
[ 0.002685] tsc: ...
[ 0.003331] e820: ...
[ 0.004190] e820: ...
...
[ 0.421939] ..TIMER: ...
[ 0.422842] clocksource: ...
[ 0.424582] Calibrating delay loop ...
[ 0.425580] pid_max: ...

Fixes: 776f22913b8e ("sched/clock: Make local_clock() noinstr")
Signed-off-by: Aaron Thompson <[email protected]>
Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Link: https://lkml.kernel.org/r/[email protected]
---
kernel/sched/clock.c | 3 +++
1 file changed, 3 insertions(+)

diff --git a/kernel/sched/clock.c b/kernel/sched/clock.c
index 5732fa7..b5cc2b5 100644
--- a/kernel/sched/clock.c
+++ b/kernel/sched/clock.c
@@ -300,6 +300,9 @@ noinstr u64 local_clock(void)
if (static_branch_likely(&__sched_clock_stable))
return sched_clock() + __sched_clock_offset;

+ if (!static_branch_likely(&sched_clock_running))
+ return sched_clock();
+
preempt_disable_notrace();
clock = sched_clock_local(this_scd());
preempt_enable_notrace();