2022-05-18 18:50:24

by Uros Bizjak

Subject: [PATCH] sched/clock: Use try_cmpxchg64 in sched_clock_{local,remote}

Use try_cmpxchg64 instead of cmpxchg64 (*ptr, old, new) != old in
sched_clock_{local,remote}. x86 cmpxchg returns success in ZF flag,
so this change saves a compare after cmpxchg (and related move
instruction in front of cmpxchg).

Signed-off-by: Uros Bizjak <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Juri Lelli <[email protected]>
Cc: Vincent Guittot <[email protected]>
Cc: Dietmar Eggemann <[email protected]>
Cc: Steven Rostedt <[email protected]>
Cc: Ben Segall <[email protected]>
Cc: Mel Gorman <[email protected]>
Cc: Daniel Bristot de Oliveira <[email protected]>
---
Patch requires commits 0aa7be05d83cc584da0782405e8007e351dfb6cc
and c2df0a6af177b6c06a859806a876f92b072dc624 from tip.git
---
kernel/sched/clock.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/clock.c b/kernel/sched/clock.c
index d9272d9061a3..e374c0c923da 100644
--- a/kernel/sched/clock.c
+++ b/kernel/sched/clock.c
@@ -287,7 +287,7 @@ notrace static u64 sched_clock_local(struct sched_clock_data *scd)
clock = wrap_max(clock, min_clock);
clock = wrap_min(clock, max_clock);

- if (cmpxchg64(&scd->clock, old_clock, clock) != old_clock)
+ if (!try_cmpxchg64(&scd->clock, &old_clock, clock))
goto again;

return clock;
@@ -349,7 +349,7 @@ notrace static u64 sched_clock_remote(struct sched_clock_data *scd)
val = remote_clock;
}

- if (cmpxchg64(ptr, old_val, val) != old_val)
+ if (!try_cmpxchg64(ptr, &old_val, val))
goto again;

return val;
--
2.35.1

2022-05-19 12:57:12

by Peter Zijlstra

Subject: Re: [PATCH] sched/clock: Use try_cmpxchg64 in sched_clock_{local,remote}

On Wed, May 18, 2022 at 08:49:53PM +0200, Uros Bizjak wrote:
> Use try_cmpxchg64 instead of cmpxchg64 (*ptr, old, new) != old in
> sched_clock_{local,remote}. x86 cmpxchg returns success in ZF flag,
> so this change saves a compare after cmpxchg (and related move
> instruction in front of cmpxchg).
>
> Signed-off-by: Uros Bizjak <[email protected]>

Thanks!

2022-05-23 08:03:51

by tip-bot2 for Alexey Makhalov

Subject: [tip: locking/core] sched/clock: Use try_cmpxchg64 in sched_clock_{local,remote}

The following commit has been merged into the locking/core branch of tip:

Commit-ID: 8491d1bdf5de152f27fc941e2dcdc4e66c950542
Gitweb: https://git.kernel.org/tip/8491d1bdf5de152f27fc941e2dcdc4e66c950542
Author: Uros Bizjak <[email protected]>
AuthorDate: Wed, 18 May 2022 20:49:53 +02:00
Committer: Peter Zijlstra <[email protected]>
CommitterDate: Thu, 19 May 2022 23:46:09 +02:00

sched/clock: Use try_cmpxchg64 in sched_clock_{local,remote}

Use try_cmpxchg64 instead of cmpxchg64 (*ptr, old, new) != old in
sched_clock_{local,remote}. x86 cmpxchg returns success in ZF flag,
so this change saves a compare after cmpxchg (and related move
instruction in front of cmpxchg).

Signed-off-by: Uros Bizjak <[email protected]>
Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Link: https://lkml.kernel.org/r/[email protected]
---
kernel/sched/clock.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/clock.c b/kernel/sched/clock.c
index d9272d9..e374c0c 100644
--- a/kernel/sched/clock.c
+++ b/kernel/sched/clock.c
@@ -287,7 +287,7 @@ again:
clock = wrap_max(clock, min_clock);
clock = wrap_min(clock, max_clock);

- if (cmpxchg64(&scd->clock, old_clock, clock) != old_clock)
+ if (!try_cmpxchg64(&scd->clock, &old_clock, clock))
goto again;

return clock;
@@ -349,7 +349,7 @@ again:
val = remote_clock;
}

- if (cmpxchg64(ptr, old_val, val) != old_val)
+ if (!try_cmpxchg64(ptr, &old_val, val))
goto again;

return val;