by Paul E. McKenney

[permalink] [raw]

Subject: [PATCH v2 cmpxchg 05/13] parisc: __cmpxchg_u32(): lift conversion into the callers

2024-05-01 23:02:04

by Paul E. McKenney

[permalink] [raw]

Subject: [PATCH v2 cmpxchg 02/13] sparc32: make the first argument of __cmpxchg_u64() volatile u64 *

2024-05-01 23:03:06

by Paul E. McKenney

[permalink] [raw]

2024-05-01 23:03:08

by Paul E. McKenney

[permalink] [raw]

Subject: [PATCH v2 cmpxchg 11/13] csky: Emulate one-byte cmpxchg

2024-05-01 23:03:09

by Paul E. McKenney

[permalink] [raw]

Subject: [PATCH v2 cmpxchg 12/13] sh: Emulate one-byte cmpxchg

Use the new cmpxchg_emu_u8() to emulate one-byte cmpxchg() on sh.

[ paulmck: Drop two-byte support per Arnd Bergmann feedback. ]
[ paulmck: Apply feedback from Naresh Kamboju. ]
[ Apply Geert Uytterhoeven feedback. ]

Signed-off-by: Paul E. McKenney <[email protected]>
Cc: Andi Shyti <[email protected]>
Cc: Palmer Dabbelt <[email protected]>
Cc: Masami Hiramatsu <[email protected]>
Cc: Arnd Bergmann <[email protected]>
Cc: <[email protected]>
---
arch/sh/Kconfig | 1 +
arch/sh/include/asm/cmpxchg.h | 3 +++
2 files changed, 4 insertions(+)

diff --git a/arch/sh/Kconfig b/arch/sh/Kconfig
index 2ad3e29f0ebec..f47e9ccf4efd2 100644
--- a/arch/sh/Kconfig
+++ b/arch/sh/Kconfig
@@ -16,6 +16,7 @@ config SUPERH
select ARCH_HIBERNATION_POSSIBLE if MMU
select ARCH_MIGHT_HAVE_PC_PARPORT
select ARCH_WANT_IPC_PARSE_VERSION
+ select ARCH_NEED_CMPXCHG_1_EMU
select CPU_NO_EFFICIENT_FFS
select DMA_DECLARE_COHERENT
select GENERIC_ATOMIC64
diff --git a/arch/sh/include/asm/cmpxchg.h b/arch/sh/include/asm/cmpxchg.h
index 5d617b3ef78f7..1e5dc5ccf7bf5 100644
--- a/arch/sh/include/asm/cmpxchg.h
+++ b/arch/sh/include/asm/cmpxchg.h
@@ -9,6 +9,7 @@

#include <linux/compiler.h>
#include <linux/types.h>
+#include <linux/cmpxchg-emu.h>

#if defined(CONFIG_GUSA_RB)
#include <asm/cmpxchg-grb.h>
@@ -56,6 +57,8 @@ static inline unsigned long __cmpxchg(volatile void * ptr, unsigned long old,
unsigned long new, int size)
{
switch (size) {
+ case 1:
+ return cmpxchg_emu_u8(ptr, old, new);
case 4:
return __cmpxchg_u32(ptr, old, new);
}
--
2.40.1

2024-05-01 23:03:43

by Paul E. McKenney

[permalink] [raw]

Subject: [PATCH v2 cmpxchg 01/13] sparc32: make __cmpxchg_u32() return u32

2024-05-01 23:03:44

by Paul E. McKenney

[permalink] [raw]

Subject: [PATCH v2 cmpxchg 03/13] sparc32: unify __cmpxchg_u{32,64}

2024-05-01 23:03:45

by Paul E. McKenney

[permalink] [raw]

Subject: [PATCH v2 cmpxchg 04/13] sparc32: add __cmpxchg_u{8,16}() and teach __cmpxchg() to handle those sizes

2024-05-01 23:04:23

by Paul E. McKenney

[permalink] [raw]

Subject: [PATCH v2 cmpxchg 09/13] lib: Add one-byte emulation function

Architectures are required to provide four-byte cmpxchg() and 64-bit
architectures are additionally required to provide eight-byte cmpxchg().
However, there are cases where one-byte cmpxchg() would be extremely
useful. Therefore, provide cmpxchg_emu_u8() that emulates one-byte
cmpxchg() in terms of four-byte cmpxchg().

Note that this emulations is fully ordered, and can (for example) cause
one-byte cmpxchg_relaxed() to incur the overhead of full ordering.
If this causes problems for a given architecture, that architecture is
free to provide its own lighter-weight primitives.

[ paulmck: Apply Marco Elver feedback. ]
[ paulmck: Apply kernel test robot feedback. ]
[ paulmck: Drop two-byte support per Arnd Bergmann feedback. ]

Link: https://lore.kernel.org/all/0733eb10-5e7a-4450-9b8a-527b97c842ff@paulmck-laptop/

Signed-off-by: Paul E. McKenney <[email protected]>
Acked-by: Marco Elver <[email protected]>
Cc: Andrew Morton <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: "Peter Zijlstra (Intel)" <[email protected]>
Cc: Douglas Anderson <[email protected]>
Cc: Petr Mladek <[email protected]>
Cc: Arnd Bergmann <[email protected]>
Cc: <[email protected]>
---
arch/Kconfig | 3 +++
include/linux/cmpxchg-emu.h | 15 +++++++++++++
lib/Makefile | 1 +
lib/cmpxchg-emu.c | 45 +++++++++++++++++++++++++++++++++++++
4 files changed, 64 insertions(+)
create mode 100644 include/linux/cmpxchg-emu.h
create mode 100644 lib/cmpxchg-emu.c

diff --git a/arch/Kconfig b/arch/Kconfig
index 9f066785bb71d..284663392eef8 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -1609,4 +1609,7 @@ config CC_HAS_SANE_FUNCTION_ALIGNMENT
# strict alignment always, even with -falign-functions.
def_bool CC_HAS_MIN_FUNCTION_ALIGNMENT || CC_IS_CLANG

+config ARCH_NEED_CMPXCHG_1_EMU
+ bool
+
endmenu
diff --git a/include/linux/cmpxchg-emu.h b/include/linux/cmpxchg-emu.h
new file mode 100644
index 0000000000000..998deec67740a
--- /dev/null
+++ b/include/linux/cmpxchg-emu.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Emulated 1-byte and 2-byte cmpxchg operations for architectures
+ * lacking direct support for these sizes. These are implemented in terms
+ * of 4-byte cmpxchg operations.
+ *
+ * Copyright (C) 2024 Paul E. McKenney.
+ */
+
+#ifndef __LINUX_CMPXCHG_EMU_H
+#define __LINUX_CMPXCHG_EMU_H
+
+uintptr_t cmpxchg_emu_u8(volatile u8 *p, uintptr_t old, uintptr_t new);
+
+#endif /* __LINUX_CMPXCHG_EMU_H */
diff --git a/lib/Makefile b/lib/Makefile
index ffc6b2341b45a..cc3d52fdb477d 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -236,6 +236,7 @@ obj-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o
lib-$(CONFIG_GENERIC_BUG) += bug.o

obj-$(CONFIG_HAVE_ARCH_TRACEHOOK) += syscall.o
+obj-$(CONFIG_ARCH_NEED_CMPXCHG_1_EMU) += cmpxchg-emu.o

obj-$(CONFIG_DYNAMIC_DEBUG_CORE) += dynamic_debug.o
#ensure exported functions have prototypes
diff --git a/lib/cmpxchg-emu.c b/lib/cmpxchg-emu.c
new file mode 100644
index 0000000000000..27f6f97cb60dd
--- /dev/null
+++ b/lib/cmpxchg-emu.c
@@ -0,0 +1,45 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Emulated 1-byte cmpxchg operation for architectures lacking direct
+ * support for this size. This is implemented in terms of 4-byte cmpxchg
+ * operations.
+ *
+ * Copyright (C) 2024 Paul E. McKenney.
+ */
+
+#include <linux/types.h>
+#include <linux/export.h>
+#include <linux/instrumented.h>
+#include <linux/atomic.h>
+#include <linux/panic.h>
+#include <linux/bug.h>
+#include <asm-generic/rwonce.h>
+#include <linux/cmpxchg-emu.h>
+
+union u8_32 {
+ u8 b[4];
+ u32 w;
+};
+
+/* Emulate one-byte cmpxchg() in terms of 4-byte cmpxchg. */
+uintptr_t cmpxchg_emu_u8(volatile u8 *p, uintptr_t old, uintptr_t new)
+{
+ u32 *p32 = (u32 *)(((uintptr_t)p) & ~0x3);
+ int i = ((uintptr_t)p) & 0x3;
+ union u8_32 old32;
+ union u8_32 new32;
+ u32 ret;
+
+ ret = READ_ONCE(*p32);
+ do {
+ old32.w = ret;
+ if (old32.b[i] != old)
+ return old32.b[i];
+ new32.w = old32.w;
+ new32.b[i] = new;
+ instrument_atomic_read_write(p, 1);
+ ret = data_race(cmpxchg(p32, old32.w, new32.w)); // Overridden above.
+ } while (ret != old32.w);
+ return old;
+}
+EXPORT_SYMBOL_GPL(cmpxchg_emu_u8);
--
2.40.1

2024-05-01 23:05:29

by Paul E. McKenney

[permalink] [raw]

Hello!

This v3 series uses one-byte cmpxchg emulation for those architectures
that do not support this in hardware and that are not already using this
emulation. The emulation is in terms of the fully ordered four-byte
cmpxchg() that is supplied by all of these architectures. This was
tested by the csky commit already in mainline and by making x86 forget
that it can do one-byte cmpxchg() natively:

f0183ab28489 ("EXP arch/x86: Test one-byte cmpxchg emulation")

This x86 commit has since been dropped from RCU in favor of the
aforementioned csky commit in mainline.

Once one-byte cmpxchg emulation patches are in mainline for all remaining
architectures in need of it, RCU Tasks will use one-byte cmpxchg()
in place of the current rcu_trc_cmpxchg_need_qs() open-coding of this
emulation.

The remaining patches are as follows:

1. ARC: Emulate one-byte cmpxchg.

2. Emulate one-byte cmpxchg.

3. Emulate one-byte cmpxchg.

4. ARM: Emulate one-byte cmpxchg.

Changes since v2:

o Dropped the sparc32, parisc, lib, and csky patches due to their
having been accepted into mainline.

o Added a 32-bit ARM patch for systems with v6 and earlier CPUs.

o Apply other feedback from review and testing.

Changes since v1:

o Dropped riscv patch in favor of alternative patch that
provides native support.

o Fixed yet more casting bugs spotted by kernel test robot
and by Geert Uytterhoeven.

Changes since RFC:

o Add native support for sparc32 and parisc, courtesy of Al Viro.

o Remove two-byte emulation due to architectures that still do not
support two-byte load and store instructions, per Arnd Bergmann
feedback. (Yes, there are a few systems out there that do not
even support one-byte load instructions, but these are slated
for removal anyway.)

o Fix numerous casting bugs spotted by kernel test robot.

o Fix SPDX header. "//" for .c files and "/*" for .h files.
I am sure that there is a good reason for this. ;-)

Thanx, Paul

------------------------------------------------------------------------

arc/Kconfig | 1 +
arc/include/asm/cmpxchg.h | 33 ++++++++++++++++++++++++---------
arm/Kconfig | 1 +
arm/include/asm/cmpxchg.h | 7 ++++++-
sh/Kconfig | 1 +
sh/include/asm/cmpxchg.h | 3 +++
xtensa/Kconfig | 1 +
xtensa/include/asm/cmpxchg.h | 2 ++
8 files changed, 39 insertions(+), 10 deletions(-)