2021-11-09 23:59:10

by Valentin Schneider

[permalink] [raw]
Subject: [PATCH] sched: Split preemption model selection between DYNAMIC & !DYNAMIC

From: Mike Galbraith <[email protected]>

Commit c597bfddc9e9 ("sched: Provide Kconfig support for default dynamic
preempt mode") changed the selectable config names for the preemption
model. This means a config file must now select

CONFIG_PREEMPT_BEHAVIOUR=y

rather than

CONFIG_PREEMPT=y

to get a preemptible kernel. This means all arch config files would need to
be updated - right now they'll all end up with the default
CONFIG_PREEMPT_NONE_BEHAVIOUR.

Rather than touch a good hundred of config files, split the preemption
model selection based on PREEMPT_DYNAMIC. !PREEMPT_DYNAMIC reverts to the
old behaviour with CONFIG_PREEMPT & friends being the user-visible config,
while PREEMPT_DYNAMIC exposes the _BEHAVIOUR counterparts.

Signed-off-by: Mike Galbraith <[email protected]>
[Changelog; deduplicated some config logic]
Signed-off-by: Valentin Schneider <[email protected]>
---
kernel/Kconfig.preempt | 80 +++++++++++++++++++++++-------------------
1 file changed, 43 insertions(+), 37 deletions(-)

diff --git a/kernel/Kconfig.preempt b/kernel/Kconfig.preempt
index 60f1bfc3c7b2..4e60afae248e 100644
--- a/kernel/Kconfig.preempt
+++ b/kernel/Kconfig.preempt
@@ -1,12 +1,32 @@
# SPDX-License-Identifier: GPL-2.0-only

+config PREEMPT_DYNAMIC
+ bool "Preemption behaviour defined on boot"
+ depends on HAVE_PREEMPT_DYNAMIC && !ARCH_NO_PREEMPT
+ select PREEMPT
+ help
+ This option allows to define the preemption model on the kernel
+ command line parameter and thus override the default preemption
+ model defined during compile time.
+
+ The feature is primarily interesting for Linux distributions which
+ provide a pre-built kernel binary to reduce the number of kernel
+ flavors they offer while still offering different usecases.
+
+ The runtime overhead is negligible with HAVE_STATIC_CALL_INLINE enabled
+ but if runtime patching is not available for the specific architecture
+ then the potential overhead should be considered.
+
+ Interesting if you want the same pre-built kernel should be used for
+ both Server and Desktop workloads.
+
choice
prompt "Preemption Model"
- default PREEMPT_NONE_BEHAVIOUR
+ default PREEMPT_NONE

-config PREEMPT_NONE_BEHAVIOUR
+config PREEMPT_NONE
bool "No Forced Preemption (Server)"
- select PREEMPT_NONE if !PREEMPT_DYNAMIC
+ depends on !PREEMPT_DYNAMIC
help
This is the traditional Linux preemption model, geared towards
throughput. It will still provide good latencies most of the
@@ -18,10 +38,9 @@ config PREEMPT_NONE_BEHAVIOUR
raw processing power of the kernel, irrespective of scheduling
latencies.

-config PREEMPT_VOLUNTARY_BEHAVIOUR
+config PREEMPT_VOLUNTARY
bool "Voluntary Kernel Preemption (Desktop)"
- depends on !ARCH_NO_PREEMPT
- select PREEMPT_VOLUNTARY if !PREEMPT_DYNAMIC
+ depends on !ARCH_NO_PREEMPT && !PREEMPT_DYNAMIC
help
This option reduces the latency of the kernel by adding more
"explicit preemption points" to the kernel code. These new
@@ -37,10 +56,11 @@ config PREEMPT_VOLUNTARY_BEHAVIOUR

Select this if you are building a kernel for a desktop system.

-config PREEMPT_BEHAVIOUR
+config PREEMPT
bool "Preemptible Kernel (Low-Latency Desktop)"
depends on !ARCH_NO_PREEMPT
- select PREEMPT
+ select PREEMPTION
+ select UNINLINE_SPIN_UNLOCK if !ARCH_INLINE_SPIN_UNLOCK
help
This option reduces the latency of the kernel by making
all kernel code (that is not executing in a critical section)
@@ -75,16 +95,23 @@ config PREEMPT_RT

endchoice

-config PREEMPT_NONE
- bool
+choice
+ prompt "Default boot-time Preemption Model"
+ depends on PREEMPT_DYNAMIC
+ default PREEMPT_NONE_BEHAVIOUR
+ help
+ This option defines the default preemption model of the kernel
+ if it hasn't been specified by the "preempt=" command line parameter.

-config PREEMPT_VOLUNTARY
- bool
+config PREEMPT_NONE_BEHAVIOUR
+ bool "No Forced Preemption (Server)"

-config PREEMPT
- bool
- select PREEMPTION
- select UNINLINE_SPIN_UNLOCK if !ARCH_INLINE_SPIN_UNLOCK
+config PREEMPT_VOLUNTARY_BEHAVIOUR
+ bool "Voluntary Kernel Preemption (Desktop)"
+
+config PREEMPT_BEHAVIOUR
+ bool "Preemptible Kernel (Low-Latency Desktop)"
+endchoice

config PREEMPT_COUNT
bool
@@ -93,27 +120,6 @@ config PREEMPTION
bool
select PREEMPT_COUNT

-config PREEMPT_DYNAMIC
- bool "Preemption behaviour defined on boot"
- depends on HAVE_PREEMPT_DYNAMIC
- select PREEMPT
- default y
- help
- This option allows to define the preemption model on the kernel
- command line parameter and thus override the default preemption
- model defined during compile time.
-
- The feature is primarily interesting for Linux distributions which
- provide a pre-built kernel binary to reduce the number of kernel
- flavors they offer while still offering different usecases.
-
- The runtime overhead is negligible with HAVE_STATIC_CALL_INLINE enabled
- but if runtime patching is not available for the specific architecture
- then the potential overhead should be considered.
-
- Interesting if you want the same pre-built kernel should be used for
- both Server and Desktop workloads.
-
config SCHED_CORE
bool "Core Scheduling for SMT"
depends on SCHED_SMT
--
2.25.1


2021-11-10 01:23:23

by Frederic Weisbecker

[permalink] [raw]
Subject: Re: [PATCH] sched: Split preemption model selection between DYNAMIC & !DYNAMIC

On Tue, Nov 09, 2021 at 03:10:57PM +0000, Valentin Schneider wrote:
> +choice
> + prompt "Default boot-time Preemption Model"
> + depends on PREEMPT_DYNAMIC
> + default PREEMPT_NONE_BEHAVIOUR
> + help
> + This option defines the default preemption model of the kernel
> + if it hasn't been specified by the "preempt=" command line parameter.
>
> -config PREEMPT_VOLUNTARY
> - bool
> +config PREEMPT_NONE_BEHAVIOUR
> + bool "No Forced Preemption (Server)"
>
> -config PREEMPT
> - bool
> - select PREEMPTION
> - select UNINLINE_SPIN_UNLOCK if !ARCH_INLINE_SPIN_UNLOCK
> +config PREEMPT_VOLUNTARY_BEHAVIOUR
> + bool "Voluntary Kernel Preemption (Desktop)"
> +
> +config PREEMPT_BEHAVIOUR
> + bool "Preemptible Kernel (Low-Latency Desktop)"
> +endchoice

The problem here is that you're duplicating the existing titles
for the static entries and you're losing all the help that text that used to come
along. The point of the BEHAVIOUR thing, further mapping to either
static or dynamic preemption, was to avoid that duplication and keep the
same titles and help for the three behaviour, whether static or dynamic.

Thanks.

2021-11-10 04:43:35

by Mike Galbraith

[permalink] [raw]
Subject: Re: [PATCH] sched: Split preemption model selection between DYNAMIC & !DYNAMIC

On Wed, 2021-11-10 at 02:17 +0100, Frederic Weisbecker wrote:
> On Tue, Nov 09, 2021 at 03:10:57PM +0000, Valentin Schneider wrote:
> > +choice
> > +       prompt "Default boot-time Preemption Model"
> > +       depends on PREEMPT_DYNAMIC
> > +       default PREEMPT_NONE_BEHAVIOUR
> > +       help
> > +         This option defines the default preemption model of the kernel
> > +         if it hasn't been specified by the "preempt=" command line parameter.
> >  
> > -config PREEMPT_VOLUNTARY
> > -       bool
> > +config PREEMPT_NONE_BEHAVIOUR
> > +       bool "No Forced Preemption (Server)"
> >  
> > -config PREEMPT
> > -       bool
> > -       select PREEMPTION
> > -       select UNINLINE_SPIN_UNLOCK if !ARCH_INLINE_SPIN_UNLOCK
> > +config PREEMPT_VOLUNTARY_BEHAVIOUR
> > +       bool "Voluntary Kernel Preemption (Desktop)"
> > +
> > +config PREEMPT_BEHAVIOUR
> > +       bool "Preemptible Kernel (Low-Latency Desktop)"
> > +endchoice
>
> The problem here is that you're duplicating the existing titles
> for the static entries and you're losing all the help that text that used to come
> along. The point of the BEHAVIOUR thing, further mapping to either
> static or dynamic preemption, was to avoid that duplication and keep the
> same titles and help for the three behaviour, whether static or dynamic.

Adding "depends on !PREEMPT_DYNAMIC" to the first (legacy?) choice made
it appear/disappear at the appropriate time as well, but I have no idea
how duplication of help text can be avoided.

-Mike

2021-11-10 11:27:21

by Valentin Schneider

[permalink] [raw]
Subject: Re: [PATCH] sched: Split preemption model selection between DYNAMIC & !DYNAMIC

On 10/11/21 05:37, Mike Galbraith wrote:
> On Wed, 2021-11-10 at 02:17 +0100, Frederic Weisbecker wrote:
>> The problem here is that you're duplicating the existing titles
>> for the static entries and you're losing all the help that text that used to come
>> along. The point of the BEHAVIOUR thing, further mapping to either
>> static or dynamic preemption, was to avoid that duplication and keep the
>> same titles and help for the three behaviour, whether static or dynamic.
>
> Adding "depends on !PREEMPT_DYNAMIC" to the first (legacy?) choice made
> it appear/disappear at the appropriate time as well, but I have no idea
> how duplication of help text can be avoided.
>
> -Mike

Following Frederic's _STATIC suggestion, I got to the below. The nice thing
is if a defconfig picks say PREEMPT_VOLUNTARY and the arch supports
PREEMPT_DYNAMIC, then it'll pick PREEMPT_VOLUNTARY_BEHAVIOUR. The less nice
thing is that we end up selecting PREEMPT_STATIC for PREEMPT_DYNAMIC
kernels, naming's hard... Maybe _BUILD rather than _STATIC?

---
diff --git a/include/linux/kernel.h b/include/linux/kernel.h
index 2776423a587e..1fa8f6fb3c55 100644
--- a/include/linux/kernel.h
+++ b/include/linux/kernel.h
@@ -88,7 +88,7 @@
struct completion;
struct user;

-#ifdef CONFIG_PREEMPT_VOLUNTARY
+#ifdef CONFIG_PREEMPT_VOLUNTARY_STATIC

extern int __cond_resched(void);
# define might_resched() __cond_resched()
diff --git a/include/linux/vermagic.h b/include/linux/vermagic.h
index 1eaaa93c37bf..0a311d7aab62 100644
--- a/include/linux/vermagic.h
+++ b/include/linux/vermagic.h
@@ -15,7 +15,7 @@
#else
#define MODULE_VERMAGIC_SMP ""
#endif
-#ifdef CONFIG_PREEMPT
+#ifdef CONFIG_PREEMPT_STATIC
#define MODULE_VERMAGIC_PREEMPT "preempt "
#elif defined(CONFIG_PREEMPT_RT)
#define MODULE_VERMAGIC_PREEMPT "preempt_rt "
@@ -39,9 +39,9 @@
#define MODULE_RANDSTRUCT_PLUGIN
#endif

-#define VERMAGIC_STRING \
+#define VERMAGIC_STRING \
UTS_RELEASE " " \
- MODULE_VERMAGIC_SMP MODULE_VERMAGIC_PREEMPT \
+ MODULE_VERMAGIC_SMP MODULE_VERMAGIC_PREEMPT \
MODULE_VERMAGIC_MODULE_UNLOAD MODULE_VERMAGIC_MODVERSIONS \
MODULE_ARCH_VERMAGIC \
MODULE_RANDSTRUCT_PLUGIN
diff --git a/init/Makefile b/init/Makefile
index 2846113677ee..989f43d23b83 100644
--- a/init/Makefile
+++ b/init/Makefile
@@ -30,7 +30,7 @@ $(obj)/version.o: include/generated/compile.h
quiet_cmd_compile.h = CHK $@
cmd_compile.h = \
$(CONFIG_SHELL) $(srctree)/scripts/mkcompile_h $@ \
- "$(UTS_MACHINE)" "$(CONFIG_SMP)" "$(CONFIG_PREEMPT)" \
+ "$(UTS_MACHINE)" "$(CONFIG_SMP)" "$(CONFIG_PREEMPT_STATIC)" \
"$(CONFIG_PREEMPT_RT)" $(CONFIG_CC_VERSION_TEXT) "$(LD)"

include/generated/compile.h: FORCE
diff --git a/kernel/Kconfig.preempt b/kernel/Kconfig.preempt
index 60f1bfc3c7b2..1761966c815e 100644
--- a/kernel/Kconfig.preempt
+++ b/kernel/Kconfig.preempt
@@ -1,12 +1,40 @@
# SPDX-License-Identifier: GPL-2.0-only

+
+choice
+ prompt "[Legacy compatibility] Preemption Model"
+
+config PREEMPT_NONE
+ bool "No Forced Preemption (Server)"
+
+config PREEMPT_VOLUNTARY
+ bool "Voluntary Kernel Preemption (Desktop)"
+
+config PREEMPT
+ bool "Preemptible Kernel (Low-Latency Desktop)"
+
+endchoice
+
+config PREEMPT_NONE_STATIC
+ bool
+
+config PREEMPT_VOLUNTARY_STATIC
+ bool
+
+config PREEMPT_STATIC
+ bool
+ select PREEMPTION
+ select UNINLINE_SPIN_UNLOCK if !ARCH_INLINE_SPIN_UNLOCK
+
choice
prompt "Preemption Model"
+ default PREEMPT_BEHAVIOUR if PREEMPT
+ default PREEMPT_VOLUNTARY_BEHAVIOUR if PREEMPT_VOLUNTARY
default PREEMPT_NONE_BEHAVIOUR

config PREEMPT_NONE_BEHAVIOUR
bool "No Forced Preemption (Server)"
- select PREEMPT_NONE if !PREEMPT_DYNAMIC
+ select PREEMPT_NONE_STATIC if !PREEMPT_DYNAMIC
help
This is the traditional Linux preemption model, geared towards
throughput. It will still provide good latencies most of the
@@ -21,7 +49,7 @@ config PREEMPT_NONE_BEHAVIOUR
config PREEMPT_VOLUNTARY_BEHAVIOUR
bool "Voluntary Kernel Preemption (Desktop)"
depends on !ARCH_NO_PREEMPT
- select PREEMPT_VOLUNTARY if !PREEMPT_DYNAMIC
+ select PREEMPT_VOLUNTARY_STATIC if !PREEMPT_DYNAMIC
help
This option reduces the latency of the kernel by adding more
"explicit preemption points" to the kernel code. These new
@@ -40,7 +68,7 @@ config PREEMPT_VOLUNTARY_BEHAVIOUR
config PREEMPT_BEHAVIOUR
bool "Preemptible Kernel (Low-Latency Desktop)"
depends on !ARCH_NO_PREEMPT
- select PREEMPT
+ select PREEMPT_STATIC
help
This option reduces the latency of the kernel by making
all kernel code (that is not executing in a critical section)
@@ -58,7 +86,7 @@ config PREEMPT_BEHAVIOUR

config PREEMPT_RT
bool "Fully Preemptible Kernel (Real-Time)"
- depends on EXPERT && ARCH_SUPPORTS_RT && !PREEMPT_DYNAMIC
+ depends on EXPERT && ARCH_SUPPORTS_RT
select PREEMPTION
help
This option turns the kernel into a real-time kernel by replacing
@@ -75,17 +103,6 @@ config PREEMPT_RT

endchoice

-config PREEMPT_NONE
- bool
-
-config PREEMPT_VOLUNTARY
- bool
-
-config PREEMPT
- bool
- select PREEMPTION
- select UNINLINE_SPIN_UNLOCK if !ARCH_INLINE_SPIN_UNLOCK
-
config PREEMPT_COUNT
bool

@@ -95,8 +112,8 @@ config PREEMPTION

config PREEMPT_DYNAMIC
bool "Preemption behaviour defined on boot"
- depends on HAVE_PREEMPT_DYNAMIC
- select PREEMPT
+ depends on HAVE_PREEMPT_DYNAMIC && !PREEMPT_RT
+ select PREEMPT_STATIC
default y
help
This option allows to define the preemption model on the kernel
diff --git a/kernel/kcsan/kcsan_test.c b/kernel/kcsan/kcsan_test.c
index dc55fd5a36fc..83dd9e6a8235 100644
--- a/kernel/kcsan/kcsan_test.c
+++ b/kernel/kcsan/kcsan_test.c
@@ -1005,13 +1005,13 @@ static const void *nthreads_gen_params(const void *prev, char *desc)
else
nthreads *= 2;

- if (!IS_ENABLED(CONFIG_PREEMPT) || !IS_ENABLED(CONFIG_KCSAN_INTERRUPT_WATCHER)) {
+ if (!IS_ENABLED(CONFIG_PREEMPTION) || !IS_ENABLED(CONFIG_KCSAN_INTERRUPT_WATCHER)) {
/*
* Without any preemption, keep 2 CPUs free for other tasks, one
* of which is the main test case function checking for
* completion or failure.
*/
- const long min_unused_cpus = IS_ENABLED(CONFIG_PREEMPT_NONE) ? 2 : 0;
+ const long min_unused_cpus = IS_ENABLED(CONFIG_PREEMPT_NONE_STATIC) ? 2 : 0;
const long min_required_cpus = 2 + min_unused_cpus;

if (num_online_cpus() < min_required_cpus) {
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index 7896d30d90f7..67413be56f22 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -4271,11 +4271,11 @@ print_trace_header(struct seq_file *m, struct trace_iterator *iter)
entries,
total,
buf->cpu,
-#if defined(CONFIG_PREEMPT_NONE)
+#if defined(CONFIG_PREEMPT_NONE_STATIC)
"server",
-#elif defined(CONFIG_PREEMPT_VOLUNTARY)
+#elif defined(CONFIG_PREEMPT_VOLUNTARY_STATIC)
"desktop",
-#elif defined(CONFIG_PREEMPT)
+#elif defined(CONFIG_PREEMPT_STATIC)
"preempt",
#elif defined(CONFIG_PREEMPT_RT)
"preempt_rt",

2021-11-10 14:49:17

by Valentin Schneider

[permalink] [raw]
Subject: Re: [PATCH] sched: Split preemption model selection between DYNAMIC & !DYNAMIC

On 10/11/21 11:24, Valentin Schneider wrote:
> Following Frederic's _STATIC suggestion, I got to the below. The nice thing
> is if a defconfig picks say PREEMPT_VOLUNTARY and the arch supports
> PREEMPT_DYNAMIC, then it'll pick PREEMPT_VOLUNTARY_BEHAVIOUR. The less nice
> thing is that we end up selecting PREEMPT_STATIC for PREEMPT_DYNAMIC
> kernels, naming's hard... Maybe _BUILD rather than _STATIC?
>

And now without a legacy compatibility layer crud...

---
diff --git a/include/linux/kernel.h b/include/linux/kernel.h
index 2776423a587e..9c7d774ef809 100644
--- a/include/linux/kernel.h
+++ b/include/linux/kernel.h
@@ -88,7 +88,7 @@
struct completion;
struct user;

-#ifdef CONFIG_PREEMPT_VOLUNTARY
+#ifdef CONFIG_PREEMPT_VOLUNTARY_BUILD

extern int __cond_resched(void);
# define might_resched() __cond_resched()
diff --git a/include/linux/vermagic.h b/include/linux/vermagic.h
index 1eaaa93c37bf..f2b71920024c 100644
--- a/include/linux/vermagic.h
+++ b/include/linux/vermagic.h
@@ -15,7 +15,7 @@
#else
#define MODULE_VERMAGIC_SMP ""
#endif
-#ifdef CONFIG_PREEMPT
+#ifdef CONFIG_PREEMPT_BUILD
#define MODULE_VERMAGIC_PREEMPT "preempt "
#elif defined(CONFIG_PREEMPT_RT)
#define MODULE_VERMAGIC_PREEMPT "preempt_rt "
@@ -39,9 +39,9 @@
#define MODULE_RANDSTRUCT_PLUGIN
#endif

-#define VERMAGIC_STRING \
+#define VERMAGIC_STRING \
UTS_RELEASE " " \
- MODULE_VERMAGIC_SMP MODULE_VERMAGIC_PREEMPT \
+ MODULE_VERMAGIC_SMP MODULE_VERMAGIC_PREEMPT \
MODULE_VERMAGIC_MODULE_UNLOAD MODULE_VERMAGIC_MODVERSIONS \
MODULE_ARCH_VERMAGIC \
MODULE_RANDSTRUCT_PLUGIN
diff --git a/init/Makefile b/init/Makefile
index 2846113677ee..04eeee12c076 100644
--- a/init/Makefile
+++ b/init/Makefile
@@ -30,7 +30,7 @@ $(obj)/version.o: include/generated/compile.h
quiet_cmd_compile.h = CHK $@
cmd_compile.h = \
$(CONFIG_SHELL) $(srctree)/scripts/mkcompile_h $@ \
- "$(UTS_MACHINE)" "$(CONFIG_SMP)" "$(CONFIG_PREEMPT)" \
+ "$(UTS_MACHINE)" "$(CONFIG_SMP)" "$(CONFIG_PREEMPT_BUILD)" \
"$(CONFIG_PREEMPT_RT)" $(CONFIG_CC_VERSION_TEXT) "$(LD)"

include/generated/compile.h: FORCE
diff --git a/kernel/Kconfig.preempt b/kernel/Kconfig.preempt
index 60f1bfc3c7b2..ce77f0265660 100644
--- a/kernel/Kconfig.preempt
+++ b/kernel/Kconfig.preempt
@@ -1,12 +1,23 @@
# SPDX-License-Identifier: GPL-2.0-only

+config PREEMPT_NONE_BUILD
+ bool
+
+config PREEMPT_VOLUNTARY_BUILD
+ bool
+
+config PREEMPT_BUILD
+ bool
+ select PREEMPTION
+ select UNINLINE_SPIN_UNLOCK if !ARCH_INLINE_SPIN_UNLOCK
+
choice
prompt "Preemption Model"
- default PREEMPT_NONE_BEHAVIOUR
+ default PREEMPT_NONE

-config PREEMPT_NONE_BEHAVIOUR
+config PREEMPT_NONE
bool "No Forced Preemption (Server)"
- select PREEMPT_NONE if !PREEMPT_DYNAMIC
+ select PREEMPT_NONE_BUILD if !PREEMPT_DYNAMIC
help
This is the traditional Linux preemption model, geared towards
throughput. It will still provide good latencies most of the
@@ -18,10 +29,10 @@ config PREEMPT_NONE_BEHAVIOUR
raw processing power of the kernel, irrespective of scheduling
latencies.

-config PREEMPT_VOLUNTARY_BEHAVIOUR
+config PREEMPT_VOLUNTARY
bool "Voluntary Kernel Preemption (Desktop)"
depends on !ARCH_NO_PREEMPT
- select PREEMPT_VOLUNTARY if !PREEMPT_DYNAMIC
+ select PREEMPT_VOLUNTARY_BUILD if !PREEMPT_DYNAMIC
help
This option reduces the latency of the kernel by adding more
"explicit preemption points" to the kernel code. These new
@@ -37,10 +48,10 @@ config PREEMPT_VOLUNTARY_BEHAVIOUR

Select this if you are building a kernel for a desktop system.

-config PREEMPT_BEHAVIOUR
+config PREEMPT
bool "Preemptible Kernel (Low-Latency Desktop)"
depends on !ARCH_NO_PREEMPT
- select PREEMPT
+ select PREEMPT_BUILD
help
This option reduces the latency of the kernel by making
all kernel code (that is not executing in a critical section)
@@ -58,7 +69,7 @@ config PREEMPT_BEHAVIOUR

config PREEMPT_RT
bool "Fully Preemptible Kernel (Real-Time)"
- depends on EXPERT && ARCH_SUPPORTS_RT && !PREEMPT_DYNAMIC
+ depends on EXPERT && ARCH_SUPPORTS_RT
select PREEMPTION
help
This option turns the kernel into a real-time kernel by replacing
@@ -75,17 +86,6 @@ config PREEMPT_RT

endchoice

-config PREEMPT_NONE
- bool
-
-config PREEMPT_VOLUNTARY
- bool
-
-config PREEMPT
- bool
- select PREEMPTION
- select UNINLINE_SPIN_UNLOCK if !ARCH_INLINE_SPIN_UNLOCK
-
config PREEMPT_COUNT
bool

@@ -95,8 +95,8 @@ config PREEMPTION

config PREEMPT_DYNAMIC
bool "Preemption behaviour defined on boot"
- depends on HAVE_PREEMPT_DYNAMIC
- select PREEMPT
+ depends on HAVE_PREEMPT_DYNAMIC && !PREEMPT_RT
+ select PREEMPT_BUILD
default y
help
This option allows to define the preemption model on the kernel
diff --git a/kernel/kcsan/kcsan_test.c b/kernel/kcsan/kcsan_test.c
index dc55fd5a36fc..845b17cf7811 100644
--- a/kernel/kcsan/kcsan_test.c
+++ b/kernel/kcsan/kcsan_test.c
@@ -1005,13 +1005,13 @@ static const void *nthreads_gen_params(const void *prev, char *desc)
else
nthreads *= 2;

- if (!IS_ENABLED(CONFIG_PREEMPT) || !IS_ENABLED(CONFIG_KCSAN_INTERRUPT_WATCHER)) {
+ if (!IS_ENABLED(CONFIG_PREEMPTION) || !IS_ENABLED(CONFIG_KCSAN_INTERRUPT_WATCHER)) {
/*
* Without any preemption, keep 2 CPUs free for other tasks, one
* of which is the main test case function checking for
* completion or failure.
*/
- const long min_unused_cpus = IS_ENABLED(CONFIG_PREEMPT_NONE) ? 2 : 0;
+ const long min_unused_cpus = IS_ENABLED(CONFIG_PREEMPT_NONE_BUILD) ? 2 : 0;
const long min_required_cpus = 2 + min_unused_cpus;

if (num_online_cpus() < min_required_cpus) {
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index 7896d30d90f7..3b3226ffbafa 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -4271,11 +4271,11 @@ print_trace_header(struct seq_file *m, struct trace_iterator *iter)
entries,
total,
buf->cpu,
-#if defined(CONFIG_PREEMPT_NONE)
+#if defined(CONFIG_PREEMPT_NONE_BUILD)
"server",
-#elif defined(CONFIG_PREEMPT_VOLUNTARY)
+#elif defined(CONFIG_PREEMPT_VOLUNTARY_BUILD)
"desktop",
-#elif defined(CONFIG_PREEMPT)
+#elif defined(CONFIG_PREEMPT_BUILD)
"preempt",
#elif defined(CONFIG_PREEMPT_RT)
"preempt_rt",

2021-11-10 15:07:43

by Marco Elver

[permalink] [raw]
Subject: Re: [PATCH] sched: Split preemption model selection between DYNAMIC & !DYNAMIC

On Wed, 10 Nov 2021 at 15:45, Valentin Schneider
<[email protected]> wrote:
>
> On 10/11/21 11:24, Valentin Schneider wrote:
> > Following Frederic's _STATIC suggestion, I got to the below. The nice thing
> > is if a defconfig picks say PREEMPT_VOLUNTARY and the arch supports
> > PREEMPT_DYNAMIC, then it'll pick PREEMPT_VOLUNTARY_BEHAVIOUR. The less nice
> > thing is that we end up selecting PREEMPT_STATIC for PREEMPT_DYNAMIC
> > kernels, naming's hard... Maybe _BUILD rather than _STATIC?
> >
>
> And now without a legacy compatibility layer crud...

If this works for everyone else, then I think this is the least
intrusive and hopefully won't require telling everyone about this
config change.

For the final version, it seems to be missing s/_BEHAVIOUR// in
kernel/sched/core.c.

And the kcsan_test.c change can be reverted, I think it's broken
either way now, because it wants the dynamic state (the test still
works in most configs, so fixing this isn't super urgent). Similarly
for kernel/trace/trace.c, which appears mainly cosmetic. So I'd defer
changing these until there are real helpers to get the true preemption
behaviour of the running system.

Thanks,
-- Marco

> ---
> diff --git a/include/linux/kernel.h b/include/linux/kernel.h
> index 2776423a587e..9c7d774ef809 100644
> --- a/include/linux/kernel.h
> +++ b/include/linux/kernel.h
> @@ -88,7 +88,7 @@
> struct completion;
> struct user;
>
> -#ifdef CONFIG_PREEMPT_VOLUNTARY
> +#ifdef CONFIG_PREEMPT_VOLUNTARY_BUILD
>
> extern int __cond_resched(void);
> # define might_resched() __cond_resched()
> diff --git a/include/linux/vermagic.h b/include/linux/vermagic.h
> index 1eaaa93c37bf..f2b71920024c 100644
> --- a/include/linux/vermagic.h
> +++ b/include/linux/vermagic.h
> @@ -15,7 +15,7 @@
> #else
> #define MODULE_VERMAGIC_SMP ""
> #endif
> -#ifdef CONFIG_PREEMPT
> +#ifdef CONFIG_PREEMPT_BUILD
> #define MODULE_VERMAGIC_PREEMPT "preempt "
> #elif defined(CONFIG_PREEMPT_RT)
> #define MODULE_VERMAGIC_PREEMPT "preempt_rt "
> @@ -39,9 +39,9 @@
> #define MODULE_RANDSTRUCT_PLUGIN
> #endif
>
> -#define VERMAGIC_STRING \
> +#define VERMAGIC_STRING \
> UTS_RELEASE " " \
> - MODULE_VERMAGIC_SMP MODULE_VERMAGIC_PREEMPT \
> + MODULE_VERMAGIC_SMP MODULE_VERMAGIC_PREEMPT \
> MODULE_VERMAGIC_MODULE_UNLOAD MODULE_VERMAGIC_MODVERSIONS \
> MODULE_ARCH_VERMAGIC \
> MODULE_RANDSTRUCT_PLUGIN
> diff --git a/init/Makefile b/init/Makefile
> index 2846113677ee..04eeee12c076 100644
> --- a/init/Makefile
> +++ b/init/Makefile
> @@ -30,7 +30,7 @@ $(obj)/version.o: include/generated/compile.h
> quiet_cmd_compile.h = CHK $@
> cmd_compile.h = \
> $(CONFIG_SHELL) $(srctree)/scripts/mkcompile_h $@ \
> - "$(UTS_MACHINE)" "$(CONFIG_SMP)" "$(CONFIG_PREEMPT)" \
> + "$(UTS_MACHINE)" "$(CONFIG_SMP)" "$(CONFIG_PREEMPT_BUILD)" \
> "$(CONFIG_PREEMPT_RT)" $(CONFIG_CC_VERSION_TEXT) "$(LD)"
>
> include/generated/compile.h: FORCE
> diff --git a/kernel/Kconfig.preempt b/kernel/Kconfig.preempt
> index 60f1bfc3c7b2..ce77f0265660 100644
> --- a/kernel/Kconfig.preempt
> +++ b/kernel/Kconfig.preempt
> @@ -1,12 +1,23 @@
> # SPDX-License-Identifier: GPL-2.0-only
>
> +config PREEMPT_NONE_BUILD
> + bool
> +
> +config PREEMPT_VOLUNTARY_BUILD
> + bool
> +
> +config PREEMPT_BUILD
> + bool
> + select PREEMPTION
> + select UNINLINE_SPIN_UNLOCK if !ARCH_INLINE_SPIN_UNLOCK
> +
> choice
> prompt "Preemption Model"
> - default PREEMPT_NONE_BEHAVIOUR
> + default PREEMPT_NONE
>
> -config PREEMPT_NONE_BEHAVIOUR
> +config PREEMPT_NONE
> bool "No Forced Preemption (Server)"
> - select PREEMPT_NONE if !PREEMPT_DYNAMIC
> + select PREEMPT_NONE_BUILD if !PREEMPT_DYNAMIC
> help
> This is the traditional Linux preemption model, geared towards
> throughput. It will still provide good latencies most of the
> @@ -18,10 +29,10 @@ config PREEMPT_NONE_BEHAVIOUR
> raw processing power of the kernel, irrespective of scheduling
> latencies.
>
> -config PREEMPT_VOLUNTARY_BEHAVIOUR
> +config PREEMPT_VOLUNTARY
> bool "Voluntary Kernel Preemption (Desktop)"
> depends on !ARCH_NO_PREEMPT
> - select PREEMPT_VOLUNTARY if !PREEMPT_DYNAMIC
> + select PREEMPT_VOLUNTARY_BUILD if !PREEMPT_DYNAMIC
> help
> This option reduces the latency of the kernel by adding more
> "explicit preemption points" to the kernel code. These new
> @@ -37,10 +48,10 @@ config PREEMPT_VOLUNTARY_BEHAVIOUR
>
> Select this if you are building a kernel for a desktop system.
>
> -config PREEMPT_BEHAVIOUR
> +config PREEMPT
> bool "Preemptible Kernel (Low-Latency Desktop)"
> depends on !ARCH_NO_PREEMPT
> - select PREEMPT
> + select PREEMPT_BUILD
> help
> This option reduces the latency of the kernel by making
> all kernel code (that is not executing in a critical section)
> @@ -58,7 +69,7 @@ config PREEMPT_BEHAVIOUR
>
> config PREEMPT_RT
> bool "Fully Preemptible Kernel (Real-Time)"
> - depends on EXPERT && ARCH_SUPPORTS_RT && !PREEMPT_DYNAMIC
> + depends on EXPERT && ARCH_SUPPORTS_RT
> select PREEMPTION
> help
> This option turns the kernel into a real-time kernel by replacing
> @@ -75,17 +86,6 @@ config PREEMPT_RT
>
> endchoice
>
> -config PREEMPT_NONE
> - bool
> -
> -config PREEMPT_VOLUNTARY
> - bool
> -
> -config PREEMPT
> - bool
> - select PREEMPTION
> - select UNINLINE_SPIN_UNLOCK if !ARCH_INLINE_SPIN_UNLOCK
> -
> config PREEMPT_COUNT
> bool
>
> @@ -95,8 +95,8 @@ config PREEMPTION
>
> config PREEMPT_DYNAMIC
> bool "Preemption behaviour defined on boot"
> - depends on HAVE_PREEMPT_DYNAMIC
> - select PREEMPT
> + depends on HAVE_PREEMPT_DYNAMIC && !PREEMPT_RT
> + select PREEMPT_BUILD
> default y
> help
> This option allows to define the preemption model on the kernel
> diff --git a/kernel/kcsan/kcsan_test.c b/kernel/kcsan/kcsan_test.c
> index dc55fd5a36fc..845b17cf7811 100644
> --- a/kernel/kcsan/kcsan_test.c
> +++ b/kernel/kcsan/kcsan_test.c
> @@ -1005,13 +1005,13 @@ static const void *nthreads_gen_params(const void *prev, char *desc)
> else
> nthreads *= 2;
>
> - if (!IS_ENABLED(CONFIG_PREEMPT) || !IS_ENABLED(CONFIG_KCSAN_INTERRUPT_WATCHER)) {
> + if (!IS_ENABLED(CONFIG_PREEMPTION) || !IS_ENABLED(CONFIG_KCSAN_INTERRUPT_WATCHER)) {
> /*
> * Without any preemption, keep 2 CPUs free for other tasks, one
> * of which is the main test case function checking for
> * completion or failure.
> */
> - const long min_unused_cpus = IS_ENABLED(CONFIG_PREEMPT_NONE) ? 2 : 0;
> + const long min_unused_cpus = IS_ENABLED(CONFIG_PREEMPT_NONE_BUILD) ? 2 : 0;
> const long min_required_cpus = 2 + min_unused_cpus;
>
> if (num_online_cpus() < min_required_cpus) {
> diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
> index 7896d30d90f7..3b3226ffbafa 100644
> --- a/kernel/trace/trace.c
> +++ b/kernel/trace/trace.c
> @@ -4271,11 +4271,11 @@ print_trace_header(struct seq_file *m, struct trace_iterator *iter)
> entries,
> total,
> buf->cpu,
> -#if defined(CONFIG_PREEMPT_NONE)
> +#if defined(CONFIG_PREEMPT_NONE_BUILD)
> "server",
> -#elif defined(CONFIG_PREEMPT_VOLUNTARY)
> +#elif defined(CONFIG_PREEMPT_VOLUNTARY_BUILD)
> "desktop",
> -#elif defined(CONFIG_PREEMPT)
> +#elif defined(CONFIG_PREEMPT_BUILD)
> "preempt",
> #elif defined(CONFIG_PREEMPT_RT)
> "preempt_rt",

2021-11-10 15:19:35

by Marco Elver

[permalink] [raw]
Subject: Re: [PATCH] sched: Split preemption model selection between DYNAMIC & !DYNAMIC

On Wed, 10 Nov 2021 at 16:03, Marco Elver <[email protected]> wrote:
[...]
> And the kcsan_test.c change can be reverted, I think it's broken
> either way now, because it wants the dynamic state (the test still
> works in most configs, so fixing this isn't super urgent). Similarly
> for kernel/trace/trace.c, which appears mainly cosmetic. So I'd defer
> changing these until there are real helpers to get the true preemption
> behaviour of the running system.

I just realised that in both these cases the original version is less
broken than the below, because the non-suffixed CONFIG_PREEMPT*
variables now denote the dynamic default. So if someone doesn't use
'preempt=' at boot, the original code will actually do the right
thing.

> > diff --git a/kernel/kcsan/kcsan_test.c b/kernel/kcsan/kcsan_test.c
> > index dc55fd5a36fc..845b17cf7811 100644
> > --- a/kernel/kcsan/kcsan_test.c
> > +++ b/kernel/kcsan/kcsan_test.c
> > @@ -1005,13 +1005,13 @@ static const void *nthreads_gen_params(const void *prev, char *desc)
> > else
> > nthreads *= 2;
> >
> > - if (!IS_ENABLED(CONFIG_PREEMPT) || !IS_ENABLED(CONFIG_KCSAN_INTERRUPT_WATCHER)) {
> > + if (!IS_ENABLED(CONFIG_PREEMPTION) || !IS_ENABLED(CONFIG_KCSAN_INTERRUPT_WATCHER)) {
> > /*
> > * Without any preemption, keep 2 CPUs free for other tasks, one
> > * of which is the main test case function checking for
> > * completion or failure.
> > */
> > - const long min_unused_cpus = IS_ENABLED(CONFIG_PREEMPT_NONE) ? 2 : 0;
> > + const long min_unused_cpus = IS_ENABLED(CONFIG_PREEMPT_NONE_BUILD) ? 2 : 0;
> > const long min_required_cpus = 2 + min_unused_cpus;
> >
> > if (num_online_cpus() < min_required_cpus) {
> > diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
> > index 7896d30d90f7..3b3226ffbafa 100644
> > --- a/kernel/trace/trace.c
> > +++ b/kernel/trace/trace.c
> > @@ -4271,11 +4271,11 @@ print_trace_header(struct seq_file *m, struct trace_iterator *iter)
> > entries,
> > total,
> > buf->cpu,
> > -#if defined(CONFIG_PREEMPT_NONE)
> > +#if defined(CONFIG_PREEMPT_NONE_BUILD)
> > "server",
> > -#elif defined(CONFIG_PREEMPT_VOLUNTARY)
> > +#elif defined(CONFIG_PREEMPT_VOLUNTARY_BUILD)
> > "desktop",
> > -#elif defined(CONFIG_PREEMPT)
> > +#elif defined(CONFIG_PREEMPT_BUILD)
> > "preempt",
> > #elif defined(CONFIG_PREEMPT_RT)
> > "preempt_rt",

2021-11-10 16:49:48

by Frederic Weisbecker

[permalink] [raw]
Subject: Re: [PATCH] sched: Split preemption model selection between DYNAMIC & !DYNAMIC

On Wed, Nov 10, 2021 at 02:45:45PM +0000, Valentin Schneider wrote:
> On 10/11/21 11:24, Valentin Schneider wrote:
> > Following Frederic's _STATIC suggestion, I got to the below. The nice thing
> > is if a defconfig picks say PREEMPT_VOLUNTARY and the arch supports
> > PREEMPT_DYNAMIC, then it'll pick PREEMPT_VOLUNTARY_BEHAVIOUR. The less nice
> > thing is that we end up selecting PREEMPT_STATIC for PREEMPT_DYNAMIC
> > kernels, naming's hard... Maybe _BUILD rather than _STATIC?
> >
>
> And now without a legacy compatibility layer crud...
>
> ---
> diff --git a/include/linux/kernel.h b/include/linux/kernel.h
> index 2776423a587e..9c7d774ef809 100644
> --- a/include/linux/kernel.h
> +++ b/include/linux/kernel.h
> @@ -88,7 +88,7 @@
> struct completion;
> struct user;
>
> -#ifdef CONFIG_PREEMPT_VOLUNTARY
> +#ifdef CONFIG_PREEMPT_VOLUNTARY_BUILD
>
> extern int __cond_resched(void);
> # define might_resched() __cond_resched()
> diff --git a/include/linux/vermagic.h b/include/linux/vermagic.h
> index 1eaaa93c37bf..f2b71920024c 100644
> --- a/include/linux/vermagic.h
> +++ b/include/linux/vermagic.h
> @@ -15,7 +15,7 @@
> #else
> #define MODULE_VERMAGIC_SMP ""
> #endif
> -#ifdef CONFIG_PREEMPT
> +#ifdef CONFIG_PREEMPT_BUILD
> #define MODULE_VERMAGIC_PREEMPT "preempt "
> #elif defined(CONFIG_PREEMPT_RT)
> #define MODULE_VERMAGIC_PREEMPT "preempt_rt "
> @@ -39,9 +39,9 @@
> #define MODULE_RANDSTRUCT_PLUGIN
> #endif
>
> -#define VERMAGIC_STRING \
> +#define VERMAGIC_STRING \
> UTS_RELEASE " " \
> - MODULE_VERMAGIC_SMP MODULE_VERMAGIC_PREEMPT \
> + MODULE_VERMAGIC_SMP MODULE_VERMAGIC_PREEMPT \
> MODULE_VERMAGIC_MODULE_UNLOAD MODULE_VERMAGIC_MODVERSIONS \
> MODULE_ARCH_VERMAGIC \
> MODULE_RANDSTRUCT_PLUGIN
> diff --git a/init/Makefile b/init/Makefile
> index 2846113677ee..04eeee12c076 100644
> --- a/init/Makefile
> +++ b/init/Makefile
> @@ -30,7 +30,7 @@ $(obj)/version.o: include/generated/compile.h
> quiet_cmd_compile.h = CHK $@
> cmd_compile.h = \
> $(CONFIG_SHELL) $(srctree)/scripts/mkcompile_h $@ \
> - "$(UTS_MACHINE)" "$(CONFIG_SMP)" "$(CONFIG_PREEMPT)" \
> + "$(UTS_MACHINE)" "$(CONFIG_SMP)" "$(CONFIG_PREEMPT_BUILD)" \
> "$(CONFIG_PREEMPT_RT)" $(CONFIG_CC_VERSION_TEXT) "$(LD)"
>
> include/generated/compile.h: FORCE
> diff --git a/kernel/Kconfig.preempt b/kernel/Kconfig.preempt
> index 60f1bfc3c7b2..ce77f0265660 100644
> --- a/kernel/Kconfig.preempt
> +++ b/kernel/Kconfig.preempt
> @@ -1,12 +1,23 @@
> # SPDX-License-Identifier: GPL-2.0-only
>
> +config PREEMPT_NONE_BUILD
> + bool
> +
> +config PREEMPT_VOLUNTARY_BUILD
> + bool
> +
> +config PREEMPT_BUILD
> + bool
> + select PREEMPTION
> + select UNINLINE_SPIN_UNLOCK if !ARCH_INLINE_SPIN_UNLOCK
> +
> choice
> prompt "Preemption Model"
> - default PREEMPT_NONE_BEHAVIOUR
> + default PREEMPT_NONE
>
> -config PREEMPT_NONE_BEHAVIOUR
> +config PREEMPT_NONE
> bool "No Forced Preemption (Server)"
> - select PREEMPT_NONE if !PREEMPT_DYNAMIC
> + select PREEMPT_NONE_BUILD if !PREEMPT_DYNAMIC
> help
> This is the traditional Linux preemption model, geared towards
> throughput. It will still provide good latencies most of the
> @@ -18,10 +29,10 @@ config PREEMPT_NONE_BEHAVIOUR
> raw processing power of the kernel, irrespective of scheduling
> latencies.
>
> -config PREEMPT_VOLUNTARY_BEHAVIOUR
> +config PREEMPT_VOLUNTARY
> bool "Voluntary Kernel Preemption (Desktop)"
> depends on !ARCH_NO_PREEMPT
> - select PREEMPT_VOLUNTARY if !PREEMPT_DYNAMIC
> + select PREEMPT_VOLUNTARY_BUILD if !PREEMPT_DYNAMIC
> help
> This option reduces the latency of the kernel by adding more
> "explicit preemption points" to the kernel code. These new
> @@ -37,10 +48,10 @@ config PREEMPT_VOLUNTARY_BEHAVIOUR
>
> Select this if you are building a kernel for a desktop system.
>
> -config PREEMPT_BEHAVIOUR
> +config PREEMPT
> bool "Preemptible Kernel (Low-Latency Desktop)"
> depends on !ARCH_NO_PREEMPT
> - select PREEMPT
> + select PREEMPT_BUILD
> help
> This option reduces the latency of the kernel by making
> all kernel code (that is not executing in a critical section)

Looks nicer that way!

But don't forget to also update preempt_dynamic_init().

2021-11-10 16:56:43

by Frederic Weisbecker

[permalink] [raw]
Subject: Re: [PATCH] sched: Split preemption model selection between DYNAMIC & !DYNAMIC

On Wed, Nov 10, 2021 at 02:45:45PM +0000, Valentin Schneider wrote:
> diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
> index 7896d30d90f7..3b3226ffbafa 100644
> --- a/kernel/trace/trace.c
> +++ b/kernel/trace/trace.c
> @@ -4271,11 +4271,11 @@ print_trace_header(struct seq_file *m, struct trace_iterator *iter)
> entries,
> total,
> buf->cpu,
> -#if defined(CONFIG_PREEMPT_NONE)
> +#if defined(CONFIG_PREEMPT_NONE_BUILD)
> "server",
> -#elif defined(CONFIG_PREEMPT_VOLUNTARY)
> +#elif defined(CONFIG_PREEMPT_VOLUNTARY_BUILD)
> "desktop",
> -#elif defined(CONFIG_PREEMPT)
> +#elif defined(CONFIG_PREEMPT_BUILD)
> "preempt",
> #elif defined(CONFIG_PREEMPT_RT)
> "preempt_rt",

I also found those:

arch/powerpc/kernel/interrupt.c: if (IS_ENABLED(CONFIG_PREEMPT)) {
arch/powerpc/kernel/traps.c: IS_ENABLED(CONFIG_PREEMPT) ? " PREEMPT" : "",