2024-06-11 13:34:03

by Masami Hiramatsu

[permalink] [raw]
Subject: [PATCH v3 0/3] tracing: Fix some selftest issues

Hi,

Here is v3 of a series of some fixes/cleanups for the test modules and
boot time selftest of kprobe events. The previous version is here;

https://lore.kernel.org/all/171805478534.52471.6269290579314514778.stgit@devnote2/

In this version, I updated the 2nd patch to integrate WARN_ON_ONCE() and
pr_warn() instead of removing WARN_ONCE() because this warning messages
are needed to ktest to handle errors.

Thank you,

---

Masami Hiramatsu (Google) (3):
tracing: Build event generation tests only as modules
tracing/kprobe: Integrate test warnings into WARN_ONCE
tracing/kprobe: Remove cleanup code unrelated to selftest


kernel/trace/Kconfig | 4 ++-
kernel/trace/trace_kprobe.c | 54 ++++++++++++++-----------------------------
2 files changed, 19 insertions(+), 39 deletions(-)

--
Masami Hiramatsu (Google) <[email protected]>


2024-06-11 13:34:40

by Masami Hiramatsu

[permalink] [raw]
Subject: [PATCH v3 3/3] tracing/kprobe: Remove cleanup code unrelated to selftest

From: Masami Hiramatsu (Google) <[email protected]>

This cleanup all kprobe events code is not related to the selftest
itself, and it can fail by the reason unrelated to this test.
If the test is successful, the generated events are cleaned up.
And if not, we cannot guarantee that the kprobe events will work
correctly. So, anyway, there is no need to clean it up.

Signed-off-by: Masami Hiramatsu (Google) <[email protected]>
---
kernel/trace/trace_kprobe.c | 4 ----
1 file changed, 4 deletions(-)

diff --git a/kernel/trace/trace_kprobe.c b/kernel/trace/trace_kprobe.c
index 8c5816c04bd2..7fd0f8576e4c 100644
--- a/kernel/trace/trace_kprobe.c
+++ b/kernel/trace/trace_kprobe.c
@@ -2114,10 +2114,6 @@ static __init int kprobe_trace_self_tests_init(void)


end:
- ret = dyn_events_release_all(&trace_kprobe_ops);
- if (WARN_ONCE(ret, "error on cleaning up probes."))
- warn++;
-
/*
* Wait for the optimizer work to finish. Otherwise it might fiddle
* with probes in already freed __init text.


2024-06-11 13:38:01

by Masami Hiramatsu

[permalink] [raw]
Subject: [PATCH v3 1/3] tracing: Build event generation tests only as modules

From: Masami Hiramatsu (Google) <[email protected]>

The kprobes and synth event generation test modules add events and lock
(get a reference) those event file reference in module init function,
and unlock and delete it in module exit function. This is because those
are designed for playing as modules.

If we make those modules as built-in, those events are left locked in the
kernel, and never be removed. This causes kprobe event self-test failure
as below.

[ 97.349708] ------------[ cut here ]------------
[ 97.353453] WARNING: CPU: 3 PID: 1 at kernel/trace/trace_kprobe.c:2133 kprobe_trace_self_tests_init+0x3f1/0x480
[ 97.357106] Modules linked in:
[ 97.358488] CPU: 3 PID: 1 Comm: swapper/0 Not tainted 6.9.0-g699646734ab5-dirty #14
[ 97.361556] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
[ 97.363880] RIP: 0010:kprobe_trace_self_tests_init+0x3f1/0x480
[ 97.365538] Code: a8 24 08 82 e9 ae fd ff ff 90 0f 0b 90 48 c7 c7 e5 aa 0b 82 e9 ee fc ff ff 90 0f 0b 90 48 c7 c7 2d 61 06 82 e9 8e fd ff ff 90 <0f> 0b 90 48 c7 c7 33 0b 0c 82 89 c6 e8 6e 03 1f ff 41 ff c7 e9 90
[ 97.370429] RSP: 0000:ffffc90000013b50 EFLAGS: 00010286
[ 97.371852] RAX: 00000000fffffff0 RBX: ffff888005919c00 RCX: 0000000000000000
[ 97.373829] RDX: ffff888003f40000 RSI: ffffffff8236a598 RDI: ffff888003f40a68
[ 97.375715] RBP: 0000000000000000 R08: 0000000000000001 R09: 0000000000000000
[ 97.377675] R10: ffffffff811c9ae5 R11: ffffffff8120c4e0 R12: 0000000000000000
[ 97.379591] R13: 0000000000000001 R14: 0000000000000015 R15: 0000000000000000
[ 97.381536] FS: 0000000000000000(0000) GS:ffff88807dcc0000(0000) knlGS:0000000000000000
[ 97.383813] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 97.385449] CR2: 0000000000000000 CR3: 0000000002244000 CR4: 00000000000006b0
[ 97.387347] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 97.389277] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 97.391196] Call Trace:
[ 97.391967] <TASK>
[ 97.392647] ? __warn+0xcc/0x180
[ 97.393640] ? kprobe_trace_self_tests_init+0x3f1/0x480
[ 97.395181] ? report_bug+0xbd/0x150
[ 97.396234] ? handle_bug+0x3e/0x60
[ 97.397311] ? exc_invalid_op+0x1a/0x50
[ 97.398434] ? asm_exc_invalid_op+0x1a/0x20
[ 97.399652] ? trace_kprobe_is_busy+0x20/0x20
[ 97.400904] ? tracing_reset_all_online_cpus+0x15/0x90
[ 97.402304] ? kprobe_trace_self_tests_init+0x3f1/0x480
[ 97.403773] ? init_kprobe_trace+0x50/0x50
[ 97.404972] do_one_initcall+0x112/0x240
[ 97.406113] do_initcall_level+0x95/0xb0
[ 97.407286] ? kernel_init+0x1a/0x1a0
[ 97.408401] do_initcalls+0x3f/0x70
[ 97.409452] kernel_init_freeable+0x16f/0x1e0
[ 97.410662] ? rest_init+0x1f0/0x1f0
[ 97.411738] kernel_init+0x1a/0x1a0
[ 97.412788] ret_from_fork+0x39/0x50
[ 97.413817] ? rest_init+0x1f0/0x1f0
[ 97.414844] ret_from_fork_asm+0x11/0x20
[ 97.416285] </TASK>
[ 97.417134] irq event stamp: 13437323
[ 97.418376] hardirqs last enabled at (13437337): [<ffffffff8110bc0c>] console_unlock+0x11c/0x150
[ 97.421285] hardirqs last disabled at (13437370): [<ffffffff8110bbf1>] console_unlock+0x101/0x150
[ 97.423838] softirqs last enabled at (13437366): [<ffffffff8108e17f>] handle_softirqs+0x23f/0x2a0
[ 97.426450] softirqs last disabled at (13437393): [<ffffffff8108e346>] __irq_exit_rcu+0x66/0xd0
[ 97.428850] ---[ end trace 0000000000000000 ]---

And also, since we can not cleanup dynamic_event file, ftracetest are
failed too.

To avoid these issues, build these tests only as modules.

Fixes: 9fe41efaca08 ("tracing: Add synth event generation test module")
Fixes: 64836248dda2 ("tracing: Add kprobe event command generation test module")
Signed-off-by: Masami Hiramatsu (Google) <[email protected]>
Reviewed-by: Steven Rostedt (Google) <[email protected]>
---
kernel/trace/Kconfig | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index 166ad5444eea..721c3b221048 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -1136,7 +1136,7 @@ config PREEMPTIRQ_DELAY_TEST

config SYNTH_EVENT_GEN_TEST
tristate "Test module for in-kernel synthetic event generation"
- depends on SYNTH_EVENTS
+ depends on SYNTH_EVENTS && m
help
This option creates a test module to check the base
functionality of in-kernel synthetic event definition and
@@ -1149,7 +1149,7 @@ config SYNTH_EVENT_GEN_TEST

config KPROBE_EVENT_GEN_TEST
tristate "Test module for in-kernel kprobe event generation"
- depends on KPROBE_EVENTS
+ depends on KPROBE_EVENTS && m
help
This option creates a test module to check the base
functionality of in-kernel kprobe event definition.


2024-06-11 14:24:53

by Steven Rostedt

[permalink] [raw]
Subject: Re: [PATCH v3 3/3] tracing/kprobe: Remove cleanup code unrelated to selftest

On Tue, 11 Jun 2024 22:30:56 +0900
"Masami Hiramatsu (Google)" <[email protected]> wrote:

> From: Masami Hiramatsu (Google) <[email protected]>
>
> This cleanup all kprobe events code is not related to the selftest
> itself, and it can fail by the reason unrelated to this test.
> If the test is successful, the generated events are cleaned up.
> And if not, we cannot guarantee that the kprobe events will work
> correctly. So, anyway, there is no need to clean it up.
>
> Signed-off-by: Masami Hiramatsu (Google) <[email protected]>

Reviewed-by: Steven Rostedt (Google) <[email protected]>

-- Steve

> ---
> kernel/trace/trace_kprobe.c | 4 ----
> 1 file changed, 4 deletions(-)
>
> diff --git a/kernel/trace/trace_kprobe.c b/kernel/trace/trace_kprobe.c
> index 8c5816c04bd2..7fd0f8576e4c 100644
> --- a/kernel/trace/trace_kprobe.c
> +++ b/kernel/trace/trace_kprobe.c
> @@ -2114,10 +2114,6 @@ static __init int kprobe_trace_self_tests_init(void)
>
>
> end:
> - ret = dyn_events_release_all(&trace_kprobe_ops);
> - if (WARN_ONCE(ret, "error on cleaning up probes."))
> - warn++;
> -
> /*
> * Wait for the optimizer work to finish. Otherwise it might fiddle
> * with probes in already freed __init text.


2024-06-11 23:11:15

by Masami Hiramatsu

[permalink] [raw]
Subject: Re: [PATCH v3 3/3] tracing/kprobe: Remove cleanup code unrelated to selftest

On Tue, 11 Jun 2024 10:25:00 -0400
Steven Rostedt <[email protected]> wrote:

> On Tue, 11 Jun 2024 22:30:56 +0900
> "Masami Hiramatsu (Google)" <[email protected]> wrote:
>
> > From: Masami Hiramatsu (Google) <[email protected]>
> >
> > This cleanup all kprobe events code is not related to the selftest
> > itself, and it can fail by the reason unrelated to this test.
> > If the test is successful, the generated events are cleaned up.
> > And if not, we cannot guarantee that the kprobe events will work
> > correctly. So, anyway, there is no need to clean it up.
> >
> > Signed-off-by: Masami Hiramatsu (Google) <[email protected]>
>
> Reviewed-by: Steven Rostedt (Google) <[email protected]>

Thanks for review!

>
> -- Steve
>
> > ---
> > kernel/trace/trace_kprobe.c | 4 ----
> > 1 file changed, 4 deletions(-)
> >
> > diff --git a/kernel/trace/trace_kprobe.c b/kernel/trace/trace_kprobe.c
> > index 8c5816c04bd2..7fd0f8576e4c 100644
> > --- a/kernel/trace/trace_kprobe.c
> > +++ b/kernel/trace/trace_kprobe.c
> > @@ -2114,10 +2114,6 @@ static __init int kprobe_trace_self_tests_init(void)
> >
> >
> > end:
> > - ret = dyn_events_release_all(&trace_kprobe_ops);
> > - if (WARN_ONCE(ret, "error on cleaning up probes."))
> > - warn++;
> > -
> > /*
> > * Wait for the optimizer work to finish. Otherwise it might fiddle
> > * with probes in already freed __init text.
>
>


--
Masami Hiramatsu (Google) <[email protected]>