While running bpf selftests it's possible to get following fault:
general protection fault, probably for non-canonical address \
0x6b6b6b6b6b6b6b6b: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC NOPTI
...
Call Trace:
<TASK>
fprobe_handler+0xc1/0x270
? __pfx_bpf_testmod_init+0x10/0x10
? __pfx_bpf_testmod_init+0x10/0x10
? bpf_fentry_test1+0x5/0x10
? bpf_fentry_test1+0x5/0x10
? bpf_testmod_init+0x22/0x80
? do_one_initcall+0x63/0x2e0
? rcu_is_watching+0xd/0x40
? kmalloc_trace+0xaf/0xc0
? do_init_module+0x60/0x250
? __do_sys_finit_module+0xac/0x120
? do_syscall_64+0x37/0x90
? entry_SYSCALL_64_after_hwframe+0x72/0xdc
</TASK>
In unregister_fprobe function we can't release fp->rethook while it's
possible there are some of its users still running on another cpu.
Moving rethook_free call after fp->ops is unregistered with
unregister_ftrace_function call.
Fixes: 5b0ab78998e3 ("fprobe: Add exit_handler support")
Reviewed-by: Steven Rostedt (Google) <[email protected]>
Signed-off-by: Jiri Olsa <[email protected]>
---
kernel/trace/fprobe.c | 12 +++---------
1 file changed, 3 insertions(+), 9 deletions(-)
diff --git a/kernel/trace/fprobe.c b/kernel/trace/fprobe.c
index 18d36842faf5..0121e8c0d54e 100644
--- a/kernel/trace/fprobe.c
+++ b/kernel/trace/fprobe.c
@@ -364,19 +364,13 @@ int unregister_fprobe(struct fprobe *fp)
fp->ops.saved_func != fprobe_kprobe_handler))
return -EINVAL;
- /*
- * rethook_free() starts disabling the rethook, but the rethook handlers
- * may be running on other processors at this point. To make sure that all
- * current running handlers are finished, call unregister_ftrace_function()
- * after this.
- */
- if (fp->rethook)
- rethook_free(fp->rethook);
-
ret = unregister_ftrace_function(&fp->ops);
if (ret < 0)
return ret;
+ if (fp->rethook)
+ rethook_free(fp->rethook);
+
ftrace_free_filter(&fp->ops);
return ret;
--
2.40.1
Masami,
Want to take this via your probes/urgent branch and send it off to Linus?
-- Steve
On Thu, 15 Jun 2023 13:52:36 +0200
Jiri Olsa <[email protected]> wrote:
> While running bpf selftests it's possible to get following fault:
>
> general protection fault, probably for non-canonical address \
> 0x6b6b6b6b6b6b6b6b: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC NOPTI
> ...
> Call Trace:
> <TASK>
> fprobe_handler+0xc1/0x270
> ? __pfx_bpf_testmod_init+0x10/0x10
> ? __pfx_bpf_testmod_init+0x10/0x10
> ? bpf_fentry_test1+0x5/0x10
> ? bpf_fentry_test1+0x5/0x10
> ? bpf_testmod_init+0x22/0x80
> ? do_one_initcall+0x63/0x2e0
> ? rcu_is_watching+0xd/0x40
> ? kmalloc_trace+0xaf/0xc0
> ? do_init_module+0x60/0x250
> ? __do_sys_finit_module+0xac/0x120
> ? do_syscall_64+0x37/0x90
> ? entry_SYSCALL_64_after_hwframe+0x72/0xdc
> </TASK>
>
> In unregister_fprobe function we can't release fp->rethook while it's
> possible there are some of its users still running on another cpu.
>
> Moving rethook_free call after fp->ops is unregistered with
> unregister_ftrace_function call.
>
> Fixes: 5b0ab78998e3 ("fprobe: Add exit_handler support")
> Reviewed-by: Steven Rostedt (Google) <[email protected]>
> Signed-off-by: Jiri Olsa <[email protected]>
> ---
> kernel/trace/fprobe.c | 12 +++---------
> 1 file changed, 3 insertions(+), 9 deletions(-)
>
> diff --git a/kernel/trace/fprobe.c b/kernel/trace/fprobe.c
> index 18d36842faf5..0121e8c0d54e 100644
> --- a/kernel/trace/fprobe.c
> +++ b/kernel/trace/fprobe.c
> @@ -364,19 +364,13 @@ int unregister_fprobe(struct fprobe *fp)
> fp->ops.saved_func != fprobe_kprobe_handler))
> return -EINVAL;
>
> - /*
> - * rethook_free() starts disabling the rethook, but the rethook handlers
> - * may be running on other processors at this point. To make sure that all
> - * current running handlers are finished, call unregister_ftrace_function()
> - * after this.
> - */
> - if (fp->rethook)
> - rethook_free(fp->rethook);
> -
> ret = unregister_ftrace_function(&fp->ops);
> if (ret < 0)
> return ret;
>
> + if (fp->rethook)
> + rethook_free(fp->rethook);
> +
> ftrace_free_filter(&fp->ops);
>
> return ret;
On Thu, Jun 15, 2023 at 08:59:20AM -0400, Steven Rostedt wrote:
>
> Masami,
>
> Want to take this via your probes/urgent branch and send it off to Linus?
hi,
did this one make it into some tree?
thanks,
jirka
>
> -- Steve
>
>
> On Thu, 15 Jun 2023 13:52:36 +0200
> Jiri Olsa <[email protected]> wrote:
>
> > While running bpf selftests it's possible to get following fault:
> >
> > general protection fault, probably for non-canonical address \
> > 0x6b6b6b6b6b6b6b6b: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC NOPTI
> > ...
> > Call Trace:
> > <TASK>
> > fprobe_handler+0xc1/0x270
> > ? __pfx_bpf_testmod_init+0x10/0x10
> > ? __pfx_bpf_testmod_init+0x10/0x10
> > ? bpf_fentry_test1+0x5/0x10
> > ? bpf_fentry_test1+0x5/0x10
> > ? bpf_testmod_init+0x22/0x80
> > ? do_one_initcall+0x63/0x2e0
> > ? rcu_is_watching+0xd/0x40
> > ? kmalloc_trace+0xaf/0xc0
> > ? do_init_module+0x60/0x250
> > ? __do_sys_finit_module+0xac/0x120
> > ? do_syscall_64+0x37/0x90
> > ? entry_SYSCALL_64_after_hwframe+0x72/0xdc
> > </TASK>
> >
> > In unregister_fprobe function we can't release fp->rethook while it's
> > possible there are some of its users still running on another cpu.
> >
> > Moving rethook_free call after fp->ops is unregistered with
> > unregister_ftrace_function call.
> >
> > Fixes: 5b0ab78998e3 ("fprobe: Add exit_handler support")
> > Reviewed-by: Steven Rostedt (Google) <[email protected]>
> > Signed-off-by: Jiri Olsa <[email protected]>
> > ---
> > kernel/trace/fprobe.c | 12 +++---------
> > 1 file changed, 3 insertions(+), 9 deletions(-)
> >
> > diff --git a/kernel/trace/fprobe.c b/kernel/trace/fprobe.c
> > index 18d36842faf5..0121e8c0d54e 100644
> > --- a/kernel/trace/fprobe.c
> > +++ b/kernel/trace/fprobe.c
> > @@ -364,19 +364,13 @@ int unregister_fprobe(struct fprobe *fp)
> > fp->ops.saved_func != fprobe_kprobe_handler))
> > return -EINVAL;
> >
> > - /*
> > - * rethook_free() starts disabling the rethook, but the rethook handlers
> > - * may be running on other processors at this point. To make sure that all
> > - * current running handlers are finished, call unregister_ftrace_function()
> > - * after this.
> > - */
> > - if (fp->rethook)
> > - rethook_free(fp->rethook);
> > -
> > ret = unregister_ftrace_function(&fp->ops);
> > if (ret < 0)
> > return ret;
> >
> > + if (fp->rethook)
> > + rethook_free(fp->rethook);
> > +
> > ftrace_free_filter(&fp->ops);
> >
> > return ret;
>
Hi Jiri,
On Thu, 15 Jun 2023 13:52:36 +0200
Jiri Olsa <[email protected]> wrote:
> While running bpf selftests it's possible to get following fault:
>
> general protection fault, probably for non-canonical address \
> 0x6b6b6b6b6b6b6b6b: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC NOPTI
> ...
> Call Trace:
> <TASK>
> fprobe_handler+0xc1/0x270
> ? __pfx_bpf_testmod_init+0x10/0x10
> ? __pfx_bpf_testmod_init+0x10/0x10
> ? bpf_fentry_test1+0x5/0x10
> ? bpf_fentry_test1+0x5/0x10
> ? bpf_testmod_init+0x22/0x80
> ? do_one_initcall+0x63/0x2e0
> ? rcu_is_watching+0xd/0x40
> ? kmalloc_trace+0xaf/0xc0
> ? do_init_module+0x60/0x250
> ? __do_sys_finit_module+0xac/0x120
> ? do_syscall_64+0x37/0x90
> ? entry_SYSCALL_64_after_hwframe+0x72/0xdc
> </TASK>
>
> In unregister_fprobe function we can't release fp->rethook while it's
> possible there are some of its users still running on another cpu.
Ah, OK. rethook_free() invoked call_rcu(rethook_free_rcu) to free the
rethook, and it is possible rethook_free_rcu() is called before disabling
all fprobe, then `rethook_try_get(fp->rethook)` will access fp->rethook
which has been freed.
>
> Moving rethook_free call after fp->ops is unregistered with
> unregister_ftrace_function call.
>
> Fixes: 5b0ab78998e3 ("fprobe: Add exit_handler support")
> Reviewed-by: Steven Rostedt (Google) <[email protected]>
> Signed-off-by: Jiri Olsa <[email protected]>
Acked-by: Masami Hiramatsu (Google) <[email protected]>
Thank you!
> ---
> kernel/trace/fprobe.c | 12 +++---------
> 1 file changed, 3 insertions(+), 9 deletions(-)
>
> diff --git a/kernel/trace/fprobe.c b/kernel/trace/fprobe.c
> index 18d36842faf5..0121e8c0d54e 100644
> --- a/kernel/trace/fprobe.c
> +++ b/kernel/trace/fprobe.c
> @@ -364,19 +364,13 @@ int unregister_fprobe(struct fprobe *fp)
> fp->ops.saved_func != fprobe_kprobe_handler))
> return -EINVAL;
>
> - /*
> - * rethook_free() starts disabling the rethook, but the rethook handlers
> - * may be running on other processors at this point. To make sure that all
> - * current running handlers are finished, call unregister_ftrace_function()
> - * after this.
> - */
> - if (fp->rethook)
> - rethook_free(fp->rethook);
> -
> ret = unregister_ftrace_function(&fp->ops);
> if (ret < 0)
> return ret;
>
> + if (fp->rethook)
> + rethook_free(fp->rethook);
> +
> ftrace_free_filter(&fp->ops);
>
> return ret;
> --
> 2.40.1
>
--
Masami Hiramatsu (Google) <[email protected]>
On Tue, 27 Jun 2023 23:33:06 +0900
Masami Hiramatsu (Google) <[email protected]> wrote:
> > diff --git a/kernel/trace/fprobe.c b/kernel/trace/fprobe.c
> > index 18d36842faf5..0121e8c0d54e 100644
> > --- a/kernel/trace/fprobe.c
> > +++ b/kernel/trace/fprobe.c
> > @@ -364,19 +364,13 @@ int unregister_fprobe(struct fprobe *fp)
> > fp->ops.saved_func != fprobe_kprobe_handler))
> > return -EINVAL;
> >
> > - /*
> > - * rethook_free() starts disabling the rethook, but the rethook handlers
> > - * may be running on other processors at this point. To make sure that all
> > - * current running handlers are finished, call unregister_ftrace_function()
> > - * after this.
> > - */
Oh, wait, here is an important comment. If a rethook handler is still running
(because it hooks target function exit), returning from unregister_fprobe()
right after rethook_free() may cause another issue.
rethook_free() clears 'rh->handler', so after calling rethook_free(), we
can ensure no NEW rethook handler (means fprobe_exit_handler()) is called.
However, it doesn't mean there is no current running fprobe_exit_handler().
Thus if unregister_fprobe() caller releases the 'fp' right after returning
from unregister_fprobe(), current running fprobe_exit_handler() can access
'fp' (use-after-free).
Thus we need to add below code with this patch;
/*
* The rethook handlers may be running on other processors at this point.
* To make sure that all current running handlers are finished, disable
* rethook by clearing handler and call unregister_ftrace_function()
* to ensure all running rethook handlers exit. And call rethook_free().
*/
if (fp->rethook)
WRITE_ONCE(fp->rethook->handler, NULL);
> > - if (fp->rethook)
> > - rethook_free(fp->rethook);
> > -
> > ret = unregister_ftrace_function(&fp->ops);
> > if (ret < 0)
> > return ret;
> >
> > + if (fp->rethook)
> > + rethook_free(fp->rethook);
> > +
> > ftrace_free_filter(&fp->ops);
> >
> > return ret;
Thank you,
> > --
> > 2.40.1
> >
>
>
> --
> Masami Hiramatsu (Google) <[email protected]>
--
Masami Hiramatsu (Google) <[email protected]>
From: Masami Hiramatsu (Google) <[email protected]>
Ensure running fprobe_exit_handler() has finished before
calling rethook_free() in the unregister_fprobe() so that caller can free
the fprobe right after unregister_fprobe().
unregister_fprobe() ensured that all running fprobe_entry/exit_handler()
have finished by calling unregister_ftrace_function() which synchronizes
RCU. But commit 5f81018753df ("fprobe: Release rethook after the ftrace_ops
is unregistered") changed to call rethook_free() after
unregister_ftrace_function(). So call rethook_stop() to make rethook
disabled before unregister_ftrace_function() and ensure it again.
Fixes: 5f81018753df ("fprobe: Release rethook after the ftrace_ops is unregistered")
Cc: [email protected]
Signed-off-by: Masami Hiramatsu (Google) <[email protected]>
---
include/linux/rethook.h | 1 +
kernel/trace/fprobe.c | 3 +++
kernel/trace/rethook.c | 13 +++++++++++++
3 files changed, 17 insertions(+)
diff --git a/include/linux/rethook.h b/include/linux/rethook.h
index c8ac1e5afcd1..bdbe6717f45a 100644
--- a/include/linux/rethook.h
+++ b/include/linux/rethook.h
@@ -59,6 +59,7 @@ struct rethook_node {
};
struct rethook *rethook_alloc(void *data, rethook_handler_t handler);
+void rethook_stop(struct rethook *rh);
void rethook_free(struct rethook *rh);
void rethook_add_node(struct rethook *rh, struct rethook_node *node);
struct rethook_node *rethook_try_get(struct rethook *rh);
diff --git a/kernel/trace/fprobe.c b/kernel/trace/fprobe.c
index 0121e8c0d54e..75517667b54f 100644
--- a/kernel/trace/fprobe.c
+++ b/kernel/trace/fprobe.c
@@ -364,6 +364,9 @@ int unregister_fprobe(struct fprobe *fp)
fp->ops.saved_func != fprobe_kprobe_handler))
return -EINVAL;
+ if (fp->rethook)
+ rethook_stop(fp->rethook);
+
ret = unregister_ftrace_function(&fp->ops);
if (ret < 0)
return ret;
diff --git a/kernel/trace/rethook.c b/kernel/trace/rethook.c
index 60f6cb2b486b..468006cce7ca 100644
--- a/kernel/trace/rethook.c
+++ b/kernel/trace/rethook.c
@@ -53,6 +53,19 @@ static void rethook_free_rcu(struct rcu_head *head)
kfree(rh);
}
+/**
+ * rethook_stop() - Stop using a rethook.
+ * @rh: the struct rethook to stop.
+ *
+ * Stop using a rethook to prepare for freeing it. If you want to wait for
+ * all running rethook handler before calling rethook_free(), you need to
+ * call this first and wait RCU, and call rethook_free().
+ */
+void rethook_stop(struct rethook *rh)
+{
+ WRITE_ONCE(rh->handler, NULL);
+}
+
/**
* rethook_free() - Free struct rethook.
* @rh: the struct rethook to be freed.
On Wed, 28 Jun 2023 23:44:02 +0900
"Masami Hiramatsu (Google)" <[email protected]> wrote:
> From: Masami Hiramatsu (Google) <[email protected]>
>
> Ensure running fprobe_exit_handler() has finished before
> calling rethook_free() in the unregister_fprobe() so that caller can free
> the fprobe right after unregister_fprobe().
>
> unregister_fprobe() ensured that all running fprobe_entry/exit_handler()
> have finished by calling unregister_ftrace_function() which synchronizes
> RCU. But commit 5f81018753df ("fprobe: Release rethook after the ftrace_ops
> is unregistered") changed to call rethook_free() after
> unregister_ftrace_function(). So call rethook_stop() to make rethook
> disabled before unregister_ftrace_function() and ensure it again.
>
Steve, can you review this? without this fix, Jiri's patch may cause another
timing issue.
Thanks,
> Fixes: 5f81018753df ("fprobe: Release rethook after the ftrace_ops is unregistered")
> Cc: [email protected]
> Signed-off-by: Masami Hiramatsu (Google) <[email protected]>
> ---
> include/linux/rethook.h | 1 +
> kernel/trace/fprobe.c | 3 +++
> kernel/trace/rethook.c | 13 +++++++++++++
> 3 files changed, 17 insertions(+)
>
> diff --git a/include/linux/rethook.h b/include/linux/rethook.h
> index c8ac1e5afcd1..bdbe6717f45a 100644
> --- a/include/linux/rethook.h
> +++ b/include/linux/rethook.h
> @@ -59,6 +59,7 @@ struct rethook_node {
> };
>
> struct rethook *rethook_alloc(void *data, rethook_handler_t handler);
> +void rethook_stop(struct rethook *rh);
> void rethook_free(struct rethook *rh);
> void rethook_add_node(struct rethook *rh, struct rethook_node *node);
> struct rethook_node *rethook_try_get(struct rethook *rh);
> diff --git a/kernel/trace/fprobe.c b/kernel/trace/fprobe.c
> index 0121e8c0d54e..75517667b54f 100644
> --- a/kernel/trace/fprobe.c
> +++ b/kernel/trace/fprobe.c
> @@ -364,6 +364,9 @@ int unregister_fprobe(struct fprobe *fp)
> fp->ops.saved_func != fprobe_kprobe_handler))
> return -EINVAL;
>
> + if (fp->rethook)
> + rethook_stop(fp->rethook);
> +
> ret = unregister_ftrace_function(&fp->ops);
> if (ret < 0)
> return ret;
> diff --git a/kernel/trace/rethook.c b/kernel/trace/rethook.c
> index 60f6cb2b486b..468006cce7ca 100644
> --- a/kernel/trace/rethook.c
> +++ b/kernel/trace/rethook.c
> @@ -53,6 +53,19 @@ static void rethook_free_rcu(struct rcu_head *head)
> kfree(rh);
> }
>
> +/**
> + * rethook_stop() - Stop using a rethook.
> + * @rh: the struct rethook to stop.
> + *
> + * Stop using a rethook to prepare for freeing it. If you want to wait for
> + * all running rethook handler before calling rethook_free(), you need to
> + * call this first and wait RCU, and call rethook_free().
> + */
> +void rethook_stop(struct rethook *rh)
> +{
> + WRITE_ONCE(rh->handler, NULL);
> +}
> +
> /**
> * rethook_free() - Free struct rethook.
> * @rh: the struct rethook to be freed.
>
--
Masami Hiramatsu (Google) <[email protected]>
On Wed, 28 Jun 2023 23:44:02 +0900
"Masami Hiramatsu (Google)" <[email protected]> wrote:
> From: Masami Hiramatsu (Google) <[email protected]>
>
> Ensure running fprobe_exit_handler() has finished before
> calling rethook_free() in the unregister_fprobe() so that caller can free
> the fprobe right after unregister_fprobe().
>
> unregister_fprobe() ensured that all running fprobe_entry/exit_handler()
> have finished by calling unregister_ftrace_function() which synchronizes
> RCU. But commit 5f81018753df ("fprobe: Release rethook after the
> ftrace_ops is unregistered") changed to call rethook_free() after
> unregister_ftrace_function(). So call rethook_stop() to make rethook
> disabled before unregister_ftrace_function() and ensure it again.
I'm confused. I still don't understand why it is bad to call
unregister_ftrace_function() *before* rethook_free().
Can you show the race condition you are trying to avoid?
-- Steve
>
> Fixes: 5f81018753df ("fprobe: Release rethook after the ftrace_ops is
> unregistered") Cc: [email protected]
> Signed-off-by: Masami Hiramatsu (Google) <[email protected]>
> ---
> include/linux/rethook.h | 1 +
> kernel/trace/fprobe.c | 3 +++
> kernel/trace/rethook.c | 13 +++++++++++++
> 3 files changed, 17 insertions(+)
>
> diff --git a/include/linux/rethook.h b/include/linux/rethook.h
> index c8ac1e5afcd1..bdbe6717f45a 100644
> --- a/include/linux/rethook.h
> +++ b/include/linux/rethook.h
> @@ -59,6 +59,7 @@ struct rethook_node {
> };
>
> struct rethook *rethook_alloc(void *data, rethook_handler_t handler);
> +void rethook_stop(struct rethook *rh);
> void rethook_free(struct rethook *rh);
> void rethook_add_node(struct rethook *rh, struct rethook_node *node);
> struct rethook_node *rethook_try_get(struct rethook *rh);
> diff --git a/kernel/trace/fprobe.c b/kernel/trace/fprobe.c
> index 0121e8c0d54e..75517667b54f 100644
> --- a/kernel/trace/fprobe.c
> +++ b/kernel/trace/fprobe.c
> @@ -364,6 +364,9 @@ int unregister_fprobe(struct fprobe *fp)
> fp->ops.saved_func != fprobe_kprobe_handler))
> return -EINVAL;
>
> + if (fp->rethook)
> + rethook_stop(fp->rethook);
> +
> ret = unregister_ftrace_function(&fp->ops);
> if (ret < 0)
> return ret;
> diff --git a/kernel/trace/rethook.c b/kernel/trace/rethook.c
> index 60f6cb2b486b..468006cce7ca 100644
> --- a/kernel/trace/rethook.c
> +++ b/kernel/trace/rethook.c
> @@ -53,6 +53,19 @@ static void rethook_free_rcu(struct rcu_head *head)
> kfree(rh);
> }
>
> +/**
> + * rethook_stop() - Stop using a rethook.
> + * @rh: the struct rethook to stop.
> + *
> + * Stop using a rethook to prepare for freeing it. If you want to wait
> for
> + * all running rethook handler before calling rethook_free(), you need to
> + * call this first and wait RCU, and call rethook_free().
> + */
> +void rethook_stop(struct rethook *rh)
> +{
> + WRITE_ONCE(rh->handler, NULL);
> +}
> +
> /**
> * rethook_free() - Free struct rethook.
> * @rh: the struct rethook to be freed.
On Wed, 5 Jul 2023 21:26:57 -0400
Steven Rostedt <[email protected]> wrote:
> On Wed, 28 Jun 2023 23:44:02 +0900
> "Masami Hiramatsu (Google)" <[email protected]> wrote:
>
> > From: Masami Hiramatsu (Google) <[email protected]>
> >
> > Ensure running fprobe_exit_handler() has finished before
> > calling rethook_free() in the unregister_fprobe() so that caller can free
> > the fprobe right after unregister_fprobe().
> >
> > unregister_fprobe() ensured that all running fprobe_entry/exit_handler()
> > have finished by calling unregister_ftrace_function() which synchronizes
> > RCU. But commit 5f81018753df ("fprobe: Release rethook after the
> > ftrace_ops is unregistered") changed to call rethook_free() after
> > unregister_ftrace_function(). So call rethook_stop() to make rethook
> > disabled before unregister_ftrace_function() and ensure it again.
>
> I'm confused. I still don't understand why it is bad to call
> unregister_ftrace_function() *before* rethook_free().
>
> Can you show the race condition you are trying to avoid?
Yes. This is ensuring all handlers exit when returning from
unregister_fprobe() so that the caller can release the data which will be
accessed from the handlers. The entry handler is safe because
unregister_ftrace_function() waits for the ftrace handlers. But that is
not enough for the exit handler.
With only Jiri's patch, following flow can happen;
------
CPU1 CPU2
call unregister_fprobe()
...
__fprobe_handler()
rethook_hook() on probed function
unregister_ftrace_function()
return from probed function
rethook hooks
find rh->handler == fprobe_exit_handler
call fprobe_exit_handler()
rethook_free():
set rh->handler = NULL;
return from unreigster_fprobe;
call fp->exit_handler() <- (*)
(*) In this point, the exit handler is called after returning from
unregister_fprobe().
------
So, this patch changes it as following;
------
CPU1 CPU2
call unregister_fprobe()
...
rethook_stop():
set rh->handler = NULL;
__fprobe_handler()
rethook_hook() on probed function
unregister_ftrace_function()
return from probed function
rethook hooks
find rh->handler == NULL
return from rethook
rethook_free()
return from unreigster_fprobe;
------
I can also just put a synchronize_sched_rcu() right after rethook_free()
to wait for all running fprobe_exit_handler() too.
Thank you,
>
> -- Steve
>
>
>
> >
> > Fixes: 5f81018753df ("fprobe: Release rethook after the ftrace_ops is
> > unregistered") Cc: [email protected]
> > Signed-off-by: Masami Hiramatsu (Google) <[email protected]>
> > ---
> > include/linux/rethook.h | 1 +
> > kernel/trace/fprobe.c | 3 +++
> > kernel/trace/rethook.c | 13 +++++++++++++
> > 3 files changed, 17 insertions(+)
> >
> > diff --git a/include/linux/rethook.h b/include/linux/rethook.h
> > index c8ac1e5afcd1..bdbe6717f45a 100644
> > --- a/include/linux/rethook.h
> > +++ b/include/linux/rethook.h
> > @@ -59,6 +59,7 @@ struct rethook_node {
> > };
> >
> > struct rethook *rethook_alloc(void *data, rethook_handler_t handler);
> > +void rethook_stop(struct rethook *rh);
> > void rethook_free(struct rethook *rh);
> > void rethook_add_node(struct rethook *rh, struct rethook_node *node);
> > struct rethook_node *rethook_try_get(struct rethook *rh);
> > diff --git a/kernel/trace/fprobe.c b/kernel/trace/fprobe.c
> > index 0121e8c0d54e..75517667b54f 100644
> > --- a/kernel/trace/fprobe.c
> > +++ b/kernel/trace/fprobe.c
> > @@ -364,6 +364,9 @@ int unregister_fprobe(struct fprobe *fp)
> > fp->ops.saved_func != fprobe_kprobe_handler))
> > return -EINVAL;
> >
> > + if (fp->rethook)
> > + rethook_stop(fp->rethook);
> > +
> > ret = unregister_ftrace_function(&fp->ops);
> > if (ret < 0)
> > return ret;
> > diff --git a/kernel/trace/rethook.c b/kernel/trace/rethook.c
> > index 60f6cb2b486b..468006cce7ca 100644
> > --- a/kernel/trace/rethook.c
> > +++ b/kernel/trace/rethook.c
> > @@ -53,6 +53,19 @@ static void rethook_free_rcu(struct rcu_head *head)
> > kfree(rh);
> > }
> >
> > +/**
> > + * rethook_stop() - Stop using a rethook.
> > + * @rh: the struct rethook to stop.
> > + *
> > + * Stop using a rethook to prepare for freeing it. If you want to wait
> > for
> > + * all running rethook handler before calling rethook_free(), you need to
> > + * call this first and wait RCU, and call rethook_free().
> > + */
> > +void rethook_stop(struct rethook *rh)
> > +{
> > + WRITE_ONCE(rh->handler, NULL);
> > +}
> > +
> > /**
> > * rethook_free() - Free struct rethook.
> > * @rh: the struct rethook to be freed.
>
--
Masami Hiramatsu (Google) <[email protected]>
On Thu, 6 Jul 2023 14:10:12 +0900
Masami Hiramatsu (Google) <[email protected]> wrote:
> With only Jiri's patch, following flow can happen;
>
> ------
> CPU1 CPU2
> call unregister_fprobe()
> ...
> __fprobe_handler()
> rethook_hook() on probed function
> unregister_ftrace_function()
> return from probed function
> rethook hooks
> find rh->handler == fprobe_exit_handler
> call fprobe_exit_handler()
> rethook_free():
> set rh->handler = NULL;
> return from unreigster_fprobe;
> call fp->exit_handler() <- (*)
>
> (*) In this point, the exit handler is called after returning from
> unregister_fprobe().
> ------
>
> So, this patch changes it as following;
> ------
> CPU1 CPU2
> call unregister_fprobe()
> ...
> rethook_stop():
> set rh->handler = NULL;
> __fprobe_handler()
> rethook_hook() on probed function
> unregister_ftrace_function()
> return from probed function
> rethook hooks
> find rh->handler == NULL
> return from rethook
> rethook_free()
> return from unreigster_fprobe;
> ------
>
> I can also just put a synchronize_sched_rcu() right after rethook_free()
> to wait for all running fprobe_exit_handler() too.
>
This makes more sense. Can you please add the above to the change log.
Thanks,
-- Steve
On Thu, 6 Jul 2023 09:56:24 -0400
Steven Rostedt <[email protected]> wrote:
> On Thu, 6 Jul 2023 14:10:12 +0900
> Masami Hiramatsu (Google) <[email protected]> wrote:
>
> > With only Jiri's patch, following flow can happen;
> >
> > ------
> > CPU1 CPU2
> > call unregister_fprobe()
> > ...
> > __fprobe_handler()
> > rethook_hook() on probed function
> > unregister_ftrace_function()
> > return from probed function
> > rethook hooks
> > find rh->handler == fprobe_exit_handler
> > call fprobe_exit_handler()
> > rethook_free():
> > set rh->handler = NULL;
> > return from unreigster_fprobe;
> > call fp->exit_handler() <- (*)
> >
> > (*) In this point, the exit handler is called after returning from
> > unregister_fprobe().
> > ------
> >
> > So, this patch changes it as following;
> > ------
> > CPU1 CPU2
> > call unregister_fprobe()
> > ...
> > rethook_stop():
> > set rh->handler = NULL;
> > __fprobe_handler()
> > rethook_hook() on probed function
> > unregister_ftrace_function()
> > return from probed function
> > rethook hooks
> > find rh->handler == NULL
> > return from rethook
> > rethook_free()
> > return from unreigster_fprobe;
> > ------
> >
> > I can also just put a synchronize_sched_rcu() right after rethook_free()
> > to wait for all running fprobe_exit_handler() too.
> >
>
> This makes more sense. Can you please add the above to the change log.
OK, let me update it.
Thanks!
>
> Thanks,
>
> -- Steve
--
Masami Hiramatsu (Google) <[email protected]>