2018-12-04 19:42:06

by Anders Roxell

[permalink] [raw]
Subject: [PATCH v3] tracing: add cond_resched to ftrace_replace_code()

When running in qemu on an kernel built with allmodconfig and debug
options (in particular kcov and ubsan) enabled, ftrace_replace_code
function call take minutes. The ftrace selftest calls
ftrace_replace_code to look >40000 through
ftrace_make_call/ftrace_make_nop, and these end up calling
__aarch64_insn_write/aarch64_insn_patch_text_nosync.

Microseconds add up because this is called in a loop for each dyn_ftrace
record, and this triggers the softlockup watchdog unless we let it sleep
occasionally.

Rework so that we call cond_resched() if !irqs_disabled() && !preempt_count().

Suggested-by: Steven Rostedt (VMware) <[email protected]>
Signed-off-by: Anders Roxell <[email protected]>
---
kernel/trace/ftrace.c | 10 ++++++++++
1 file changed, 10 insertions(+)

diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
index c375e33239f7..7c9f49a5a1ab 100644
--- a/kernel/trace/ftrace.c
+++ b/kernel/trace/ftrace.c
@@ -2419,11 +2419,19 @@ void __weak ftrace_replace_code(int enable)
{
struct dyn_ftrace *rec;
struct ftrace_page *pg;
+ bool schedulable;
int failed;

if (unlikely(ftrace_disabled))
return;

+ /*
+ * Some archs call this function with interrupts or preemption
+ * disabled. However, for other archs that can preempt, this can cause
+ * an tremendous unneeded latency.
+ */
+ schedulable = !irqs_disabled() && !preempt_count();
+
do_for_each_ftrace_rec(pg, rec) {

if (rec->flags & FTRACE_FL_DISABLED)
@@ -2435,6 +2443,8 @@ void __weak ftrace_replace_code(int enable)
/* Stop processing */
return;
}
+ if (schedulable)
+ cond_resched();
} while_for_each_ftrace_rec();
}

--
2.19.2



2018-12-04 22:33:53

by Steven Rostedt

[permalink] [raw]
Subject: Re: [PATCH v3] tracing: add cond_resched to ftrace_replace_code()

On Tue, 4 Dec 2018 20:40:44 +0100
Anders Roxell <[email protected]> wrote:

> When running in qemu on an kernel built with allmodconfig and debug
> options (in particular kcov and ubsan) enabled, ftrace_replace_code
> function call take minutes. The ftrace selftest calls
> ftrace_replace_code to look >40000 through
> ftrace_make_call/ftrace_make_nop, and these end up calling
> __aarch64_insn_write/aarch64_insn_patch_text_nosync.
>
> Microseconds add up because this is called in a loop for each dyn_ftrace
> record, and this triggers the softlockup watchdog unless we let it sleep
> occasionally.
>
> Rework so that we call cond_resched() if !irqs_disabled() && !preempt_count().

This isn't urgent is it? That is, it doesn't need a stable tag?

-- Steve



2018-12-05 10:51:42

by Anders Roxell

[permalink] [raw]
Subject: Re: [PATCH v3] tracing: add cond_resched to ftrace_replace_code()

On Tue, 4 Dec 2018 at 23:33, Steven Rostedt <[email protected]> wrote:
>
> On Tue, 4 Dec 2018 20:40:44 +0100
> Anders Roxell <[email protected]> wrote:
>
> > When running in qemu on an kernel built with allmodconfig and debug
> > options (in particular kcov and ubsan) enabled, ftrace_replace_code
> > function call take minutes. The ftrace selftest calls
> > ftrace_replace_code to look >40000 through
> > ftrace_make_call/ftrace_make_nop, and these end up calling
> > __aarch64_insn_write/aarch64_insn_patch_text_nosync.
> >
> > Microseconds add up because this is called in a loop for each dyn_ftrace
> > record, and this triggers the softlockup watchdog unless we let it sleep
> > occasionally.
> >
> > Rework so that we call cond_resched() if !irqs_disabled() && !preempt_count().
>
> This isn't urgent is it?

No, it is not urgent. I'm trying to get allmodconfig to boot on arm64.

> That is, it doesn't need a stable tag?

Also I don't think its a regression, since clearly nobody has ever been
able to boot an arm64 allmodconfig kernel or anything close to that.

Cheers,
Anders