The perf has basic kernel ftrace support but lack support of most tracing
options. This serias is target to enhance the perf ftrace functionality so
that we can make full use of kernel ftrace with perf.
In general, this serias be cataloged into two main changes:
1) Improve usability of existing functions. For example, we don't need to type
extra option to select the tracer.
2) Add new options to support all other ftrace functions.
Here is a glance of all ftrace functions with this serias:
$ sudo perf ftrace -h
Usage: perf ftrace [<options>] [<command>]
or: perf ftrace [<options>] -- <command> [<options>]
-a, --all-cpus system-wide collection from all CPUs
-C, --cpu <cpu> list of cpus to monitor
-D, --delay <n> ms to wait before starting tracing after program start
-F, --funcs Show available functions to filter
-G, --graph-funcs <func>
trace given functions using function_graph tracer
-g, --nograph-funcs <func>
Set nograph filter on given functions
-m, --buffer-size <size>
size of per cpu buffer
-N, --notrace-funcs <func>
do not trace given functions
-p, --pid <pid> trace on existing process id
-T, --trace-funcs <func>
trace given functions using function tracer
-t, --tracer <tracer>
tracer to use: function_graph(default) or function
-v, --verbose be more verbose
--func-opts <options>
function tracer options, available options: call-graph,irq-info
--graph-opts <options>
graph tracer options, available options: nosleep-time,noirqs,verbose,thresh=<n>,depth=<n>
--inherit trace children processes
--tid <tid> trace on existing thread id (exclusive to --pid)
v8:
o change tracer selection policy ([PATCH 01/18]).
v7:
o add back '--tid <tid>'.
v6:
o fix return value of read_tracing_file_to_stdout().
o make __cmd_ftrace() shorter.
o remove option '-t, --tid <tid>'.
v5:
o trivial fixes.
v4:
o add util/parse-sublevel-options.c
O remove -D/--graph-depth
v3:
o add --func-opts and --graph-opts to set tracer specific options.
o support units as a suffix for option '-m/--buffer-size'.
v2:
o patches for option '-u/--userstacktrace' and '--no-pager' are dropped.
o update all related perf documentation.
o rename some options. Now all funcgraph tracer options are prefixed with
'--graph-', while all function tracer options are prefixed with '--func-'.
o mark old options deprecated instead of removing them.
Acked-by: Namhyung Kim <[email protected]>
Changbin Du (18):
perf ftrace: select function/function_graph tracer automatically
perf ftrace: add option '-F/--funcs' to list available functions
perf ftrace: factor out function write_tracing_file_int()
perf ftrace: add option '-m/--buffer-size' to set per-cpu buffer size
perf ftrace: show trace column header
perf ftrace: add option '--inherit' to trace children processes
perf: util: add general function to parse sublevel options
perf ftrace: add support for tracing option 'func_stack_trace'
perf ftrace: add support for trace option sleep-time
perf ftrace: add support for trace option funcgraph-irqs
perf ftrace: add support for tracing option 'irq-info'
perf ftrace: add option 'verbose' to show more info for graph tracer
perf ftrace: add support for trace option tracing_thresh
perf: ftrace: allow set graph depth by '--graph-opts'
perf ftrace: add option -D/--delay to delay tracing
perf ftrace: add option --tid to filter by thread id
perf: ftrace: Add set_tracing_options() to set all trace options
perf ftrace: add change log
tools/perf/Documentation/perf-config.txt | 5 +-
tools/perf/Documentation/perf-ftrace.txt | 75 ++--
tools/perf/builtin-ftrace.c | 424 +++++++++++++++++++++--
tools/perf/util/Build | 1 +
tools/perf/util/debug.c | 61 +---
tools/perf/util/parse-sublevel-options.c | 70 ++++
tools/perf/util/parse-sublevel-options.h | 11 +
7 files changed, 550 insertions(+), 97 deletions(-)
create mode 100644 tools/perf/util/parse-sublevel-options.c
create mode 100644 tools/perf/util/parse-sublevel-options.h
--
2.25.1
The '-g/-G' options have already implied function_graph tracer should be
used instead of function tracer. So we don't need extra option '--tracer'
in this case.
This patch changes the behavior as below:
- If '-g' or '-G' option is on, then function_graph tracer is used.
- If '-T' or '-N' option is on, then function tracer is used.
- The function_graph has priority over function tracer.
- The option '--tracer' only take effect if neither -g/-G nor -T/-N
is specified.
Here are some examples.
This will start tracing all functions using default tracer:
$ sudo perf ftrace
This will trace all functions using function graph tracer:
$ sudo perf ftrace -G '*'
This will trace function vfs_read using function graph tracer:
$ sudo perf ftrace -G vfs_read
This will trace function vfs_read using function tracer:
$ sudo perf ftrace -T vfs_read
Signed-off-by: Changbin Du <[email protected]>
---
v4: changed tracer selection policy.
v3: remove default '*' for -G/-T.
---
tools/perf/Documentation/perf-config.txt | 5 +--
tools/perf/Documentation/perf-ftrace.txt | 41 +++++++++++++-----------
tools/perf/builtin-ftrace.c | 25 +++++++++++++--
3 files changed, 48 insertions(+), 23 deletions(-)
diff --git a/tools/perf/Documentation/perf-config.txt b/tools/perf/Documentation/perf-config.txt
index c7d3df5798e2..76408d986aed 100644
--- a/tools/perf/Documentation/perf-config.txt
+++ b/tools/perf/Documentation/perf-config.txt
@@ -614,8 +614,9 @@ trace.*::
ftrace.*::
ftrace.tracer::
- Can be used to select the default tracer. Possible values are
- 'function' and 'function_graph'.
+ Can be used to select the default tracer when neither -G nor
+ -F option is not specified. Possible values are 'function' and
+ 'function_graph'.
llvm.*::
llvm.clang-path::
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index b80c84307dc9..821d4d334a09 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -24,7 +24,8 @@ OPTIONS
-t::
--tracer=::
- Tracer to use: function_graph or function.
+ Tracer to use when neither -G nor -F option is not
+ specified: function_graph or function.
-v::
--verbose=::
@@ -50,33 +51,35 @@ OPTIONS
-T::
--trace-funcs=::
- Only trace functions given by the argument. Multiple functions
- can be given by using this option more than once. The function
- argument also can be a glob pattern. It will be passed to
- 'set_ftrace_filter' in tracefs.
+ Select function tracer and set function filter on the given
+ function (or a glob pattern). Multiple functions can be given
+ by using this option more than once. The function argument also
+ can be a glob pattern. It will be passed to 'set_ftrace_filter'
+ in tracefs.
-N::
--notrace-funcs=::
- Do not trace functions given by the argument. Like -T option,
- this can be used more than once to specify multiple functions
- (or glob patterns). It will be passed to 'set_ftrace_notrace'
- in tracefs.
+ Select function tracer and do not trace functions given by the
+ argument. Like -T option, this can be used more than once to
+ specify multiple functions (or glob patterns). It will be
+ passed to 'set_ftrace_notrace' in tracefs.
-G::
--graph-funcs=::
- Set graph filter on the given function (or a glob pattern).
- This is useful for the function_graph tracer only and enables
- tracing for functions executed from the given function.
- This can be used more than once to specify multiple functions.
- It will be passed to 'set_graph_function' in tracefs.
+ Select function_graph tracer and set graph filter on the given
+ function (or a glob pattern). This is useful to trace for
+ functions executed from the given function. This can be used more
+ than once to specify multiple functions. It will be passed to
+ 'set_graph_function' in tracefs.
-g::
--nograph-funcs=::
- Set graph notrace filter on the given function (or a glob pattern).
- Like -G option, this is useful for the function_graph tracer only
- and disables tracing for function executed from the given function.
- This can be used more than once to specify multiple functions.
- It will be passed to 'set_graph_notrace' in tracefs.
+ Select function_graph tracer and set graph notrace filter on the
+ given function (or a glob pattern). Like -G option, this is useful
+ for the function_graph tracer only and disables tracing for function
+ executed from the given function. This can be used more than once to
+ specify multiple functions. It will be passed to 'set_graph_notrace'
+ in tracefs.
-D::
--graph-depth=::
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index 2bfc1b0db536..048a11192b59 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -455,6 +455,23 @@ static void delete_filter_func(struct list_head *head)
}
}
+static void select_tracer(struct perf_ftrace *ftrace)
+{
+ bool graph = !list_empty(&ftrace->graph_funcs) ||
+ !list_empty(&ftrace->nograph_funcs);
+ bool func = !list_empty(&ftrace->filters) ||
+ !list_empty(&ftrace->notrace);
+
+ /* The function_graph has priority over function tracer. */
+ if (graph)
+ ftrace->tracer = "function_graph";
+ else if (func)
+ ftrace->tracer = "function";
+ /* Otherwise, the default tracer is used. */
+
+ pr_debug("%s tracer is used\n", ftrace->tracer);
+}
+
int cmd_ftrace(int argc, const char **argv)
{
int ret;
@@ -479,11 +496,13 @@ int cmd_ftrace(int argc, const char **argv)
OPT_STRING('C', "cpu", &ftrace.target.cpu_list, "cpu",
"list of cpus to monitor"),
OPT_CALLBACK('T', "trace-funcs", &ftrace.filters, "func",
- "trace given functions only", parse_filter_func),
+ "trace given functions using function tracer",
+ parse_filter_func),
OPT_CALLBACK('N', "notrace-funcs", &ftrace.notrace, "func",
"do not trace given functions", parse_filter_func),
OPT_CALLBACK('G', "graph-funcs", &ftrace.graph_funcs, "func",
- "Set graph filter on given functions", parse_filter_func),
+ "trace given functions using function_graph tracer",
+ parse_filter_func),
OPT_CALLBACK('g', "nograph-funcs", &ftrace.nograph_funcs, "func",
"Set nograph filter on given functions", parse_filter_func),
OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
@@ -505,6 +524,8 @@ int cmd_ftrace(int argc, const char **argv)
if (!argc && target__none(&ftrace.target))
ftrace.target.system_wide = true;
+ select_tracer(&ftrace);
+
ret = target__validate(&ftrace.target);
if (ret) {
char errbuf[512];
--
2.25.1
This adds an option '-F/--funcs' to list all available functions to trace,
which is read from tracing file 'available_filter_functions'.
$ sudo ./perf ftrace -F | head
trace_initcall_finish_cb
initcall_blacklisted
do_one_initcall
do_one_initcall
trace_initcall_start_cb
run_init_process
try_to_run_init_process
match_dev_by_label
match_dev_by_uuid
rootfs_init_fs_context
Signed-off-by: Changbin Du <[email protected]>
---
v3: fix return value issue.
v2: option name '-l/--list-functions' -> '-F/--funcs'
---
tools/perf/Documentation/perf-ftrace.txt | 4 +++
tools/perf/builtin-ftrace.c | 46 ++++++++++++++++++++++++
2 files changed, 50 insertions(+)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index 821d4d334a09..4f5628445a63 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -31,6 +31,10 @@ OPTIONS
--verbose=::
Verbosity level.
+-F::
+--funcs::
+ List all available functions to trace.
+
-p::
--pid=::
Trace on existing process id (comma separated list).
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index 048a11192b59..1ebf71ecc1a8 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -33,6 +33,7 @@ struct perf_ftrace {
struct evlist *evlist;
struct target target;
const char *tracer;
+ bool list_avail_functions;
struct list_head filters;
struct list_head notrace;
struct list_head graph_funcs;
@@ -128,6 +129,46 @@ static int append_tracing_file(const char *name, const char *val)
return __write_tracing_file(name, val, true);
}
+static int read_tracing_file_to_stdout(const char *name)
+{
+ char buf[4096];
+ char *file;
+ int fd;
+ int ret = -1;
+
+ file = get_tracing_file(name);
+ if (!file) {
+ pr_debug("cannot get tracing file: %s\n", name);
+ return -1;
+ }
+
+ fd = open(file, O_RDONLY);
+ if (fd < 0) {
+ pr_debug("cannot open tracing file: %s: %s\n",
+ name, str_error_r(errno, buf, sizeof(buf)));
+ goto out;
+ }
+
+ /* read contents to stdout */
+ while (true) {
+ int n = read(fd, buf, sizeof(buf));
+ if (n == 0)
+ break;
+ else if (n < 0)
+ goto out_close;
+
+ if (fwrite(buf, n, 1, stdout) != 1)
+ goto out_close;
+ }
+ ret = 0;
+
+out_close:
+ close(fd);
+out:
+ put_tracing_file(file);
+ return ret;
+}
+
static int reset_tracing_cpu(void);
static void reset_tracing_filters(void);
@@ -302,6 +343,9 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
signal(SIGCHLD, sig_handler);
signal(SIGPIPE, sig_handler);
+ if (ftrace->list_avail_functions)
+ return read_tracing_file_to_stdout("available_filter_functions");
+
if (reset_tracing_files(ftrace) < 0) {
pr_err("failed to reset ftrace\n");
goto out;
@@ -487,6 +531,8 @@ int cmd_ftrace(int argc, const char **argv)
const struct option ftrace_options[] = {
OPT_STRING('t', "tracer", &ftrace.tracer, "tracer",
"tracer to use: function_graph(default) or function"),
+ OPT_BOOLEAN('F', "funcs", &ftrace.list_avail_functions,
+ "Show available functions to filter"),
OPT_STRING('p', "pid", &ftrace.target.pid, "pid",
"trace on existing process id"),
OPT_INCR('v', "verbose", &verbose,
--
2.25.1
We will reuse this function later.
Signed-off-by: Changbin Du <[email protected]>
---
tools/perf/builtin-ftrace.c | 17 ++++++++++++-----
1 file changed, 12 insertions(+), 5 deletions(-)
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index 1ebf71ecc1a8..4b3fcee5725a 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -169,6 +169,17 @@ static int read_tracing_file_to_stdout(const char *name)
return ret;
}
+static int write_tracing_file_int(const char *name, int value)
+{
+ char buf[16];
+
+ snprintf(buf, sizeof(buf), "%d", value);
+ if (write_tracing_file(name, buf) < 0)
+ return -1;
+
+ return 0;
+}
+
static int reset_tracing_cpu(void);
static void reset_tracing_filters(void);
@@ -299,8 +310,6 @@ static void reset_tracing_filters(void)
static int set_tracing_depth(struct perf_ftrace *ftrace)
{
- char buf[16];
-
if (ftrace->graph_depth == 0)
return 0;
@@ -309,9 +318,7 @@ static int set_tracing_depth(struct perf_ftrace *ftrace)
return -1;
}
- snprintf(buf, sizeof(buf), "%d", ftrace->graph_depth);
-
- if (write_tracing_file("max_graph_depth", buf) < 0)
+ if (write_tracing_file_int("max_graph_depth", ftrace->graph_depth) < 0)
return -1;
return 0;
--
2.25.1
This adds an option '-m/--buffer-size' to allow us set the size of per-cpu
tracing buffer.
Signed-off-by: Changbin Du <[email protected]>
---
v2: support units as a suffix.
---
tools/perf/Documentation/perf-ftrace.txt | 5 +++
tools/perf/builtin-ftrace.c | 55 ++++++++++++++++++++++++
2 files changed, 60 insertions(+)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index 4f5628445a63..7a5d915f60b0 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -53,6 +53,11 @@ OPTIONS
Ranges of CPUs are specified with -: 0-2.
Default is to trace on all online CPUs.
+-m::
+--buffer-size::
+ Set the size of per-cpu tracing buffer, <size> is expected to
+ be a number with appended unit character - B/K/M/G.
+
-T::
--trace-funcs=::
Select function tracer and set function filter on the given
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index 4b3fcee5725a..a3a4f4be9dde 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -26,6 +26,7 @@
#include "thread_map.h"
#include "util/cap.h"
#include "util/config.h"
+#include "util/units.h"
#define DEFAULT_TRACER "function_graph"
@@ -39,6 +40,7 @@ struct perf_ftrace {
struct list_head graph_funcs;
struct list_head nograph_funcs;
int graph_depth;
+ unsigned long percpu_buffer_size;
};
struct filter_entry {
@@ -324,6 +326,21 @@ static int set_tracing_depth(struct perf_ftrace *ftrace)
return 0;
}
+static int set_tracing_percpu_buffer_size(struct perf_ftrace *ftrace)
+{
+ int ret;
+
+ if (ftrace->percpu_buffer_size == 0)
+ return 0;
+
+ ret = write_tracing_file_int("buffer_size_kb",
+ ftrace->percpu_buffer_size / 1024);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
+
static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
{
char *trace_file;
@@ -388,6 +405,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
goto out_reset;
}
+ if (set_tracing_percpu_buffer_size(ftrace) < 0) {
+ pr_err("failed to set tracing per-cpu buffer size\n");
+ goto out_reset;
+ }
+
if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
goto out_reset;
@@ -506,6 +528,37 @@ static void delete_filter_func(struct list_head *head)
}
}
+static int parse_buffer_size(const struct option *opt,
+ const char *str, int unset)
+{
+ unsigned long *s = (unsigned long *)opt->value;
+ static struct parse_tag tags_size[] = {
+ { .tag = 'B', .mult = 1 },
+ { .tag = 'K', .mult = 1 << 10 },
+ { .tag = 'M', .mult = 1 << 20 },
+ { .tag = 'G', .mult = 1 << 30 },
+ { .tag = 0 },
+ };
+ unsigned long val;
+
+ if (unset) {
+ *s = 0;
+ return 0;
+ }
+
+ val = parse_tag_value(str, tags_size);
+ if (val != (unsigned long) -1) {
+ if (val < 1024) {
+ pr_err("buffer size too small, must larger than 1KB.");
+ return -1;
+ }
+ *s = val;
+ return 0;
+ }
+
+ return -1;
+}
+
static void select_tracer(struct perf_ftrace *ftrace)
{
bool graph = !list_empty(&ftrace->graph_funcs) ||
@@ -560,6 +613,8 @@ int cmd_ftrace(int argc, const char **argv)
"Set nograph filter on given functions", parse_filter_func),
OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
"Max depth for function graph tracer"),
+ OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
+ "size of per cpu buffer", parse_buffer_size),
OPT_END()
};
--
2.25.1
This makes perf-ftrace display column header before printing trace.
$ sudo perf ftrace
# tracer: function
#
# entries-in-buffer/entries-written: 0/0 #P:8
#
# TASK-PID CPU# TIMESTAMP FUNCTION
# | | | | |
<...>-9246 [006] 10726.262760: mutex_unlock <-rb_simple_write
<...>-9246 [006] 10726.262764: __fsnotify_parent <-vfs_write
<...>-9246 [006] 10726.262765: fsnotify <-vfs_write
<...>-9246 [006] 10726.262766: __sb_end_write <-vfs_write
<...>-9246 [006] 10726.262767: fpregs_assert_state_consistent <-do_syscall_64
Signed-off-by: Changbin Du <[email protected]>
---
tools/perf/builtin-ftrace.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index a3a4f4be9dde..39c694be2b71 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -435,6 +435,9 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
fcntl(trace_fd, F_SETFL, O_NONBLOCK);
pollfd.fd = trace_fd;
+ /* display column headers */
+ read_tracing_file_to_stdout("trace");
+
if (write_tracing_file("tracing_on", "1") < 0) {
pr_err("can't enable tracing\n");
goto out_close_fd;
--
2.25.1
This adds an option '--inherit' to allow us trace children
processes spawned by our target.
Signed-off-by: Changbin Du <[email protected]>
---
v2: option name '--trace-children' -> '--inherit'.
---
tools/perf/Documentation/perf-ftrace.txt | 3 ++
tools/perf/builtin-ftrace.c | 38 ++++++++++++++++++++++++
2 files changed, 41 insertions(+)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index 7a5d915f60b0..c46d0a09b38c 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -58,6 +58,9 @@ OPTIONS
Set the size of per-cpu tracing buffer, <size> is expected to
be a number with appended unit character - B/K/M/G.
+--inherit::
+ Trace children processes spawned by our target.
+
-T::
--trace-funcs=::
Select function tracer and set function filter on the given
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index 39c694be2b71..07b81d0c1658 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -41,6 +41,7 @@ struct perf_ftrace {
struct list_head nograph_funcs;
int graph_depth;
unsigned long percpu_buffer_size;
+ bool inherit;
};
struct filter_entry {
@@ -182,9 +183,27 @@ static int write_tracing_file_int(const char *name, int value)
return 0;
}
+static int write_tracing_option_file(const char *name, const char *val)
+{
+ char *file;
+ int ret;
+
+ if (asprintf(&file, "options/%s", name) < 0)
+ return -1;
+
+ ret = __write_tracing_file(file, val, false);
+ free(file);
+ return ret;
+}
+
static int reset_tracing_cpu(void);
static void reset_tracing_filters(void);
+static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
+{
+ write_tracing_option_file("function-fork", "0");
+}
+
static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
{
if (write_tracing_file("tracing_on", "0") < 0)
@@ -203,6 +222,7 @@ static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
return -1;
reset_tracing_filters();
+ reset_tracing_options(ftrace);
return 0;
}
@@ -341,6 +361,17 @@ static int set_tracing_percpu_buffer_size(struct perf_ftrace *ftrace)
return 0;
}
+static int set_tracing_trace_inherit(struct perf_ftrace *ftrace)
+{
+ if (!ftrace->inherit)
+ return 0;
+
+ if (write_tracing_option_file("function-fork", "1") < 0)
+ return -1;
+
+ return 0;
+}
+
static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
{
char *trace_file;
@@ -410,6 +441,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
goto out_reset;
}
+ if (set_tracing_trace_inherit(ftrace) < 0) {
+ pr_err("failed to set tracing option function-fork\n");
+ goto out_reset;
+ }
+
if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
goto out_reset;
@@ -618,6 +654,8 @@ int cmd_ftrace(int argc, const char **argv)
"Max depth for function graph tracer"),
OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
"size of per cpu buffer", parse_buffer_size),
+ OPT_BOOLEAN(0, "inherit", &ftrace.inherit,
+ "trace children processes"),
OPT_END()
};
--
2.25.1
This adds support to display call trace for function tracer. To do this,
just specify a '--func-opts call-graph' option.
$ sudo perf ftrace -T vfs_read --func-opts call-graph
iio-sensor-prox-855 [003] 6168.369657: vfs_read <-ksys_read
iio-sensor-prox-855 [003] 6168.369677: <stack trace>
=> vfs_read
=> ksys_read
=> __x64_sys_read
=> do_syscall_64
=> entry_SYSCALL_64_after_hwframe
...
Signed-off-by: Changbin Du <[email protected]>
---
v3: switch to uniform option --func-opts.
v2: option name '-s' -> '--func-call-graph'
---
tools/perf/Documentation/perf-ftrace.txt | 4 +++
tools/perf/builtin-ftrace.c | 42 ++++++++++++++++++++++++
2 files changed, 46 insertions(+)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index c46d0a09b38c..8f08ad0992c2 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -76,6 +76,10 @@ OPTIONS
specify multiple functions (or glob patterns). It will be
passed to 'set_ftrace_notrace' in tracefs.
+--func-opts::
+ List of options allowed to set:
+ call-graph - Display kernel stack trace for function tracer.
+
-G::
--graph-funcs=::
Select function_graph tracer and set graph filter on the given
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index 07b81d0c1658..469b89748c42 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -27,6 +27,7 @@
#include "util/cap.h"
#include "util/config.h"
#include "util/units.h"
+#include "util/parse-sublevel-options.h"
#define DEFAULT_TRACER "function_graph"
@@ -42,6 +43,7 @@ struct perf_ftrace {
int graph_depth;
unsigned long percpu_buffer_size;
bool inherit;
+ int func_stack_trace;
};
struct filter_entry {
@@ -202,6 +204,7 @@ static void reset_tracing_filters(void);
static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
{
write_tracing_option_file("function-fork", "0");
+ write_tracing_option_file("func_stack_trace", "0");
}
static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
@@ -278,6 +281,17 @@ static int set_tracing_cpu(struct perf_ftrace *ftrace)
return set_tracing_cpumask(cpumap);
}
+static int set_tracing_func_stack_trace(struct perf_ftrace *ftrace)
+{
+ if (!ftrace->func_stack_trace)
+ return 0;
+
+ if (write_tracing_option_file("func_stack_trace", "1") < 0)
+ return -1;
+
+ return 0;
+}
+
static int reset_tracing_cpu(void)
{
struct perf_cpu_map *cpumap = perf_cpu_map__new(NULL);
@@ -426,6 +440,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
goto out_reset;
}
+ if (set_tracing_func_stack_trace(ftrace) < 0) {
+ pr_err("failed to set tracing option func_stack_trace\n");
+ goto out_reset;
+ }
+
if (set_tracing_filters(ftrace) < 0) {
pr_err("failed to set tracing filters\n");
goto out_reset;
@@ -598,6 +617,26 @@ static int parse_buffer_size(const struct option *opt,
return -1;
}
+static int parse_func_tracer_opts(const struct option *opt,
+ const char *str, int unset)
+{
+ int ret;
+ struct perf_ftrace *ftrace = (struct perf_ftrace *) opt->value;
+ struct sublevel_option func_tracer_opts[] = {
+ { .name = "call-graph", .value_ptr = &ftrace->func_stack_trace },
+ { .name = NULL, }
+ };
+
+ if (unset)
+ return 0;
+
+ ret = perf_parse_sublevel_options(str, func_tracer_opts);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
static void select_tracer(struct perf_ftrace *ftrace)
{
bool graph = !list_empty(&ftrace->graph_funcs) ||
@@ -645,6 +684,9 @@ int cmd_ftrace(int argc, const char **argv)
parse_filter_func),
OPT_CALLBACK('N', "notrace-funcs", &ftrace.notrace, "func",
"do not trace given functions", parse_filter_func),
+ OPT_CALLBACK(0, "func-opts", &ftrace, "options",
+ "function tracer options, available options: call-graph",
+ parse_func_tracer_opts),
OPT_CALLBACK('G', "graph-funcs", &ftrace.graph_funcs, "func",
"trace given functions using function_graph tracer",
parse_filter_func),
--
2.25.1
This adds an option '--graph-opts nosleep-time' which allow us
only to measure on-CPU time. This option is function_graph tracer
only.
Signed-off-by: Changbin Du <[email protected]>
---
v3: switch to uniform option --graph-opts.
v2: option name '--nosleep-time' -> '--graph-nosleep-time'.
---
tools/perf/Documentation/perf-ftrace.txt | 4 +++
tools/perf/builtin-ftrace.c | 41 ++++++++++++++++++++++++
2 files changed, 45 insertions(+)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index 8f08ad0992c2..3380a2e2c9ad 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -101,6 +101,10 @@ OPTIONS
--graph-depth=::
Set max depth for function graph tracer to follow
+--graph-opts::
+ List of options allowed to set:
+ nosleep-time - Measure on-CPU time only for function_graph tracer.
+
SEE ALSO
--------
linkperf:perf-record[1], linkperf:perf-trace[1]
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index 469b89748c42..47d63bba6a48 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -44,6 +44,7 @@ struct perf_ftrace {
unsigned long percpu_buffer_size;
bool inherit;
int func_stack_trace;
+ int graph_nosleep_time;
};
struct filter_entry {
@@ -205,6 +206,7 @@ static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
{
write_tracing_option_file("function-fork", "0");
write_tracing_option_file("func_stack_trace", "0");
+ write_tracing_option_file("sleep-time", "1");
}
static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
@@ -386,6 +388,17 @@ static int set_tracing_trace_inherit(struct perf_ftrace *ftrace)
return 0;
}
+static int set_tracing_sleep_time(struct perf_ftrace *ftrace)
+{
+ if (!ftrace->graph_nosleep_time)
+ return 0;
+
+ if (write_tracing_option_file("sleep-time", "0") < 0)
+ return -1;
+
+ return 0;
+}
+
static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
{
char *trace_file;
@@ -465,6 +478,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
goto out_reset;
}
+ if (set_tracing_sleep_time(ftrace) < 0) {
+ pr_err("failed to set tracing option sleep-time\n");
+ goto out_reset;
+ }
+
if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
goto out_reset;
@@ -637,6 +655,26 @@ static int parse_func_tracer_opts(const struct option *opt,
return 0;
}
+static int parse_graph_tracer_opts(const struct option *opt,
+ const char *str, int unset)
+{
+ int ret;
+ struct perf_ftrace *ftrace = (struct perf_ftrace *) opt->value;
+ struct sublevel_option graph_tracer_opts[] = {
+ { .name = "nosleep-time", .value_ptr = &ftrace->graph_nosleep_time },
+ { .name = NULL, }
+ };
+
+ if (unset)
+ return 0;
+
+ ret = perf_parse_sublevel_options(str, graph_tracer_opts);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
static void select_tracer(struct perf_ftrace *ftrace)
{
bool graph = !list_empty(&ftrace->graph_funcs) ||
@@ -694,6 +732,9 @@ int cmd_ftrace(int argc, const char **argv)
"Set nograph filter on given functions", parse_filter_func),
OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
"Max depth for function graph tracer"),
+ OPT_CALLBACK(0, "graph-opts", &ftrace, "options",
+ "graph tracer options, available options: nosleep-time",
+ parse_graph_tracer_opts),
OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
"size of per cpu buffer", parse_buffer_size),
OPT_BOOLEAN(0, "inherit", &ftrace.inherit,
--
2.25.1
This adds an option '--graph-opts noirqs' to filter out functions executed
in irq context.
Signed-off-by: Changbin Du <[email protected]>
---
v2: option name '--nofuncgraph-irqs' -> '--graph-noirqs'.
---
tools/perf/Documentation/perf-ftrace.txt | 1 +
tools/perf/builtin-ftrace.c | 21 ++++++++++++++++++++-
2 files changed, 21 insertions(+), 1 deletion(-)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index 3380a2e2c9ad..fca55ac55ff3 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -104,6 +104,7 @@ OPTIONS
--graph-opts::
List of options allowed to set:
nosleep-time - Measure on-CPU time only for function_graph tracer.
+ noirqs - Ignore functions that happen inside interrupt.
SEE ALSO
--------
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index 47d63bba6a48..b4c821be4fb5 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -45,6 +45,7 @@ struct perf_ftrace {
bool inherit;
int func_stack_trace;
int graph_nosleep_time;
+ int graph_noirqs;
};
struct filter_entry {
@@ -207,6 +208,7 @@ static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
write_tracing_option_file("function-fork", "0");
write_tracing_option_file("func_stack_trace", "0");
write_tracing_option_file("sleep-time", "1");
+ write_tracing_option_file("funcgraph-irqs", "1");
}
static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
@@ -399,6 +401,17 @@ static int set_tracing_sleep_time(struct perf_ftrace *ftrace)
return 0;
}
+static int set_tracing_funcgraph_irqs(struct perf_ftrace *ftrace)
+{
+ if (!ftrace->graph_noirqs)
+ return 0;
+
+ if (write_tracing_option_file("funcgraph-irqs", "0") < 0)
+ return -1;
+
+ return 0;
+}
+
static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
{
char *trace_file;
@@ -483,6 +496,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
goto out_reset;
}
+ if (set_tracing_funcgraph_irqs(ftrace) < 0) {
+ pr_err("failed to set tracing option funcgraph-irqs\n");
+ goto out_reset;
+ }
+
if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
goto out_reset;
@@ -662,6 +680,7 @@ static int parse_graph_tracer_opts(const struct option *opt,
struct perf_ftrace *ftrace = (struct perf_ftrace *) opt->value;
struct sublevel_option graph_tracer_opts[] = {
{ .name = "nosleep-time", .value_ptr = &ftrace->graph_nosleep_time },
+ { .name = "noirqs", .value_ptr = &ftrace->graph_noirqs },
{ .name = NULL, }
};
@@ -733,7 +752,7 @@ int cmd_ftrace(int argc, const char **argv)
OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
"Max depth for function graph tracer"),
OPT_CALLBACK(0, "graph-opts", &ftrace, "options",
- "graph tracer options, available options: nosleep-time",
+ "graph tracer options, available options: nosleep-time,noirqs",
parse_graph_tracer_opts),
OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
"size of per cpu buffer", parse_buffer_size),
--
2.25.1
This adds support to display irq context info for function tracer. To do
this, just specify a '--func-opts irq-info' option.
Signed-off-by: Changbin Du <[email protected]>
---
tools/perf/Documentation/perf-ftrace.txt | 1 +
tools/perf/builtin-ftrace.c | 21 ++++++++++++++++++++-
2 files changed, 21 insertions(+), 1 deletion(-)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index fca55ac55ff3..37ea3ea97922 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -79,6 +79,7 @@ OPTIONS
--func-opts::
List of options allowed to set:
call-graph - Display kernel stack trace for function tracer.
+ irq-info - Display irq context info for function tracer.
-G::
--graph-funcs=::
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index b4c821be4fb5..d1241febe143 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -44,6 +44,7 @@ struct perf_ftrace {
unsigned long percpu_buffer_size;
bool inherit;
int func_stack_trace;
+ int func_irq_info;
int graph_nosleep_time;
int graph_noirqs;
};
@@ -209,6 +210,7 @@ static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
write_tracing_option_file("func_stack_trace", "0");
write_tracing_option_file("sleep-time", "1");
write_tracing_option_file("funcgraph-irqs", "1");
+ write_tracing_option_file("irq-info", "0");
}
static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
@@ -296,6 +298,17 @@ static int set_tracing_func_stack_trace(struct perf_ftrace *ftrace)
return 0;
}
+static int set_tracing_func_irqinfo(struct perf_ftrace *ftrace)
+{
+ if (!ftrace->func_irq_info)
+ return 0;
+
+ if (write_tracing_option_file("irq-info", "1") < 0)
+ return -1;
+
+ return 0;
+}
+
static int reset_tracing_cpu(void)
{
struct perf_cpu_map *cpumap = perf_cpu_map__new(NULL);
@@ -471,6 +484,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
goto out_reset;
}
+ if (set_tracing_func_irqinfo(ftrace) < 0) {
+ pr_err("failed to set tracing option irq-info\n");
+ goto out_reset;
+ }
+
if (set_tracing_filters(ftrace) < 0) {
pr_err("failed to set tracing filters\n");
goto out_reset;
@@ -660,6 +678,7 @@ static int parse_func_tracer_opts(const struct option *opt,
struct perf_ftrace *ftrace = (struct perf_ftrace *) opt->value;
struct sublevel_option func_tracer_opts[] = {
{ .name = "call-graph", .value_ptr = &ftrace->func_stack_trace },
+ { .name = "irq-info", .value_ptr = &ftrace->func_irq_info },
{ .name = NULL, }
};
@@ -742,7 +761,7 @@ int cmd_ftrace(int argc, const char **argv)
OPT_CALLBACK('N', "notrace-funcs", &ftrace.notrace, "func",
"do not trace given functions", parse_filter_func),
OPT_CALLBACK(0, "func-opts", &ftrace, "options",
- "function tracer options, available options: call-graph",
+ "function tracer options, available options: call-graph,irq-info",
parse_func_tracer_opts),
OPT_CALLBACK('G', "graph-funcs", &ftrace.graph_funcs, "func",
"trace given functions using function_graph tracer",
--
2.25.1
Sometimes we want ftrace display more and longer information about
the trace.
$ sudo perf ftrace -G '*'
2) 0.979 us | mutex_unlock();
2) 1.540 us | __fsnotify_parent();
2) 0.433 us | fsnotify();
$ sudo perf ftrace -G '*' --graph-opts verbose
14160.770883 | 0) <...>-47814 | .... | 1.289 us | mutex_unlock();
14160.770886 | 0) <...>-47814 | .... | 1.624 us | __fsnotify_parent();
14160.770887 | 0) <...>-47814 | .... | 0.636 us | fsnotify();
14160.770888 | 0) <...>-47814 | .... | 0.328 us | __sb_end_write();
14160.770888 | 0) <...>-47814 | d... | 0.430 us | fpregs_assert_state_consistent();
14160.770889 | 0) <...>-47814 | d... | | do_syscall_64() {
14160.770889 | 0) <...>-47814 | .... | | __x64_sys_close() {
Signed-off-by: Changbin Du <[email protected]>
---
tools/perf/Documentation/perf-ftrace.txt | 1 +
tools/perf/builtin-ftrace.c | 29 +++++++++++++++++++++++-
2 files changed, 29 insertions(+), 1 deletion(-)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index 37ea3ea97922..a2056aaf2ece 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -106,6 +106,7 @@ OPTIONS
List of options allowed to set:
nosleep-time - Measure on-CPU time only for function_graph tracer.
noirqs - Ignore functions that happen inside interrupt.
+ verbose - Show process names, PIDs, timestamps, etc.
SEE ALSO
--------
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index d1241febe143..8ee5287bd84a 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -47,6 +47,7 @@ struct perf_ftrace {
int func_irq_info;
int graph_nosleep_time;
int graph_noirqs;
+ int graph_verbose;
};
struct filter_entry {
@@ -210,6 +211,9 @@ static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
write_tracing_option_file("func_stack_trace", "0");
write_tracing_option_file("sleep-time", "1");
write_tracing_option_file("funcgraph-irqs", "1");
+ write_tracing_option_file("funcgraph-proc", "0");
+ write_tracing_option_file("funcgraph-abstime", "0");
+ write_tracing_option_file("latency-format", "0");
write_tracing_option_file("irq-info", "0");
}
@@ -425,6 +429,23 @@ static int set_tracing_funcgraph_irqs(struct perf_ftrace *ftrace)
return 0;
}
+static int set_tracing_funcgraph_verbose(struct perf_ftrace *ftrace)
+{
+ if (!ftrace->graph_verbose)
+ return 0;
+
+ if (write_tracing_option_file("funcgraph-proc", "1") < 0)
+ return -1;
+
+ if (write_tracing_option_file("funcgraph-abstime", "1") < 0)
+ return -1;
+
+ if (write_tracing_option_file("latency-format", "1") < 0)
+ return -1;
+
+ return 0;
+}
+
static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
{
char *trace_file;
@@ -519,6 +540,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
goto out_reset;
}
+ if (set_tracing_funcgraph_verbose(ftrace) < 0) {
+ pr_err("failed to set tracing option funcgraph-proc/funcgraph-abstime\n");
+ goto out_reset;
+ }
+
if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
goto out_reset;
@@ -700,6 +726,7 @@ static int parse_graph_tracer_opts(const struct option *opt,
struct sublevel_option graph_tracer_opts[] = {
{ .name = "nosleep-time", .value_ptr = &ftrace->graph_nosleep_time },
{ .name = "noirqs", .value_ptr = &ftrace->graph_noirqs },
+ { .name = "verbose", .value_ptr = &ftrace->graph_verbose },
{ .name = NULL, }
};
@@ -771,7 +798,7 @@ int cmd_ftrace(int argc, const char **argv)
OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
"Max depth for function graph tracer"),
OPT_CALLBACK(0, "graph-opts", &ftrace, "options",
- "graph tracer options, available options: nosleep-time,noirqs",
+ "graph tracer options, available options: nosleep-time,noirqs,verbose",
parse_graph_tracer_opts),
OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
"size of per cpu buffer", parse_buffer_size),
--
2.25.1
This factors out a general function perf_parse_sublevel_options() to parse
sublevel options. The 'sublevel' options is something like the '--debug'
options which allow more sublevel options.
Signed-off-by: Changbin Du <[email protected]>
---
v2: add util/parse-sublevel-options.c
---
tools/perf/util/Build | 1 +
tools/perf/util/debug.c | 61 ++++++---------------
tools/perf/util/parse-sublevel-options.c | 70 ++++++++++++++++++++++++
tools/perf/util/parse-sublevel-options.h | 11 ++++
4 files changed, 99 insertions(+), 44 deletions(-)
create mode 100644 tools/perf/util/parse-sublevel-options.c
create mode 100644 tools/perf/util/parse-sublevel-options.h
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index 8d18380ecd10..e86607ada0b5 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -117,6 +117,7 @@ endif
perf-y += parse-branch-options.o
perf-y += dump-insn.o
perf-y += parse-regs-options.o
+perf-y += parse-sublevel-options.o
perf-y += term.o
perf-y += help-unknown-cmd.o
perf-y += mem-events.o
diff --git a/tools/perf/util/debug.c b/tools/perf/util/debug.c
index adb656745ecc..5cda5565777a 100644
--- a/tools/perf/util/debug.c
+++ b/tools/perf/util/debug.c
@@ -20,6 +20,7 @@
#include "target.h"
#include "ui/helpline.h"
#include "ui/ui.h"
+#include "util/parse-sublevel-options.h"
#include <linux/ctype.h>
@@ -173,65 +174,37 @@ void trace_event(union perf_event *event)
trace_event_printer, event);
}
-static struct debug_variable {
- const char *name;
- int *ptr;
-} debug_variables[] = {
- { .name = "verbose", .ptr = &verbose },
- { .name = "ordered-events", .ptr = &debug_ordered_events},
- { .name = "stderr", .ptr = &redirect_to_stderr},
- { .name = "data-convert", .ptr = &debug_data_convert },
- { .name = "perf-event-open", .ptr = &debug_peo_args },
+static struct sublevel_option debug_opts[] = {
+ { .name = "verbose", .value_ptr = &verbose },
+ { .name = "ordered-events", .value_ptr = &debug_ordered_events},
+ { .name = "stderr", .value_ptr = &redirect_to_stderr},
+ { .name = "data-convert", .value_ptr = &debug_data_convert },
+ { .name = "perf-event-open", .value_ptr = &debug_peo_args },
{ .name = NULL, }
};
int perf_debug_option(const char *str)
{
- struct debug_variable *var = &debug_variables[0];
- char *vstr, *s = strdup(str);
- int v = 1;
-
- vstr = strchr(s, '=');
- if (vstr)
- *vstr++ = 0;
-
- while (var->name) {
- if (!strcmp(s, var->name))
- break;
- var++;
- }
-
- if (!var->name) {
- pr_err("Unknown debug variable name '%s'\n", s);
- free(s);
- return -1;
- }
+ int ret;
- if (vstr) {
- v = atoi(vstr);
- /*
- * Allow only values in range (0, 10),
- * otherwise set 0.
- */
- v = (v < 0) || (v > 10) ? 0 : v;
- }
+ ret = perf_parse_sublevel_options(str, debug_opts);
+ if (ret)
+ return ret;
- if (quiet)
- v = -1;
+ /* Allow only verbose value in range (0, 10), otherwise set 0. */
+ verbose = (verbose < 0) || (verbose > 10) ? 0 : verbose;
- *var->ptr = v;
- free(s);
return 0;
}
int perf_quiet_option(void)
{
- struct debug_variable *var = &debug_variables[0];
+ struct sublevel_option *opt = &debug_opts[0];
/* disable all debug messages */
- while (var->name) {
- *var->ptr = -1;
- var++;
+ while (opt->name) {
+ *opt->value_ptr = -1;
+ opt++;
}
return 0;
diff --git a/tools/perf/util/parse-sublevel-options.c b/tools/perf/util/parse-sublevel-options.c
new file mode 100644
index 000000000000..a841d17ffd57
--- /dev/null
+++ b/tools/perf/util/parse-sublevel-options.c
@@ -0,0 +1,70 @@
+#include <stdlib.h>
+#include <stdint.h>
+#include <string.h>
+#include <stdio.h>
+
+#include "util/debug.h"
+#include "util/parse-sublevel-options.h"
+
+static int parse_one_sublevel_option(const char *str,
+ struct sublevel_option *opts)
+{
+ struct sublevel_option *opt = opts;
+ char *vstr, *s = strdup(str);
+ int v = 1;
+
+ if (!s) {
+ pr_err("no memory\n");
+ return -1;
+ }
+
+ vstr = strchr(s, '=');
+ if (vstr)
+ *vstr++ = 0;
+
+ while (opt->name) {
+ if (!strcmp(s, opt->name))
+ break;
+ opt++;
+ }
+
+ if (!opt->name) {
+ pr_err("Unknown option name '%s'\n", s);
+ free(s);
+ return -1;
+ }
+
+ if (vstr)
+ v = atoi(vstr);
+
+ *opt->value_ptr = v;
+ free(s);
+ return 0;
+}
+
+/* parse options like --foo a=<n>,b,c... */
+int perf_parse_sublevel_options(const char *str, struct sublevel_option *opts)
+{
+ char *s = strdup(str);
+ char *p = NULL;
+ int ret;
+
+ if (!s) {
+ pr_err("no memory\n");
+ return -1;
+ }
+
+ p = strtok(s, ",");
+ while (p) {
+ ret = parse_one_sublevel_option(p, opts);
+ if (ret) {
+ free(s);
+ return ret;
+ }
+
+ p = strtok(NULL, ",");
+ }
+
+ free(s);
+ return 0;
+}
diff --git a/tools/perf/util/parse-sublevel-options.h b/tools/perf/util/parse-sublevel-options.h
new file mode 100644
index 000000000000..9b9efcc2aaad
--- /dev/null
+++ b/tools/perf/util/parse-sublevel-options.h
@@ -0,0 +1,11 @@
+#ifndef _PERF_PARSE_SUBLEVEL_OPTIONS_H
+#define _PERF_PARSE_SUBLEVEL_OPTIONS_H
+
+struct sublevel_option {
+ const char *name;
+ int *value_ptr;
+};
+
+int perf_parse_sublevel_options(const char *str, struct sublevel_option *opts);
+
+#endif
\ No newline at end of file
--
2.25.1
This adds an option '-D/--delay' to allow us to start tracing some
times later after workload is launched.
Signed-off-by: Changbin Du <[email protected]>
---
tools/perf/Documentation/perf-ftrace.txt | 4 ++++
tools/perf/builtin-ftrace.c | 19 ++++++++++++++++---
2 files changed, 20 insertions(+), 3 deletions(-)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index 08216634d2bc..29c684b3b3fd 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -39,6 +39,10 @@ OPTIONS
--pid=::
Trace on existing process id (comma separated list).
+-D::
+--delay::
+ Time (ms) to wait before starting tracing after program start.
+
-a::
--all-cpus::
Force system-wide collection. Scripts run without a <command>
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index bc3b35d18167..5f9a9ebea0a2 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -49,6 +49,7 @@ struct perf_ftrace {
int graph_noirqs;
int graph_verbose;
int graph_thresh;
+ unsigned int initial_delay;
};
struct filter_entry {
@@ -596,13 +597,23 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
/* display column headers */
read_tracing_file_to_stdout("trace");
- if (write_tracing_file("tracing_on", "1") < 0) {
- pr_err("can't enable tracing\n");
- goto out_close_fd;
+ if (!ftrace->initial_delay) {
+ if (write_tracing_file("tracing_on", "1") < 0) {
+ pr_err("can't enable tracing\n");
+ goto out_close_fd;
+ }
}
perf_evlist__start_workload(ftrace->evlist);
+ if (ftrace->initial_delay) {
+ usleep(ftrace->initial_delay * 1000);
+ if (write_tracing_file("tracing_on", "1") < 0) {
+ pr_err("can't enable tracing\n");
+ goto out_close_fd;
+ }
+ }
+
while (!done) {
if (poll(&pollfd, 1, -1) < 0)
break;
@@ -827,6 +838,8 @@ int cmd_ftrace(int argc, const char **argv)
"size of per cpu buffer", parse_buffer_size),
OPT_BOOLEAN(0, "inherit", &ftrace.inherit,
"trace children processes"),
+ OPT_UINTEGER('D', "delay", &ftrace.initial_delay,
+ "ms to wait before starting tracing after program start"),
OPT_END()
};
--
2.25.1
Add a change log after previous enhancements.
Signed-off-by: Changbin Du <[email protected]>
---
tools/perf/builtin-ftrace.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index 9da38dc8b542..167f1f57aba3 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -3,6 +3,7 @@
* builtin-ftrace.c
*
* Copyright (c) 2013 LG Electronics, Namhyung Kim <[email protected]>
+ * Copyright (c) 2020 Changbin Du <[email protected]>, significant enhancement.
*/
#include "builtin.h"
--
2.25.1
This adds an option '--graph-opts thresh' to setup trace duration
threshold for funcgraph tracer.
$ sudo ./perf ftrace -G '*' --graph-opts thresh=100
3) ! 184.060 us | } /* schedule */
3) ! 185.600 us | } /* exit_to_usermode_loop */
2) ! 225.989 us | } /* schedule_idle */
2) # 4140.051 us | } /* do_idle */
Signed-off-by: Changbin Du <[email protected]>
---
tools/perf/Documentation/perf-ftrace.txt | 1 +
tools/perf/builtin-ftrace.c | 26 +++++++++++++++++++++++-
2 files changed, 26 insertions(+), 1 deletion(-)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index a2056aaf2ece..6fa927e5971b 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -107,6 +107,7 @@ OPTIONS
nosleep-time - Measure on-CPU time only for function_graph tracer.
noirqs - Ignore functions that happen inside interrupt.
verbose - Show process names, PIDs, timestamps, etc.
+ thresh=<n> - Setup trace duration threshold in microseconds.
SEE ALSO
--------
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index 8ee5287bd84a..ca61f8b8bbf6 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -48,6 +48,7 @@ struct perf_ftrace {
int graph_nosleep_time;
int graph_noirqs;
int graph_verbose;
+ int graph_thresh;
};
struct filter_entry {
@@ -234,6 +235,9 @@ static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
if (write_tracing_file("max_graph_depth", "0") < 0)
return -1;
+ if (write_tracing_file("tracing_thresh", "0") < 0)
+ return -1;
+
reset_tracing_filters();
reset_tracing_options(ftrace);
return 0;
@@ -446,6 +450,20 @@ static int set_tracing_funcgraph_verbose(struct perf_ftrace *ftrace)
return 0;
}
+static int set_tracing_thresh(struct perf_ftrace *ftrace)
+{
+ int ret;
+
+ if (ftrace->graph_thresh == 0)
+ return 0;
+
+ ret = write_tracing_file_int("tracing_thresh", ftrace->graph_thresh);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
+
static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
{
char *trace_file;
@@ -545,6 +563,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
goto out_reset;
}
+ if (set_tracing_thresh(ftrace) < 0) {
+ pr_err("failed to set tracing thresh\n");
+ goto out_reset;
+ }
+
if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
goto out_reset;
@@ -727,6 +750,7 @@ static int parse_graph_tracer_opts(const struct option *opt,
{ .name = "nosleep-time", .value_ptr = &ftrace->graph_nosleep_time },
{ .name = "noirqs", .value_ptr = &ftrace->graph_noirqs },
{ .name = "verbose", .value_ptr = &ftrace->graph_verbose },
+ { .name = "thresh", .value_ptr = &ftrace->graph_thresh },
{ .name = NULL, }
};
@@ -798,7 +822,7 @@ int cmd_ftrace(int argc, const char **argv)
OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
"Max depth for function graph tracer"),
OPT_CALLBACK(0, "graph-opts", &ftrace, "options",
- "graph tracer options, available options: nosleep-time,noirqs,verbose",
+ "graph tracer options, available options: nosleep-time,noirqs,verbose,thresh=<n>",
parse_graph_tracer_opts),
OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
"size of per cpu buffer", parse_buffer_size),
--
2.25.1
This is to have a consistent view of all graph tracer options.
The original option '--graph-depth' is marked as deprecated.
Signed-off-by: Changbin Du <[email protected]>
---
tools/perf/Documentation/perf-ftrace.txt | 5 +----
tools/perf/builtin-ftrace.c | 5 ++---
2 files changed, 3 insertions(+), 7 deletions(-)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index 6fa927e5971b..08216634d2bc 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -98,16 +98,13 @@ OPTIONS
specify multiple functions. It will be passed to 'set_graph_notrace'
in tracefs.
--D::
---graph-depth=::
- Set max depth for function graph tracer to follow
-
--graph-opts::
List of options allowed to set:
nosleep-time - Measure on-CPU time only for function_graph tracer.
noirqs - Ignore functions that happen inside interrupt.
verbose - Show process names, PIDs, timestamps, etc.
thresh=<n> - Setup trace duration threshold in microseconds.
+ depth=<n> - Set max depth for function graph tracer to follow.
SEE ALSO
--------
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index ca61f8b8bbf6..bc3b35d18167 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -751,6 +751,7 @@ static int parse_graph_tracer_opts(const struct option *opt,
{ .name = "noirqs", .value_ptr = &ftrace->graph_noirqs },
{ .name = "verbose", .value_ptr = &ftrace->graph_verbose },
{ .name = "thresh", .value_ptr = &ftrace->graph_thresh },
+ { .name = "depth", .value_ptr = &ftrace->graph_depth },
{ .name = NULL, }
};
@@ -819,10 +820,8 @@ int cmd_ftrace(int argc, const char **argv)
parse_filter_func),
OPT_CALLBACK('g', "nograph-funcs", &ftrace.nograph_funcs, "func",
"Set nograph filter on given functions", parse_filter_func),
- OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
- "Max depth for function graph tracer"),
OPT_CALLBACK(0, "graph-opts", &ftrace, "options",
- "graph tracer options, available options: nosleep-time,noirqs,verbose,thresh=<n>",
+ "graph tracer options, available options: nosleep-time,noirqs,verbose,thresh=<n>,depth=<n>",
parse_graph_tracer_opts),
OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
"size of per cpu buffer", parse_buffer_size),
--
2.25.1
This allows us to trace single thread instead of the whole process.
Signed-off-by: Changbin Du <[email protected]>
---
tools/perf/Documentation/perf-ftrace.txt | 3 +++
tools/perf/builtin-ftrace.c | 3 +++
2 files changed, 6 insertions(+)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index 29c684b3b3fd..78358af9a1c4 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -39,6 +39,9 @@ OPTIONS
--pid=::
Trace on existing process id (comma separated list).
+--tid=::
+ Trace on existing thread id (comma separated list).
+
-D::
--delay::
Time (ms) to wait before starting tracing after program start.
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index 5f9a9ebea0a2..13abb737fd12 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -812,6 +812,9 @@ int cmd_ftrace(int argc, const char **argv)
"Show available functions to filter"),
OPT_STRING('p', "pid", &ftrace.target.pid, "pid",
"trace on existing process id"),
+ /* TODO: Add short option -t after -t/--tracer can be removed. */
+ OPT_STRING(0, "tid", &ftrace.target.tid, "tid",
+ "trace on existing thread id (exclusive to --pid)"),
OPT_INCR('v', "verbose", &verbose,
"be more verbose"),
OPT_BOOLEAN('a', "all-cpus", &ftrace.target.system_wide,
--
2.25.1
Now the __cmd_ftrace() becomes a bit long. This moves the trace
option setting code to a separate function set_tracing_options().
Suggested-by: Namhyung Kim <[email protected]>
Signed-off-by: Changbin Du <[email protected]>
---
tools/perf/builtin-ftrace.c | 118 +++++++++++++++++++-----------------
1 file changed, 63 insertions(+), 55 deletions(-)
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index 13abb737fd12..9da38dc8b542 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -465,110 +465,118 @@ static int set_tracing_thresh(struct perf_ftrace *ftrace)
return 0;
}
-static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
+static int set_tracing_options(struct perf_ftrace *ftrace)
{
- char *trace_file;
- int trace_fd;
- char buf[4096];
- struct pollfd pollfd = {
- .events = POLLIN,
- };
-
- if (!(perf_cap__capable(CAP_PERFMON) ||
- perf_cap__capable(CAP_SYS_ADMIN))) {
- pr_err("ftrace only works for %s!\n",
-#ifdef HAVE_LIBCAP_SUPPORT
- "users with the CAP_PERFMON or CAP_SYS_ADMIN capability"
-#else
- "root"
-#endif
- );
- return -1;
- }
-
- signal(SIGINT, sig_handler);
- signal(SIGUSR1, sig_handler);
- signal(SIGCHLD, sig_handler);
- signal(SIGPIPE, sig_handler);
-
- if (ftrace->list_avail_functions)
- return read_tracing_file_to_stdout("available_filter_functions");
-
- if (reset_tracing_files(ftrace) < 0) {
- pr_err("failed to reset ftrace\n");
- goto out;
- }
-
- /* reset ftrace buffer */
- if (write_tracing_file("trace", "0") < 0)
- goto out;
-
- if (argc && perf_evlist__prepare_workload(ftrace->evlist,
- &ftrace->target, argv, false,
- ftrace__workload_exec_failed_signal) < 0) {
- goto out;
- }
-
if (set_tracing_pid(ftrace) < 0) {
pr_err("failed to set ftrace pid\n");
- goto out_reset;
+ return -1;
}
if (set_tracing_cpu(ftrace) < 0) {
pr_err("failed to set tracing cpumask\n");
- goto out_reset;
+ return -1;
}
if (set_tracing_func_stack_trace(ftrace) < 0) {
pr_err("failed to set tracing option func_stack_trace\n");
- goto out_reset;
+ return -1;
}
if (set_tracing_func_irqinfo(ftrace) < 0) {
pr_err("failed to set tracing option irq-info\n");
- goto out_reset;
+ return -1;
}
if (set_tracing_filters(ftrace) < 0) {
pr_err("failed to set tracing filters\n");
- goto out_reset;
+ return -1;
}
if (set_tracing_depth(ftrace) < 0) {
pr_err("failed to set graph depth\n");
- goto out_reset;
+ return -1;
}
if (set_tracing_percpu_buffer_size(ftrace) < 0) {
pr_err("failed to set tracing per-cpu buffer size\n");
- goto out_reset;
+ return -1;
}
if (set_tracing_trace_inherit(ftrace) < 0) {
pr_err("failed to set tracing option function-fork\n");
- goto out_reset;
+ return -1;
}
if (set_tracing_sleep_time(ftrace) < 0) {
pr_err("failed to set tracing option sleep-time\n");
- goto out_reset;
+ return -1;
}
if (set_tracing_funcgraph_irqs(ftrace) < 0) {
pr_err("failed to set tracing option funcgraph-irqs\n");
- goto out_reset;
+ return -1;
}
if (set_tracing_funcgraph_verbose(ftrace) < 0) {
pr_err("failed to set tracing option funcgraph-proc/funcgraph-abstime\n");
- goto out_reset;
+ return -1;
}
if (set_tracing_thresh(ftrace) < 0) {
pr_err("failed to set tracing thresh\n");
- goto out_reset;
+ return -1;
+ }
+
+ return 0;
+}
+
+static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
+{
+ char *trace_file;
+ int trace_fd;
+ char buf[4096];
+ struct pollfd pollfd = {
+ .events = POLLIN,
+ };
+
+ if (!(perf_cap__capable(CAP_PERFMON) ||
+ perf_cap__capable(CAP_SYS_ADMIN))) {
+ pr_err("ftrace only works for %s!\n",
+#ifdef HAVE_LIBCAP_SUPPORT
+ "users with the CAP_PERFMON or CAP_SYS_ADMIN capability"
+#else
+ "root"
+#endif
+ );
+ return -1;
}
+ signal(SIGINT, sig_handler);
+ signal(SIGUSR1, sig_handler);
+ signal(SIGCHLD, sig_handler);
+ signal(SIGPIPE, sig_handler);
+
+ if (ftrace->list_avail_functions)
+ return read_tracing_file_to_stdout("available_filter_functions");
+
+ if (reset_tracing_files(ftrace) < 0) {
+ pr_err("failed to reset ftrace\n");
+ goto out;
+ }
+
+ /* reset ftrace buffer */
+ if (write_tracing_file("trace", "0") < 0)
+ goto out;
+
+ if (argc && perf_evlist__prepare_workload(ftrace->evlist,
+ &ftrace->target, argv, false,
+ ftrace__workload_exec_failed_signal) < 0) {
+ goto out;
+ }
+
+ if (set_tracing_options(ftrace) < 0)
+ goto out_reset;
+
if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
goto out_reset;
--
2.25.1
Em Sat, Aug 08, 2020 at 10:31:25AM +0800, Changbin Du escreveu:
> This adds an option '-F/--funcs' to list all available functions to trace,
> which is read from tracing file 'available_filter_functions'.
>
> $ sudo ./perf ftrace -F | head
> trace_initcall_finish_cb
> initcall_blacklisted
> do_one_initcall
> do_one_initcall
> trace_initcall_start_cb
> run_init_process
> try_to_run_init_process
> match_dev_by_label
> match_dev_by_uuid
> rootfs_init_fs_context
>
> Signed-off-by: Changbin Du <[email protected]>
>
> ---
> v3: fix return value issue.
> v2: option name '-l/--list-functions' -> '-F/--funcs'
> ---
> tools/perf/Documentation/perf-ftrace.txt | 4 +++
> tools/perf/builtin-ftrace.c | 46 ++++++++++++++++++++++++
> 2 files changed, 50 insertions(+)
>
> diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> index 821d4d334a09..4f5628445a63 100644
> --- a/tools/perf/Documentation/perf-ftrace.txt
> +++ b/tools/perf/Documentation/perf-ftrace.txt
> @@ -31,6 +31,10 @@ OPTIONS
> --verbose=::
> Verbosity level.
>
> +-F::
> +--funcs::
> + List all available functions to trace.
> +
> -p::
> --pid=::
> Trace on existing process id (comma separated list).
> diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> index 048a11192b59..1ebf71ecc1a8 100644
> --- a/tools/perf/builtin-ftrace.c
> +++ b/tools/perf/builtin-ftrace.c
> @@ -33,6 +33,7 @@ struct perf_ftrace {
> struct evlist *evlist;
> struct target target;
> const char *tracer;
> + bool list_avail_functions;
> struct list_head filters;
> struct list_head notrace;
> struct list_head graph_funcs;
> @@ -128,6 +129,46 @@ static int append_tracing_file(const char *name, const char *val)
> return __write_tracing_file(name, val, true);
> }
>
> +static int read_tracing_file_to_stdout(const char *name)
> +{
> + char buf[4096];
> + char *file;
> + int fd;
> + int ret = -1;
> +
> + file = get_tracing_file(name);
> + if (!file) {
> + pr_debug("cannot get tracing file: %s\n", name);
> + return -1;
> + }
> +
> + fd = open(file, O_RDONLY);
> + if (fd < 0) {
> + pr_debug("cannot open tracing file: %s: %s\n",
> + name, str_error_r(errno, buf, sizeof(buf)));
> + goto out;
> + }
> +
> + /* read contents to stdout */
> + while (true) {
> + int n = read(fd, buf, sizeof(buf));
> + if (n == 0)
> + break;
> + else if (n < 0)
> + goto out_close;
> +
> + if (fwrite(buf, n, 1, stdout) != 1)
> + goto out_close;
> + }
> + ret = 0;
> +
> +out_close:
> + close(fd);
> +out:
> + put_tracing_file(file);
> + return ret;
> +}
> +
> static int reset_tracing_cpu(void);
> static void reset_tracing_filters(void);
>
> @@ -302,6 +343,9 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> signal(SIGCHLD, sig_handler);
> signal(SIGPIPE, sig_handler);
>
> + if (ftrace->list_avail_functions)
> + return read_tracing_file_to_stdout("available_filter_functions");
> +
> if (reset_tracing_files(ftrace) < 0) {
> pr_err("failed to reset ftrace\n");
> goto out;
> @@ -487,6 +531,8 @@ int cmd_ftrace(int argc, const char **argv)
> const struct option ftrace_options[] = {
> OPT_STRING('t', "tracer", &ftrace.tracer, "tracer",
> "tracer to use: function_graph(default) or function"),
> + OPT_BOOLEAN('F', "funcs", &ftrace.list_avail_functions,
> + "Show available functions to filter"),
A boolean? I guess this can be a first step, but why not do it like
'perf probe' and accept a patterna? Allows for less typing:
# perf probe -h -F
-F, --funcs <[FILTER]>
Show potential probe-able functions.
[root@quaco ~]#
# perf probe -F '*btf_parse*'
btf_parse_hdr
btf_parse_str_sec
btf_parse_vmlinux
#
# perf ftrace -F | grep btf_parse
btf_parse_str_sec
btf_parse_hdr
btf_parse_vmlinux
#
I'm applying this to make progress, but please consider sending a patch
to make this behave in the same way as 'perf probe'.
- Arnaldo
> OPT_STRING('p', "pid", &ftrace.target.pid, "pid",
> "trace on existing process id"),
> OPT_INCR('v', "verbose", &verbose,
> --
> 2.25.1
>
--
- Arnaldo
Em Fri, Aug 14, 2020 at 08:56:55AM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Sat, Aug 08, 2020 at 10:31:28AM +0800, Changbin Du escreveu:
> > This makes perf-ftrace display column header before printing trace.
> >
> > $ sudo perf ftrace
> > # tracer: function
> > #
> > # entries-in-buffer/entries-written: 0/0 #P:8
> > #
> > # TASK-PID CPU# TIMESTAMP FUNCTION
> > # | | | | |
> > <...>-9246 [006] 10726.262760: mutex_unlock <-rb_simple_write
> > <...>-9246 [006] 10726.262764: __fsnotify_parent <-vfs_write
> > <...>-9246 [006] 10726.262765: fsnotify <-vfs_write
> > <...>-9246 [006] 10726.262766: __sb_end_write <-vfs_write
> > <...>-9246 [006] 10726.262767: fpregs_assert_state_consistent <-do_syscall_64
>
> We have:
>
> # perf report -h header
>
> Usage: perf report [<options>]
>
> --header Show data header.
> --header-only Show only data header.
>
> #
>
> So perhaps we should have those as well, in the 'ftrace' case one would
> perhaps want to use --no-header, for instance, to do some scripting
> parsing just the trace lines.
This as well can be done after I process this series, to make progress.
Applied.
- Arnaldo
Em Sat, Aug 08, 2020 at 10:31:28AM +0800, Changbin Du escreveu:
> This makes perf-ftrace display column header before printing trace.
>
> $ sudo perf ftrace
> # tracer: function
> #
> # entries-in-buffer/entries-written: 0/0 #P:8
> #
> # TASK-PID CPU# TIMESTAMP FUNCTION
> # | | | | |
> <...>-9246 [006] 10726.262760: mutex_unlock <-rb_simple_write
> <...>-9246 [006] 10726.262764: __fsnotify_parent <-vfs_write
> <...>-9246 [006] 10726.262765: fsnotify <-vfs_write
> <...>-9246 [006] 10726.262766: __sb_end_write <-vfs_write
> <...>-9246 [006] 10726.262767: fpregs_assert_state_consistent <-do_syscall_64
We have:
# perf report -h header
Usage: perf report [<options>]
--header Show data header.
--header-only Show only data header.
#
So perhaps we should have those as well, in the 'ftrace' case one would
perhaps want to use --no-header, for instance, to do some scripting
parsing just the trace lines.
- Arnaldo
> Signed-off-by: Changbin Du <[email protected]>
> ---
> tools/perf/builtin-ftrace.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> index a3a4f4be9dde..39c694be2b71 100644
> --- a/tools/perf/builtin-ftrace.c
> +++ b/tools/perf/builtin-ftrace.c
> @@ -435,6 +435,9 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> fcntl(trace_fd, F_SETFL, O_NONBLOCK);
> pollfd.fd = trace_fd;
>
> + /* display column headers */
> + read_tracing_file_to_stdout("trace");
> +
> if (write_tracing_file("tracing_on", "1") < 0) {
> pr_err("can't enable tracing\n");
> goto out_close_fd;
> --
> 2.25.1
>
--
- Arnaldo
Em Sat, Aug 08, 2020 at 10:31:29AM +0800, Changbin Du escreveu:
> This adds an option '--inherit' to allow us trace children
> processes spawned by our target.
One thing I'm missing from your commit log messages is the exact
sequence of commands and the outputs for the options you're introducing.
- Arnaldo
> Signed-off-by: Changbin Du <[email protected]>
>
> ---
> v2: option name '--trace-children' -> '--inherit'.
> ---
> tools/perf/Documentation/perf-ftrace.txt | 3 ++
> tools/perf/builtin-ftrace.c | 38 ++++++++++++++++++++++++
> 2 files changed, 41 insertions(+)
>
> diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> index 7a5d915f60b0..c46d0a09b38c 100644
> --- a/tools/perf/Documentation/perf-ftrace.txt
> +++ b/tools/perf/Documentation/perf-ftrace.txt
> @@ -58,6 +58,9 @@ OPTIONS
> Set the size of per-cpu tracing buffer, <size> is expected to
> be a number with appended unit character - B/K/M/G.
>
> +--inherit::
> + Trace children processes spawned by our target.
> +
> -T::
> --trace-funcs=::
> Select function tracer and set function filter on the given
> diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> index 39c694be2b71..07b81d0c1658 100644
> --- a/tools/perf/builtin-ftrace.c
> +++ b/tools/perf/builtin-ftrace.c
> @@ -41,6 +41,7 @@ struct perf_ftrace {
> struct list_head nograph_funcs;
> int graph_depth;
> unsigned long percpu_buffer_size;
> + bool inherit;
> };
>
> struct filter_entry {
> @@ -182,9 +183,27 @@ static int write_tracing_file_int(const char *name, int value)
> return 0;
> }
>
> +static int write_tracing_option_file(const char *name, const char *val)
> +{
> + char *file;
> + int ret;
> +
> + if (asprintf(&file, "options/%s", name) < 0)
> + return -1;
> +
> + ret = __write_tracing_file(file, val, false);
> + free(file);
> + return ret;
> +}
> +
> static int reset_tracing_cpu(void);
> static void reset_tracing_filters(void);
>
> +static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
> +{
> + write_tracing_option_file("function-fork", "0");
> +}
> +
> static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
> {
> if (write_tracing_file("tracing_on", "0") < 0)
> @@ -203,6 +222,7 @@ static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
> return -1;
>
> reset_tracing_filters();
> + reset_tracing_options(ftrace);
> return 0;
> }
>
> @@ -341,6 +361,17 @@ static int set_tracing_percpu_buffer_size(struct perf_ftrace *ftrace)
> return 0;
> }
>
> +static int set_tracing_trace_inherit(struct perf_ftrace *ftrace)
> +{
> + if (!ftrace->inherit)
> + return 0;
> +
> + if (write_tracing_option_file("function-fork", "1") < 0)
> + return -1;
> +
> + return 0;
> +}
> +
> static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> {
> char *trace_file;
> @@ -410,6 +441,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> goto out_reset;
> }
>
> + if (set_tracing_trace_inherit(ftrace) < 0) {
> + pr_err("failed to set tracing option function-fork\n");
> + goto out_reset;
> + }
> +
> if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
> pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
> goto out_reset;
> @@ -618,6 +654,8 @@ int cmd_ftrace(int argc, const char **argv)
> "Max depth for function graph tracer"),
> OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
> "size of per cpu buffer", parse_buffer_size),
> + OPT_BOOLEAN(0, "inherit", &ftrace.inherit,
> + "trace children processes"),
> OPT_END()
> };
>
> --
> 2.25.1
>
--
- Arnaldo
Em Fri, Aug 14, 2020 at 08:53:18AM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Sat, Aug 08, 2020 at 10:31:27AM +0800, Changbin Du escreveu:
> > This adds an option '-m/--buffer-size' to allow us set the size of per-cpu
> > tracing buffer.
> >
> > Signed-off-by: Changbin Du <[email protected]>
>
> So this is a little bit confusing, i.e. if I don't specify a suffix (B,
> K, M, G) I get:
>
> # perf ftrace -m 2048
>
> Usage: perf ftrace [<options>] [<command>]
> or: perf ftrace [<options>] -- <command> [<options>]
>
> -m, --buffer-size <size>
> size of per cpu buffer
> #
>
> Which is not very helpful, and you also forgot to add an entry to 'man
> perf-ftrace' (tools/perf/Documentation/perf-ftrace.txt), so only by
I must be blind, the entry _was_ added to perf-ftrace.txt, nevermind :-)
But the other points still stand :)
> looking at the patch I was able to figure out that a suffix is
> required.
>
> Now look how 'perf trace' works:
>
> # perf trace -m 2048 -e open* --max-events 1
> 0.772 sh/343169 openat(dfd: CWD, filename: /etc/ld.so.cache, flags: RDONLY|CLOEXEC) = 3
> #
>
> I.e. it doesn't require a unit suffix, which is more natural.
>
> # perf trace -h -m
>
> -m, --mmap-pages <pages>
> number of mmap data pages
>
> #
>
> It indicates that the argument is in units of 'pages', while you don't
> state that in:
>
> $ perf ftrace -h -m
>
> -m, --buffer-size <size>
> size of per cpu buffer
>
> $
>
> So please improve the option description and add its longer explanation
> in the man page, also I suggest you interpret:
>
> # perf ftrace -m 2048
>
> as asking for 2048K since the file as _kb in its name, so people used to
> ftrace will get what they ask for, i.e. N kb. Please add this to the one
> line description in 'perf ftrace -h', namely that the default unit os
> 'K'.
>
> All this can be made on top of this series, so I'm applying it now to
> make progress.
>
> - Arnaldo
>
> > ---
> > v2: support units as a suffix.
> > ---
> > tools/perf/Documentation/perf-ftrace.txt | 5 +++
> > tools/perf/builtin-ftrace.c | 55 ++++++++++++++++++++++++
> > 2 files changed, 60 insertions(+)
> >
> > diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> > index 4f5628445a63..7a5d915f60b0 100644
> > --- a/tools/perf/Documentation/perf-ftrace.txt
> > +++ b/tools/perf/Documentation/perf-ftrace.txt
> > @@ -53,6 +53,11 @@ OPTIONS
> > Ranges of CPUs are specified with -: 0-2.
> > Default is to trace on all online CPUs.
> >
> > +-m::
> > +--buffer-size::
> > + Set the size of per-cpu tracing buffer, <size> is expected to
> > + be a number with appended unit character - B/K/M/G.
> > +
> > -T::
> > --trace-funcs=::
> > Select function tracer and set function filter on the given
> > diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> > index 4b3fcee5725a..a3a4f4be9dde 100644
> > --- a/tools/perf/builtin-ftrace.c
> > +++ b/tools/perf/builtin-ftrace.c
> > @@ -26,6 +26,7 @@
> > #include "thread_map.h"
> > #include "util/cap.h"
> > #include "util/config.h"
> > +#include "util/units.h"
> >
> > #define DEFAULT_TRACER "function_graph"
> >
> > @@ -39,6 +40,7 @@ struct perf_ftrace {
> > struct list_head graph_funcs;
> > struct list_head nograph_funcs;
> > int graph_depth;
> > + unsigned long percpu_buffer_size;
> > };
> >
> > struct filter_entry {
> > @@ -324,6 +326,21 @@ static int set_tracing_depth(struct perf_ftrace *ftrace)
> > return 0;
> > }
> >
> > +static int set_tracing_percpu_buffer_size(struct perf_ftrace *ftrace)
> > +{
> > + int ret;
> > +
> > + if (ftrace->percpu_buffer_size == 0)
> > + return 0;
> > +
> > + ret = write_tracing_file_int("buffer_size_kb",
> > + ftrace->percpu_buffer_size / 1024);
> > + if (ret < 0)
> > + return ret;
> > +
> > + return 0;
> > +}
> > +
> > static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> > {
> > char *trace_file;
> > @@ -388,6 +405,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> > goto out_reset;
> > }
> >
> > + if (set_tracing_percpu_buffer_size(ftrace) < 0) {
> > + pr_err("failed to set tracing per-cpu buffer size\n");
> > + goto out_reset;
> > + }
> > +
> > if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
> > pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
> > goto out_reset;
> > @@ -506,6 +528,37 @@ static void delete_filter_func(struct list_head *head)
> > }
> > }
> >
> > +static int parse_buffer_size(const struct option *opt,
> > + const char *str, int unset)
> > +{
> > + unsigned long *s = (unsigned long *)opt->value;
> > + static struct parse_tag tags_size[] = {
> > + { .tag = 'B', .mult = 1 },
> > + { .tag = 'K', .mult = 1 << 10 },
> > + { .tag = 'M', .mult = 1 << 20 },
> > + { .tag = 'G', .mult = 1 << 30 },
> > + { .tag = 0 },
> > + };
> > + unsigned long val;
> > +
> > + if (unset) {
> > + *s = 0;
> > + return 0;
> > + }
> > +
> > + val = parse_tag_value(str, tags_size);
> > + if (val != (unsigned long) -1) {
> > + if (val < 1024) {
> > + pr_err("buffer size too small, must larger than 1KB.");
> > + return -1;
> > + }
> > + *s = val;
> > + return 0;
> > + }
> > +
> > + return -1;
> > +}
> > +
> > static void select_tracer(struct perf_ftrace *ftrace)
> > {
> > bool graph = !list_empty(&ftrace->graph_funcs) ||
> > @@ -560,6 +613,8 @@ int cmd_ftrace(int argc, const char **argv)
> > "Set nograph filter on given functions", parse_filter_func),
> > OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
> > "Max depth for function graph tracer"),
> > + OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
> > + "size of per cpu buffer", parse_buffer_size),
> > OPT_END()
> > };
> >
> > --
> > 2.25.1
> >
>
> --
>
> - Arnaldo
--
- Arnaldo
Em Sat, Aug 08, 2020 at 10:31:27AM +0800, Changbin Du escreveu:
> This adds an option '-m/--buffer-size' to allow us set the size of per-cpu
> tracing buffer.
>
> Signed-off-by: Changbin Du <[email protected]>
So this is a little bit confusing, i.e. if I don't specify a suffix (B,
K, M, G) I get:
# perf ftrace -m 2048
Usage: perf ftrace [<options>] [<command>]
or: perf ftrace [<options>] -- <command> [<options>]
-m, --buffer-size <size>
size of per cpu buffer
#
Which is not very helpful, and you also forgot to add an entry to 'man
perf-ftrace' (tools/perf/Documentation/perf-ftrace.txt), so only by
looking at the patch I was able to figure out that a suffix is
required.
Now look how 'perf trace' works:
# perf trace -m 2048 -e open* --max-events 1
0.772 sh/343169 openat(dfd: CWD, filename: /etc/ld.so.cache, flags: RDONLY|CLOEXEC) = 3
#
I.e. it doesn't require a unit suffix, which is more natural.
# perf trace -h -m
-m, --mmap-pages <pages>
number of mmap data pages
#
It indicates that the argument is in units of 'pages', while you don't
state that in:
$ perf ftrace -h -m
-m, --buffer-size <size>
size of per cpu buffer
$
So please improve the option description and add its longer explanation
in the man page, also I suggest you interpret:
# perf ftrace -m 2048
as asking for 2048K since the file as _kb in its name, so people used to
ftrace will get what they ask for, i.e. N kb. Please add this to the one
line description in 'perf ftrace -h', namely that the default unit os
'K'.
All this can be made on top of this series, so I'm applying it now to
make progress.
- Arnaldo
> ---
> v2: support units as a suffix.
> ---
> tools/perf/Documentation/perf-ftrace.txt | 5 +++
> tools/perf/builtin-ftrace.c | 55 ++++++++++++++++++++++++
> 2 files changed, 60 insertions(+)
>
> diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> index 4f5628445a63..7a5d915f60b0 100644
> --- a/tools/perf/Documentation/perf-ftrace.txt
> +++ b/tools/perf/Documentation/perf-ftrace.txt
> @@ -53,6 +53,11 @@ OPTIONS
> Ranges of CPUs are specified with -: 0-2.
> Default is to trace on all online CPUs.
>
> +-m::
> +--buffer-size::
> + Set the size of per-cpu tracing buffer, <size> is expected to
> + be a number with appended unit character - B/K/M/G.
> +
> -T::
> --trace-funcs=::
> Select function tracer and set function filter on the given
> diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> index 4b3fcee5725a..a3a4f4be9dde 100644
> --- a/tools/perf/builtin-ftrace.c
> +++ b/tools/perf/builtin-ftrace.c
> @@ -26,6 +26,7 @@
> #include "thread_map.h"
> #include "util/cap.h"
> #include "util/config.h"
> +#include "util/units.h"
>
> #define DEFAULT_TRACER "function_graph"
>
> @@ -39,6 +40,7 @@ struct perf_ftrace {
> struct list_head graph_funcs;
> struct list_head nograph_funcs;
> int graph_depth;
> + unsigned long percpu_buffer_size;
> };
>
> struct filter_entry {
> @@ -324,6 +326,21 @@ static int set_tracing_depth(struct perf_ftrace *ftrace)
> return 0;
> }
>
> +static int set_tracing_percpu_buffer_size(struct perf_ftrace *ftrace)
> +{
> + int ret;
> +
> + if (ftrace->percpu_buffer_size == 0)
> + return 0;
> +
> + ret = write_tracing_file_int("buffer_size_kb",
> + ftrace->percpu_buffer_size / 1024);
> + if (ret < 0)
> + return ret;
> +
> + return 0;
> +}
> +
> static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> {
> char *trace_file;
> @@ -388,6 +405,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> goto out_reset;
> }
>
> + if (set_tracing_percpu_buffer_size(ftrace) < 0) {
> + pr_err("failed to set tracing per-cpu buffer size\n");
> + goto out_reset;
> + }
> +
> if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
> pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
> goto out_reset;
> @@ -506,6 +528,37 @@ static void delete_filter_func(struct list_head *head)
> }
> }
>
> +static int parse_buffer_size(const struct option *opt,
> + const char *str, int unset)
> +{
> + unsigned long *s = (unsigned long *)opt->value;
> + static struct parse_tag tags_size[] = {
> + { .tag = 'B', .mult = 1 },
> + { .tag = 'K', .mult = 1 << 10 },
> + { .tag = 'M', .mult = 1 << 20 },
> + { .tag = 'G', .mult = 1 << 30 },
> + { .tag = 0 },
> + };
> + unsigned long val;
> +
> + if (unset) {
> + *s = 0;
> + return 0;
> + }
> +
> + val = parse_tag_value(str, tags_size);
> + if (val != (unsigned long) -1) {
> + if (val < 1024) {
> + pr_err("buffer size too small, must larger than 1KB.");
> + return -1;
> + }
> + *s = val;
> + return 0;
> + }
> +
> + return -1;
> +}
> +
> static void select_tracer(struct perf_ftrace *ftrace)
> {
> bool graph = !list_empty(&ftrace->graph_funcs) ||
> @@ -560,6 +613,8 @@ int cmd_ftrace(int argc, const char **argv)
> "Set nograph filter on given functions", parse_filter_func),
> OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
> "Max depth for function graph tracer"),
> + OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
> + "size of per cpu buffer", parse_buffer_size),
> OPT_END()
> };
>
> --
> 2.25.1
>
--
- Arnaldo
Em Sat, Aug 08, 2020 at 10:31:31AM +0800, Changbin Du escreveu:
> This adds support to display call trace for function tracer. To do this,
> just specify a '--func-opts call-graph' option.
Applied, thanks for providing the usage example and the output of the
tool, please consider making this more compact and similar to the other
perf tools as:
# perf ftrace -g -T vfs_read
# perf report -h -g
Usage: perf report [<options>]
-g, --call-graph <print_type,threshold[,print_limit],order,sort_key[,branch],value>
Display call graph (stack chain/backtrace):
print_type: call graph printing style (graph|flat|fractal|folded|none)
threshold: minimum call graph inclusion threshold (<percent>)
print_limit: maximum number of call graph entry (<number>)
order: call graph order (caller|callee)
sort_key: call graph sort key (function|address)
branch: include last branch info to call graph (branch)
value: call graph value (percent|period|count)
Default: graph,0.5,caller,function,percent
#
Is there a way to ask for a limit in the number of levels? That is
similar to /proc/sys/kernel/perf_event_max_stack (system wide) and
perf_event_attr.sample_max_stack (per event)?
One can ask for that per event:
# perf trace -e sched:*_wakeup/max-stack=2/ -e sched:*switch/max-stack=5/ --max-events=4
0.000 migration/0/12 sched:sched_switch(prev_comm: "migration/0", prev_pid: 12 (migration/0), prev_prio: 0, prev_state: 1, next_comm: "swapper/0", next_pid: 0, next_prio: 120)
__schedule ([kernel.kallsyms])
__schedule ([kernel.kallsyms])
schedule ([kernel.kallsyms])
smpboot_thread_fn ([kernel.kallsyms])
kthread ([kernel.kallsyms])
2.381 :0/0 sched:sched_wakeup(comm: "PacerThread", pid: 111637 (PacerThread), prio: 120, success: 1, target_cpu: 0)
ttwu_do_wakeup ([kernel.kallsyms])
ttwu_do_wakeup ([kernel.kallsyms])
2.387 :0/0 sched:sched_switch(prev_comm: "swapper/0", prev_pid: 0, prev_prio: 120, prev_state: 0, next_comm: "PacerThread", next_pid: 111637 (PacerThread), next_prio: 120)
__schedule ([kernel.kallsyms])
__schedule ([kernel.kallsyms])
schedule_idle ([kernel.kallsyms])
do_idle ([kernel.kallsyms])
cpu_startup_entry ([kernel.kallsyms])
2.410 PacerThread/111637 sched:sched_switch(prev_comm: "PacerThread", prev_pid: 111637 (PacerThread), prev_prio: 120, prev_state: 1, next_comm: "swapper/0", next_pid: 0, next_prio: 120)
__schedule ([kernel.kallsyms])
__schedule ([kernel.kallsyms])
schedule ([kernel.kallsyms])
futex_wait_queue_me ([kernel.kallsyms])
futex_wait ([kernel.kallsyms])
#
- Arnaldo
> $ sudo perf ftrace -T vfs_read --func-opts call-graph
> iio-sensor-prox-855 [003] 6168.369657: vfs_read <-ksys_read
> iio-sensor-prox-855 [003] 6168.369677: <stack trace>
> => vfs_read
> => ksys_read
> => __x64_sys_read
> => do_syscall_64
> => entry_SYSCALL_64_after_hwframe
> ...
>
> Signed-off-by: Changbin Du <[email protected]>
>
> ---
> v3: switch to uniform option --func-opts.
> v2: option name '-s' -> '--func-call-graph'
> ---
> tools/perf/Documentation/perf-ftrace.txt | 4 +++
> tools/perf/builtin-ftrace.c | 42 ++++++++++++++++++++++++
> 2 files changed, 46 insertions(+)
>
> diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> index c46d0a09b38c..8f08ad0992c2 100644
> --- a/tools/perf/Documentation/perf-ftrace.txt
> +++ b/tools/perf/Documentation/perf-ftrace.txt
> @@ -76,6 +76,10 @@ OPTIONS
> specify multiple functions (or glob patterns). It will be
> passed to 'set_ftrace_notrace' in tracefs.
>
> +--func-opts::
> + List of options allowed to set:
> + call-graph - Display kernel stack trace for function tracer.
> +
> -G::
> --graph-funcs=::
> Select function_graph tracer and set graph filter on the given
> diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> index 07b81d0c1658..469b89748c42 100644
> --- a/tools/perf/builtin-ftrace.c
> +++ b/tools/perf/builtin-ftrace.c
> @@ -27,6 +27,7 @@
> #include "util/cap.h"
> #include "util/config.h"
> #include "util/units.h"
> +#include "util/parse-sublevel-options.h"
>
> #define DEFAULT_TRACER "function_graph"
>
> @@ -42,6 +43,7 @@ struct perf_ftrace {
> int graph_depth;
> unsigned long percpu_buffer_size;
> bool inherit;
> + int func_stack_trace;
> };
>
> struct filter_entry {
> @@ -202,6 +204,7 @@ static void reset_tracing_filters(void);
> static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
> {
> write_tracing_option_file("function-fork", "0");
> + write_tracing_option_file("func_stack_trace", "0");
> }
>
> static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
> @@ -278,6 +281,17 @@ static int set_tracing_cpu(struct perf_ftrace *ftrace)
> return set_tracing_cpumask(cpumap);
> }
>
> +static int set_tracing_func_stack_trace(struct perf_ftrace *ftrace)
> +{
> + if (!ftrace->func_stack_trace)
> + return 0;
> +
> + if (write_tracing_option_file("func_stack_trace", "1") < 0)
> + return -1;
> +
> + return 0;
> +}
> +
> static int reset_tracing_cpu(void)
> {
> struct perf_cpu_map *cpumap = perf_cpu_map__new(NULL);
> @@ -426,6 +440,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> goto out_reset;
> }
>
> + if (set_tracing_func_stack_trace(ftrace) < 0) {
> + pr_err("failed to set tracing option func_stack_trace\n");
> + goto out_reset;
> + }
> +
> if (set_tracing_filters(ftrace) < 0) {
> pr_err("failed to set tracing filters\n");
> goto out_reset;
> @@ -598,6 +617,26 @@ static int parse_buffer_size(const struct option *opt,
> return -1;
> }
>
> +static int parse_func_tracer_opts(const struct option *opt,
> + const char *str, int unset)
> +{
> + int ret;
> + struct perf_ftrace *ftrace = (struct perf_ftrace *) opt->value;
> + struct sublevel_option func_tracer_opts[] = {
> + { .name = "call-graph", .value_ptr = &ftrace->func_stack_trace },
> + { .name = NULL, }
> + };
> +
> + if (unset)
> + return 0;
> +
> + ret = perf_parse_sublevel_options(str, func_tracer_opts);
> + if (ret)
> + return ret;
> +
> + return 0;
> +}
> +
> static void select_tracer(struct perf_ftrace *ftrace)
> {
> bool graph = !list_empty(&ftrace->graph_funcs) ||
> @@ -645,6 +684,9 @@ int cmd_ftrace(int argc, const char **argv)
> parse_filter_func),
> OPT_CALLBACK('N', "notrace-funcs", &ftrace.notrace, "func",
> "do not trace given functions", parse_filter_func),
> + OPT_CALLBACK(0, "func-opts", &ftrace, "options",
> + "function tracer options, available options: call-graph",
> + parse_func_tracer_opts),
> OPT_CALLBACK('G', "graph-funcs", &ftrace.graph_funcs, "func",
> "trace given functions using function_graph tracer",
> parse_filter_func),
> --
> 2.25.1
>
--
- Arnaldo
Em Sat, Aug 08, 2020 at 10:31:32AM +0800, Changbin Du escreveu:
> This adds an option '--graph-opts nosleep-time' which allow us
> only to measure on-CPU time. This option is function_graph tracer
> only.
Here an example showing its usage would be great. Even better, on
perf-ftrace.txt, so that people can see it in action and learn about its
output and usefullness.
Applied,
- Arnaldo
> Signed-off-by: Changbin Du <[email protected]>
>
> ---
> v3: switch to uniform option --graph-opts.
> v2: option name '--nosleep-time' -> '--graph-nosleep-time'.
> ---
> tools/perf/Documentation/perf-ftrace.txt | 4 +++
> tools/perf/builtin-ftrace.c | 41 ++++++++++++++++++++++++
> 2 files changed, 45 insertions(+)
>
> diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> index 8f08ad0992c2..3380a2e2c9ad 100644
> --- a/tools/perf/Documentation/perf-ftrace.txt
> +++ b/tools/perf/Documentation/perf-ftrace.txt
> @@ -101,6 +101,10 @@ OPTIONS
> --graph-depth=::
> Set max depth for function graph tracer to follow
>
> +--graph-opts::
> + List of options allowed to set:
> + nosleep-time - Measure on-CPU time only for function_graph tracer.
> +
> SEE ALSO
> --------
> linkperf:perf-record[1], linkperf:perf-trace[1]
> diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> index 469b89748c42..47d63bba6a48 100644
> --- a/tools/perf/builtin-ftrace.c
> +++ b/tools/perf/builtin-ftrace.c
> @@ -44,6 +44,7 @@ struct perf_ftrace {
> unsigned long percpu_buffer_size;
> bool inherit;
> int func_stack_trace;
> + int graph_nosleep_time;
> };
>
> struct filter_entry {
> @@ -205,6 +206,7 @@ static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
> {
> write_tracing_option_file("function-fork", "0");
> write_tracing_option_file("func_stack_trace", "0");
> + write_tracing_option_file("sleep-time", "1");
> }
>
> static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
> @@ -386,6 +388,17 @@ static int set_tracing_trace_inherit(struct perf_ftrace *ftrace)
> return 0;
> }
>
> +static int set_tracing_sleep_time(struct perf_ftrace *ftrace)
> +{
> + if (!ftrace->graph_nosleep_time)
> + return 0;
> +
> + if (write_tracing_option_file("sleep-time", "0") < 0)
> + return -1;
> +
> + return 0;
> +}
> +
> static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> {
> char *trace_file;
> @@ -465,6 +478,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> goto out_reset;
> }
>
> + if (set_tracing_sleep_time(ftrace) < 0) {
> + pr_err("failed to set tracing option sleep-time\n");
> + goto out_reset;
> + }
> +
> if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
> pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
> goto out_reset;
> @@ -637,6 +655,26 @@ static int parse_func_tracer_opts(const struct option *opt,
> return 0;
> }
>
> +static int parse_graph_tracer_opts(const struct option *opt,
> + const char *str, int unset)
> +{
> + int ret;
> + struct perf_ftrace *ftrace = (struct perf_ftrace *) opt->value;
> + struct sublevel_option graph_tracer_opts[] = {
> + { .name = "nosleep-time", .value_ptr = &ftrace->graph_nosleep_time },
> + { .name = NULL, }
> + };
> +
> + if (unset)
> + return 0;
> +
> + ret = perf_parse_sublevel_options(str, graph_tracer_opts);
> + if (ret)
> + return ret;
> +
> + return 0;
> +}
> +
> static void select_tracer(struct perf_ftrace *ftrace)
> {
> bool graph = !list_empty(&ftrace->graph_funcs) ||
> @@ -694,6 +732,9 @@ int cmd_ftrace(int argc, const char **argv)
> "Set nograph filter on given functions", parse_filter_func),
> OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
> "Max depth for function graph tracer"),
> + OPT_CALLBACK(0, "graph-opts", &ftrace, "options",
> + "graph tracer options, available options: nosleep-time",
> + parse_graph_tracer_opts),
> OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
> "size of per cpu buffer", parse_buffer_size),
> OPT_BOOLEAN(0, "inherit", &ftrace.inherit,
> --
> 2.25.1
>
--
- Arnaldo
Em Sat, Aug 08, 2020 at 10:31:34AM +0800, Changbin Du escreveu:
> This adds support to display irq context info for function tracer. To do
> this, just specify a '--func-opts irq-info' option.
Applied, ditto about adding an example to perf-ftrace.txt
> Signed-off-by: Changbin Du <[email protected]>
> ---
> tools/perf/Documentation/perf-ftrace.txt | 1 +
> tools/perf/builtin-ftrace.c | 21 ++++++++++++++++++++-
> 2 files changed, 21 insertions(+), 1 deletion(-)
>
> diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> index fca55ac55ff3..37ea3ea97922 100644
> --- a/tools/perf/Documentation/perf-ftrace.txt
> +++ b/tools/perf/Documentation/perf-ftrace.txt
> @@ -79,6 +79,7 @@ OPTIONS
> --func-opts::
> List of options allowed to set:
> call-graph - Display kernel stack trace for function tracer.
> + irq-info - Display irq context info for function tracer.
>
> -G::
> --graph-funcs=::
> diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> index b4c821be4fb5..d1241febe143 100644
> --- a/tools/perf/builtin-ftrace.c
> +++ b/tools/perf/builtin-ftrace.c
> @@ -44,6 +44,7 @@ struct perf_ftrace {
> unsigned long percpu_buffer_size;
> bool inherit;
> int func_stack_trace;
> + int func_irq_info;
> int graph_nosleep_time;
> int graph_noirqs;
> };
> @@ -209,6 +210,7 @@ static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
> write_tracing_option_file("func_stack_trace", "0");
> write_tracing_option_file("sleep-time", "1");
> write_tracing_option_file("funcgraph-irqs", "1");
> + write_tracing_option_file("irq-info", "0");
> }
>
> static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
> @@ -296,6 +298,17 @@ static int set_tracing_func_stack_trace(struct perf_ftrace *ftrace)
> return 0;
> }
>
> +static int set_tracing_func_irqinfo(struct perf_ftrace *ftrace)
> +{
> + if (!ftrace->func_irq_info)
> + return 0;
> +
> + if (write_tracing_option_file("irq-info", "1") < 0)
> + return -1;
> +
> + return 0;
> +}
> +
> static int reset_tracing_cpu(void)
> {
> struct perf_cpu_map *cpumap = perf_cpu_map__new(NULL);
> @@ -471,6 +484,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> goto out_reset;
> }
>
> + if (set_tracing_func_irqinfo(ftrace) < 0) {
> + pr_err("failed to set tracing option irq-info\n");
> + goto out_reset;
> + }
> +
> if (set_tracing_filters(ftrace) < 0) {
> pr_err("failed to set tracing filters\n");
> goto out_reset;
> @@ -660,6 +678,7 @@ static int parse_func_tracer_opts(const struct option *opt,
> struct perf_ftrace *ftrace = (struct perf_ftrace *) opt->value;
> struct sublevel_option func_tracer_opts[] = {
> { .name = "call-graph", .value_ptr = &ftrace->func_stack_trace },
> + { .name = "irq-info", .value_ptr = &ftrace->func_irq_info },
> { .name = NULL, }
> };
>
> @@ -742,7 +761,7 @@ int cmd_ftrace(int argc, const char **argv)
> OPT_CALLBACK('N', "notrace-funcs", &ftrace.notrace, "func",
> "do not trace given functions", parse_filter_func),
> OPT_CALLBACK(0, "func-opts", &ftrace, "options",
> - "function tracer options, available options: call-graph",
> + "function tracer options, available options: call-graph,irq-info",
> parse_func_tracer_opts),
> OPT_CALLBACK('G', "graph-funcs", &ftrace.graph_funcs, "func",
> "trace given functions using function_graph tracer",
> --
> 2.25.1
>
--
- Arnaldo
Em Fri, Aug 14, 2020 at 09:12:50AM -0300, Arnaldo Carvalho de Melo escreveu:
> > It indicates that the argument is in units of 'pages', while you don't
> > state that in:
> >
> > $ perf ftrace -h -m
> >
> > -m, --buffer-size <size>
> > size of per cpu buffer
I did this already:
# perf ftrace -h -m
Usage: perf ftrace [<options>] [<command>]
or: perf ftrace [<options>] -- <command> [<options>]
-m, --buffer-size <size>
Size of per cpu buffer, needs to use a B, K, M or G suffix.
#
- Arnaldo
Em Sat, Aug 08, 2020 at 10:31:33AM +0800, Changbin Du escreveu:
> This adds an option '--graph-opts noirqs' to filter out functions executed
> in irq context.
ditto, no example provided, please consider adding an example +
explanation to perf-ftrace.txt.
> Signed-off-by: Changbin Du <[email protected]>
>
> ---
> v2: option name '--nofuncgraph-irqs' -> '--graph-noirqs'.
> ---
> tools/perf/Documentation/perf-ftrace.txt | 1 +
> tools/perf/builtin-ftrace.c | 21 ++++++++++++++++++++-
> 2 files changed, 21 insertions(+), 1 deletion(-)
>
> diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> index 3380a2e2c9ad..fca55ac55ff3 100644
> --- a/tools/perf/Documentation/perf-ftrace.txt
> +++ b/tools/perf/Documentation/perf-ftrace.txt
> @@ -104,6 +104,7 @@ OPTIONS
> --graph-opts::
> List of options allowed to set:
> nosleep-time - Measure on-CPU time only for function_graph tracer.
> + noirqs - Ignore functions that happen inside interrupt.
>
> SEE ALSO
> --------
> diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> index 47d63bba6a48..b4c821be4fb5 100644
> --- a/tools/perf/builtin-ftrace.c
> +++ b/tools/perf/builtin-ftrace.c
> @@ -45,6 +45,7 @@ struct perf_ftrace {
> bool inherit;
> int func_stack_trace;
> int graph_nosleep_time;
> + int graph_noirqs;
> };
>
> struct filter_entry {
> @@ -207,6 +208,7 @@ static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
> write_tracing_option_file("function-fork", "0");
> write_tracing_option_file("func_stack_trace", "0");
> write_tracing_option_file("sleep-time", "1");
> + write_tracing_option_file("funcgraph-irqs", "1");
> }
>
> static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
> @@ -399,6 +401,17 @@ static int set_tracing_sleep_time(struct perf_ftrace *ftrace)
> return 0;
> }
>
> +static int set_tracing_funcgraph_irqs(struct perf_ftrace *ftrace)
> +{
> + if (!ftrace->graph_noirqs)
> + return 0;
> +
> + if (write_tracing_option_file("funcgraph-irqs", "0") < 0)
> + return -1;
> +
> + return 0;
> +}
> +
> static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> {
> char *trace_file;
> @@ -483,6 +496,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> goto out_reset;
> }
>
> + if (set_tracing_funcgraph_irqs(ftrace) < 0) {
> + pr_err("failed to set tracing option funcgraph-irqs\n");
> + goto out_reset;
> + }
> +
> if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
> pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
> goto out_reset;
> @@ -662,6 +680,7 @@ static int parse_graph_tracer_opts(const struct option *opt,
> struct perf_ftrace *ftrace = (struct perf_ftrace *) opt->value;
> struct sublevel_option graph_tracer_opts[] = {
> { .name = "nosleep-time", .value_ptr = &ftrace->graph_nosleep_time },
> + { .name = "noirqs", .value_ptr = &ftrace->graph_noirqs },
> { .name = NULL, }
> };
>
> @@ -733,7 +752,7 @@ int cmd_ftrace(int argc, const char **argv)
> OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
> "Max depth for function graph tracer"),
> OPT_CALLBACK(0, "graph-opts", &ftrace, "options",
> - "graph tracer options, available options: nosleep-time",
> + "graph tracer options, available options: nosleep-time,noirqs",
> parse_graph_tracer_opts),
> OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
> "size of per cpu buffer", parse_buffer_size),
> --
> 2.25.1
>
--
- Arnaldo
Em Sat, Aug 08, 2020 at 10:31:36AM +0800, Changbin Du escreveu:
> This adds an option '--graph-opts thresh' to setup trace duration
> threshold for funcgraph tracer.
Applied, please consider making --duration an alias for this, as this is
also available in 'perf trace':
[root@quaco ~]# perf trace --duration 500 --max-events=5
369.860 gnome-shell/2084 poll(ufds: 0x559577983080, nfds: 21, timeout_msecs: 7194) = 1
493.654 pool-geoclue/386046 futex(uaddr: 0x56024843b610, op: WAIT|PRIVATE_FLAG, val: 1201, utime: 0x7f32bd7f9cc0) = -1 ETIMEDOUT (Connection timed out)
71.604 WebRTC_Worker/111634 futex(uaddr: 0x7fb4c364d608, op: WAIT_BITSET|PRIVATE_FLAG, val: 0, utime: 0x7fb4c364d410, val3: MATCH_ANY) = -1 ETIMEDOUT (Connection timed out)
601.645 pool-gsd-smart/2469 clock_nanosleep(which_clock: 0, flags: 0, rqtp: 0x7fb82cbd1cf0, rmtp: 0x7fb82cbd1d00) = 0
623.823 chromium-brows/92363 futex(uaddr: 0x7ffcfd064e38, op: WAIT_BITSET|PRIVATE_FLAG, val: 0, utime: 0x7ffcfd064c40, val3: MATCH_ANY) = -1 ETIMEDOUT (Connection timed out)
[root@quaco ~]#
$ perf trace -h duration
--duration <float>
show only events with duration > N.M ms
$
This way one would be able to do:
# perf trace --duration N.M
To see at syscall level, then add 'f':
# perf ftrace --duration N.M
To see at kernel function level. Switching from one perf tool to another
with such ease is the main objective here, I'd say :-)
- Arnaldo
> $ sudo ./perf ftrace -G '*' --graph-opts thresh=100
> 3) ! 184.060 us | } /* schedule */
> 3) ! 185.600 us | } /* exit_to_usermode_loop */
> 2) ! 225.989 us | } /* schedule_idle */
> 2) # 4140.051 us | } /* do_idle */
>
> Signed-off-by: Changbin Du <[email protected]>
> ---
> tools/perf/Documentation/perf-ftrace.txt | 1 +
> tools/perf/builtin-ftrace.c | 26 +++++++++++++++++++++++-
> 2 files changed, 26 insertions(+), 1 deletion(-)
>
> diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> index a2056aaf2ece..6fa927e5971b 100644
> --- a/tools/perf/Documentation/perf-ftrace.txt
> +++ b/tools/perf/Documentation/perf-ftrace.txt
> @@ -107,6 +107,7 @@ OPTIONS
> nosleep-time - Measure on-CPU time only for function_graph tracer.
> noirqs - Ignore functions that happen inside interrupt.
> verbose - Show process names, PIDs, timestamps, etc.
> + thresh=<n> - Setup trace duration threshold in microseconds.
>
> SEE ALSO
> --------
> diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> index 8ee5287bd84a..ca61f8b8bbf6 100644
> --- a/tools/perf/builtin-ftrace.c
> +++ b/tools/perf/builtin-ftrace.c
> @@ -48,6 +48,7 @@ struct perf_ftrace {
> int graph_nosleep_time;
> int graph_noirqs;
> int graph_verbose;
> + int graph_thresh;
> };
>
> struct filter_entry {
> @@ -234,6 +235,9 @@ static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
> if (write_tracing_file("max_graph_depth", "0") < 0)
> return -1;
>
> + if (write_tracing_file("tracing_thresh", "0") < 0)
> + return -1;
> +
> reset_tracing_filters();
> reset_tracing_options(ftrace);
> return 0;
> @@ -446,6 +450,20 @@ static int set_tracing_funcgraph_verbose(struct perf_ftrace *ftrace)
> return 0;
> }
>
> +static int set_tracing_thresh(struct perf_ftrace *ftrace)
> +{
> + int ret;
> +
> + if (ftrace->graph_thresh == 0)
> + return 0;
> +
> + ret = write_tracing_file_int("tracing_thresh", ftrace->graph_thresh);
> + if (ret < 0)
> + return ret;
> +
> + return 0;
> +}
> +
> static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> {
> char *trace_file;
> @@ -545,6 +563,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> goto out_reset;
> }
>
> + if (set_tracing_thresh(ftrace) < 0) {
> + pr_err("failed to set tracing thresh\n");
> + goto out_reset;
> + }
> +
> if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
> pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
> goto out_reset;
> @@ -727,6 +750,7 @@ static int parse_graph_tracer_opts(const struct option *opt,
> { .name = "nosleep-time", .value_ptr = &ftrace->graph_nosleep_time },
> { .name = "noirqs", .value_ptr = &ftrace->graph_noirqs },
> { .name = "verbose", .value_ptr = &ftrace->graph_verbose },
> + { .name = "thresh", .value_ptr = &ftrace->graph_thresh },
> { .name = NULL, }
> };
>
> @@ -798,7 +822,7 @@ int cmd_ftrace(int argc, const char **argv)
> OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
> "Max depth for function graph tracer"),
> OPT_CALLBACK(0, "graph-opts", &ftrace, "options",
> - "graph tracer options, available options: nosleep-time,noirqs,verbose",
> + "graph tracer options, available options: nosleep-time,noirqs,verbose,thresh=<n>",
> parse_graph_tracer_opts),
> OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
> "size of per cpu buffer", parse_buffer_size),
> --
> 2.25.1
>
--
- Arnaldo
Em Sat, Aug 08, 2020 at 10:31:23AM +0800, Changbin Du escreveu:
> The perf has basic kernel ftrace support but lack support of most tracing
> options. This serias is target to enhance the perf ftrace functionality so
> that we can make full use of kernel ftrace with perf.
>
> In general, this serias be cataloged into two main changes:
> 1) Improve usability of existing functions. For example, we don't need to type
> extra option to select the tracer.
> 2) Add new options to support all other ftrace functions.
Applied, sent review comments for further work, thanks!
Now it will go together with other changes to extensive build testing
and then to Linus,
- Arnaldo
On Fri, Aug 14, 2020 at 09:33:10AM -0300, Arnaldo Carvalho de Melo wrote:
> Em Sat, Aug 08, 2020 at 10:31:36AM +0800, Changbin Du escreveu:
> > This adds an option '--graph-opts thresh' to setup trace duration
> > threshold for funcgraph tracer.
>
> Applied, please consider making --duration an alias for this, as this is
> also available in 'perf trace':
>
> [root@quaco ~]# perf trace --duration 500 --max-events=5
> 369.860 gnome-shell/2084 poll(ufds: 0x559577983080, nfds: 21, timeout_msecs: 7194) = 1
> 493.654 pool-geoclue/386046 futex(uaddr: 0x56024843b610, op: WAIT|PRIVATE_FLAG, val: 1201, utime: 0x7f32bd7f9cc0) = -1 ETIMEDOUT (Connection timed out)
> 71.604 WebRTC_Worker/111634 futex(uaddr: 0x7fb4c364d608, op: WAIT_BITSET|PRIVATE_FLAG, val: 0, utime: 0x7fb4c364d410, val3: MATCH_ANY) = -1 ETIMEDOUT (Connection timed out)
> 601.645 pool-gsd-smart/2469 clock_nanosleep(which_clock: 0, flags: 0, rqtp: 0x7fb82cbd1cf0, rmtp: 0x7fb82cbd1d00) = 0
> 623.823 chromium-brows/92363 futex(uaddr: 0x7ffcfd064e38, op: WAIT_BITSET|PRIVATE_FLAG, val: 0, utime: 0x7ffcfd064c40, val3: MATCH_ANY) = -1 ETIMEDOUT (Connection timed out)
> [root@quaco ~]#
>
> $ perf trace -h duration
>
> --duration <float>
> show only events with duration > N.M ms
>
> $
>
> This way one would be able to do:
>
> # perf trace --duration N.M
>
> To see at syscall level, then add 'f':
>
> # perf ftrace --duration N.M
>
> To see at kernel function level. Switching from one perf tool to another
> with such ease is the main objective here, I'd say :-)
>
Sure. Currently the unit for ftrace is 'ns'. I could change it to 'ms' at perf
level.
> - Arnaldo
>
> > $ sudo ./perf ftrace -G '*' --graph-opts thresh=100
> > 3) ! 184.060 us | } /* schedule */
> > 3) ! 185.600 us | } /* exit_to_usermode_loop */
> > 2) ! 225.989 us | } /* schedule_idle */
> > 2) # 4140.051 us | } /* do_idle */
> >
> > Signed-off-by: Changbin Du <[email protected]>
> > ---
> > tools/perf/Documentation/perf-ftrace.txt | 1 +
> > tools/perf/builtin-ftrace.c | 26 +++++++++++++++++++++++-
> > 2 files changed, 26 insertions(+), 1 deletion(-)
> >
> > diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> > index a2056aaf2ece..6fa927e5971b 100644
> > --- a/tools/perf/Documentation/perf-ftrace.txt
> > +++ b/tools/perf/Documentation/perf-ftrace.txt
> > @@ -107,6 +107,7 @@ OPTIONS
> > nosleep-time - Measure on-CPU time only for function_graph tracer.
> > noirqs - Ignore functions that happen inside interrupt.
> > verbose - Show process names, PIDs, timestamps, etc.
> > + thresh=<n> - Setup trace duration threshold in microseconds.
> >
> > SEE ALSO
> > --------
> > diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> > index 8ee5287bd84a..ca61f8b8bbf6 100644
> > --- a/tools/perf/builtin-ftrace.c
> > +++ b/tools/perf/builtin-ftrace.c
> > @@ -48,6 +48,7 @@ struct perf_ftrace {
> > int graph_nosleep_time;
> > int graph_noirqs;
> > int graph_verbose;
> > + int graph_thresh;
> > };
> >
> > struct filter_entry {
> > @@ -234,6 +235,9 @@ static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
> > if (write_tracing_file("max_graph_depth", "0") < 0)
> > return -1;
> >
> > + if (write_tracing_file("tracing_thresh", "0") < 0)
> > + return -1;
> > +
> > reset_tracing_filters();
> > reset_tracing_options(ftrace);
> > return 0;
> > @@ -446,6 +450,20 @@ static int set_tracing_funcgraph_verbose(struct perf_ftrace *ftrace)
> > return 0;
> > }
> >
> > +static int set_tracing_thresh(struct perf_ftrace *ftrace)
> > +{
> > + int ret;
> > +
> > + if (ftrace->graph_thresh == 0)
> > + return 0;
> > +
> > + ret = write_tracing_file_int("tracing_thresh", ftrace->graph_thresh);
> > + if (ret < 0)
> > + return ret;
> > +
> > + return 0;
> > +}
> > +
> > static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> > {
> > char *trace_file;
> > @@ -545,6 +563,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> > goto out_reset;
> > }
> >
> > + if (set_tracing_thresh(ftrace) < 0) {
> > + pr_err("failed to set tracing thresh\n");
> > + goto out_reset;
> > + }
> > +
> > if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
> > pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
> > goto out_reset;
> > @@ -727,6 +750,7 @@ static int parse_graph_tracer_opts(const struct option *opt,
> > { .name = "nosleep-time", .value_ptr = &ftrace->graph_nosleep_time },
> > { .name = "noirqs", .value_ptr = &ftrace->graph_noirqs },
> > { .name = "verbose", .value_ptr = &ftrace->graph_verbose },
> > + { .name = "thresh", .value_ptr = &ftrace->graph_thresh },
> > { .name = NULL, }
> > };
> >
> > @@ -798,7 +822,7 @@ int cmd_ftrace(int argc, const char **argv)
> > OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
> > "Max depth for function graph tracer"),
> > OPT_CALLBACK(0, "graph-opts", &ftrace, "options",
> > - "graph tracer options, available options: nosleep-time,noirqs,verbose",
> > + "graph tracer options, available options: nosleep-time,noirqs,verbose,thresh=<n>",
> > parse_graph_tracer_opts),
> > OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
> > "size of per cpu buffer", parse_buffer_size),
> > --
> > 2.25.1
> >
>
> --
>
> - Arnaldo
--
Cheers,
Changbin Du
On Fri, Aug 14, 2020 at 09:24:40AM -0300, Arnaldo Carvalho de Melo wrote:
> Em Sat, Aug 08, 2020 at 10:31:31AM +0800, Changbin Du escreveu:
> > This adds support to display call trace for function tracer. To do this,
> > just specify a '--func-opts call-graph' option.
>
> Applied, thanks for providing the usage example and the output of the
> tool, please consider making this more compact and similar to the other
> perf tools as:
>
> # perf ftrace -g -T vfs_read
>
> # perf report -h -g
>
> Usage: perf report [<options>]
>
> -g, --call-graph <print_type,threshold[,print_limit],order,sort_key[,branch],value>
> Display call graph (stack chain/backtrace):
>
> print_type: call graph printing style (graph|flat|fractal|folded|none)
> threshold: minimum call graph inclusion threshold (<percent>)
> print_limit: maximum number of call graph entry (<number>)
> order: call graph order (caller|callee)
> sort_key: call graph sort key (function|address)
> branch: include last branch info to call graph (branch)
> value: call graph value (percent|period|count)
>
> Default: graph,0.5,caller,function,percent
>
> #
>
> Is there a way to ask for a limit in the number of levels? That is
> similar to /proc/sys/kernel/perf_event_max_stack (system wide) and
> perf_event_attr.sample_max_stack (per event)?
>
No, there is no such limit for function tracer yet (kernel doesn't support it).
It just backtraces from current to top.
For graph tracer, the option is --graph-opts depth=<n>.
> One can ask for that per event:
>
> # perf trace -e sched:*_wakeup/max-stack=2/ -e sched:*switch/max-stack=5/ --max-events=4
> 0.000 migration/0/12 sched:sched_switch(prev_comm: "migration/0", prev_pid: 12 (migration/0), prev_prio: 0, prev_state: 1, next_comm: "swapper/0", next_pid: 0, next_prio: 120)
> __schedule ([kernel.kallsyms])
> __schedule ([kernel.kallsyms])
> schedule ([kernel.kallsyms])
> smpboot_thread_fn ([kernel.kallsyms])
> kthread ([kernel.kallsyms])
> 2.381 :0/0 sched:sched_wakeup(comm: "PacerThread", pid: 111637 (PacerThread), prio: 120, success: 1, target_cpu: 0)
> ttwu_do_wakeup ([kernel.kallsyms])
> ttwu_do_wakeup ([kernel.kallsyms])
> 2.387 :0/0 sched:sched_switch(prev_comm: "swapper/0", prev_pid: 0, prev_prio: 120, prev_state: 0, next_comm: "PacerThread", next_pid: 111637 (PacerThread), next_prio: 120)
> __schedule ([kernel.kallsyms])
> __schedule ([kernel.kallsyms])
> schedule_idle ([kernel.kallsyms])
> do_idle ([kernel.kallsyms])
> cpu_startup_entry ([kernel.kallsyms])
> 2.410 PacerThread/111637 sched:sched_switch(prev_comm: "PacerThread", prev_pid: 111637 (PacerThread), prev_prio: 120, prev_state: 1, next_comm: "swapper/0", next_pid: 0, next_prio: 120)
> __schedule ([kernel.kallsyms])
> __schedule ([kernel.kallsyms])
> schedule ([kernel.kallsyms])
> futex_wait_queue_me ([kernel.kallsyms])
> futex_wait ([kernel.kallsyms])
> #
>
> - Arnaldo
>
> > $ sudo perf ftrace -T vfs_read --func-opts call-graph
> > iio-sensor-prox-855 [003] 6168.369657: vfs_read <-ksys_read
> > iio-sensor-prox-855 [003] 6168.369677: <stack trace>
> > => vfs_read
> > => ksys_read
> > => __x64_sys_read
> > => do_syscall_64
> > => entry_SYSCALL_64_after_hwframe
> > ...
> >
> > Signed-off-by: Changbin Du <[email protected]>
> >
> > ---
> > v3: switch to uniform option --func-opts.
> > v2: option name '-s' -> '--func-call-graph'
> > ---
> > tools/perf/Documentation/perf-ftrace.txt | 4 +++
> > tools/perf/builtin-ftrace.c | 42 ++++++++++++++++++++++++
> > 2 files changed, 46 insertions(+)
> >
> > diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> > index c46d0a09b38c..8f08ad0992c2 100644
> > --- a/tools/perf/Documentation/perf-ftrace.txt
> > +++ b/tools/perf/Documentation/perf-ftrace.txt
> > @@ -76,6 +76,10 @@ OPTIONS
> > specify multiple functions (or glob patterns). It will be
> > passed to 'set_ftrace_notrace' in tracefs.
> >
> > +--func-opts::
> > + List of options allowed to set:
> > + call-graph - Display kernel stack trace for function tracer.
> > +
> > -G::
> > --graph-funcs=::
> > Select function_graph tracer and set graph filter on the given
> > diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> > index 07b81d0c1658..469b89748c42 100644
> > --- a/tools/perf/builtin-ftrace.c
> > +++ b/tools/perf/builtin-ftrace.c
> > @@ -27,6 +27,7 @@
> > #include "util/cap.h"
> > #include "util/config.h"
> > #include "util/units.h"
> > +#include "util/parse-sublevel-options.h"
> >
> > #define DEFAULT_TRACER "function_graph"
> >
> > @@ -42,6 +43,7 @@ struct perf_ftrace {
> > int graph_depth;
> > unsigned long percpu_buffer_size;
> > bool inherit;
> > + int func_stack_trace;
> > };
> >
> > struct filter_entry {
> > @@ -202,6 +204,7 @@ static void reset_tracing_filters(void);
> > static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
> > {
> > write_tracing_option_file("function-fork", "0");
> > + write_tracing_option_file("func_stack_trace", "0");
> > }
> >
> > static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
> > @@ -278,6 +281,17 @@ static int set_tracing_cpu(struct perf_ftrace *ftrace)
> > return set_tracing_cpumask(cpumap);
> > }
> >
> > +static int set_tracing_func_stack_trace(struct perf_ftrace *ftrace)
> > +{
> > + if (!ftrace->func_stack_trace)
> > + return 0;
> > +
> > + if (write_tracing_option_file("func_stack_trace", "1") < 0)
> > + return -1;
> > +
> > + return 0;
> > +}
> > +
> > static int reset_tracing_cpu(void)
> > {
> > struct perf_cpu_map *cpumap = perf_cpu_map__new(NULL);
> > @@ -426,6 +440,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> > goto out_reset;
> > }
> >
> > + if (set_tracing_func_stack_trace(ftrace) < 0) {
> > + pr_err("failed to set tracing option func_stack_trace\n");
> > + goto out_reset;
> > + }
> > +
> > if (set_tracing_filters(ftrace) < 0) {
> > pr_err("failed to set tracing filters\n");
> > goto out_reset;
> > @@ -598,6 +617,26 @@ static int parse_buffer_size(const struct option *opt,
> > return -1;
> > }
> >
> > +static int parse_func_tracer_opts(const struct option *opt,
> > + const char *str, int unset)
> > +{
> > + int ret;
> > + struct perf_ftrace *ftrace = (struct perf_ftrace *) opt->value;
> > + struct sublevel_option func_tracer_opts[] = {
> > + { .name = "call-graph", .value_ptr = &ftrace->func_stack_trace },
> > + { .name = NULL, }
> > + };
> > +
> > + if (unset)
> > + return 0;
> > +
> > + ret = perf_parse_sublevel_options(str, func_tracer_opts);
> > + if (ret)
> > + return ret;
> > +
> > + return 0;
> > +}
> > +
> > static void select_tracer(struct perf_ftrace *ftrace)
> > {
> > bool graph = !list_empty(&ftrace->graph_funcs) ||
> > @@ -645,6 +684,9 @@ int cmd_ftrace(int argc, const char **argv)
> > parse_filter_func),
> > OPT_CALLBACK('N', "notrace-funcs", &ftrace.notrace, "func",
> > "do not trace given functions", parse_filter_func),
> > + OPT_CALLBACK(0, "func-opts", &ftrace, "options",
> > + "function tracer options, available options: call-graph",
> > + parse_func_tracer_opts),
> > OPT_CALLBACK('G', "graph-funcs", &ftrace.graph_funcs, "func",
> > "trace given functions using function_graph tracer",
> > parse_filter_func),
> > --
> > 2.25.1
> >
>
> --
>
> - Arnaldo
--
Cheers,
Changbin Du
On Fri, Aug 14, 2020 at 08:35:35AM -0300, Arnaldo Carvalho de Melo wrote:
> Em Sat, Aug 08, 2020 at 10:31:25AM +0800, Changbin Du escreveu:
> > This adds an option '-F/--funcs' to list all available functions to trace,
> > which is read from tracing file 'available_filter_functions'.
> >
> > $ sudo ./perf ftrace -F | head
> > trace_initcall_finish_cb
> > initcall_blacklisted
> > do_one_initcall
> > do_one_initcall
> > trace_initcall_start_cb
> > run_init_process
> > try_to_run_init_process
> > match_dev_by_label
> > match_dev_by_uuid
> > rootfs_init_fs_context
> >
> > Signed-off-by: Changbin Du <[email protected]>
> >
> > ---
> > v3: fix return value issue.
> > v2: option name '-l/--list-functions' -> '-F/--funcs'
> > ---
> > tools/perf/Documentation/perf-ftrace.txt | 4 +++
> > tools/perf/builtin-ftrace.c | 46 ++++++++++++++++++++++++
> > 2 files changed, 50 insertions(+)
> >
> > diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> > index 821d4d334a09..4f5628445a63 100644
> > --- a/tools/perf/Documentation/perf-ftrace.txt
> > +++ b/tools/perf/Documentation/perf-ftrace.txt
> > @@ -31,6 +31,10 @@ OPTIONS
> > --verbose=::
> > Verbosity level.
> >
> > +-F::
> > +--funcs::
> > + List all available functions to trace.
> > +
> > -p::
> > --pid=::
> > Trace on existing process id (comma separated list).
> > diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> > index 048a11192b59..1ebf71ecc1a8 100644
> > --- a/tools/perf/builtin-ftrace.c
> > +++ b/tools/perf/builtin-ftrace.c
> > @@ -33,6 +33,7 @@ struct perf_ftrace {
> > struct evlist *evlist;
> > struct target target;
> > const char *tracer;
> > + bool list_avail_functions;
> > struct list_head filters;
> > struct list_head notrace;
> > struct list_head graph_funcs;
> > @@ -128,6 +129,46 @@ static int append_tracing_file(const char *name, const char *val)
> > return __write_tracing_file(name, val, true);
> > }
> >
> > +static int read_tracing_file_to_stdout(const char *name)
> > +{
> > + char buf[4096];
> > + char *file;
> > + int fd;
> > + int ret = -1;
> > +
> > + file = get_tracing_file(name);
> > + if (!file) {
> > + pr_debug("cannot get tracing file: %s\n", name);
> > + return -1;
> > + }
> > +
> > + fd = open(file, O_RDONLY);
> > + if (fd < 0) {
> > + pr_debug("cannot open tracing file: %s: %s\n",
> > + name, str_error_r(errno, buf, sizeof(buf)));
> > + goto out;
> > + }
> > +
> > + /* read contents to stdout */
> > + while (true) {
> > + int n = read(fd, buf, sizeof(buf));
> > + if (n == 0)
> > + break;
> > + else if (n < 0)
> > + goto out_close;
> > +
> > + if (fwrite(buf, n, 1, stdout) != 1)
> > + goto out_close;
> > + }
> > + ret = 0;
> > +
> > +out_close:
> > + close(fd);
> > +out:
> > + put_tracing_file(file);
> > + return ret;
> > +}
> > +
> > static int reset_tracing_cpu(void);
> > static void reset_tracing_filters(void);
> >
> > @@ -302,6 +343,9 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> > signal(SIGCHLD, sig_handler);
> > signal(SIGPIPE, sig_handler);
> >
> > + if (ftrace->list_avail_functions)
> > + return read_tracing_file_to_stdout("available_filter_functions");
> > +
> > if (reset_tracing_files(ftrace) < 0) {
> > pr_err("failed to reset ftrace\n");
> > goto out;
> > @@ -487,6 +531,8 @@ int cmd_ftrace(int argc, const char **argv)
> > const struct option ftrace_options[] = {
> > OPT_STRING('t', "tracer", &ftrace.tracer, "tracer",
> > "tracer to use: function_graph(default) or function"),
> > + OPT_BOOLEAN('F', "funcs", &ftrace.list_avail_functions,
> > + "Show available functions to filter"),
>
> A boolean? I guess this can be a first step, but why not do it like
> 'perf probe' and accept a patterna? Allows for less typing:
>
> # perf probe -h -F
>
> -F, --funcs <[FILTER]>
> Show potential probe-able functions.
>
> [root@quaco ~]#
>
> # perf probe -F '*btf_parse*'
> btf_parse_hdr
> btf_parse_str_sec
> btf_parse_vmlinux
> #
> # perf ftrace -F | grep btf_parse
> btf_parse_str_sec
> btf_parse_hdr
> btf_parse_vmlinux
> #
>
> I'm applying this to make progress, but please consider sending a patch
> to make this behave in the same way as 'perf probe'.
>
No problem, I'll try to provide similar behaviour for this. Thanks.
> - Arnaldo
>
>
> > OPT_STRING('p', "pid", &ftrace.target.pid, "pid",
> > "trace on existing process id"),
> > OPT_INCR('v', "verbose", &verbose,
> > --
> > 2.25.1
> >
>
> --
>
> - Arnaldo
--
Cheers,
Changbin Du
On Fri, Aug 14, 2020 at 09:28:07AM -0300, Arnaldo Carvalho de Melo wrote:
> Em Sat, Aug 08, 2020 at 10:31:34AM +0800, Changbin Du escreveu:
> > This adds support to display irq context info for function tracer. To do
> > this, just specify a '--func-opts irq-info' option.
>
> Applied, ditto about adding an example to perf-ftrace.txt
>
no problem, I will do it some days later. Thanks.
> > Signed-off-by: Changbin Du <[email protected]>
> > ---
> > tools/perf/Documentation/perf-ftrace.txt | 1 +
> > tools/perf/builtin-ftrace.c | 21 ++++++++++++++++++++-
> > 2 files changed, 21 insertions(+), 1 deletion(-)
> >
> > diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> > index fca55ac55ff3..37ea3ea97922 100644
> > --- a/tools/perf/Documentation/perf-ftrace.txt
> > +++ b/tools/perf/Documentation/perf-ftrace.txt
> > @@ -79,6 +79,7 @@ OPTIONS
> > --func-opts::
> > List of options allowed to set:
> > call-graph - Display kernel stack trace for function tracer.
> > + irq-info - Display irq context info for function tracer.
> >
> > -G::
> > --graph-funcs=::
> > diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> > index b4c821be4fb5..d1241febe143 100644
> > --- a/tools/perf/builtin-ftrace.c
> > +++ b/tools/perf/builtin-ftrace.c
> > @@ -44,6 +44,7 @@ struct perf_ftrace {
> > unsigned long percpu_buffer_size;
> > bool inherit;
> > int func_stack_trace;
> > + int func_irq_info;
> > int graph_nosleep_time;
> > int graph_noirqs;
> > };
> > @@ -209,6 +210,7 @@ static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
> > write_tracing_option_file("func_stack_trace", "0");
> > write_tracing_option_file("sleep-time", "1");
> > write_tracing_option_file("funcgraph-irqs", "1");
> > + write_tracing_option_file("irq-info", "0");
> > }
> >
> > static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
> > @@ -296,6 +298,17 @@ static int set_tracing_func_stack_trace(struct perf_ftrace *ftrace)
> > return 0;
> > }
> >
> > +static int set_tracing_func_irqinfo(struct perf_ftrace *ftrace)
> > +{
> > + if (!ftrace->func_irq_info)
> > + return 0;
> > +
> > + if (write_tracing_option_file("irq-info", "1") < 0)
> > + return -1;
> > +
> > + return 0;
> > +}
> > +
> > static int reset_tracing_cpu(void)
> > {
> > struct perf_cpu_map *cpumap = perf_cpu_map__new(NULL);
> > @@ -471,6 +484,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> > goto out_reset;
> > }
> >
> > + if (set_tracing_func_irqinfo(ftrace) < 0) {
> > + pr_err("failed to set tracing option irq-info\n");
> > + goto out_reset;
> > + }
> > +
> > if (set_tracing_filters(ftrace) < 0) {
> > pr_err("failed to set tracing filters\n");
> > goto out_reset;
> > @@ -660,6 +678,7 @@ static int parse_func_tracer_opts(const struct option *opt,
> > struct perf_ftrace *ftrace = (struct perf_ftrace *) opt->value;
> > struct sublevel_option func_tracer_opts[] = {
> > { .name = "call-graph", .value_ptr = &ftrace->func_stack_trace },
> > + { .name = "irq-info", .value_ptr = &ftrace->func_irq_info },
> > { .name = NULL, }
> > };
> >
> > @@ -742,7 +761,7 @@ int cmd_ftrace(int argc, const char **argv)
> > OPT_CALLBACK('N', "notrace-funcs", &ftrace.notrace, "func",
> > "do not trace given functions", parse_filter_func),
> > OPT_CALLBACK(0, "func-opts", &ftrace, "options",
> > - "function tracer options, available options: call-graph",
> > + "function tracer options, available options: call-graph,irq-info",
> > parse_func_tracer_opts),
> > OPT_CALLBACK('G', "graph-funcs", &ftrace.graph_funcs, "func",
> > "trace given functions using function_graph tracer",
> > --
> > 2.25.1
> >
>
> --
>
> - Arnaldo
--
Cheers,
Changbin Du