The perf has basic kernel ftrace support but lack support of most tracing
options. This serias is target to enhance the perf ftrace functionality so
that we can make full use of kernel ftrace with perf.
In general, this serias be cataloged into two main changes:
1) Improve usability of existing functions. For example, we don't need to type
extra option to select the tracer.
2) Add new options to support all other ftrace functions.
Here is a glance of all ftrace functions with this serias:
$ sudo perf ftrace -h
Usage: perf ftrace [<options>] [<command>]
or: perf ftrace [<options>] -- <command> [<options>]
-a, --all-cpus system-wide collection from all CPUs
-C, --cpu <cpu> list of cpus to monitor
-D, --delay <n> ms to wait before starting tracing after program start
-F, --funcs Show available functions to filter
-G, --graph-funcs <func>
trace given functions using function_graph tracer
-g, --nograph-funcs <func>
Set nograph filter on given functions
-m, --buffer-size <size>
size of per cpu buffer
-N, --notrace-funcs <func>
do not trace given functions
-p, --pid <pid> trace on existing process id
-t, --tid <tid> trace on existing thread id (exclusive to --pid)
-T, --trace-funcs <func>
trace given functions using function tracer
-t, --tracer <tracer>
tracer to use: function or function_graph (This option is deprecated)
-v, --verbose be more verbose
--func-opts <options>
function tracer options, available options: call-graph,irq-info
--graph-opts <options>
graph tracer options, available options: nosleep-time,noirqs,verbose,thresh=<n>,depth=<n>
--inherit trace children processes
v5:
o trivial fixes.
v4:
o add util/parse-sublevel-options.c
O remove -D/--graph-depth
v3:
o add --func-opts and --graph-opts to set tracer specific options.
o support units as a suffix for option '-m/--buffer-size'.
v2:
o patches for option '-u/--userstacktrace' and '--no-pager' are dropped.
o update all related perf documentation.
o rename some options. Now all funcgraph tracer options are prefixed with
'--graph-', while all function tracer options are prefixed with '--func-'.
o mark old options deprecated instead of removing them.
Changbin Du (17):
perf ftrace: select function/function_graph tracer automatically
perf ftrace: add option '-F/--funcs' to list available functions
perf ftrace: add option -t/--tid to filter by thread id
perf ftrace: factor out function write_tracing_file_int()
perf ftrace: add option '-m/--buffer-size' to set per-cpu buffer size
perf ftrace: show trace column header
perf ftrace: add option '--inherit' to trace children processes
perf: util: add general function to parse sublevel options
perf ftrace: add support for tracing option 'func_stack_trace'
perf ftrace: add support for trace option sleep-time
perf ftrace: add support for trace option funcgraph-irqs
perf ftrace: add support for tracing option 'irq-info'
perf ftrace: add option 'verbose' to show more info for graph tracer
perf ftrace: add support for trace option tracing_thresh
perf: ftrace: allow set graph depth by '--graph-opts'
perf ftrace: add option -D/--delay to delay tracing
perf ftrace: add change log
tools/perf/Documentation/perf-config.txt | 5 -
tools/perf/Documentation/perf-ftrace.txt | 37 ++-
tools/perf/builtin-ftrace.c | 367 ++++++++++++++++++++++-
tools/perf/util/Build | 1 +
tools/perf/util/debug.c | 61 ++--
tools/perf/util/parse-sublevel-options.c | 70 +++++
tools/perf/util/parse-sublevel-options.h | 11 +
7 files changed, 484 insertions(+), 68 deletions(-)
create mode 100644 tools/perf/util/parse-sublevel-options.c
create mode 100644 tools/perf/util/parse-sublevel-options.h
--
2.25.1
The '-g/-G' options have already implied function_graph tracer should be
used instead of function tracer. So the extra option '--tracer' can be
killed.
This patch changes the behavior as below:
- By default, function tracer is used.
- If '-g' or '-G' option is on, then function_graph tracer is used.
- The perf configuration item 'ftrace.tracer' is marked as deprecated.
- The option '--tracer' is marked as deprecated.
Here are some examples.
This will start tracing all functions using function tracer:
$ sudo perf ftrace
This will trace all functions using function graph tracer:
$ sudo perf ftrace -G '*'
This will trace function vfs_read using function graph tracer:
$ sudo perf ftrace -G vfs_read
Signed-off-by: Changbin Du <[email protected]>
---
v3: remove default '*' for -G/-T.
---
tools/perf/Documentation/perf-config.txt | 5 -----
tools/perf/Documentation/perf-ftrace.txt | 2 +-
tools/perf/builtin-ftrace.c | 15 ++++++++++-----
3 files changed, 11 insertions(+), 11 deletions(-)
diff --git a/tools/perf/Documentation/perf-config.txt b/tools/perf/Documentation/perf-config.txt
index c7d3df5798e2..a25fee7de3b2 100644
--- a/tools/perf/Documentation/perf-config.txt
+++ b/tools/perf/Documentation/perf-config.txt
@@ -612,11 +612,6 @@ trace.*::
"libbeauty", the default, to use the same argument beautifiers used in the
strace-like sys_enter+sys_exit lines.
-ftrace.*::
- ftrace.tracer::
- Can be used to select the default tracer. Possible values are
- 'function' and 'function_graph'.
-
llvm.*::
llvm.clang-path::
Path to clang. If omit, search it from $PATH.
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index b80c84307dc9..952e46669168 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -24,7 +24,7 @@ OPTIONS
-t::
--tracer=::
- Tracer to use: function_graph or function.
+ Tracer to use: function_graph or function. This option is deprecated.
-v::
--verbose=::
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index 2bfc1b0db536..5f53da87040d 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -27,7 +27,6 @@
#include "util/cap.h"
#include "util/config.h"
-#define DEFAULT_TRACER "function_graph"
struct perf_ftrace {
struct evlist *evlist;
@@ -419,6 +418,7 @@ static int perf_ftrace_config(const char *var, const char *value, void *cb)
if (strcmp(var, "ftrace.tracer"))
return -1;
+ pr_warning("Configuration ftrace.tracer is deprecated\n");
if (!strcmp(value, "function_graph") ||
!strcmp(value, "function")) {
ftrace->tracer = value;
@@ -459,7 +459,7 @@ int cmd_ftrace(int argc, const char **argv)
{
int ret;
struct perf_ftrace ftrace = {
- .tracer = DEFAULT_TRACER,
+ .tracer = "function",
.target = { .uid = UINT_MAX, },
};
const char * const ftrace_usage[] = {
@@ -469,7 +469,7 @@ int cmd_ftrace(int argc, const char **argv)
};
const struct option ftrace_options[] = {
OPT_STRING('t', "tracer", &ftrace.tracer, "tracer",
- "tracer to use: function_graph(default) or function"),
+ "tracer to use: function or function_graph (This option is deprecated)"),
OPT_STRING('p', "pid", &ftrace.target.pid, "pid",
"trace on existing process id"),
OPT_INCR('v', "verbose", &verbose,
@@ -479,11 +479,13 @@ int cmd_ftrace(int argc, const char **argv)
OPT_STRING('C', "cpu", &ftrace.target.cpu_list, "cpu",
"list of cpus to monitor"),
OPT_CALLBACK('T', "trace-funcs", &ftrace.filters, "func",
- "trace given functions only", parse_filter_func),
+ "trace given functions using function tracer",
+ parse_filter_func),
OPT_CALLBACK('N', "notrace-funcs", &ftrace.notrace, "func",
"do not trace given functions", parse_filter_func),
OPT_CALLBACK('G', "graph-funcs", &ftrace.graph_funcs, "func",
- "Set graph filter on given functions", parse_filter_func),
+ "trace given functions using function_graph tracer",
+ parse_filter_func),
OPT_CALLBACK('g', "nograph-funcs", &ftrace.nograph_funcs, "func",
"Set nograph filter on given functions", parse_filter_func),
OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
@@ -505,6 +507,9 @@ int cmd_ftrace(int argc, const char **argv)
if (!argc && target__none(&ftrace.target))
ftrace.target.system_wide = true;
+ if (!list_empty(&ftrace.graph_funcs) || !list_empty(&ftrace.nograph_funcs))
+ ftrace.tracer = "function_graph";
+
ret = target__validate(&ftrace.target);
if (ret) {
char errbuf[512];
--
2.25.1
This adds an option '-F/--funcs' to list all available functions to trace,
which is read from tracing file 'available_filter_functions'.
$ sudo ./perf ftrace -F | head
trace_initcall_finish_cb
initcall_blacklisted
do_one_initcall
do_one_initcall
trace_initcall_start_cb
run_init_process
try_to_run_init_process
match_dev_by_label
match_dev_by_uuid
rootfs_init_fs_context
Signed-off-by: Changbin Du <[email protected]>
---
v2: option name '-l/--list-functions' -> '-F/--funcs'
---
tools/perf/Documentation/perf-ftrace.txt | 4 +++
tools/perf/builtin-ftrace.c | 43 ++++++++++++++++++++++++
2 files changed, 47 insertions(+)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index 952e46669168..d79560dea19f 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -30,6 +30,10 @@ OPTIONS
--verbose=::
Verbosity level.
+-F::
+--funcs::
+ List all available functions to trace.
+
-p::
--pid=::
Trace on existing process id (comma separated list).
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index 5f53da87040d..244cc8e6bd60 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -32,6 +32,7 @@ struct perf_ftrace {
struct evlist *evlist;
struct target target;
const char *tracer;
+ bool list_avail_functions;
struct list_head filters;
struct list_head notrace;
struct list_head graph_funcs;
@@ -127,6 +128,43 @@ static int append_tracing_file(const char *name, const char *val)
return __write_tracing_file(name, val, true);
}
+static int read_tracing_file_to_stdout(const char *name)
+{
+ char buf[4096];
+ char *file;
+ int fd;
+ int ret = -1;
+
+ file = get_tracing_file(name);
+ if (!file) {
+ pr_debug("cannot get tracing file: %s\n", name);
+ return -1;
+ }
+
+ fd = open(file, O_RDONLY);
+ if (fd < 0) {
+ pr_debug("cannot open tracing file: %s: %s\n",
+ name, str_error_r(errno, buf, sizeof(buf)));
+ goto out;
+ }
+
+ /* read contents to stdout */
+ while (true) {
+ int n = read(fd, buf, sizeof(buf));
+ if (n <= 0)
+ goto out_close;
+ if (fwrite(buf, n, 1, stdout) != 1)
+ goto out_close;
+ }
+ ret = 0;
+
+out_close:
+ close(fd);
+out:
+ put_tracing_file(file);
+ return ret;
+}
+
static int reset_tracing_cpu(void);
static void reset_tracing_filters(void);
@@ -301,6 +339,9 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
signal(SIGCHLD, sig_handler);
signal(SIGPIPE, sig_handler);
+ if (ftrace->list_avail_functions)
+ return read_tracing_file_to_stdout("available_filter_functions");
+
if (reset_tracing_files(ftrace) < 0) {
pr_err("failed to reset ftrace\n");
goto out;
@@ -470,6 +511,8 @@ int cmd_ftrace(int argc, const char **argv)
const struct option ftrace_options[] = {
OPT_STRING('t', "tracer", &ftrace.tracer, "tracer",
"tracer to use: function or function_graph (This option is deprecated)"),
+ OPT_BOOLEAN('F', "funcs", &ftrace.list_avail_functions,
+ "Show available functions to filter"),
OPT_STRING('p', "pid", &ftrace.target.pid, "pid",
"trace on existing process id"),
OPT_INCR('v', "verbose", &verbose,
--
2.25.1
This allows us to trace single thread instead of the whole process.
Signed-off-by: Changbin Du <[email protected]>
---
tools/perf/Documentation/perf-ftrace.txt | 4 ++++
tools/perf/builtin-ftrace.c | 2 ++
2 files changed, 6 insertions(+)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index d79560dea19f..e204bf6d50d8 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -38,6 +38,10 @@ OPTIONS
--pid=::
Trace on existing process id (comma separated list).
+-t::
+--tid=::
+ Trace on existing thread id (comma separated list).
+
-a::
--all-cpus::
Force system-wide collection. Scripts run without a <command>
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index 244cc8e6bd60..1188b82c6541 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -515,6 +515,8 @@ int cmd_ftrace(int argc, const char **argv)
"Show available functions to filter"),
OPT_STRING('p', "pid", &ftrace.target.pid, "pid",
"trace on existing process id"),
+ OPT_STRING('t', "tid", &ftrace.target.tid, "tid",
+ "trace on existing thread id (exclusive to --pid)"),
OPT_INCR('v', "verbose", &verbose,
"be more verbose"),
OPT_BOOLEAN('a', "all-cpus", &ftrace.target.system_wide,
--
2.25.1
We will reuse this function later.
Signed-off-by: Changbin Du <[email protected]>
---
tools/perf/builtin-ftrace.c | 17 ++++++++++++-----
1 file changed, 12 insertions(+), 5 deletions(-)
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index 1188b82c6541..342861a1d152 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -165,6 +165,17 @@ static int read_tracing_file_to_stdout(const char *name)
return ret;
}
+static int write_tracing_file_int(const char *name, int value)
+{
+ char buf[16];
+
+ snprintf(buf, sizeof(buf), "%d", value);
+ if (write_tracing_file(name, buf) < 0)
+ return -1;
+
+ return 0;
+}
+
static int reset_tracing_cpu(void);
static void reset_tracing_filters(void);
@@ -295,8 +306,6 @@ static void reset_tracing_filters(void)
static int set_tracing_depth(struct perf_ftrace *ftrace)
{
- char buf[16];
-
if (ftrace->graph_depth == 0)
return 0;
@@ -305,9 +314,7 @@ static int set_tracing_depth(struct perf_ftrace *ftrace)
return -1;
}
- snprintf(buf, sizeof(buf), "%d", ftrace->graph_depth);
-
- if (write_tracing_file("max_graph_depth", buf) < 0)
+ if (write_tracing_file_int("max_graph_depth", ftrace->graph_depth) < 0)
return -1;
return 0;
--
2.25.1
This makes perf-ftrace display column header before printing trace.
$ sudo perf ftrace
# tracer: function
#
# entries-in-buffer/entries-written: 0/0 #P:8
#
# TASK-PID CPU# TIMESTAMP FUNCTION
# | | | | |
<...>-9246 [006] 10726.262760: mutex_unlock <-rb_simple_write
<...>-9246 [006] 10726.262764: __fsnotify_parent <-vfs_write
<...>-9246 [006] 10726.262765: fsnotify <-vfs_write
<...>-9246 [006] 10726.262766: __sb_end_write <-vfs_write
<...>-9246 [006] 10726.262767: fpregs_assert_state_consistent <-do_syscall_64
Signed-off-by: Changbin Du <[email protected]>
---
tools/perf/builtin-ftrace.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index 348e2d960987..887e78b23a82 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -430,6 +430,9 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
fcntl(trace_fd, F_SETFL, O_NONBLOCK);
pollfd.fd = trace_fd;
+ /* display column headers */
+ read_tracing_file_to_stdout("trace");
+
if (write_tracing_file("tracing_on", "1") < 0) {
pr_err("can't enable tracing\n");
goto out_close_fd;
--
2.25.1
This factors out a general function perf_parse_sublevel_options() to parse
sublevel options. The 'sublevel' options is something like the '--debug'
options which allow more sublevel options.
Signed-off-by: Changbin Du <[email protected]>
---
v2: add util/parse-sublevel-options.c
---
tools/perf/util/Build | 1 +
tools/perf/util/debug.c | 61 ++++++---------------
tools/perf/util/parse-sublevel-options.c | 70 ++++++++++++++++++++++++
tools/perf/util/parse-sublevel-options.h | 11 ++++
4 files changed, 99 insertions(+), 44 deletions(-)
create mode 100644 tools/perf/util/parse-sublevel-options.c
create mode 100644 tools/perf/util/parse-sublevel-options.h
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index 8d18380ecd10..e86607ada0b5 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -117,6 +117,7 @@ endif
perf-y += parse-branch-options.o
perf-y += dump-insn.o
perf-y += parse-regs-options.o
+perf-y += parse-sublevel-options.o
perf-y += term.o
perf-y += help-unknown-cmd.o
perf-y += mem-events.o
diff --git a/tools/perf/util/debug.c b/tools/perf/util/debug.c
index adb656745ecc..5cda5565777a 100644
--- a/tools/perf/util/debug.c
+++ b/tools/perf/util/debug.c
@@ -20,6 +20,7 @@
#include "target.h"
#include "ui/helpline.h"
#include "ui/ui.h"
+#include "util/parse-sublevel-options.h"
#include <linux/ctype.h>
@@ -173,65 +174,37 @@ void trace_event(union perf_event *event)
trace_event_printer, event);
}
-static struct debug_variable {
- const char *name;
- int *ptr;
-} debug_variables[] = {
- { .name = "verbose", .ptr = &verbose },
- { .name = "ordered-events", .ptr = &debug_ordered_events},
- { .name = "stderr", .ptr = &redirect_to_stderr},
- { .name = "data-convert", .ptr = &debug_data_convert },
- { .name = "perf-event-open", .ptr = &debug_peo_args },
+static struct sublevel_option debug_opts[] = {
+ { .name = "verbose", .value_ptr = &verbose },
+ { .name = "ordered-events", .value_ptr = &debug_ordered_events},
+ { .name = "stderr", .value_ptr = &redirect_to_stderr},
+ { .name = "data-convert", .value_ptr = &debug_data_convert },
+ { .name = "perf-event-open", .value_ptr = &debug_peo_args },
{ .name = NULL, }
};
int perf_debug_option(const char *str)
{
- struct debug_variable *var = &debug_variables[0];
- char *vstr, *s = strdup(str);
- int v = 1;
-
- vstr = strchr(s, '=');
- if (vstr)
- *vstr++ = 0;
-
- while (var->name) {
- if (!strcmp(s, var->name))
- break;
- var++;
- }
-
- if (!var->name) {
- pr_err("Unknown debug variable name '%s'\n", s);
- free(s);
- return -1;
- }
+ int ret;
- if (vstr) {
- v = atoi(vstr);
- /*
- * Allow only values in range (0, 10),
- * otherwise set 0.
- */
- v = (v < 0) || (v > 10) ? 0 : v;
- }
+ ret = perf_parse_sublevel_options(str, debug_opts);
+ if (ret)
+ return ret;
- if (quiet)
- v = -1;
+ /* Allow only verbose value in range (0, 10), otherwise set 0. */
+ verbose = (verbose < 0) || (verbose > 10) ? 0 : verbose;
- *var->ptr = v;
- free(s);
return 0;
}
int perf_quiet_option(void)
{
- struct debug_variable *var = &debug_variables[0];
+ struct sublevel_option *opt = &debug_opts[0];
/* disable all debug messages */
- while (var->name) {
- *var->ptr = -1;
- var++;
+ while (opt->name) {
+ *opt->value_ptr = -1;
+ opt++;
}
return 0;
diff --git a/tools/perf/util/parse-sublevel-options.c b/tools/perf/util/parse-sublevel-options.c
new file mode 100644
index 000000000000..a841d17ffd57
--- /dev/null
+++ b/tools/perf/util/parse-sublevel-options.c
@@ -0,0 +1,70 @@
+#include <stdlib.h>
+#include <stdint.h>
+#include <string.h>
+#include <stdio.h>
+
+#include "util/debug.h"
+#include "util/parse-sublevel-options.h"
+
+static int parse_one_sublevel_option(const char *str,
+ struct sublevel_option *opts)
+{
+ struct sublevel_option *opt = opts;
+ char *vstr, *s = strdup(str);
+ int v = 1;
+
+ if (!s) {
+ pr_err("no memory\n");
+ return -1;
+ }
+
+ vstr = strchr(s, '=');
+ if (vstr)
+ *vstr++ = 0;
+
+ while (opt->name) {
+ if (!strcmp(s, opt->name))
+ break;
+ opt++;
+ }
+
+ if (!opt->name) {
+ pr_err("Unknown option name '%s'\n", s);
+ free(s);
+ return -1;
+ }
+
+ if (vstr)
+ v = atoi(vstr);
+
+ *opt->value_ptr = v;
+ free(s);
+ return 0;
+}
+
+/* parse options like --foo a=<n>,b,c... */
+int perf_parse_sublevel_options(const char *str, struct sublevel_option *opts)
+{
+ char *s = strdup(str);
+ char *p = NULL;
+ int ret;
+
+ if (!s) {
+ pr_err("no memory\n");
+ return -1;
+ }
+
+ p = strtok(s, ",");
+ while (p) {
+ ret = parse_one_sublevel_option(p, opts);
+ if (ret) {
+ free(s);
+ return ret;
+ }
+
+ p = strtok(NULL, ",");
+ }
+
+ free(s);
+ return 0;
+}
diff --git a/tools/perf/util/parse-sublevel-options.h b/tools/perf/util/parse-sublevel-options.h
new file mode 100644
index 000000000000..9b9efcc2aaad
--- /dev/null
+++ b/tools/perf/util/parse-sublevel-options.h
@@ -0,0 +1,11 @@
+#ifndef _PERF_PARSE_SUBLEVEL_OPTIONS_H
+#define _PERF_PARSE_SUBLEVEL_OPTIONS_H
+
+struct sublevel_option {
+ const char *name;
+ int *value_ptr;
+};
+
+int perf_parse_sublevel_options(const char *str, struct sublevel_option *opts);
+
+#endif
\ No newline at end of file
--
2.25.1
This adds support to display call trace for function tracer. To do this,
just specify a '--func-opts call-graph' option.
$ sudo perf ftrace -T vfs_read --func-opts call-graph
iio-sensor-prox-855 [003] 6168.369657: vfs_read <-ksys_read
iio-sensor-prox-855 [003] 6168.369677: <stack trace>
=> vfs_read
=> ksys_read
=> __x64_sys_read
=> do_syscall_64
=> entry_SYSCALL_64_after_hwframe
...
Signed-off-by: Changbin Du <[email protected]>
---
v3: switch to uniform option --func-opts.
v2: option name '-s' -> '--func-call-graph'
---
tools/perf/Documentation/perf-ftrace.txt | 4 +++
tools/perf/builtin-ftrace.c | 42 ++++++++++++++++++++++++
2 files changed, 46 insertions(+)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index fd632bd9b2c1..676a30cb9b5a 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -78,6 +78,10 @@ OPTIONS
(or glob patterns). It will be passed to 'set_ftrace_notrace'
in tracefs.
+--func-opts::
+ List of options allowed to set:
+ call-graph - Display kernel stack trace for function tracer.
+
-G::
--graph-funcs=::
Set graph filter on the given function (or a glob pattern).
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index 4efaa7b6a906..91611eef5deb 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -27,6 +27,7 @@
#include "util/cap.h"
#include "util/config.h"
#include "util/units.h"
+#include "util/parse-sublevel-options.h"
struct perf_ftrace {
struct evlist *evlist;
@@ -40,6 +41,7 @@ struct perf_ftrace {
int graph_depth;
unsigned long percpu_buffer_size;
bool inherit;
+ int func_stack_trace;
};
struct filter_entry {
@@ -197,6 +199,7 @@ static void reset_tracing_filters(void);
static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
{
write_tracing_option_file("function-fork", "0");
+ write_tracing_option_file("func_stack_trace", "0");
}
static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
@@ -273,6 +276,17 @@ static int set_tracing_cpu(struct perf_ftrace *ftrace)
return set_tracing_cpumask(cpumap);
}
+static int set_tracing_func_stack_trace(struct perf_ftrace *ftrace)
+{
+ if (!ftrace->func_stack_trace)
+ return 0;
+
+ if (write_tracing_option_file("func_stack_trace", "1") < 0)
+ return -1;
+
+ return 0;
+}
+
static int reset_tracing_cpu(void)
{
struct perf_cpu_map *cpumap = perf_cpu_map__new(NULL);
@@ -421,6 +435,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
goto out_reset;
}
+ if (set_tracing_func_stack_trace(ftrace) < 0) {
+ pr_err("failed to set tracing option func_stack_trace\n");
+ goto out_reset;
+ }
+
if (set_tracing_filters(ftrace) < 0) {
pr_err("failed to set tracing filters\n");
goto out_reset;
@@ -594,6 +613,26 @@ static int parse_buffer_size(const struct option *opt,
return -1;
}
+static int parse_func_tracer_opts(const struct option *opt,
+ const char *str, int unset)
+{
+ int ret;
+ struct perf_ftrace *ftrace = (struct perf_ftrace *) opt->value;
+ struct sublevel_option func_tracer_opts[] = {
+ { .name = "call-graph", .value_ptr = &ftrace->func_stack_trace },
+ { .name = NULL, }
+ };
+
+ if (unset)
+ return 0;
+
+ ret = perf_parse_sublevel_options(str, func_tracer_opts);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
int cmd_ftrace(int argc, const char **argv)
{
int ret;
@@ -626,6 +665,9 @@ int cmd_ftrace(int argc, const char **argv)
parse_filter_func),
OPT_CALLBACK('N', "notrace-funcs", &ftrace.notrace, "func",
"do not trace given functions", parse_filter_func),
+ OPT_CALLBACK(0, "func-opts", &ftrace, "options",
+ "function tracer options, available options: call-graph",
+ parse_func_tracer_opts),
OPT_CALLBACK('G', "graph-funcs", &ftrace.graph_funcs, "func",
"trace given functions using function_graph tracer",
parse_filter_func),
--
2.25.1
This adds an option '--graph-opts nosleep-time' which allow us
only to measure on-CPU time. This option is function_graph tracer
only.
Signed-off-by: Changbin Du <[email protected]>
---
v3: switch to uniform option --graph-opts.
v2: option name '--nosleep-time' -> '--graph-nosleep-time'.
---
tools/perf/Documentation/perf-ftrace.txt | 4 +++
tools/perf/builtin-ftrace.c | 41 ++++++++++++++++++++++++
2 files changed, 45 insertions(+)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index 676a30cb9b5a..760c2b78c305 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -102,6 +102,10 @@ OPTIONS
--graph-depth=::
Set max depth for function graph tracer to follow
+--graph-opts::
+ List of options allowed to set:
+ nosleep-time - Measure on-CPU time only for function_graph tracer.
+
SEE ALSO
--------
linkperf:perf-record[1], linkperf:perf-trace[1]
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index 91611eef5deb..b4eda459ba78 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -42,6 +42,7 @@ struct perf_ftrace {
unsigned long percpu_buffer_size;
bool inherit;
int func_stack_trace;
+ int graph_nosleep_time;
};
struct filter_entry {
@@ -200,6 +201,7 @@ static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
{
write_tracing_option_file("function-fork", "0");
write_tracing_option_file("func_stack_trace", "0");
+ write_tracing_option_file("sleep-time", "1");
}
static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
@@ -381,6 +383,17 @@ static int set_tracing_trace_inherit(struct perf_ftrace *ftrace)
return 0;
}
+static int set_tracing_sleep_time(struct perf_ftrace *ftrace)
+{
+ if (!ftrace->graph_nosleep_time)
+ return 0;
+
+ if (write_tracing_option_file("sleep-time", "0") < 0)
+ return -1;
+
+ return 0;
+}
+
static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
{
char *trace_file;
@@ -460,6 +473,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
goto out_reset;
}
+ if (set_tracing_sleep_time(ftrace) < 0) {
+ pr_err("failed to set tracing option sleep-time\n");
+ goto out_reset;
+ }
+
if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
goto out_reset;
@@ -633,6 +651,26 @@ static int parse_func_tracer_opts(const struct option *opt,
return 0;
}
+static int parse_graph_tracer_opts(const struct option *opt,
+ const char *str, int unset)
+{
+ int ret;
+ struct perf_ftrace *ftrace = (struct perf_ftrace *) opt->value;
+ struct sublevel_option graph_tracer_opts[] = {
+ { .name = "nosleep-time", .value_ptr = &ftrace->graph_nosleep_time },
+ { .name = NULL, }
+ };
+
+ if (unset)
+ return 0;
+
+ ret = perf_parse_sublevel_options(str, graph_tracer_opts);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
int cmd_ftrace(int argc, const char **argv)
{
int ret;
@@ -675,6 +713,9 @@ int cmd_ftrace(int argc, const char **argv)
"Set nograph filter on given functions", parse_filter_func),
OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
"Max depth for function graph tracer"),
+ OPT_CALLBACK(0, "graph-opts", &ftrace, "options",
+ "graph tracer options, available options: nosleep-time",
+ parse_graph_tracer_opts),
OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
"size of per cpu buffer", parse_buffer_size),
OPT_BOOLEAN(0, "inherit", &ftrace.inherit,
--
2.25.1
This adds an option '--graph-opts noirqs' to filter out functions executed
in irq context.
Signed-off-by: Changbin Du <[email protected]>
---
v2: option name '--nofuncgraph-irqs' -> '--graph-noirqs'.
---
tools/perf/Documentation/perf-ftrace.txt | 1 +
tools/perf/builtin-ftrace.c | 21 ++++++++++++++++++++-
2 files changed, 21 insertions(+), 1 deletion(-)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index 760c2b78c305..2e16e1e588d9 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -105,6 +105,7 @@ OPTIONS
--graph-opts::
List of options allowed to set:
nosleep-time - Measure on-CPU time only for function_graph tracer.
+ noirqs - Ignore functions that happen inside interrupt.
SEE ALSO
--------
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index b4eda459ba78..179c5da678e3 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -43,6 +43,7 @@ struct perf_ftrace {
bool inherit;
int func_stack_trace;
int graph_nosleep_time;
+ int graph_noirqs;
};
struct filter_entry {
@@ -202,6 +203,7 @@ static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
write_tracing_option_file("function-fork", "0");
write_tracing_option_file("func_stack_trace", "0");
write_tracing_option_file("sleep-time", "1");
+ write_tracing_option_file("funcgraph-irqs", "1");
}
static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
@@ -394,6 +396,17 @@ static int set_tracing_sleep_time(struct perf_ftrace *ftrace)
return 0;
}
+static int set_tracing_funcgraph_irqs(struct perf_ftrace *ftrace)
+{
+ if (!ftrace->graph_noirqs)
+ return 0;
+
+ if (write_tracing_option_file("funcgraph-irqs", "0") < 0)
+ return -1;
+
+ return 0;
+}
+
static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
{
char *trace_file;
@@ -478,6 +491,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
goto out_reset;
}
+ if (set_tracing_funcgraph_irqs(ftrace) < 0) {
+ pr_err("failed to set tracing option funcgraph-irqs\n");
+ goto out_reset;
+ }
+
if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
goto out_reset;
@@ -658,6 +676,7 @@ static int parse_graph_tracer_opts(const struct option *opt,
struct perf_ftrace *ftrace = (struct perf_ftrace *) opt->value;
struct sublevel_option graph_tracer_opts[] = {
{ .name = "nosleep-time", .value_ptr = &ftrace->graph_nosleep_time },
+ { .name = "noirqs", .value_ptr = &ftrace->graph_noirqs },
{ .name = NULL, }
};
@@ -714,7 +733,7 @@ int cmd_ftrace(int argc, const char **argv)
OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
"Max depth for function graph tracer"),
OPT_CALLBACK(0, "graph-opts", &ftrace, "options",
- "graph tracer options, available options: nosleep-time",
+ "graph tracer options, available options: nosleep-time,noirqs",
parse_graph_tracer_opts),
OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
"size of per cpu buffer", parse_buffer_size),
--
2.25.1
This adds support to display irq context info for function tracer. To do
this, just specify a '--func-opts irq-info' option.
Signed-off-by: Changbin Du <[email protected]>
---
tools/perf/Documentation/perf-ftrace.txt | 1 +
tools/perf/builtin-ftrace.c | 21 ++++++++++++++++++++-
2 files changed, 21 insertions(+), 1 deletion(-)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index 2e16e1e588d9..e6e9564d6c2e 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -81,6 +81,7 @@ OPTIONS
--func-opts::
List of options allowed to set:
call-graph - Display kernel stack trace for function tracer.
+ irq-info - Display irq context info for function tracer.
-G::
--graph-funcs=::
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index 179c5da678e3..a5906258c413 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -42,6 +42,7 @@ struct perf_ftrace {
unsigned long percpu_buffer_size;
bool inherit;
int func_stack_trace;
+ int func_irq_info;
int graph_nosleep_time;
int graph_noirqs;
};
@@ -204,6 +205,7 @@ static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
write_tracing_option_file("func_stack_trace", "0");
write_tracing_option_file("sleep-time", "1");
write_tracing_option_file("funcgraph-irqs", "1");
+ write_tracing_option_file("irq-info", "0");
}
static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
@@ -291,6 +293,17 @@ static int set_tracing_func_stack_trace(struct perf_ftrace *ftrace)
return 0;
}
+static int set_tracing_func_irqinfo(struct perf_ftrace *ftrace)
+{
+ if (!ftrace->func_irq_info)
+ return 0;
+
+ if (write_tracing_option_file("irq-info", "1") < 0)
+ return -1;
+
+ return 0;
+}
+
static int reset_tracing_cpu(void)
{
struct perf_cpu_map *cpumap = perf_cpu_map__new(NULL);
@@ -466,6 +479,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
goto out_reset;
}
+ if (set_tracing_func_irqinfo(ftrace) < 0) {
+ pr_err("failed to set tracing option irq-info\n");
+ goto out_reset;
+ }
+
if (set_tracing_filters(ftrace) < 0) {
pr_err("failed to set tracing filters\n");
goto out_reset;
@@ -656,6 +674,7 @@ static int parse_func_tracer_opts(const struct option *opt,
struct perf_ftrace *ftrace = (struct perf_ftrace *) opt->value;
struct sublevel_option func_tracer_opts[] = {
{ .name = "call-graph", .value_ptr = &ftrace->func_stack_trace },
+ { .name = "irq-info", .value_ptr = &ftrace->func_irq_info },
{ .name = NULL, }
};
@@ -723,7 +742,7 @@ int cmd_ftrace(int argc, const char **argv)
OPT_CALLBACK('N', "notrace-funcs", &ftrace.notrace, "func",
"do not trace given functions", parse_filter_func),
OPT_CALLBACK(0, "func-opts", &ftrace, "options",
- "function tracer options, available options: call-graph",
+ "function tracer options, available options: call-graph,irq-info",
parse_func_tracer_opts),
OPT_CALLBACK('G', "graph-funcs", &ftrace.graph_funcs, "func",
"trace given functions using function_graph tracer",
--
2.25.1
This adds an option '--graph-opts thresh' to setup trace duration
threshold for funcgraph tracer.
$ sudo ./perf ftrace -G --graph-opts thresh=100
3) ! 184.060 us | } /* schedule */
3) ! 185.600 us | } /* exit_to_usermode_loop */
2) ! 225.989 us | } /* schedule_idle */
2) # 4140.051 us | } /* do_idle */
Signed-off-by: Changbin Du <[email protected]>
---
tools/perf/Documentation/perf-ftrace.txt | 1 +
tools/perf/builtin-ftrace.c | 26 +++++++++++++++++++++++-
2 files changed, 26 insertions(+), 1 deletion(-)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index 0dd5ef4f9c65..a103e3fd8d42 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -108,6 +108,7 @@ OPTIONS
nosleep-time - Measure on-CPU time only for function_graph tracer.
noirqs - Ignore functions that happen inside interrupt.
verbose - Show process names, PIDs, timestamps, etc.
+ thresh=<n> - Setup trace duration threshold in microseconds.
SEE ALSO
--------
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index d169d6329454..c6d0194e4d0e 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -46,6 +46,7 @@ struct perf_ftrace {
int graph_nosleep_time;
int graph_noirqs;
int graph_verbose;
+ int graph_thresh;
};
struct filter_entry {
@@ -229,6 +230,9 @@ static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
if (write_tracing_file("max_graph_depth", "0") < 0)
return -1;
+ if (write_tracing_file("tracing_thresh", "0") < 0)
+ return -1;
+
reset_tracing_filters();
reset_tracing_options(ftrace);
return 0;
@@ -441,6 +445,20 @@ static int set_tracing_funcgraph_verbose(struct perf_ftrace *ftrace)
return 0;
}
+static int set_tracing_thresh(struct perf_ftrace *ftrace)
+{
+ int ret;
+
+ if (ftrace->graph_thresh == 0)
+ return 0;
+
+ ret = write_tracing_file_int("tracing_thresh", ftrace->graph_thresh);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
+
static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
{
char *trace_file;
@@ -540,6 +558,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
goto out_reset;
}
+ if (set_tracing_thresh(ftrace) < 0) {
+ pr_err("failed to set tracing thresh\n");
+ goto out_reset;
+ }
+
if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
goto out_reset;
@@ -723,6 +746,7 @@ static int parse_graph_tracer_opts(const struct option *opt,
{ .name = "nosleep-time", .value_ptr = &ftrace->graph_nosleep_time },
{ .name = "noirqs", .value_ptr = &ftrace->graph_noirqs },
{ .name = "verbose", .value_ptr = &ftrace->graph_verbose },
+ { .name = "thresh", .value_ptr = &ftrace->graph_thresh },
{ .name = NULL, }
};
@@ -779,7 +803,7 @@ int cmd_ftrace(int argc, const char **argv)
OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
"Max depth for function graph tracer"),
OPT_CALLBACK(0, "graph-opts", &ftrace, "options",
- "graph tracer options, available options: nosleep-time,noirqs,verbose",
+ "graph tracer options, available options: nosleep-time,noirqs,verbose,thresh=<n>",
parse_graph_tracer_opts),
OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
"size of per cpu buffer", parse_buffer_size),
--
2.25.1
This is to have a consistent view of all graph tracer options.
The original option '--graph-depth' is marked as deprecated.
Signed-off-by: Changbin Du <[email protected]>
---
tools/perf/Documentation/perf-ftrace.txt | 5 +----
tools/perf/builtin-ftrace.c | 5 ++---
2 files changed, 3 insertions(+), 7 deletions(-)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index a103e3fd8d42..97d89478ff5f 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -99,16 +99,13 @@ OPTIONS
This can be used more than once to specify multiple functions.
It will be passed to 'set_graph_notrace' in tracefs.
--D::
---graph-depth=::
- Set max depth for function graph tracer to follow
-
--graph-opts::
List of options allowed to set:
nosleep-time - Measure on-CPU time only for function_graph tracer.
noirqs - Ignore functions that happen inside interrupt.
verbose - Show process names, PIDs, timestamps, etc.
thresh=<n> - Setup trace duration threshold in microseconds.
+ depth=<n> - Set max depth for function graph tracer to follow.
SEE ALSO
--------
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index c6d0194e4d0e..86fc5d466e97 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -747,6 +747,7 @@ static int parse_graph_tracer_opts(const struct option *opt,
{ .name = "noirqs", .value_ptr = &ftrace->graph_noirqs },
{ .name = "verbose", .value_ptr = &ftrace->graph_verbose },
{ .name = "thresh", .value_ptr = &ftrace->graph_thresh },
+ { .name = "depth", .value_ptr = &ftrace->graph_depth },
{ .name = NULL, }
};
@@ -800,10 +801,8 @@ int cmd_ftrace(int argc, const char **argv)
parse_filter_func),
OPT_CALLBACK('g', "nograph-funcs", &ftrace.nograph_funcs, "func",
"Set nograph filter on given functions", parse_filter_func),
- OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
- "Max depth for function graph tracer"),
OPT_CALLBACK(0, "graph-opts", &ftrace, "options",
- "graph tracer options, available options: nosleep-time,noirqs,verbose,thresh=<n>",
+ "graph tracer options, available options: nosleep-time,noirqs,verbose,thresh=<n>,depth=<n>",
parse_graph_tracer_opts),
OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
"size of per cpu buffer", parse_buffer_size),
--
2.25.1
This adds an option '-D/--delay' to allow us to start tracing some
times later after workload is launched.
Signed-off-by: Changbin Du <[email protected]>
---
tools/perf/Documentation/perf-ftrace.txt | 4 ++++
tools/perf/builtin-ftrace.c | 19 ++++++++++++++++---
2 files changed, 20 insertions(+), 3 deletions(-)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index 97d89478ff5f..62f56058de54 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -42,6 +42,10 @@ OPTIONS
--tid=::
Trace on existing thread id (comma separated list).
+-D::
+--delay::
+ Time (ms) to wait before starting tracing after program start.
+
-a::
--all-cpus::
Force system-wide collection. Scripts run without a <command>
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index 86fc5d466e97..a70951938308 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -47,6 +47,7 @@ struct perf_ftrace {
int graph_noirqs;
int graph_verbose;
int graph_thresh;
+ unsigned int initial_delay;
};
struct filter_entry {
@@ -591,13 +592,23 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
/* display column headers */
read_tracing_file_to_stdout("trace");
- if (write_tracing_file("tracing_on", "1") < 0) {
- pr_err("can't enable tracing\n");
- goto out_close_fd;
+ if (!ftrace->initial_delay) {
+ if (write_tracing_file("tracing_on", "1") < 0) {
+ pr_err("can't enable tracing\n");
+ goto out_close_fd;
+ }
}
perf_evlist__start_workload(ftrace->evlist);
+ if (ftrace->initial_delay) {
+ usleep(ftrace->initial_delay * 1000);
+ if (write_tracing_file("tracing_on", "1") < 0) {
+ pr_err("can't enable tracing\n");
+ goto out_close_fd;
+ }
+ }
+
while (!done) {
if (poll(&pollfd, 1, -1) < 0)
break;
@@ -808,6 +819,8 @@ int cmd_ftrace(int argc, const char **argv)
"size of per cpu buffer", parse_buffer_size),
OPT_BOOLEAN(0, "inherit", &ftrace.inherit,
"trace children processes"),
+ OPT_UINTEGER('D', "delay", &ftrace.initial_delay,
+ "ms to wait before starting tracing after program start"),
OPT_END()
};
--
2.25.1
Add a change log after previous enhancements.
Signed-off-by: Changbin Du <[email protected]>
---
tools/perf/builtin-ftrace.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index a70951938308..56dfce646d30 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -3,6 +3,7 @@
* builtin-ftrace.c
*
* Copyright (c) 2013 LG Electronics, Namhyung Kim <[email protected]>
+ * Copyright (c) 2020 Changbin Du <[email protected]>, significant enhancement.
*/
#include "builtin.h"
--
2.25.1
This adds an option '-m/--buffer-size' to allow us set the size of per-cpu
tracing buffer.
Signed-off-by: Changbin Du <[email protected]>
---
v2: support units as a suffix.
---
tools/perf/Documentation/perf-ftrace.txt | 5 +++
tools/perf/builtin-ftrace.c | 56 +++++++++++++++++++++++-
2 files changed, 60 insertions(+), 1 deletion(-)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index e204bf6d50d8..98fe01d354d1 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -56,6 +56,11 @@ OPTIONS
Ranges of CPUs are specified with -: 0-2.
Default is to trace on all online CPUs.
+-m::
+--buffer-size::
+ Set the size of per-cpu tracing buffer, <size> is expected to
+ be a number with appended unit character - B/K/M/G.
+
-T::
--trace-funcs=::
Only trace functions given by the argument. Multiple functions
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index 342861a1d152..348e2d960987 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -26,7 +26,7 @@
#include "thread_map.h"
#include "util/cap.h"
#include "util/config.h"
-
+#include "util/units.h"
struct perf_ftrace {
struct evlist *evlist;
@@ -38,6 +38,7 @@ struct perf_ftrace {
struct list_head graph_funcs;
struct list_head nograph_funcs;
int graph_depth;
+ unsigned long percpu_buffer_size;
};
struct filter_entry {
@@ -320,6 +321,21 @@ static int set_tracing_depth(struct perf_ftrace *ftrace)
return 0;
}
+static int set_tracing_percpu_buffer_size(struct perf_ftrace *ftrace)
+{
+ int ret;
+
+ if (ftrace->percpu_buffer_size == 0)
+ return 0;
+
+ ret = write_tracing_file_int("buffer_size_kb",
+ ftrace->percpu_buffer_size / 1024);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
+
static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
{
char *trace_file;
@@ -384,6 +400,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
goto out_reset;
}
+ if (set_tracing_percpu_buffer_size(ftrace) < 0) {
+ pr_err("failed to set tracing per-cpu buffer size\n");
+ goto out_reset;
+ }
+
if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
goto out_reset;
@@ -503,6 +524,37 @@ static void delete_filter_func(struct list_head *head)
}
}
+static int parse_buffer_size(const struct option *opt,
+ const char *str, int unset)
+{
+ unsigned long *s = (unsigned long *)opt->value;
+ static struct parse_tag tags_size[] = {
+ { .tag = 'B', .mult = 1 },
+ { .tag = 'K', .mult = 1 << 10 },
+ { .tag = 'M', .mult = 1 << 20 },
+ { .tag = 'G', .mult = 1 << 30 },
+ { .tag = 0 },
+ };
+ unsigned long val;
+
+ if (unset) {
+ *s = 0;
+ return 0;
+ }
+
+ val = parse_tag_value(str, tags_size);
+ if (val != (unsigned long) -1) {
+ if (val < 1024) {
+ pr_err("buffer size too small, must larger than 1KB.");
+ return -1;
+ }
+ *s = val;
+ return 0;
+ }
+
+ return -1;
+}
+
int cmd_ftrace(int argc, const char **argv)
{
int ret;
@@ -542,6 +594,8 @@ int cmd_ftrace(int argc, const char **argv)
"Set nograph filter on given functions", parse_filter_func),
OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
"Max depth for function graph tracer"),
+ OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
+ "size of per cpu buffer", parse_buffer_size),
OPT_END()
};
--
2.25.1
This adds an option '--inherit' to allow us trace children
processes spawned by our target.
Signed-off-by: Changbin Du <[email protected]>
---
v2: option name '--trace-children' -> '--inherit'.
---
tools/perf/Documentation/perf-ftrace.txt | 3 ++
tools/perf/builtin-ftrace.c | 38 ++++++++++++++++++++++++
2 files changed, 41 insertions(+)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index 98fe01d354d1..fd632bd9b2c1 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -61,6 +61,9 @@ OPTIONS
Set the size of per-cpu tracing buffer, <size> is expected to
be a number with appended unit character - B/K/M/G.
+--inherit::
+ Trace children processes spawned by our target.
+
-T::
--trace-funcs=::
Only trace functions given by the argument. Multiple functions
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index 887e78b23a82..4efaa7b6a906 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -39,6 +39,7 @@ struct perf_ftrace {
struct list_head nograph_funcs;
int graph_depth;
unsigned long percpu_buffer_size;
+ bool inherit;
};
struct filter_entry {
@@ -177,9 +178,27 @@ static int write_tracing_file_int(const char *name, int value)
return 0;
}
+static int write_tracing_option_file(const char *name, const char *val)
+{
+ char *file;
+ int ret;
+
+ if (asprintf(&file, "options/%s", name) < 0)
+ return -1;
+
+ ret = __write_tracing_file(file, val, false);
+ free(file);
+ return ret;
+}
+
static int reset_tracing_cpu(void);
static void reset_tracing_filters(void);
+static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
+{
+ write_tracing_option_file("function-fork", "0");
+}
+
static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
{
if (write_tracing_file("tracing_on", "0") < 0)
@@ -198,6 +217,7 @@ static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
return -1;
reset_tracing_filters();
+ reset_tracing_options(ftrace);
return 0;
}
@@ -336,6 +356,17 @@ static int set_tracing_percpu_buffer_size(struct perf_ftrace *ftrace)
return 0;
}
+static int set_tracing_trace_inherit(struct perf_ftrace *ftrace)
+{
+ if (!ftrace->inherit)
+ return 0;
+
+ if (write_tracing_option_file("function-fork", "1") < 0)
+ return -1;
+
+ return 0;
+}
+
static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
{
char *trace_file;
@@ -405,6 +436,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
goto out_reset;
}
+ if (set_tracing_trace_inherit(ftrace) < 0) {
+ pr_err("failed to set tracing option function-fork\n");
+ goto out_reset;
+ }
+
if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
goto out_reset;
@@ -599,6 +635,8 @@ int cmd_ftrace(int argc, const char **argv)
"Max depth for function graph tracer"),
OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
"size of per cpu buffer", parse_buffer_size),
+ OPT_BOOLEAN(0, "inherit", &ftrace.inherit,
+ "trace children processes"),
OPT_END()
};
--
2.25.1
Sometimes we want ftrace display more and longer information about
the trace.
$ sudo perf ftrace -G
2) 0.979 us | mutex_unlock();
2) 1.540 us | __fsnotify_parent();
2) 0.433 us | fsnotify();
$ sudo perf ftrace -G --graph-opts verbose
14160.770883 | 0) <...>-47814 | .... | 1.289 us | mutex_unlock();
14160.770886 | 0) <...>-47814 | .... | 1.624 us | __fsnotify_parent();
14160.770887 | 0) <...>-47814 | .... | 0.636 us | fsnotify();
14160.770888 | 0) <...>-47814 | .... | 0.328 us | __sb_end_write();
14160.770888 | 0) <...>-47814 | d... | 0.430 us | fpregs_assert_state_consistent();
14160.770889 | 0) <...>-47814 | d... | | do_syscall_64() {
14160.770889 | 0) <...>-47814 | .... | | __x64_sys_close() {
Signed-off-by: Changbin Du <[email protected]>
---
tools/perf/Documentation/perf-ftrace.txt | 1 +
tools/perf/builtin-ftrace.c | 29 +++++++++++++++++++++++-
2 files changed, 29 insertions(+), 1 deletion(-)
diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
index e6e9564d6c2e..0dd5ef4f9c65 100644
--- a/tools/perf/Documentation/perf-ftrace.txt
+++ b/tools/perf/Documentation/perf-ftrace.txt
@@ -107,6 +107,7 @@ OPTIONS
List of options allowed to set:
nosleep-time - Measure on-CPU time only for function_graph tracer.
noirqs - Ignore functions that happen inside interrupt.
+ verbose - Show process names, PIDs, timestamps, etc.
SEE ALSO
--------
diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
index a5906258c413..d169d6329454 100644
--- a/tools/perf/builtin-ftrace.c
+++ b/tools/perf/builtin-ftrace.c
@@ -45,6 +45,7 @@ struct perf_ftrace {
int func_irq_info;
int graph_nosleep_time;
int graph_noirqs;
+ int graph_verbose;
};
struct filter_entry {
@@ -205,6 +206,9 @@ static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
write_tracing_option_file("func_stack_trace", "0");
write_tracing_option_file("sleep-time", "1");
write_tracing_option_file("funcgraph-irqs", "1");
+ write_tracing_option_file("funcgraph-proc", "0");
+ write_tracing_option_file("funcgraph-abstime", "0");
+ write_tracing_option_file("latency-format", "0");
write_tracing_option_file("irq-info", "0");
}
@@ -420,6 +424,23 @@ static int set_tracing_funcgraph_irqs(struct perf_ftrace *ftrace)
return 0;
}
+static int set_tracing_funcgraph_verbose(struct perf_ftrace *ftrace)
+{
+ if (!ftrace->graph_verbose)
+ return 0;
+
+ if (write_tracing_option_file("funcgraph-proc", "1") < 0)
+ return -1;
+
+ if (write_tracing_option_file("funcgraph-abstime", "1") < 0)
+ return -1;
+
+ if (write_tracing_option_file("latency-format", "1") < 0)
+ return -1;
+
+ return 0;
+}
+
static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
{
char *trace_file;
@@ -514,6 +535,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
goto out_reset;
}
+ if (set_tracing_funcgraph_verbose(ftrace) < 0) {
+ pr_err("failed to set tracing option funcgraph-proc/funcgraph-abstime\n");
+ goto out_reset;
+ }
+
if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
goto out_reset;
@@ -696,6 +722,7 @@ static int parse_graph_tracer_opts(const struct option *opt,
struct sublevel_option graph_tracer_opts[] = {
{ .name = "nosleep-time", .value_ptr = &ftrace->graph_nosleep_time },
{ .name = "noirqs", .value_ptr = &ftrace->graph_noirqs },
+ { .name = "verbose", .value_ptr = &ftrace->graph_verbose },
{ .name = NULL, }
};
@@ -752,7 +779,7 @@ int cmd_ftrace(int argc, const char **argv)
OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
"Max depth for function graph tracer"),
OPT_CALLBACK(0, "graph-opts", &ftrace, "options",
- "graph tracer options, available options: nosleep-time,noirqs",
+ "graph tracer options, available options: nosleep-time,noirqs,verbose",
parse_graph_tracer_opts),
OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
"size of per cpu buffer", parse_buffer_size),
--
2.25.1
Hello,
On Sat, Jul 11, 2020 at 9:42 PM Changbin Du <[email protected]> wrote:
>
> This adds an option '-F/--funcs' to list all available functions to trace,
> which is read from tracing file 'available_filter_functions'.
>
> $ sudo ./perf ftrace -F | head
> trace_initcall_finish_cb
> initcall_blacklisted
> do_one_initcall
> do_one_initcall
> trace_initcall_start_cb
> run_init_process
> try_to_run_init_process
> match_dev_by_label
> match_dev_by_uuid
> rootfs_init_fs_context
>
> Signed-off-by: Changbin Du <[email protected]>
>
> ---
> v2: option name '-l/--list-functions' -> '-F/--funcs'
> ---
> tools/perf/Documentation/perf-ftrace.txt | 4 +++
> tools/perf/builtin-ftrace.c | 43 ++++++++++++++++++++++++
> 2 files changed, 47 insertions(+)
>
> diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> index 952e46669168..d79560dea19f 100644
> --- a/tools/perf/Documentation/perf-ftrace.txt
> +++ b/tools/perf/Documentation/perf-ftrace.txt
> @@ -30,6 +30,10 @@ OPTIONS
> --verbose=::
> Verbosity level.
>
> +-F::
> +--funcs::
> + List all available functions to trace.
> +
> -p::
> --pid=::
> Trace on existing process id (comma separated list).
> diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> index 5f53da87040d..244cc8e6bd60 100644
> --- a/tools/perf/builtin-ftrace.c
> +++ b/tools/perf/builtin-ftrace.c
> @@ -32,6 +32,7 @@ struct perf_ftrace {
> struct evlist *evlist;
> struct target target;
> const char *tracer;
> + bool list_avail_functions;
> struct list_head filters;
> struct list_head notrace;
> struct list_head graph_funcs;
> @@ -127,6 +128,43 @@ static int append_tracing_file(const char *name, const char *val)
> return __write_tracing_file(name, val, true);
> }
>
> +static int read_tracing_file_to_stdout(const char *name)
> +{
> + char buf[4096];
> + char *file;
> + int fd;
> + int ret = -1;
> +
> + file = get_tracing_file(name);
> + if (!file) {
> + pr_debug("cannot get tracing file: %s\n", name);
> + return -1;
> + }
> +
> + fd = open(file, O_RDONLY);
> + if (fd < 0) {
> + pr_debug("cannot open tracing file: %s: %s\n",
> + name, str_error_r(errno, buf, sizeof(buf)));
> + goto out;
> + }
> +
> + /* read contents to stdout */
> + while (true) {
> + int n = read(fd, buf, sizeof(buf));
> + if (n <= 0)
> + goto out_close;
> + if (fwrite(buf, n, 1, stdout) != 1)
> + goto out_close;
> + }
> + ret = 0;
As I said before, there's no break in the loop above
so the ret can never be 0.
Thanks
Namhyung
> +
> +out_close:
> + close(fd);
> +out:
> + put_tracing_file(file);
> + return ret;
> +}
> +
> static int reset_tracing_cpu(void);
> static void reset_tracing_filters(void);
>
> @@ -301,6 +339,9 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> signal(SIGCHLD, sig_handler);
> signal(SIGPIPE, sig_handler);
>
> + if (ftrace->list_avail_functions)
> + return read_tracing_file_to_stdout("available_filter_functions");
> +
> if (reset_tracing_files(ftrace) < 0) {
> pr_err("failed to reset ftrace\n");
> goto out;
> @@ -470,6 +511,8 @@ int cmd_ftrace(int argc, const char **argv)
> const struct option ftrace_options[] = {
> OPT_STRING('t', "tracer", &ftrace.tracer, "tracer",
> "tracer to use: function or function_graph (This option is deprecated)"),
> + OPT_BOOLEAN('F', "funcs", &ftrace.list_avail_functions,
> + "Show available functions to filter"),
> OPT_STRING('p', "pid", &ftrace.target.pid, "pid",
> "trace on existing process id"),
> OPT_INCR('v', "verbose", &verbose,
> --
> 2.25.1
>
On Sat, Jul 11, 2020 at 9:42 PM Changbin Du <[email protected]> wrote:
>
> This adds an option '--inherit' to allow us trace children
> processes spawned by our target.
>
> Signed-off-by: Changbin Du <[email protected]>
>
> ---
> v2: option name '--trace-children' -> '--inherit'.
> ---
> tools/perf/Documentation/perf-ftrace.txt | 3 ++
> tools/perf/builtin-ftrace.c | 38 ++++++++++++++++++++++++
> 2 files changed, 41 insertions(+)
>
> diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> index 98fe01d354d1..fd632bd9b2c1 100644
> --- a/tools/perf/Documentation/perf-ftrace.txt
> +++ b/tools/perf/Documentation/perf-ftrace.txt
> @@ -61,6 +61,9 @@ OPTIONS
> Set the size of per-cpu tracing buffer, <size> is expected to
> be a number with appended unit character - B/K/M/G.
>
> +--inherit::
> + Trace children processes spawned by our target.
> +
> -T::
> --trace-funcs=::
> Only trace functions given by the argument. Multiple functions
> diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> index 887e78b23a82..4efaa7b6a906 100644
> --- a/tools/perf/builtin-ftrace.c
> +++ b/tools/perf/builtin-ftrace.c
> @@ -39,6 +39,7 @@ struct perf_ftrace {
> struct list_head nograph_funcs;
> int graph_depth;
> unsigned long percpu_buffer_size;
> + bool inherit;
> };
>
> struct filter_entry {
> @@ -177,9 +178,27 @@ static int write_tracing_file_int(const char *name, int value)
> return 0;
> }
>
> +static int write_tracing_option_file(const char *name, const char *val)
> +{
> + char *file;
> + int ret;
> +
> + if (asprintf(&file, "options/%s", name) < 0)
> + return -1;
> +
> + ret = __write_tracing_file(file, val, false);
> + free(file);
> + return ret;
> +}
> +
> static int reset_tracing_cpu(void);
> static void reset_tracing_filters(void);
>
> +static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
> +{
> + write_tracing_option_file("function-fork", "0");
> +}
> +
> static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
> {
> if (write_tracing_file("tracing_on", "0") < 0)
> @@ -198,6 +217,7 @@ static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
> return -1;
>
> reset_tracing_filters();
> + reset_tracing_options(ftrace);
> return 0;
> }
>
> @@ -336,6 +356,17 @@ static int set_tracing_percpu_buffer_size(struct perf_ftrace *ftrace)
> return 0;
> }
>
> +static int set_tracing_trace_inherit(struct perf_ftrace *ftrace)
> +{
> + if (!ftrace->inherit)
> + return 0;
> +
> + if (write_tracing_option_file("function-fork", "1") < 0)
> + return -1;
> +
> + return 0;
> +}
> +
> static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> {
> char *trace_file;
> @@ -405,6 +436,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> goto out_reset;
> }
>
> + if (set_tracing_trace_inherit(ftrace) < 0) {
> + pr_err("failed to set tracing option function-fork\n");
> + goto out_reset;
> + }
Can we have set_tracing_options() here instead to make
the __cmd_ftrace() shorter? It'd set other options added later.
It's also symmetric to reset_tracing_options().
Thanks
Namhyung
> +
> if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
> pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
> goto out_reset;
> @@ -599,6 +635,8 @@ int cmd_ftrace(int argc, const char **argv)
> "Max depth for function graph tracer"),
> OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
> "size of per cpu buffer", parse_buffer_size),
> + OPT_BOOLEAN(0, "inherit", &ftrace.inherit,
> + "trace children processes"),
> OPT_END()
> };
>
> --
> 2.25.1
>
On Sat, Jul 11, 2020 at 9:43 PM Changbin Du <[email protected]> wrote:
>
> Sometimes we want ftrace display more and longer information about
> the trace.
>
> $ sudo perf ftrace -G
> 2) 0.979 us | mutex_unlock();
> 2) 1.540 us | __fsnotify_parent();
> 2) 0.433 us | fsnotify();
>
> $ sudo perf ftrace -G --graph-opts verbose
These -G option usage should be changed..
Thanks
Namhyung
> 14160.770883 | 0) <...>-47814 | .... | 1.289 us | mutex_unlock();
> 14160.770886 | 0) <...>-47814 | .... | 1.624 us | __fsnotify_parent();
> 14160.770887 | 0) <...>-47814 | .... | 0.636 us | fsnotify();
> 14160.770888 | 0) <...>-47814 | .... | 0.328 us | __sb_end_write();
> 14160.770888 | 0) <...>-47814 | d... | 0.430 us | fpregs_assert_state_consistent();
> 14160.770889 | 0) <...>-47814 | d... | | do_syscall_64() {
> 14160.770889 | 0) <...>-47814 | .... | | __x64_sys_close() {
>
> Signed-off-by: Changbin Du <[email protected]>
> ---
> tools/perf/Documentation/perf-ftrace.txt | 1 +
> tools/perf/builtin-ftrace.c | 29 +++++++++++++++++++++++-
> 2 files changed, 29 insertions(+), 1 deletion(-)
>
> diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> index e6e9564d6c2e..0dd5ef4f9c65 100644
> --- a/tools/perf/Documentation/perf-ftrace.txt
> +++ b/tools/perf/Documentation/perf-ftrace.txt
> @@ -107,6 +107,7 @@ OPTIONS
> List of options allowed to set:
> nosleep-time - Measure on-CPU time only for function_graph tracer.
> noirqs - Ignore functions that happen inside interrupt.
> + verbose - Show process names, PIDs, timestamps, etc.
>
> SEE ALSO
> --------
> diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> index a5906258c413..d169d6329454 100644
> --- a/tools/perf/builtin-ftrace.c
> +++ b/tools/perf/builtin-ftrace.c
> @@ -45,6 +45,7 @@ struct perf_ftrace {
> int func_irq_info;
> int graph_nosleep_time;
> int graph_noirqs;
> + int graph_verbose;
> };
>
> struct filter_entry {
> @@ -205,6 +206,9 @@ static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
> write_tracing_option_file("func_stack_trace", "0");
> write_tracing_option_file("sleep-time", "1");
> write_tracing_option_file("funcgraph-irqs", "1");
> + write_tracing_option_file("funcgraph-proc", "0");
> + write_tracing_option_file("funcgraph-abstime", "0");
> + write_tracing_option_file("latency-format", "0");
> write_tracing_option_file("irq-info", "0");
> }
>
> @@ -420,6 +424,23 @@ static int set_tracing_funcgraph_irqs(struct perf_ftrace *ftrace)
> return 0;
> }
>
> +static int set_tracing_funcgraph_verbose(struct perf_ftrace *ftrace)
> +{
> + if (!ftrace->graph_verbose)
> + return 0;
> +
> + if (write_tracing_option_file("funcgraph-proc", "1") < 0)
> + return -1;
> +
> + if (write_tracing_option_file("funcgraph-abstime", "1") < 0)
> + return -1;
> +
> + if (write_tracing_option_file("latency-format", "1") < 0)
> + return -1;
> +
> + return 0;
> +}
> +
> static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> {
> char *trace_file;
> @@ -514,6 +535,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> goto out_reset;
> }
>
> + if (set_tracing_funcgraph_verbose(ftrace) < 0) {
> + pr_err("failed to set tracing option funcgraph-proc/funcgraph-abstime\n");
> + goto out_reset;
> + }
> +
> if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
> pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
> goto out_reset;
> @@ -696,6 +722,7 @@ static int parse_graph_tracer_opts(const struct option *opt,
> struct sublevel_option graph_tracer_opts[] = {
> { .name = "nosleep-time", .value_ptr = &ftrace->graph_nosleep_time },
> { .name = "noirqs", .value_ptr = &ftrace->graph_noirqs },
> + { .name = "verbose", .value_ptr = &ftrace->graph_verbose },
> { .name = NULL, }
> };
>
> @@ -752,7 +779,7 @@ int cmd_ftrace(int argc, const char **argv)
> OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
> "Max depth for function graph tracer"),
> OPT_CALLBACK(0, "graph-opts", &ftrace, "options",
> - "graph tracer options, available options: nosleep-time,noirqs",
> + "graph tracer options, available options: nosleep-time,noirqs,verbose",
> parse_graph_tracer_opts),
> OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
> "size of per cpu buffer", parse_buffer_size),
> --
> 2.25.1
>
Em Mon, Jul 13, 2020 at 10:49:40AM +0900, Namhyung Kim escreveu:
> Hello,
>
> On Sat, Jul 11, 2020 at 9:42 PM Changbin Du <[email protected]> wrote:
> >
> > This adds an option '-F/--funcs' to list all available functions to trace,
> > which is read from tracing file 'available_filter_functions'.
> >
> > $ sudo ./perf ftrace -F | head
> > trace_initcall_finish_cb
> > initcall_blacklisted
> > do_one_initcall
> > do_one_initcall
> > trace_initcall_start_cb
> > run_init_process
> > try_to_run_init_process
> > match_dev_by_label
> > match_dev_by_uuid
> > rootfs_init_fs_context
> >
> > Signed-off-by: Changbin Du <[email protected]>
> >
> > ---
> > v2: option name '-l/--list-functions' -> '-F/--funcs'
> > ---
> > tools/perf/Documentation/perf-ftrace.txt | 4 +++
> > tools/perf/builtin-ftrace.c | 43 ++++++++++++++++++++++++
> > 2 files changed, 47 insertions(+)
> >
> > diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> > index 952e46669168..d79560dea19f 100644
> > --- a/tools/perf/Documentation/perf-ftrace.txt
> > +++ b/tools/perf/Documentation/perf-ftrace.txt
> > @@ -30,6 +30,10 @@ OPTIONS
> > --verbose=::
> > Verbosity level.
> >
> > +-F::
> > +--funcs::
> > + List all available functions to trace.
> > +
> > -p::
> > --pid=::
> > Trace on existing process id (comma separated list).
> > diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> > index 5f53da87040d..244cc8e6bd60 100644
> > --- a/tools/perf/builtin-ftrace.c
> > +++ b/tools/perf/builtin-ftrace.c
> > @@ -32,6 +32,7 @@ struct perf_ftrace {
> > struct evlist *evlist;
> > struct target target;
> > const char *tracer;
> > + bool list_avail_functions;
> > struct list_head filters;
> > struct list_head notrace;
> > struct list_head graph_funcs;
> > @@ -127,6 +128,43 @@ static int append_tracing_file(const char *name, const char *val)
> > return __write_tracing_file(name, val, true);
> > }
> >
> > +static int read_tracing_file_to_stdout(const char *name)
> > +{
> > + char buf[4096];
> > + char *file;
> > + int fd;
> > + int ret = -1;
> > +
> > + file = get_tracing_file(name);
> > + if (!file) {
> > + pr_debug("cannot get tracing file: %s\n", name);
> > + return -1;
> > + }
> > +
> > + fd = open(file, O_RDONLY);
> > + if (fd < 0) {
> > + pr_debug("cannot open tracing file: %s: %s\n",
> > + name, str_error_r(errno, buf, sizeof(buf)));
> > + goto out;
> > + }
> > +
> > + /* read contents to stdout */
> > + while (true) {
> > + int n = read(fd, buf, sizeof(buf));
> > + if (n <= 0)
> > + goto out_close;
> > + if (fwrite(buf, n, 1, stdout) != 1)
> > + goto out_close;
> > + }
> > + ret = 0;
>
> As I said before, there's no break in the loop above
> so the ret can never be 0.
Yeah, this is clearly wrong, that ret = 0 will never be reached.
- Arnaldo
> Thanks
> Namhyung
>
> > +
> > +out_close:
> > + close(fd);
> > +out:
> > + put_tracing_file(file);
> > + return ret;
> > +}
> > +
> > static int reset_tracing_cpu(void);
> > static void reset_tracing_filters(void);
> >
> > @@ -301,6 +339,9 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> > signal(SIGCHLD, sig_handler);
> > signal(SIGPIPE, sig_handler);
> >
> > + if (ftrace->list_avail_functions)
> > + return read_tracing_file_to_stdout("available_filter_functions");
> > +
> > if (reset_tracing_files(ftrace) < 0) {
> > pr_err("failed to reset ftrace\n");
> > goto out;
> > @@ -470,6 +511,8 @@ int cmd_ftrace(int argc, const char **argv)
> > const struct option ftrace_options[] = {
> > OPT_STRING('t', "tracer", &ftrace.tracer, "tracer",
> > "tracer to use: function or function_graph (This option is deprecated)"),
> > + OPT_BOOLEAN('F', "funcs", &ftrace.list_avail_functions,
> > + "Show available functions to filter"),
> > OPT_STRING('p', "pid", &ftrace.target.pid, "pid",
> > "trace on existing process id"),
> > OPT_INCR('v', "verbose", &verbose,
> > --
> > 2.25.1
> >
--
- Arnaldo
Em Sat, Jul 11, 2020 at 08:40:21PM +0800, Changbin Du escreveu:
> This allows us to trace single thread instead of the whole process.
>
> Signed-off-by: Changbin Du <[email protected]>
> ---
> tools/perf/Documentation/perf-ftrace.txt | 4 ++++
> tools/perf/builtin-ftrace.c | 2 ++
> 2 files changed, 6 insertions(+)
>
> diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> index d79560dea19f..e204bf6d50d8 100644
> --- a/tools/perf/Documentation/perf-ftrace.txt
> +++ b/tools/perf/Documentation/perf-ftrace.txt
> @@ -38,6 +38,10 @@ OPTIONS
> --pid=::
> Trace on existing process id (comma separated list).
>
> +-t::
> +--tid=::
> + Trace on existing thread id (comma separated list).
> +
Humm, I just tried:
[root@five ~]# yes > /dev/null &
[1] 18265
[root@five ~]# perf ftrace --tid 18265
^C[root@five ~]#
After waiting for a while, nothing, what am I doing wrong?
- Arnaldo
> -a::
> --all-cpus::
> Force system-wide collection. Scripts run without a <command>
> diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> index 244cc8e6bd60..1188b82c6541 100644
> --- a/tools/perf/builtin-ftrace.c
> +++ b/tools/perf/builtin-ftrace.c
> @@ -515,6 +515,8 @@ int cmd_ftrace(int argc, const char **argv)
> "Show available functions to filter"),
> OPT_STRING('p', "pid", &ftrace.target.pid, "pid",
> "trace on existing process id"),
> + OPT_STRING('t', "tid", &ftrace.target.tid, "tid",
> + "trace on existing thread id (exclusive to --pid)"),
> OPT_INCR('v', "verbose", &verbose,
> "be more verbose"),
> OPT_BOOLEAN('a', "all-cpus", &ftrace.target.system_wide,
> --
> 2.25.1
>
--
- Arnaldo
On Thu, Jul 16, 2020 at 12:36:30PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Sat, Jul 11, 2020 at 08:40:21PM +0800, Changbin Du escreveu:
> > This allows us to trace single thread instead of the whole process.
> >
> > Signed-off-by: Changbin Du <[email protected]>
> > ---
> > tools/perf/Documentation/perf-ftrace.txt | 4 ++++
> > tools/perf/builtin-ftrace.c | 2 ++
> > 2 files changed, 6 insertions(+)
> >
> > diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> > index d79560dea19f..e204bf6d50d8 100644
> > --- a/tools/perf/Documentation/perf-ftrace.txt
> > +++ b/tools/perf/Documentation/perf-ftrace.txt
> > @@ -38,6 +38,10 @@ OPTIONS
> > --pid=::
> > Trace on existing process id (comma separated list).
> >
> > +-t::
> > +--tid=::
> > + Trace on existing thread id (comma separated list).
> > +
>
>
> Humm, I just tried:
>
> [root@five ~]# yes > /dev/null &
> [1] 18265
> [root@five ~]# perf ftrace --tid 18265
> ^C[root@five ~]#
>
> After waiting for a while, nothing, what am I doing wrong?
>
I got it wrong. Currently ftrace only can filter by pid. If the pid is not
the main thread it won't work.
So this patch makes no sense. will drop this.
> - Arnaldo
>
>
> > -a::
> > --all-cpus::
> > Force system-wide collection. Scripts run without a <command>
> > diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> > index 244cc8e6bd60..1188b82c6541 100644
> > --- a/tools/perf/builtin-ftrace.c
> > +++ b/tools/perf/builtin-ftrace.c
> > @@ -515,6 +515,8 @@ int cmd_ftrace(int argc, const char **argv)
> > "Show available functions to filter"),
> > OPT_STRING('p', "pid", &ftrace.target.pid, "pid",
> > "trace on existing process id"),
> > + OPT_STRING('t', "tid", &ftrace.target.tid, "tid",
> > + "trace on existing thread id (exclusive to --pid)"),
> > OPT_INCR('v', "verbose", &verbose,
> > "be more verbose"),
> > OPT_BOOLEAN('a', "all-cpus", &ftrace.target.system_wide,
> > --
> > 2.25.1
> >
>
> --
>
> - Arnaldo
--
Cheers,
Changbin Du
On Mon, Jul 13, 2020 at 10:49:40AM +0900, Namhyung Kim wrote:
> Hello,
>
> On Sat, Jul 11, 2020 at 9:42 PM Changbin Du <[email protected]> wrote:
> >
> > This adds an option '-F/--funcs' to list all available functions to trace,
> > which is read from tracing file 'available_filter_functions'.
> >
> > $ sudo ./perf ftrace -F | head
> > trace_initcall_finish_cb
> > initcall_blacklisted
> > do_one_initcall
> > do_one_initcall
> > trace_initcall_start_cb
> > run_init_process
> > try_to_run_init_process
> > match_dev_by_label
> > match_dev_by_uuid
> > rootfs_init_fs_context
> >
> > Signed-off-by: Changbin Du <[email protected]>
> >
> > ---
> > v2: option name '-l/--list-functions' -> '-F/--funcs'
> > ---
> > tools/perf/Documentation/perf-ftrace.txt | 4 +++
> > tools/perf/builtin-ftrace.c | 43 ++++++++++++++++++++++++
> > 2 files changed, 47 insertions(+)
> >
> > diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> > index 952e46669168..d79560dea19f 100644
> > --- a/tools/perf/Documentation/perf-ftrace.txt
> > +++ b/tools/perf/Documentation/perf-ftrace.txt
> > @@ -30,6 +30,10 @@ OPTIONS
> > --verbose=::
> > Verbosity level.
> >
> > +-F::
> > +--funcs::
> > + List all available functions to trace.
> > +
> > -p::
> > --pid=::
> > Trace on existing process id (comma separated list).
> > diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> > index 5f53da87040d..244cc8e6bd60 100644
> > --- a/tools/perf/builtin-ftrace.c
> > +++ b/tools/perf/builtin-ftrace.c
> > @@ -32,6 +32,7 @@ struct perf_ftrace {
> > struct evlist *evlist;
> > struct target target;
> > const char *tracer;
> > + bool list_avail_functions;
> > struct list_head filters;
> > struct list_head notrace;
> > struct list_head graph_funcs;
> > @@ -127,6 +128,43 @@ static int append_tracing_file(const char *name, const char *val)
> > return __write_tracing_file(name, val, true);
> > }
> >
> > +static int read_tracing_file_to_stdout(const char *name)
> > +{
> > + char buf[4096];
> > + char *file;
> > + int fd;
> > + int ret = -1;
> > +
> > + file = get_tracing_file(name);
> > + if (!file) {
> > + pr_debug("cannot get tracing file: %s\n", name);
> > + return -1;
> > + }
> > +
> > + fd = open(file, O_RDONLY);
> > + if (fd < 0) {
> > + pr_debug("cannot open tracing file: %s: %s\n",
> > + name, str_error_r(errno, buf, sizeof(buf)));
> > + goto out;
> > + }
> > +
> > + /* read contents to stdout */
> > + while (true) {
> > + int n = read(fd, buf, sizeof(buf));
> > + if (n <= 0)
> > + goto out_close;
> > + if (fwrite(buf, n, 1, stdout) != 1)
> > + goto out_close;
> > + }
> > + ret = 0;
>
> As I said before, there's no break in the loop above
> so the ret can never be 0.
>
I see now. Just fixed it as:
while (true) {
int n = read(fd, buf, sizeof(buf));
- if (n <= 0)
+ if (n == 0)
+ break;
+ else if (n < 0)
goto out_close;
+
Thanks!
> Thanks
> Namhyung
>
> > +
> > +out_close:
> > + close(fd);
> > +out:
> > + put_tracing_file(file);
> > + return ret;
> > +}
> > +
> > static int reset_tracing_cpu(void);
> > static void reset_tracing_filters(void);
> >
> > @@ -301,6 +339,9 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> > signal(SIGCHLD, sig_handler);
> > signal(SIGPIPE, sig_handler);
> >
> > + if (ftrace->list_avail_functions)
> > + return read_tracing_file_to_stdout("available_filter_functions");
> > +
> > if (reset_tracing_files(ftrace) < 0) {
> > pr_err("failed to reset ftrace\n");
> > goto out;
> > @@ -470,6 +511,8 @@ int cmd_ftrace(int argc, const char **argv)
> > const struct option ftrace_options[] = {
> > OPT_STRING('t', "tracer", &ftrace.tracer, "tracer",
> > "tracer to use: function or function_graph (This option is deprecated)"),
> > + OPT_BOOLEAN('F', "funcs", &ftrace.list_avail_functions,
> > + "Show available functions to filter"),
> > OPT_STRING('p', "pid", &ftrace.target.pid, "pid",
> > "trace on existing process id"),
> > OPT_INCR('v', "verbose", &verbose,
> > --
> > 2.25.1
> >
--
Cheers,
Changbin Du
On Mon, Jul 13, 2020 at 11:07:56AM +0900, Namhyung Kim wrote:
> On Sat, Jul 11, 2020 at 9:43 PM Changbin Du <[email protected]> wrote:
> >
> > Sometimes we want ftrace display more and longer information about
> > the trace.
> >
> > $ sudo perf ftrace -G
> > 2) 0.979 us | mutex_unlock();
> > 2) 1.540 us | __fsnotify_parent();
> > 2) 0.433 us | fsnotify();
> >
> > $ sudo perf ftrace -G --graph-opts verbose
>
> These -G option usage should be changed..
>
The commit mesge is update now. Thanks.
> Thanks
> Namhyung
>
>
> > 14160.770883 | 0) <...>-47814 | .... | 1.289 us | mutex_unlock();
> > 14160.770886 | 0) <...>-47814 | .... | 1.624 us | __fsnotify_parent();
> > 14160.770887 | 0) <...>-47814 | .... | 0.636 us | fsnotify();
> > 14160.770888 | 0) <...>-47814 | .... | 0.328 us | __sb_end_write();
> > 14160.770888 | 0) <...>-47814 | d... | 0.430 us | fpregs_assert_state_consistent();
> > 14160.770889 | 0) <...>-47814 | d... | | do_syscall_64() {
> > 14160.770889 | 0) <...>-47814 | .... | | __x64_sys_close() {
> >
> > Signed-off-by: Changbin Du <[email protected]>
> > ---
> > tools/perf/Documentation/perf-ftrace.txt | 1 +
> > tools/perf/builtin-ftrace.c | 29 +++++++++++++++++++++++-
> > 2 files changed, 29 insertions(+), 1 deletion(-)
> >
> > diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> > index e6e9564d6c2e..0dd5ef4f9c65 100644
> > --- a/tools/perf/Documentation/perf-ftrace.txt
> > +++ b/tools/perf/Documentation/perf-ftrace.txt
> > @@ -107,6 +107,7 @@ OPTIONS
> > List of options allowed to set:
> > nosleep-time - Measure on-CPU time only for function_graph tracer.
> > noirqs - Ignore functions that happen inside interrupt.
> > + verbose - Show process names, PIDs, timestamps, etc.
> >
> > SEE ALSO
> > --------
> > diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> > index a5906258c413..d169d6329454 100644
> > --- a/tools/perf/builtin-ftrace.c
> > +++ b/tools/perf/builtin-ftrace.c
> > @@ -45,6 +45,7 @@ struct perf_ftrace {
> > int func_irq_info;
> > int graph_nosleep_time;
> > int graph_noirqs;
> > + int graph_verbose;
> > };
> >
> > struct filter_entry {
> > @@ -205,6 +206,9 @@ static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
> > write_tracing_option_file("func_stack_trace", "0");
> > write_tracing_option_file("sleep-time", "1");
> > write_tracing_option_file("funcgraph-irqs", "1");
> > + write_tracing_option_file("funcgraph-proc", "0");
> > + write_tracing_option_file("funcgraph-abstime", "0");
> > + write_tracing_option_file("latency-format", "0");
> > write_tracing_option_file("irq-info", "0");
> > }
> >
> > @@ -420,6 +424,23 @@ static int set_tracing_funcgraph_irqs(struct perf_ftrace *ftrace)
> > return 0;
> > }
> >
> > +static int set_tracing_funcgraph_verbose(struct perf_ftrace *ftrace)
> > +{
> > + if (!ftrace->graph_verbose)
> > + return 0;
> > +
> > + if (write_tracing_option_file("funcgraph-proc", "1") < 0)
> > + return -1;
> > +
> > + if (write_tracing_option_file("funcgraph-abstime", "1") < 0)
> > + return -1;
> > +
> > + if (write_tracing_option_file("latency-format", "1") < 0)
> > + return -1;
> > +
> > + return 0;
> > +}
> > +
> > static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> > {
> > char *trace_file;
> > @@ -514,6 +535,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> > goto out_reset;
> > }
> >
> > + if (set_tracing_funcgraph_verbose(ftrace) < 0) {
> > + pr_err("failed to set tracing option funcgraph-proc/funcgraph-abstime\n");
> > + goto out_reset;
> > + }
> > +
> > if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
> > pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
> > goto out_reset;
> > @@ -696,6 +722,7 @@ static int parse_graph_tracer_opts(const struct option *opt,
> > struct sublevel_option graph_tracer_opts[] = {
> > { .name = "nosleep-time", .value_ptr = &ftrace->graph_nosleep_time },
> > { .name = "noirqs", .value_ptr = &ftrace->graph_noirqs },
> > + { .name = "verbose", .value_ptr = &ftrace->graph_verbose },
> > { .name = NULL, }
> > };
> >
> > @@ -752,7 +779,7 @@ int cmd_ftrace(int argc, const char **argv)
> > OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
> > "Max depth for function graph tracer"),
> > OPT_CALLBACK(0, "graph-opts", &ftrace, "options",
> > - "graph tracer options, available options: nosleep-time,noirqs",
> > + "graph tracer options, available options: nosleep-time,noirqs,verbose",
> > parse_graph_tracer_opts),
> > OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
> > "size of per cpu buffer", parse_buffer_size),
> > --
> > 2.25.1
> >
--
Cheers,
Changbin Du
On Mon, Jul 13, 2020 at 10:59:56AM +0900, Namhyung Kim wrote:
> On Sat, Jul 11, 2020 at 9:42 PM Changbin Du <[email protected]> wrote:
> >
> > This adds an option '--inherit' to allow us trace children
> > processes spawned by our target.
> >
> > Signed-off-by: Changbin Du <[email protected]>
> >
> > ---
> > v2: option name '--trace-children' -> '--inherit'.
> > ---
> > tools/perf/Documentation/perf-ftrace.txt | 3 ++
> > tools/perf/builtin-ftrace.c | 38 ++++++++++++++++++++++++
> > 2 files changed, 41 insertions(+)
> >
> > diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> > index 98fe01d354d1..fd632bd9b2c1 100644
> > --- a/tools/perf/Documentation/perf-ftrace.txt
> > +++ b/tools/perf/Documentation/perf-ftrace.txt
> > @@ -61,6 +61,9 @@ OPTIONS
> > Set the size of per-cpu tracing buffer, <size> is expected to
> > be a number with appended unit character - B/K/M/G.
> >
> > +--inherit::
> > + Trace children processes spawned by our target.
> > +
> > -T::
> > --trace-funcs=::
> > Only trace functions given by the argument. Multiple functions
> > diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> > index 887e78b23a82..4efaa7b6a906 100644
> > --- a/tools/perf/builtin-ftrace.c
> > +++ b/tools/perf/builtin-ftrace.c
> > @@ -39,6 +39,7 @@ struct perf_ftrace {
> > struct list_head nograph_funcs;
> > int graph_depth;
> > unsigned long percpu_buffer_size;
> > + bool inherit;
> > };
> >
> > struct filter_entry {
> > @@ -177,9 +178,27 @@ static int write_tracing_file_int(const char *name, int value)
> > return 0;
> > }
> >
> > +static int write_tracing_option_file(const char *name, const char *val)
> > +{
> > + char *file;
> > + int ret;
> > +
> > + if (asprintf(&file, "options/%s", name) < 0)
> > + return -1;
> > +
> > + ret = __write_tracing_file(file, val, false);
> > + free(file);
> > + return ret;
> > +}
> > +
> > static int reset_tracing_cpu(void);
> > static void reset_tracing_filters(void);
> >
> > +static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
> > +{
> > + write_tracing_option_file("function-fork", "0");
> > +}
> > +
> > static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
> > {
> > if (write_tracing_file("tracing_on", "0") < 0)
> > @@ -198,6 +217,7 @@ static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
> > return -1;
> >
> > reset_tracing_filters();
> > + reset_tracing_options(ftrace);
> > return 0;
> > }
> >
> > @@ -336,6 +356,17 @@ static int set_tracing_percpu_buffer_size(struct perf_ftrace *ftrace)
> > return 0;
> > }
> >
> > +static int set_tracing_trace_inherit(struct perf_ftrace *ftrace)
> > +{
> > + if (!ftrace->inherit)
> > + return 0;
> > +
> > + if (write_tracing_option_file("function-fork", "1") < 0)
> > + return -1;
> > +
> > + return 0;
> > +}
> > +
> > static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> > {
> > char *trace_file;
> > @@ -405,6 +436,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> > goto out_reset;
> > }
> >
> > + if (set_tracing_trace_inherit(ftrace) < 0) {
> > + pr_err("failed to set tracing option function-fork\n");
> > + goto out_reset;
> > + }
>
> Can we have set_tracing_options() here instead to make
> the __cmd_ftrace() shorter? It'd set other options added later.
> It's also symmetric to reset_tracing_options().
>
I appended a patch to do this. Thanks for your suggestion.
> Thanks
> Namhyung
>
> > +
> > if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
> > pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
> > goto out_reset;
> > @@ -599,6 +635,8 @@ int cmd_ftrace(int argc, const char **argv)
> > "Max depth for function graph tracer"),
> > OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
> > "size of per cpu buffer", parse_buffer_size),
> > + OPT_BOOLEAN(0, "inherit", &ftrace.inherit,
> > + "trace children processes"),
> > OPT_END()
> > };
> >
> > --
> > 2.25.1
> >
--
Cheers,
Changbin Du
On Sat, 11 Jul 2020 20:40:20 +0800
Changbin Du <[email protected]> wrote:
> This adds an option '-F/--funcs' to list all available functions to trace,
> which is read from tracing file 'available_filter_functions'.
>
> $ sudo ./perf ftrace -F | head
> trace_initcall_finish_cb
> initcall_blacklisted
> do_one_initcall
> do_one_initcall
> trace_initcall_start_cb
> run_init_process
> try_to_run_init_process
> match_dev_by_label
> match_dev_by_uuid
> rootfs_init_fs_context
>
> Signed-off-by: Changbin Du <[email protected]>
>
> ---
> v2: option name '-l/--list-functions' -> '-F/--funcs'
> ---
> tools/perf/Documentation/perf-ftrace.txt | 4 +++
> tools/perf/builtin-ftrace.c | 43 ++++++++++++++++++++++++
> 2 files changed, 47 insertions(+)
>
> diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> index 952e46669168..d79560dea19f 100644
> --- a/tools/perf/Documentation/perf-ftrace.txt
> +++ b/tools/perf/Documentation/perf-ftrace.txt
> @@ -30,6 +30,10 @@ OPTIONS
> --verbose=::
> Verbosity level.
>
> +-F::
> +--funcs::
> + List all available functions to trace.
> +
> -p::
> --pid=::
> Trace on existing process id (comma separated list).
> diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> index 5f53da87040d..244cc8e6bd60 100644
> --- a/tools/perf/builtin-ftrace.c
> +++ b/tools/perf/builtin-ftrace.c
> @@ -32,6 +32,7 @@ struct perf_ftrace {
> struct evlist *evlist;
> struct target target;
> const char *tracer;
> + bool list_avail_functions;
> struct list_head filters;
> struct list_head notrace;
> struct list_head graph_funcs;
> @@ -127,6 +128,43 @@ static int append_tracing_file(const char *name, const char *val)
> return __write_tracing_file(name, val, true);
> }
>
> +static int read_tracing_file_to_stdout(const char *name)
> +{
All this is looking like its duplicating code that we are working on
for libtracefs.
Would you like to start contributing to that, and when we get the
libtracefs.so packed in distributions, we can easily create the
perf ftrace without having to rewrite the wheel 10 times?
-- Steve
> + char buf[4096];
> + char *file;
> + int fd;
> + int ret = -1;
> +
> + file = get_tracing_file(name);
> + if (!file) {
> + pr_debug("cannot get tracing file: %s\n", name);
> + return -1;
> + }
> +
> + fd = open(file, O_RDONLY);
> + if (fd < 0) {
> + pr_debug("cannot open tracing file: %s: %s\n",
> + name, str_error_r(errno, buf, sizeof(buf)));
> + goto out;
> + }
> +
> + /* read contents to stdout */
> + while (true) {
> + int n = read(fd, buf, sizeof(buf));
> + if (n <= 0)
> + goto out_close;
> + if (fwrite(buf, n, 1, stdout) != 1)
> + goto out_close;
> + }
> + ret = 0;
> +
> +out_close:
> + close(fd);
> +out:
> + put_tracing_file(file);
> + return ret;
> +}
> +
> static int reset_tracing_cpu(void);
> static void reset_tracing_filters(void);
>
> @@ -301,6 +339,9 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> signal(SIGCHLD, sig_handler);
> signal(SIGPIPE, sig_handler);
>
> + if (ftrace->list_avail_functions)
> + return read_tracing_file_to_stdout("available_filter_functions");
> +
> if (reset_tracing_files(ftrace) < 0) {
> pr_err("failed to reset ftrace\n");
> goto out;
> @@ -470,6 +511,8 @@ int cmd_ftrace(int argc, const char **argv)
> const struct option ftrace_options[] = {
> OPT_STRING('t', "tracer", &ftrace.tracer, "tracer",
> "tracer to use: function or function_graph (This option is deprecated)"),
> + OPT_BOOLEAN('F', "funcs", &ftrace.list_avail_functions,
> + "Show available functions to filter"),
> OPT_STRING('p', "pid", &ftrace.target.pid, "pid",
> "trace on existing process id"),
> OPT_INCR('v', "verbose", &verbose,
Em Fri, Jul 17, 2020 at 11:05:04AM -0400, Steven Rostedt escreveu:
> On Sat, 11 Jul 2020 20:40:20 +0800
> Changbin Du <[email protected]> wrote:
>
> > This adds an option '-F/--funcs' to list all available functions to trace,
> > which is read from tracing file 'available_filter_functions'.
> >
> > $ sudo ./perf ftrace -F | head
> > trace_initcall_finish_cb
> > initcall_blacklisted
> > do_one_initcall
> > do_one_initcall
> > trace_initcall_start_cb
> > run_init_process
> > try_to_run_init_process
> > match_dev_by_label
> > match_dev_by_uuid
> > rootfs_init_fs_context
> >
> > Signed-off-by: Changbin Du <[email protected]>
> >
> > ---
> > v2: option name '-l/--list-functions' -> '-F/--funcs'
> > ---
> > tools/perf/Documentation/perf-ftrace.txt | 4 +++
> > tools/perf/builtin-ftrace.c | 43 ++++++++++++++++++++++++
> > 2 files changed, 47 insertions(+)
> >
> > diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> > index 952e46669168..d79560dea19f 100644
> > --- a/tools/perf/Documentation/perf-ftrace.txt
> > +++ b/tools/perf/Documentation/perf-ftrace.txt
> > @@ -30,6 +30,10 @@ OPTIONS
> > --verbose=::
> > Verbosity level.
> >
> > +-F::
> > +--funcs::
> > + List all available functions to trace.
> > +
> > -p::
> > --pid=::
> > Trace on existing process id (comma separated list).
> > diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> > index 5f53da87040d..244cc8e6bd60 100644
> > --- a/tools/perf/builtin-ftrace.c
> > +++ b/tools/perf/builtin-ftrace.c
> > @@ -32,6 +32,7 @@ struct perf_ftrace {
> > struct evlist *evlist;
> > struct target target;
> > const char *tracer;
> > + bool list_avail_functions;
> > struct list_head filters;
> > struct list_head notrace;
> > struct list_head graph_funcs;
> > @@ -127,6 +128,43 @@ static int append_tracing_file(const char *name, const char *val)
> > return __write_tracing_file(name, val, true);
> > }
> > +static int read_tracing_file_to_stdout(const char *name)
> > +{
> All this is looking like its duplicating code that we are working on
> for libtracefs.
> Would you like to start contributing to that, and when we get the
> libtracefs.so packed in distributions, we can easily create the
> perf ftrace without having to rewrite the wheel 10 times?
Or we can use as soon as it is available, not preventing 'perf ftrace'
from having to wait for libtracefs.so?
Duplication is normal at some point, Changbin is moving 'perf ftrace'
forward, and has been doing this thru several patch series revisions, if
we continue putting new requirements, it gets tiresome at some point :-\
- Arnaldo
> -- Steve
>
>
> > + char buf[4096];
> > + char *file;
> > + int fd;
> > + int ret = -1;
> > +
> > + file = get_tracing_file(name);
> > + if (!file) {
> > + pr_debug("cannot get tracing file: %s\n", name);
> > + return -1;
> > + }
> > +
> > + fd = open(file, O_RDONLY);
> > + if (fd < 0) {
> > + pr_debug("cannot open tracing file: %s: %s\n",
> > + name, str_error_r(errno, buf, sizeof(buf)));
> > + goto out;
> > + }
> > +
> > + /* read contents to stdout */
> > + while (true) {
> > + int n = read(fd, buf, sizeof(buf));
> > + if (n <= 0)
> > + goto out_close;
> > + if (fwrite(buf, n, 1, stdout) != 1)
> > + goto out_close;
> > + }
> > + ret = 0;
> > +
> > +out_close:
> > + close(fd);
> > +out:
> > + put_tracing_file(file);
> > + return ret;
> > +}
> > +
> > static int reset_tracing_cpu(void);
> > static void reset_tracing_filters(void);
> >
> > @@ -301,6 +339,9 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> > signal(SIGCHLD, sig_handler);
> > signal(SIGPIPE, sig_handler);
> >
> > + if (ftrace->list_avail_functions)
> > + return read_tracing_file_to_stdout("available_filter_functions");
> > +
> > if (reset_tracing_files(ftrace) < 0) {
> > pr_err("failed to reset ftrace\n");
> > goto out;
> > @@ -470,6 +511,8 @@ int cmd_ftrace(int argc, const char **argv)
> > const struct option ftrace_options[] = {
> > OPT_STRING('t', "tracer", &ftrace.tracer, "tracer",
> > "tracer to use: function or function_graph (This option is deprecated)"),
> > + OPT_BOOLEAN('F', "funcs", &ftrace.list_avail_functions,
> > + "Show available functions to filter"),
> > OPT_STRING('p', "pid", &ftrace.target.pid, "pid",
> > "trace on existing process id"),
> > OPT_INCR('v', "verbose", &verbose,
>
--
- Arnaldo
On Fri, 17 Jul 2020 13:21:16 -0300
Arnaldo Carvalho de Melo <[email protected]> wrote:
>
> > Would you like to start contributing to that, and when we get the
> > libtracefs.so packed in distributions, we can easily create the
> > perf ftrace without having to rewrite the wheel 10 times?
>
> Or we can use as soon as it is available, not preventing 'perf ftrace'
> from having to wait for libtracefs.so?
>
> Duplication is normal at some point, Changbin is moving 'perf ftrace'
> forward, and has been doing this thru several patch series revisions, if
> we continue putting new requirements, it gets tiresome at some point :-\
We're finally at the point to move libtracefs.so and libtraceevent.so
into their own repository.
My fear is that the two will become incompatible, and forked forever.
-- Steve
Em Fri, Jul 17, 2020 at 12:27:40PM -0400, Steven Rostedt escreveu:
> On Fri, 17 Jul 2020 13:21:16 -0300 Arnaldo Carvalho de Melo <[email protected]> wrote:
> > > Would you like to start contributing to that, and when we get the
> > > libtracefs.so packed in distributions, we can easily create the
> > > perf ftrace without having to rewrite the wheel 10 times?
> > Or we can use as soon as it is available, not preventing 'perf ftrace'
> > from having to wait for libtracefs.so?
> > Duplication is normal at some point, Changbin is moving 'perf ftrace'
> > forward, and has been doing this thru several patch series revisions, if
> > we continue putting new requirements, it gets tiresome at some point :-\
> We're finally at the point to move libtracefs.so and libtraceevent.so
> into their own repository.
> My fear is that the two will become incompatible, and forked forever.
I don't share this fear, and since libtracefs is not generally
available, this will make perf progress to be slowed down, so its better
to merge what he has so far, after some review issues that surfaced are
solved, and when a better way of achieving that is available, consider
using it.
- Arnaldo
Em Fri, Jul 17, 2020 at 09:26:50PM +0800, Changbin Du escreveu:
> On Thu, Jul 16, 2020 at 12:36:30PM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Sat, Jul 11, 2020 at 08:40:21PM +0800, Changbin Du escreveu:
> > > +++ b/tools/perf/Documentation/perf-ftrace.txt
> > > @@ -38,6 +38,10 @@ OPTIONS
> > > --pid=::
> > > Trace on existing process id (comma separated list).
> > > +-t::
> > > +--tid=::
> > > + Trace on existing thread id (comma separated list).
> > Humm, I just tried:
> > [root@five ~]# yes > /dev/null &
> > [1] 18265
> > [root@five ~]# perf ftrace --tid 18265
> > ^C[root@five ~]#
> > After waiting for a while, nothing, what am I doing wrong?
> I got it wrong. Currently ftrace only can filter by pid. If the pid is not
> the main thread it won't work.
> So this patch makes no sense. will drop this.
I think you could alternatively keep it but inform the user that this
target, available to other perf commands, isn't available for ftrace as
it doesn't support it, this way when the user goes from:
perf trace|top|record|script|report --tid 1234
to:
perf ftrace --tid 1234
He gets a message like:
ERROR: 'ftrace' doesn't support the --tid target, try with --pid
And that would be more useful, provides an explanation as why that
target can't be used and suggests an alternative.
- Arnaldo
On Fri, 17 Jul 2020 21:26:50 +0800
Changbin Du <[email protected]> wrote:
> On Thu, Jul 16, 2020 at 12:36:30PM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Sat, Jul 11, 2020 at 08:40:21PM +0800, Changbin Du escreveu:
> > > This allows us to trace single thread instead of the whole process.
> > >
> > > Signed-off-by: Changbin Du <[email protected]>
> > > ---
> > > tools/perf/Documentation/perf-ftrace.txt | 4 ++++
> > > tools/perf/builtin-ftrace.c | 2 ++
> > > 2 files changed, 6 insertions(+)
> > >
> > > diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> > > index d79560dea19f..e204bf6d50d8 100644
> > > --- a/tools/perf/Documentation/perf-ftrace.txt
> > > +++ b/tools/perf/Documentation/perf-ftrace.txt
> > > @@ -38,6 +38,10 @@ OPTIONS
> > > --pid=::
> > > Trace on existing process id (comma separated list).
> > >
> > > +-t::
> > > +--tid=::
> > > + Trace on existing thread id (comma separated list).
> > > +
> >
> >
> > Humm, I just tried:
> >
> > [root@five ~]# yes > /dev/null &
> > [1] 18265
> > [root@five ~]# perf ftrace --tid 18265
> > ^C[root@five ~]#
> >
> > After waiting for a while, nothing, what am I doing wrong?
> >
> I got it wrong. Currently ftrace only can filter by pid. If the pid is not
> the main thread it won't work.
Wait what?
The "pid" for ftrace is really a "task id" which is a thread id.
[root@bxtest ~]# yes > /dev/null &
[1] 6799
[root@bxtest ~]# trace-cmd record -e all -P 6799
Hit Ctrl^C to stop recording
^CCPU 0: 3573031 events lost
CPU0 data recorded at offset=0x838000
627675136 bytes in size
CPU1 data recorded at offset=0x25ed1000
0 bytes in size
CPU2 data recorded at offset=0x25ed1000
0 bytes in size
CPU3 data recorded at offset=0x25ed1000
0 bytes in size
CPU4 data recorded at offset=0x25ed1000
0 bytes in size
CPU5 data recorded at offset=0x25ed1000
0 bytes in size
CPU6 data recorded at offset=0x25ed1000
0 bytes in size
CPU7 data recorded at offset=0x25ed1000
0 bytes in size
[root@bxtest ~]# trace-cmd report | head
CPU 1 is empty
CPU 2 is empty
CPU 3 is empty
CPU 4 is empty
CPU 5 is empty
CPU 6 is empty
CPU 7 is empty
cpus=8
yes-6799 [000] 67825.580902: sys_exit: NR 1 = 8192
yes-6799 [000] 67825.580903: sys_exit_write: 0x2000
What's different about tid vs pid?
-- Steve
>
> So this patch makes no sense. will drop this.
>
On Sat, Jul 11, 2020 at 5:42 AM Changbin Du <[email protected]> wrote:
>
> This adds an option '-F/--funcs' to list all available functions to trace,
> which is read from tracing file 'available_filter_functions'.
>
> $ sudo ./perf ftrace -F | head
> trace_initcall_finish_cb
> initcall_blacklisted
> do_one_initcall
> do_one_initcall
> trace_initcall_start_cb
> run_init_process
> try_to_run_init_process
> match_dev_by_label
> match_dev_by_uuid
> rootfs_init_fs_context
>
> Signed-off-by: Changbin Du <[email protected]>
>
> ---
> v2: option name '-l/--list-functions' -> '-F/--funcs'
> ---
> tools/perf/Documentation/perf-ftrace.txt | 4 +++
> tools/perf/builtin-ftrace.c | 43 ++++++++++++++++++++++++
> 2 files changed, 47 insertions(+)
>
> diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> index 952e46669168..d79560dea19f 100644
> --- a/tools/perf/Documentation/perf-ftrace.txt
> +++ b/tools/perf/Documentation/perf-ftrace.txt
> @@ -30,6 +30,10 @@ OPTIONS
> --verbose=::
> Verbosity level.
>
> +-F::
> +--funcs::
> + List all available functions to trace.
> +
Does "perf list tracepoint" contain this information? Perhaps add a
note to clarify.
Thanks,
Ian
> -p::
> --pid=::
> Trace on existing process id (comma separated list).
> diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> index 5f53da87040d..244cc8e6bd60 100644
> --- a/tools/perf/builtin-ftrace.c
> +++ b/tools/perf/builtin-ftrace.c
> @@ -32,6 +32,7 @@ struct perf_ftrace {
> struct evlist *evlist;
> struct target target;
> const char *tracer;
> + bool list_avail_functions;
> struct list_head filters;
> struct list_head notrace;
> struct list_head graph_funcs;
> @@ -127,6 +128,43 @@ static int append_tracing_file(const char *name, const char *val)
> return __write_tracing_file(name, val, true);
> }
>
> +static int read_tracing_file_to_stdout(const char *name)
> +{
> + char buf[4096];
> + char *file;
> + int fd;
> + int ret = -1;
> +
> + file = get_tracing_file(name);
> + if (!file) {
> + pr_debug("cannot get tracing file: %s\n", name);
> + return -1;
> + }
> +
> + fd = open(file, O_RDONLY);
> + if (fd < 0) {
> + pr_debug("cannot open tracing file: %s: %s\n",
> + name, str_error_r(errno, buf, sizeof(buf)));
> + goto out;
> + }
> +
> + /* read contents to stdout */
> + while (true) {
> + int n = read(fd, buf, sizeof(buf));
> + if (n <= 0)
> + goto out_close;
> + if (fwrite(buf, n, 1, stdout) != 1)
> + goto out_close;
> + }
> + ret = 0;
> +
> +out_close:
> + close(fd);
> +out:
> + put_tracing_file(file);
> + return ret;
> +}
> +
> static int reset_tracing_cpu(void);
> static void reset_tracing_filters(void);
>
> @@ -301,6 +339,9 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
> signal(SIGCHLD, sig_handler);
> signal(SIGPIPE, sig_handler);
>
> + if (ftrace->list_avail_functions)
> + return read_tracing_file_to_stdout("available_filter_functions");
> +
> if (reset_tracing_files(ftrace) < 0) {
> pr_err("failed to reset ftrace\n");
> goto out;
> @@ -470,6 +511,8 @@ int cmd_ftrace(int argc, const char **argv)
> const struct option ftrace_options[] = {
> OPT_STRING('t', "tracer", &ftrace.tracer, "tracer",
> "tracer to use: function or function_graph (This option is deprecated)"),
> + OPT_BOOLEAN('F', "funcs", &ftrace.list_avail_functions,
> + "Show available functions to filter"),
> OPT_STRING('p', "pid", &ftrace.target.pid, "pid",
> "trace on existing process id"),
> OPT_INCR('v', "verbose", &verbose,
> --
> 2.25.1
>
Em Fri, Jul 17, 2020 at 01:01:24PM -0400, Steven Rostedt escreveu:
> On Fri, 17 Jul 2020 21:26:50 +0800
> Changbin Du <[email protected]> wrote:
>
> > On Thu, Jul 16, 2020 at 12:36:30PM -0300, Arnaldo Carvalho de Melo wrote:
> > > Em Sat, Jul 11, 2020 at 08:40:21PM +0800, Changbin Du escreveu:
> > > > This allows us to trace single thread instead of the whole process.
> > > >
> > > > Signed-off-by: Changbin Du <[email protected]>
> > > > ---
> > > > tools/perf/Documentation/perf-ftrace.txt | 4 ++++
> > > > tools/perf/builtin-ftrace.c | 2 ++
> > > > 2 files changed, 6 insertions(+)
> > > >
> > > > diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> > > > index d79560dea19f..e204bf6d50d8 100644
> > > > --- a/tools/perf/Documentation/perf-ftrace.txt
> > > > +++ b/tools/perf/Documentation/perf-ftrace.txt
> > > > @@ -38,6 +38,10 @@ OPTIONS
> > > > --pid=::
> > > > Trace on existing process id (comma separated list).
> > > >
> > > > +-t::
> > > > +--tid=::
> > > > + Trace on existing thread id (comma separated list).
> > > > +
> > > Humm, I just tried:
> > > [root@five ~]# yes > /dev/null &
> > > [1] 18265
> > > [root@five ~]# perf ftrace --tid 18265
> > > ^C[root@five ~]#
> > > After waiting for a while, nothing, what am I doing wrong?
> > I got it wrong. Currently ftrace only can filter by pid. If the pid is not
> > the main thread it won't work.
> Wait what?
> The "pid" for ftrace is really a "task id" which is a thread id.
> [root@bxtest ~]# yes > /dev/null &
> [1] 6799
> [root@bxtest ~]# trace-cmd record -e all -P 6799
> Hit Ctrl^C to stop recording
> ^CCPU 0: 3573031 events lost
> CPU0 data recorded at offset=0x838000
> 627675136 bytes in size
> CPU1 data recorded at offset=0x25ed1000
> 0 bytes in size
> CPU2 data recorded at offset=0x25ed1000
> 0 bytes in size
> CPU3 data recorded at offset=0x25ed1000
> 0 bytes in size
> CPU4 data recorded at offset=0x25ed1000
> 0 bytes in size
> CPU5 data recorded at offset=0x25ed1000
> 0 bytes in size
> CPU6 data recorded at offset=0x25ed1000
> 0 bytes in size
> CPU7 data recorded at offset=0x25ed1000
> 0 bytes in size
> [root@bxtest ~]# trace-cmd report | head
> CPU 1 is empty
> CPU 2 is empty
> CPU 3 is empty
> CPU 4 is empty
> CPU 5 is empty
> CPU 6 is empty
> CPU 7 is empty
> cpus=8
> yes-6799 [000] 67825.580902: sys_exit: NR 1 = 8192
> yes-6799 [000] 67825.580903: sys_exit_write: 0x2000
>
>
> What's different about tid vs pid?
Say you use:
^C[root@ssdandy ~]# cyclictest --smp -um -p95
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 0.05 0.03 0.06 2/409 29072
T: 0 (29065) P:95 I:1000 C: 518 Min: 2 Act: 2 Avg: 2 Max: 6
T: 1 (29066) P:95 I:1500 C: 343 Min: 2 Act: 2 Avg: 2 Max: 5
T: 2 (29067) P:95 I:2000 C: 256 Min: 2 Act: 2 Avg: 2 Max: 7
T: 3 (29068) P:95 I:2500 C: 203 Min: 2 Act: 2 Avg: 2 Max: 5
T: 4 (29069) P:95 I:3000 C: 168 Min: 2 Act: 2 Avg: 3 Max: 6
T: 5 (29070) P:95 I:3500 C: 143 Min: 2 Act: 2 Avg: 2 Max: 6
T: 6 (29071) P:95 I:4000 C: 124 Min: 2 Act: 2 Avg: 2 Max: 7
T: 7 (29072) P:95 I:4500 C: 110 Min: 2 Act: 2 Avg: 2 Max:
Then we do:
# pidof cyclictest
29064
#
If we use:
[root@ssdandy ~]# perf record --pid 29064
^C[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.052 MB perf.data (866 samples) ]
[root@ssdandy ~]# perf report --task
# pid tid ppid comm
0 0 -1 |swapper
29064 29064 -1 |cyclictest
29064 29065 -1 |cyclictest
29064 29066 -1 |cyclictest
29064 29067 -1 |cyclictest
29064 29068 -1 |cyclictest
29064 29069 -1 |cyclictest
29064 29070 -1 |cyclictest
29064 29071 -1 |cyclictest
29064 29072 -1 |cyclictest
[root@ssdandy ~]#
If we are interested only on the thread running on CPU3 we can do:
[root@ssdandy ~]# perf report --task
# pid tid ppid comm
0 0 -1 |swapper
29064 29064 -1 |cyclictest
29064 29068 -1 |cyclictest
[root@ssdandy ~]#
The first 29064 is just to have info on who created 29068, i.e.:
Its just the synthesized info for 29068 creator:
[root@ssdandy ~]# perf report -D | grep -w 29064/29064
0 0x4690 [0x30]: PERF_RECORD_COMM: cyclictest:29064/29064
0 0x46c0 [0x70]: PERF_RECORD_MMAP2 29064/29064: [0x400000(0xa000) @ 0 fd:00 136246288 0]: r-xp /usr/bin/cyclictest
0 0x4730 [0x80]: PERF_RECORD_MMAP2 29064/29064: [0x7f990f505000(0x15000) @ 0 fd:00 201479398 0]: r-xp /usr/lib64/libgcc_s-4.8.5-20150702.so.1
0 0x47b0 [0x70]: PERF_RECORD_MMAP2 29064/29064: [0x7f990f71b000(0x1c3000) @ 0 fd:00 201334455 0]: r-xp /usr/lib64/libc-2.17.so
0 0x4820 [0x78]: PERF_RECORD_MMAP2 29064/29064: [0x7f990fae9000(0xa000) @ 0 fd:00 204604380 0]: r-xp /usr/lib64/libnuma.so.1.0.0
0 0x4898 [0x78]: PERF_RECORD_MMAP2 29064/29064: [0x7f990fcf5000(0x17000) @ 0 fd:00 201335636 0]: r-xp /usr/lib64/libpthread-2.17.so
0 0x4910 [0x78]: PERF_RECORD_MMAP2 29064/29064: [0x7f990ff11000(0x7000) @ 0 fd:00 201335640 0]: r-xp /usr/lib64/librt-2.17.so
0 0x4988 [0x70]: PERF_RECORD_MMAP2 29064/29064: [0x7f9910119000(0x22000) @ 0 fd:00 203595299 0]: r-xp /usr/lib64/ld-2.17.so
0 0x49f8 [0x60]: PERF_RECORD_MMAP2 29064/29064: [0x7fff0b52d000(0x2000) @ 0 00:00 0 0]: r-xp [vdso]
0 0x4a58 [0x68]: PERF_RECORD_MMAP2 29064/29064: [0xffffffffff600000(0x1000) @ 0 00:00 0 0]: r-xp [vsyscall]
[root@ssdandy ~]#
No PERF_RECORD_SAMPLEs.
Those are only for:
[root@ssdandy ~]# perf report -D | grep PERF_RECORD_SAMPLE | head -5
147224656598815 0x4ac0 [0x28]: PERF_RECORD_SAMPLE(IP, 0x4001): 29064/29068: 0xffffffffa8e5b568 period: 1 addr: 0
147224656606270 0x4ae8 [0x28]: PERF_RECORD_SAMPLE(IP, 0x4001): 29064/29068: 0xffffffffa8e5b568 period: 1 addr: 0
147224656611284 0x4b10 [0x28]: PERF_RECORD_SAMPLE(IP, 0x4001): 29064/29068: 0xffffffffa8e5b568 period: 5 addr: 0
147224656616225 0x4b38 [0x28]: PERF_RECORD_SAMPLE(IP, 0x4001): 29064/29068: 0xffffffffa8e5b568 period: 35 addr: 0
147224656621152 0x4b60 [0x28]: PERF_RECORD_SAMPLE(IP, 0x4001): 29064/29068: 0xffffffffa8e5b568 period: 252 addr: 0
[root@ssdandy ~]#
[root@ssdandy ~]# perf report -D | grep PERF_RECORD_SAMPLE | cut -d: -f3 | sort -u
29064/29068
[root@ssdandy ~]#
Is there a way in ftrace to ask for a pid and its children? Pre-existing
and and new ones the --pid specified will create after we start
monitoring?
- Arnaldo
On Fri, 17 Jul 2020 14:40:53 -0300
Arnaldo Carvalho de Melo <[email protected]> wrote:
> Say you use:
>
> ^C[root@ssdandy ~]# cyclictest --smp -um -p95
> # /dev/cpu_dma_latency set to 0us
> policy: fifo: loadavg: 0.05 0.03 0.06 2/409 29072
>
> T: 0 (29065) P:95 I:1000 C: 518 Min: 2 Act: 2 Avg: 2 Max: 6
> T: 1 (29066) P:95 I:1500 C: 343 Min: 2 Act: 2 Avg: 2 Max: 5
> T: 2 (29067) P:95 I:2000 C: 256 Min: 2 Act: 2 Avg: 2 Max: 7
> T: 3 (29068) P:95 I:2500 C: 203 Min: 2 Act: 2 Avg: 2 Max: 5
> T: 4 (29069) P:95 I:3000 C: 168 Min: 2 Act: 2 Avg: 3 Max: 6
> T: 5 (29070) P:95 I:3500 C: 143 Min: 2 Act: 2 Avg: 2 Max: 6
> T: 6 (29071) P:95 I:4000 C: 124 Min: 2 Act: 2 Avg: 2 Max: 7
> T: 7 (29072) P:95 I:4500 C: 110 Min: 2 Act: 2 Avg: 2 Max:
>
> Then we do:
>
> # pidof cyclictest
> 29064
> #
>
> If we use:
>
> [root@ssdandy ~]# perf record --pid 29064
> ^C[ perf record: Woken up 1 times to write data ]
> [ perf record: Captured and wrote 0.052 MB perf.data (866 samples) ]
>
> [root@ssdandy ~]# perf report --task
> # pid tid ppid comm
> 0 0 -1 |swapper
> 29064 29064 -1 |cyclictest
> 29064 29065 -1 |cyclictest
> 29064 29066 -1 |cyclictest
> 29064 29067 -1 |cyclictest
> 29064 29068 -1 |cyclictest
> 29064 29069 -1 |cyclictest
> 29064 29070 -1 |cyclictest
> 29064 29071 -1 |cyclictest
> 29064 29072 -1 |cyclictest
> [root@ssdandy ~]#
"ftrace" is inside the kernel. But you could specify all those PIDs in
the set_ftrace_pid and set_event_pid files, and they will be traced. If
you want to trace the children of those PIDs, you would need to set the
options function function-fork and event-fork respectively. And then
any time a process with a pid in the set_ftrace_pid or set_event_pid
file forks, its child will also be added to that file and it too will
be traced. If the fork options are set, then when a task exits, its pid
will be removed from the file.
echo 1 > options/function-fork
echo 1 > options/event-fork
>
> If we are interested only on the thread running on CPU3 we can do:
>
> [root@ssdandy ~]# perf report --task
> # pid tid ppid comm
> 0 0 -1 |swapper
> 29064 29064 -1 |cyclictest
> 29064 29068 -1 |cyclictest
> [root@ssdandy ~]#
>
> The first 29064 is just to have info on who created 29068, i.e.:
>
> Its just the synthesized info for 29068 creator:
>
> [root@ssdandy ~]# perf report -D | grep -w 29064/29064
> 0 0x4690 [0x30]: PERF_RECORD_COMM: cyclictest:29064/29064
> 0 0x46c0 [0x70]: PERF_RECORD_MMAP2 29064/29064: [0x400000(0xa000) @ 0 fd:00 136246288 0]: r-xp /usr/bin/cyclictest
> 0 0x4730 [0x80]: PERF_RECORD_MMAP2 29064/29064: [0x7f990f505000(0x15000) @ 0 fd:00 201479398 0]: r-xp /usr/lib64/libgcc_s-4.8.5-20150702.so.1
> 0 0x47b0 [0x70]: PERF_RECORD_MMAP2 29064/29064: [0x7f990f71b000(0x1c3000) @ 0 fd:00 201334455 0]: r-xp /usr/lib64/libc-2.17.so
> 0 0x4820 [0x78]: PERF_RECORD_MMAP2 29064/29064: [0x7f990fae9000(0xa000) @ 0 fd:00 204604380 0]: r-xp /usr/lib64/libnuma.so.1.0.0
> 0 0x4898 [0x78]: PERF_RECORD_MMAP2 29064/29064: [0x7f990fcf5000(0x17000) @ 0 fd:00 201335636 0]: r-xp /usr/lib64/libpthread-2.17.so
> 0 0x4910 [0x78]: PERF_RECORD_MMAP2 29064/29064: [0x7f990ff11000(0x7000) @ 0 fd:00 201335640 0]: r-xp /usr/lib64/librt-2.17.so
> 0 0x4988 [0x70]: PERF_RECORD_MMAP2 29064/29064: [0x7f9910119000(0x22000) @ 0 fd:00 203595299 0]: r-xp /usr/lib64/ld-2.17.so
> 0 0x49f8 [0x60]: PERF_RECORD_MMAP2 29064/29064: [0x7fff0b52d000(0x2000) @ 0 00:00 0 0]: r-xp [vdso]
> 0 0x4a58 [0x68]: PERF_RECORD_MMAP2 29064/29064: [0xffffffffff600000(0x1000) @ 0 00:00 0 0]: r-xp [vsyscall]
> [root@ssdandy ~]#
>
> No PERF_RECORD_SAMPLEs.
>
> Those are only for:
>
> [root@ssdandy ~]# perf report -D | grep PERF_RECORD_SAMPLE | head -5
> 147224656598815 0x4ac0 [0x28]: PERF_RECORD_SAMPLE(IP, 0x4001): 29064/29068: 0xffffffffa8e5b568 period: 1 addr: 0
> 147224656606270 0x4ae8 [0x28]: PERF_RECORD_SAMPLE(IP, 0x4001): 29064/29068: 0xffffffffa8e5b568 period: 1 addr: 0
> 147224656611284 0x4b10 [0x28]: PERF_RECORD_SAMPLE(IP, 0x4001): 29064/29068: 0xffffffffa8e5b568 period: 5 addr: 0
> 147224656616225 0x4b38 [0x28]: PERF_RECORD_SAMPLE(IP, 0x4001): 29064/29068: 0xffffffffa8e5b568 period: 35 addr: 0
> 147224656621152 0x4b60 [0x28]: PERF_RECORD_SAMPLE(IP, 0x4001): 29064/29068: 0xffffffffa8e5b568 period: 252 addr: 0
> [root@ssdandy ~]#
>
> [root@ssdandy ~]# perf report -D | grep PERF_RECORD_SAMPLE | cut -d: -f3 | sort -u
> 29064/29068
The above can somewhat be done in trace-cmd, but not fully. But that's
all userspace commands, nothing with the kernel.
> [root@ssdandy ~]#
>
> Is there a way in ftrace to ask for a pid and its children? Pre-existing
> and and new ones the --pid specified will create after we start
> monitoring?
>
As described above, yes. :-)
-- Steve
Em Fri, Jul 17, 2020 at 01:53:51PM -0400, Steven Rostedt escreveu:
> On Fri, 17 Jul 2020 14:40:53 -0300 Arnaldo Carvalho de Melo <[email protected]> wrote:
> > Say you use:
> > ^C[root@ssdandy ~]# cyclictest --smp -um -p95
> > # /dev/cpu_dma_latency set to 0us
> > policy: fifo: loadavg: 0.05 0.03 0.06 2/409 29072
> > T: 0 (29065) P:95 I:1000 C: 518 Min: 2 Act: 2 Avg: 2 Max: 6
> > T: 1 (29066) P:95 I:1500 C: 343 Min: 2 Act: 2 Avg: 2 Max: 5
> > T: 2 (29067) P:95 I:2000 C: 256 Min: 2 Act: 2 Avg: 2 Max: 7
> > T: 3 (29068) P:95 I:2500 C: 203 Min: 2 Act: 2 Avg: 2 Max: 5
> > T: 4 (29069) P:95 I:3000 C: 168 Min: 2 Act: 2 Avg: 3 Max: 6
> > T: 5 (29070) P:95 I:3500 C: 143 Min: 2 Act: 2 Avg: 2 Max: 6
> > T: 6 (29071) P:95 I:4000 C: 124 Min: 2 Act: 2 Avg: 2 Max: 7
> > T: 7 (29072) P:95 I:4500 C: 110 Min: 2 Act: 2 Avg: 2 Max:
> > Then we do:
> > # pidof cyclictest
> > 29064
> > #
> > If we use:
> > [root@ssdandy ~]# perf record --pid 29064
> > ^C[ perf record: Woken up 1 times to write data ]
> > [ perf record: Captured and wrote 0.052 MB perf.data (866 samples) ]
> > [root@ssdandy ~]# perf report --task
> > # pid tid ppid comm
> > 0 0 -1 |swapper
> > 29064 29064 -1 |cyclictest
> > 29064 29065 -1 |cyclictest
> > 29064 29066 -1 |cyclictest
> > 29064 29067 -1 |cyclictest
> > 29064 29068 -1 |cyclictest
> > 29064 29069 -1 |cyclictest
> > 29064 29070 -1 |cyclictest
> > 29064 29071 -1 |cyclictest
> > 29064 29072 -1 |cyclictest
> > [root@ssdandy ~]#
> "ftrace" is inside the kernel. But you could specify all those PIDs in
> the set_ftrace_pid and set_event_pid files, and they will be traced. If
> you want to trace the children of those PIDs, you would need to set the
> options function function-fork and event-fork respectively. And then
> any time a process with a pid in the set_ftrace_pid or set_event_pid
> file forks, its child will also be added to that file and it too will
> be traced. If the fork options are set, then when a task exits, its pid
> will be removed from the file.
> echo 1 > options/function-fork
> echo 1 > options/event-fork
cool, so its possible to have the same sort of behaviour one expects
using --pid or --tid and --inherit with perf record/trace/top/etc
Its just some confusion about what perf understands by --pid and --tid
and the jargon used by ftrace, we can make 'perf ftrace' use the knobs
you mentioned and achieve the expected results as with other perf
subcommands, good.
> > If we are interested only on the thread running on CPU3 we can do:
> >
> > [root@ssdandy ~]# perf report --task
> > # pid tid ppid comm
> > 0 0 -1 |swapper
> > 29064 29064 -1 |cyclictest
> > 29064 29068 -1 |cyclictest
> > [root@ssdandy ~]#
> > The first 29064 is just to have info on who created 29068, i.e.:
> > Its just the synthesized info for 29068 creator:
> > [root@ssdandy ~]# perf report -D | grep -w 29064/29064
> > 0 0x4690 [0x30]: PERF_RECORD_COMM: cyclictest:29064/29064
> > 0 0x46c0 [0x70]: PERF_RECORD_MMAP2 29064/29064: [0x400000(0xa000) @ 0 fd:00 136246288 0]: r-xp /usr/bin/cyclictest
> > 0 0x4730 [0x80]: PERF_RECORD_MMAP2 29064/29064: [0x7f990f505000(0x15000) @ 0 fd:00 201479398 0]: r-xp /usr/lib64/libgcc_s-4.8.5-20150702.so.1
> > 0 0x47b0 [0x70]: PERF_RECORD_MMAP2 29064/29064: [0x7f990f71b000(0x1c3000) @ 0 fd:00 201334455 0]: r-xp /usr/lib64/libc-2.17.so
> > 0 0x4820 [0x78]: PERF_RECORD_MMAP2 29064/29064: [0x7f990fae9000(0xa000) @ 0 fd:00 204604380 0]: r-xp /usr/lib64/libnuma.so.1.0.0
> > 0 0x4898 [0x78]: PERF_RECORD_MMAP2 29064/29064: [0x7f990fcf5000(0x17000) @ 0 fd:00 201335636 0]: r-xp /usr/lib64/libpthread-2.17.so
> > 0 0x4910 [0x78]: PERF_RECORD_MMAP2 29064/29064: [0x7f990ff11000(0x7000) @ 0 fd:00 201335640 0]: r-xp /usr/lib64/librt-2.17.so
> > 0 0x4988 [0x70]: PERF_RECORD_MMAP2 29064/29064: [0x7f9910119000(0x22000) @ 0 fd:00 203595299 0]: r-xp /usr/lib64/ld-2.17.so
> > 0 0x49f8 [0x60]: PERF_RECORD_MMAP2 29064/29064: [0x7fff0b52d000(0x2000) @ 0 00:00 0 0]: r-xp [vdso]
> > 0 0x4a58 [0x68]: PERF_RECORD_MMAP2 29064/29064: [0xffffffffff600000(0x1000) @ 0 00:00 0 0]: r-xp [vsyscall]
> > [root@ssdandy ~]#
> > No PERF_RECORD_SAMPLEs.
> > Those are only for:
> > [root@ssdandy ~]# perf report -D | grep PERF_RECORD_SAMPLE | head -5
> > 147224656598815 0x4ac0 [0x28]: PERF_RECORD_SAMPLE(IP, 0x4001): 29064/29068: 0xffffffffa8e5b568 period: 1 addr: 0
> > 147224656606270 0x4ae8 [0x28]: PERF_RECORD_SAMPLE(IP, 0x4001): 29064/29068: 0xffffffffa8e5b568 period: 1 addr: 0
> > 147224656611284 0x4b10 [0x28]: PERF_RECORD_SAMPLE(IP, 0x4001): 29064/29068: 0xffffffffa8e5b568 period: 5 addr: 0
> > 147224656616225 0x4b38 [0x28]: PERF_RECORD_SAMPLE(IP, 0x4001): 29064/29068: 0xffffffffa8e5b568 period: 35 addr: 0
> > 147224656621152 0x4b60 [0x28]: PERF_RECORD_SAMPLE(IP, 0x4001): 29064/29068: 0xffffffffa8e5b568 period: 252 addr: 0
> > [root@ssdandy ~]#
> > [root@ssdandy ~]# perf report -D | grep PERF_RECORD_SAMPLE | cut -d: -f3 | sort -u
> > 29064/29068
> The above can somewhat be done in trace-cmd, but not fully. But that's
> all userspace commands, nothing with the kernel.
> > [root@ssdandy ~]#
> > Is there a way in ftrace to ask for a pid and its children? Pre-existing
> > and and new ones the --pid specified will create after we start
> > monitoring?
> As described above, yes. :-)
Changbin, you can take from here :-)
And to reiterate, for me the value of 'perf ftrace' is to allow people
used to perf to be able to switch to ftrace quickly, just changing:
perf record/top/stat/trace/report/script/etc --pid 1234
by:
perf ftrace --pid 1234
And have the tracefs ftrace knobs set up to have what is expected in
terms of targets to trace as the other perf tools.
And not just --pid and --tid, but --cgroup, --cpu, etc.
i.e., 'perf ftrace' being _a_ front-end aplication to ftrace.
:-)
- Arnaldo
On Fri, 17 Jul 2020 16:34:55 -0300
Arnaldo Carvalho de Melo <[email protected]> wrote:
Thinking a bit more, I have to ask. Does perf use the kernel when
getting all the children of an existing task, or is that done only in
userspace?
That is, is there a perf syscall that says "start tracing this task and
all its existing children"?
Or is it done by perf user space looking at the /proc filesystem (like
ps does).
I'm asking because if perf has a syscall to do that, then I probably
should add a way to do that with ftrace as well. But that's really
trivial, because all it would take is grabbing the task_list lock and
iterating over all the children. Getting new children was the
non-trivial part, which was what I focused on (with the fork options).
If perf does it with proc files, then we don't need to change anything
as that could still be used with ftrace.
> Changbin, you can take from here :-)
>
> And to reiterate, for me the value of 'perf ftrace' is to allow people
> used to perf to be able to switch to ftrace quickly, just changing:
>
> perf record/top/stat/trace/report/script/etc --pid 1234
>
> by:
>
> perf ftrace --pid 1234
>
> And have the tracefs ftrace knobs set up to have what is expected in
> terms of targets to trace as the other perf tools.
>
> And not just --pid and --tid, but --cgroup, --cpu, etc.
>
> i.e., 'perf ftrace' being _a_ front-end aplication to ftrace.
>
> :-)
I have no problem with this, and I'm quite excited about it. I would
like it to use libtracefs, as it looks to be exactly what we are
working on. And this is now a high priority to get out, and I don't
expect another year (or two) in doing so ;-)
-- Steve
On Fri, Jul 17, 2020 at 01:01:24PM -0400, Steven Rostedt wrote:
> On Fri, 17 Jul 2020 21:26:50 +0800
> Changbin Du <[email protected]> wrote:
>
> > On Thu, Jul 16, 2020 at 12:36:30PM -0300, Arnaldo Carvalho de Melo wrote:
> > > Em Sat, Jul 11, 2020 at 08:40:21PM +0800, Changbin Du escreveu:
> > > > This allows us to trace single thread instead of the whole process.
> > > >
> > > > Signed-off-by: Changbin Du <[email protected]>
> > > > ---
> > > > tools/perf/Documentation/perf-ftrace.txt | 4 ++++
> > > > tools/perf/builtin-ftrace.c | 2 ++
> > > > 2 files changed, 6 insertions(+)
> > > >
> > > > diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> > > > index d79560dea19f..e204bf6d50d8 100644
> > > > --- a/tools/perf/Documentation/perf-ftrace.txt
> > > > +++ b/tools/perf/Documentation/perf-ftrace.txt
> > > > @@ -38,6 +38,10 @@ OPTIONS
> > > > --pid=::
> > > > Trace on existing process id (comma separated list).
> > > >
> > > > +-t::
> > > > +--tid=::
> > > > + Trace on existing thread id (comma separated list).
> > > > +
> > >
> > >
> > > Humm, I just tried:
> > >
> > > [root@five ~]# yes > /dev/null &
> > > [1] 18265
> > > [root@five ~]# perf ftrace --tid 18265
> > > ^C[root@five ~]#
> > >
> > > After waiting for a while, nothing, what am I doing wrong?
> > >
> > I got it wrong. Currently ftrace only can filter by pid. If the pid is not
> > the main thread it won't work.
>
> Wait what?
>
> The "pid" for ftrace is really a "task id" which is a thread id.
>
My bad. I traced a sleeping thread yesterday so no event generated.
Now it works:
$ pstree -p 2378
qemu-system-x86(2378)─┬─{qemu-system-x86}(2379)
├─{qemu-system-x86}(2382)
├─{qemu-system-x86}(2385)
├─{qemu-system-x86}(2387)
├─{qemu-system-x86}(2388)
├─{qemu-system-x86}(2389)
├─{qemu-system-x86}(2390)
├─{qemu-system-x86}(2391)
├─{qemu-system-x86}(2392)
$ sudo ./perf ftrace --tid 2388
[sudo] password for changbin:
# tracer: function
#
# entries-in-buffer/entries-written: 0/0 #P:8
#
# TASK-PID CPU# TIMESTAMP FUNCTION
# | | | | |
<idle>-0 [001] 6561.553989: switch_mm_irqs_off <-__schedule
<idle>-0 [001] 6561.553996: load_new_mm_cr3 <-switch_mm_irqs_off
qemu-system-x86-2388 [001] 6561.553997: finish_task_switch <-__schedule
qemu-system-x86-2388 [001] 6561.553998: smp_irq_work_interrupt <-irq_work_interrupt
qemu-system-x86-2388 [001] 6561.553999: irq_enter <-smp_irq_work_interrupt
qemu-system-x86-2388 [001] 6561.553999: rcu_irq_enter <-irq_enter
qemu-system-x86-2388 [001] 6561.554000: __wake_up <-rb_wake_up_waiters
qemu-system-x86-2388 [001] 6561.554000: __wake_up_common_lock <-__wake_up
qemu-system-x86-2388 [001] 6561.554000: _raw_spin_lock_irqsave <-__wake_up_common_lock
qemu-system-x86-2388 [001] 6561.554000: __wake_up_common <-__wake_up_common_lock
...
> [root@bxtest ~]# yes > /dev/null &
> [1] 6799
> [root@bxtest ~]# trace-cmd record -e all -P 6799
> Hit Ctrl^C to stop recording
> ^CCPU 0: 3573031 events lost
> CPU0 data recorded at offset=0x838000
> 627675136 bytes in size
> CPU1 data recorded at offset=0x25ed1000
> 0 bytes in size
> CPU2 data recorded at offset=0x25ed1000
> 0 bytes in size
> CPU3 data recorded at offset=0x25ed1000
> 0 bytes in size
> CPU4 data recorded at offset=0x25ed1000
> 0 bytes in size
> CPU5 data recorded at offset=0x25ed1000
> 0 bytes in size
> CPU6 data recorded at offset=0x25ed1000
> 0 bytes in size
> CPU7 data recorded at offset=0x25ed1000
> 0 bytes in size
> [root@bxtest ~]# trace-cmd report | head
> CPU 1 is empty
> CPU 2 is empty
> CPU 3 is empty
> CPU 4 is empty
> CPU 5 is empty
> CPU 6 is empty
> CPU 7 is empty
> cpus=8
> yes-6799 [000] 67825.580902: sys_exit: NR 1 = 8192
> yes-6799 [000] 67825.580903: sys_exit_write: 0x2000
>
>
> What's different about tid vs pid?
>
> -- Steve
>
>
>
> >
> > So this patch makes no sense. will drop this.
> >
--
Cheers,
Changbin Du
On Fri, Jul 17, 2020 at 11:05:04AM -0400, Steven Rostedt wrote:
> On Sat, 11 Jul 2020 20:40:20 +0800
> Changbin Du <[email protected]> wrote:
>
> > This adds an option '-F/--funcs' to list all available functions to trace,
> > which is read from tracing file 'available_filter_functions'.
> >
> > $ sudo ./perf ftrace -F | head
> > trace_initcall_finish_cb
> > initcall_blacklisted
> > do_one_initcall
> > do_one_initcall
> > trace_initcall_start_cb
> > run_init_process
> > try_to_run_init_process
> > match_dev_by_label
> > match_dev_by_uuid
> > rootfs_init_fs_context
> >
> > Signed-off-by: Changbin Du <[email protected]>
> >
> > ---
> > v2: option name '-l/--list-functions' -> '-F/--funcs'
> > ---
> > tools/perf/Documentation/perf-ftrace.txt | 4 +++
> > tools/perf/builtin-ftrace.c | 43 ++++++++++++++++++++++++
> > 2 files changed, 47 insertions(+)
> >
> > diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt
> > index 952e46669168..d79560dea19f 100644
> > --- a/tools/perf/Documentation/perf-ftrace.txt
> > +++ b/tools/perf/Documentation/perf-ftrace.txt
> > @@ -30,6 +30,10 @@ OPTIONS
> > --verbose=::
> > Verbosity level.
> >
> > +-F::
> > +--funcs::
> > + List all available functions to trace.
> > +
> > -p::
> > --pid=::
> > Trace on existing process id (comma separated list).
> > diff --git a/tools/perf/builtin-ftrace.c b/tools/perf/builtin-ftrace.c
> > index 5f53da87040d..244cc8e6bd60 100644
> > --- a/tools/perf/builtin-ftrace.c
> > +++ b/tools/perf/builtin-ftrace.c
> > @@ -32,6 +32,7 @@ struct perf_ftrace {
> > struct evlist *evlist;
> > struct target target;
> > const char *tracer;
> > + bool list_avail_functions;
> > struct list_head filters;
> > struct list_head notrace;
> > struct list_head graph_funcs;
> > @@ -127,6 +128,43 @@ static int append_tracing_file(const char *name, const char *val)
> > return __write_tracing_file(name, val, true);
> > }
> >
> > +static int read_tracing_file_to_stdout(const char *name)
> > +{
>
> All this is looking like its duplicating code that we are working on
> for libtracefs.
>
> Would you like to start contributing to that, and when we get the
> libtracefs.so packed in distributions, we can easily create the
> perf ftrace without having to rewrite the wheel 10 times?
>
> -- Steve
Yes, I'd like to join the development of libtracefs. But honestly speaking,
I am not sure whether I can get enough time on it since I have to take care
of my little baby :).
I have the same opinion with Arnaldo that we can merge what we'v already
done here. Then consider to migrate to libtracefs instead before adding
more new features.
--
Cheers,
Changbin Du
Hi Steve,
On Sat, Jul 18, 2020 at 4:44 AM Steven Rostedt <[email protected]> wrote:
>
> On Fri, 17 Jul 2020 16:34:55 -0300
> Arnaldo Carvalho de Melo <[email protected]> wrote:
>
> Thinking a bit more, I have to ask. Does perf use the kernel when
> getting all the children of an existing task, or is that done only in
> userspace?
>
> That is, is there a perf syscall that says "start tracing this task and
> all its existing children"?
>
> Or is it done by perf user space looking at the /proc filesystem (like
> ps does).
Yep, perf does look up the /proc to get a list of threads in a process.
Thanks
Namhyung
>
> I'm asking because if perf has a syscall to do that, then I probably
> should add a way to do that with ftrace as well. But that's really
> trivial, because all it would take is grabbing the task_list lock and
> iterating over all the children. Getting new children was the
> non-trivial part, which was what I focused on (with the fork options).
>
> If perf does it with proc files, then we don't need to change anything
> as that could still be used with ftrace.
>
> > Changbin, you can take from here :-)
> >
> > And to reiterate, for me the value of 'perf ftrace' is to allow people
> > used to perf to be able to switch to ftrace quickly, just changing:
> >
> > perf record/top/stat/trace/report/script/etc --pid 1234
> >
> > by:
> >
> > perf ftrace --pid 1234
> >
> > And have the tracefs ftrace knobs set up to have what is expected in
> > terms of targets to trace as the other perf tools.
> >
> > And not just --pid and --tid, but --cgroup, --cpu, etc.
> >
> > i.e., 'perf ftrace' being _a_ front-end aplication to ftrace.
> >
> > :-)
>
>
> I have no problem with this, and I'm quite excited about it. I would
> like it to use libtracefs, as it looks to be exactly what we are
> working on. And this is now a high priority to get out, and I don't
> expect another year (or two) in doing so ;-)
>
> -- Steve