2009-11-04 00:17:46

by Masami Hiramatsu

[permalink] [raw]
Subject: [PATCH -tip perf/probes 0/5] perf-probe and kprobe-tracer updates

Hi,

Here are some updates according to previous LKML threads.
- Update perf-probe document.
- Improve error messages.
- Fall back to non-dwarf mode if possible.
- Change group name to probe.
- Rename kprobe-tracer to kprobe-event.

BTW, I think perf-probe and kprobe-event might better share
similar syntax for not confusing users. And for that purpose,
perf-probe syntax should introduce event/group specifier,
for example,

perf probe "newgroup:newevnt=func:10 arg1 arg2"

adds the newevent under newgroup. On the other hand, ftrace
users can also add a new event as below;

echo 'newgroup:newevent=func+0x18 arg1=$a1 arg2=$a2' > kprobe_events

Any thoughts?

TODO:
- Support --list option to show probes.
- Support --del option to remove probes.
- Simplify probe names.
- Support --line option to show which lines user can probe.
- Support lazy string matching.

Thank you,

---

Masami Hiramatsu (5):
tracing/kprobes: Rename Kprobe-tracer to kprobe-event
perf/probes: Rename perf probe events group name
perf/probes: Fall back to non-dwarf if possible
perf/probes: Improve error messages
perf/probe: Update Documentation/perf-probe.txt


Documentation/trace/kprobetrace.txt | 34 ++++++-------
kernel/trace/Kconfig | 19 ++++---
kernel/trace/Makefile | 2 -
kernel/trace/trace_kprobe.c | 6 +-
tools/perf/Documentation/perf-probe.txt | 17 +++---
tools/perf/builtin-probe.c | 84 ++++++++++++++++++-------------
tools/perf/util/probe-finder.c | 7 ++-
7 files changed, 95 insertions(+), 74 deletions(-)

--
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division
e-mail: [email protected]


2009-11-04 00:17:42

by Masami Hiramatsu

[permalink] [raw]
Subject: [PATCH -tip perf/probes 1/5] perf/probe: Update Documentation/perf-probe.txt

Update Documentation/perf-probe.txt accoding to syntax change.

Signed-off-by: Masami Hiramatsu <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Steven Rostedt <[email protected]>
Cc: Jim Keniston <[email protected]>
Cc: Ananth N Mavinakayanahalli <[email protected]>
Cc: Christoph Hellwig <[email protected]>
Cc: Frank Ch. Eigler <[email protected]>
Cc: Frederic Weisbecker <[email protected]>
Cc: Jason Baron <[email protected]>
Cc: K.Prasad <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Srikar Dronamraju <[email protected]>
---

tools/perf/Documentation/perf-probe.txt | 17 +++++++++--------
1 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/tools/perf/Documentation/perf-probe.txt b/tools/perf/Documentation/perf-probe.txt
index 6b6c6ae..c47e54e 100644
--- a/tools/perf/Documentation/perf-probe.txt
+++ b/tools/perf/Documentation/perf-probe.txt
@@ -8,7 +8,9 @@ perf-probe - Define new dynamic tracepoints
SYNOPSIS
--------
[verse]
-'perf probe' [-k <file>] -P 'PROBE' [-P 'PROBE' ...]
+'perf probe' [options] --add 'PROBE' [--add 'PROBE' ...]
+or
+'perf probe' [options] 'PROBE' ['PROBE' ...]


DESCRIPTION
@@ -21,26 +23,25 @@ and C local variables) with debuginfo.
OPTIONS
-------
-k::
---vmlinux::
+--vmlinux=PATH::
Specify vmlinux path which has debuginfo (Dwarf binary).

-v::
--verbose::
Be more verbose (show parsed arguments, etc).

--P::
---probe::
+-a::
+--add::
Define a probe point (see PROBE SYNTAX for detail)

PROBE SYNTAX
------------
Probe points are defined by following syntax.

- "TYPE:[GRP/]NAME FUNC[+OFFS][@SRC]|@SRC:LINE [ARG ...]"
+ "FUNC[+OFFS|:RLN|%return][@SRC]|SRC:ALN [ARG ...]"

-'TYPE' specifies the type of probe point, you can use either "p" (kprobe) or "r" (kretprobe) for 'TYPE'. 'GRP' specifies the group name of this probe, and 'NAME' specifies the event name. If 'GRP' is omitted, "kprobes" is used for its group name.
-'FUNC' and 'OFFS' specifies function and offset (in byte) where probe will be put. In addition, 'SRC' specifies a source file which has that function (this is mainly for inline functions).
-You can specify a probe point by the source line number by using '@SRC:LINE' syntax, where 'SRC' is the source file path and 'LINE' is the line number.
+'FUNC' specifies a probed function name, and it may have one of following options; '+OFFS' is the offset from function entry address in byte, 'RLN' is the relative-line number from function entry line, and '%return' means that it probes function return. In addition, 'SRC' specifies a source file which has that function.
+It is also possible to specify a probe point by the source line number by using 'SRC:ALN' syntax, where 'SRC' is the source file path and 'ALN' is the line number.
'ARG' specifies the arguments of this probe point. You can use the name of local variable, or kprobe-tracer argument format (e.g. $retval, %ax, etc).

SEE ALSO


--
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: [email protected]

2009-11-04 00:17:52

by Masami Hiramatsu

[permalink] [raw]
Subject: [PATCH -tip perf/probes 2/5] perf/probes: Improve error messages

Improve error messages in perf-probe so that users can figure
out problems easily.

Signed-off-by: Masami Hiramatsu <[email protected]>
Reported-by: Ingo Molnar <[email protected]>
Cc: Steven Rostedt <[email protected]>
Cc: Jim Keniston <[email protected]>
Cc: Ananth N Mavinakayanahalli <[email protected]>
Cc: Christoph Hellwig <[email protected]>
Cc: Frank Ch. Eigler <[email protected]>
Cc: Frederic Weisbecker <[email protected]>
Cc: Jason Baron <[email protected]>
Cc: K.Prasad <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Srikar Dronamraju <[email protected]>
---

tools/perf/builtin-probe.c | 21 ++++++++++++++-------
tools/perf/util/probe-finder.c | 3 ++-
2 files changed, 16 insertions(+), 8 deletions(-)

diff --git a/tools/perf/builtin-probe.c b/tools/perf/builtin-probe.c
index a99a366..454ddfc 100644
--- a/tools/perf/builtin-probe.c
+++ b/tools/perf/builtin-probe.c
@@ -294,10 +294,11 @@ static int write_new_event(int fd, const char *buf)
{
int ret;

- printf("Adding new event: %s\n", buf);
ret = write(fd, buf, strlen(buf));
if (ret <= 0)
- die("failed to create event.");
+ die("Failed to create event.");
+ else
+ printf("Added new event: %s\n", buf);

return ret;
}
@@ -310,7 +311,7 @@ static int synthesize_probe_event(struct probe_point *pp)
int i, len, ret;
pp->probes[0] = buf = (char *)calloc(MAX_CMDLEN, sizeof(char));
if (!buf)
- die("calloc");
+ die("Failed to allocate memory by calloc.");
ret = snprintf(buf, MAX_CMDLEN, "%s+%d", pp->function, pp->offset);
if (ret <= 0 || ret >= MAX_CMDLEN)
goto error;
@@ -363,7 +364,7 @@ int cmd_probe(int argc, const char **argv, const char *prefix __used)
if (ret == -E2BIG)
semantic_error("probe point is too long.");
else if (ret < 0)
- die("snprintf");
+ die("Failed to synthesize a probe point.");
}

#ifndef NO_LIBDWARF
@@ -375,7 +376,7 @@ int cmd_probe(int argc, const char **argv, const char *prefix __used)
else
fd = open_default_vmlinux();
if (fd < 0)
- die("vmlinux/module file open");
+ die("Could not open vmlinux/module file.");

/* Searching probe points */
for (j = 0; j < session.nr_probe; j++) {
@@ -396,8 +397,14 @@ setup_probes:
/* Settng up probe points */
snprintf(buf, MAX_CMDLEN, "%s/../kprobe_events", debugfs_path);
fd = open(buf, O_WRONLY, O_APPEND);
- if (fd < 0)
- die("kprobe_events open");
+ if (fd < 0) {
+ if (errno == ENOENT)
+ die("kprobe_events file does not exist"
+ " - please rebuild with CONFIG_KPROBE_TRACER.");
+ else
+ die("Could not open kprobe_events file: %s",
+ strerror(errno));
+ }
for (j = 0; j < session.nr_probe; j++) {
pp = &session.probes[j];
if (pp->found == 1) {
diff --git a/tools/perf/util/probe-finder.c b/tools/perf/util/probe-finder.c
index db96186..86cead3 100644
--- a/tools/perf/util/probe-finder.c
+++ b/tools/perf/util/probe-finder.c
@@ -688,7 +688,8 @@ int find_probepoint(int fd, struct probe_point *pp)

ret = dwarf_init(fd, DW_DLC_READ, 0, 0, &__dw_debug, &__dw_error);
if (ret != DW_DLV_OK)
- die("Failed to call dwarf_init(). Maybe, not a dwarf file.\n");
+ die("Not found dwarf info in the vmlinux"
+ " - please rebuild with CONFIG_DEBUG_INFO.\n");

pp->found = 0;
while (++cu_number) {


--
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: [email protected]

2009-11-04 00:18:10

by Masami Hiramatsu

[permalink] [raw]
Subject: [PATCH -tip perf/probes 3/5] perf/probes: Fall back to non-dwarf if possible

Fall back to non-dwarf probe point if the probe definition may not
need dwarf analysis, when perf can't find vmlinux/debuginfo.
This might skip some inlined code of target function.

Signed-off-by: Masami Hiramatsu <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Steven Rostedt <[email protected]>
Cc: Jim Keniston <[email protected]>
Cc: Ananth N Mavinakayanahalli <[email protected]>
Cc: Christoph Hellwig <[email protected]>
Cc: Frank Ch. Eigler <[email protected]>
Cc: Frederic Weisbecker <[email protected]>
Cc: Jason Baron <[email protected]>
Cc: K.Prasad <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Srikar Dronamraju <[email protected]>
---

tools/perf/builtin-probe.c | 65 +++++++++++++++++++++++-----------------
tools/perf/util/probe-finder.c | 8 +++--
2 files changed, 42 insertions(+), 31 deletions(-)

diff --git a/tools/perf/builtin-probe.c b/tools/perf/builtin-probe.c
index 454ddfc..e44491b 100644
--- a/tools/perf/builtin-probe.c
+++ b/tools/perf/builtin-probe.c
@@ -189,7 +189,7 @@ static void parse_probe_event(const char *str)
/* Parse probe point */
parse_probe_point(argv[0], pp);
free(argv[0]);
- if (pp->file)
+ if (pp->file || pp->line)
session.need_dwarf = 1;

/* Copy arguments */
@@ -347,36 +347,24 @@ int cmd_probe(int argc, const char **argv, const char *prefix __used)
if (session.nr_probe == 0)
usage_with_options(probe_usage, options);

-#ifdef NO_LIBDWARF
if (session.need_dwarf)
- semantic_error("Dwarf-analysis is not supported");
-#endif
-
- /* Synthesize probes without dwarf */
- for (j = 0; j < session.nr_probe; j++) {
-#ifndef NO_LIBDWARF
- if (!session.probes[j].retprobe) {
- session.need_dwarf = 1;
- continue;
- }
-#endif
- ret = synthesize_probe_event(&session.probes[j]);
- if (ret == -E2BIG)
- semantic_error("probe point is too long.");
- else if (ret < 0)
- die("Failed to synthesize a probe point.");
- }
-
-#ifndef NO_LIBDWARF
- if (!session.need_dwarf)
- goto setup_probes;
+#ifdef NO_LIBDWARF
+ semantic_error("Debuginfo-analysis is not supported");
+#else /* !NO_LIBDWARF */
+ pr_info("Some probes require debuginfo.\n");

if (session.vmlinux)
fd = open(session.vmlinux, O_RDONLY);
else
fd = open_default_vmlinux();
- if (fd < 0)
- die("Could not open vmlinux/module file.");
+ if (fd < 0) {
+ if (session.need_dwarf)
+ die("Could not open vmlinux/module file.");
+
+ pr_warning("Could not open vmlinux/module file."
+ " Try to use symbols.\n");
+ goto end_dwarf;
+ }

/* Searching probe points */
for (j = 0; j < session.nr_probe; j++) {
@@ -386,14 +374,35 @@ int cmd_probe(int argc, const char **argv, const char *prefix __used)

lseek(fd, SEEK_SET, 0);
ret = find_probepoint(fd, pp);
- if (ret <= 0)
- die("No probe point found.\n");
+ if (ret < 0) {
+ if (session.need_dwarf)
+ die("Could not analyze debuginfo.");
+
+ pr_warning("An error was occurred in debuginfo"
+ " analysis. Try to use symbols.\n");
+ break;
+ }
+ if (ret == 0) /* No error but failed to find probe point. */
+ die("No probe point found.");
}
close(fd);

-setup_probes:
+end_dwarf:
#endif /* !NO_LIBDWARF */

+ /* Synthesize probes without dwarf */
+ for (j = 0; j < session.nr_probe; j++) {
+ pp = &session.probes[j];
+ if (pp->found) /* This probe is already found. */
+ continue;
+
+ ret = synthesize_probe_event(pp);
+ if (ret == -E2BIG)
+ semantic_error("probe point is too long.");
+ else if (ret < 0)
+ die("Failed to synthesize a probe point.");
+ }
+
/* Settng up probe points */
snprintf(buf, MAX_CMDLEN, "%s/../kprobe_events", debugfs_path);
fd = open(buf, O_WRONLY, O_APPEND);
diff --git a/tools/perf/util/probe-finder.c b/tools/perf/util/probe-finder.c
index 86cead3..f628dee 100644
--- a/tools/perf/util/probe-finder.c
+++ b/tools/perf/util/probe-finder.c
@@ -687,9 +687,11 @@ int find_probepoint(int fd, struct probe_point *pp)
struct probe_finder pf = {.pp = pp};

ret = dwarf_init(fd, DW_DLC_READ, 0, 0, &__dw_debug, &__dw_error);
- if (ret != DW_DLV_OK)
- die("Not found dwarf info in the vmlinux"
- " - please rebuild with CONFIG_DEBUG_INFO.\n");
+ if (ret != DW_DLV_OK) {
+ pr_warning("Not found dwarf info in the vmlinux"
+ " - please rebuild with CONFIG_DEBUG_INFO.\n");
+ return -ENOENT;
+ }

pp->found = 0;
while (++cu_number) {


--
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: [email protected]

2009-11-04 00:18:19

by Masami Hiramatsu

[permalink] [raw]
Subject: [PATCH -tip perf/probes 4/5] perf/probes: Rename perf probe events group name

Rename the group name of perf probe events to 'probe'.

Signed-off-by: Masami Hiramatsu <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Steven Rostedt <[email protected]>
Cc: Jim Keniston <[email protected]>
Cc: Ananth N Mavinakayanahalli <[email protected]>
Cc: Christoph Hellwig <[email protected]>
Cc: Frank Ch. Eigler <[email protected]>
Cc: Frederic Weisbecker <[email protected]>
Cc: Jason Baron <[email protected]>
Cc: K.Prasad <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Srikar Dronamraju <[email protected]>
---

tools/perf/builtin-probe.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/tools/perf/builtin-probe.c b/tools/perf/builtin-probe.c
index e44491b..73dd7d1 100644
--- a/tools/perf/builtin-probe.c
+++ b/tools/perf/builtin-probe.c
@@ -52,7 +52,7 @@ const char *default_search_path[NR_SEARCH_PATH] = {
#define MAX_PATH_LEN 256
#define MAX_PROBES 128
#define MAX_PROBE_ARGS 128
-#define PERFPROBE_GROUP "perfprobe"
+#define PERFPROBE_GROUP "probe"

/* Session management structure */
static struct {


--
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: [email protected]

2009-11-04 00:13:06

by Masami Hiramatsu

[permalink] [raw]
Subject: [PATCH -tip perf/probes 5/5] tracing/kprobes: Rename Kprobe-tracer to kprobe-event

Rename Kprobes-based event tracer to kprobes-based tracing event
(kprobe-event), since it is not a tracer but an extensible tracing
event interface.
This also changes CONFIG_KPROBE_TRACER to CONFIG_KPROBE_EVENT
and set y it by default.

Signed-off-by: Masami Hiramatsu <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Steven Rostedt <[email protected]>
Cc: Jim Keniston <[email protected]>
Cc: Ananth N Mavinakayanahalli <[email protected]>
Cc: Christoph Hellwig <[email protected]>
Cc: Frank Ch. Eigler <[email protected]>
Cc: Frederic Weisbecker <[email protected]>
Cc: Jason Baron <[email protected]>
Cc: K.Prasad <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Srikar Dronamraju <[email protected]>
---

Documentation/trace/kprobetrace.txt | 34 ++++++++++++++++------------------
kernel/trace/Kconfig | 19 ++++++++++++-------
kernel/trace/Makefile | 2 +-
kernel/trace/trace_kprobe.c | 6 ++----
4 files changed, 31 insertions(+), 30 deletions(-)

diff --git a/Documentation/trace/kprobetrace.txt b/Documentation/trace/kprobetrace.txt
index 1541524..bd6a2e2 100644
--- a/Documentation/trace/kprobetrace.txt
+++ b/Documentation/trace/kprobetrace.txt
@@ -1,26 +1,23 @@
- Kprobe-based Event Tracer
- =========================
+ Kprobe-based Tracing Event
+ ==========================

Documentation is written by Masami Hiramatsu


Overview
--------
-This tracer is similar to the events tracer which is based on Tracepoint
-infrastructure. Instead of Tracepoint, this tracer is based on kprobes(kprobe
-and kretprobe). It probes anywhere where kprobes can probe(this means, all
-functions body except for __kprobes functions).
+This event is similar to the tracepoint based event. Instead of Tracepoint,
+this is based on kprobes (kprobe and kretprobe). So it can probe wherever
+kprobes can probe (this means, all functions body except for __kprobes
+functions). Unlike the Tracepoint based event, this can be added and removed
+on the fly.

-Unlike the function tracer, this tracer can probe instructions inside of
-kernel functions. It allows you to check which instruction has been executed.
+For enabling this feature, build your kernel with CONFIG_KPROBE_TRACING=y.

-Unlike the Tracepoint based events tracer, this tracer can add and remove
-probe points on the fly.
-
-Similar to the events tracer, this tracer doesn't need to be activated via
-current_tracer, instead of that, just set probe points via
-/sys/kernel/debug/tracing/kprobe_events. And you can set filters on each
-probe events via /sys/kernel/debug/tracing/events/kprobes/<EVENT>/filter.
+Similar to the events tracer, this doesn't need to be activated via
+current_tracer. Instead of that, add probe points via
+/sys/kernel/debug/tracing/kprobe_events, and enable it via
+/sys/kernel/debug/tracing/events/kprobes/<EVENT>/enabled.


Synopsis of kprobe_events
@@ -55,9 +52,9 @@ Per-Probe Event Filtering
-------------------------
Per-probe event filtering feature allows you to set different filter on each
probe and gives you what arguments will be shown in trace buffer. If an event
-name is specified right after 'p:' or 'r:' in kprobe_events, the tracer adds
-an event under tracing/events/kprobes/<EVENT>, at the directory you can see
-'id', 'enabled', 'format' and 'filter'.
+name is specified right after 'p:' or 'r:' in kprobe_events, it adds an event
+under tracing/events/kprobes/<EVENT>, at the directory you can see 'id',
+'enabled', 'format' and 'filter'.

enabled:
You can enable/disable the probe by writing 1 or 0 on it.
@@ -71,6 +68,7 @@ filter:
id:
This shows the id of this probe event.

+
Event Profiling
---------------
You can check the total number of probe hits and probe miss-hits via
diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index 15372a9..d2ad714 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -428,17 +428,22 @@ config BLK_DEV_IO_TRACE

If unsure, say N.

-config KPROBE_TRACER
+config KPROBE_EVENT
depends on KPROBES
depends on X86
- bool "Trace kprobes"
+ bool "Enable kprobes-based dynamic events"
select TRACING
- select GENERIC_TRACER
+ default y
help
- This tracer probes everywhere where kprobes can probe it, and
- records various registers and memories specified by user.
- This also allows you to trace kprobe probe points as a dynamic
- defined events. It provides per-probe event filtering interface.
+ This allows user to add tracing events (like tracepoint) on the fly
+ via the ftrace interface. See Documentation/trace/kprobetrace.txt
+ for more details.
+
+ Those events can probe wherever kprobes can probe, and record
+ various registers and memories.
+
+ This option is required by perf-probe subcommand of perf tools. If
+ you want to use perf tools, this option is strongly recommended.

config DYNAMIC_FTRACE
bool "enable/disable ftrace tracepoints dynamically"
diff --git a/kernel/trace/Makefile b/kernel/trace/Makefile
index c8cb75d..edc3a3c 100644
--- a/kernel/trace/Makefile
+++ b/kernel/trace/Makefile
@@ -53,7 +53,7 @@ obj-$(CONFIG_EVENT_TRACING) += trace_export.o
obj-$(CONFIG_FTRACE_SYSCALLS) += trace_syscalls.o
obj-$(CONFIG_EVENT_PROFILE) += trace_event_profile.o
obj-$(CONFIG_EVENT_TRACING) += trace_events_filter.o
-obj-$(CONFIG_KPROBE_TRACER) += trace_kprobe.o
+obj-$(CONFIG_KPROBE_EVENT) += trace_kprobe.o
obj-$(CONFIG_EVENT_TRACING) += power-traces.o

libftrace-y := ftrace.o
diff --git a/kernel/trace/trace_kprobe.c b/kernel/trace/trace_kprobe.c
index a86c3ac..cf17a66 100644
--- a/kernel/trace/trace_kprobe.c
+++ b/kernel/trace/trace_kprobe.c
@@ -1,5 +1,5 @@
/*
- * kprobe based kernel tracer
+ * Kprobes-based tracing events
*
* Created by Masami Hiramatsu <[email protected]>
*
@@ -57,8 +57,6 @@ const char *reserved_field_names[] = {
FIELD_STRING_FUNC,
};

-/* currently, trace_kprobe only supports X86. */
-
struct fetch_func {
unsigned long (*func)(struct pt_regs *, void *);
void *data;
@@ -191,7 +189,7 @@ static __kprobes void free_indirect_fetch_data(struct indirect_fetch_data *data)
}

/**
- * Kprobe tracer core functions
+ * Kprobe event core functions
*/

struct probe_arg {


--
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: [email protected]

2009-11-04 02:15:30

by Frederic Weisbecker

[permalink] [raw]
Subject: Re: [PATCH -tip perf/probes 0/5] perf-probe and kprobe-tracer updates

On Tue, Nov 03, 2009 at 07:12:04PM -0500, Masami Hiramatsu wrote:
> Hi,
>
> Here are some updates according to previous LKML threads.
> - Update perf-probe document.
> - Improve error messages.
> - Fall back to non-dwarf mode if possible.
> - Change group name to probe.
> - Rename kprobe-tracer to kprobe-event.
>
> BTW, I think perf-probe and kprobe-event might better share
> similar syntax for not confusing users. And for that purpose,
> perf-probe syntax should introduce event/group specifier,
> for example,


I personally more imagine the debugfs kprobe-event interface as
something used by higher level applications rather than users.

I've tried to use kprobe events directly by the past to do
some debugging, and once I wanted to go further a simple function
probe, like fetching a variable or putting a probe in a given branch,
it rapidly grew into a pain: I had to read assembly code, guess
which register was matching which variable, etc... It works, but
it takes too much time, and printk() rapidly becomes a temptation :)

It too low-level, but its use through perf brings all that to the
human dimension.

So, I'm not sure we really need to have such tight syntax between
both, since the debugfs more likely behaves as a gateway, something
I don't imagine to be used broadly as an end-user interface but mostly
as a kernel interface.

Especially we shouldn't break the perf probe syntax simplicity
just because we want both syntaxes to be tight.

(Nothing related to the event/group feature itself, it's just an
opinion about the need of this similarity between two interfaces).


> perf probe "newgroup:newevnt=func:10 arg1 arg2"
>
> adds the newevent under newgroup. On the other hand, ftrace
> users can also add a new event as below;
>
> echo 'newgroup:newevent=func+0x18 arg1=$a1 arg2=$a2' > kprobe_events
>
> Any thoughts?


Yeah, that would probably be nice, especially once we have a good
collection of probes to handle and to organize in a sensical output.

But it would be better to have that as an optional thing:

perf probe "[group:name=]func...."


so that we keep the simplicity of:

perf probe func

I guess you meant it as optional already, but just in case... :)

Thanks.

2009-11-04 14:12:25

by Masami Hiramatsu

[permalink] [raw]
Subject: Re: [PATCH -tip perf/probes 0/5] perf-probe and kprobe-tracer updates

Frederic Weisbecker wrote:
> On Tue, Nov 03, 2009 at 07:12:04PM -0500, Masami Hiramatsu wrote:
>> BTW, I think perf-probe and kprobe-event might better share
>> similar syntax for not confusing users. And for that purpose,
>> perf-probe syntax should introduce event/group specifier,
>> for example,
>
>
> I personally more imagine the debugfs kprobe-event interface as
> something used by higher level applications rather than users.
>
> I've tried to use kprobe events directly by the past to do
> some debugging, and once I wanted to go further a simple function
> probe, like fetching a variable or putting a probe in a given branch,
> it rapidly grew into a pain: I had to read assembly code, guess
> which register was matching which variable, etc... It works, but
> it takes too much time, and printk() rapidly becomes a temptation :)
>
> It too low-level, but its use through perf brings all that to the
> human dimension.
>
> So, I'm not sure we really need to have such tight syntax between
> both, since the debugfs more likely behaves as a gateway, something
> I don't imagine to be used broadly as an end-user interface but mostly
> as a kernel interface.

I see, and I also found that the syntax never be same, since
perf-probe doesn't need argument names etc. kprobe_events
interface may be mostly for higher level scripts or programs.

> Especially we shouldn't break the perf probe syntax simplicity
> just because we want both syntaxes to be tight.

Agreed. OK, so let it be :-)

> (Nothing related to the event/group feature itself, it's just an
> opinion about the need of this similarity between two interfaces).
>
>
>> perf probe "newgroup:newevnt=func:10 arg1 arg2"
>>
>> adds the newevent under newgroup. On the other hand, ftrace
>> users can also add a new event as below;
>>
>> echo 'newgroup:newevent=func+0x18 arg1=$a1 arg2=$a2'> kprobe_events
>>
>> Any thoughts?
>
>
> Yeah, that would probably be nice, especially once we have a good
> collection of probes to handle and to organize in a sensical output.
>
> But it would be better to have that as an optional thing:
>
> perf probe "[group:name=]func...."


Sure, of course it should be optional. :-)

> so that we keep the simplicity of:
>
> perf probe func
>
> I guess you meant it as optional already, but just in case... :)

Thank you :-)

--
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: [email protected]

2009-11-04 15:25:27

by Masami Hiramatsu

[permalink] [raw]
Subject: [tip:perf/probes] perf/probes: Update Documentation/perf-probe.txt

Commit-ID: c43f9d1e61e265c6bfafdd65c7f07c8d71a7efc3
Gitweb: http://git.kernel.org/tip/c43f9d1e61e265c6bfafdd65c7f07c8d71a7efc3
Author: Masami Hiramatsu <[email protected]>
AuthorDate: Tue, 3 Nov 2009 19:12:13 -0500
Committer: Ingo Molnar <[email protected]>
CommitDate: Wed, 4 Nov 2009 13:02:46 +0100

perf/probes: Update Documentation/perf-probe.txt

Update Documentation/perf-probe.txt accoding to recent
syntax changes.

Signed-off-by: Masami Hiramatsu <[email protected]>
Acked-by: Frederic Weisbecker <[email protected]>
Cc: Steven Rostedt <[email protected]>
Cc: Jim Keniston <[email protected]>
Cc: Ananth N Mavinakayanahalli <[email protected]>
Cc: Christoph Hellwig <[email protected]>
Cc: Frank Ch. Eigler <[email protected]>
Cc: Jason Baron <[email protected]>
Cc: K.Prasad <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Srikar Dronamraju <[email protected]>
LKML-Reference: <20091104001212.3454.19415.stgit@harusame>
Signed-off-by: Ingo Molnar <[email protected]>
---
tools/perf/Documentation/perf-probe.txt | 17 +++++++++--------
1 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/tools/perf/Documentation/perf-probe.txt b/tools/perf/Documentation/perf-probe.txt
index 6b6c6ae..9270594 100644
--- a/tools/perf/Documentation/perf-probe.txt
+++ b/tools/perf/Documentation/perf-probe.txt
@@ -8,7 +8,9 @@ perf-probe - Define new dynamic tracepoints
SYNOPSIS
--------
[verse]
-'perf probe' [-k <file>] -P 'PROBE' [-P 'PROBE' ...]
+'perf probe' [options] --add 'PROBE' [--add 'PROBE' ...]
+or
+'perf probe' [options] 'PROBE' ['PROBE' ...]


DESCRIPTION
@@ -21,26 +23,25 @@ and C local variables) with debuginfo.
OPTIONS
-------
-k::
---vmlinux::
+--vmlinux=PATH::
Specify vmlinux path which has debuginfo (Dwarf binary).

-v::
--verbose::
Be more verbose (show parsed arguments, etc).

--P::
---probe::
+-a::
+--add::
Define a probe point (see PROBE SYNTAX for detail)

PROBE SYNTAX
------------
Probe points are defined by following syntax.

- "TYPE:[GRP/]NAME FUNC[+OFFS][@SRC]|@SRC:LINE [ARG ...]"
+ "FUNC[+OFFS|:RLN|%return][@SRC]|SRC:ALN [ARG ...]"

-'TYPE' specifies the type of probe point, you can use either "p" (kprobe) or "r" (kretprobe) for 'TYPE'. 'GRP' specifies the group name of this probe, and 'NAME' specifies the event name. If 'GRP' is omitted, "kprobes" is used for its group name.
-'FUNC' and 'OFFS' specifies function and offset (in byte) where probe will be put. In addition, 'SRC' specifies a source file which has that function (this is mainly for inline functions).
-You can specify a probe point by the source line number by using '@SRC:LINE' syntax, where 'SRC' is the source file path and 'LINE' is the line number.
+'FUNC' specifies a probed function name, and it may have one of the following options; '+OFFS' is the offset from function entry address in bytes, 'RLN' is the relative-line number from function entry line, and '%return' means that it probes function return. In addition, 'SRC' specifies a source file which has that function.
+It is also possible to specify a probe point by the source line number by using 'SRC:ALN' syntax, where 'SRC' is the source file path and 'ALN' is the line number.
'ARG' specifies the arguments of this probe point. You can use the name of local variable, or kprobe-tracer argument format (e.g. $retval, %ax, etc).

SEE ALSO

2009-11-04 15:25:43

by Masami Hiramatsu

[permalink] [raw]
Subject: [tip:perf/probes] perf/probes: Improve error messages

Commit-ID: a7f4328b91fb6e71dbe1fa4d46f3597c9555014d
Gitweb: http://git.kernel.org/tip/a7f4328b91fb6e71dbe1fa4d46f3597c9555014d
Author: Masami Hiramatsu <[email protected]>
AuthorDate: Tue, 3 Nov 2009 19:12:21 -0500
Committer: Ingo Molnar <[email protected]>
CommitDate: Wed, 4 Nov 2009 13:02:46 +0100

perf/probes: Improve error messages

Improve error messages in perf-probe so that users can figure
out problems easily.

Reported-by: Ingo Molnar <[email protected]>
Signed-off-by: Masami Hiramatsu <[email protected]>
Acked-by: Frederic Weisbecker <[email protected]>
Cc: Steven Rostedt <[email protected]>
Cc: Jim Keniston <[email protected]>
Cc: Ananth N Mavinakayanahalli <[email protected]>
Cc: Christoph Hellwig <[email protected]>
Cc: Frank Ch. Eigler <[email protected]>
Cc: Jason Baron <[email protected]>
Cc: K.Prasad <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Srikar Dronamraju <[email protected]>
LKML-Reference: <20091104001221.3454.52030.stgit@harusame>
Signed-off-by: Ingo Molnar <[email protected]>
---
tools/perf/builtin-probe.c | 20 +++++++++++++-------
tools/perf/util/probe-finder.c | 2 +-
2 files changed, 14 insertions(+), 8 deletions(-)

diff --git a/tools/perf/builtin-probe.c b/tools/perf/builtin-probe.c
index 8124523..65bcaed 100644
--- a/tools/perf/builtin-probe.c
+++ b/tools/perf/builtin-probe.c
@@ -294,10 +294,11 @@ static int write_new_event(int fd, const char *buf)
{
int ret;

- printf("Adding new event: %s\n", buf);
ret = write(fd, buf, strlen(buf));
if (ret <= 0)
- die("failed to create event.");
+ die("Failed to create event.");
+ else
+ printf("Added new event: %s\n", buf);

return ret;
}
@@ -310,7 +311,7 @@ static int synthesize_probe_event(struct probe_point *pp)
int i, len, ret;
pp->probes[0] = buf = (char *)calloc(MAX_CMDLEN, sizeof(char));
if (!buf)
- die("calloc");
+ die("Failed to allocate memory by calloc.");
ret = snprintf(buf, MAX_CMDLEN, "%s+%d", pp->function, pp->offset);
if (ret <= 0 || ret >= MAX_CMDLEN)
goto error;
@@ -363,7 +364,7 @@ int cmd_probe(int argc, const char **argv, const char *prefix __used)
if (ret == -E2BIG)
semantic_error("probe point is too long.");
else if (ret < 0)
- die("snprintf");
+ die("Failed to synthesize a probe point.");
}

#ifndef NO_LIBDWARF
@@ -375,7 +376,7 @@ int cmd_probe(int argc, const char **argv, const char *prefix __used)
else
fd = open_default_vmlinux();
if (fd < 0)
- die("vmlinux/module file open");
+ die("Could not open vmlinux/module file.");

/* Searching probe points */
for (j = 0; j < session.nr_probe; j++) {
@@ -396,8 +397,13 @@ setup_probes:
/* Settng up probe points */
snprintf(buf, MAX_CMDLEN, "%s/../kprobe_events", debugfs_path);
fd = open(buf, O_WRONLY, O_APPEND);
- if (fd < 0)
- die("kprobe_events open");
+ if (fd < 0) {
+ if (errno == ENOENT)
+ die("kprobe_events file does not exist - please rebuild with CONFIG_KPROBE_TRACER.");
+ else
+ die("Could not open kprobe_events file: %s",
+ strerror(errno));
+ }
for (j = 0; j < session.nr_probe; j++) {
pp = &session.probes[j];
if (pp->found == 1) {
diff --git a/tools/perf/util/probe-finder.c b/tools/perf/util/probe-finder.c
index db96186..35d5a69 100644
--- a/tools/perf/util/probe-finder.c
+++ b/tools/perf/util/probe-finder.c
@@ -688,7 +688,7 @@ int find_probepoint(int fd, struct probe_point *pp)

ret = dwarf_init(fd, DW_DLC_READ, 0, 0, &__dw_debug, &__dw_error);
if (ret != DW_DLV_OK)
- die("Failed to call dwarf_init(). Maybe, not a dwarf file.\n");
+ die("No dwarf info found in the vmlinux - please rebuild with CONFIG_DEBUG_INFO.\n");

pp->found = 0;
while (++cu_number) {

2009-11-04 15:26:21

by Masami Hiramatsu

[permalink] [raw]
Subject: [tip:perf/probes] perf/probes: Fall back to non-dwarf if possible

Commit-ID: a225a1d911f0e434dc0407df29fd08e4388f3fa4
Gitweb: http://git.kernel.org/tip/a225a1d911f0e434dc0407df29fd08e4388f3fa4
Author: Masami Hiramatsu <[email protected]>
AuthorDate: Tue, 3 Nov 2009 19:12:30 -0500
Committer: Ingo Molnar <[email protected]>
CommitDate: Wed, 4 Nov 2009 13:02:47 +0100

perf/probes: Fall back to non-dwarf if possible

Fall back to non-dwarf probe point if the probe definition may
not need dwarf analysis, when perf can't find vmlinux/debuginfo.
This might skip some inlined code of target function.

Signed-off-by: Masami Hiramatsu <[email protected]>
Acked-by: Frederic Weisbecker <[email protected]>
Cc: Steven Rostedt <[email protected]>
Cc: Jim Keniston <[email protected]>
Cc: Ananth N Mavinakayanahalli <[email protected]>
Cc: Christoph Hellwig <[email protected]>
Cc: Frank Ch. Eigler <[email protected]>
Cc: Jason Baron <[email protected]>
Cc: K.Prasad <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Srikar Dronamraju <[email protected]>
LKML-Reference: <20091104001229.3454.63987.stgit@harusame>
Signed-off-by: Ingo Molnar <[email protected]>
---
tools/perf/builtin-probe.c | 64 ++++++++++++++++++++++-----------------
tools/perf/util/probe-finder.c | 6 ++-
2 files changed, 40 insertions(+), 30 deletions(-)

diff --git a/tools/perf/builtin-probe.c b/tools/perf/builtin-probe.c
index 65bcaed..d111a93 100644
--- a/tools/perf/builtin-probe.c
+++ b/tools/perf/builtin-probe.c
@@ -189,7 +189,7 @@ static void parse_probe_event(const char *str)
/* Parse probe point */
parse_probe_point(argv[0], pp);
free(argv[0]);
- if (pp->file)
+ if (pp->file || pp->line)
session.need_dwarf = 1;

/* Copy arguments */
@@ -347,36 +347,24 @@ int cmd_probe(int argc, const char **argv, const char *prefix __used)
if (session.nr_probe == 0)
usage_with_options(probe_usage, options);

-#ifdef NO_LIBDWARF
if (session.need_dwarf)
- semantic_error("Dwarf-analysis is not supported");
-#endif
-
- /* Synthesize probes without dwarf */
- for (j = 0; j < session.nr_probe; j++) {
-#ifndef NO_LIBDWARF
- if (!session.probes[j].retprobe) {
- session.need_dwarf = 1;
- continue;
- }
-#endif
- ret = synthesize_probe_event(&session.probes[j]);
- if (ret == -E2BIG)
- semantic_error("probe point is too long.");
- else if (ret < 0)
- die("Failed to synthesize a probe point.");
- }
-
-#ifndef NO_LIBDWARF
- if (!session.need_dwarf)
- goto setup_probes;
+#ifdef NO_LIBDWARF
+ semantic_error("Debuginfo-analysis is not supported");
+#else /* !NO_LIBDWARF */
+ pr_info("Some probes require debuginfo.\n");

if (session.vmlinux)
fd = open(session.vmlinux, O_RDONLY);
else
fd = open_default_vmlinux();
- if (fd < 0)
- die("Could not open vmlinux/module file.");
+ if (fd < 0) {
+ if (session.need_dwarf)
+ die("Could not open vmlinux/module file.");
+
+ pr_warning("Could not open vmlinux/module file."
+ " Try to use symbols.\n");
+ goto end_dwarf;
+ }

/* Searching probe points */
for (j = 0; j < session.nr_probe; j++) {
@@ -386,14 +374,34 @@ int cmd_probe(int argc, const char **argv, const char *prefix __used)

lseek(fd, SEEK_SET, 0);
ret = find_probepoint(fd, pp);
- if (ret <= 0)
- die("No probe point found.\n");
+ if (ret < 0) {
+ if (session.need_dwarf)
+ die("Could not analyze debuginfo.");
+
+ pr_warning("An error occurred in debuginfo analysis. Try to use symbols.\n");
+ break;
+ }
+ if (ret == 0) /* No error but failed to find probe point. */
+ die("No probe point found.");
}
close(fd);

-setup_probes:
+end_dwarf:
#endif /* !NO_LIBDWARF */

+ /* Synthesize probes without dwarf */
+ for (j = 0; j < session.nr_probe; j++) {
+ pp = &session.probes[j];
+ if (pp->found) /* This probe is already found. */
+ continue;
+
+ ret = synthesize_probe_event(pp);
+ if (ret == -E2BIG)
+ semantic_error("probe point is too long.");
+ else if (ret < 0)
+ die("Failed to synthesize a probe point.");
+ }
+
/* Settng up probe points */
snprintf(buf, MAX_CMDLEN, "%s/../kprobe_events", debugfs_path);
fd = open(buf, O_WRONLY, O_APPEND);
diff --git a/tools/perf/util/probe-finder.c b/tools/perf/util/probe-finder.c
index 35d5a69..293cdfc 100644
--- a/tools/perf/util/probe-finder.c
+++ b/tools/perf/util/probe-finder.c
@@ -687,8 +687,10 @@ int find_probepoint(int fd, struct probe_point *pp)
struct probe_finder pf = {.pp = pp};

ret = dwarf_init(fd, DW_DLC_READ, 0, 0, &__dw_debug, &__dw_error);
- if (ret != DW_DLV_OK)
- die("No dwarf info found in the vmlinux - please rebuild with CONFIG_DEBUG_INFO.\n");
+ if (ret != DW_DLV_OK) {
+ pr_warning("No dwarf info found in the vmlinux - please rebuild with CONFIG_DEBUG_INFO.\n");
+ return -ENOENT;
+ }

pp->found = 0;
while (++cu_number) {

2009-11-04 15:26:36

by Masami Hiramatsu

[permalink] [raw]
Subject: [tip:perf/probes] perf/probes: Rename perf probe events group name

Commit-ID: 91365bbe4f8c39a821f390f785d606304d6dee3c
Gitweb: http://git.kernel.org/tip/91365bbe4f8c39a821f390f785d606304d6dee3c
Author: Masami Hiramatsu <[email protected]>
AuthorDate: Tue, 3 Nov 2009 19:12:38 -0500
Committer: Ingo Molnar <[email protected]>
CommitDate: Wed, 4 Nov 2009 13:02:47 +0100

perf/probes: Rename perf probe events group name

Rename the group name of perf probe events to 'probe'.

Signed-off-by: Masami Hiramatsu <[email protected]>
Acked-by: Frederic Weisbecker <[email protected]>
Cc: Steven Rostedt <[email protected]>
Cc: Jim Keniston <[email protected]>
Cc: Ananth N Mavinakayanahalli <[email protected]>
Cc: Christoph Hellwig <[email protected]>
Cc: Frank Ch. Eigler <[email protected]>
Cc: Jason Baron <[email protected]>
Cc: K.Prasad <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Srikar Dronamraju <[email protected]>
LKML-Reference: <20091104001238.3454.70508.stgit@harusame>
Signed-off-by: Ingo Molnar <[email protected]>
---
tools/perf/builtin-probe.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/tools/perf/builtin-probe.c b/tools/perf/builtin-probe.c
index d111a93..d78a3d9 100644
--- a/tools/perf/builtin-probe.c
+++ b/tools/perf/builtin-probe.c
@@ -52,7 +52,7 @@ const char *default_search_path[NR_SEARCH_PATH] = {
#define MAX_PATH_LEN 256
#define MAX_PROBES 128
#define MAX_PROBE_ARGS 128
-#define PERFPROBE_GROUP "perfprobe"
+#define PERFPROBE_GROUP "probe"

/* Session management structure */
static struct {

2009-11-04 15:26:59

by Masami Hiramatsu

[permalink] [raw]
Subject: [tip:perf/probes] tracing/kprobes: Rename Kprobe-tracer to kprobe-event

Commit-ID: 77b44d1b7c28360910cdbd427fb62d485c08674c
Gitweb: http://git.kernel.org/tip/77b44d1b7c28360910cdbd427fb62d485c08674c
Author: Masami Hiramatsu <[email protected]>
AuthorDate: Tue, 3 Nov 2009 19:12:47 -0500
Committer: Ingo Molnar <[email protected]>
CommitDate: Wed, 4 Nov 2009 13:02:48 +0100

tracing/kprobes: Rename Kprobe-tracer to kprobe-event

Rename Kprobes-based event tracer to kprobes-based tracing event
(kprobe-event), since it is not a tracer but an extensible
tracing event interface.

This also changes CONFIG_KPROBE_TRACER to CONFIG_KPROBE_EVENT
and sets it y by default.

Signed-off-by: Masami Hiramatsu <[email protected]>
Acked-by: Frederic Weisbecker <[email protected]>
Cc: Steven Rostedt <[email protected]>
Cc: Jim Keniston <[email protected]>
Cc: Ananth N Mavinakayanahalli <[email protected]>
Cc: Christoph Hellwig <[email protected]>
Cc: Frank Ch. Eigler <[email protected]>
Cc: Jason Baron <[email protected]>
Cc: K.Prasad <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Srikar Dronamraju <[email protected]>
LKML-Reference: <20091104001247.3454.14131.stgit@harusame>
Signed-off-by: Ingo Molnar <[email protected]>
---
Documentation/trace/kprobetrace.txt | 34 ++++++++++++++++------------------
kernel/trace/Kconfig | 19 ++++++++++++-------
kernel/trace/Makefile | 2 +-
kernel/trace/trace_kprobe.c | 6 ++----
4 files changed, 31 insertions(+), 30 deletions(-)

diff --git a/Documentation/trace/kprobetrace.txt b/Documentation/trace/kprobetrace.txt
index 1541524..47aabee 100644
--- a/Documentation/trace/kprobetrace.txt
+++ b/Documentation/trace/kprobetrace.txt
@@ -1,26 +1,23 @@
- Kprobe-based Event Tracer
- =========================
+ Kprobe-based Event Tracing
+ ==========================

Documentation is written by Masami Hiramatsu


Overview
--------
-This tracer is similar to the events tracer which is based on Tracepoint
-infrastructure. Instead of Tracepoint, this tracer is based on kprobes(kprobe
-and kretprobe). It probes anywhere where kprobes can probe(this means, all
-functions body except for __kprobes functions).
+These events are similar to tracepoint based events. Instead of Tracepoint,
+this is based on kprobes (kprobe and kretprobe). So it can probe wherever
+kprobes can probe (this means, all functions body except for __kprobes
+functions). Unlike the Tracepoint based event, this can be added and removed
+dynamically, on the fly.

-Unlike the function tracer, this tracer can probe instructions inside of
-kernel functions. It allows you to check which instruction has been executed.
+To enable this feature, build your kernel with CONFIG_KPROBE_TRACING=y.

-Unlike the Tracepoint based events tracer, this tracer can add and remove
-probe points on the fly.
-
-Similar to the events tracer, this tracer doesn't need to be activated via
-current_tracer, instead of that, just set probe points via
-/sys/kernel/debug/tracing/kprobe_events. And you can set filters on each
-probe events via /sys/kernel/debug/tracing/events/kprobes/<EVENT>/filter.
+Similar to the events tracer, this doesn't need to be activated via
+current_tracer. Instead of that, add probe points via
+/sys/kernel/debug/tracing/kprobe_events, and enable it via
+/sys/kernel/debug/tracing/events/kprobes/<EVENT>/enabled.


Synopsis of kprobe_events
@@ -55,9 +52,9 @@ Per-Probe Event Filtering
-------------------------
Per-probe event filtering feature allows you to set different filter on each
probe and gives you what arguments will be shown in trace buffer. If an event
-name is specified right after 'p:' or 'r:' in kprobe_events, the tracer adds
-an event under tracing/events/kprobes/<EVENT>, at the directory you can see
-'id', 'enabled', 'format' and 'filter'.
+name is specified right after 'p:' or 'r:' in kprobe_events, it adds an event
+under tracing/events/kprobes/<EVENT>, at the directory you can see 'id',
+'enabled', 'format' and 'filter'.

enabled:
You can enable/disable the probe by writing 1 or 0 on it.
@@ -71,6 +68,7 @@ filter:
id:
This shows the id of this probe event.

+
Event Profiling
---------------
You can check the total number of probe hits and probe miss-hits via
diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index 15372a9..f056716 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -428,17 +428,22 @@ config BLK_DEV_IO_TRACE

If unsure, say N.

-config KPROBE_TRACER
+config KPROBE_EVENT
depends on KPROBES
depends on X86
- bool "Trace kprobes"
+ bool "Enable kprobes-based dynamic events"
select TRACING
- select GENERIC_TRACER
+ default y
help
- This tracer probes everywhere where kprobes can probe it, and
- records various registers and memories specified by user.
- This also allows you to trace kprobe probe points as a dynamic
- defined events. It provides per-probe event filtering interface.
+ This allows the user to add tracing events (similar to tracepoints) on the fly
+ via the ftrace interface. See Documentation/trace/kprobetrace.txt
+ for more details.
+
+ Those events can be inserted wherever kprobes can probe, and record
+ various register and memory values.
+
+ This option is also required by perf-probe subcommand of perf tools. If
+ you want to use perf tools, this option is strongly recommended.

config DYNAMIC_FTRACE
bool "enable/disable ftrace tracepoints dynamically"
diff --git a/kernel/trace/Makefile b/kernel/trace/Makefile
index c8cb75d..edc3a3c 100644
--- a/kernel/trace/Makefile
+++ b/kernel/trace/Makefile
@@ -53,7 +53,7 @@ obj-$(CONFIG_EVENT_TRACING) += trace_export.o
obj-$(CONFIG_FTRACE_SYSCALLS) += trace_syscalls.o
obj-$(CONFIG_EVENT_PROFILE) += trace_event_profile.o
obj-$(CONFIG_EVENT_TRACING) += trace_events_filter.o
-obj-$(CONFIG_KPROBE_TRACER) += trace_kprobe.o
+obj-$(CONFIG_KPROBE_EVENT) += trace_kprobe.o
obj-$(CONFIG_EVENT_TRACING) += power-traces.o

libftrace-y := ftrace.o
diff --git a/kernel/trace/trace_kprobe.c b/kernel/trace/trace_kprobe.c
index a86c3ac..cf17a66 100644
--- a/kernel/trace/trace_kprobe.c
+++ b/kernel/trace/trace_kprobe.c
@@ -1,5 +1,5 @@
/*
- * kprobe based kernel tracer
+ * Kprobes-based tracing events
*
* Created by Masami Hiramatsu <[email protected]>
*
@@ -57,8 +57,6 @@ const char *reserved_field_names[] = {
FIELD_STRING_FUNC,
};

-/* currently, trace_kprobe only supports X86. */
-
struct fetch_func {
unsigned long (*func)(struct pt_regs *, void *);
void *data;
@@ -191,7 +189,7 @@ static __kprobes void free_indirect_fetch_data(struct indirect_fetch_data *data)
}

/**
- * Kprobe tracer core functions
+ * Kprobe event core functions
*/

struct probe_arg {