2008-08-03 07:50:31

by Ming Lei

[permalink] [raw]
Subject: [RFC] ftrace: support tracing functions in one module

Hi,

IMO, ftrace is a very good tools, which can monitor almost all
functions calling
in the running kernel. The traced result is very complete and intact.
But it seems
too large to grasp the interested content. For example, one may only
have interest in
functions calling in usbcore.ko, but he must trace all the functions
calling in the
kernel, so the tracing result is too large to use it.

Could you add the support of tracing functions in one module only to ftrace?

Thanks,


2008-08-03 12:21:48

by Frank Ch. Eigler

[permalink] [raw]
Subject: Re: [RFC] ftrace: support tracing functions in one module

"Ming Lei" <[email protected]> writes:

> [...] For example, one may only have interest in functions calling
> in usbcore.ko, but he must trace all the functions calling in the
> kernel, so the tracing result is too large to use it. [...]

Until ftrace gains this ability, you could use systemtap:

# cat callret.stp
probe $1.call { printf ("%s <- %s\n", thread_indent(1), probefunc()) }
probe $1.return { printf ("%s -> %s\n", thread_indent(-1), probefunc()) }
# stap callret.stp 'module("usbcore").function("*")'
[...]
^C
#

(One can also add conditions based on function parameters, processes,
whatever; soon it'll be easy to trace all function parameters.)


- FChE

2008-08-03 14:45:24

by Steven Rostedt

[permalink] [raw]
Subject: Re: [RFC] ftrace: support tracing functions in one module


On Sun, 3 Aug 2008, Ming Lei wrote:

> Hi,
>
> IMO, ftrace is a very good tools, which can monitor almost all
> functions calling
> in the running kernel. The traced result is very complete and intact.
> But it seems
> too large to grasp the interested content. For example, one may only
> have interest in
> functions calling in usbcore.ko, but he must trace all the functions
> calling in the
> kernel, so the tracing result is too large to use it.
>
> Could you add the support of tracing functions in one module only to ftrace?

Look at the set_ftrace_filter in ftrace.txt. You can pick and choose which
functions to trace. All the functions that can be traced is in
available_filter_functions in the debugfs/tracing directory.

Note, the functions must execute at least once before they can be
recorded.

-- Steve

2008-08-03 15:16:52

by Abhishek Sagar

[permalink] [raw]
Subject: Re: [RFC] ftrace: support tracing functions in one module

On Sun, Aug 3, 2008 at 1:20 PM, Ming Lei <[email protected]> wrote:
> Could you add the support of tracing functions in one module only to ftrace?

You can do that at compile time only by selective compilation of
files/modules which need to be traced with the -pg CFLAG. For
instance, the inlined patch enables tracing of all functions in
softirq.o only.

Regards,
Abhishek Sagar

--
## For testing only

diff --git a/Makefile b/Makefile
index 40f2481..09d216b 100644
--- a/Makefile
+++ b/Makefile
@@ -532,10 +532,6 @@ KBUILD_CFLAGS += -g
KBUILD_AFLAGS += -gdwarf-2
endif

-ifdef CONFIG_FTRACE
-KBUILD_CFLAGS += -pg
-endif
-
# We trigger additional mismatches with less inlining
ifdef CONFIG_DEBUG_SECTION_MISMATCH
KBUILD_CFLAGS += $(call cc-option, -fno-inline-functions-called-once)
diff --git a/kernel/Makefile b/kernel/Makefile
index 82f1f3f..c099be3 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -14,14 +14,8 @@ obj-y = sched.o fork.o exec_domain.o panic.o printk.o \
CFLAGS_REMOVE_sched.o = -mno-spe

ifdef CONFIG_FTRACE
-# Do not trace debug files and internal ftrace files
-CFLAGS_REMOVE_lockdep.o = -pg
-CFLAGS_REMOVE_lockdep_proc.o = -pg
-CFLAGS_REMOVE_mutex-debug.o = -pg
-CFLAGS_REMOVE_rtmutex-debug.o = -pg
-CFLAGS_REMOVE_cgroup-debug.o = -pg
-CFLAGS_REMOVE_sched_clock.o = -pg
-CFLAGS_REMOVE_sched.o = -mno-spe -pg
+CFLAGS_softirq.o = -pg
+CFLAGS_REMOVE_sched.o = -mno-spe
endif

obj-$(CONFIG_PROFILING) += profile.o

2008-08-04 02:11:28

by Ming Lei

[permalink] [raw]
Subject: Re: [RFC] ftrace: support tracing functions in one module

2008/8/3 Steven Rostedt <[email protected]>:
>
> On Sun, 3 Aug 2008, Ming Lei wrote:
>
>> Hi,
>>
>> IMO, ftrace is a very good tools, which can monitor almost all
>> functions calling
>> in the running kernel. The traced result is very complete and intact.
>> But it seems
>> too large to grasp the interested content. For example, one may only
>> have interest in
>> functions calling in usbcore.ko, but he must trace all the functions
>> calling in the
>> kernel, so the tracing result is too large to use it.
>>
>> Could you add the support of tracing functions in one module only to ftrace?
>
> Look at the set_ftrace_filter in ftrace.txt. You can pick and choose which
> functions to trace. All the functions that can be traced is in

It seems not ver easy to opearte to trace all functions in a module. You need to
write all function names to set_ftrace_filter. Also some functions have
same names in kernel and modules. This can lead to some messed trace result.

Do you have the plan to support tracing functions in one module or in one kernel
address range?

> available_filter_functions in the debugfs/tracing directory.
>
> Note, the functions must execute at least once before they can be
> recorded.
>
> -- Steve
>
>



--
Lei Ming

2008-08-04 16:18:25

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: [RFC] ftrace: support tracing functions in one module

Em Mon, Aug 04, 2008 at 10:11:17AM +0800, Ming Lei escreveu:
> 2008/8/3 Steven Rostedt <[email protected]>:
> >
> > On Sun, 3 Aug 2008, Ming Lei wrote:
> >
> >> Hi,
> >>
> >> IMO, ftrace is a very good tools, which can monitor almost all
> >> functions calling
> >> in the running kernel. The traced result is very complete and intact.
> >> But it seems
> >> too large to grasp the interested content. For example, one may only
> >> have interest in
> >> functions calling in usbcore.ko, but he must trace all the functions
> >> calling in the
> >> kernel, so the tracing result is too large to use it.
> >>
> >> Could you add the support of tracing functions in one module only to ftrace?
> >
> > Look at the set_ftrace_filter in ftrace.txt. You can pick and choose which
> > functions to trace. All the functions that can be traced is in
>
> It seems not ver easy to opearte to trace all functions in a module. You need to
> write all function names to set_ftrace_filter. Also some functions have
> same names in kernel and modules. This can lead to some messed trace result.
>
> Do you have the plan to support tracing functions in one module or in one kernel
> address range?

What about using:

[acme@doppio pahole]$ nm --defined-only /usr/lib/debug/lib/modules/2.6.24.7-75.el5rt/kernel/drivers/net/tg3.ko.debug | grep ' t ' | cut -d' ' -f3 | head
__tg3_set_coalesce
__tg3_set_mac_addr
__tg3_set_rx_mode
_tw32_flush
tg3_5700_link_polarity
tg3_abort_hw
tg3_alloc_rx_skb
tg3_ape_driver_state_change
tg3_bmcr_reset
tg3_change_mtu
[acme@doppio pahole]$

Feed this to /sys/kernel/debug/tracing/set_ftrace_filter and you should
be set, no?

Ok, you need to have the kernel-debuginfo package installed, and I guess
we can get away with that, but for now, isn't that enough?

- Arnaldo

2008-08-04 16:28:26

by Steven Rostedt

[permalink] [raw]
Subject: Re: [RFC] ftrace: support tracing functions in one module


On Mon, 4 Aug 2008, Arnaldo Carvalho de Melo wrote:
> >
> > Do you have the plan to support tracing functions in one module or in one kernel
> > address range?
>
> What about using:
>
> [acme@doppio pahole]$ nm --defined-only /usr/lib/debug/lib/modules/2.6.24.7-75.el5rt/kernel/drivers/net/tg3.ko.debug | grep ' t ' | cut -d' ' -f3 | head
> __tg3_set_coalesce
> __tg3_set_mac_addr
> __tg3_set_rx_mode
> _tw32_flush
> tg3_5700_link_polarity
> tg3_abort_hw
> tg3_alloc_rx_skb
> tg3_ape_driver_state_change
> tg3_bmcr_reset
> tg3_change_mtu
> [acme@doppio pahole]$
>
> Feed this to /sys/kernel/debug/tracing/set_ftrace_filter and you should
> be set, no?
>
> Ok, you need to have the kernel-debuginfo package installed, and I guess
> we can get away with that, but for now, isn't that enough?


Or simply (again for tg3 :


# awk '/\[tg3\]/ { print $3 }' /proc/kallsyms \
> /debug/tracing/set_ftrace_filter

-- Steve

2008-08-04 17:21:47

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: [RFC] ftrace: support tracing functions in one module

Em Mon, Aug 04, 2008 at 12:28:12PM -0400, Steven Rostedt escreveu:
>
> On Mon, 4 Aug 2008, Arnaldo Carvalho de Melo wrote:
> > >
> > > Do you have the plan to support tracing functions in one module or in one kernel
> > > address range?
> >
> > What about using:
> >
> > [acme@doppio pahole]$ nm --defined-only /usr/lib/debug/lib/modules/2.6.24.7-75.el5rt/kernel/drivers/net/tg3.ko.debug | grep ' t ' | cut -d' ' -f3 | head
> > __tg3_set_coalesce
> > __tg3_set_mac_addr
> > __tg3_set_rx_mode
> > _tw32_flush
> > tg3_5700_link_polarity
> > tg3_abort_hw
> > tg3_alloc_rx_skb
> > tg3_ape_driver_state_change
> > tg3_bmcr_reset
> > tg3_change_mtu
> > [acme@doppio pahole]$
> >
> > Feed this to /sys/kernel/debug/tracing/set_ftrace_filter and you should
> > be set, no?
> >
> > Ok, you need to have the kernel-debuginfo package installed, and I guess
> > we can get away with that, but for now, isn't that enough?
>
>
> Or simply (again for tg3 :
>
>
> # awk '/\[tg3\]/ { print $3 }' /proc/kallsyms \
> > /debug/tracing/set_ftrace_filter

For the drivers that do proper namespacing, yes, that should be enough
in most cases :)

- Arnaldo