2008-02-26 22:38:54

by David Wilder

[permalink] [raw]
Subject: [PATCH 0/3] A kernel tracing interface

These patches provide a kernel tracing interface called "trace".

ChangeLog:
-Updated to 2.6.25-rc2-mm1
-Removed sem_watch example, to hard to maintain.
-Added a new example that demonstrates per-cpu continuous tracing
of data generated using marker probes.
-Removed inline from relay patch.
-Moved examples into /sample directory.

The motivation for "trace" is to:
- Provide a simple set of tracing primitives that will utilize the high-
performance and low-overhead of relayfs for passing traces data from
kernel to user space.
- Provide a common user interface for managing kernel traces.
- Allow for binary as well as ascii trace data.
- Incorporate features from the systemtap runtime that are
useful to others.

Patches are built against 2.6.25-rc2-mm1.
Tested on: 2.6.25-rc3

Summary of patches:
[patch 1/3] Relay Reset Consumed
[patch 2/3] Trace code and documentation
[patch 3/3] Trace sample

Note: Patch 1/3 provides a service used by patch 2/3
thus must be applied first or at the same time.

Signed-off-by: David Wilder <[email protected]>


2008-02-28 10:06:29

by Andrew Morton

[permalink] [raw]
Subject: Re: [PATCH 0/3] A kernel tracing interface

On Tue, 26 Feb 2008 14:38:37 -0800 "David J. Wilder" <[email protected]> wrote:

> These patches provide a kernel tracing interface called "trace".
>
> ChangeLog:
> -Updated to 2.6.25-rc2-mm1
> -Removed sem_watch example, to hard to maintain.
> -Added a new example that demonstrates per-cpu continuous tracing
> of data generated using marker probes.
> -Removed inline from relay patch.
> -Moved examples into /sample directory.
>
> The motivation for "trace" is to:
> - Provide a simple set of tracing primitives that will utilize the high-
> performance and low-overhead of relayfs for passing traces data from
> kernel to user space.
> - Provide a common user interface for managing kernel traces.
> - Allow for binary as well as ascii trace data.
> - Incorporate features from the systemtap runtime that are
> useful to others.

So... what's the story on this versus lttng?

Is there some userspace code available for people to test this? Maybe it's
mentioned and I missed it.

It'd be interesting to see a writeup of the proposed kernel<->userspace
interfaces. I see a description of kernel-internal interfaces, but how do
users use it?

I see it uses things from blktrace. Can blktrace be switched over to being
a client of this code?

2008-02-28 19:41:17

by David Wilder

[permalink] [raw]
Subject: Re: [PATCH 0/3] A kernel tracing interface

Andrew Morton wrote:
> On Tue, 26 Feb 2008 14:38:37 -0800 "David J. Wilder" <[email protected]> wrote:
>
>> These patches provide a kernel tracing interface called "trace".
>>
>> ChangeLog:
>> -Updated to 2.6.25-rc2-mm1
>> -Removed sem_watch example, to hard to maintain.
>> -Added a new example that demonstrates per-cpu continuous tracing
>> of data generated using marker probes.
>> -Removed inline from relay patch.
>> -Moved examples into /sample directory.
>>
>> The motivation for "trace" is to:
>> - Provide a simple set of tracing primitives that will utilize the high-
>> performance and low-overhead of relayfs for passing traces data from
>> kernel to user space.
>> - Provide a common user interface for managing kernel traces.
>> - Allow for binary as well as ascii trace data.
>> - Incorporate features from the systemtap runtime that are
>> useful to others.
>
> So... what's the story on this versus lttng?

Trace is simpler to use and a smaller bit of code than lttng.
It may not have all the features of lttng but it is good starting point
that can be expanded upon. I have several users of trace that just
wanted a simple way to get data out of the kernel without the overhead
of lttng trace is working well for them. Trace also works well with
markers and kprobes. For example systemtap is is using the basic trace
code to gather data from kprobes and markers.
>
> Is there some userspace code available for people to test this? Maybe it's
> mentioned and I missed it.

I purposely designed trace so no special user code is required to gather
trace data. Standard user utilities like cat, grep, sort and more is all
that is needed. I show an example in /samples/trace/fork_trace.c.

However since trace uses relay as its transport any user interfaces
provided by relay can be used to read trace data. (documented in
/Documentation/relay.txt)

>
> It'd be interesting to see a writeup of the proposed kernel<->userspace
> interfaces. I see a description of kernel-internal interfaces, but how do
> users use it?

I will add some text in the documentation.
>
> I see it uses things from blktrace. Can blktrace be switched over to being
> a client of this code?

yep.

>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>

2008-02-29 05:36:19

by K.Prasad

[permalink] [raw]
Subject: Re: [PATCH 0/3] A kernel tracing interface

On Thu, Feb 28, 2008 at 11:40:21AM -0800, David Wilder wrote:
> Andrew Morton wrote:
> >On Tue, 26 Feb 2008 14:38:37 -0800 "David J. Wilder" <[email protected]>
> >wrote:
> >
> >>These patches provide a kernel tracing interface called "trace".
> >>
> >>ChangeLog:
> >>-Updated to 2.6.25-rc2-mm1
> >>-Removed sem_watch example, to hard to maintain.
> >>-Added a new example that demonstrates per-cpu continuous tracing
> >> of data generated using marker probes.
> >>-Removed inline from relay patch.
> >>-Moved examples into /sample directory.
> >>
> >>The motivation for "trace" is to:
> >>- Provide a simple set of tracing primitives that will utilize the high-
> >> performance and low-overhead of relayfs for passing traces data from
> >> kernel to user space.
> >>- Provide a common user interface for managing kernel traces.
> >>- Allow for binary as well as ascii trace data.
> >>- Incorporate features from the systemtap runtime that are
> >> useful to others.
> >
> >So... what's the story on this versus lttng?
>
> Trace is simpler to use and a smaller bit of code than lttng.
> It may not have all the features of lttng but it is good starting point
> that can be expanded upon. I have several users of trace that just
> wanted a simple way to get data out of the kernel without the overhead
> of lttng trace is working well for them. Trace also works well with
> markers and kprobes. For example systemtap is is using the basic trace
> code to gather data from kprobes and markers.
> >
> >Is there some userspace code available for people to test this? Maybe it's
> >mentioned and I missed it.
>
> I purposely designed trace so no special user code is required to gather
> trace data. Standard user utilities like cat, grep, sort and more is all
> that is needed. I show an example in /samples/trace/fork_trace.c.
>
> However since trace uses relay as its transport any user interfaces
> provided by relay can be used to read trace data. (documented in
> /Documentation/relay.txt)
>
> >
> >It'd be interesting to see a writeup of the proposed kernel<->userspace
> >interfaces. I see a description of kernel-internal interfaces, but how do
> >users use it?
>
> I will add some text in the documentation.
> >
> >I see it uses things from blktrace. Can blktrace be switched over to being
> >a client of this code?
>
> yep.
>
Dave,
I might also want to add to the discussion by stating that you
will find a user for the 'trace' infrastructure in the marker handler
code of the proposed markers for futex subsystem. I hope to post the
patches soon to the community.

Thanks,
K.Prasad

2008-02-29 12:08:55

by K.Prasad

[permalink] [raw]
Subject: Re: [PATCH 0/3] A kernel tracing interface

On Fri, Feb 29, 2008 at 11:06:05AM +0530, K. Prasad wrote:
> On Thu, Feb 28, 2008 at 11:40:21AM -0800, David Wilder wrote:
> > Andrew Morton wrote:
> > >On Tue, 26 Feb 2008 14:38:37 -0800 "David J. Wilder" <[email protected]>
> > >wrote:
> > >
> > >>These patches provide a kernel tracing interface called "trace".
> > >>
> > >>ChangeLog:
> > >>-Updated to 2.6.25-rc2-mm1
> > >>-Removed sem_watch example, to hard to maintain.
> > >>-Added a new example that demonstrates per-cpu continuous tracing
> > >> of data generated using marker probes.
> > >>-Removed inline from relay patch.
> > >>-Moved examples into /sample directory.
> > >>
> > >>The motivation for "trace" is to:
> > >>- Provide a simple set of tracing primitives that will utilize the high-
> > >> performance and low-overhead of relayfs for passing traces data from
> > >> kernel to user space.
> > >>- Provide a common user interface for managing kernel traces.
> > >>- Allow for binary as well as ascii trace data.
> > >>- Incorporate features from the systemtap runtime that are
> > >> useful to others.
> > >
> > >So... what's the story on this versus lttng?
> >
> > Trace is simpler to use and a smaller bit of code than lttng.
> > It may not have all the features of lttng but it is good starting point
> > that can be expanded upon. I have several users of trace that just
> > wanted a simple way to get data out of the kernel without the overhead
> > of lttng trace is working well for them. Trace also works well with
> > markers and kprobes. For example systemtap is is using the basic trace
> > code to gather data from kprobes and markers.
> > >
> > >Is there some userspace code available for people to test this? Maybe it's
> > >mentioned and I missed it.
> >
> > I purposely designed trace so no special user code is required to gather
> > trace data. Standard user utilities like cat, grep, sort and more is all
> > that is needed. I show an example in /samples/trace/fork_trace.c.
> >
> > However since trace uses relay as its transport any user interfaces
> > provided by relay can be used to read trace data. (documented in
> > /Documentation/relay.txt)
> >
> > >
> > >It'd be interesting to see a writeup of the proposed kernel<->userspace
> > >interfaces. I see a description of kernel-internal interfaces, but how do
> > >users use it?
> >
> > I will add some text in the documentation.
> > >
> > >I see it uses things from blktrace. Can blktrace be switched over to being
> > >a client of this code?
> >
> > yep.
> >
> Dave,
> I might also want to add to the discussion by stating that you
> will find a user for the 'trace' infrastructure in the marker handler
> code of the proposed markers for futex subsystem. I hope to post the
> patches soon to the community.
>
Also to let you know that I tested the patches (sent against
2.6.25-rc2-mm1) on a x86 machine and they've been working fine....

Thanks,
K.Prasad