(sorry for repost, I messed up lkml address in the original post, please
reply to this one)
Hi everyone,
Let's see what the current status is and what we still have to focus on
before submission. First I'd like to say a big thanks to Zhaolei from
Fujitsu who did a great job for the debugfs-based LTTng control we now
have.
- trace_clock()
- Port to the -tip tree. Submit. Should be ready for inclusion after
the few rounds of review on LKML it had.
Assigned-to: Mathieu Desnoyers <[email protected]>
- LTTng instrumentation
- Port to the -tip tree, submit incrementally.
Assigned-to: Mathieu Desnoyers <[email protected]>
- Add periodical subbuffer flush for streaming
- Will be required by ascii text output.
- Also requires modification to lttd so it can only transmit the
buffer data, without the padding. Some modification to LTTV might be
needed to cope with variable-sized subbuffers (we would need an
index to be able to seek with O(log(num subbuffers)) as we currently
do, or we would have to use sparse files for trace files).
Assigned-to: Fran?ois Prenoveau <[email protected]>
- Marker ID management
Background :
The markers are currently used to identify the numerical event IDs
associated with a named event as well as the event types (format
string) associated with that event. Those two informations are kept in
a table written in the metadata channel. A "channel" could be thought
as being a set of per-cpu buffers, where information transits. This
information would typically be related to a specific tracer. Note,
however, that the information generated by a tracer should always be
parseable generically and be transformable into text output by a
simple parser, because we want to permit analysis across information
logged by the various tracer.
Action item :
The current event IDs are global to the whole kernel. It would be
beneficial to change their scope so they are per-channel instead,
because we can then encode most events in the low-order bits of the
event ID bits, therefore keeping events as small as possible.
- Channel management
Background :
LTTng currently has the following statically declared channels :
cpu (default channel)
metadata (marker name, marker ID, marker format string table)
interrupts (interrupt handlers table, interrupt handler connexion
events)
modules (loaded module table, module load/free events)
network (active interfaces table, interface up/down events)
processes (processes existing at trace start table, scheduling events)
Action item :
But if we want to integrate ftrace easily within this framework, we
would like to offer an API so tracers can dynamically register a new
channel whenever they are loaded.
Note that the marker ID management and channel management are closely
tied together; IDs would be per-channel, and channels would be
registered by tracers.
- trimmed-down lttv for the kernel tree
Need to look at
git://git.kernel.org/pub/scm/linux/kernel/git/sam/test.git#master
tests/ directory, which permits building userspace tools with the
Linux kernel. I recently got the idea of populating debugfs with
userspace tools that would sits in kernel module data. How (in)sane
does this idea look ? That would seems like a rather good solution to
ship the userspace tool with the kernel.
- Port ftrace to LTTng
ftrace could ask LTTng to create channels for its data, and would use
this handle to record into the buffers. Given that LTTng allows
multiple traces to be taken at once, and given LTTng exports all its
trace control primitives as in-kernel API, ftrace could use those to
automatically control tracing behavior from within the kernel wrt the
buffers for which it produces _and_ consumes information.
I think we might need to add the notion of traces for which buffers
are meant to be consumed within the kernel (private) and not exported
to userspace. Or maybe the current reference counting which insure
that at most one reader is connected on a trace buffer, is enough.
- ASCII text output.
- Merge sort done within the kernel. Binary-to-text translation also
done within the kernel.
Assigned-to: Zhaolei <[email protected]>
Assigned-to: Lai Jiangshan <[email protected]>
Mathieu
--
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
On Mon, Nov 24, 2008 at 06:28:42AM -0500, Mathieu Desnoyers wrote:
> - trimmed-down lttv for the kernel tree
> Need to look at
> git://git.kernel.org/pub/scm/linux/kernel/git/sam/test.git#master
> tests/ directory, which permits building userspace tools with the
> Linux kernel. I recently got the idea of populating debugfs with
> userspace tools that would sits in kernel module data. How (in)sane
> does this idea look ? That would seems like a rather good solution to
> ship the userspace tool with the kernel.
Completely insane I'd say. Debugfs is per defintion populated from
kernelspace, and adding another mixed user/kernel populated fs like
devfs is a rather bad idea. I don't quite see any point in shipping
lttv with the kernel tree either, it shouldn't really need knowledge of
the kernel version.
* Christoph Hellwig ([email protected]) wrote:
> On Mon, Nov 24, 2008 at 06:28:42AM -0500, Mathieu Desnoyers wrote:
> > - trimmed-down lttv for the kernel tree
> > Need to look at
> > git://git.kernel.org/pub/scm/linux/kernel/git/sam/test.git#master
> > tests/ directory, which permits building userspace tools with the
> > Linux kernel. I recently got the idea of populating debugfs with
> > userspace tools that would sits in kernel module data. How (in)sane
> > does this idea look ? That would seems like a rather good solution to
> > ship the userspace tool with the kernel.
>
> Completely insane I'd say. Debugfs is per defintion populated from
> kernelspace, and adding another mixed user/kernel populated fs like
> devfs is a rather bad idea. I don't quite see any point in shipping
> lttv with the kernel tree either, it shouldn't really need knowledge of
> the kernel version.
>
The key idea behind this is to answer to Thomas Gleixner concerns, who
supports that a tracer should output data in text-format only so it can
be used with tools kernel developers have on their system, like "cat".
However, getting data out of the kernel efficiently simply cannot be
done with such approach. Therefore, LTTng needs its own userspace tools
to splice the data out of the kernel efficiently. Another tool is used
to pretty-print the binary data into text.
Then the problem becomes : we have to make the userspace tool easy
enough to deploy so even Linus can find and use it. ;)
But indeed, the trace buffers are versioned, so if the format changes
between kernel versions, the userspace tools will detect it and the user
will know it must update its tools. So it's not really a problem there.
The question that prevails is therefore : should we ship userspace
binary with the kernel tree at all ? And if yes, how should the resuting
executables be packaged and deployed ? Should it be installed in the
system along with kernel modules or should it be populated into a
filesystem populated by kernelspace ?
Or is it better to do as we have always done and keep the userspace
tools separated from the kernel tree ?
Mathieu
--
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
On Mon, 24 Nov 2008, Mathieu Desnoyers wrote:
>
> The key idea behind this is to answer to Thomas Gleixner concerns, who
> supports that a tracer should output data in text-format only so it can
> be used with tools kernel developers have on their system, like "cat".
>
> However, getting data out of the kernel efficiently simply cannot be
> done with such approach. Therefore, LTTng needs its own userspace tools
> to splice the data out of the kernel efficiently. Another tool is used
> to pretty-print the binary data into text.
>
> Then the problem becomes : we have to make the userspace tool easy
> enough to deploy so even Linus can find and use it. ;)
>
> But indeed, the trace buffers are versioned, so if the format changes
> between kernel versions, the userspace tools will detect it and the user
> will know it must update its tools. So it's not really a problem there.
>
> The question that prevails is therefore : should we ship userspace
> binary with the kernel tree at all ? And if yes, how should the resuting
> executables be packaged and deployed ? Should it be installed in the
> system along with kernel modules or should it be populated into a
> filesystem populated by kernelspace ?
>
> Or is it better to do as we have always done and keep the userspace
> tools separated from the kernel tree ?
I say keep the user space tools separate as much as possible.
What about having a meta-data file for all binary files. This meta-data
could explain the format that is read. Big endian, little endian, the
fields and offsets, the event ids etc. This way we will not need a
"version" file, which means absolutely nothing if you do not know what
comes with that version. Any tool could look at the meta-data file and
figure out what is in the buffers.
-- Steve
* Steven Rostedt ([email protected]) wrote:
>
> On Mon, 24 Nov 2008, Mathieu Desnoyers wrote:
> >
> > The key idea behind this is to answer to Thomas Gleixner concerns, who
> > supports that a tracer should output data in text-format only so it can
> > be used with tools kernel developers have on their system, like "cat".
> >
> > However, getting data out of the kernel efficiently simply cannot be
> > done with such approach. Therefore, LTTng needs its own userspace tools
> > to splice the data out of the kernel efficiently. Another tool is used
> > to pretty-print the binary data into text.
> >
> > Then the problem becomes : we have to make the userspace tool easy
> > enough to deploy so even Linus can find and use it. ;)
> >
> > But indeed, the trace buffers are versioned, so if the format changes
> > between kernel versions, the userspace tools will detect it and the user
> > will know it must update its tools. So it's not really a problem there.
> >
> > The question that prevails is therefore : should we ship userspace
> > binary with the kernel tree at all ? And if yes, how should the resuting
> > executables be packaged and deployed ? Should it be installed in the
> > system along with kernel modules or should it be populated into a
> > filesystem populated by kernelspace ?
> >
> > Or is it better to do as we have always done and keep the userspace
> > tools separated from the kernel tree ?
>
> I say keep the user space tools separate as much as possible.
>
I'd be in favor of that too. We should just document and package it so
it's easy to find.
> What about having a meta-data file for all binary files. This meta-data
> could explain the format that is read. Big endian, little endian, the
> fields and offsets, the event ids etc. This way we will not need a
> "version" file, which means absolutely nothing if you do not know what
> comes with that version. Any tool could look at the meta-data file and
> figure out what is in the buffers.
>
> -- Steve
This is exactly what I do in LTTng, modulo the fact that I repeat this
information also in other buffer headers, but only use the information
located in the metadata buffer header. I duplicated the information to
make sure all subbuffer headers looks the same, but I could easily
change that.
I would however keep a small subbuffer header with a version number for
each subbuffers though, just so the parser can "know" what file this is
and what metadata should be expected with it. I think about the poor
user who lost its metadata file and wonders what tool could open the
other tracefiles he has... without a header containing at least a magic
number and a version, those files won't be identified. But we can keep
this information as minimalistic as possible.
Thanks for the feedback.
Mathieu
--
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
* Mathieu Desnoyers ([email protected]) wrote:
> * Steven Rostedt ([email protected]) wrote:
> >
> > On Mon, 24 Nov 2008, Mathieu Desnoyers wrote:
> > >
> > > The key idea behind this is to answer to Thomas Gleixner concerns, who
> > > supports that a tracer should output data in text-format only so it can
> > > be used with tools kernel developers have on their system, like "cat".
> > >
> > > However, getting data out of the kernel efficiently simply cannot be
> > > done with such approach. Therefore, LTTng needs its own userspace tools
> > > to splice the data out of the kernel efficiently. Another tool is used
> > > to pretty-print the binary data into text.
> > >
> > > Then the problem becomes : we have to make the userspace tool easy
> > > enough to deploy so even Linus can find and use it. ;)
> > >
> > > But indeed, the trace buffers are versioned, so if the format changes
> > > between kernel versions, the userspace tools will detect it and the user
> > > will know it must update its tools. So it's not really a problem there.
> > >
> > > The question that prevails is therefore : should we ship userspace
> > > binary with the kernel tree at all ? And if yes, how should the resuting
> > > executables be packaged and deployed ? Should it be installed in the
> > > system along with kernel modules or should it be populated into a
> > > filesystem populated by kernelspace ?
> > >
> > > Or is it better to do as we have always done and keep the userspace
> > > tools separated from the kernel tree ?
> >
> > I say keep the user space tools separate as much as possible.
> >
>
> I'd be in favor of that too. We should just document and package it so
> it's easy to find.
>
> > What about having a meta-data file for all binary files. This meta-data
> > could explain the format that is read. Big endian, little endian, the
> > fields and offsets, the event ids etc. This way we will not need a
> > "version" file, which means absolutely nothing if you do not know what
> > comes with that version. Any tool could look at the meta-data file and
> > figure out what is in the buffers.
> >
> > -- Steve
>
> This is exactly what I do in LTTng, modulo the fact that I repeat this
> information also in other buffer headers, but only use the information
> located in the metadata buffer header. I duplicated the information to
> make sure all subbuffer headers looks the same, but I could easily
> change that.
>
> I would however keep a small subbuffer header with a version number for
> each subbuffers though, just so the parser can "know" what file this is
> and what metadata should be expected with it. I think about the poor
> user who lost its metadata file and wonders what tool could open the
> other tracefiles he has... without a header containing at least a magic
> number and a version, those files won't be identified. But we can keep
> this information as minimalistic as possible.
>
I now remember why I duplicate the endianness and the type size
information in the subbuffer header.
The main use-case for this is userspace tracing on a mixed 32/64-bits
system, where I'd like to be able to write from a 32-bits userspace
program directly into their own memory buffers. I would therefore have
one "channel" per process, or something like that. The thing is : I have
to know the pointer size so I know how the event headers are aligned in
the traces. However, with mixed 32/64-bits processes, I would happen to
have specific data types for specific channels (note that size_t is also
special : it changes depending on the compiling options for 32-bits
executables). Therefore, it seems to make sense to export this type of
information directly in the subbuffer headers.
I think we can still iterate on the userspace tracing aspects of tracing
later on, so we don't have to come up with a "perfect solution" right
now. I just want us to keep an open mind for the requirements that will
come with its implementation and try to plan a little bit ahead.
Mathieu
> Thanks for the feedback.
>
> Mathieu
>
> --
> Mathieu Desnoyers
> OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
>
> _______________________________________________
> ltt-dev mailing list
> [email protected]
> http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
>
--
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
On Mon, Nov 24, 2008 at 07:20:55AM -0500, Mathieu Desnoyers wrote:
> The key idea behind this is to answer to Thomas Gleixner concerns, who
> supports that a tracer should output data in text-format only so it can
> be used with tools kernel developers have on their system, like "cat".
Last time I talked to Thomas and others it wasn't "text only" but text
also in that you can just cat a debugfs file for simple tracing. And I
totally agree with that. If you don't trace too much that's absolutely
enough. It's not enough for large traces on highly loaded enterprise
or high-end embedded systems or things like flight recorder tracing, but
for those requiring an external viewer is just fine.
* Christoph Hellwig ([email protected]) wrote:
> On Mon, Nov 24, 2008 at 07:20:55AM -0500, Mathieu Desnoyers wrote:
> > The key idea behind this is to answer to Thomas Gleixner concerns, who
> > supports that a tracer should output data in text-format only so it can
> > be used with tools kernel developers have on their system, like "cat".
>
> Last time I talked to Thomas and others it wasn't "text only" but text
> also in that you can just cat a debugfs file for simple tracing. And I
> totally agree with that. If you don't trace too much that's absolutely
> enough. It's not enough for large traces on highly loaded enterprise
> or high-end embedded systems or things like flight recorder tracing, but
> for those requiring an external viewer is just fine.
>
Ok, thanks for the clarification. This will be pretty much
straightforward to implement.
Mathieu
>
> _______________________________________________
> ltt-dev mailing list
> [email protected]
> http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
>
--
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
On Tue, Nov 25, 2008 at 03:24:14AM -0500, Christoph Hellwig wrote:
> On Mon, Nov 24, 2008 at 07:20:55AM -0500, Mathieu Desnoyers wrote:
> > The key idea behind this is to answer to Thomas Gleixner concerns, who
> > supports that a tracer should output data in text-format only so it can
> > be used with tools kernel developers have on their system, like "cat".
>
> Last time I talked to Thomas and others it wasn't "text only" but text
> also in that you can just cat a debugfs file for simple tracing. And I
> totally agree with that. If you don't trace too much that's absolutely
> enough. It's not enough for large traces on highly loaded enterprise
> or high-end embedded systems or things like flight recorder tracing, but
> for those requiring an external viewer is just fine.
I agree that we should have a text-based output format for kernel
hackers, as well as a binary high volume format. I'd argue also that
since the text-based one would be used by kernel hackers, we don't
need to worry so much about keeping the format absolutely stable from
kernel version to kernel version (and it's not worth it to add version
tags to the text stream); people who care about a long-term, stable
interface should use the binary output format, IMHO.
I'm not sure everyone will agree with me, but I think it helps to
simplify the effort behind the text-based output.
- Ted
On Tue, Nov 25, 2008 at 03:57:22AM -0500, Theodore Tso wrote:
> I agree that we should have a text-based output format for kernel
> hackers, as well as a binary high volume format. I'd argue also that
> since the text-based one would be used by kernel hackers, we don't
> need to worry so much about keeping the format absolutely stable from
> kernel version to kernel version (and it's not worth it to add version
> tags to the text stream); people who care about a long-term, stable
> interface should use the binary output format, IMHO.
>
> I'm not sure everyone will agree with me, but I think it helps to
> simplify the effort behind the text-based output.
Yes, agreed.
On Mon, 24 Nov 2008 11:58:40 -0500 (EST) Steven Rostedt <[email protected]> wrote:
> > The question that prevails is therefore : should we ship userspace
> > binary with the kernel tree at all ? And if yes, how should the resuting
> > executables be packaged and deployed ? Should it be installed in the
> > system along with kernel modules or should it be populated into a
> > filesystem populated by kernelspace ?
> >
> > Or is it better to do as we have always done and keep the userspace
> > tools separated from the kernel tree ?
>
> I say keep the user space tools separate as much as possible.
So... we're now planning to build even more userspace tools into the
kernel merely because we don't know how to deliver userspace tools in
userspace?
Worse, those kernel-based userspace tools will simply be presenting a
textual form of something which the kernel already makes available in a
binary form?
This is so lame.
Any proposal to fix all this will result in vast amounts of
inconclusive chin-scratching again. Last time this came up, we ended
up with the old impractical put-it-in-util-linux chestnut. I think the
time has come for us to admit that this isn't working out.
Mathieu, if you're feeling keen I'd suggest that you just type `mkdir
-p userspace/lttng' and build your userspace tools in there. One they
are there and real, the issues around versioning, installation and
delivery will still need tobe sorted out, but we need to start
somewhere.
>
> Mathieu, if you're feeling keen I'd suggest that you just type `mkdir
> -p userspace/lttng' and build your userspace tools in there.
Maybe this can help...
Sam
diff --git a/usr/Makefile b/usr/Makefile
index 201f27f..627fafc 100644
--- a/usr/Makefile
+++ b/usr/Makefile
@@ -2,6 +2,8 @@
# kbuild file for usr/ - including initramfs image
#
+subdir-y := lttng
+
klibcdirs:;
PHONY += klibcdirs
diff --git a/usr/lttng/Makefile b/usr/lttng/Makefile
new file mode 100644
index 0000000..8667998
--- /dev/null
+++ b/usr/lttng/Makefile
@@ -0,0 +1,4 @@
+
+always := ltt
+
+hostprogs-y := ltt
diff --git a/usr/lttng/ltt.c b/usr/lttng/ltt.c
new file mode 100644
index 0000000..9eca2ed
--- /dev/null
+++ b/usr/lttng/ltt.c
@@ -0,0 +1,7 @@
+#include <stdio.h>
+
+int main(int ac, char *argv[])
+{
+ printf("lttng\n");
+ return 0;
+}
On Tue, Nov 25, 2008 at 04:40:11PM +0100, Sam Ravnborg wrote:
> >
> > Mathieu, if you're feeling keen I'd suggest that you just type `mkdir
> > -p userspace/lttng' and build your userspace tools in there.
>
> Maybe this can help...
>
> Sam
>
> diff --git a/usr/lttng/Makefile b/usr/lttng/Makefile
> new file mode 100644
> index 0000000..8667998
> --- /dev/null
> +++ b/usr/lttng/Makefile
> @@ -0,0 +1,4 @@
> +
> +always := ltt
> +
> +hostprogs-y := ltt
hostprogs are intended to be run on the host during the compilation
process, right? What we really want is something that allows us to
build userspace programs intended for use on the target, not the host,
since they aren't going to be used when the system is compiled, but
when the system is run.
This also brings up some interesting questions such as where would we
install these userspace programs (which could be kernel version
specific), and how they would be packaged. In some ways, this is
similar to a previously unsolved and unresolved problem with firmware
which people want to move out of the kernel (but not for legal
reasons, they swear, but because most of the time the firmware stays
constant from version to version --- except when it doesn't).
The same arguments that are used for ejecting firmware from the kernel
and not wanting to ship multiple versions of binaries with the kernel
can also be used for these sorts of low-level userspace binaries which
may be kernel version specific which are often duplicated.
I happen to disagree with those who wish to eject firmware from the
kernel sources (because I don't believe it is for technology reasons,
but because they want to be able to chant, "Unclean, Unclean!" as they
type rm -rf firmware). And I think we should need to be able to
distribute userspace binaries in the kernel, because of the versioning
issues and so we can guarantee that we get upgraded userspace tools
into the hands of users when they upgrade to the latest kernel.org
kernel and need certian updated userspace utilities where we want the
stable kernel interface to be located at a kernel-shipped userspace
program or library.
However, we do need to understand that there are issues around
building target userspace progs, where they get installed, changing
the kernel packaging systems to include them in the kernel image
package, creating some convention so users can set the appropriate
search path in their shells depending on which kernel they have
booted, etc. These are not insurmountable problems. But they are
ones that we need to solve first.
- Ted
On Tue, 25 Nov 2008 12:59:37 -0500 Theodore Tso <[email protected]> wrote:
> On Tue, Nov 25, 2008 at 04:40:11PM +0100, Sam Ravnborg wrote:
> > >
> > > Mathieu, if you're feeling keen I'd suggest that you just type `mkdir
> > > -p userspace/lttng' and build your userspace tools in there.
> >
> > Maybe this can help...
> >
> > Sam
> >
> > diff --git a/usr/lttng/Makefile b/usr/lttng/Makefile
> > new file mode 100644
> > index 0000000..8667998
> > --- /dev/null
> > +++ b/usr/lttng/Makefile
> > @@ -0,0 +1,4 @@
> > +
> > +always := ltt
> > +
> > +hostprogs-y := ltt
^^ good start!
> hostprogs are intended to be run on the host during the compilation
> process, right? What we really want is something that allows us to
> build userspace programs intended for use on the target, not the host,
> since they aren't going to be used when the system is compiled, but
> when the system is run.
>
> This also brings up some interesting questions such as where would we
> install these userspace programs (which could be kernel version
> specific), and how they would be packaged. In some ways, this is
> similar to a previously unsolved and unresolved problem with firmware
> which people want to move out of the kernel (but not for legal
> reasons, they swear, but because most of the time the firmware stays
> constant from version to version --- except when it doesn't).
>
> The same arguments that are used for ejecting firmware from the kernel
> and not wanting to ship multiple versions of binaries with the kernel
> can also be used for these sorts of low-level userspace binaries which
> may be kernel version specific which are often duplicated.
>
> I happen to disagree with those who wish to eject firmware from the
> kernel sources (because I don't believe it is for technology reasons,
> but because they want to be able to chant, "Unclean, Unclean!" as they
> type rm -rf firmware). And I think we should need to be able to
> distribute userspace binaries in the kernel, because of the versioning
> issues and so we can guarantee that we get upgraded userspace tools
> into the hands of users when they upgrade to the latest kernel.org
> kernel and need certian updated userspace utilities where we want the
> stable kernel interface to be located at a kernel-shipped userspace
> program or library.
>
> However, we do need to understand that there are issues around
> building target userspace progs, where they get installed, changing
> the kernel packaging systems to include them in the kernel image
> package, creating some convention so users can set the appropriate
> search path in their shells depending on which kernel they have
> booted, etc. These are not insurmountable problems. But they are
> ones that we need to solve first.
>
I think when you add in internationalisation, they _are_ insurmountable
problems. I don't think that full-on shipping of userspace-ready
applications by the kernel team will prove practical.
But let's look at the problem which we're actually trying to solve.
Developer A wishes to write some kernel monitoring/controlling code, so
he is forced to stick it on his website, keep reminding people to
download updates, act as an independent target of other people's
patches, etc, etc. It's all a pain and horror, so developer A gives up
and implements his userspace code in the kernel instead. It is, as a
result, technically inferior and English-only, but at least it got
there.
So I'd propose that userspace/foo/ be viewed as a code *development*
location, not a *delivery* location. Once a tool has reached a
sufficient level of maturity, stability and usefulness then it's time
to ask the util-linux people "please take this", then remove it from
the kernel tree. Or to create a separate package altogether, and to
remove the original from the kernel tree.
So userspace/foo/ doesn't have to be *fancy*. It's a three- to twelve-
month thing where code can sit while kernel developers evolve it. Its
main consumers at that time will be kernel developers and testers.
On Tue, Nov 25, 2008 at 12:59:37PM -0500, Theodore Tso wrote:
> On Tue, Nov 25, 2008 at 04:40:11PM +0100, Sam Ravnborg wrote:
> > >
> > > Mathieu, if you're feeling keen I'd suggest that you just type `mkdir
> > > -p userspace/lttng' and build your userspace tools in there.
> >
> > Maybe this can help...
> >
> > Sam
> >
> > diff --git a/usr/lttng/Makefile b/usr/lttng/Makefile
> > new file mode 100644
> > index 0000000..8667998
> > --- /dev/null
> > +++ b/usr/lttng/Makefile
> > @@ -0,0 +1,4 @@
> > +
> > +always := ltt
> > +
> > +hostprogs-y := ltt
>
> hostprogs are intended to be run on the host during the compilation
> process, right? What we really want is something that allows us to
> build userspace programs intended for use on the target, not the host,
> since they aren't going to be used when the system is compiled, but
> when the system is run.
Yes. And if this fly I will add support for that.
The majority build their kernel on the same host where they run it,
or at least a binary compatible host.
>
> This also brings up some interesting questions such as where would we
> install these userspace programs (which could be kernel version
> specific), and how they would be packaged.
We already have similar userspace tools which we have no problem distributing.
Think of checkpatch, bloat-o-meter, cleanfile etc.
They happens to be scripts and not binaries. But otherwise I
see no big difference.
Sam
* Sam Ravnborg ([email protected]) wrote:
> >
> > Mathieu, if you're feeling keen I'd suggest that you just type `mkdir
> > -p userspace/lttng' and build your userspace tools in there.
>
> Maybe this can help...
>
Hi Sam,
Thanks for the code snipped, I'll start from there for my kernel-tree
integrated userspace tools when I come back from travelling around
Europe.
Mathieu
> Sam
>
> diff --git a/usr/Makefile b/usr/Makefile
> index 201f27f..627fafc 100644
> --- a/usr/Makefile
> +++ b/usr/Makefile
> @@ -2,6 +2,8 @@
> # kbuild file for usr/ - including initramfs image
> #
>
> +subdir-y := lttng
> +
> klibcdirs:;
> PHONY += klibcdirs
>
> diff --git a/usr/lttng/Makefile b/usr/lttng/Makefile
> new file mode 100644
> index 0000000..8667998
> --- /dev/null
> +++ b/usr/lttng/Makefile
> @@ -0,0 +1,4 @@
> +
> +always := ltt
> +
> +hostprogs-y := ltt
> diff --git a/usr/lttng/ltt.c b/usr/lttng/ltt.c
> new file mode 100644
> index 0000000..9eca2ed
> --- /dev/null
> +++ b/usr/lttng/ltt.c
> @@ -0,0 +1,7 @@
> +#include <stdio.h>
> +
> +int main(int ac, char *argv[])
> +{
> + printf("lttng\n");
> + return 0;
> +}
>
> _______________________________________________
> ltt-dev mailing list
> [email protected]
> http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
>
--
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
* Andrew Morton ([email protected]) wrote:
> On Mon, 24 Nov 2008 11:58:40 -0500 (EST) Steven Rostedt <[email protected]> wrote:
>
> > > The question that prevails is therefore : should we ship userspace
> > > binary with the kernel tree at all ? And if yes, how should the resuting
> > > executables be packaged and deployed ? Should it be installed in the
> > > system along with kernel modules or should it be populated into a
> > > filesystem populated by kernelspace ?
> > >
> > > Or is it better to do as we have always done and keep the userspace
> > > tools separated from the kernel tree ?
> >
> > I say keep the user space tools separate as much as possible.
>
> So... we're now planning to build even more userspace tools into the
> kernel merely because we don't know how to deliver userspace tools in
> userspace?
>
> Worse, those kernel-based userspace tools will simply be presenting a
> textual form of something which the kernel already makes available in a
> binary form?
>
> This is so lame.
>
> Any proposal to fix all this will result in vast amounts of
> inconclusive chin-scratching again. Last time this came up, we ended
> up with the old impractical put-it-in-util-linux chestnut. I think the
> time has come for us to admit that this isn't working out.
>
> Mathieu, if you're feeling keen I'd suggest that you just type `mkdir
> -p userspace/lttng' and build your userspace tools in there. One they
> are there and real, the issues around versioning, installation and
> delivery will still need tobe sorted out, but we need to start
> somewhere.
>
Yes, I like the idea of having a kernel-developer oriented version of
the userspace tools, which may not have packaging requirements as strong
as standard userspace packages. Just keeping them around with the kernel
build tree would be a good start, and we could probably go as far as
installing them along with the kernel modules in /lib/modules/`uname
-r`.
Mathieu
>
> _______________________________________________
> ltt-dev mailing list
> [email protected]
> http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
>
--
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
Mathieu Desnoyers wrote:
> - Marker ID management
> Background :
> The markers are currently used to identify the numerical event IDs
> associated with a named event as well as the event types (format
> string) associated with that event. Those two informations are kept in
> a table written in the metadata channel. A "channel" could be thought
> as being a set of per-cpu buffers, where information transits. This
> information would typically be related to a specific tracer. Note,
> however, that the information generated by a tracer should always be
> parseable generically and be transformable into text output by a
> simple parser, because we want to permit analysis across information
> logged by the various tracer.
>
> Action item :
> The current event IDs are global to the whole kernel. It would be
> beneficial to change their scope so they are per-channel instead,
> because we can then encode most events in the low-order bits of the
> event ID bits, therefore keeping events as small as possible.
>
Hi, Mathieu Desnoyers:
It also required by ASCII text output.
ASCII text output needs get name&fmt from eID, but In current code,
I cannot find a reliable way to get name&fmt from eID. And also,
in the kernel space, we must get name&fmt from eID as fast as possible
by reason of performance. So I cannot use metadata channel for it.
When "Add periodical subbuffer flush for streaming" and "Marker ID management"
will done?
Thanks, Lai.