LKMB version 0.1.16 is the humble codeware beginning of the CLAN project.
It is essentially a Perl package (proper with Makefile.PL and all, but not
CPANed yet), which enables one to process LKMB packages.
The latter ones are packages that LKMB can create installation and
compilation packages for kernel modules that can run in any enviornment
the Linux kernel can be compiled and installed on. (a GNU environment).
It contains an example module for the Ethernet DMFE module. Currently, the
makefile for the kernel module's package supports only the "all" and
"install" targets.
I will upload it to CPAN soon, but would like to get some initial feedback
beforehand.
Regards,
Shlomi Fish
----------------------------------------------------------------------
Shlomi Fish [email protected]
Home Page: http://t2.technion.ac.il/~shlomif/
He who re-invents the wheel, understands much better how a wheel works.
Ooop! Forgot the URL:
http://fc-solve.berlios.de/CLAN/download/arcs/Linux-Kernel-Modules-Builder-0.1.16.tar.gz
And the CLAN temporary homepage is at:
http://fc-solve.berlios.de/CLAN/
Sorry about that.
Regards,
Shlomi Fish
----------------------------------------------------------------------
Shlomi Fish [email protected]
Home Page: http://t2.technion.ac.il/~shlomif/
He who re-invents the wheel, understands much better how a wheel works.
> print MAKEFILE "CFLAGS = -I/usr/src/linux/include -O2 -Wall -DMODULE -D__KERNEL__ -fno-common -fomit-frame-pointer -pipe -mpreferred-stack-boundary=2\n\n";
That's broken. you need to get the proper kernel CFLAGS, and you shouldn't
assume there's anything useful in /usr/src/linux.
Use "/lib/modules/`uname -r`/build" as a default kernel directory, but
allow it to be overridden somehow from the command line. Then do something
like...
make -C $(LINUXDIR) SUBDIRS=`pwd` modules
... to build your module. That way, all the kernel build stuff will be
correct; it'll be just as if you were in a normal subdirectory of the
kernel tree during a 'make modules' run.
--
dwmw2
Wow! My first patch suggestion! Thanks.
On Fri, 17 Jan 2003, David Woodhouse wrote:
>
> > print MAKEFILE "CFLAGS = -I/usr/src/linux/include -O2 -Wall -DMODULE -D__KERNEL__ -fno-common -fomit-frame-pointer -pipe -mpreferred-stack-boundary=2\n\n";
>
> That's broken. you need to get the proper kernel CFLAGS, and you shouldn't
> assume there's anything useful in /usr/src/linux.
>
Ahah. What do you suggest me to do instead? The LKMB package needs to
compile on every system it was intended to. It's still a source package
that has to compile on any GNU system that has the Linux kernel headers.
> Use "/lib/modules/`uname -r`/build" as a default kernel directory, but
> allow it to be overridden somehow from the command line. Then do something
> like...
>
> make -C $(LINUXDIR) SUBDIRS=`pwd` modules
>
> ... to build your module. That way, all the kernel build stuff will be
> correct; it'll be just as if you were in a normal subdirectory of the
> kernel tree during a 'make modules' run.
>
Do you mean I'll need a live Linux kernel to build the kernel module
package?
I'm not a big kbuild expert and it show.
Regards,
Shlomi Fish
> --
> dwmw2
>
>
----------------------------------------------------------------------
Shlomi Fish [email protected]
Home Page: http://t2.technion.ac.il/~shlomif/
He who re-invents the wheel, understands much better how a wheel works.
On Fri, Jan 17, 2003 at 07:00:58PM +0200, Shlomi Fish wrote:
>
> Do you mean I'll need a live Linux kernel to build the kernel module
> package?
Yes, you fundamentally need the full kernel to compile a module.
Modules may refer to different headers, and some may even be arch specific.
The trick dwmw2 gave you is the only _sane_ way to build a module.
Sam
On Fri, 17 Jan 2003, Sam Ravnborg wrote:
> On Fri, Jan 17, 2003 at 07:00:58PM +0200, Shlomi Fish wrote:
> >
> > Do you mean I'll need a live Linux kernel to build the kernel module
> > package?
>
> Yes, you fundamentally need the full kernel to compile a module.
> Modules may refer to different headers, and some may even be arch specific.
>
> The trick dwmw2 gave you is the only _sane_ way to build a module.
>
Yes I gathered it from him and #kernelnewbies. In the future I want
CLAN to be able to package entire kernels. But I'll guess I'll cross the
bridge when I get there, and there's no need to overengineer now.
Regards,
Shlomi Fish
> Sam
>
----------------------------------------------------------------------
Shlomi Fish [email protected]
Home Page: http://t2.technion.ac.il/~shlomif/
He who re-invents the wheel, understands much better how a wheel works.
> > Use "/lib/modules/`uname -r`/build" as a default kernel directory, but
> > allow it to be overridden somehow from the command line. Then do something
> > like...
>...
> Do you mean I'll need a live Linux kernel to build the kernel module
> package?
Whoever invented this /lib/modules/... scheme should have known that
it provokes this sort of misunderstandings, not to mention is broken
in other ways too.
You need the _source_ of the kernel the module will run on to compile
modules. You don't need to _run_ this kernel while compiling. Putting
build infrastructure into a deployment directory at the least causes
confusion, not to mention that the deployment directory might not even
exist on the development machine. (I routinely compile kernels and
modules of different configurations for three boxes on one of them,
the other two don't even have a complete development toolset.)
Compiling modules is one of the things which always have been among
the most broken things in the kernel build systems, can this please be
fixed and properly documented?
Olaf
On Sat, Jan 18, 2003 at 11:37:19PM +0100, Olaf Titz wrote:
> Compiling modules is one of the things which always have been among
> the most broken things in the kernel build systems, can this please be
> fixed and properly documented?
^^^^^
Some actual bug reports would be good.
regards
john
--
"This is just the kind of crackpot scheme I've been looking to champion!!!"
- P.M. Hartke on 6U campaign
On Sat, 2003-01-18 at 23:37, Olaf Titz wrote:
> > > Use "/lib/modules/`uname -r`/build" as a default kernel directory, but
> > > allow it to be overridden somehow from the command line. Then do something
> > > like...
> >...
> > Do you mean I'll need a live Linux kernel to build the kernel module
> > package?
>
> Whoever invented this /lib/modules/... scheme should have known that
> it provokes this sort of misunderstandings, not to mention is broken
> in other ways too.
it was Linus who decreed this to be the standard;)
>
> You need the _source_ of the kernel the module will run on to compile
> modules. You don't need to _run_ this kernel while compiling. Putting
> build infrastructure into a deployment directory at the least causes
> confusion, not to mention that the deployment directory might not even
> exist on the development machine. (I routinely compile kernels and
> modules of different configurations for three boxes on one of them,
> the other two don't even have a complete development toolset.)
yes, and most of the time you want to compile against the currently
running kernel, at which point `uname -r` comes in handy; for other
kernels you just change the path a bit.
make install and make modules_install make the symlink right already....
it's a 99% solution, sure, but it's ok for all but a few cases.
> > Compiling modules is one of the things which always have been among
> > the most broken things in the kernel build systems, can this please be
> > fixed and properly documented?
> ^^^^^
>
> Some actual bug reports would be good.
The general bug is that there is incomplete infrastructure for
building modules outside of the kernel. You see the problem when
looking at the CIPE configure scripts, or the ALSA configure scripts.
Up to kernel version 2.4, it takes considerable effort to find out
just what compiler options to use. This is information which belongs
in some easily accessed location.
The desirable situation for module developers would be that a kernel
tree after configure run contains a Makefile (or equivalent) with all
necessary definitions which can be called from an outside module
source tree and just DTRT. The 2.5 kbuild stuff is close, but not
complete.
It is a bug that Documentation/modules.txt is so outdated that it
contains little useful information any more. It is a bug that
Documentation/kbuild/makefiles.txt is at least a bit outdated.
It is a bug that the build process outside of the kernel tree changes
files inside the kernel tree when MODVERSIONS is enabled. (At least
this was the case last time I checked.) This means the kernel tree
can't be mounted read-only, or at least you would have to do dirty
tricks with symlinks.
It is a bug that the current Makefile can't compile modules in an
object directory different from the source directory. This means the
module source tree can't be mounted read-only (again, without
resorting to symlinks).
It is also a bug that parts of the development infrastructure are
installed in /lib/modules/<version> and it's somewhat documented that
compiling modules needs this /lib/modules/<version> stuff. That may be
true for the ideal, simplified Red Hat world but in reality the
machine and running OS version of the development machine is likely
different from the box it will run on. Mixing development environment
and install target only causes confusion.
I don't know if real cross-compilation (i.e. for a different
architecture than the compiler runs on) of modules is possible
yet. If not, that's a bug too.[1]
Olaf
[1] Okay, on this count I'm guilty too with CIPE (just trying to sort
that out...)
On Sun, 2003-01-19 at 13:55, Olaf Titz wrote:
> It is also a bug that parts of the development infrastructure are
> installed in /lib/modules/<version> and it's somewhat documented that
> compiling modules needs this /lib/modules/<version> stuff. That may be
> true for the ideal, simplified Red Hat world but in reality the
> machine and running OS version of the development machine is likely
> different from the box it will run on. Mixing development environment
> and install target only causes confusion.
you make a series of good points before this. However
/lib/modules/<version>/build is nothing Red Hat specific. It's something
that is the result of a similar discussion long ago where Linus finally
decreed this location for finding the full source of modules.
Combine that with the makefile dwmw2 showed and you can compile external
modules EVERYWHERE on ANY distribution (assuming said distribution
doesn't go out of the way to break the decree). Afaik RHL, SuSE,
Mandrake, Debian and Slackware at least have this correct.
Yes it breaks if you move around your source after doing make
modules_install. Yes it breaks if you don't have the tree at all. But
both situations are "invalid" wrt the decree, and need a fixed symlink.
On Sun, Jan 19, 2003 at 01:55:11PM +0100, Olaf Titz wrote:
>
> It is a bug that Documentation/modules.txt is so outdated that it
> contains little useful information any more.
An updated version that at least describe how to build modules sanely
outside the kernel tree is queued for inclusion[1].
> It is a bug that
> Documentation/kbuild/makefiles.txt is at least a bit outdated.
A totally revised makefiles.txt is queued for inclusion[1].
> It is a bug that the build process outside of the kernel tree changes
> files inside the kernel tree when MODVERSIONS is enabled. (At least
> this was the case last time I checked.) This means the kernel tree
> can't be mounted read-only, or at least you would have to do dirty
> tricks with symlinks.
I have posted 2-3 version of a "separate src-tree" patch.
But due to the existence of such weird things as oprofile and xfs it did
not work in all cases. I symlinked all makefiles, but wanted to avoid
catching the rest of the directories.
I want my queued stuff applied before revisiting this though.
Main usage has been to build kernels based on different configurations,
not to mount src on a read-only media, that's sort of a side-effect.
> It is a bug that the current Makefile can't compile modules in an
> object directory different from the source directory. This means the
> module source tree can't be mounted read-only (again, without
> resorting to symlinks).
make -C path/to/kernel/src SUBDIRS=$PWD modules
So simple when you know the trick. That what I have documented
in modules.txt.
[1] Most of what I have done has been submitted to lkml within the
past month. It can be found at http://linux-sam.bkbits.net in
the kbuild repository, where it is queued for inclusion.
Sam
>> It is also a bug that parts of the development infrastructure are
>> installed in /lib/modules/<version> and it's somewhat documented that
>> compiling modules needs this /lib/modules/<version> stuff. That may be
>> true for the ideal, simplified Red Hat world but in reality the
>> machine and running OS version of the development machine is likely
>> different from the box it will run on. Mixing development environment
>> and install target only causes confusion.
>
> you make a series of good points before this. However
> /lib/modules/<version>/build is nothing Red Hat specific. It's something
> that is the result of a similar discussion long ago where Linus finally
> decreed this location for finding the full source of modules.
> Combine that with the makefile dwmw2 showed and you can compile external
> modules EVERYWHERE on ANY distribution (assuming said distribution
> doesn't go out of the way to break the decree). Afaik RHL, SuSE,
> Mandrake, Debian and Slackware at least have this correct.
>
> Yes it breaks if you move around your source after doing make
> modules_install. Yes it breaks if you don't have the tree at all. But
> both situations are "invalid" wrt the decree, and need a fixed symlink.
Try "make modules INSTALL_MOD_PATH=<whatever>". Then modules
will use <whatever>/lib/modules instead of /lib/modules. This works in 2.4
and early 2.5, I haven't tried it with the new kbuild system in recent 2.5.
And
I don't know if this is properly documented anywhere.
Paul
> Try "make modules INSTALL_MOD_PATH=<whatever>".
Oops, that should have been:
"make modules_install INSTALL_MOD_PATH=<whatever>"
Paul
> > It is a bug that the current Makefile can't compile modules in an
> > object directory different from the source directory. This means the
> > module source tree can't be mounted read-only (again, without
> > resorting to symlinks).
>
> make -C path/to/kernel/src SUBDIRS=$PWD modules
> So simple when you know the trick. That what I have documented
no, this doesn't work when you have the source and object files _of
the module_ in separate directories. You'd have to run make in the
object directory, but there seems no way to tell it where the source
is. (At least I have not found any.)
The reason to do this is compiling one module for different kernel
configurations as well as compiling one kernel for different
configurations. I.e. I want to have a tree like this:
first the kernel:
/usr/src/linux/2.5.99/ this is directory X
/var/obj/linux/2.5.99/boxa here is X compiled for boxa
/var/obj/linux/2.5.99/boxb here is X compiled for boxb
(this doesn't work either at the moment, another bug/missing feature...)
then the external module:
/usr/src/cipe this is directory Y
/var/obj/cipe/2.5.99/boxa here is Y compiled against X for boxa
/var/obj/cipe/2.5.99/boxb here is Y compiled against X for boxb
and /usr/src might be mounted read-only.
Olaf
> Yes it breaks if you move around your source after doing make
> modules_install. Yes it breaks if you don't have the tree at all. But
> both situations are "invalid" wrt the decree, and need a fixed symlink.
This means the decree also rules out compiling a kernel for one box on
another?
Olaf
> > Whoever invented this /lib/modules/... scheme should have known that
> > it provokes this sort of misunderstandings, not to mention is broken
> > in other ways too.
>
> it was Linus who decreed this to be the standard;)
no matter who decreed it. Even Linus may be wrong sometimes.
> yes, and most of the time you want to compile against the currently
> running kernel, at which point `uname -r` comes in handy; for other
> kernels you just change the path a bit.
> make install and make modules_install make the symlink right already....
> it's a 99% solution, sure, but it's ok for all but a few cases.
And what's exactly wrong with the other 99% solution of putting it in
/usr/src/linux-`uname -r` ? This has exactly the same advantages but
doesn't mix up between development and runtime environment; /usr/src
is clearly where source belongs and /lib/modules is an install target.
Even Linus has finally accepted that the root of the source tree is
best called linux-$VERSION rather than just linux, so this is not an
obstacle either.
Olaf
On Mon, Jan 20, 2003 at 09:14:17PM +0100, Olaf Titz wrote:
> first the kernel:
>
> /usr/src/linux/2.5.99/ this is directory X
> /var/obj/linux/2.5.99/boxa here is X compiled for boxa
> /var/obj/linux/2.5.99/boxb here is X compiled for boxb
>
> (this doesn't work either at the moment, another bug/missing feature...)
Call it a missing feature, I have posted patches for this before.
Small details missing to enable it.
> then the external module:
>
> /usr/src/cipe this is directory Y
> /var/obj/cipe/2.5.99/boxa here is Y compiled against X for boxa
> /var/obj/cipe/2.5.99/boxb here is Y compiled against X for boxb
>
> and /usr/src might be mounted read-only.
If I remember when I return to the "separate src tree" patch, I will
enable this as well.
I have not thought about this before, thanks for input.
Sam
> And what's exactly wrong with the other 99% solution of putting it in
> /usr/src/linux-`uname -r` ? This has exactly the same advantages but
> doesn't mix up between development and runtime environment; /usr/src
> is clearly where source belongs and /lib/modules is an install target.
back then the argument (not mine btw) was that /usr on a lot of machines
is RO (I think debian has an option for that) so that sysadmins there
compile stuff in /root. /lib/modules however IS standardized and needs
to be writable to install a new kernel so making a symlink to the real
place there isn't too bad. In addition it already is the only directory
with per kernel files.. adding a second one was judged not needed. It
has to be somewhere. /lib/modules/ or /usr/src.. who cares. Linus made
the final call and everybody complies with it since then, just because
it doesn't matter THAT much. It just needs to be SOMEWHERE standard and
/lib/modules suffices so far it seems.
> back then the argument (not mine btw) was that /usr on a lot of machines
> is RO (I think debian has an option for that) so that sysadmins there
and because of that (among other reasons) my other report about
compiling outside of the source directory...
> compile stuff in /root. /lib/modules however IS standardized and needs
> to be writable to install a new kernel so making a symlink to the real
Only while installing a new kernel. And you install the kernel on the
box it needs to run on, which is not necessarily the box it's compiled
on.
> place there isn't too bad. In addition it already is the only directory
> with per kernel files.. adding a second one was judged not needed. It
Only for stand-alone machines which only ever compile and run one
kernel. You don't need a data center to violate that, you just need
the fairly usual three-boxes home network (one of which is mainly a
router/firewall which has no development environment if only for
security reasons, or because it's a scavenged '486).
> has to be somewhere. /lib/modules/ or /usr/src.. who cares. Linus made
> the final call and everybody complies with it since then, just because
> it doesn't matter THAT much. It just needs to be SOMEWHERE standard and
> /lib/modules suffices so far it seems.
Frankly, I think the main reason is that Linus doesn't care at all
about the kernel build process. We've had a _much_ better solution
already in the 2.5 cycle which was rejected for completely bogus
formal reasons coupled with an explicit "why do we need this at all",
even though it was pointed out over and over again what is broken
currently (or was back then, granted it has improved but not as much
as is desirable and possible).
This blatantly erroneous decision[1] just smells of a blind spot or
carelessness more than careful consideration.
Olaf
[1] it's about of the same category as the original Unix' designers
putting utmp and wtmp in /etc. Linus would never have approved _that_.
On Tue, Jan 21, 2003 at 10:16:06PM +0100, Olaf Titz wrote:
> currently (or was back then, granted it has improved but not as much
> as is desirable and possible).
>From your previous posts I have only noted down one feature that is
not yet planned:
1) Building modules separate from module src.
The other issues you have rasied are either queued for inclusion, or
being worked on.
Could you try to come up with some sort of list what you really miss.
It is always nice to get inputs from others, not only judging based
on your own needs.
Any constructive inputs are greatly appreciated.
Sam
On Tue, 21 Jan 2003, Olaf Titz wrote:
> Frankly, I think the main reason is that Linus doesn't care at all
> about the kernel build process. We've had a _much_ better solution
> already in the 2.5 cycle which was rejected for completely bogus
> formal reasons coupled with an explicit "why do we need this at all",
> even though it was pointed out over and over again what is broken
> currently (or was back then, granted it has improved but not as much
> as is desirable and possible).
in this case the real reason is probably that Linus seldom uses modules.
David Lang
[email protected] said:
> From your previous posts I have only noted down one feature that is
> not yet planned: 1) Building modules separate from module src.
This would be cute. When testing a single module under both 2.4 and 2.5
kernels, I often have to 'make clean ; make LINUXDIR=/usr/src/linux-2.5'
and 'make clean; make' to switch between them. Having separate object trees
would make life a lot nicer.
--
dwmw2
[email protected] said:
> Only for stand-alone machines which only ever compile and run one
> kernel. You don't need a data center to violate that, you just need
> the fairly usual three-boxes home network (one of which is mainly a
> router/firewall which has no development environment if only for
> security reasons, or because it's a scavenged '486).
Er, if it has no development environment, why are you bitching about the
fact that it's not possible to compile kernel modules on it?
The box which holds your firewall's kernel source can be used to compile
extra out-of-tree modules. The directory /lib/modules/`uname -r`/build,
while a reasonable _default_ for out-of-tree modules to use, should
generally be overridable with a directory specified by yourself.
--
dwmw2
On Wed, 22 Jan 2003, David Woodhouse wrote:
>
> [email protected] said:
> > Only for stand-alone machines which only ever compile and run one
> > kernel. You don't need a data center to violate that, you just need
> > the fairly usual three-boxes home network (one of which is mainly a
> > router/firewall which has no development environment if only for
> > security reasons, or because it's a scavenged '486).
>
> Er, if it has no development environment, why are you bitching about the
> fact that it's not possible to compile kernel modules on it?
>
> The box which holds your firewall's kernel source can be used to compile
> extra out-of-tree modules. The directory /lib/modules/`uname -r`/build,
> while a reasonable _default_ for out-of-tree modules to use, should
> generally be overridable with a directory specified by yourself.
`uname -r` is the kernel version of the running kernel. It is NOT by magic
the kernel version of the kernel you are building...
--
bill davidsen <[email protected]>
CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
[email protected] said:
> `uname -r` is the kernel version of the running kernel. It is NOT by
> magic the kernel version of the kernel you are building...
Er, yes. And what's your point?
There is _no_ magic that will find the kernel you want to build against
today without any input from you. Using the build tree for the
currently-running kernel, if installed in the standard place, is as good a
default as any. Of course you should be permitted to override that default.
You remain free to put your build trees wherever you want -- with the
obvious proviso that if you put them somewhere other than the standard
place, you need to tell the out-of-tree build process where to find the tree
you want to build against. This seems to be entirely irrelevant to the
original question.
--
dwmw2
I use "INSTALL_MOD_PATH=put/the/modules/here/instead/of/lib/modules" in my
.profile or whatever in order to drop the modules into another directory
at "make modules_install" time. Is this one of the things folks are
talking about?
Hal Duston
On Wed, 22 Jan 2003, David Woodhouse wrote:
>
> [email protected] said:
> > `uname -r` is the kernel version of the running kernel. It is NOT by
> > magic the kernel version of the kernel you are building...
>
> Er, yes. And what's your point?
>
> There is _no_ magic that will find the kernel you want to build against
> today without any input from you. Using the build tree for the
> currently-running kernel, if installed in the standard place, is as good a
> default as any. Of course you should be permitted to override that default.
You make my point for me, there is no magic, and when building a module it
should require that the directory be specified by either a command line
option (as noted below) or by being built as part of a source tree. There
*is* no good default in that particular case.
On Wed, 22 Jan 2003, Hal Duston wrote:
> I use "INSTALL_MOD_PATH=put/the/modules/here/instead/of/lib/modules" in my
> .profile or whatever in order to drop the modules into another directory
> at "make modules_install" time. Is this one of the things folks are
> talking about?
Related for sure, the point I was making was that there is no good default
place to put modules built outside a kernel source tree (and probably also
when built for multiple kernels). I was suggesting that the module tree of
the running kernel might be a really poor choice. I don't think I was
clear in my first post, I was not suggesting a better default, I was
suggesting that any default is likely to bite.
I'm not unhappy that Mr. Woodhouse disagrees, I just think he missed my
point the first time and I'm trying to clarify.
--
bill davidsen <[email protected]>
CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
[email protected] said:
> You make my point for me, there is no magic, and when building a
> module it should require that the directory be specified by either a
> command line option (as noted below) or by being built as part of a
> source tree. There *is* no good default in that particular case.
/lib/modules/`uname -r`/build _is_ a good default for a module to build
again. It is correct in more cases than a simple failure to do anything.
For _installing_, the correct place to install the built objects is surely
/lib/modules/$(VERSION).$(PATCHLEVEL).$(SUBLEVEL)$(EXTRAVERSION) where the
variables are obtained from the top-level Makefile in the kernel sources
you built against.
You _default_ to building and installing for the current kernel, if it's
installed properly. But of course you should be permitted to override that
on the command line.
> Related for sure, the point I was making was that there is no good
> default place to put modules built outside a kernel source tree (and
> probably also when built for multiple kernels).
I disagree. Modutils will look in only one place -- the /lib/modules/...
directory corresponding to the kernel version for which you built each
module. Each module, therefore, should go into the directory corresponding
to the version of the kernel against which it was built.
Finding the appropriate _installation_ directory is trivial, surely? You
can even find it from the 'kernel_version' stamp _inside_ the object file,
without any other information?
--
dwmw2