Hi,
I have writen a small kernel module and it works perfectly, but currently (I
think it just begun with 2.5.59) I get the warning above when the module is
inserted. Now I am just interested what I have to change so this message
won't appear anymore...
Thank you much!
Thomas Schlichter
P.S.: If my Makefile or source will help I'll give it to you...
On Thu, 23 Jan 2003, Thomas Schlichter wrote:
| I have writen a small kernel module and it works perfectly, but currently (I
| think it just begun with 2.5.59) I get the warning above when the module is
| inserted. Now I am just interested what I have to change so this message
| won't appear anymore...
|
| P.S.: If my Makefile or source will help I'll give it to you...
| -
Did you rebuild the module with a 2.5.59 Makefile?
Yes, it's a 2.5.59 change according to the Changelog at kernel.org:
<QUOTE>
<[email protected]>
Module Sanity Check
This patch, based on Rusty's implementation,
adds a special section to vmlinux and all modules, which
contain the kernel version string, values of some
particularly important config options (SMP,preempt,proc
family) and the gcc version.
When inserting a module, the version string is checked against the
kernel version string and loading is rejected if they don't match.
The version string is actually added to the modules during the final
.ko generation, so that a changed version string does only cause relinking,
not recompilation, which is a major performance improvement over the old 2.4
way of doing things.
</QUOTE>
--
~Randy
On Thu, Jan 23, 2003 at 02:59:22PM +0100, Thomas Schlichter wrote:
> Hi,
>
> I have writen a small kernel module and it works perfectly, but currently (I
> think it just begun with 2.5.59) I get the warning above when the module is
> inserted. Now I am just interested what I have to change so this message
> won't appear anymore...
>
> Thank you much!
>
> Thomas Schlichter
>
> P.S.: If my Makefile or source will help I'll give it to you...
What command did you use to build your module?
If you did no use:
make -C path/to/kernel/src SUBDIRS=$PWD modules
chances are big you did not compile the module correct.
This requires the Makefile to look like any other kernel (kbuild) makefile.
Sam
Thanks for your answers!
I did not compile my module with a kernel Makefile, I used the very small and
simple one attatched to this mail. So it seems I miss something when the
module is linked and I have to know what I have to link to the module or
which header-file I have to include...
For me it seems link I have to link the init/vermagic.c file to my module, but
how would this be possible if only the kernel includes were available??
I think only these should be needed to compile a module...
Thomas Schlichter
Am Donnerstag, 23. Januar 2003 17:29 schrieb Randy.Dunlap:
> Did you rebuild the module with a 2.5.59 Makefile?
>
>
> Yes, it's a 2.5.59 change according to the Changelog at kernel.org:
>
> <QUOTE>
> <[email protected]>
> Module Sanity Check
>
> This patch, based on Rusty's implementation,
> adds a special section to vmlinux and all modules, which
> contain the kernel version string, values of some
> particularly important config options (SMP,preempt,proc
> family) and the gcc version.
>
> When inserting a module, the version string is checked against the
> kernel version string and loading is rejected if they don't match.
>
> The version string is actually added to the modules during the final
> .ko generation, so that a changed version string does only cause relinking,
> not recompilation, which is a major performance improvement over the old
> 2.4 way of doing things.
> </QUOTE>
Am Donnerstag, 23. Januar 2003 17:52 schrieb Sam Ravnborg:
> What command did you use to build your module?
> If you did no use:
> make -C path/to/kernel/src SUBDIRS=$PWD modules
>
> chances are big you did not compile the module correct.
> This requires the Makefile to look like any other kernel (kbuild) makefile.
>
> Sam
On Thu, Jan 23, 2003 at 06:32:59PM +0100, Thomas Schlichter wrote:
> Thanks for your answers!
>
> I did not compile my module with a kernel Makefile, I used the very small and
> simple one attatched to this mail. So it seems I miss something when the
> module is linked and I have to know what I have to link to the module or
> which header-file I have to include...
>
> For me it seems link I have to link the init/vermagic.c file to my module, but
> how would this be possible if only the kernel includes were available??
> I think only these should be needed to compile a module...
As it is today the only sane way is to have the full kernel src available.
It should be possible to minimize that - but I do not feel tempted to
do so.
We want to do too much to rely on whatever makefile people write for
their module.
Sam
Can't the stuff in init/vermagic.c be moved into a header file? Maybe
vermagic.h? Most of the code can be cut 'n pasted right out of vermagic.c
and the bit that defines "const char vermagic[]..." could be placed inside a
macro which modules would then stick in the bottom of one of their c files.
This is what I'm getting at (warning I haven't checked this code or even
tried to clean it up):
in vermagic.h:
#include <linux/version.h>
#include <linux/module.h>
/* Simply sanity version stamp for modules. */
#ifdef CONFIG_SMP
#define MODULE_VERMAGIC_SMP "SMP "
#else
#define MODULE_VERMAGIC_SMP ""
#endif
#ifdef CONFIG_PREEMPT
#define MODULE_VERMAGIC_PREEMPT "preempt "
#else
#define MODULE_VERMAGIC_PREEMPT ""
#endif
#ifndef MODULE_ARCH_VERMAGIC
#define MODULE_ARCH_VERMAGIC ""
#endif
#define KERNEL_VERSIONMAGIC const char vermagic[] \
__attribute__((section("__vermagic"))) = \
UTS_RELEASE " " \
MODULE_VERMAGIC_SMP MODULE_VERMAGIC_PREEMPT MODULE_ARCH_VERMAGIC \
"gcc-" __stringify(__GNUC__) "." __stringify(__GNUC_MINOR__)
in my_external_module.c, and init/vermagic.c I'd just do:
#include <linux/vermagic.h>
KERNEL_VERSIONMAGIC();
and be done with it.
Modules that ship with the kernel wouldn't have to change a thing (and still
be linked against vermagic.c, and it'd only add two lines of code to
everyones externally shipped modules. I don't really think that this would be
"doing too much" for those whose modules are included in Linus's kernel, and
it wouldn't require people to keep entire source tree's around to compile a
module or two...
Am I totally missing something here or wouldn't this solve our problem?
Someone please correct me if I'm wrong :)
--Mark
On Thu, Jan 23, 2003 at 07:22:36PM +0100, Sam Ravnborg wrote:
> As it is today the only sane way is to have the full kernel src available.
> It should be possible to minimize that - but I do not feel tempted to
> do so.
>
> We want to do too much to rely on whatever makefile people write for
> their module.
>
> Sam
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
--
Mark Fasheh
Software Developer, Oracle Corp
[email protected]
On Thu, Jan 23, 2003 at 11:35:40AM -0800, Mark Fasheh wrote:
>
> Can't the stuff in init/vermagic.c be moved into a header file?
> Maybe vermagic.h? Most of the code can be cut 'n pasted right out of
> vermagic.c and the bit that defines "const char vermagic[]..." could
> be placed inside a macro which modules would then stick in the
> bottom of one of their c files. This is what I'm getting at
> (warning I haven't checked this code or even tried to clean it up):
>
> in vermagic.h:
> #include <linux/version.h>
> #include <linux/module.h>
>
> /* Simply sanity version stamp for modules. */
> #ifdef CONFIG_SMP
> #define MODULE_VERMAGIC_SMP "SMP "
> #else
> #define MODULE_VERMAGIC_SMP ""
> #endif
> #ifdef CONFIG_PREEMPT
> #define MODULE_VERMAGIC_PREEMPT "preempt "
> #else
> #define MODULE_VERMAGIC_PREEMPT ""
> #endif
> #ifndef MODULE_ARCH_VERMAGIC
> #define MODULE_ARCH_VERMAGIC ""
> #endif
>
> #define KERNEL_VERSIONMAGIC const char vermagic[] \
> __attribute__((section("__vermagic"))) = \
> UTS_RELEASE " " \
> MODULE_VERMAGIC_SMP MODULE_VERMAGIC_PREEMPT MODULE_ARCH_VERMAGIC \
> "gcc-" __stringify(__GNUC__) "." __stringify(__GNUC_MINOR__)
>
>
> in my_external_module.c, and init/vermagic.c I'd just do:
> #include <linux/vermagic.h>
> KERNEL_VERSIONMAGIC();
>
> and be done with it.
>
> Modules that ship with the kernel wouldn't have to change a thing
> (and still be linked against vermagic.c, and it'd only add two lines
> of code to everyones externally shipped modules. I don't really
> think that this would be "doing too much" for those whose modules
> are included in Linus's kernel, and it wouldn't require people to
> keep entire source tree's around to compile a module or two... Am I
> totally missing something here or wouldn't this solve our problem?
> Someone please correct me if I'm wrong :)
>
I wouldn't go as far as saying that the current solution imposes an
onerous requirement on any user wishing to build third-party modules
by relying on a complete and properly configured kernel source tree,
but it certainly seems like a step backwards from Linux 2.4, where
properly configured kernel headers are sufficient.
Of course, this only holds true for external projects using kbuild to
build the modules; other build systems would not only require that a
complete, configured kernel source tree be installed, they would also
rely on that source tree to be uncleaned since they have no knowledge
of how init/vermagic.o is to be built (and shouldn't make assumptions,
not considering possible legal/licensing implications). This I really
do consider an unnecessary, burdensome prerequisite.
Since I'm not intimately familiar with kbuild, I'm also wondering if
this scenario would be a problem:
Somebody downloaded Linux 2.5.59, configured it and built it using a
pre-3.0 version of gcc, e.g. gcc 2.95. The user had plenty of disk
space and decided, based on past experiences, that leaving the source
tree uncleaned is least likely to cause problems, should he/she ever
be intersted in building third-party modules. For some time, the user
placed no interest in external modules and used his/her system quite
happily; with the release of a new, improved version of a driver, the
user decided that he wants to give it a try, however, and built the
driver module, which picked up init/vermagic.o; our imaginary user is
using a distribution that provides frequent updates and he/she makes
regular use of this service - it just so happens that one of these
updates installed gcc 3.0 as the new default compiler. The new module
is thus built using gcc 3.0, but init/vermagic.o still indicates gcc
2.95; the module loader will erroneously believe everything is fine.
I agree with Mark that moving the code in init/vermagic.c into a new
header file like linux/vermagic.h is a better solution.
--
christian zander
[email protected]
On Sun, 26 Jan 2003 14:29:23 +0100,
Christian Zander <[email protected]> wrote:
>The new module
>is thus built using gcc 3.0, but init/vermagic.o still indicates gcc
>2.95; the module loader will erroneously believe everything is fine.
Congratulations, you have put your finger on a major design flaw in
modversions that has been there since 2.0 kernel days. The modversion
data is generated once and everything else blindly uses it, with _NO_
checks on whether it is still valid or not. Rusty knows damn well that
this is broken, but appears to be ignoring that fact (Rusty, see my
mail to you and Alan Cox on Wed, 24 Oct 2001 14:14:18 +1000).
See http://unc.dl.sourceforge.net/sourceforge/kbuild/kbuild-2.5-history.tar.bz2,
in particular makefile-2.5_whereto.html. modversions relies on
assumptions that are not valid and therefore modversions are broken, as
noted by Christian Zander and others.
To do module versioning right needs global information about this
definitions and usage of exported symbols. It was one of the reasons
that kbuild 2.5 had a global database, in order to track symbol usage
and handle ABI versioning correctly.
On Thu, 23 Jan 2003, Mark Fasheh wrote:
> Can't the stuff in init/vermagic.c be moved into a header file? Maybe
> vermagic.h? Most of the code can be cut 'n pasted right out of vermagic.c
> and the bit that defines "const char vermagic[]..." could be placed inside a
> macro which modules would then stick in the bottom of one of their c files.
> This is what I'm getting at (warning I haven't checked this code or even
> tried to clean it up):
Your suggestion is sensible, yet it is just an indication that you're
using the wrong way to build your external module.
The thing is, open source projects like the linux kernel tend to move
fast, and they don't care about changing the interfaces are the way to
build things much, since you get the source and can recompile yourself.
Due to that fact, all solutions which try to build modules externally are
bound to fail sooner or later. IMO the only sensible way to overcome this
is to accept help from the kernel build system instead of adding one
kludge after the other to your home-made Makefile.
The kernel build provides this facility today, use the
make -C $KERNELSRC SUBDIRS=$PWD approach
that Sam Ravnborg pointed out. It's true that it has not been clearly
separated how much of the kernel source tree is needed to do that, (it's
at least include/*, .config, scripts, init/, ...) so the rule is: You need
the entire configured kernel tree.
Now, is that so bad? When you're building kernels yourself, you obviously
have enough room for a full tree anyway. When you're using a distribution,
you have to install the kernel source rpm anyway, to get the headers. For
all I know, these days the headers are not distributed separately from the
rest of the kernel source anymore..
You might say that this is a regression w.r.t 2.4. But actually, even in
2.4, you need e.g. Makefile and arch/i386/Makefile to figure out the
correct flags and things for your compile, and those are not headers,
either.
> in my_external_module.c, and init/vermagic.c I'd just do:
> #include <linux/vermagic.h>
> KERNEL_VERSIONMAGIC();
This is a good solution to this specific problem. But it does not solve
the rest, e.g. your Makefile doesn't set -fomit-frame-pointer depending on
CONFIG_FRAME_POINTER. It doesn't set the proper march=x86 flags. IA-64
even needs a special flag just for modules. And it'll get even worse with
the reintroduction of module symbol versioning.
So the above would work around this specific problem, leaving the other
more subtle ones unsolved. And if you're using modules which have been
built in such a fragile way with subtle differences, I think it's
justified to have your kernel tainted.
--Kai
On Sun, 26 Jan 2003, Christian Zander wrote:
> Of course, this only holds true for external projects using kbuild to
> build the modules; other build systems would not only require that a
> complete, configured kernel source tree be installed, they would also
> rely on that source tree to be uncleaned since they have no knowledge
> of how init/vermagic.o is to be built (and shouldn't make assumptions,
> not considering possible legal/licensing implications). This I really
> do consider an unnecessary, burdensome prerequisite.
Well, what I'm trying to say is that external build system will always
break one way or other. Since they're external, they're naturally out of
reach for me to influence, so there's really nothing I can do it but
telling people to using the internal system instead.
> Somebody downloaded Linux 2.5.59, configured it and built it using a
> pre-3.0 version of gcc, e.g. gcc 2.95. The user had plenty of disk
> space and decided, based on past experiences, that leaving the source
> tree uncleaned is least likely to cause problems, should he/she ever
> be intersted in building third-party modules. For some time, the user
> placed no interest in external modules and used his/her system quite
> happily; with the release of a new, improved version of a driver, the
> user decided that he wants to give it a try, however, and built the
> driver module, which picked up init/vermagic.o; our imaginary user is
> using a distribution that provides frequent updates and he/she makes
> regular use of this service - it just so happens that one of these
> updates installed gcc 3.0 as the new default compiler. The new module
> is thus built using gcc 3.0, but init/vermagic.o still indicates gcc
> 2.95; the module loader will erroneously believe everything is fine.
Again, when you're using your own external build system, it's up to you to
mess it up in all possible ways, the above being one of them.
When you're using the kernel build, the above cannot happen, since the
kernel build system knows about this dependency and builds a new
init/vermagic.o with the correct information before it gets linked into
the external module.
--Kai
On Mon, 27 Jan 2003, Keith Owens wrote:
> Christian Zander <[email protected]> wrote:
> >The new module
> >is thus built using gcc 3.0, but init/vermagic.o still indicates gcc
> >2.95; the module loader will erroneously believe everything is fine.
>
> Congratulations, you have put your finger on a major design flaw in
> modversions that has been there since 2.0 kernel days. The modversion
> data is generated once and everything else blindly uses it, with _NO_
> checks on whether it is still valid or not. Rusty knows damn well that
> this is broken, but appears to be ignoring that fact (Rusty, see my
> mail to you and Alan Cox on Wed, 24 Oct 2001 14:14:18 +1000).
First of all, "modversions" has been used to designate the process of
module symbols versioning in the past years, which the above is not. The
version magic string is supposed to be a simple check against obvious
errors like kernel version mismatch between kernel and module and also to
protect against incompatibilities which the ABI checksums (which
modversions is about) cannot detect, like incompatible compiler versions.
As such, the information is known beforehand and now complicated
bookkeeping needed here. (Also, as I just pointed out in a another mail,
the above scenario is not possible).
Now your comments seem to rather apply to the actual module symbol
versioning process, which I just posted a patch for. Things are handled
very much differently now, if you have technical comments on that patch,
they'd be much appreciated.
--Kai
On Sun, Jan 26, 2003 at 11:51:22AM -0600, Kai Germaschewski wrote:
>
> Well, what I'm trying to say is that external build system will
> always break one way or other. Since they're external, they're
> naturally out of reach for me to influence, so there's really
> nothing I can do it but telling people to using the internal system
> instead.
>
External build systems have been working just fine even after kbuild
was adopted as the build system of choice for the Linux kernel, there
should be more convincing reasons for deliberately breaking them now.
> Again, when you're using your own external build system, it's up to
> you to mess it up in all possible ways, the above being one of them.
>
> When you're using the kernel build, the above cannot happen, since
> the kernel build system knows about this dependency and builds a new
> init/vermagic.o with the correct information before it gets linked
> into the external module.
>
Is that so? Does kbuild determine that vermagic.o needs rebuilding if
the compiler version changed?
--
christian zander
[email protected]
On Sun, Jan 26, 2003 at 11:43:44AM -0600, Kai Germaschewski wrote:
>
> Now, is that so bad? When you're building kernels yourself, you
> obviously have enough room for a full tree anyway. When you're using
> a distribution, you have to install the kernel source rpm anyway, to
> get the headers. For all I know, these days the headers are not
> distributed separately from the rest of the kernel source anymore..
>
Debian GNU/Linux is a good example of a distribution shipping source
and configured kernel header packages seperately. It seems fair enough
to require a configured source tree installed, however, and I don't
have a strong opinion on this particular question.
> You might say that this is a regression w.r.t 2.4. But actually,
> even in 2.4, you need e.g. Makefile and arch/i386/Makefile to figure
> out the correct flags and things for your compile, and those are not
> headers, either.
>
Not necessarily.
> This is a good solution to this specific problem. But it does not
> solve the rest, e.g. your Makefile doesn't set -fomit-frame-pointer
> depending on CONFIG_FRAME_POINTER. It doesn't set the proper
> march=x86 flags. IA-64 even needs a special flag just for modules.
> And it'll get even worse with the reintroduction of module symbol
> versioning.
>
> So the above would work around this specific problem, leaving the
> other more subtle ones unsolved. And if you're using modules which
> have been built in such a fragile way with subtle differences, I
> think it's justified to have your kernel tainted.
>
External projects may not want to use the build flags unchanged, they
may have good reasons for using their own. It seems sensible to make
the kernel build system available to those who wish to use it, but it
should be optional rather than mandatory. In this specific case, there
is no technical reason to require the use of kbuild.
--
christian zander
[email protected]
On Sun, Jan 26, 2003 at 10:57:14PM +0100, Christian Zander wrote:
>
> Is that so? Does kbuild determine that vermagic.o needs rebuilding if
> the compiler version changed?
>
Nevermind, I just checked - it does.
--
christian zander
[email protected]
On Sun, Jan 26, 2003 at 11:08:42PM +0100, Christian Zander wrote:
> External projects may not want to use the build flags unchanged, they
> may have good reasons for using their own. It seems sensible to make
> the kernel build system available to those who wish to use it, but it
> should be optional rather than mandatory. In this specific case, there
> is no technical reason to require the use of kbuild.
Judging on the number of people that got bid by the mach- include changes,
I conclude that we better ask people to use kbuild.
The whole issues boils down to the following options:
1) The "kernel-headers" package shall be extended to include the vital
part of kbuild
2) To develop modules you need the full kernel src
3) Do-it-yourself makefiles
Can we agree on that - and take the discussion from there?
In 1) and 2) you have total freedom to change options etc.
Several architectures filter out generic options they dislike.
Adding extra options is supported by kbuild (EXTRA_CFLAGS).
In 3) you have all possibilities to screw up things. You would probarly
argue you have full flexibility.
But then I wonder what kind of flexibility you need for your module, that
is not needed for all the modules included in the kernel?
To take your argument and turn it around:
What technical reasons are there to avoid kbuild?
Please realise that you will be hit by changes in include paths,
compiler options etc.
That is visible in the number of mails seen on lkml the last couple of
months.
Sam
On Sun, 26 Jan 2003, Christian Zander wrote:
> On Sun, Jan 26, 2003 at 11:51:22AM -0600, Kai Germaschewski wrote:
> >
> > Well, what I'm trying to say is that external build system will
> > always break one way or other. Since they're external, they're
> > naturally out of reach for me to influence, so there's really
> > nothing I can do it but telling people to using the internal system
> > instead.
> >
>
> External build systems have been working just fine even after kbuild
> was adopted as the build system of choice for the Linux kernel, there
> should be more convincing reasons for deliberately breaking them now.
That's not true. For example, how would an old external build system
magically starting to compile modules as .ko without updating? How would
it have added -DKBUILD_BASENAME and -DKBUILD_MODNAME, which are required
by the new module code. And, how did they avoid subtle breakage like not
giving the same switches on the command line? (This list goes on...)
Also, it's not true that they've been broken deliberately. As work
progresses, breakage occurs, that's just a fact of live. However,
introduction of __vermagic was not introduced in order to make live for
maintainers of external modules harder, it was introduced since loading
modules compiled with gcc3 into a kernel compiled with gcc2 caused crashes
for people.
> > When you're using the kernel build, the above cannot happen, since
> > the kernel build system knows about this dependency and builds a new
> > init/vermagic.o with the correct information before it gets linked
> > into the external module.
>
> Is that so? Does kbuild determine that vermagic.o needs rebuilding if
> the compiler version changed?
Okay, you have a point here, there's still a bug. vermagic.o will be
rebuilt when the version changes or any of the recorded config options
change, but it doesn't pick up changes in the compiler version, if the
new gcc has the same name.
That's a bug for internal use as well, the patch below fixes it.
Two more things:
o You say kbuild has been adopted as the build system of choice. That's
misleading, the new build system for 2.5, called kbuild-2.5 has never
been adopted. It's just that the existing build system has been improved
quite a bit by me and others.
o One thing I do not understand at all: What is the problem with using
the internal build system? It makes maintainance of external modules
much easier than keeping track of what happens in the kernel and
patching a private solution all the time.
I don't even see any license issues, first of all you don't even
distribute it, the user who's building the module will already have it
along with his kernel source. And if you're using it to compile
(possibly binary) modules you want to distribute, you can just use it
just like gcc without any further obligations, so no problem there
either. (IANAL, of course)
--Kai
===== init/Makefile 1.18 vs edited =====
--- 1.18/init/Makefile Tue Jan 14 14:29:02 2003
+++ edited/init/Makefile Sun Jan 26 15:24:21 2003
@@ -12,6 +12,13 @@
$(obj)/version.o: include/linux/compile.h
+# vermagic doesn't actually include compile.h, but it records
+# the compiler version. To make sure we notice when the compiler
+# changes, add a dependency on compile.h, which changes when the
+# compiler changes.
+
+$(obj)/vermagic.o: include/linux/compile.h
+
# compile.h changes depending on hostname, generation number, etc,
# so we regenerate it always.
# mkcompile_h will make sure to only update the
[email protected] said:
> External projects may not want to use the build flags unchanged, they
> may have good reasons for using their own.
That's what EXTRA_CFLAGS and CFLAGS_someobject.o are for.
> It seems sensible to make the kernel build system available to those
> who wish to use it, but it should be optional rather than mandatory.
> In this specific case, there is no technical reason to require the use
> of kbuild.
The use of the kernel build process to build kernel modules is not
_mandatory_, it's just that it's the only sane option.
You are, of course, welcome to hack up your own broken and short-term
solutions which happen to work this week for some platforms. But don't come
crying to us when (not if) they stop working.
The use of vermagic.c doesn't stop you from making your own build system;
you can have your own vermagic.c to make your hacks work this week.
--
dwmw2
On Sun, Jan 26, 2003 at 10:29:50PM +0100, Sam Ravnborg wrote:
>
> Judging on the number of people that got bid by the mach- include
> changes, I conclude that we better ask people to use kbuild.
>
Fair enough.
> The whole issues boils down to the following options:
> 1) The "kernel-headers" package shall be extended to include the
> vital part of kbuild
> 2) To develop modules you need the full kernel src
> 3) Do-it-yourself makefiles
>
> Can we agree on that - and take the discussion from there?
>
> In 1) and 2) you have total freedom to change options etc.
> Several architectures filter out generic options they dislike.
> Adding extra options is supported by kbuild (EXTRA_CFLAGS).
>
> In 3) you have all possibilities to screw up things. You would
> probarly argue you have full flexibility. But then I wonder what
> kind of flexibility you need for your module, that is not needed for
> all the modules included in the kernel? To take your argument and
> turn it around: What technical reasons are there to avoid kbuild?
>
> Please realise that you will be hit by changes in include paths,
> compiler options etc. That is visible in the number of mails seen
> on lkml the last couple of months.
>
The problem isn't necessarily lack of flexibility, but the lack of
unity across kernel versions. I agree that kbuild is the preferable
solution for Linux 2.5, but it isn't for all incarnations of Linux
2.4 and definetely not for Linux 2.2. I do realize that changes have
resulted in problems for external build systems and understand that
future changes may result in similar problems.
I guess a reasonable solution is adjusting existing Makefiles to be
smart enough to detect kbuild and use it when available.
--
christian zander
[email protected]
On Sun, Jan 26, 2003 at 03:46:30PM -0600, Kai Germaschewski wrote:
>
> That's not true. For example, how would an old external build system
> magically starting to compile modules as .ko without updating? How
> would it have added -DKBUILD_BASENAME and -DKBUILD_MODNAME, which
> are required by the new module code. And, how did they avoid subtle
> breakage like not giving the same switches on the command line?
> (This list goes on...)
>
I hear you, but these changes were easy enough to adapt to.
> Also, it's not true that they've been broken deliberately. As work
> progresses, breakage occurs, that's just a fact of live. However,
> introduction of __vermagic was not introduced in order to make live
> for maintainers of external modules harder, it was introduced since
> loading modules compiled with gcc3 into a kernel compiled with gcc2
> caused crashes for people.
>
Well, in this specific case an alternate solution was proposed that
would have solved any of the potential problems pointed out.
> Okay, you have a point here, there's still a bug. vermagic.o will be
> rebuilt when the version changes or any of the recorded config
> options change, but it doesn't pick up changes in the compiler
> version, if the new gcc has the same name.
>
> That's a bug for internal use as well, the patch below fixes it.
>
Fair enough.
> o One thing I do not understand at all: What is the problem with
> using the internal build system? It makes maintainance of external
> modules much easier than keeping track of what happens in the kernel
> and patching a private solution all the time.
>
My primary concern is compatibility with those kernels that do not use
kbuild or a different version of it. Ideally, one would want to use
the same build system for all possible kernel versions rather than use
Makefiles that attempt to pick the best choice. I guess I'm convinced
that the latter is the "best" solution to dealing with this problem at
this point, and I can live with that.
What's the most reliable way to tell if kbuild is available, and what
differences among kbuild versions will one have to look out for?
> I don't even see any license issues, first of all you don't even
> distribute it, the user who's building the module will already
> have it along with his kernel source. And if you're using it to
> compile (possibly binary) modules you want to distribute, you can
> just use it just like gcc without any further obligations, so no
> problem there either. (IANAL, of course)
>
I don't see any problems with kbuild, I was referring to vermagic.c.
--
christian zander
[email protected]
On Sun, Jan 26, 2003 at 09:40:22PM +0000, David Woodhouse wrote:
>
> That's what EXTRA_CFLAGS and CFLAGS_someobject.o are for.
>
I am well aware of the documentation in Documentation/kbuild and know
that kbuild is flexible enough to support customized CFLAGS. I didn't
argue that.
> The use of the kernel build process to build kernel modules is not
> _mandatory_, it's just that it's the only sane option.
>
> You are, of course, welcome to hack up your own broken and
> short-term solutions which happen to work this week for some
> platforms. But don't come crying to us when (not if) they stop
> working.
>
> The use of vermagic.c doesn't stop you from making your own build
> system; you can have your own vermagic.c to make your hacks work
> this week.
>
I can't follow your hostility in this matter; I didn't "come crying"
to you when things broke in the past and I'm not crying now. Last time
I checked, voicing concerns was a legitimate thing to do.
The "broken and short-term solutions" are needed with any kernel that
doesn't provide the module building support introduced with Linux 2.5,
which will likely be the majority of kernels in use for quite a while,
still.
--
christian zander
[email protected]
[email protected] said:
> I am well aware of the documentation in Documentation/kbuild and know
> that kbuild is flexible enough to support customized CFLAGS. I didn't
> argue that.
Ok, but why else would you want your own makefiles if that's not that you
wanted them for?
> I can't follow your hostility in this matter; I didn't "come crying"
> to you when things broke in the past and I'm not crying now. Last time
> I checked, voicing concerns was a legitimate thing to do.
True, but in this case you are voicing concern about the potential breakage
of something which was always known to be bad practice, fragile and
unreliable.
Your expression of concern is noted, but with about as much sympathy as is
granted to those who express concern because kernel headers which they were
including from userspace have changed.
Yes, it breaks if you invent you own makefiles. We knew that.
Don't Do That Then -- or if you must, then just deal with it breaking in
the kernel-de-jour.
> The "broken and short-term solutions" are needed with any kernel that
> doesn't provide the module building support introduced with Linux 2.5,
> which will likely be the majority of kernels in use for quite a while,
> still.
'make -C $LINUXDIR SUBDIRS=$PWD modules' has worked for as long as I can
remember; it's not new in 2.5. It's _always_ been the only reliable way to
get kernel modules to build with the correct options.
--
dwmw2
[email protected] said:
> The problem isn't necessarily lack of flexibility, but the lack of
> unity across kernel versions. I agree that kbuild is the preferable
> solution for Linux 2.5, but it isn't for all incarnations of Linux 2.4
> and definetely not for Linux 2.2.
/me blinks... what's wrong with 2.2? Looks fine to me...
imladris /home/dwmw2/working/mtd/drivers/mtd $ make LINUXDIR=/inst/linux/linux-2.2
make -C /inst/linux/linux-2.2 SUBDIRS=`pwd` modules
make[1]: Entering directory `/inst/linux/linux-2.2'
make -C /home/dwmw2/working/mtd/drivers/mtd CFLAGS="-Wall -Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strict-aliasing -D__SMP__ -pipe -fno-strength-reduce -m486 -malign-loops=2 -malign-jumps=2 -malign-functions=2 -DCPU=686 -DMODULE"
MAKING_MODULES=1 modules
make[2]: Entering directory `/home/dwmw2/working/mtd/drivers/mtd'
gcc296 -D__KERNEL__ -I/inst/linux/linux-2.2/include -I/home/dwmw2/working/mtd/drivers/mtd/../../include -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strict-aliasing -D__SMP__ -pipe -fno-strength-reduce -m486 -malign-loops=2 -malign-jumps=2 -malign-functions=2 -DCPU=686 -DMODULE -DEXPORT_SYMTAB -c afs.c
{standard input}: Assembler messages:
{standard input}:9: Warning: ignoring changed section attributes for .modinfo
gcc296 -D__KERNEL__ -I/inst/linux/linux-2.2/include -I/home/dwmw2/working/mtd/drivers/mtd/../../include -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strict-aliasing -D__SMP__ -pipe -fno-strength-reduce -m486 -malign-loops=2 -malign-jumps=2 -malign-functions=2 -DCPU=686 -DMODULE -DEXPORT_SYMTAB -c mtdcore.cmake[2]: *** Deleting file `mtdcore.o'
make[2]: *** [mtdcore.o] Interrupt
make[1]: *** [_mod_/home/dwmw2/working/mtd/drivers/mtd] Interrupt
make: *** [modules] Interrupt
--
dwmw2
On Sun, Jan 26, 2003 at 10:46:39PM +0000, David Woodhouse wrote:
>
> Ok, but why else would you want your own makefiles if that's not
> that you wanted them for?
>
Essentially because they already existed and worked (well enough).
> True, but in this case you are voicing concern about the potential
> breakage of something which was always known to be bad practice,
> fragile and unreliable.
>
> Your expression of concern is noted, but with about as much sympathy
> as is granted to those who express concern because kernel headers
> which they were including from userspace have changed.
>
> Yes, it breaks if you invent you own makefiles. We knew that. Don't
> Do That Then -- or if you must, then just deal with it breaking in
> the kernel-de-jour.
>
Fair enough.
> 'make -C $LINUXDIR SUBDIRS=$PWD modules' has worked for as long as
> I can remember; it's not new in 2.5. It's _always_ been the only
> reliable way to get kernel modules to build with the correct
> options.
>
Since Linux 2.2 and including any specifics involved in the process of
customizing CFLAGS, ...? If that's the case, I admit ignorance and ask
that my earlier remarks be ignored.
--
christian zander
[email protected]
[email protected] said:
> Since Linux 2.2 and including any specifics involved in the process
> of customizing CFLAGS, ...? If that's the case, I admit ignorance and
> ask that my earlier remarks be ignored.
imladris /home/dwmw2/working/mtd/drivers/mtd $ make LINUXDIR=/inst/linux/linux-2.2 CFLAGS_afs.o=-DFISH EXTRA_CFLAGS=-DTURNIP
make -C /inst/linux/linux-2.2 SUBDIRS=`pwd` modules
make[1]: Entering directory `/inst/linux/linux-2.2'
make -C /home/dwmw2/working/mtd/drivers/mtd CFLAGS="-Wall -Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strict-aliasing -D__SMP__ -pipe -fno-strength-reduce -m486 -malign-loops=2 -malign-jumps=2 -malign-functions=2 -DCPU=686 -DMODULE"
MAKING_MODULES=1 modules
make[2]: Entering directory `/home/dwmw2/working/mtd/drivers/mtd'
gcc296 -D__KERNEL__ -I/inst/linux/linux-2.2/include -I/home/dwmw2/working/mtd/drivers/mtd/../../include -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strict-aliasing -D__SMP__ -pipe -fno-strength-reduce -m486 -malign-loops=2 -malign-jumps=2 -malign-functions=2 -DCPU=686 -DMODULE -DTURNIP -DFISH -DEXPORT_SYMTAB -c afs.c
make[2]: *** Deleting file `afs.o'
make[2]: *** [afs.o] Interrupt
make[1]: *** [_mod_/home/dwmw2/working/mtd/drivers/mtd] Interrupt
make: *** [modules] Interrupt
--
dwmw2
On Sun, Jan 26, 2003 at 10:55:49PM +0000, David Woodhouse wrote:
>
> /me blinks... what's wrong with 2.2? Looks fine to me...
>
> imladris /home/dwmw2/working/mtd/drivers/mtd $ make LINUXDIR=/inst/linux/linux-2.2
> make -C /inst/linux/linux-2.2 SUBDIRS=`pwd` modules
> make[1]: Entering directory `/inst/linux/linux-2.2'
> make -C /home/dwmw2/working/mtd/drivers/mtd CFLAGS="-Wall -Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strict-aliasing -D__SMP__ -pipe -fno-strength-reduce -m486 -malign-loops=2 -malign-jumps=2 -malign-functions=2 -DCPU=686 -DMODULE"
> MAKING_MODULES=1 modules
>
I apologize if I have argued based on false assumptions; is it true
then that a Makefile written for use with Linux 2.5 kbuild will work
unchanged with any Linux 2.2/2.5 kernel (w/ custom CFLAGS, ...)?
--
christian zander
[email protected]
[email protected] said:
> I apologize if I have argued based on false assumptions; is it true
> then that a Makefile written for use with Linux 2.5 kbuild will work
> unchanged with any Linux 2.2/2.5 kernel (w/ custom CFLAGS, ...)?
There are some differences, but nothing insurmountable.
For <=2.4 you have to include Rules.make at the end of your own Makefile,
and for 2.5 that file doesn't exist any more.
For older kernels you must set O_TARGET, for 2.5 I think the mere act of
setting it causes the build to break -- that one is gratuitously making it
harder to make external modules which compile in both and I've complained
about it before.
Per-file CFLAGS must have a full path specified now in 2.5, whereas in 2.4
and earlier it was just 'CFLAGS_filename.o'.
--
dwmw2
On Sun, Jan 26, 2003 at 11:16:11PM +0000, David Woodhouse wrote:
>
> There are some differences, but nothing insurmountable.
>
> For <=2.4 you have to include Rules.make at the end of your own
> Makefile, and for 2.5 that file doesn't exist any more.
>
> For older kernels you must set O_TARGET, for 2.5 I think the mere
> act of setting it causes the build to break -- that one is
> gratuitously making it harder to make external modules which compile
> in both and I've complained about it before.
>
> Per-file CFLAGS must have a full path specified now in 2.5, whereas
> in 2.4 and earlier it was just 'CFLAGS_filename.o'.
>
Thanks for pointing that out, I'll look into this more closely. If it
is as simple as you say, I'll be inclined to share your sentiment
towards custom build systems interfacing with the Linux kernel.
--
christian zander
[email protected]
On Sun, Jan 26, 2003 at 03:46:30PM -0600, Kai Germaschewski wrote:
>
> o One thing I do not understand at all: What is the problem with using
> the internal build system? It makes maintainance of external modules
> much easier than keeping track of what happens in the kernel and
> patching a private solution all the time.
>
> I don't even see any license issues, first of all you don't even
> distribute it, the user who's building the module will already have it
> along with his kernel source. And if you're using it to compile
> (possibly binary) modules you want to distribute, you can just use it
> just like gcc without any further obligations, so no problem there
> either. (IANAL, of course)
>From my exprience at VMware newsgroups distros have bad troubles even
with delivery of basic configured kernel headers matching to kernel
binaries they provide (it is not unusual that for example they go from
1GB to 4GB kernel without make mrproper so kmap/kunmap do not have proper
versions attached :-( or they even sell headers and binaries with
different configs). Trying to ask them to provide usable configured kernel
sources (and not one which automagically replace -3 with -3custom
so that modules built by internal build system do not match to kernel
at all...) is not going to work - it did not work in the past, and I see
no reason why it should start working now. And dialup users do not want to
download 30MB package when 2MB headers were always enough (and especially
if they'll use 1.5MB from the downloaded package to build 3rd party module).
There are few exceptinons of distros which provide kernel sources in
usable form (== without any user intervention /lib/modules/`uname -r`/build
points to working correctly configured kernel tree), but usually their
users build their own kernels anyway, so nobody profits from that :-(
Petr Vandrovec
[email protected]
[email protected] said:
> From my exprience at VMware newsgroups distros have bad troubles even
> with delivery of basic configured kernel headers matching to kernel
> binaries they provide (it is not unusual that for example they go from
> 1GB to 4GB kernel without make mrproper so kmap/kunmap do not have
> proper versions attached :-( or they even sell headers and binaries
> with different configs).
But you _need_ the config. Even with your own makefiles, how are you going
to get it right for all new kernels that $CRAPDISTRO ships in a broken form,
if you don't have the configs?
If distros ship broken crap which doesn't let you build modules, there's
really not a lot you can do except note their quality control for the
record and report it in their bug tracking system.
You are _always_ going to have problems with people shipping shite, or
possibly even actively going our of their way to prevent out-of-tree modules
building. It's fairly much an orthogonal problem though, isn't it?
--
dwmw2
On Mon, Jan 27, 2003 at 09:02:11AM +0000, David Woodhouse wrote:
>
> [email protected] said:
> > From my exprience at VMware newsgroups distros have bad troubles even
> > with delivery of basic configured kernel headers matching to kernel
> > binaries they provide (it is not unusual that for example they go from
> > 1GB to 4GB kernel without make mrproper so kmap/kunmap do not have
> > proper versions attached :-( or they even sell headers and binaries
> > with different configs).
>
> But you _need_ the config. Even with your own makefiles, how are you going
> to get it right for all new kernels that $CRAPDISTRO ships in a broken form,
> if you don't have the configs?
Yes, but currently you need only 3 files: autoconf.h, version.h, and, eventually
.config, as other headers are read-only and nobody managed to get them
wrong (yet).
When using kernel Makefiles there are at least two more files (main and arch Makefile),
of which main Makefile is known to be modified by vendors to get their cloneconfig
working: they define additional rules and variables, and you may run into conflict.
> If distros ship broken crap which doesn't let you build modules, there's
> really not a lot you can do except note their quality control for the
> record and report it in their bug tracking system.
>
> You are _always_ going to have problems with people shipping shite, or
> possibly even actively going our of their way to prevent out-of-tree modules
> building. It's fairly much an orthogonal problem though, isn't it?
Increasing number of files needed to build module also increases possibility
that something will go wrong :-( Ok, I'll modify modules build script to
use kernel's build system if /lib/modules/`uname -r`/build/Makefile exists,
and I'll see.
BTW, is there any way how code outside of Makefile can detect what file was
generated and where it was stored: whether vmmon.o or vmmon.ko? As it is now,
Makefile contains obj-y := vmmon.o, but you can end up with vmmon.ko... I
need it to make a note for uninstaller which files we added to the system
(and which we can overwrite without prompting on upgrade).
Petr Vandrovec
On Sun, 26 Jan 2003, David Woodhouse wrote:
> For older kernels you must set O_TARGET, for 2.5 I think the mere act of
> setting it causes the build to break -- that one is gratuitously making it
> harder to make external modules which compile in both and I've complained
> about it before.
Okay, I might get persuaded to take that check out if it really makes life
harder. Wouldn't it work to not have O_TARGET in the Makefile at all, I
think 2.4 shouldn't care as long as you just "make modules"?
> Per-file CFLAGS must have a full path specified now in 2.5, whereas in 2.4
> and earlier it was just 'CFLAGS_filename.o'.
It shouldn't, I kinda deliberately decided to keep the old syntax for
this. So it would be a bug.
--Kai
[email protected] said:
> Okay, I might get persuaded to take that check out if it really makes
> life harder. Wouldn't it work to not have O_TARGET in the Makefile at
> all, I think 2.4 shouldn't care as long as you just "make modules"?
Er, I think O_TARGET is in fact the target _module_ name when building
a file system as a module. Try removing O_TARGET from 2.4 fs/ext2/Makefile
and building ext2 as a module.
> It shouldn't, I kinda deliberately decided to keep the old syntax for
> this. So it would be a bug.
Ah, OK -- sorry.
--
dwmw2
On Mon, 27 Jan 2003, David Woodhouse wrote:
> Er, I think O_TARGET is in fact the target _module_ name when building
> a file system as a module. Try removing O_TARGET from 2.4 fs/ext2/Makefile
> and building ext2 as a module.
Well, okay, there's two ways you can build a single module in a directory
in 2.4, one is to have
O_TARGET := module.o
obj-m := $(O_TARGET)
obj-y := part1.o part2.o
The other is
O_TARGET := something.o
obj-m := module.o
module-objs := part1.o part2.o
(plus a link rule for module.o)
In 2.5, only the second way is legal, so if you're aiming for a compatible
Makefile, you'd use that one, and then O_TARGET shouldn't matter for
"make modules".
--Kai
On Sun, Jan 26, 2003 at 03:46:30PM -0600, Kai Germaschewski wrote:
> it have added -DKBUILD_BASENAME and -DKBUILD_MODNAME, which are required
> by the new module code. And, how did they avoid subtle breakage like not
> giving the same switches on the command line? (This list goes on...)
As opposed to the kernel forcing subtle breakage by specifying
options that break said module? Don't say that doesn't happen (just
like the kernel has to add/remove compiler switches to make some of its
code work).
> Okay, you have a point here, there's still a bug. vermagic.o will be
> rebuilt when the version changes or any of the recorded config options
> change, but it doesn't pick up changes in the compiler version, if the
> new gcc has the same name.
IOW, you not only need a kernel source tree (built, no less,
taking up space) but it needs to be writable!
> o One thing I do not understand at all: What is the problem with using
> the internal build system? It makes maintainance of external modules
> much easier than keeping track of what happens in the kernel and
> patching a private solution all the time.
Well, the Red Hat kernel tree is designed to allow you to build
against there UP, SMP, and large memory kernels from one tree. In your
example, you require three(!) fully built kernel trees lying around, each
with a different configuration. That's a lot of space.
Granted, Red Hat (as our example) can do what they have to do,
since the generic kernel isn't responsible for whatever Red Hat does.
However, even without their setup, in 2.4 all I have to do is keep
around version.h and autoconf.h and I can usually do the right thing
with one completely clean kernel tree.
Joel
--
"And yet I fight,
And yet I fight this battle all alone.
No one to cry to;
No place to call home."
Joel Becker
Senior Member of Technical Staff
Oracle Corporation
E-mail: [email protected]
Phone: (650) 506-8127
On Mon, 27 Jan 2003, Joel Becker wrote:
> On Sun, Jan 26, 2003 at 03:46:30PM -0600, Kai Germaschewski wrote:
> > it have added -DKBUILD_BASENAME and -DKBUILD_MODNAME, which are required
> > by the new module code. And, how did they avoid subtle breakage like not
> > giving the same switches on the command line? (This list goes on...)
>
> As opposed to the kernel forcing subtle breakage by specifying
> options that break said module? Don't say that doesn't happen (just
> like the kernel has to add/remove compiler switches to make some of its
> code work).
Well, if you're doing things in your module which break with the command
line options the rest of the kernel is using, I'd claim you're playing
tricks in your module which you shouldn't. The only place I'm aware of
where someone really needs to change the options is kernel/sched.c, and
that's a very internal part of the kernel, an area which surely shouldn't
be touched by an external module. (Of course mere adding of -I -Dmacro
happens quite a bit, you can do that for your module as you like)
> > Okay, you have a point here, there's still a bug. vermagic.o will be
> > rebuilt when the version changes or any of the recorded config options
> > change, but it doesn't pick up changes in the compiler version, if the
> > new gcc has the same name.
>
> IOW, you not only need a kernel source tree (built, no less,
> taking up space) but it needs to be writable!
Basically, yes. The build process needs to be able to write, e.g. to
compile its helper code in scripts/, so init/vermagic.o is just another
file being written.
I guess you'll hate me even more with CONFIG_MODVERSIONING coming back
soon. Because that means that we need to generate the versions checksum
from all the kernel objects, and we need to write the somewhere, too. In
fact, these checksums are generated as part of the compiled objects, so
recording checksums needs all other compiled objects to be around. If you
know a sensible way to get around this limitation, I'd be happy to hear
it, but I fear there isn't.
So it boils down to the fact that we need a full writeable kernel tree. I
agree that this is asking a lot, and I'm sure open for suggestions. The
writeable part can and will be lifted using separate object directories,
which means that the source can remain read-only and you can specify your
own writeable object tree.
> > o One thing I do not understand at all: What is the problem with using
> > the internal build system? It makes maintainance of external modules
> > much easier than keeping track of what happens in the kernel and
> > patching a private solution all the time.
>
> Well, the Red Hat kernel tree is designed to allow you to build
> against there UP, SMP, and large memory kernels from one tree. In your
> example, you require three(!) fully built kernel trees lying around, each
> with a different configuration. That's a lot of space.
As I said, I am sure interested in working with people and distros to get
something which everybody can live with. I'm wondering how RedHat manages
to have one tree for different configurations, since in that case, at
least .config/autoconf.h, EXTRAVERSION and the module version files
(*.ver) need to differ, so that kinda seems not possible in one
(read-only) tree.
Generally, you need to have the configured source around for the kernel
you want to build the module against. If that's three kernels, you need
the source three times. (Or, of course, you reconfigure it between the
builds)
> Granted, Red Hat (as our example) can do what they have to do,
> since the generic kernel isn't responsible for whatever Red Hat does.
> However, even without their setup, in 2.4 all I have to do is keep
> around version.h and autoconf.h and I can usually do the right thing
> with one completely clean kernel tree.
And that's with module versioning?
--Kai
The solution
make -C KERNEL_SOURCE SUBDIRS=$PWD modules
is fine, but you have to have a Makefile in the current directory,
and that Makefile needs a somewhat different form for 2.4 and
2.5 kernels. Here is a quick and dirty script for generating
a quick boilerplate for the sources in that directory. Straightforward
to customize and extend etc. (You may have to change some whitespace
to TABS for make...)
======================================================================
Jerry Cooperstein, Senior Consultant, <[email protected]>
Axian, Inc., Software Consulting and Training
4800 SW Griffith Dr., Ste. 202, Beaverton, OR 97005 USA
http://www.axian.com/
======================================================================
#!/bin/bash
# script for generating external Makefile for kernel modules
# Jerry Cooperstein, Axian Inc 2003_01_27
# Too trivial to GPL; use as desired.
# do either "makeit"; assumes kernel source at /usr/src/linux-`uname -r`
# or "makeit /usr/src/linux-2.5.59 or makeit /usr/src/linux-2.4.20" etc.
# Should work on all 2.4, 2.5 kernels.
# assumes all .c files in current directory are modules
if [ "$1" == "" ] ; then
KROOT=/usr/src/linux-`uname -r`
else
KROOT="$1"
fi
if [ `echo $KROOT | grep 2.5` ] ; then
VERSION=2.5
else
VERSION=2.4
fi
OBJS=""
for names in *.c ; do
OBJS=$OBJS" `basename $names .c`.o"
done
PHONYS="$OBJS"
rm -f Makefile
cat <<EOF > Makefile
obj-m += $OBJS
KROOT=$KROOT
all: mmodules
EOF
if [ $VERSION == 2.5 ] ; then
CLEANSTUFF="*.o *.ko .*cmd"
cat <<EOF >> Makefile
PHONYS=$PHONYS
.PHONY: \$(PHONYS) clean
EOF
else
CLEANSTUFF="*.o .*flags"
cat <<EOF >> Makefile
TOPDIR=\$(KROOT)
include \$(TOPDIR)/Rules.make
.PHONY: clean
EOF
fi
cat <<EOF >> Makefile
mmodules:
\$(MAKE) -C \$(KROOT) SUBDIRS=$PWD modules
clean:
rm $CLEANSTUFF
EOF
On Mon, Jan 27, 2003 at 10:52:28AM -0800, Jerry Cooperstein wrote:
> The solution
>
> make -C KERNEL_SOURCE SUBDIRS=$PWD modules
>
> is fine, but you have to have a Makefile in the current directory,
> and that Makefile needs a somewhat different form for 2.4 and
> 2.5 kernels.
Notes solely to the Kernel 2.5 part of your script:
1) You do not need any targets like all: or mmodules:
2) The clean: rule will not be used in 2.5, use clean-files instead.
3) No reason to special case on .ko or not. .ko will not harm on 2.4
4) I cannot see any reason to declare all OBJS as .PHONY
5) It looks like you try to enable the use of make in the directory
where the module exists.
This is an unusual extension, which will clutter up the makefile
a bit.
I would prefer a mmake script or similar that did the make -c ... trick.
Sam
On Mon, Jan 27, 2003 at 08:12:25PM +0100, Sam Ravnborg wrote:
>
> 1) You do not need any targets like all: or mmodules:
> 2) The clean: rule will not be used in 2.5, use clean-files instead.
> 3) No reason to special case on .ko or not. .ko will not harm on 2.4
> 4) I cannot see any reason to declare all OBJS as .PHONY
> 5) It looks like you try to enable the use of make in the directory
> where the module exists.
> This is an unusual extension, which will clutter up the makefile
> a bit.
> I would prefer a mmake script or similar that did the make -c ... trick.
>
> Sam
on 4): yep, no need for PHONY stuff (simplified script attached)
The reason for this cropping in is that I had a more complex script
that reflected my actual situation; I have a directory that contains
some kernel modules, some testing programs and other junk, and I
build a complete Makefile that has rules for all these guys, not
just the kernel modules. Which is why I violated your sensibilties
about the other points I guess. I didn't just want to pass the
work off to the kernel building stuff.
If you want to do this I agree all you need is the part to generate
a objs-m list and the Rules.make stuff for 2.4, and that is
simpler. You could indeed remove all the targets etc, but then
as you note, you can't just say make, but have to do the
make -C KERNEL_SOURCE SUBDIRS=$PWD modules
business.
I've attached that one too.
coop
======================================================================
Jerry Cooperstein, Senior Consultant, <[email protected]>
Axian, Inc., Software Consulting and Training
4800 SW Griffith Dr., Ste. 202, Beaverton, OR 97005 USA
http://www.axian.com/
======================================================================
#(FULL SCRIPT, NO PHONY):
######################################################################
#!/bin/bash
# script for generating external Makefile for kernel modules
# Jerry Cooperstein, Axian Inc 2003_01_27
# Too trivial to GPL; use as desired.
# do either "makeit"; assumes kernel source at /usr/src/linux-`uname -r`
# or "makeit /usr/src/linux-2.5.59 or makeit /usr/src/linux-2.4.20" etc.
# Should work on all 2.4, 2.5 kernels.
# assumes all .c files in current directory are modules
if [ "$1" == "" ] ; then
KROOT=/usr/src/linux-`uname -r`
else
KROOT="$1"
fi
if [ `echo $KROOT | grep 2.5` ] ; then
VERSION=2.5
else
VERSION=2.4
fi
OBJS=""
for names in *.c ; do
OBJS=$OBJS" `basename $names .c`.o"
done
rm -f Makefile
cat <<EOF > Makefile
obj-m += $OBJS
KROOT=$KROOT
all: mmodules
EOF
if [ $VERSION == 2.5 ] ; then
CLEANSTUFF="*.o *.ko .*cmd"
cat <<EOF >> Makefile
EOF
else
CLEANSTUFF="*.o .*flags"
cat <<EOF >> Makefile
TOPDIR=\$(KROOT)
include \$(TOPDIR)/Rules.make
EOF
fi
cat <<EOF >> Makefile
mmodules:
\$(MAKE) -C \$(KROOT) SUBDIRS=$PWD modules
clean:
rm $CLEANSTUFF
EOF
######################################################################
#(SHORT SCRIPT: no local make targets)
#!/bin/bash
# script for generating external Makefile for kernel modules
# Jerry Cooperstein, Axian Inc 2003_01_27
# Too trivial to GPL; use as desired.
# do either "makeit"; assumes kernel source at /usr/src/linux-`uname -r`
# or "makeit /usr/src/linux-2.5.59 or makeit /usr/src/linux-2.4.20" etc.
# Should work on all 2.4, 2.5 kernels.
# assumes all .c files in current directory are modules
if [ "$1" == "" ] ; then
KROOT=/usr/src/linux-`uname -r`
else
KROOT="$1"
fi
if [ `echo $KROOT | grep 2.5` ] ; then
VERSION=2.5
else
VERSION=2.4
fi
OBJS=""
for names in *.c ; do
OBJS=$OBJS" `basename $names .c`.o"
done
rm -f Makefile
cat <<EOF > Makefile
obj-m += $OBJS
KROOT=$KROOT
EOF
if [ $VERSION == 2.5 ] ; then
cat <<EOF >> Makefile
EOF
else
cat <<EOF >> Makefile
TOPDIR=\$(KROOT)
include \$(TOPDIR)/Rules.make
EOF
fi
Jerry Cooperstein <[email protected]> writes:
> is fine, but you have to have a Makefile in the current directory,
> and that Makefile needs a somewhat different form for 2.4 and
> 2.5 kernels.
It is no problem to use the same Makefile for both 2.4.x and 2.5.x,
and even let the Makefile do the RightThing[tm] in case the user just
types 'make'. My Makefiles for kernel stuff look like this (stripped
down a bit):
==============================[ cut here ]==============================
ifneq ($(KERNELRELEASE),)
# call from kernel build system
obj-m := btaudio.o
-include $(TOPDIR)/Rules.make
else
KDIR := /lib/modules/$(shell uname -r)/build
PWD := $(shell pwd)
default:
$(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules
endif
==============================[ cut here ]==============================
If you need different stuff for 2.4.x and 2.5.x you can handle it
using ifeq ($(VERSION).$(PATCHLEVEL),2.4)
Gerd
On Mon, Jan 27, 2003 at 12:31:07PM -0600, Kai Germaschewski wrote:
> Well, if you're doing things in your module which break with the command
> line options the rest of the kernel is using, I'd claim you're playing
> tricks in your module which you shouldn't. The only place I'm aware of
I'm not so sure about that. Some gcc things tweak us, and the
some code has had to deal with it. This isn't something that happens
often, but it still can. In addition, CFLAGS_filename.o does not allow
removal of options, merely the addition if I am not mistaken.
> Basically, yes. The build process needs to be able to write, e.g. to
> compile its helper code in scripts/, so init/vermagic.o is just another
> file being written.
If my distribution has installed /usr/src/linux-x.y, I can't
compile against it. Even though the 200MB of a kernel tree is already
taking up space on my system, I have to download *another* 30MB and
install it as *another* 200MB and build it to an eventual *another*
260MB of kernel tree. So, for every kernel I want to support, I have to
have 260MB of built tree. And that's just for my userid. Anyone else
on the box has to have their own n_kernels * 260MB of space waste.
> fact, these checksums are generated as part of the compiled objects, so
> recording checksums needs all other compiled objects to be around. If you
But, once the checksums are recorded, the compiled objects are
no longer needed, no? It still remains that a kernel header package
with associated correct autoconf.h and checksums is at least an order of
magnitude smaller than a built kernel tree.
> As I said, I am sure interested in working with people and distros to get
> something which everybody can live with. I'm wondering how RedHat manages
> to have one tree for different configurations, since in that case, at
> least .config/autoconf.h, EXTRAVERSION and the module version files
> (*.ver) need to differ, so that kinda seems not possible in one
> (read-only) tree.
Red Hat plays tricks. They add a <rhconfig.h> to the top of
autoconf.h and have some extra defines so that chunks of autoconf.h look
like:
#ifdef UP_FLAG
... some UP CONFIG_* options
#else
#ifdef SMP_FLAG
... some SMP CONFIG_* options
and so on.
This does indeed track modversions as well (I don't recall which
files do the switching). This actually works pretty well, but it depends
on the fact that their kernel flavours (up, smp, large ram) are known
at the time they build this setup. This isn't necessarily the proper
solution for the generic kernel.
It still remains that in 2.4 you need the headers for the kernel
plus the proper bits created by config/modversions. You don't need
anything else, and you don't need any writability after the initial
generation. This takes significantly less space than an entire built
tree, and is usable from /usr/src as a readonly entity. Requiring that
*each user* have the kernels they wish to build installed and fully
built is a step back, IMHO.
Joel
--
"I always thought the hardest questions were those I could not answer.
Now I know they are the ones I can never ask."
- Charlie Watkins
Joel Becker
Senior Member of Technical Staff
Oracle Corporation
E-mail: [email protected]
Phone: (650) 506-8127
On Mon, 27 Jan 2003, Joel Becker wrote:
> On Mon, Jan 27, 2003 at 12:31:07PM -0600, Kai Germaschewski wrote:
> > Well, if you're doing things in your module which break with the command
> > line options the rest of the kernel is using, I'd claim you're playing
> > tricks in your module which you shouldn't. The only place I'm aware of
>
> I'm not so sure about that. Some gcc things tweak us, and the
> some code has had to deal with it. This isn't something that happens
> often, but it still can. In addition, CFLAGS_filename.o does not allow
> removal of options, merely the addition if I am not mistaken.
Well, I suppose arguing about that without a concrete example is kinda
pointless.
> > Basically, yes. The build process needs to be able to write, e.g. to
> > compile its helper code in scripts/, so init/vermagic.o is just another
> > file being written.
>
> If my distribution has installed /usr/src/linux-x.y, I can't
> compile against it. Even though the 200MB of a kernel tree is already
> taking up space on my system, I have to download *another* 30MB and
> install it as *another* 200MB and build it to an eventual *another*
> 260MB of kernel tree. So, for every kernel I want to support, I have to
> have 260MB of built tree. And that's just for my userid. Anyone else
> on the box has to have their own n_kernels * 260MB of space waste.
You ignored the fact that I said you will be able to use separate
src/objdir, which means you can have your source read-only.
> > fact, these checksums are generated as part of the compiled objects, so
> > recording checksums needs all other compiled objects to be around. If you
>
> But, once the checksums are recorded, the compiled objects are
> no longer needed, no? It still remains that a kernel header package
> with associated correct autoconf.h and checksums is at least an order of
> magnitude smaller than a built kernel tree.
Yes, all you really need is the checksums. Then again, you also want a way
to verify a way that the checksums match the ABI as determined by the
current .config. I mean, just using the previously recorded checksums
without verifying is kinda pointless, they'll just always match and not
fulfill their function.
> > As I said, I am sure interested in working with people and distros to get
> > something which everybody can live with. I'm wondering how RedHat manages
> > to have one tree for different configurations, since in that case, at
> > least .config/autoconf.h, EXTRAVERSION and the module version files
> > (*.ver) need to differ, so that kinda seems not possible in one
> > (read-only) tree.
>
> Red Hat plays tricks. They add a <rhconfig.h> to the top of
> autoconf.h and have some extra defines so that chunks of autoconf.h look
> like:
>
> #ifdef UP_FLAG
> ... some UP CONFIG_* options
> #else
> #ifdef SMP_FLAG
> ... some SMP CONFIG_* options
>
> and so on.
>
> This does indeed track modversions as well (I don't recall which
> files do the switching). This actually works pretty well, but it depends
> on the fact that their kernel flavours (up, smp, large ram) are known
> at the time they build this setup. This isn't necessarily the proper
> solution for the generic kernel.
> It still remains that in 2.4 you need the headers for the kernel
> plus the proper bits created by config/modversions. You don't need
> anything else, and you don't need any writability after the initial
> generation. This takes significantly less space than an entire built
> tree, and is usable from /usr/src as a readonly entity. Requiring that
> *each user* have the kernels they wish to build installed and fully
> built is a step back, IMHO.
Yup, I now looked into what Redhat does. It's an obvious sign that there
is work to be done, in particular making the build system work in a way
that vendors don't need to kludge around it would definitely be nice.
I have to admit that I did not think about the needs of distro vendors
when implementing the new module version code (the other small issues I
think can be worked around easily enough), and it's definitely an area
which needs serious thinking and improvement. However, I think I need to
finish the current work first, i.e. make sure the module versions actually
work and then the separate obj / src dir stuff.
Afterwards I think it's a really good idea to come up with a way to
sensibly handle external modules, and it'd be much appreciated when all of
the affected people (i.e. distro and external module guys) would provide
input for that.
--Kai
On Mon, Jan 27, 2003 at 05:08:50PM -0600, Kai Germaschewski wrote:
> Well, I suppose arguing about that without a concrete example is kinda
> pointless.
Fair enough.
> You ignored the fact that I said you will be able to use separate
> src/objdir, which means you can have your source read-only.
Yes. That still doesn't change the fact that in 2.4 I need the
headers. In 2.5 I need all 200MB of source + 60MB per different .config
of built tree. And that's per user using that source.
> Yes, all you really need is the checksums. Then again, you also want a way
> to verify a way that the checksums match the ABI as determined by the
> current .config. I mean, just using the previously recorded checksums
> without verifying is kinda pointless, they'll just always match and not
> fulfill their function.
You'd obviously have to have a way in the readonly /usr/src bits
to make sure that autoconf.h and the checksums match. That would have
to be enforced (as Red Hat does now with their method).
> Yup, I now looked into what Redhat does. It's an obvious sign that there
> is work to be done, in particular making the build system work in a way
> that vendors don't need to kludge around it would definitely be nice.
Maybe a way to track the differnet builds. I'm not sure. But
say you had a way of keeping multiple autoconf.h files in a tree. If
the kernel sources could be smart enough to figure out the version.h and
modversions locations from that path, then you could have an external
module create a kbuild makefile for itself and run something like:
make KERN_CONF=/usr/src/linux-2.6.1/include/autoconf/build-smp/autoconf.h modules
where kbuild figures out from KERN_CONF that modversions stuff is in
autoconf/buid-smp/modversions/ and version.h is in
autoconf/build-smp/version.h, etc. Something like that (I can beef with
this example, so no one tell me exactly why this specific example is bad
:-)
> which needs serious thinking and improvement. However, I think I need to
> finish the current work first, i.e. make sure the module versions actually
> work and then the separate obj / src dir stuff.
Yes, I understand that there are priorities.
Joel
--
"The cynics are right nine times out of ten."
- H. L. Mencken
Joel Becker
Senior Member of Technical Staff
Oracle Corporation
E-mail: [email protected]
Phone: (650) 506-8127
In message <[email protected]> you write:
> Can't the stuff in init/vermagic.c be moved into a header file? Maybe
> vermagic.h? Most of the code can be cut 'n pasted right out of vermagic.c
> and the bit that defines "const char vermagic[]..." could be placed inside a
> macro which modules would then stick in the bottom of one of their c files.
And then you'll die horribly next time Kai or I change the way modules
are built.
Really, using the Makefiles is always the most future-proof way!
Hope that helps!
Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.
[email protected] said:
> If my distribution has installed /usr/src/linux-x.y, I can't compile
> against it. Even though the 200MB of a kernel tree is already taking
> up space on my system, I have to download *another* 30MB and install
> it as *another* 200MB and build it to an eventual *another* 260MB of
> kernel tree. So, for every kernel I want to support, I have to have
> 260MB of built tree. And that's just for my userid. Anyone else on
> the box has to have their own n_kernels * 260MB of space waste.
Er, if vermagic.o needed to change, then your module build was broken
already and wouldn't have worked against the precompiled kernel. That's
what vermagic.o is there for. You shouldn't need write permissions to the
kernel source tree. Neither do you need _all_ of the kernel source;
just the headers and appropriate bits of infrastructure.
--
dwmw2
On Tue, Jan 28, 2003 at 03:43:57PM +0000, David Woodhouse wrote:
> Er, if vermagic.o needed to change, then your module build was broken
> already and wouldn't have worked against the precompiled kernel. That's
If you've precompiled the kernel, you have a full kernel built
tree. If you haven't, then vermagic.o isn't there. Plus, modversions
requires the objects according to kai.
In 2.4, a "make config" and "make dep" pretty much does what is
needed. The kernel is ready to be used by external modules.
Joel
--
"You can get more with a kind word and a gun than you can with
a kind word alone."
- Al Capone
Joel Becker
Senior Member of Technical Staff
Oracle Corporation
E-mail: [email protected]
Phone: (650) 506-8127
On Tue, Jan 28, 2003 at 12:58:40PM +1100, Rusty Russell wrote:
> And then you'll die horribly next time Kai or I change the way modules
> are built.
>
> Really, using the Makefiles is always the most future-proof way!
I've actually converted my stuff to use the Makefiles, which does simplify
things, but the problem I had was mainly with the whole vermagic.c stuff and
needing a full source tree + compiled objects (or having the tree writeable
by users who want to compile modules) around just to build my module.
Anyway, Joel Becker, Christian Zander and others have eloquently voiced my
complaints previously in this thread, so I won't go over them again :)
As far as using the kernel build system, so far so good - it's nice, and
builds things correctly though I'm having some trouble getting it to
install my module correctly, which may just indicate that I need to re-read
the docs.
--Mark
--
Mark Fasheh
Software Developer, Oracle Corp
[email protected]
On Tue, 28 Jan 2003, Mark Fasheh wrote:
> On Tue, Jan 28, 2003 at 12:58:40PM +1100, Rusty Russell wrote:
> > And then you'll die horribly next time Kai or I change the way modules
> > are built.
> >
> > Really, using the Makefiles is always the most future-proof way!
> I've actually converted my stuff to use the Makefiles, which does simplify
> things, but the problem I had was mainly with the whole vermagic.c stuff and
> needing a full source tree + compiled objects (or having the tree writeable
> by users who want to compile modules) around just to build my module.
> Anyway, Joel Becker, Christian Zander and others have eloquently voiced my
> complaints previously in this thread, so I won't go over them again :)
>
> As far as using the kernel build system, so far so good - it's nice, and
> builds things correctly though I'm having some trouble getting it to
> install my module correctly, which may just indicate that I need to re-read
> the docs.
The docs are kinda out-of-date, expect an update (thanks to Sam Ravnborg)
soon. However, there aren't provisions for installing third party modules
yet, so that's on my todo list together with the points above.
--Kai