2004-06-03 16:58:18

by Andreas Gruenbacher

[permalink] [raw]
Subject: [PATCH] Symlinks for building external modules

Hi Sam,

modules not in the kernel source tree need to locate both the source tree and
the object tree (O=). Currently, the /lib/modules/$(uname -r)/build symlink
is the only reference we have; it historically points to the source tree from
2.4 times. The following patch changes this as follows (this is what we have
in the current SUSE tree now):

/lib/modules/$(uname -r)/source ==> source tree
/lib/modules/$(uname -r)/build ==> object tree

Both links are required for building external modules with:

make -C /lib/modules/$(uname -r)/source \
O=/lib/modules/$(uname -r)/build \
M=$(pwd)


Index: linux-2.6.7-rc2/Makefile
===================================================================
--- linux-2.6.7-rc2.orig/Makefile
+++ linux-2.6.7-rc2/Makefile
@@ -732,9 +732,10 @@ _modinst_:
sleep 1; \
fi
@rm -rf $(MODLIB)/kernel
- @rm -f $(MODLIB)/build
+ @rm -f $(MODLIB)/{source,build}
@mkdir -p $(MODLIB)/kernel
- @ln -s $(TOPDIR) $(MODLIB)/build
+ @ln -s $(srctree) $(MODLIB)/source
+ @ln -s $(objtree) $(MODLIB)/build
$(Q)$(MAKE) -rR -f $(srctree)/scripts/Makefile.modinst

# If System.map exists, run depmod. This deliberately does not have a


Thanks,
--
Andreas Gruenbacher <[email protected]>
SUSE Labs, SUSE LINUX AG


2004-06-03 17:10:22

by Måns Rullgård

[permalink] [raw]
Subject: Re: [PATCH] Symlinks for building external modules

Andreas Gruenbacher <[email protected]> writes:

> Hi Sam,
>
> modules not in the kernel source tree need to locate both the source
> tree and the object tree (O=). Currently, the /lib/modules/$(uname
> -r)/build symlink is the only reference we have; it historically
> points to the source tree from 2.4 times. The following patch
> changes this as follows (this is what we have in the current SUSE
> tree now):
>
> /lib/modules/$(uname -r)/source ==> source tree
> /lib/modules/$(uname -r)/build ==> object tree

This will break the building of all external modules until they are
updated, and break updated modules building against older kernels
unless they check the kernel version in the makefiles.. I suggest
leaving the 'build' link as is, and using a difference name for the
build directory, perhaps 'object'. This might look confusing, so we
could have a 'source' link as well and remove the 'build' link when
most external modules have been updated.

--
M?ns Rullg?rd
[email protected]

2004-06-03 17:38:40

by Sam Ravnborg

[permalink] [raw]
Subject: Re: [PATCH] Symlinks for building external modules

On Thu, Jun 03, 2004 at 07:09:42PM +0200, M?ns Rullg?rd wrote:
> Andreas Gruenbacher <[email protected]> writes:
>
> > Hi Sam,
> >
> > modules not in the kernel source tree need to locate both the source
> > tree and the object tree (O=). Currently, the /lib/modules/$(uname
> > -r)/build symlink is the only reference we have; it historically
> > points to the source tree from 2.4 times. The following patch
> > changes this as follows (this is what we have in the current SUSE
> > tree now):
> >
> > /lib/modules/$(uname -r)/source ==> source tree
> > /lib/modules/$(uname -r)/build ==> object tree
>
> This will break the building of all external modules until they are
> updated, and break updated modules building against older kernels
> unless they check the kernel version in the makefiles.. I suggest
> leaving the 'build' link as is, and using a difference name for the
> build directory, perhaps 'object'. This might look confusing, so we
> could have a 'source' link as well and remove the 'build' link when
> most external modules have been updated.

The existing external modules are anyway broken when using separate
directories for source and output directories. So noting lost here.
In the case where the kernel is build in the traditional way
the build and source tree will point to the same place.

So I do not see this patch breaking existing setups, but I see
external modules not being prepared for separate build and source
directories.

Patch looks good to me, and I will forward to Andrew soon.

Sam

2004-06-03 17:55:42

by Måns Rullgård

[permalink] [raw]
Subject: Re: [PATCH] Symlinks for building external modules

Sam Ravnborg <[email protected]> writes:

> On Thu, Jun 03, 2004 at 07:09:42PM +0200, M?ns Rullg?rd wrote:
>> Andreas Gruenbacher <[email protected]> writes:
>>
>> > Hi Sam,
>> >
>> > modules not in the kernel source tree need to locate both the source
>> > tree and the object tree (O=). Currently, the /lib/modules/$(uname
>> > -r)/build symlink is the only reference we have; it historically
>> > points to the source tree from 2.4 times. The following patch
>> > changes this as follows (this is what we have in the current SUSE
>> > tree now):
>> >
>> > /lib/modules/$(uname -r)/source ==> source tree
>> > /lib/modules/$(uname -r)/build ==> object tree
>>
>> This will break the building of all external modules until they are
>> updated, and break updated modules building against older kernels
>> unless they check the kernel version in the makefiles.. I suggest
>> leaving the 'build' link as is, and using a difference name for the
>> build directory, perhaps 'object'. This might look confusing, so we
>> could have a 'source' link as well and remove the 'build' link when
>> most external modules have been updated.
>
> The existing external modules are anyway broken when using separate
> directories for source and output directories.

They work fine if you pass O=/some/path on the make command line.

> So noting lost here. In the case where the kernel is build in the
> traditional way the build and source tree will point to the same
> place.

True.

> So I do not see this patch breaking existing setups, but I see
> external modules not being prepared for separate build and source
> directories.
>
> Patch looks good to me, and I will forward to Andrew soon.

I'll just have to go fix my kernel modules then.

--
M?ns Rullg?rd
[email protected]

2004-06-04 17:14:30

by Jari Ruusu

[permalink] [raw]
Subject: Re: [PATCH] Symlinks for building external modules

Sam Ravnborg wrote:
> On Thu, Jun 03, 2004 at 07:09:42PM +0200, M?ns Rullg?rd wrote:
> > Andreas Gruenbacher <[email protected]> writes:
> > > modules not in the kernel source tree need to locate both the source
> > > tree and the object tree (O=). Currently, the /lib/modules/$(uname
> > > -r)/build symlink is the only reference we have; it historically
> > > points to the source tree from 2.4 times. The following patch
> > > changes this as follows (this is what we have in the current SUSE
> > > tree now):
> > >
> > > /lib/modules/$(uname -r)/source ==> source tree
> > > /lib/modules/$(uname -r)/build ==> object tree
> >
> > This will break the building of all external modules until they are
> > updated, and break updated modules building against older kernels
> > unless they check the kernel version in the makefiles.. I suggest
> > leaving the 'build' link as is, and using a difference name for the
> > build directory, perhaps 'object'. This might look confusing, so we
> > could have a 'source' link as well and remove the 'build' link when
> > most external modules have been updated.
>
> The existing external modules are anyway broken when using separate
> directories for source and output directories. So noting lost here.

Wrong! Existing external modules build fine when specified an object
directory.

> In the case where the kernel is build in the traditional way
> the build and source tree will point to the same place.
>
> So I do not see this patch breaking existing setups, but I see
> external modules not being prepared for separate build and source
> directories.

How long have you recommended building external modules like this:

make -C /lib/modules/`uname -r`/build modules SUBDIRS=`pwd`
or
make -C /lib/modules/`uname -r`/build modules M=`pwd`

Now they all have to be changed to:

make -C /lib/modules/`uname -r`/source modules SUBDIRS=`pwd`
or
make -C /lib/modules/`uname -r`/source modules M=`pwd`

And the worst part is that modules that have to support old and new
kernel versions have to first detect which method to use, and
select method that applies to that paricular kernel version.

> Patch looks good to me, and I will forward to Andrew soon.

That patch only breaks stuff. Please use the 'object' symlink
suggested by M?ns Rullg?rd. That does not break anything.

--
Jari Ruusu 1024R/3A220F51 5B 4B F9 BB D3 3F 52 E9 DB 1D EB E3 24 0E A9 DD

2004-06-04 19:19:14

by Sam Ravnborg

[permalink] [raw]
Subject: Re: [PATCH] Symlinks for building external modules

On Fri, Jun 04, 2004 at 08:15:02PM +0300, Jari Ruusu wrote:
>
> How long have you recommended building external modules like this:
>
> make -C /lib/modules/`uname -r`/build modules SUBDIRS=`pwd`
> or
> make -C /lib/modules/`uname -r`/build modules M=`pwd`
>
> Now they all have to be changed to:
>
> make -C /lib/modules/`uname -r`/source modules SUBDIRS=`pwd`
> or
> make -C /lib/modules/`uname -r`/source modules M=`pwd`

That would not work either.
You need to tell kbuild both where to find the source and the output files.
So you have to specify both -C ... and O=...
KERNEL=/lib/modules/`uname -r`
make -C $KERNEL/source O=$KERNEL/build M=`pwd`

There is no way to do the proposed chang and be backward compatible
when the kernel is build using seperate output directories.

Think of the following situation
/usr/src/linux-2.6.6-xx/ <= kernel src

/lib/modules/linux-2.6.6-xx-smp/build/ <= output files (or a symlink to them)
/lib/modules/linux-2.6.6-xx-smp/source/ <= symlink to kernel src

/lib/modules/linux-2.6.6-xx-up/build/ <= output files (or symlink)
/lib/modules/linux-2.6.6-xx-up/source/ <= symlink to kernel src

/lib/modules/linux-2.6.6-xx-4g/build/ <= output files (or symlink)
/lib/modules/linux-2.6.6-xx-4g/source/ <= symlink to kernel src

Notice they all share the _same_ kernel src.
We just have three different .config files.

If there is a way to be backward compatible with
this feature please demonstrate it.

Plese note that the patch Andreas made did not break existing setups
if a seperate output directory was not used. The only effect
would be an additional symlink to the same dir. (build and source would
be links to the same dir).


Andreas - please expalin why you want build to be a symlink, and not
the directory used when actually building the kernel.

Sam

2004-06-04 19:45:37

by Måns Rullgård

[permalink] [raw]
Subject: Re: [PATCH] Symlinks for building external modules

Sam Ravnborg <[email protected]> writes:

> On Fri, Jun 04, 2004 at 08:15:02PM +0300, Jari Ruusu wrote:
>>
>> How long have you recommended building external modules like this:
>>
>> make -C /lib/modules/`uname -r`/build modules SUBDIRS=`pwd`
>> or
>> make -C /lib/modules/`uname -r`/build modules M=`pwd`
>>
>> Now they all have to be changed to:
>>
>> make -C /lib/modules/`uname -r`/source modules SUBDIRS=`pwd`
>> or
>> make -C /lib/modules/`uname -r`/source modules M=`pwd`
>
> That would not work either.
> You need to tell kbuild both where to find the source and the output files.
> So you have to specify both -C ... and O=...
> KERNEL=/lib/modules/`uname -r`
> make -C $KERNEL/source O=$KERNEL/build M=`pwd`
>
> There is no way to do the proposed chang and be backward compatible
> when the kernel is build using seperate output directories.

There are many external module packages that assume
/lib/modules/`uname -r`/build to contain the kernel sources. These
can be built with

make O=/some/path

If the content of /lib/modules/`uname -r`/build changes this will no
longer work.

> Think of the following situation
> /usr/src/linux-2.6.6-xx/ <= kernel src
>
> /lib/modules/linux-2.6.6-xx-smp/build/ <= output files (or a symlink to them)
> /lib/modules/linux-2.6.6-xx-smp/source/ <= symlink to kernel src
>
> /lib/modules/linux-2.6.6-xx-up/build/ <= output files (or symlink)
> /lib/modules/linux-2.6.6-xx-up/source/ <= symlink to kernel src
>
> /lib/modules/linux-2.6.6-xx-4g/build/ <= output files (or symlink)
> /lib/modules/linux-2.6.6-xx-4g/source/ <= symlink to kernel src
>
> Notice they all share the _same_ kernel src.
> We just have three different .config files.
>
> If there is a way to be backward compatible with
> this feature please demonstrate it.
>
> Plese note that the patch Andreas made did not break existing setups
> if a seperate output directory was not used. The only effect
> would be an additional symlink to the same dir. (build and source would
> be links to the same dir).
>
> Andreas - please expalin why you want build to be a symlink, and not
> the directory used when actually building the kernel.

I can't speak for Andreas, but I prefer to keep my root filesystem as
clean as possible. Often it's mounted read-only.

--
M?ns Rullg?rd
[email protected]

2004-06-05 13:19:48

by Jari Ruusu

[permalink] [raw]
Subject: Re: [PATCH] Symlinks for building external modules

Sam Ravnborg wrote:
> On Fri, Jun 04, 2004 at 08:15:02PM +0300, Jari Ruusu wrote:
> > How long have you recommended building external modules like this:
> >
> > make -C /lib/modules/`uname -r`/build modules SUBDIRS=`pwd`
> > or
> > make -C /lib/modules/`uname -r`/build modules M=`pwd`
> >
> > Now they all have to be changed to:
> >
> > make -C /lib/modules/`uname -r`/source modules SUBDIRS=`pwd`
> > or
> > make -C /lib/modules/`uname -r`/source modules M=`pwd`
>
> That would not work either.
> You need to tell kbuild both where to find the source and the output files.
> So you have to specify both -C ... and O=...
> KERNEL=/lib/modules/`uname -r`
> make -C $KERNEL/source O=$KERNEL/build M=`pwd`
>
> There is no way to do the proposed chang and be backward compatible
> when the kernel is build using seperate output directories.

What I and M?ns Rullg?rd are saying is this: A symlink to object directory
is missing. The fix is to add the damn symlink. However, breaking symlink to
source directory at the same time is unacceptable.

In other words:
build -> Symlink to kernel sources and build machinery, Makefiles and
scripts. No change here.
object -> Symlink to separate object tree. This is new.

Existing functionality when using separate source and object directories is
that only the 'build' symlink exists, and that points to kernel build
machinery. And that 'build' symlink must stay that way.

> Plese note that the patch Andreas made did not break existing setups
> if a seperate output directory was not used. The only effect
> would be an additional symlink to the same dir. (build and source would
> be links to the same dir).

All code that uses the 'build' symlink needs to be changed if Andreas' patch
is merged to mainline. That is quite breakage.

Sam, you are wrong here. I hope you have enough balls to admit that instead
of forcing everyone to deal with the proposed breakage.

--
Jari Ruusu 1024R/3A220F51 5B 4B F9 BB D3 3F 52 E9 DB 1D EB E3 24 0E A9 DD

2004-06-05 13:43:35

by Andreas Gruenbacher

[permalink] [raw]
Subject: Re: [PATCH] Symlinks for building external modules

On Friday 04 June 2004 21:45, M?ns Rullg?rd wrote:
> Sam Ravnborg <[email protected]> writes:
> > Andreas - please expalin why you want build to be a symlink, and not
> > the directory used when actually building the kernel.
>
> I can't speak for Andreas, but I prefer to keep my root filesystem as
> clean as possible. Often it's mounted read-only.

Either way would work, but I prefer to have the actual files below /usr/src.
The /lib directory is meant for files required during boot, whereas /usr may
be mounted later. If we want to allow both possibilities, we probably want to
add the following check:

Index: linux-2.6.7-rc2/Makefile
===================================================================
--- linux-2.6.7-rc2.orig/Makefile
+++ linux-2.6.7-rc2/Makefile
@@ -735,7 +735,8 @@ _modinst_:
@rm -f $(MODLIB)/{source,build}
@mkdir -p $(MODLIB)/kernel
@ln -s $(srctree) $(MODLIB)/source
- @ln -s $(objtree) $(MODLIB)/build
+ @[ $(objtree) -ef $(MODLIB)/build ] \
+ || ln -s $(objtree) $(MODLIB)/build
$(Q)$(MAKE) -rR -f $(srctree)/scripts/Makefile.modinst


Next, we may want to add a Makefile to the output directory so that external
modules will build the same way they did before, without specifying the
source tree separately:

Index: linux-2.6.7-rc2/Makefile
===================================================================
--- linux-2.6.7-rc2.orig/Makefile
+++ linux-2.6.7-rc2/Makefile
@@ -125,11 +125,19 @@ ifeq ($(skip-makefile),)
# but instead _all depend on modules
.PHONY: all
ifeq ($(KBUILD_EXTMOD),)
-_all: all
+_all: $(objtree)/Makefile all
else
_all: modules
endif

+.PHONY: $(objtree)/Makefile
+$(objtree)/Makefile:
+ $(Q)if [ ! $(srctree) -ef $(objtree) ]; then \
+ ( echo "modules modules_install clean:" ; \
+ echo -e "\t\$$(MAKE) -C $(srctree) \$$@ O=\$$(CURDIR)" \
+ ) > $(objtree)/Makefile ; \
+ fi
+
# Make sure we're not wasting cpu-cycles doing locale handling, yet do make
# sure error messages appear in the user-desired language
ifdef LC_ALL


Cheers,
--
Andreas Gruenbacher <[email protected]>
SUSE Labs, SUSE LINUX AG