Hi Andrew, Linus.
Here follows two kbuild patches.
1) Add generic infrastructure for creating kernel packages.
This moves make rpm support to scripts/package
and add the deb-pkg target.
The infrastructure were added because there is requests
to add .tar.gz, .tar.gz2 support as well, and the functionality
really did not belong in the top-level makefile.
The implementation is simplified based on feedback on first
patch, so one just have to set KBUILD_IMAGE in arch Makefiel
to say what kernel image to include in the package.
2) Improved support for external modules.
It has been debated what to name the symlink in /lib/modules/`uname -r`
and where it should point.
Now that there is a possibility to build the kernel with a separate output
directory, there is a need to utilise this in the install.
From now on build will point to the output directory, and source will point
to the kernel source.
A small MAkefile is created in the output directory allowing external modules
to continue to build independent of the kernel being built with separate
output directory, or with ouput and source mixed.
If the kernel is build with source and output mixed there is no change
in behaviour.
If the kernel is build with separate source and output directories,
all external modules that has not yet picked up on using the kbuild
infrastructure will fail...
No effort whatsoever will be done to keep external modules working if they
do not use the kbuild infrastructure. There is no reason not to do so.
Sam
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/06/19 23:41:15+02:00 [email protected]
# kbuild: Add new package format: deb-pkg
#
# To prepare kbuild for more kernel packaging formats move all packaging
# support to scripts/package.
# In top-level Makefile introduce generic support for all package
# formats using target names *-pkg. Included the old rpm target for backward
# compatibility.
#
# A new variable KBUILD_IMAGE is used to specify what kernel image
# will be part of the final package, and is to be set by the arch specific
# makefile. KBUILD_IMAGE may be overridden from command line or environment.
# KBUILD_IMAGE will see wider usage later, mainly when installing
# kernel images.
# Introducing KBUILD_IMAGE allowed arch specific details to be deleted from the mkspec
# and builddeb scripts.
#
# While in the process added the deb packet format. Script is
# From: Wichert Akkerman <[email protected]>
#
# To create a RPM packet use 'make rpm-pkg'.
# To create a deb packet use 'make deb-pkg'.
# Both targets are included in 'make help'
#
# Signed-off-by: Sam Ravnborg <[email protected]>
#
# scripts/package/builddeb
# 2004/06/19 23:41:00+02:00 [email protected] +79 -0
#
# scripts/package/builddeb
# 2004/06/19 23:41:00+02:00 [email protected] +0 -0
# BitKeeper file /home/sam/bk/kbuild/scripts/package/builddeb
#
# scripts/package/Makefile
# 2004/06/19 23:40:59+02:00 [email protected] +71 -0
#
# scripts/package/mkspec
# 2004/06/19 23:40:59+02:00 [email protected] +6 -15
# KBUILD_IMAGE tell what image is being built.
# Therefore remove hardcoded architecture specific info
#
# scripts/package/Makefile
# 2004/06/19 23:40:59+02:00 [email protected] +0 -0
# BitKeeper file /home/sam/bk/kbuild/scripts/package/Makefile
#
# scripts/Makefile
# 2004/06/19 23:40:59+02:00 [email protected] +1 -1
# Add package/ to dirs being visited during make mrproper
#
# arch/i386/Makefile
# 2004/06/19 23:40:59+02:00 [email protected] +7 -6
# Set KBUILD_IMAGE to kernel image being build.
# Since BOOTIMAGE and KBUILD_IMAGE contained the same info kill
# local version of BOOTIMAGE
#
# Makefile
# 2004/06/19 23:40:59+02:00 [email protected] +25 -37
# Move rpm target to scripts/package
# Added support for more generic package handling
#
# scripts/package/mkspec
# 2004/06/19 14:53:10+02:00 [email protected] +0 -0
# Rename: scripts/mkspec -> scripts/package/mkspec
#
diff -Nru a/Makefile b/Makefile
--- a/Makefile 2004-06-20 23:05:13 +02:00
+++ b/Makefile 2004-06-20 23:05:13 +02:00
@@ -290,8 +290,6 @@
OBJCOPY = $(CROSS_COMPILE)objcopy
OBJDUMP = $(CROSS_COMPILE)objdump
AWK = awk
-RPM := $(shell if [ -x "/usr/bin/rpmbuild" ]; then echo rpmbuild; \
- else echo rpm; fi)
GENKSYMS = scripts/genksyms/genksyms
DEPMOD = /sbin/depmod
KALLSYMS = scripts/kallsyms
@@ -409,13 +407,6 @@
scripts_basic: include/linux/autoconf.h
-
-# That's our default target when none is given on the command line
-# Note that 'modules' will be added as a prerequisite as well,
-# in the CONFIG_MODULES part below
-
-all: vmlinux
-
# Objects we will link into vmlinux / subdirs we need to visit
init-y := init/
drivers-y := drivers/ sound/
@@ -449,6 +440,19 @@
include $(srctree)/arch/$(ARCH)/Makefile
+# Default kernel image to build when no specific target is given.
+# KBUILD_IMAGE may be overruled on the commandline or
+# set in the environment
+# Also any assingments in arch/$(ARCH)/Makefiel take precedence over
+# this default value
+export KBUILD_IMAGE ?= vmlinux
+
+# The all: target is the default when no target is given on the
+# command line.
+# This allow a user to issue only 'make' to build a kernel including modules
+# Defaults vmlinux but it is usually overriden in the arch makefile
+all: vmlinux
+
ifdef CONFIG_CC_OPTIMIZE_FOR_SIZE
CFLAGS += -Os
else
@@ -795,7 +799,7 @@
# Directories & files removed with 'make clean'
CLEAN_DIRS += $(MODVERDIR)
-CLEAN_FILES += vmlinux System.map kernel.spec \
+CLEAN_FILES += vmlinux System.map \
.tmp_kallsyms* .tmp_version .tmp_vmlinux*
# Directories & files removed with 'make mrproper'
@@ -848,37 +852,19 @@
-o -name '*%' -o -name '.*.cmd' -o -name 'core' \) \
-type f -print | xargs rm -f
-# RPM target
-# ---------------------------------------------------------------------------
-
-.PHONY: rpm
-# Remove hyphens since they have special meaning in RPM filenames
-KERNELPATH=kernel-$(subst -,,$(KERNELRELEASE))
+# Packaging of the kernel to various formats
+# ---------------------------------------------------------------------------
+# rpm target kept for backward compatibility
+package-dir := $(srctree)/scripts/package
-# If you do a make spec before packing the tarball you can rpm -ta it
+.PHONY: %-pkg rpm
-spec:
- $(CONFIG_SHELL) $(srctree)/scripts/mkspec > $(objtree)/kernel.spec
-
-# a) Build a tar ball
-# b) generate an rpm from it
-# c) and pack the result
-# - Use /. to avoid tar packing just the symlink
-
-rpm: clean spec
- set -e; \
- cd .. ; \
- ln -sf $(srctree) $(KERNELPATH) ; \
- tar -cvz $(RCS_TAR_IGNORE) -f $(KERNELPATH).tar.gz $(KERNELPATH)/. ; \
- rm $(KERNELPATH)
-
- set -e; \
- $(CONFIG_SHELL) $(srctree)/scripts/mkversion > $(objtree)/.tmp_version;\
- mv -f $(objtree)/.tmp_version $(objtree)/.version;
+%pkg: FORCE
+ $(Q)$(MAKE) -f $(package-dir)/Makefile $@
+rpm: FORCE
+ $(Q)$(MAKE) -f $(package-dir)/Makefile $@
- $(RPM) --target $(UTS_MACHINE) -ta ../$(KERNELPATH).tar.gz
- rm ../$(KERNELPATH).tar.gz
# Brief documentation of the typical targets used
# ---------------------------------------------------------------------------
@@ -905,6 +891,8 @@
@echo ' tags/TAGS - Generate tags file for editors'
@echo ' cscope - Generate cscope index'
@echo ' checkstack - Generate a list of stack hogs'
+ @echo 'Kernel packaging:'
+ @$(MAKE) -f $(package-dir)/Makefile help
@echo ''
@echo 'Documentation targets:'
@$(MAKE) -f $(srctree)/Documentation/DocBook/Makefile dochelp
diff -Nru a/arch/i386/Makefile b/arch/i386/Makefile
--- a/arch/i386/Makefile 2004-06-20 23:05:13 +02:00
+++ b/arch/i386/Makefile 2004-06-20 23:05:13 +02:00
@@ -121,22 +121,23 @@
all: bzImage
-BOOTIMAGE=arch/i386/boot/bzImage
-zImage zlilo zdisk: BOOTIMAGE=arch/i386/boot/zImage
+# KBUILD_IMAGE specify target image being built
+ KBUILD_IMAGE := $(boot)/bzImage
+zImage zlilo zdisk: KBUILD_IMAGE := arch/i386/boot/zImage
zImage bzImage: vmlinux
- $(Q)$(MAKE) $(build)=$(boot) $(BOOTIMAGE)
+ $(Q)$(MAKE) $(build)=$(boot) $(KBUILD_IMAGE)
compressed: zImage
zlilo bzlilo: vmlinux
- $(Q)$(MAKE) $(build)=$(boot) BOOTIMAGE=$(BOOTIMAGE) zlilo
+ $(Q)$(MAKE) $(build)=$(boot) BOOTIMAGE=$(KBUILD_IMAGE) zlilo
zdisk bzdisk: vmlinux
- $(Q)$(MAKE) $(build)=$(boot) BOOTIMAGE=$(BOOTIMAGE) zdisk
+ $(Q)$(MAKE) $(build)=$(boot) BOOTIMAGE=$(KBUILD_IMAGE) zdisk
install fdimage fdimage144 fdimage288: vmlinux
- $(Q)$(MAKE) $(build)=$(boot) BOOTIMAGE=$(BOOTIMAGE) $@
+ $(Q)$(MAKE) $(build)=$(boot) BOOTIMAGE=$(KBUILD_IMAGE) $@
prepare: include/asm-$(ARCH)/asm_offsets.h
CLEAN_FILES += include/asm-$(ARCH)/asm_offsets.h
diff -Nru a/scripts/Makefile b/scripts/Makefile
--- a/scripts/Makefile 2004-06-20 23:05:13 +02:00
+++ b/scripts/Makefile 2004-06-20 23:05:13 +02:00
@@ -13,7 +13,7 @@
subdir-$(CONFIG_MODVERSIONS) += genksyms
# Let clean descend into subdirs
-subdir- += basic lxdialog kconfig
+subdir- += basic lxdialog kconfig package
# dependencies on generated files need to be listed explicitly
diff -Nru a/scripts/mkspec b/scripts/mkspec
--- a/scripts/mkspec 2004-06-20 23:05:13 +02:00
+++ /dev/null Wed Dec 31 16:00:00 196900
@@ -1,72 +0,0 @@
-#!/bin/sh
-#
-# Output a simple RPM spec file that uses no fancy features requring
-# RPM v4. This is intended to work with any RPM distro.
-#
-# The only gothic bit here is redefining install_post to avoid
-# stripping the symbols from files in the kernel which we want
-#
-# Patched for non-x86 by Opencon (L) 2002 <[email protected]>
-#
-# That's the voodoo to see if it's a x86.
-ISX86=`echo ${ARCH:=\`arch\`} | grep -ie i.86`
-if [ ! -z $ISX86 ]; then
- PC=1
-else
- PC=0
-fi
-# starting to output the spec
-if [ "`grep CONFIG_DRM=y .config | cut -f2 -d\=`" = "y" ]; then
- PROVIDES=kernel-drm
-fi
-
-PROVIDES="$PROVIDES kernel-$VERSION.$PATCHLEVEL.$SUBLEVEL$EXTRAVERSION"
-
-echo "Name: kernel"
-echo "Summary: The Linux Kernel"
-echo "Version: "$VERSION.$PATCHLEVEL.$SUBLEVEL$EXTRAVERSION | sed -e "s/-//g"
-# we need to determine the NEXT version number so that uname and
-# rpm -q will agree
-echo "Release: `. $srctree/scripts/mkversion`"
-echo "License: GPL"
-echo "Group: System Environment/Kernel"
-echo "Vendor: The Linux Community"
-echo "URL: http://www.kernel.org"
-echo -n "Source: kernel-$VERSION.$PATCHLEVEL.$SUBLEVEL"
-echo "$EXTRAVERSION.tar.gz" | sed -e "s/-//g"
-echo "BuildRoot: /var/tmp/%{name}-%{PACKAGE_VERSION}-root"
-echo "Provides: $PROVIDES"
-echo "%define __spec_install_post /usr/lib/rpm/brp-compress || :"
-echo "%define debug_package %{nil}"
-echo ""
-echo "%description"
-echo "The Linux Kernel, the operating system core itself"
-echo ""
-echo "%prep"
-echo "%setup -q"
-echo ""
-echo "%build"
-echo "make clean all"
-echo ""
-echo "%install"
-echo 'mkdir -p $RPM_BUILD_ROOT/boot $RPM_BUILD_ROOT/lib $RPM_BUILD_ROOT/lib/modules'
-echo 'INSTALL_MOD_PATH=$RPM_BUILD_ROOT make modules_install'
-# This is the first disagreement between i386 and most others
-if [ $PC = 1 ]; then
- echo 'cp arch/i386/boot/bzImage $RPM_BUILD_ROOT'"/boot/vmlinuz-$VERSION.$PATCHLEVEL.$SUBLEVEL$EXTRAVERSION"
-else
- echo 'cp vmlinux $RPM_BUILD_ROOT'"/boot/vmlinux-$VERSION.$PATCHLEVEL.$SUBLEVEL$EXTRAVERSION"
-fi
-# Back on track
-echo 'cp System.map $RPM_BUILD_ROOT'"/boot/System.map-$VERSION.$PATCHLEVEL.$SUBLEVEL$EXTRAVERSION"
-echo 'cp .config $RPM_BUILD_ROOT'"/boot/config-$VERSION.$PATCHLEVEL.$SUBLEVEL$EXTRAVERSION"
-echo ""
-echo "%clean"
-echo '#echo -rf $RPM_BUILD_ROOT'
-echo ""
-echo "%files"
-echo '%defattr (-, root, root)'
-echo "%dir /lib/modules"
-echo "/lib/modules/$VERSION.$PATCHLEVEL.$SUBLEVEL$EXTRAVERSION"
-echo "/boot/*"
-echo ""
diff -Nru a/scripts/package/Makefile b/scripts/package/Makefile
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/scripts/package/Makefile 2004-06-20 23:05:13 +02:00
@@ -0,0 +1,71 @@
+# Makefile for the different targets used to generate full packages of a kernel
+# It uses the generic clean infrastructure of kbuild
+
+# Ignore the following files/directories during tar operation
+TAR_IGNORE := --exclude SCCS --exclude BitKeeper --exclude .svn --exclude CVS
+
+
+# RPM target
+# ---------------------------------------------------------------------------
+# The rpm target generates two rpm files:
+# /usr/src/packages/SRPMS/kernel-2.6.7rc2-1.src.rpm
+# /usr/src/packages/RPMS/i386/kernel-2.6.7rc2-1.<arch>.rpm
+# The src.rpm files includes all source for the kernel being built
+# The <arch>.rpm includes kernel configuration, modules etc.
+#
+# Process to create the rpm files
+# a) clean the kernel
+# b) Generate .spec file
+# c) Build a tar ball, using symlink to make kernel version
+# first entry in the path
+# d) and pack the result to a tar.gz file
+# e) generate the rpm files, based on kernel.spec
+# - Use /. to avoid tar packing just the symlink
+
+# Do we have rpmbuild, otherwise fall back to the older rpm
+RPM := $(shell if [ -x "/usr/bin/rpmbuild" ]; then echo rpmbuild; \
+ else echo rpm; fi)
+
+# Remove hyphens since they have special meaning in RPM filenames
+KERNELPATH := kernel-$(subst -,,$(KERNELRELEASE))
+MKSPEC := $(srctree)/scripts/package/mkspec
+PREV := set -e; cd ..;
+
+.PHONY: rpm-pkg rpm
+
+$(objtree)/kernel.spec: $(MKSPEC)
+ $(CONFIG_SHELL) $(MKSPEC) > $@
+
+rpm-pkg rpm: $(objtree)/kernel.spec
+ $(MAKE) clean
+ $(PREV) ln -sf $(srctree) $(KERNELPATH)
+ $(PREV) tar -cz $(RCS_TAR_IGNORE) -f $(KERNELPATH).tar.gz $(KERNELPATH)/.
+ $(PREV) rm $(KERNELPATH)
+
+ set -e; \
+ $(CONFIG_SHELL) $(srctree)/scripts/mkversion > $(objtree)/.tmp_version
+ set -e; \
+ mv -f $(objtree)/.tmp_version $(objtree)/.version
+
+ $(RPM) --target $(UTS_MACHINE) -ta ../$(KERNELPATH).tar.gz
+ rm ../$(KERNELPATH).tar.gz
+
+clean-rule += rm -f $(objtree)/kernel.spec
+
+# Deb target
+# ---------------------------------------------------------------------------
+#
+.PHONY: deb-pkg
+deb-pkg:
+ $(MAKE)
+ $(CONFIG_SHELL) $(srctree)/scripts/package/builddeb
+
+clean-rule += && rm -rf $(objtree)/debian/
+
+
+# Help text displayed when executing 'make help'
+# ---------------------------------------------------------------------------
+help:
+ @echo ' rpm-pkg - Build the kernel as an RPM package'
+ @echo ' deb-pkg - Build the kernel as an deb package'
+
diff -Nru a/scripts/package/builddeb b/scripts/package/builddeb
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/scripts/package/builddeb 2004-06-20 23:05:13 +02:00
@@ -0,0 +1,79 @@
+#!/bin/sh
+#
+# builddeb 1.2
+# Copyright 2003 Wichert Akkerman <[email protected]>
+#
+# Simple script to generate a deb package for a Linux kernel. All the
+# complexity of what to do with a kernel after it is installer or removed
+# is left to other scripts and packages: they can install scripts in the
+# /etc/kernel/{pre,post}{inst,rm}.d/ directories that will be called on
+# package install and removal.
+
+set -e
+
+# Some variables and settings used throughout the script
+version="$VERSION.$PATCHLEVEL.$SUBLEVEL$EXTRAVERSION"
+tmpdir="$objtree/debian/tmp"
+
+# Setup the directory structure
+rm -rf "$tmpdir"
+mkdir -p "$tmpdir/DEBIAN" "$tmpdir/lib" "$tmpdir/boot"
+
+# Build and install the kernel
+cp System.map "$tmpdir/boot/System.map-$version"
+cp .config "$tmpdir/boot/config-$version"
+cp $KBUILD_IMAGE "$tmpdir/boot/vmlinuz-$version"
+
+if grep -q '^CONFIG_MODULES=y' .config ; then
+ INSTALL_MOD_PATH="$tmpdir" make modules_install
+fi
+
+# Install the maintainer scripts
+for script in postinst postrm preinst prerm ; do
+ mkdir -p "$tmpdir/etc/kernel/$script.d"
+ cat <<EOF > "$tmpdir/DEBIAN/$script"
+#!/bin/sh
+
+set -e
+
+test -d /etc/kernel/$script.d && run-parts --arg="$version" /etc/kernel/$script.d
+exit 0
+EOF
+ chmod 755 "$tmpdir/DEBIAN/$script"
+done
+
+name="Kernel Compiler <$(id -nu)@$(hostname -f)>"
+# Generate a simple changelog template
+cat <<EOF > debian/changelog
+linux ($version) unstable; urgency=low
+
+ * A standard release
+
+ -- $name $(date -R)
+EOF
+
+# Generate a control file
+cat <<EOF > debian/control
+Source: linux
+Section: base
+Priority: optional
+Maintainer: $name
+Standards-Version: 3.6.1
+
+Package: linux-$version
+Architecture: any
+Description: Linux kernel, version $version
+ This package contains the Linux kernel, modules and corresponding other
+ files version $version.
+EOF
+
+# Fix some ownership and permissions
+chown -R root:root "$tmpdir"
+chmod -R go-w "$tmpdir"
+
+# Perform the final magic
+dpkg-gencontrol -isp
+dpkg --build "$tmpdir" ..
+
+exit 0
+
diff -Nru a/scripts/package/mkspec b/scripts/package/mkspec
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/scripts/package/mkspec 2004-06-20 23:05:13 +02:00
@@ -0,0 +1,63 @@
+#!/bin/sh
+#
+# Output a simple RPM spec file that uses no fancy features requring
+# RPM v4. This is intended to work with any RPM distro.
+#
+# The only gothic bit here is redefining install_post to avoid
+# stripping the symbols from files in the kernel which we want
+#
+# Patched for non-x86 by Opencon (L) 2002 <[email protected]>
+#
+
+# starting to output the spec
+if [ "`grep CONFIG_DRM=y .config | cut -f2 -d\=`" = "y" ]; then
+ PROVIDES=kernel-drm
+fi
+
+PROVIDES="$PROVIDES kernel-$VERSION.$PATCHLEVEL.$SUBLEVEL$EXTRAVERSION"
+
+echo "Name: kernel"
+echo "Summary: The Linux Kernel"
+echo "Version: "$VERSION.$PATCHLEVEL.$SUBLEVEL$EXTRAVERSION | sed -e "s/-//g"
+# we need to determine the NEXT version number so that uname and
+# rpm -q will agree
+echo "Release: `. $srctree/scripts/mkversion`"
+echo "License: GPL"
+echo "Group: System Environment/Kernel"
+echo "Vendor: The Linux Community"
+echo "URL: http://www.kernel.org"
+echo -n "Source: kernel-$VERSION.$PATCHLEVEL.$SUBLEVEL"
+echo "$EXTRAVERSION.tar.gz" | sed -e "s/-//g"
+echo "BuildRoot: /var/tmp/%{name}-%{PACKAGE_VERSION}-root"
+echo "Provides: $PROVIDES"
+echo "%define __spec_install_post /usr/lib/rpm/brp-compress || :"
+echo "%define debug_package %{nil}"
+echo ""
+echo "%description"
+echo "The Linux Kernel, the operating system core itself"
+echo ""
+echo "%prep"
+echo "%setup -q"
+echo ""
+echo "%build"
+echo "make clean && make"
+echo ""
+echo "%install"
+echo 'mkdir -p $RPM_BUILD_ROOT/boot $RPM_BUILD_ROOT/lib $RPM_BUILD_ROOT/lib/modules'
+
+echo 'INSTALL_MOD_PATH=$RPM_BUILD_ROOT make modules_install'
+echo 'cp $KBUILD_IMAGE $RPM_BUILD_ROOT'"/boot/vmlinuz-$VERSION.$PATCHLEVEL.$SUBLEVEL$EXTRAVERSION"
+
+echo 'cp System.map $RPM_BUILD_ROOT'"/boot/System.map-$VERSION.$PATCHLEVEL.$SUBLEVEL$EXTRAVERSION"
+
+echo 'cp .config $RPM_BUILD_ROOT'"/boot/config-$VERSION.$PATCHLEVEL.$SUBLEVEL$EXTRAVERSION"
+echo ""
+echo "%clean"
+echo '#echo -rf $RPM_BUILD_ROOT'
+echo ""
+echo "%files"
+echo '%defattr (-, root, root)'
+echo "%dir /lib/modules"
+echo "/lib/modules/$VERSION.$PATCHLEVEL.$SUBLEVEL$EXTRAVERSION"
+echo "/boot/*"
+echo ""
On Sun, Jun 20, 2004 at 11:19:06PM +0200, Sam Ravnborg wrote:
> Hi Andrew, Linus.
>
> Here follows two kbuild patches.
Available also at bkbits:
bk pull bk://linux-sam.bkbits.net/kbuild
Sam
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/06/20 22:59:03+02:00 [email protected]
# kbuild: External module support improved
#
# To improve support for external modules and to make usage
# of separate output directory easier the following
# changes has been implemented:
#
# 1) When using a separate output directory create a small
# Makefile that is a simple wrapper, calling the Makefile
# in the kernel tree.
# - This allows the user to shift to the output directory
# and execute make.
# - The Makefile is also useful to document the location
# source used for the kernel
#
# 2) When installing the kernel a new symlink is now created
# pointing to the source of kernel.
#
# 3) The build symlink now points to the output of the kernel
# compile.
# - When a kernel is compiled with output and source
# mixed, the build and source symlinks will point
# to the same directory. In this case there is
# no change in behaviour.
#
# Adding the Makefile in step 1) allow the for a long
# time recommended way to build an external module to
# continue working even with separate output directory.
#
# The following command now works independent of the kernel
# being build with separate output directory, or with
# output and source mixed.
#
# make -C /lib/modules/`uname -r`/build M=`pwd`
#
# [Substituting M= with SUBDIRS= give same effect].
#
# It is recommended that distributions pick up this
# method, and especially start shipping kernel output and
# source separately.
#
# Please note that when the kernel is being build
# using a separate output directory it may (will?) break
# modules that has not yet picked up the recommended
# way to build modules for 2.6.
# See Documentation/kbuild/modules.txt for details.
#
# Patch includes contributions from:
# Andreas Gruenbacher <[email protected]> and
# Geert Uytterhoeven <[email protected]>
#
# Signed-off-by: Sam Ravnborg <[email protected]>
#
# scripts/mkmakefile
# 2004/06/20 22:58:48+02:00 [email protected] +25 -0
#
# scripts/mkmakefile
# 2004/06/20 22:58:48+02:00 [email protected] +0 -0
# BitKeeper file /home/sam/bk/kbuild/scripts/mkmakefile
#
# Makefile
# 2004/06/20 22:58:48+02:00 [email protected] +23 -4
# External module support improved
#
diff -Nru a/Makefile b/Makefile
--- a/Makefile 2004-06-20 23:06:03 +02:00
+++ b/Makefile 2004-06-20 23:06:03 +02:00
@@ -608,14 +608,23 @@
# A multi level approach is used. prepare1 is updated first, then prepare0.
# prepare-all is the collection point for the prepare targets.
-.PHONY: prepare-all prepare prepare0 prepare1
+.PHONY: prepare-all prepare prepare0 prepare1 prepare2
+
+# prepare 2 generate Makefile to be placed in output directory, if
+# using a seperate output directory. This allows convinient use
+# of make in output directory
+prepare2:
+ $(Q)if [ ! $(srctree) -ef $(objtree) ]; then \
+ $(CONFIG_SHELL) $(srctree)/scripts/mkmakefile $(srctree) $(objtree) \
+ > $(objtree)/Makefile; \
+ fi
# prepare1 is used to check if we are building in a separate output directory,
# and if so do:
# 1) Check that make has not been executed in the kernel src $(srctree)
# 2) Create the include2 directory, used for the second asm symlink
-prepare1:
+prepare1: prepare2
ifneq ($(KBUILD_SRC),)
@echo ' Using $(srctree) as source for kernel'
$(Q)if [ -h $(srctree)/include/asm -o -f $(srctree)/.config ]; then \
@@ -725,6 +734,12 @@
modules_prepare: prepare-all scripts
# Target to install modules
+# Modules are pr. default installed in /lib/modules/$(KERNELRELEASE)/...
+# Within this directory create two symlinks:
+# build => link to the directory containing the output files of the kernel build
+# source => link to the directory containing the source for the kernel
+# source and build are equal except for the case when the kernel is build using
+# a separate output directory
.PHONY: modules_install
modules_install: _modinst_ _modinst_post
@@ -736,9 +751,13 @@
sleep 1; \
fi
@rm -rf $(MODLIB)/kernel
- @rm -f $(MODLIB)/build
+ @rm -f $(MODLIB)/source
@mkdir -p $(MODLIB)/kernel
- @ln -s $(TOPDIR) $(MODLIB)/build
+ @ln -s $(srctree) $(MODLIB)/source
+ @if [ ! $(objtree) -ef $(MODLIB)/build ]; then \
+ rm -f $(MODLIB)/build ; \
+ ln -s $(objtree) $(MODLIB)/build ; \
+ fi
$(Q)$(MAKE) -rR -f $(srctree)/scripts/Makefile.modinst
# If System.map exists, run depmod. This deliberately does not have a
diff -Nru a/scripts/mkmakefile b/scripts/mkmakefile
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/scripts/mkmakefile 2004-06-20 23:06:03 +02:00
@@ -0,0 +1,25 @@
+#!/bin/sh
+# Generates a small Makefile used in the root of the output
+# directory, to allow make to be started from there.
+# The Makefile also allow for more convinient build of external modules
+
+# Usage
+# $1 - Kernel src directory
+# $2 - Output directory
+
+
+cat << EOF
+
+KERNELSRC := $1
+KERNELOUTPUT := $2
+
+MAKEFLAGS += --no-print-directory
+
+all:
+ \$(MAKE) -C \$(KERNELSRC) O=\$(KERNELOUTPUT)
+
+%:
+ \$(MAKE) -C \$(KERNELSRC) O=\$(KERNELOUTPUT) \$@
+
+EOF
+
> # 3) The build symlink now points to the output of the kernel
> # compile.
> # - When a kernel is compiled with output and source
> # mixed, the build and source symlinks will point
> # to the same directory. In this case there is
> # no change in behaviour.
> # It is recommended that distributions pick up this
> # method, and especially start shipping kernel output and
> # source separately.
> #
I don't see the point of this; module builds don't use the output of the
kernel compile but the SOURCE, eg the headers and Makefiles.
I don't see a reason for this change; at least what I ship right now for
the Fedora Core 2 kernel seems to work for all modules with sane
makefiles so far....
On Sun, 2004-06-20 at 23:19, Sam Ravnborg wrote:
Hiya,
> 2) Improved support for external modules.
> It has been debated what to name the symlink in /lib/modules/`uname -r`
> and where it should point.
> Now that there is a possibility to build the kernel with a separate output
> directory, there is a need to utilise this in the install.
> From now on build will point to the output directory, and source will point
> to the kernel source.
I know Sam's mta blocks my mail at least (lame isp), but for the rest,
please reconsider using this. Many external modules, libs, etc use
/lib/modules/`uname -r`/build to locate the _source_, and this will
break them all.
Once again I do not argue the logic behind this, but please then rather
do it in 2.7, or just keep 'build', and make the output one 'output' or
'object' or something.
> No effort whatsoever will be done to keep external modules working if
> they do not use the kbuild infrastructure. There is no reason not to
> do so.
Given, but to 'use' the kbuild infrastructure, you must still call it
via:
make -C _path_to_sources M=`pwd`
and any external project that at least tries to automate things a bit
(because some things are not always distributed with vendor distro,
or updated regularly by vendor distro, and most users know at least
how to do 'make && make install') will break.
Lastly, if the 'build'/whatever symlinks is not an 'kbuild'
infrastructure (as Sam want to make it, and thus base his reasoning why
it is Ok to build it) for finding the source of the current running
kernel, then what is, why have it in the first place?
Thanks,
--
Martin Schlemmer
On Sun, 2004-06-20 at 23:25, Arjan van de Ven wrote:
> > # 3) The build symlink now points to the output of the kernel
> > # compile.
> > # - When a kernel is compiled with output and source
> > # mixed, the build and source symlinks will point
> > # to the same directory. In this case there is
> > # no change in behaviour.
>
> > # It is recommended that distributions pick up this
> > # method, and especially start shipping kernel output and
> > # source separately.
> > #
>
> I don't see the point of this; module builds don't use the output of the
> kernel compile but the SOURCE, eg the headers and Makefiles.
>
> I don't see a reason for this change; at least what I ship right now for
> the Fedora Core 2 kernel seems to work for all modules with sane
> makefiles so far....
And is going to break a lot of things - IMHO not an change for a
'stable' release.
--
Martin Schlemmer
On Sun, Jun 20, 2004 at 11:25:29PM +0200, Arjan van de Ven wrote:
>
> > # 3) The build symlink now points to the output of the kernel
> > # compile.
> > # - When a kernel is compiled with output and source
> > # mixed, the build and source symlinks will point
> > # to the same directory. In this case there is
> > # no change in behaviour.
>
> > # It is recommended that distributions pick up this
> > # method, and especially start shipping kernel output and
> > # source separately.
> > #
>
> I don't see the point of this; module builds don't use the output of the
> kernel compile but the SOURCE, eg the headers and Makefiles.
When building modules they use _both_ the source and partly the output.
The output needed is what is required for kbuild to work, and optionally
Module.symvers for symbol versions.
> I don't see a reason for this change; at least what I ship right now for
> the Fedora Core 2 kernel seems to work for all modules with sane
> makefiles so far....
Did you put the output of the kernel compile in a separate output directory,
reusing the same src for several configurations?
If not you do not need the change, and thus do not get any benefit. On the
other hand everything should work as-is for you.
Sam
On Sun, Jun 20, 2004 at 11:45:43PM +0200, Sam Ravnborg wrote:
>
> Did you put the output of the kernel compile in a separate output directory,
> reusing the same src for several configurations?
I do not ship the output of the kernel compile actually.
But I do not share the /lib/modules/version/build for different kernels,
it's unique.
> If not you do not need the change, and thus do not get any benefit. On the
> other hand everything should work as-is for you.
> Given, but to 'use' the kbuild infrastructure, you must still call it
> via:
>
> make -C _path_to_sources M=`pwd`
I see no problem with requiring this though; requiring a correct
makefile is perfectly fine with me, and this is the only and documented
way for 2.6 already.
(And it's also the only way to build modules against Fedora Core 2
kernels by the way)
That's not my reservation to this approach.
On Sun, Jun 20, 2004 at 11:30:34PM +0200, Martin Schlemmer wrote:
>
> I know Sam's mta blocks my mail at least (lame isp), but for the rest,
> please reconsider using this.
Hmm, got your mail.
> Many external modules, libs, etc use
> /lib/modules/`uname -r`/build to locate the _source_, and this will
> break them all.
Examples please. What I have seen so far is modules that was not
adapted to use kbuild when being build.
If they fail to do so they are inherently broken.
If I get just one good example I will go for the object directory, but
what I have seen so far is whining - no examples.
Sam
On Sun, 2004-06-20 at 23:42, Arjan van de Ven wrote:
> > Given, but to 'use' the kbuild infrastructure, you must still call it
> > via:
> >
> > make -C _path_to_sources M=`pwd`
>
> I see no problem with requiring this though; requiring a correct
> makefile is perfectly fine with me, and this is the only and documented
> way for 2.6 already.
> (And it's also the only way to build modules against Fedora Core 2
> kernels by the way)
>
I did not mean I have a problem with that. Say you take svgalib, and
you want the build system to automatically compile the kernel module,
you might do something like:
---
build_2_6_module:
@make -C /lib/modules/`uname -r`/build M=`PWD`
---
will break with proposed patch ...
And the point I wanted to make was that AFIAK
'/lib/modules/`uname -r`/build' is an interface to figure
out where the _sources_ for the current running kernel are
located. And apparently I am not the only one, as most if
not all external projects use this as the way (or one of
them at least) to determine this ...
Thanks,
--
Martin Schlemmer
On Mon, Jun 21, 2004 at 12:03:19AM +0200, Sam Ravnborg wrote:
> If I get just one good example I will go for the object directory, but
> what I have seen so far is whining - no examples.
Now I recall why I did not like the object directory.
I will break all modules using the kbuild infrastructure!
Why, because there is no way the to find the output directory except
specifying both directories.
One could do:
make -C /lib/modules/`uname -r`/source O=/lib/modules/`uname -r`/build M=`pwd`
So the currect choice is:
1) Break modules that actually dive into the src, grepping, including or whatever
2) Break all modules using kbuild infrastructure, including the above ones
I go for 1), introducing minimal breakage.
And please keep in mind. The breakage wil _only_ be visible when kernels are
shipped with separate output directory.
If kernels are shipped with no output files at all then one can just
compile the kernel. Seems to be the Fedora way. No breakage happens.
If kernels are shipped with mixed source and output then no breakage happens.
If kernels are shipped with separate source and output then we better break
as few modules as possible. And the introduced change actually minimize breakage.
So the patch will not change.
Sam
On Mon, 2004-06-21 at 00:03, Sam Ravnborg wrote:
> On Sun, Jun 20, 2004 at 11:30:34PM +0200, Martin Schlemmer wrote:
> >
> > I know Sam's mta blocks my mail at least (lame isp), but for the rest,
> > please reconsider using this.
> Hmm, got your mail.
>
> > Many external modules, libs, etc use
> > /lib/modules/`uname -r`/build to locate the _source_, and this will
> > break them all.
> Examples please. What I have seen so far is modules that was not
> adapted to use kbuild when being build.
> If they fail to do so they are inherently broken.
>
Well, glibc use it for instance as an fall-through if you do not specify
it via ./configure arguments, or environment (yes, glibc should not use
it, etc, etc, no flames please =). So as well does alsa-driver,
nvidia's drivers (gah, puke, yes, its got some binary-only stuff in
there ;), ati's drivers and a lot of other stuff (if you really need
them all I can try to find time to look for more).
I am not sure about ati's drivers and alsa, but nvidia uses kbuild.
Thanks,
--
Martin Schlemmer
On Sunday 20 June 2004 23:52, Martin Schlemmer wrote:
> On Sun, 2004-06-20 at 23:42, Arjan van de Ven wrote:
> > > Given, but to 'use' the kbuild infrastructure, you must still call it
> > > via:
> > >
> > > make -C _path_to_sources M=`pwd`
> >
> > I see no problem with requiring this though; requiring a correct
> > makefile is perfectly fine with me, and this is the only and documented
> > way for 2.6 already.
> > (And it's also the only way to build modules against Fedora Core 2
> > kernels by the way)
>
> I did not mean I have a problem with that. Say you take svgalib, and
> you want the build system to automatically compile the kernel module,
> you might do something like:
>
> ---
> build_2_6_module:
> @make -C /lib/modules/`uname -r`/build M=`PWD`
> ---
>
> will break with proposed patch ...
No it won't.
You always need to figure out $(objtree) to build external modules, with or
without a separate output directory. Many modules don't need to know
$(srctree) explicitly at all.
In case you want to do something depending on the sources/confguration, there
are two ways:
- follow the new source symlink,
- let kbuild take you to $(srctree): When the makefile in the M directory
is included, the current working directory is $(srctree). besides, all the
usual variables like $(srctree), $(objtree), CONFIG_* variables, etc. are
all available. That's a good time to check for features, etc.
> And the point I wanted to make was that AFIAK
> '/lib/modules/`uname -r`/build' is an interface to figure
> out where the _sources_ for the current running kernel are
> located.
That's a misconception. At the minimum, you want to be able to build the
module. Directly messing with the sources is usually wrong. I know external
module authors like to do that nevertheless; in a few cases it's actually
useful. Most of the time it really is not. Most external modules have totally
braindead/broken makefiles.
Regards,
--
Andreas Gruenbacher <[email protected]>
SUSE Labs, SUSE LINUX AG
On Mon, 2004-06-21 at 00:18, Sam Ravnborg wrote:
> On Mon, Jun 21, 2004 at 12:03:19AM +0200, Sam Ravnborg wrote:
> > If I get just one good example I will go for the object directory, but
> > what I have seen so far is whining - no examples.
>
> Now I recall why I did not like the object directory.
> I will break all modules using the kbuild infrastructure!
>
Below do not really explain this - care to be more detailed?
> Why, because there is no way the to find the output directory except
> specifying both directories.
> One could do:
> make -C /lib/modules/`uname -r`/source O=/lib/modules/`uname -r`/build M=`pwd`
>
Huh? Explain to me how else you would do builds that have separate
output directory? And what is the difference from above to:
make -C /lib/modules/`uname -r`/build O=/lib/modules/`uname -r`/object M=`pwd`
except that you will _not_ cause existing stuff to break?
> So the currect choice is:
> 1) Break modules that actually dive into the src, grepping, including or whatever
> 2) Break all modules using kbuild infrastructure, including the above ones
>
> I go for 1), introducing minimal breakage.
>
> And please keep in mind. The breakage wil _only_ be visible when kernels are
> shipped with separate output directory.
How is that? In both cases the 'build' symlink do not point to the
source anymore.
> If kernels are shipped with no output files at all then one can just
> compile the kernel. Seems to be the Fedora way. No breakage happens.
>
> If kernels are shipped with mixed source and output then no breakage happens.
>
> If kernels are shipped with separate source and output then we better break
> as few modules as possible. And the introduced change actually minimize breakage.
>
> So the patch will not change.
>
> Sam
--
Martin Schlemmer
[snipped a few CC addresses]
On Sunday 20 June 2004 23:16, Martin Schlemmer wrote:
> On Mon, 2004-06-21 at 00:03, Sam Ravnborg wrote:
> > On Sun, Jun 20, 2004 at 11:30:34PM +0200, Martin Schlemmer wrote:
> > > I know Sam's mta blocks my mail at least (lame isp), but for the rest,
> > > please reconsider using this.
> >
> > Hmm, got your mail.
> >
> > > Many external modules, libs, etc use
> > > /lib/modules/`uname -r`/build to locate the _source_, and this will
> > > break them all.
> >
> > Examples please. What I have seen so far is modules that was not
> > adapted to use kbuild when being build.
> > If they fail to do so they are inherently broken.
>
> Well, glibc use it for instance as an fall-through if you do not specify
> it via ./configure arguments, or environment (yes, glibc should not use
> it, etc, etc, no flames please =). So as well does alsa-driver,
> nvidia's drivers (gah, puke, yes, its got some binary-only stuff in
> there ;), ati's drivers and a lot of other stuff (if you really need
> them all I can try to find time to look for more).
>
> I am not sure about ati's drivers and alsa, but nvidia uses kbuild.
>
>
> Thanks,
Sam's point is that unless you ask KBUILD to put the kernel build in a
separate directory to its sources (this is not the default
behaviour), /lib/modules/`uname -r`/build will still point to the mixture of
source and build data, therefore no breakage will occur.
I understand Sam's reasoning and I believe it is sensible to have the source
and build output in separate directories within /lib/modules/`uname -r`. The
drivers in question can easily be updated to support the exceptional case
whereby users build kernels in a different directory to the source.
Sam, maybe if there was a way to easily detect whether a kernel had been build
with or without a different output directory, it would be easier to have
vendors take this change on board. For example, I imagine in the typical case
whereby no change in build directory is made, you will have something like
this:
/lib/modules/2.6.7/build -> /home/alistair/linux-2.6
/lib/modules/2.6.7/source -> /home/alistair/linux-2.6
Whereas when O is given, it will instead be like this:
/lib/modules/2.6.7/build -> /home/alistair/my-dir
/lib/modules/2.6.7/source -> /home/alistair/linux-2.6
I presume that checking for the existence of /lib/modules/`uname -r`/source
will be enough.
#
# where's the kernel source?
#
if [ -d /lib/modules/`uname -r`/source ]; then
# 2.6.8 and newer
KERNDIR="/lib/modules/`uname -r`/source"
else
# pre 2.6.8 kernels
KERNDIR="/lib/modules/`uname -r`/build"
fi
Yeah?
--
Cheers,
Alistair.
personal: alistair()devzero!co!uk
university: s0348365()sms!ed!ac!uk
student: CS/AI Undergraduate
contact: 1F2 55 South Clerk Street,
Edinburgh. EH8 9PP.
On Mon, 21 Jun 2004 00:00:12 +0200, you wrote in linux.kernel:
> ---
> build_2_6_module:
> @make -C /lib/modules/`uname -r`/build M=3D`PWD`
> ---
>
> will break with proposed patch ...
Huh? Sam wrote there will be a Makefile in the output directory.
--
Ciao,
Pascal
On Mon, 2004-06-21 at 00:26, Andreas Gruenbacher wrote:
> On Sunday 20 June 2004 23:52, Martin Schlemmer wrote:
> > On Sun, 2004-06-20 at 23:42, Arjan van de Ven wrote:
> > > > Given, but to 'use' the kbuild infrastructure, you must still call it
> > > > via:
> > > >
> > > > make -C _path_to_sources M=`pwd`
> > >
> > > I see no problem with requiring this though; requiring a correct
> > > makefile is perfectly fine with me, and this is the only and documented
> > > way for 2.6 already.
> > > (And it's also the only way to build modules against Fedora Core 2
> > > kernels by the way)
> >
> > I did not mean I have a problem with that. Say you take svgalib, and
> > you want the build system to automatically compile the kernel module,
> > you might do something like:
> >
> > ---
> > build_2_6_module:
> > @make -C /lib/modules/`uname -r`/build M=`PWD`
> > ---
> >
> > will break with proposed patch ...
>
> No it won't.
>
> You always need to figure out $(objtree) to build external modules, with or
> without a separate output directory. Many modules don't need to know
> $(srctree) explicitly at all.
>
> In case you want to do something depending on the sources/confguration, there
> are two ways:
> - follow the new source symlink,
> - let kbuild take you to $(srctree): When the makefile in the M directory
> is included, the current working directory is $(srctree). besides, all the
> usual variables like $(srctree), $(objtree), CONFIG_* variables, etc. are
> all available. That's a good time to check for features, etc.
>
But my original concern (that the only way to figure where the source
are for the running kernel will be broken) is still valid. And to do
all that you mentioned above, you still need to figure out _where_ the
kernel source are ...
> > And the point I wanted to make was that AFIAK
> > '/lib/modules/`uname -r`/build' is an interface to figure
> > out where the _sources_ for the current running kernel are
> > located.
>
> That's a misconception. At the minimum, you want to be able to build the
> module. Directly messing with the sources is usually wrong. I know external
> module authors like to do that nevertheless; in a few cases it's actually
> useful. Most of the time it really is not. Most external modules have totally
> braindead/broken makefiles.
>
I never said anything about messing with the source. But anything that
needs access to the running kernel's headers need to know where the
sources are - and that could have been figured out by looking at the
'build' symlink.
Say you maintain an opensource (just to throw out the 'its closed
source, so screw them' arguments) external module that supports both
2.4 and 2.6, and easy way to implement it is to have:
makefile - make will first look for this, and then process it.
in here you check what kernel is running (via uname -r
or whatever), and then create the proper 'Makefile'
symlink, and then call make with arguments to properly
handle the external module build process for that
version kernel.
Makefile-pre_M_flag - 100% valid kbuild Makefile for kernels that
do not support M=
Makefile-post_M_flag - 100% valid kbuild Makefile for kernels
supporting M=
Makefile-2_4 - 100% valid kbuild/whatever Makefile for 2.4 kernels
(Ok, I am not so sure about how 2.4 handles things
these days anymore ...)
now the clueless user just calls 'make && make install' and it should
work perfectly.
Or you create an install.sh that does things properly, or whatever,
but point remains that you need to know where the source are ...
Thanks,
--
Martin Schlemmer
On Mon, 2004-06-21 at 00:26, Alistair John Strachan wrote:
> [snipped a few CC addresses]
>
> On Sunday 20 June 2004 23:16, Martin Schlemmer wrote:
> > On Mon, 2004-06-21 at 00:03, Sam Ravnborg wrote:
> > > On Sun, Jun 20, 2004 at 11:30:34PM +0200, Martin Schlemmer wrote:
> > > > I know Sam's mta blocks my mail at least (lame isp), but for the rest,
> > > > please reconsider using this.
> > >
> > > Hmm, got your mail.
> > >
> > > > Many external modules, libs, etc use
> > > > /lib/modules/`uname -r`/build to locate the _source_, and this will
> > > > break them all.
> > >
> > > Examples please. What I have seen so far is modules that was not
> > > adapted to use kbuild when being build.
> > > If they fail to do so they are inherently broken.
> >
> > Well, glibc use it for instance as an fall-through if you do not specify
> > it via ./configure arguments, or environment (yes, glibc should not use
> > it, etc, etc, no flames please =). So as well does alsa-driver,
> > nvidia's drivers (gah, puke, yes, its got some binary-only stuff in
> > there ;), ati's drivers and a lot of other stuff (if you really need
> > them all I can try to find time to look for more).
> >
> > I am not sure about ati's drivers and alsa, but nvidia uses kbuild.
> >
> >
> > Thanks,
>
> Sam's point is that unless you ask KBUILD to put the kernel build in a
> separate directory to its sources (this is not the default
> behaviour), /lib/modules/`uname -r`/build will still point to the mixture of
> source and build data, therefore no breakage will occur.
>
> I understand Sam's reasoning and I believe it is sensible to have the source
> and build output in separate directories within /lib/modules/`uname -r`. The
> drivers in question can easily be updated to support the exceptional case
> whereby users build kernels in a different directory to the source.
>
> Sam, maybe if there was a way to easily detect whether a kernel had been build
> with or without a different output directory, it would be easier to have
> vendors take this change on board. For example, I imagine in the typical case
> whereby no change in build directory is made, you will have something like
> this:
>
> /lib/modules/2.6.7/build -> /home/alistair/linux-2.6
> /lib/modules/2.6.7/source -> /home/alistair/linux-2.6
>
> Whereas when O is given, it will instead be like this:
>
> /lib/modules/2.6.7/build -> /home/alistair/my-dir
> /lib/modules/2.6.7/source -> /home/alistair/linux-2.6
>
> I presume that checking for the existence of /lib/modules/`uname -r`/source
> will be enough.
>
> #
> # where's the kernel source?
> #
>
> if [ -d /lib/modules/`uname -r`/source ]; then
> # 2.6.8 and newer
> KERNDIR="/lib/modules/`uname -r`/source"
> else
> # pre 2.6.8 kernels
> KERNDIR="/lib/modules/`uname -r`/build"
> fi
>
> Yeah?
Yes, as said before, I can understand the name change. The point is
more that the 'build' symlink will change in behavior in certain
circumstances, and because many projects already support 2.6, and
make use of the 'build' symlink, they will break.
If it was however done in 2.7, things will get supported well before
2.8 is out, or in general use ...
--
Martin Schlemmer
[CC trimmed]
On Monday 21 June 2004 00:39, Martin Schlemmer wrote:
> On Mon, 2004-06-21 at 00:26, Andreas Gruenbacher wrote:
> > On Sunday 20 June 2004 23:52, Martin Schlemmer wrote:
> > > On Sun, 2004-06-20 at 23:42, Arjan van de Ven wrote:
> > > > > Given, but to 'use' the kbuild infrastructure, you must still call
> > > > > it via:
> > > > >
> > > > > make -C _path_to_sources M=`pwd`
> > > >
> > > > I see no problem with requiring this though; requiring a correct
> > > > makefile is perfectly fine with me, and this is the only and
> > > > documented way for 2.6 already.
> > > > (And it's also the only way to build modules against Fedora Core 2
> > > > kernels by the way)
> > >
> > > I did not mean I have a problem with that. Say you take svgalib, and
> > > you want the build system to automatically compile the kernel module,
> > > you might do something like:
> > >
> > > ---
> > > build_2_6_module:
> > > @make -C /lib/modules/`uname -r`/build M=`PWD`
> > > ---
> > >
> > > will break with proposed patch ...
> >
> > No it won't.
> >
> > You always need to figure out $(objtree) to build external modules, with
> > or without a separate output directory. Many modules don't need to know
> > $(srctree) explicitly at all.
> >
> > In case you want to do something depending on the sources/confguration,
> > there are two ways:
> > - follow the new source symlink,
> > - let kbuild take you to $(srctree): When the makefile in the M
> > directory is included, the current working directory is $(srctree).
> > besides, all the usual variables like $(srctree), $(objtree), CONFIG_*
> > variables, etc. are all available. That's a good time to check for
> > features, etc.
>
> But my original concern (that the only way to figure where the source
> are for the running kernel will be broken) is still valid.
User-space stuff that needs access to kernel headers at build time is a
problem. But for those programs, depending on the running kernel instead of
simply looking in /usr/src/linux is an even bigger problem: What guarantees
that the first time the program is run, the kernel will still be the same? So
depending on the running kernel is definitely wrong. Depending
on /usr/src/linux is also wrong; ideally those programs should look
in /usr/include/{linux,asm}. Unfortunately these headers are not always
recent enough, and so recently added definitions may be missing.
> And to do
> all that you mentioned above, you still need to figure out _where_ the
> kernel source are ...
No. Use ``make -C /lib/modules/$(uname -r)/build M=$(pwd)''
[or ``make -C /lib/modules/$(uname -r)/build modules SUBDIRS=$(pwd)'']
and in Makefile, use conditionals to adapt to the configuration.
Documentation/kbuild/makefiles.txt is your friend. Use
``make -C /lib/modules/$(uname -r)/build modules_install M=$(pwd)''
to install the modules.
Caution: the last command is dangerous; it will wipe away all other modules
with kernels that don't have M= support.
In case you really think you must circumvent kbuild, check if the source
symlink exists, etc.
> > > And the point I wanted to make was that AFIAK
> > > '/lib/modules/`uname -r`/build' is an interface to figure
> > > out where the _sources_ for the current running kernel are
> > > located.
> >
> > That's a misconception. At the minimum, you want to be able to build the
> > module. Directly messing with the sources is usually wrong. I know
> > external module authors like to do that nevertheless; in a few cases it's
> > actually useful. Most of the time it really is not. Most external modules
> > have totally braindead/broken makefiles.
>
> I never said anything about messing with the source. But anything that
> needs access to the running kernel's headers need to know where the
> sources are - and that could have been figured out by looking at the
> 'build' symlink.
See above.
> Say you maintain an opensource (just to throw out the 'its closed
> source, so screw them' arguments) external module that supports both
> 2.4 and 2.6, and easy way to implement it is to have:
>
> makefile - make will first look for this, and then process it.
> in here you check what kernel is running (via uname -r
> or whatever), and then create the proper 'Makefile'
> symlink, and then call make with arguments to properly
> handle the external module build process for that
> version kernel.
And what prevents you from doing exactly that? The only question is how to
invoke kbuild, but that hasn't changed.
> Makefile-pre_M_flag - 100% valid kbuild Makefile for kernels that
> do not support M=
>
> Makefile-post_M_flag - 100% valid kbuild Makefile for kernels
> supporting M=
Right now I would collapse the pre/post Makefiles and use SUBDIRS instead.
There is no easy and reliable test for M= support, and it's only cosmetic.
Sam will probably disagree.
> Makefile-2_4 - 100% valid kbuild/whatever Makefile for 2.4 kernels
> (Ok, I am not so sure about how 2.4 handles things
> these days anymore ...)
>
> now the clueless user just calls 'make && make install' and it should
> work perfectly.
>
> Or you create an install.sh that does things properly, or whatever,
> but point remains that you need to know where the source are ...
Regards,
--
Andreas Gruenbacher <[email protected]>
SUSE Labs, SUSE LINUX AG
On Mon, 21 Jun 2004, Sam Ravnborg wrote:
> > Many external modules, libs, etc use
> > /lib/modules/`uname -r`/build to locate the _source_, and this will
> > break them all.
> Examples please. What I have seen so far is modules that was not
> adapted to use kbuild when being build.
> If they fail to do so they are inherently broken.
For example our installation script expects that
/lib/modules/`uname -r`/build behaves like a symbolic link that points to
/usr/src/linux-`uname -r`. If /lib/modules/`uname -r`/build doesn't exist
then /usr/src/linux-`uname -r` will be attempted instead.
"make -C $KERNELDIR SUBDIRS=$MYDIR \
CC=what_we_think_is_the_right_gcc_version" modules
This approach seems to work with the "stock" kernels as well as with all
the distribution kernels what we have tried so far (except the latest
kernel from SuSE that used different directory scheme).
It's relatively easy to use the dual directory scheme provided that the
object and source directories always have the same naming. However it's
possible that in the future there will be much more projects that need to
compile external modules. So it would be easier in general if the
(possible) dual directory scheme is "hidden" from the external projects.
It should be easy difficult to patch the Makefile in
/lib/modules/`uname -r`/build to locate the object directory without
requiring the caller to pass it as a parameter to the "make modules"
command.
There is one special problem that doesn't affect users who have compiled
their kernel themselves. Many distributions don't include gcc, make and
related tools in the default installation so large amount of customer
sites don't have them installed. In addition different gcc version may
have to be used to compile the kernel than the one that is the default gcc
command in the system. So before anything can be compiled the installation
scripts have to check also if the required tools are installed. If they
are not available then the customer needs to be instructed to install
them. Because the installation procedure depents on the distribution it's
very difficult to help the customer in any way. For this reason it would
be better if the re is some kind of unbrella script for building the
module. In the vanilla kernels it simply executes the right make command.
If the distribution vendors see it necessary they can improve the script
and add features like automatic installation of the required modules or
whatever else.
Does something like the following sound good?
sh /lib/modules/`uname -r`/build/make_module $MYSUBDIR CC=$CC
$MYSUBDIR is the absolute path to the directory where the module sources
are located.
$CC is optional parameter that the calling module can use if it thinks it
knows the right gcc version. If nothing is given then plain gcc is
assumed. If the script knows the right compiler then it overrides this
parameter.
In the minimalistic form the script is just
#!/bin/sh
make -C /lib/modules/`uname -r`/build SUBDIRS="$1" $2 modules
Best regards,
Hannu
-----
Hannu Savolainen ([email protected])
http://www.opensound.com (Open Sound System (OSS))
http://www.compusonic.fi (Finnish OSS pages)
OH2GLH QTH: Karkkila, Finland LOC: KP20CM
On Monday 21 June 2004 02:29, Hannu Savolainen wrote:
> On Mon, 21 Jun 2004, Sam Ravnborg wrote:
> > > Many external modules, libs, etc use
> > > /lib/modules/`uname -r`/build to locate the _source_, and this will
> > > break them all.
> >
> > Examples please. What I have seen so far is modules that was not
> > adapted to use kbuild when being build.
> > If they fail to do so they are inherently broken.
>
> For example our installation script expects that
> /lib/modules/`uname -r`/build behaves like a symbolic link that points to
> /usr/src/linux-`uname -r`. If /lib/modules/`uname -r`/build doesn't exist
> then /usr/src/linux-`uname -r` will be attempted instead.
>
> "make -C $KERNELDIR SUBDIRS=$MYDIR \
> CC=what_we_think_is_the_right_gcc_version" modules
>
> This approach seems to work with the "stock" kernels as well as with all
> the distribution kernels what we have tried so far (except the latest
> kernel from SuSE that used different directory scheme).
Did you actually try the above command with a SUSE kernel? I don't think so;
one of the earlier postings in the other thread spiced with strong words
tried to do other unusual things.
> It's relatively easy to use the dual directory scheme provided that the
> object and source directories always have the same naming. However it's
> possible that in the future there will be much more projects that need to
> compile external modules. So it would be easier in general if the
> (possible) dual directory scheme is "hidden" from the external projects.
This is already in place with Sam's patches.
> It should be easy difficult to patch the Makefile in
> /lib/modules/`uname -r`/build to locate the object directory without
> requiring the caller to pass it as a parameter to the "make modules"
> command.
There is one source tree for N object trees in general. Going from the N side
to the 1 side makes a lot more sense than going the other way around.
> There is one special problem that doesn't affect users who have compiled
> their kernel themselves. Many distributions don't include gcc, make and
> related tools in the default installation so large amount of customer
> sites don't have them installed.
You will invoke make directly, so you should check if make and the kernel
sources for the running kernel are available.
> In addition different gcc version may
> have to be used to compile the kernel than the one that is the default gcc
> command in the system.
Then this should be defined in the Makefile in the kernel sources. Fixing
problems for people who build their own kernels from vanilla sources without
paying attention to such details will take you nowhere, it will only mess
things up even worse.
> So before anything can be compiled the installation
> scripts have to check also if the required tools are installed. If they
> are not available then the customer needs to be instructed to install
> them. Because the installation procedure depents on the distribution it's
> very difficult to help the customer in any way. For this reason it would
> be better if the re is some kind of unbrella script for building the
> module. In the vanilla kernels it simply executes the right make command.
> If the distribution vendors see it necessary they can improve the script
> and add features like automatic installation of the required modules or
> whatever else.
There could be a few basic checks in kbuild whether or not the required tools
are available. Telling users who compile kernel modules on their own how to
install packages sounds a bit much asked for my taste.
Regards,
--
Andreas Gruenbacher <[email protected]>
SUSE Labs, SUSE LINUX AG
On Sun, Jun 20, 2004 at 11:23:53PM +0200, Sam Ravnborg wrote:
> # If System.map exists, run depmod. This deliberately does not have a
> diff -Nru a/scripts/mkmakefile b/scripts/mkmakefile
> --- /dev/null Wed Dec 31 16:00:00 196900
> +++ b/scripts/mkmakefile 2004-06-20 23:06:03 +02:00
> @@ -0,0 +1,25 @@
> +#!/bin/sh
> +# Generates a small Makefile used in the root of the output
> +# directory, to allow make to be started from there.
> +# The Makefile also allow for more convinient build of external modules
> +
> +# Usage
> +# $1 - Kernel src directory
> +# $2 - Output directory
> +
> +
> +cat << EOF
> +
> +KERNELSRC := $1
> +KERNELOUTPUT := $2
> +
> +MAKEFLAGS += --no-print-directory
> +
> +all:
> + \$(MAKE) -C \$(KERNELSRC) O=\$(KERNELOUTPUT)
> +
> +%:
> + \$(MAKE) -C \$(KERNELSRC) O=\$(KERNELOUTPUT) \$@
> +
> +EOF
> +
In addition to what Arjan said - If you insist on doing it this way,
it would be nice if that generated Makefile could contain also
VERSION = 2
PATCHLEVEL = 6
so I can detect (by grepping that Makefile - or by some other method which
works on 2.0 - 2.6 kernels) which kbuild system I should use - whether 2.6.x,
or 2.4.x, or standalone makefile for 2.2.x and 2.0.x. I see
no way how to find what Makefile I should prepare for this thing.
Other possible solution is that you'll guarantee that this is last version
which will be ever invented. But you probably cannot guarantee this,
so some version number is needed.
Thanks,
Petr Vandrovec
On Mon, Jun 21, 2004 at 03:29:24AM +0300, Hannu Savolainen wrote:
> Does something like the following sound good?
>
> sh /lib/modules/`uname -r`/build/make_module $MYSUBDIR CC=$CC
no you shold not override the compiler; the makefiles should do that instead
automatically
On Mon, 21 Jun 2004, Arjan van de Ven wrote:
> On Mon, Jun 21, 2004 at 03:29:24AM +0300, Hannu Savolainen wrote:
> > Does something like the following sound good?
> >
> > sh /lib/modules/`uname -r`/build/make_module $MYSUBDIR CC=$CC
>
>
> no you shold not override the compiler; the makefiles should do that instead
> automatically
Right. That would be better.
Best regards,
Hannu
-----
Hannu Savolainen ([email protected])
http://www.opensound.com (Open Sound System (OSS))
http://www.compusonic.fi (Finnish OSS pages)
OH2GLH QTH: Karkkila, Finland LOC: KP20CM
On Sun, 20 Jun 2004, Sam Ravnborg wrote:
> # 1) When using a separate output directory create a small
> # Makefile that is a simple wrapper, calling the Makefile
> # in the kernel tree.
> # - This allows the user to shift to the output directory
> # and execute make.
> # - The Makefile is also useful to document the location
> # source used for the kernel
> diff -Nru a/scripts/mkmakefile b/scripts/mkmakefile
> --- /dev/null Wed Dec 31 16:00:00 196900
> +++ b/scripts/mkmakefile 2004-06-20 23:06:03 +02:00
> @@ -0,0 +1,25 @@
> +#!/bin/sh
> +# Generates a small Makefile used in the root of the output
> +# directory, to allow make to be started from there.
> +# The Makefile also allow for more convinient build of external modules
> +
> +# Usage
> +# $1 - Kernel src directory
> +# $2 - Output directory
> +
> +
> +cat << EOF
> +
> +KERNELSRC := $1
> +KERNELOUTPUT := $2
> +
> +MAKEFLAGS += --no-print-directory
> +
> +all:
> + \$(MAKE) -C \$(KERNELSRC) O=\$(KERNELOUTPUT)
> +
> +%:
> + \$(MAKE) -C \$(KERNELSRC) O=\$(KERNELOUTPUT) \$@
> +
> +EOF
The generated Makefile looks sufficiently similar to the one I'm using, so I'm
wondering: Does it work if I say e.g. `make drivers/char/mem.o'? For me that
part never worked, but `make drivers/char/' does work.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
On Mon, Jun 21, 2004 at 11:01:34AM +0200, Geert Uytterhoeven wrote:
> > +
> > +KERNELSRC := $1
> > +KERNELOUTPUT := $2
> > +
> > +MAKEFLAGS += --no-print-directory
> > +
> > +all:
> > + \$(MAKE) -C \$(KERNELSRC) O=\$(KERNELOUTPUT)
> > +
> > +%:
> > + \$(MAKE) -C \$(KERNELSRC) O=\$(KERNELOUTPUT) \$@
> > +
> > +EOF
>
> The generated Makefile looks sufficiently similar to the one I'm using, so I'm
> wondering: Does it work if I say e.g. `make drivers/char/mem.o'? For me that
> part never worked, but `make drivers/char/' does work.
The Makefile is derived from the one you posted, I just typed it by hand.
Therefore the variable names differ.
In the "Match-Anything Rules::" part of 'info make' the trick needed to
let 'make drivers/char/mem.o' work was described - thanks for reminding me of this issue.
The match-anything rule must be marked as terminal rule using double-colon - viola.
So the rule now looks:
%::
\$(MAKE) -C \$(KERNELSRC) O=\$(KERNELOUTPUT) \$@
Sam
On Mon, Jun 21, 2004 at 01:51:43AM +0200, Andreas Gruenbacher wrote:
> > But my original concern (that the only way to figure where the source
> > are for the running kernel will be broken) is still valid.
>
> User-space stuff that needs access to kernel headers at build time is a
> problem. But for those programs, depending on the running kernel instead of
> simply looking in /usr/src/linux is an even bigger problem: What guarantees
> that the first time the program is run, the kernel will still be the same? So
> depending on the running kernel is definitely wrong. Depending
> on /usr/src/linux is also wrong; ideally those programs should look
> in /usr/include/{linux,asm}. Unfortunately these headers are not always
> recent enough, and so recently added definitions may be missing.
But Martin has a point here.
How to figure out for example the number of arguments to remap_page_range.
One could do some grepping in the mm.h file, or one could try to compile
a minimal module calling this function.
If we go for the simple version by grepping we need to figure out where
to find the source. In the past this was simple - just follow the
build symlink. But now kernels may be shipped with separate source
and output directory exposing the weakness of this method.
A much more reliable way is to build a simple module.
If the module build succeeds that specific version
of remap_page_range is OK.
nvidia does something similar, but they take the false assumption
that the running kernel is always the one being build for.
So they call gcc direct.
Other modules uses the grep method - which will fail when the kernel
is build with separate output and source directories.
>
> > Makefile-pre_M_flag - 100% valid kbuild Makefile for kernels that
> > do not support M=
> >
> > Makefile-post_M_flag - 100% valid kbuild Makefile for kernels
> > supporting M=
>
> Right now I would collapse the pre/post Makefiles and use SUBDIRS instead.
> There is no easy and reliable test for M= support, and it's only cosmetic.
> Sam will probably disagree.
SUBDIRS were kept for backward compatibility - and I realise it will stay
for a long time. The implementation were kept straightforward so no problem.
Sam
On Sun, Jun 20, 2004 at 11:26:54PM +0100, Alistair John Strachan wrote:
> [snipped a few CC addresses]
>
> On Sunday 20 June 2004 23:16, Martin Schlemmer wrote:
> > On Mon, 2004-06-21 at 00:03, Sam Ravnborg wrote:
> > > On Sun, Jun 20, 2004 at 11:30:34PM +0200, Martin Schlemmer wrote:
> > > > I know Sam's mta blocks my mail at least (lame isp), but for the rest,
> > > > please reconsider using this.
> > >
> > > Hmm, got your mail.
> > >
> > > > Many external modules, libs, etc use
> > > > /lib/modules/`uname -r`/build to locate the _source_, and this will
> > > > break them all.
> > >
> > > Examples please. What I have seen so far is modules that was not
> > > adapted to use kbuild when being build.
> > > If they fail to do so they are inherently broken.
> >
> > Well, glibc use it for instance as an fall-through if you do not specify
> > it via ./configure arguments, or environment (yes, glibc should not use
> > it, etc, etc, no flames please =). So as well does alsa-driver,
> > nvidia's drivers (gah, puke, yes, its got some binary-only stuff in
> > there ;), ati's drivers and a lot of other stuff (if you really need
> > them all I can try to find time to look for more).
> >
> > I am not sure about ati's drivers and alsa, but nvidia uses kbuild.
> >
> >
> > Thanks,
>
> Sam's point is that unless you ask KBUILD to put the kernel build in a
> separate directory to its sources (this is not the default
> behaviour), /lib/modules/`uname -r`/build will still point to the mixture of
> source and build data, therefore no breakage will occur.
Correct!
> I understand Sam's reasoning and I believe it is sensible to have the source
> and build output in separate directories within /lib/modules/`uname -r`. The
> drivers in question can easily be updated to support the exceptional case
> whereby users build kernels in a different directory to the source.
>
> Sam, maybe if there was a way to easily detect whether a kernel had been build
> with or without a different output directory, it would be easier to have
> vendors take this change on board. For example, I imagine in the typical case
> whereby no change in build directory is made, you will have something like
> this:
>
> /lib/modules/2.6.7/build -> /home/alistair/linux-2.6
> /lib/modules/2.6.7/source -> /home/alistair/linux-2.6
>
> Whereas when O is given, it will instead be like this:
>
> /lib/modules/2.6.7/build -> /home/alistair/my-dir
> /lib/modules/2.6.7/source -> /home/alistair/linux-2.6
>
> I presume that checking for the existence of /lib/modules/`uname -r`/source
> will be enough.
>
> #
> # where's the kernel source?
> #
>
> if [ -d /lib/modules/`uname -r`/source ]; then
> # 2.6.8 and newer
> KERNDIR="/lib/modules/`uname -r`/source"
> else
> # pre 2.6.8 kernels
> KERNDIR="/lib/modules/`uname -r`/build"
> fi
Look ok.
Sam
On Tue, 2004-06-22 at 00:33, Sam Ravnborg wrote:
> On Sun, Jun 20, 2004 at 11:26:54PM +0100, Alistair John Strachan wrote:
> > Sam's point is that unless you ask KBUILD to put the kernel build in a
> > separate directory to its sources (this is not the default
> > behaviour), /lib/modules/`uname -r`/build will still point to the mixture of
> > source and build data, therefore no breakage will occur.
>
> Correct!
>
> > Sam, maybe if there was a way to easily detect whether a kernel had been build
> > with or without a different output directory, it would be easier to have
> > vendors take this change on board. For example, I imagine in the typical case
> > whereby no change in build directory is made, you will have something like
> > this:
> >
> > /lib/modules/2.6.7/build -> /home/alistair/linux-2.6
> > /lib/modules/2.6.7/source -> /home/alistair/linux-2.6
> >
> > Whereas when O is given, it will instead be like this:
> >
> > /lib/modules/2.6.7/build -> /home/alistair/my-dir
> > /lib/modules/2.6.7/source -> /home/alistair/linux-2.6
> >
> > I presume that checking for the existence of /lib/modules/`uname -r`/source
> > will be enough.
> >
> > #
> > # where's the kernel source?
> > #
> >
> > if [ -d /lib/modules/`uname -r`/source ]; then
> > # 2.6.8 and newer
> > KERNDIR="/lib/modules/`uname -r`/source"
> > else
> > # pre 2.6.8 kernels
> > KERNDIR="/lib/modules/`uname -r`/build"
> > fi
>
> Look ok.
>
Yeah, understood, and already seen as the solution.
I guess I just wanted to try and prevent breakage to _existing_
projects, and the need for another hack to try and figure out
how to build for 2.6 (2.6 has been a bumpy ride in external
module building regards =).
But I guess it will not happen, so rather now then much later
when 2.6 is in much wider use ... :-)
Thanks,
--
Martin Schlemmer
On Mon, Jun 21, 2004 at 12:54:01AM +0200, Martin Schlemmer wrote:
> >
> > #
> > # where's the kernel source?
> > #
> >
> > if [ -d /lib/modules/`uname -r`/source ]; then
> > # 2.6.8 and newer
> > KERNDIR="/lib/modules/`uname -r`/source"
> > else
> > # pre 2.6.8 kernels
> > KERNDIR="/lib/modules/`uname -r`/build"
> > fi
> >
> > Yeah?
>
> Yes, as said before, I can understand the name change. The point is
> more that the 'build' symlink will change in behavior in certain
> circumstances, and because many projects already support 2.6, and
> make use of the 'build' symlink, they will break.
But they were already broken.
And the patch actually helps here.
See the following table:
Current kbuild With patch
simple module, no grepping
kernel source + output mixed OK OK
simple module, no grepping
kernel source separate from output FAIL *1 OK
module grepping in src
kernel source + output mixed OK OK
module grepping in src
kernel source separate from output FAIL *2 FAIL *3
FAIL *1 Module cannot build because it fails to locate the compiled files used for kbuild.
Also the kernel configuration is missing.
FAIL *2 Same as FAIL *1. Module cannot build because kbuild compiled files are missing and
it cannot access kernel configuration.
Also it fails to locate files when grepping.
FAIL *3 Grep will fail because it try to grep in files located under build/
The above table clearly shows that this patch fix building external modules in the
case where no grepping were preformed.
And the error were simplified in the last case.
So improvements all over.
What one should realise is that the patch makes it less painfull when kernels
are build using the separate output directory option.
And when that feature is not used - no change in behaviour occur.
Sam
On Mon, Jun 21, 2004 at 12:25:18AM +0200, Martin Schlemmer wrote:
> On Mon, 2004-06-21 at 00:18, Sam Ravnborg wrote:
> > On Mon, Jun 21, 2004 at 12:03:19AM +0200, Sam Ravnborg wrote:
> > > If I get just one good example I will go for the object directory, but
> > > what I have seen so far is whining - no examples.
> >
> > Now I recall why I did not like the object directory.
> > I will break all modules using the kbuild infrastructure!
> >
>
> Below do not really explain this - care to be more detailed?
>
> > Why, because there is no way the to find the output directory except
> > specifying both directories.
> > One could do:
> > make -C /lib/modules/`uname -r`/source O=/lib/modules/`uname -r`/build M=`pwd`
> >
>
> Huh? Explain to me how else you would do builds that have separate
> output directory? And what is the difference from above to:
>
> make -C /lib/modules/`uname -r`/build O=/lib/modules/`uname -r`/object M=`pwd`
Modules do not have O= specified.
And you did not address the issue with modules makefile grepping in src.
Sam
On Tue, 2004-06-22 at 00:31, Sam Ravnborg wrote:
> On Mon, Jun 21, 2004 at 01:51:43AM +0200, Andreas Gruenbacher wrote:
> > > But my original concern (that the only way to figure where the source
> > > are for the running kernel will be broken) is still valid.
> >
> > User-space stuff that needs access to kernel headers at build time is a
> > problem. But for those programs, depending on the running kernel instead of
> > simply looking in /usr/src/linux is an even bigger problem: What guarantees
> > that the first time the program is run, the kernel will still be the same? So
> > depending on the running kernel is definitely wrong. Depending
> > on /usr/src/linux is also wrong; ideally those programs should look
> > in /usr/include/{linux,asm}. Unfortunately these headers are not always
> > recent enough, and so recently added definitions may be missing.
>
> But Martin has a point here.
> How to figure out for example the number of arguments to remap_page_range.
> One could do some grepping in the mm.h file, or one could try to compile
> a minimal module calling this function.
> If we go for the simple version by grepping we need to figure out where
> to find the source. In the past this was simple - just follow the
> build symlink. But now kernels may be shipped with separate source
> and output directory exposing the weakness of this method.
> A much more reliable way is to build a simple module.
> If the module build succeeds that specific version
> of remap_page_range is OK.
>
> nvidia does something similar, but they take the false assumption
> that the running kernel is always the one being build for.
> So they call gcc direct.
>
> Other modules uses the grep method - which will fail when the kernel
> is build with separate output and source directories.
>
Might it help if there was some silly 'external module example toolkit'
or whatever, that shows how to figure out with kbuild how to determine
srcdir, figure out how many args remap_page_range takes, etc?
I will admit that most fixes I do to external modules is maybe not
really because their build systems is 'buggy' as such, but more because
of short-sightedness and lack of understanding kbuild. So above should
help.
Yeah, another project to maintain (in kernel if possible?). I am
willing to try at some of the tests, but my docs skills is no where
good ... :/
Thanks,
--
Martin Schlemmer
On Tuesday 22 June 2004 00:31, Sam Ravnborg wrote:
> But Martin has a point here.
Yes.
> Other modules uses the grep method - which will fail when the kernel
> is build with separate output and source directories.
Is there anything fundamentally wrong with invoking the test script from
within the module makefile, when really necessary? This doesn't work very
well if you need the test results outside of the module build.
----- Makefile -----
test_result := $(shell ...)
ifneq ($(test_result),)
EXTRA_CFLAGS := -DSOMETHING
endif
...
-------- 8< --------
Cheers,
--
Andreas Gruenbacher <[email protected]>
SUSE Labs, SUSE LINUX AG
On Tue, Jun 22, 2004 at 12:50:09AM +0200, Andreas Gruenbacher wrote:
> On Tuesday 22 June 2004 00:31, Sam Ravnborg wrote:
> > But Martin has a point here.
>
> Yes.
>
> > Other modules uses the grep method - which will fail when the kernel
> > is build with separate output and source directories.
>
> Is there anything fundamentally wrong with invoking the test script from
> within the module makefile, when really necessary? This doesn't work very
> well if you need the test results outside of the module build.
>
> ----- Makefile -----
> test_result := $(shell ...)
> ifneq ($(test_result),)
> EXTRA_CFLAGS := -DSOMETHING
> endif
> ...
> -------- 8< --------
I like keeping the kbuild makefile clean as such - but otherwise not.
One has to secure that right CFLAGS etc is used though.
Also when make -j N is used one has to be very carefully with prerequisites - as always.
Sam
On Tuesday 22 June 2004 00:29, Martin Schlemmer wrote:
> I guess I just wanted to try and prevent breakage to _existing_
> projects, and the need for another hack to try and figure out
> how to build for 2.6 (2.6 has been a bumpy ride in external
> module building regards =).
We are *a lot* better now than we ever were before. Thanks, Sam.
Cheers,
--
Andreas Gruenbacher <[email protected]>
SUSE Labs, SUSE LINUX AG
>>>>> "Alistair" == Alistair John Strachan <[email protected]> writes:
Alistair> Sam, maybe if there was a way to easily detect whether a
Alistair> kernel had been build with or without a different output
Alistair> directory, it would be easier to have vendors take this
Alistair> change on board. For example, I imagine in the typical case
Alistair> whereby no change in build directory is made, you will have
Alistair> something like this:
You can do this without any changes:
if [ -d /lib/modules/`uname -r`/build/include2 ]; then
kerndir=`cd /lib/modules/`uname -r`/build/include2/asm; cd -P../..;/bin/pwd`
else
kerndir=`cd /lib/modules/`uname -r`/build; /bin/pwd`
fi
But can the include2/asm symlink be made a relative one, please? I frequently
build on one machine, then NFS-mount the build tree and run make
modules_install somewhere else; I always at present have to convert
that link to a relative symlink before doing so.
--
Dr Peter Chubb http://www.gelato.unsw.edu.au peterc AT gelato.unsw.edu.au
The technical we do immediately, the political takes *forever*
On Tue, Jun 22, 2004 at 11:39:43AM +1000, Peter Chubb wrote:
>
> But can the include2/asm symlink be made a relative one, please? I frequently
> build on one machine, then NFS-mount the build tree and run make
> modules_install somewhere else; I always at present have to convert
> that link to a relative symlink before doing so.
Patch is welcome. I recall having trouble with it when introducing it. But that
can have been caused by other issues.
Sam
Sam Ravnborg wrote:
> On Mon, Jun 21, 2004 at 12:03:19AM +0200, Sam Ravnborg wrote:
> > If I get just one good example I will go for the object directory, but
> > what I have seen so far is whining - no examples.
>
> Now I recall why I did not like the object directory.
> I will break all modules using the kbuild infrastructure!
No it does not. If 'export KBUILD_OUTPUT=/foo' command is used used before
kernel is built, it is not any more difficult to compile external modules
with that same env variable defined.
> Why, because there is no way the to find the output directory except
> specifying both directories.
> One could do:
> make -C /lib/modules/`uname -r`/source O=/lib/modules/`uname -r`/build M=`pwd`
>
> So the currect choice is:
> 1) Break modules that actually dive into the src, grepping, including or whatever
> 2) Break all modules using kbuild infrastructure, including the above ones
or 3) Add the missing object symlink. It does not break anything.
If and when external module build scripts are updated to automatically
detect the /lib/modules/`uname -r`/object symlink, users can even unset the
KBUILD_OUTPUT= env variable and external modules will still compile ok.
> I go for 1), introducing minimal breakage.
You seem to be in some strange "I must destroy... I must destroy... I must
destroy..." mental state. There is a no-breakage alternative and you just
will not consider that at all.
> If kernels are shipped with separate source and output then we better break
> as few modules as possible. And the introduced change actually minimize breakage.
Nope. Solution #3 is the minimal breakage solution.
> So the patch will not change.
In which case Andrew and Linus should consider revoking your kbuild
maintainer status. I moved Andrew and Linus from CC list to TO list in hope
to get this small question answered:
Andrew, Linus,
How about choosing someone else less destructive to maintain kbuild?
> See the following table:
>
> Current kbuild With patch
> simple module, no grepping
> kernel source + output mixed OK OK
>
> simple module, no grepping
> kernel source separate from output FAIL *1 OK
>
> module grepping in src
> kernel source + output mixed OK OK
>
> module grepping in src
> kernel source separate from output FAIL *2 FAIL *3
>
> FAIL *1 Module cannot build because it fails to locate the compiled files used for kbuild.
> Also the kernel configuration is missing.
Same 'export KBUILD_OUTPUT=/foo' will work the same for kernel build and
external module build. Your "FAIL *1" is bogus.
> FAIL *2 Same as FAIL *1. Module cannot build because kbuild compiled files are missing and
> it cannot access kernel configuration.
> Also it fails to locate files when grepping.
Same 'export KBUILD_OUTPUT=/foo' will work the same for kernel build and
external module build. Your "FAIL *2" is also bogus.
> FAIL *3 Grep will fail because it try to grep in files located under build/
Yep.
--
Jari Ruusu 1024R/3A220F51 5B 4B F9 BB D3 3F 52 E9 DB 1D EB E3 24 0E A9 DD
On Tuesday 22 June 2004 07:20, Sam Ravnborg wrote:
> On Tue, Jun 22, 2004 at 11:39:43AM +1000, Peter Chubb wrote:
> > But can the include2/asm symlink be made a relative one, please? I
> > frequently build on one machine, then NFS-mount the build tree and run
> > make modules_install somewhere else; I always at present have to convert
> > that link to a relative symlink before doing so.
>
> Patch is welcome. I recall having trouble with it when introducing it. But
> that can have been caused by other issues.
The same issue exists with the KERNELSRC and KERNELOUTPUT paths in
$(objtree)/Makefile: they would also better be relative to each other. In our
case we have something like:
KERNELSRC := /usr/src/linux-2.6.5-7.79
KERNELOUTPUT := /usr/src/linux-2.6.5-7.79-obj/i386/default
this would become:
KERNELSRC := ../../../linux-2.6.5-7.79
KERNELOUTPUT := ../linux-2.6.5-7.79-obj/i386/default
I am currently regenerating $(objtree)/Makefile by hand by invoking mkmakefile
with the appropriate parameters. The attached script could be used to
automate this; it would work equally well for the asm symlink.
Regards,
--
Andreas Gruenbacher <[email protected]>
SUSE Labs, SUSE LINUX AG
On Tuesday 22 June 2004 07:29, Jari Ruusu wrote:
> Sam Ravnborg wrote:
> > On Mon, Jun 21, 2004 at 12:03:19AM +0200, Sam Ravnborg wrote:
> > > If I get just one good example I will go for the object directory, but
> > > what I have seen so far is whining - no examples.
> >
> > Now I recall why I did not like the object directory.
> > I will break all modules using the kbuild infrastructure!
>
> No it does not. If 'export KBUILD_OUTPUT=/foo' command is used before
> kernel is built, it is not any more difficult to compile external modules
> with that same env variable defined.
This clearly is not an option. We want the most trivial way of building
external modules to continue working, no matter whether the kernel is using a
separate output directory or not; with Sam's patch we get that:
make -C /lib/modules/$(uname -r)/build M=$(pwd)
We also want to build modules for other configurations; this is as simple as
passing another path in -C. For example, in the SUSE setup this would give
you a module for an i386 bigsmp kernel:
make -C /usr/scc/linux-obj/i386/bigsmp M=$(pwd)
The environment variable proposal is worthless: Where on earth should that
environment variable come from?
> [cut the crap]
>
> In which case Andrew and Linus should consider revoking your kbuild
> maintainer status. I moved Andrew and Linus from CC list to TO list in hope
> to get this small question answered:
>
> Andrew, Linus,
> How about choosing someone else less destructive to maintain kbuild?
Jari, you have shown that you still didn't get it. Trying harder instead of
badmouthing random people might help.
Regards,
--
Andreas Gruenbacher <[email protected]>
SUSE Labs, SUSE LINUX AG
Andreas Gruenbacher wrote:
> On Tuesday 22 June 2004 07:29, Jari Ruusu wrote:
> > Sam Ravnborg wrote:
> > > Now I recall why I did not like the object directory.
> > > I will break all modules using the kbuild infrastructure!
> >
> > No it does not. If 'export KBUILD_OUTPUT=/foo' command is used before
> > kernel is built, it is not any more difficult to compile external modules
> > with that same env variable defined.
>
> This clearly is not an option. We want the most trivial way of building
> external modules to continue working, no matter whether the kernel is using a
> separate output directory or not; with Sam's patch we get that:
>
> make -C /lib/modules/$(uname -r)/build M=$(pwd)
Kernel already does that correctly, and follows KBUILD_OUTPUT env variable
that points to where user wants the object tree. No changes needed to
external module build scripts.
Maybe you SUSE guys just didn't realize that.
> We also want to build modules for other configurations; this is as simple as
> passing another path in -C. For example, in the SUSE setup this would give
> you a module for an i386 bigsmp kernel:
>
> make -C /usr/scc/linux-obj/i386/bigsmp M=$(pwd)
This is cool and desirable, but does not need or even justify
breaking/redirecting the 'build' symlink elsewhere.
> The environment variable proposal is worthless:
It is not a proposal. It has been in mainline since 2.5.x kernels and last
time I checked, it worked fine.
> Where on earth should that environment variable come from?
The person compiling kernel decides where he wants the object tree. He only
needs to set that once, before builing a kernel with separate object and
source trees. And same env variable works just fine when used with
externally compiled modules.
--
Jari Ruusu 1024R/3A220F51 5B 4B F9 BB D3 3F 52 E9 DB 1D EB E3 24 0E A9 DD
On Tue, Jun 22, 2004 at 08:29:55AM +0300, Jari Ruusu wrote:
> Sam Ravnborg wrote:
> > On Mon, Jun 21, 2004 at 12:03:19AM +0200, Sam Ravnborg wrote:
> > > If I get just one good example I will go for the object directory, but
> > > what I have seen so far is whining - no examples.
> >
> > Now I recall why I did not like the object directory.
> > I will break all modules using the kbuild infrastructure!
>
> No it does not. If 'export KBUILD_OUTPUT=/foo' command is used used before
> kernel is built, it is not any more difficult to compile external modules
> with that same env variable defined.
>
> > Why, because there is no way the to find the output directory except
> > specifying both directories.
> > One could do:
> > make -C /lib/modules/`uname -r`/source O=/lib/modules/`uname -r`/build M=`pwd`
> >
> > So the currect choice is:
> > 1) Break modules that actually dive into the src, grepping, including or whatever
> > 2) Break all modules using kbuild infrastructure, including the above ones
>
> or 3) Add the missing object symlink. It does not break anything.
This is unfortunately not the case.
First a few facts.
The documented way to build external modules has for a while now been
to use the following command:
make -C /lib/modules/`uname -r`/build SUBDIRS=`pwd` modules
Recently this was simplifed to
make -C /lib/modules/`uname -r`/build M=`pwd`
As you pointed out in a previous post letting build point to the
output directory would break all existing drivers using the documented
approach, iff the kernel were compiled using a separate output directory.
So the breakage that was of a concern was how to build external
modules using the above command - but having the kernel build
using a separate output directory.
The was solved by adding a small makefile in the output directory.
So the situation with the patch posted is that all ordinary modules
compile independent of the kernel is compiled with separate output
directories or not.
The problem you point out is modules that to be backward compatible
needs access to the kernel source. This is solved in different ways:
a) Grepping the source direct
b) Compiling a small code fragment, and test if the .o file exists
Knowing your background you are most familiar with the a) approach.
Therefore I understand why you prefer to set an environment variable
and otherwise use your current makefile.
But doing so will break all 'ordinary' external modules.
The ordinary modules are adapted to use the above commands
to compile the module.
But if the build symlinks points to the source of the
kernel and the kernel uses separate directory for output files then
it is missing the possibility to:
- access .config
- use kbuild infrastructure
- include headers under asm symlink
All the abvoe are fatal.
The porposal to require the user to set KBUILD_OUTPUT before
compiling the module would fix this. But this is then a change
in the required command used to compile an 'ordinary' external module.
It would then look like:
KBUILD_OUTPUT=/lib/modules/`uname -r`/object; make -C /lib/modules/`uname -r`/build M=`pwd`
This change is not acceptable.
The tradeoff is that external modules that needs access to the source
needs to be tweaked to support kernels build using separate output and
source directories.
So the final result is that external modules that needs access to
the source is broken iff the kernel uses separate output directories.
> > I go for 1), introducing minimal breakage.
>
> You seem to be in some strange "I must destroy... I must destroy... I must
> destroy..." mental state. There is a no-breakage alternative and you just
> will not consider that at all.
Requiring user to set an environment variable in many common cases
are breakage.
Sam