Just a quick intro to what is pending in kbuild.git/lxdialog.git for 2.6.19.
And a short status too.
Highlights:
o unifdef is now included in the kernel source (used by
headers_* targets).
o lxdialog is now built-in (this did not remove all flickering
but thats much easier to fix now).
o Added color theme support to lxdialog (see help in menuconfig)
o modpost is run twice on vmlinux in normal (modules included)
builds - which may result in duplicated warnings.
It is run twice to make sure we run modpost in non-modules
build. modpost does the section mismatch chcks in all cases
o try the new "make V=2" thing. It tells why a target got
rebuild. It is good to catch those "why the heck did kbuild
rebuild that much after changing just a tine CONFIG_ option".
All of this is in latest -mm, and git trees at kernel.org.
Outstanding kbuild issues (I should fix a few of these for 2.6.18):
o make -j N is not as parallel as expected (latest report from Keith
Ownens but others has complained as well). I assume it is a kbuild
thing but has no clue how to fix it or debug it further.
o symlinked output directory is no good (Andi Kleen)
o dependency errors in scripts/ (Olaf Hering)
o fix -Wshadow in scripts/ (Jesper Juhl)
I add quite a few warnings in lxdialog patch serie :-(
o Add a few more color themses from Jan Engelhardt
o bugzilla has a few more issues that needs some love & care
Sam
Jan Engelhardt:
kconfig: linguistic fixes for Documentation/kbuild/kconfig-language.txt
kbuild: linguistic fixes for Documentation/kbuild/modules.txt
kbuild: linguistic fixes for Documentation/kbuild/makefiles.txt
Magnus Damm:
kbuild: ignore references from ".pci_fixup" to ".init.text"
Matthew Wilcox:
kconfig: support DOS line endings
Olaf Hering:
remove RPM_BUILD_ROOT from asm-offsets.h
Sam Ravnborg:
kbuild: consistently decide when to rebuild a target
kbuild: add unifdef
kbuild: replace use of strlcpy with a dedicated implmentation in unifdef
kbuild: use in-kernel unifdef
kbuild: create output directory for hostprogs with O=.. build
kbuild: remove debug left-over from Makefile.host
kbuild: modpost on vmlinux regardless of CONFIG_MODULES
kbuild: make V=2 tell why a target is rebuild
kbuild: make -rR is now default
kbuild: preperly align SYSMAP output
kbuild: add missing return statement in modpost.c:secref_whitelist()
Sam Ravnborg:
kconfig/lxdialog: refactor color support
kconfig/lxdialog: add support for color themes and add blackbg theme
kconfig/lxdialog: add a new theme bluetitle which is now default
kconfig/menuconfig: lxdialog is now built-in
kconfig/lxdialog: let <ESC><ESC> behave as expected
kconfig/lxdialog: support resize
kconfig/lxdialog: fix make mrproper
On Sun, 13 Aug 2006 21:45:03 +0200 Sam Ravnborg wrote:
> Just a quick intro to what is pending in kbuild.git/lxdialog.git
> for 2.6.19. And a short status too.
>
> o try the new "make V=2" thing. It tells why a target got
> rebuild. It is good to catch those "why the heck did
> kbuild rebuild that much after changing just a tine CONFIG_ option".
Thanks, will use it.
> Jan Engelhardt:
> kconfig: linguistic fixes for
> Documentation/kbuild/kconfig-language.txt kbuild: linguistic fixes
> for Documentation/kbuild/modules.txt kbuild: linguistic fixes for
> Documentation/kbuild/makefiles.txt
Were any of these used?
http://marc.theaimsgroup.com/?l=linux-kernel&m=115410865910922&w=2
---
~Randy
On Sun, 13 Aug 2006 13:54:26 -0700 Randy.Dunlap wrote:
> On Sun, 13 Aug 2006 21:45:03 +0200 Sam Ravnborg wrote:
>
> > Just a quick intro to what is pending in kbuild.git/lxdialog.git
> > for 2.6.19. And a short status too.
> >
> > o try the new "make V=2" thing. It tells why a target got
> > rebuild. It is good to catch those "why the heck did
> > kbuild rebuild that much after changing just a tine CONFIG_
> > option".
>
> Thanks, will use it.
>
> > Jan Engelhardt:
> > kconfig: linguistic fixes for
> > Documentation/kbuild/kconfig-language.txt kbuild: linguistic fixes
> > for Documentation/kbuild/modules.txt kbuild: linguistic fixes for
> > Documentation/kbuild/makefiles.txt
>
> Were any of these used?
> http://marc.theaimsgroup.com/?l=linux-kernel&m=115410865910922&w=2
and even that has a few typos :(
---
~Randy
On Sun, Aug 13, 2006 at 01:57:54PM -0700, Randy.Dunlap wrote:
> On Sun, 13 Aug 2006 13:54:26 -0700 Randy.Dunlap wrote:
>
> > On Sun, 13 Aug 2006 21:45:03 +0200 Sam Ravnborg wrote:
> >
> > > Just a quick intro to what is pending in kbuild.git/lxdialog.git
> > > for 2.6.19. And a short status too.
> > >
> > > o try the new "make V=2" thing. It tells why a target got
> > > rebuild. It is good to catch those "why the heck did
> > > kbuild rebuild that much after changing just a tine CONFIG_
> > > option".
> >
> > Thanks, will use it.
> >
> > > Jan Engelhardt:
> > > kconfig: linguistic fixes for
> > > Documentation/kbuild/kconfig-language.txt kbuild: linguistic fixes
> > > for Documentation/kbuild/modules.txt kbuild: linguistic fixes for
> > > Documentation/kbuild/makefiles.txt
> >
> > Were any of these used?
> > http://marc.theaimsgroup.com/?l=linux-kernel&m=115410865910922&w=2
No. I have your original mail queued but never got around to it.
A resubmit would be appreciated - with proper Signed-of-by:...
Sam
On Sun, 13 Aug 2006 23:09:07 +0200 Sam Ravnborg wrote:
> On Sun, Aug 13, 2006 at 01:57:54PM -0700, Randy.Dunlap wrote:
> > On Sun, 13 Aug 2006 13:54:26 -0700 Randy.Dunlap wrote:
> >
> > > On Sun, 13 Aug 2006 21:45:03 +0200 Sam Ravnborg wrote:
> > >
> > > > Just a quick intro to what is pending in
> > > > kbuild.git/lxdialog.git for 2.6.19. And a short status too.
> > > >
> > > > o try the new "make V=2" thing. It tells why a target got
> > > > rebuild. It is good to catch those "why the heck did
> > > > kbuild rebuild that much after changing just a tine CONFIG_
> > > > option".
> > >
> > > Thanks, will use it.
> > >
> > > > Jan Engelhardt:
> > > > kconfig: linguistic fixes for
> > > > Documentation/kbuild/kconfig-language.txt kbuild: linguistic
> > > > fixes for Documentation/kbuild/modules.txt kbuild: linguistic
> > > > fixes for Documentation/kbuild/makefiles.txt
> > >
> > > Were any of these used?
> > > http://marc.theaimsgroup.com/?l=linux-kernel&m=115410865910922&w=2
>
> No. I have your original mail queued but never got around to it.
> A resubmit would be appreciated - with proper Signed-of-by:...
It didn't need an sob since it was not a patch.
Anyway, I'll get around tuit (and fix my typos).
---
~Randy
On Sun, 13 Aug 2006 23:09:07 +0200 Sam Ravnborg wrote:
> On Sun, Aug 13, 2006 at 01:57:54PM -0700, Randy.Dunlap wrote:
> > On Sun, 13 Aug 2006 13:54:26 -0700 Randy.Dunlap wrote:
> >
> > > Were any of these used?
> > > http://marc.theaimsgroup.com/?l=linux-kernel&m=115410865910922&w=2
>
> No. I have your original mail queued but never got around to it.
> A resubmit would be appreciated - with proper Signed-of-by:...
---
From: Randy Dunlap <[email protected]>
Fix typos/spellos in kbuild/makefiles.txt.
Signed-off-by: Randy Dunlap <[email protected]>
---
Documentation/kbuild/makefiles.txt | 33 +++++++++++++++++----------------
1 files changed, 17 insertions(+), 16 deletions(-)
--- linux-2618-rc4-ext4.orig/Documentation/kbuild/makefiles.txt
+++ linux-2618-rc4-ext4/Documentation/kbuild/makefiles.txt
@@ -34,7 +34,7 @@ This document describes the Linux kernel
--- 6.1 Set variables to tweak the build to the architecture
--- 6.2 Add prerequisites to archprepare:
--- 6.3 List directories to visit when descending
- --- 6.4 Architecture specific boot images
+ --- 6.4 Architecture-specific boot images
--- 6.5 Building non-kbuild targets
--- 6.6 Commands useful for building a boot image
--- 6.7 Custom kbuild commands
@@ -124,7 +124,7 @@ more details, with real examples.
Example:
obj-y += foo.o
- This tell kbuild that there is one object in that directory, named
+ This tells kbuild that there is one object in that directory, named
foo.o. foo.o will be built from foo.c or foo.S.
If foo.o shall be built as a module, the variable obj-m is used.
@@ -353,7 +353,7 @@ more details, with real examples.
Special rules are used when the kbuild infrastructure does
not provide the required support. A typical example is
header files generated during the build process.
- Another example are the architecture specific Makefiles which
+ Another example are the architecture-specific Makefiles which
need special rules to prepare boot images etc.
Special rules are written as normal Make rules.
@@ -416,7 +416,7 @@ more details, with real examples.
#arch/i386/kernel/Makefile
vsyscall-flags += $(call ld-option, -Wl$(comma)--hash-style=sysv)
- In the above example vsyscall-flags will be assigned the option
+ In the above example, vsyscall-flags will be assigned the option
-Wl$(comma)--hash-style=sysv if it is supported by $(CC).
The second argument is optional, and if supplied will be used
if first argument is not supported.
@@ -429,7 +429,7 @@ more details, with real examples.
#arch/i386/Makefile
cflags-y += $(call cc-option,-march=pentium-mmx,-march=i586)
- In the above example cflags-y will be assigned the option
+ In the above example, cflags-y will be assigned the option
-march=pentium-mmx if supported by $(CC), otherwise -march=i586.
The second argument to cc-option is optional, and if omitted,
cflags-y will be assigned no value if first option is not supported.
@@ -650,14 +650,14 @@ Both possibilities are described in the
--- 4.7 Using hostprogs-$(CONFIG_FOO)
- A typcal pattern in a Kbuild file looks like this:
+ A typical pattern in a Kbuild file looks like this:
Example:
#scripts/Makefile
hostprogs-$(CONFIG_KALLSYMS) += kallsyms
Kbuild knows about both 'y' for built-in and 'm' for module.
- So if a config symbol evaluate to 'm', kbuild will still build
+ So if a config symbol evaluates to 'm', kbuild will still build
the binary. In other words, Kbuild handles hostprogs-m exactly
like hostprogs-y. But only hostprogs-y is recommended to be used
when no CONFIG symbols are involved.
@@ -744,10 +744,10 @@ When kbuild executes, the following step
located at the root of the obj tree.
The very first objects linked are listed in head-y, assigned by
arch/$(ARCH)/Makefile.
-7) Finally, the architecture specific part does any required post processing
+7) Finally, the architecture-specific part does any required post processing
and builds the final bootimage.
- This includes building boot records
- - Preparing initrd images and thelike
+ - Preparing initrd images and the like
--- 6.1 Set variables to tweak the build to the architecture
@@ -874,7 +874,7 @@ When kbuild executes, the following step
$(head-y) lists objects to be linked first in vmlinux.
$(libs-y) lists directories where a lib.a archive can be located.
- The rest lists directories where a built-in.o object file can be
+ The rest list directories where a built-in.o object file can be
located.
$(init-y) objects will be located after $(head-y).
@@ -882,7 +882,7 @@ When kbuild executes, the following step
$(core-y), $(libs-y), $(drivers-y) and $(net-y).
The top level Makefile defines values for all generic directories,
- and arch/$(ARCH)/Makefile only adds architecture specific directories.
+ and arch/$(ARCH)/Makefile only adds architecture-specific directories.
Example:
#arch/sparc64/Makefile
@@ -891,7 +891,7 @@ When kbuild executes, the following step
drivers-$(CONFIG_OPROFILE) += arch/sparc64/oprofile/
---- 6.4 Architecture specific boot images
+--- 6.4 Architecture-specific boot images
An arch Makefile specifies goals that take the vmlinux file, compress
it, wrap it in bootstrapping code, and copy the resulting files
@@ -918,7 +918,7 @@ When kbuild executes, the following step
"$(Q)$(MAKE) $(build)=<dir>" is the recommended way to invoke
make in a subdirectory.
- There are no rules for naming architecture specific targets,
+ There are no rules for naming architecture-specific targets,
but executing "make help" will list all relevant targets.
To support this, $(archhelp) must be defined.
@@ -976,7 +976,7 @@ When kbuild executes, the following step
$(call if_changed,ld/objcopy/gzip)
When the rule is evaluated, it is checked to see if any files
- needs an update, or the command line has changed since the last
+ need an update, or the command line has changed since the last
invocation. The latter will force a rebuild if any options
to the executable have changed.
Any target that utilises if_changed must be listed in $(targets),
@@ -1016,7 +1016,7 @@ When kbuild executes, the following step
In this example, there are two possible targets, requiring different
options to the linker. The linker options are specified using the
LDFLAGS_$@ syntax - one for each potential target.
- $(targets) are assinged all potential targets, by which kbuild knows
+ $(targets) are assigned all potential targets, by which kbuild knows
the targets and will:
1) check for commandline changes
2) delete target during make clean
@@ -1083,7 +1083,7 @@ When kbuild executes, the following step
assignment.
The kbuild infrastructure for *lds file are used in several
- architecture specific files.
+ architecture-specific files.
=== 7 Kbuild Variables
@@ -1127,7 +1127,7 @@ The top Makefile exports the following v
This variable defines a place for the arch Makefiles to install
the resident kernel image and System.map file.
- Use this for architecture specific install targets.
+ Use this for architecture-specific install targets.
INSTALL_MOD_PATH, MODLIB
Sam Ravnborg (on Sun, 13 Aug 2006 21:45:03 +0200) wrote:
>Outstanding kbuild issues (I should fix a few of these for 2.6.18):
>o make -j N is not as parallel as expected (latest report from Keith
> Ownens but others has complained as well). I assume it is a kbuild
> thing but has no clue how to fix it or debug it further.
It is the make jobserver code. make -j<n> causes the various make
tasks to communicate and work out how many versions are currently
running, to avoid overrunning the -j<n> value. Every recursive
invocation of make subtracts one from the -j value, reducing the value
that is left when make finally get down to doing some useful work
instead of just recursing. Jobserver problems are yet another reason
why recursive make is bad.
kbuild is full of recursive make. The user cannot just add an excess
to <n>, the number of recursive invocations changes from kernel to
kernel as people try to fix bugs in makefile generation, so the
required excess value keeps changing.
Before somebody suggests it: the makefile cannot detect the supplied
value of <n> and specify a modified -j<n> on recursive make commands.
make will detect that a sub-make has -j<n>, complain about it and turn
off the jobserver completely.
Keith, 2.4 kernel build doesn't seem to have this issue. Is this problem
specific to 2.6 kbuild?
"info make" says this.
<snip>
Note that any job that is marked recursive (*note Instead of Executing
the Commands: Instead of Execution.) doesn't count against the total
jobs (otherwise we could get `N' sub-`make's running and have no slots
left over for any real work!)
</snip>
Are we doing something different in 2.6 kbuild?
thanks,
suresh
On Mon, Aug 14, 2006 at 12:02:09AM -0700, Keith Owens wrote:
>Sam Ravnborg (on Sun, 13 Aug 2006 21:45:03 +0200) wrote:
>>Outstanding kbuild issues (I should fix a few of these for 2.6.18):
>>o make -j N is not as parallel as expected (latest report from Keith
>> Ownens but others has complained as well). I assume it is a kbuild
>> thing but has no clue how to fix it or debug it further.
>
>It is the make jobserver code. make -j<n> causes the various make
>tasks to communicate and work out how many versions are currently
>running, to avoid overrunning the -j<n> value. Every recursive
>invocation of make subtracts one from the -j value, reducing the value
>that is left when make finally get down to doing some useful work
>instead of just recursing. Jobserver problems are yet another reason
>why recursive make is bad.
>
>kbuild is full of recursive make. The user cannot just add an excess
>to <n>, the number of recursive invocations changes from kernel to
>kernel as people try to fix bugs in makefile generation, so the
>required excess value keeps changing.
>
>Before somebody suggests it: the makefile cannot detect the supplied
>value of <n> and specify a modified -j<n> on recursive make commands.
>make will detect that a sub-make has -j<n>, complain about it and turn
>off the jobserver completely.
On Mon, Aug 14, 2006 at 12:02:09AM -0700, Keith Owens wrote:
>Sam Ravnborg (on Sun, 13 Aug 2006 21:45:03 +0200) wrote:
>>Outstanding kbuild issues (I should fix a few of these for 2.6.18):
>>o make -j N is not as parallel as expected (latest report from Keith
>> Ownens but others has complained as well). I assume it is a kbuild
>> thing but has no clue how to fix it or debug it further.
>
>It is the make jobserver code. make -j<n> causes the various make
>tasks to communicate and work out how many versions are currently
>running, to avoid overrunning the -j<n> value. Every recursive
>invocation of make subtracts one from the -j value, reducing the value
>that is left when make finally get down to doing some useful work
>instead of just recursing. Jobserver problems are yet another reason
>why recursive make is bad.
>
>kbuild is full of recursive make. The user cannot just add an excess
>to <n>, the number of recursive invocations changes from kernel to
>kernel as people try to fix bugs in makefile generation, so the
>required excess value keeps changing.
This is beginning to look like a bug in make which is exacerbated by
the large number of recursive make commands issued in 2.6. Using an
instrumented version of make 3.80, I see some jobserver tokens being
consumed and not returned to the pool, which results in a restricted
number of processes actually running.
The bug is timing dependent, it depends on when child processes die and
are reaped relative to the rest of the make process tree. Recursive
make generates many more processes when compared to running make
against a single Makefile, these extra processes affect the timing.
I am testing a couple of patches against make which will be going to
gnu.org. When I get a reply from gnu.org I will do a follow up to
lkml.
BTW, make -j<n> does not quite do what you expect. info make says "If
the '-j' option is followed by an integer, this is the number of
commands to execute at once". Looking at the source for make, the top
level make task actually consumes one of those slots, even though it
does little except run sub-makes. Even with my bug fix, make -j4
really only uses about 3.3 cpus, 3 doing compiles and about 0.3 of a
load keeping track of the first level sub-makes.
On Mon, Aug 14, 2006 at 12:02:09AM -0700, Keith Owens wrote:
>Sam Ravnborg (on Sun, 13 Aug 2006 21:45:03 +0200) wrote:
>>Outstanding kbuild issues (I should fix a few of these for 2.6.18):
>>o make -j N is not as parallel as expected (latest report from Keith
>> Ownens but others has complained as well). I assume it is a kbuild
>> thing but has no clue how to fix it or debug it further.
>
>It is the make jobserver code. make -j<n> causes the various make
>tasks to communicate and work out how many versions are currently
>running, to avoid overrunning the -j<n> value. Every recursive
>invocation of make subtracts one from the -j value, reducing the value
>that is left when make finally get down to doing some useful work
>instead of just recursing. Jobserver problems are yet another reason
>why recursive make is bad.
>
>kbuild is full of recursive make. The user cannot just add an excess
>to <n>, the number of recursive invocations changes from kernel to
>kernel as people try to fix bugs in makefile generation, so the
>required excess value keeps changing.
The jobserver in make 3.80 is buggy. 3.80 appears to work for parallel
builds using a single Makefile, with recursive make it can lose
jobserver tokens. make 3.81 works fine. Now to persuade SuSE to
upgrade to make 3.81.
On Tue, Aug 15, 2006 at 04:25:59PM +1000, Keith Owens wrote:
> The jobserver in make 3.80 is buggy. 3.80 appears to work for parallel
> builds using a single Makefile, with recursive make it can lose
> jobserver tokens. make 3.81 works fine. Now to persuade SuSE to
> upgrade to make 3.81.
sam@mars ~ $ make --version
GNU Make 3.81
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
This program built for x86_64-pc-linux-gnu
That explains why I do not see it.
Thanks for investigating this!
Sam
On Sun, 13 Aug 2006 21:45:03 +0200, Sam Ravnborg wrote:
> Just a quick intro to what is pending in kbuild.git/lxdialog.git for 2.6.19.
> And a short status too.
>
> Highlights:
> o unifdef is now included in the kernel source (used by
> headers_* targets).
Hi Sam,
This apparently doesn't build:
CHK include/linux/version.h
UPD include/linux/version.h
HOSTCC scripts/basic/fixdep
HOSTCC scripts/basic/docproc
HOSTCC scripts/unifdef
/tmp/ccwcmPxS.o: In function `keywordedit':
unifdef.c:(.text+0x25c): undefined reference to `strlcpy'
collect2: ld returned 1 exit status
make[1]: *** [scripts/unifdef] Error 1
make: *** [headers_install] Error 2
AFAICT, strlcpy is a BSD'ism and isn't generally available to userland on
Linux (but of course the kernel has its own strlcpy implementation).
Debian solve this by including a separate strlcpy.c with the unifdef
source. See:
http://ftp.debian.org/debian/pool/main/u/unifdef/unifdef_1.0+20030701.orig.tar.gz
Regards
Greg
On Fri, Aug 18, 2006 at 06:26:07PM +1000, Greg Schafer wrote:
> On Sun, 13 Aug 2006 21:45:03 +0200, Sam Ravnborg wrote:
>
> > Just a quick intro to what is pending in kbuild.git/lxdialog.git for 2.6.19.
> > And a short status too.
> >
> > Highlights:
> > o unifdef is now included in the kernel source (used by
> > headers_* targets).
>
> Hi Sam,
>
> This apparently doesn't build:
>
> CHK include/linux/version.h
> UPD include/linux/version.h
> HOSTCC scripts/basic/fixdep
> HOSTCC scripts/basic/docproc
> HOSTCC scripts/unifdef
> /tmp/ccwcmPxS.o: In function `keywordedit':
> unifdef.c:(.text+0x25c): undefined reference to `strlcpy'
> collect2: ld returned 1 exit status
> make[1]: *** [scripts/unifdef] Error 1
> make: *** [headers_install] Error 2
When I grep in the unifdef sources I get only one reference to strlcpy.
That's a prototype that was missed when I replaced the use of strlcpy
with a dedicated implementation.
I have checked latest -mm - same result.
Try to doublecheck that you have properly updated your source tree.
Sam
On Fri, Aug 18, 2006 at 07:13:35PM +0200, Sam Ravnborg wrote:
> On Fri, Aug 18, 2006 at 06:26:07PM +1000, Greg Schafer wrote:
> > HOSTCC scripts/unifdef
> > /tmp/ccwcmPxS.o: In function `keywordedit':
> > unifdef.c:(.text+0x25c): undefined reference to `strlcpy'
> > collect2: ld returned 1 exit status
> > make[1]: *** [scripts/unifdef] Error 1
> > make: *** [headers_install] Error 2
>
> When I grep in the unifdef sources I get only one reference to strlcpy.
> That's a prototype that was missed when I replaced the use of strlcpy
> with a dedicated implementation.
Arghhh, I was cherry-picking patches and missed "replace use of strlcpy with
a dedicated implmentation in unifdef". Sorry for the noise..
Regards
Greg