2006-03-12 17:25:21

by Petr Vandrovec

[permalink] [raw]
Subject: [PATCH] Do not rebuild full kernel tree again and again...

Hello,
I've returned back to the computer after month and half, and I've found that
I cannot reasonably build kernel - just repeating 'make' twice without doing any
change forced full rebuild of everything, which as far as I can tell is not
intended behavior. As I did not notice this being reported on the LKML, and
2.6.16-rc6 still suffers from this problem, here comes report + patch.

Problem seems to be that new make includes FORCE prerequisite in $? - apparently
new make treats $? not as 'prerequisities newer than target', but as
'prerequisities newer than target or missing'. Due to this $(if) in make rules
always succeeded as it always received at least 'FORCE', and 'FORCE'
is not an empty string. So let's filter out 'FORCE' from $? - unless somebody
can confirm that make 3.81 is broken and unusable for kernel...
Thanks,
Petr Vandrovec


ppc:~# make -v
GNU Make 3.81rc1
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i486-pc-linux-gnu
ppc:~# dpkg -l make
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Installed/Config-files/Unpacked/Failed-config/Half-installed
|/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
||/ Name Version Description
+++-===============-=====================-============================================
ii make 3.80+3.81.rc1-1 The GNU version of the "make" utility.
ppc:~#

Signed-off-by: Petr Vandrovec <[email protected]>

diff -urdN linux/scripts/Kbuild.include linux/scripts/Kbuild.include
--- linux/scripts/Kbuild.include 2006-03-12 17:28:37.000000000 +0100
+++ linux/scripts/Kbuild.include 2006-03-12 17:55:53.000000000 +0100
@@ -79,7 +79,7 @@
# >'< substitution is for echo to work, >$< substitution to preserve $ when reloading .cmd file
# note: when using inline perl scripts [perl -e '...$$t=1;...'] in $(cmd_xxx) double $$ your perl vars
#
-if_changed = $(if $(strip $? $(call arg-check, $(cmd_$(1)), $(cmd_$@)) ), \
+if_changed = $(if $(strip $(filter-out FORCE,$?) $(call arg-check, $(cmd_$(1)), $(cmd_$@)) ), \
@set -e; \
$(echo-cmd) \
$(cmd_$(1)); \
@@ -87,7 +87,7 @@

# execute the command and also postprocess generated .d dependencies
# file
-if_changed_dep = $(if $(strip $? $(filter-out FORCE $(wildcard $^),$^)\
+if_changed_dep = $(if $(strip $(filter-out FORCE,$? $(filter-out $(wildcard $^),$^))\
$(call arg-check, $(cmd_$(1)), $(cmd_$@)) ), \
@set -e; \
$(echo-cmd) \
@@ -99,6 +99,6 @@
# Usage: $(call if_changed_rule,foo)
# will check if $(cmd_foo) changed, or any of the prequisites changed,
# and if so will execute $(rule_foo)
-if_changed_rule = $(if $(strip $? $(call arg-check, $(cmd_$(1)), $(cmd_$@)) ),\
+if_changed_rule = $(if $(strip $(filter-out FORCE,$?) $(call arg-check, $(cmd_$(1)), $(cmd_$@)) ),\
@set -e; \
$(rule_$(1)))


2006-03-12 17:43:03

by Sam Ravnborg

[permalink] [raw]
Subject: Re: [PATCH] Do not rebuild full kernel tree again and again...

On Sun, Mar 12, 2006 at 06:25:11PM +0100, Petr Vandrovec wrote:
> Hello,
> I've returned back to the computer after month and half, and I've found that
> I cannot reasonably build kernel - just repeating 'make' twice without doing any
> change forced full rebuild of everything, which as far as I can tell is not
> intended behavior. As I did not notice this being reported on the LKML, and
> 2.6.16-rc6 still suffers from this problem, here comes report + patch.
>
> Problem seems to be that new make includes FORCE prerequisite in $? - apparently
> new make treats $? not as 'prerequisities newer than target', but as
> 'prerequisities newer than target or missing'. Due to this $(if) in make rules
> always succeeded as it always received at least 'FORCE', and 'FORCE'
> is not an empty string. So let's filter out 'FORCE' from $? - unless somebody
> can confirm that make 3.81 is broken and unusable for kernel...

This issue has been discussed with Paul D. Smith which also provided
a patch similar to yours.
The patch is in the kbuild queue for 2.6.17.
But we also agreed to postpose this change in make until next stable
release. So if you update your make to latest CVS version you will no
longer see this misbehaviour.
And when 2.6.17 opens up kbuild will be 'fixed' in mainline kernel.

Sam

2006-03-13 06:10:46

by Nick Piggin

[permalink] [raw]
Subject: Re: [PATCH] Do not rebuild full kernel tree again and again...

Sam Ravnborg wrote:

> This issue has been discussed with Paul D. Smith which also provided
> a patch similar to yours.
> The patch is in the kbuild queue for 2.6.17.
> But we also agreed to postpose this change in make until next stable
> release. So if you update your make to latest CVS version you will no
> longer see this misbehaviour.
> And when 2.6.17 opens up kbuild will be 'fixed' in mainline kernel.
>

So what's going to be done about 2.6.16?

I'm seeing this behaviour too in -rc6 and it is a bad regression
for a developer. I assume there will be some workaround?

--
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com

2006-03-13 09:13:07

by Sam Ravnborg

[permalink] [raw]
Subject: Re: [PATCH] Do not rebuild full kernel tree again and again...

On Mon, Mar 13, 2006 at 05:10:31PM +1100, Nick Piggin wrote:
> Sam Ravnborg wrote:
>
> >This issue has been discussed with Paul D. Smith which also provided
> >a patch similar to yours.
> >The patch is in the kbuild queue for 2.6.17.
> >But we also agreed to postpose this change in make until next stable
> >release. So if you update your make to latest CVS version you will no
> >longer see this misbehaviour.
> >And when 2.6.17 opens up kbuild will be 'fixed' in mainline kernel.
> >
>
> So what's going to be done about 2.6.16?
>
> I'm seeing this behaviour too in -rc6 and it is a bad regression
> for a developer. I assume there will be some workaround?
I assume debian soon will update make to current version from CVS that
has this behaviour removed.

Sam

2006-03-13 10:47:14

by Nick Piggin

[permalink] [raw]
Subject: Re: [PATCH] Do not rebuild full kernel tree again and again...

Sam Ravnborg wrote:
> On Mon, Mar 13, 2006 at 05:10:31PM +1100, Nick Piggin wrote:
>
>>Sam Ravnborg wrote:
>>
>>
>>>This issue has been discussed with Paul D. Smith which also provided
>>>a patch similar to yours.
>>>The patch is in the kbuild queue for 2.6.17.
>>>But we also agreed to postpose this change in make until next stable
>>>release. So if you update your make to latest CVS version you will no
>>>longer see this misbehaviour.
>>>And when 2.6.17 opens up kbuild will be 'fixed' in mainline kernel.
>>>
>>
>>So what's going to be done about 2.6.16?
>>
>>I'm seeing this behaviour too in -rc6 and it is a bad regression
>>for a developer. I assume there will be some workaround?
>
> I assume debian soon will update make to current version from CVS that
> has this behaviour removed.
>

So long as it just requires a tools update then that's fine for me.
Thanks.

--
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com

2006-03-13 16:30:58

by Sam Ravnborg

[permalink] [raw]
Subject: Re: [PATCH] Do not rebuild full kernel tree again and again...

On Mon, Mar 13, 2006 at 09:47:08PM +1100, Nick Piggin wrote:
>>I'm seeing this behaviour too in -rc6 and it is a bad regression
> >>for a developer. I assume there will be some workaround?
> >
> >I assume debian soon will update make to current version from CVS that
> >has this behaviour removed.
> >
>
> So long as it just requires a tools update then that's fine for me.
I should note here that it was agreed with Paul that upcoming make
relase will not have this change, but next release will have it.
So 2.6.17 kbuild will take care of being forward compatible in this
matter.

Sam

2006-03-16 18:31:34

by Tejun Heo

[permalink] [raw]
Subject: Re: [PATCH] Do not rebuild full kernel tree again and again...

Sam Ravnborg wrote:
> On Mon, Mar 13, 2006 at 09:47:08PM +1100, Nick Piggin wrote:
> >>I'm seeing this behaviour too in -rc6 and it is a bad regression
>
>>>>for a developer. I assume there will be some workaround?
>>>
>>>I assume debian soon will update make to current version from CVS that
>>>has this behaviour removed.
>>>
>>
>>So long as it just requires a tools update then that's fine for me.
>
> I should note here that it was agreed with Paul that upcoming make
> relase will not have this change, but next release will have it.
> So 2.6.17 kbuild will take care of being forward compatible in this
> matter.
>

Wouldn't it be better to have an option to tell make to assume the old
behavior? I only skimmed the original thread but it didn't seem terribly
complex thing to do. A LOT of people will be doing things on pre-2.6.17
kernel for quite some time and they will be cursing a lot if they have
to rebuild everything everytime.

Thanks.

--
tejun

2006-03-16 18:37:14

by Sam Ravnborg

[permalink] [raw]
Subject: Re: [PATCH] Do not rebuild full kernel tree again and again...

On Fri, Mar 17, 2006 at 03:31:06AM +0900, Tejun Heo wrote:
Wouldn't it be better to have an option to tell make to assume the old
> behavior? I only skimmed the original thread but it didn't seem terribly
> complex thing to do. A LOT of people will be doing things on pre-2.6.17
> kernel for quite some time and they will be cursing a lot if they have
> to rebuild everything everytime.

If Paul planned for a new make relase this year - then yes.
But I assume it will take another year (almost) before next make realse
after the current 3.81. And then it should not matter much.

Sam

2006-03-16 19:06:48

by Tejun Heo

[permalink] [raw]
Subject: Re: [PATCH] Do not rebuild full kernel tree again and again...

Sam Ravnborg wrote:
> On Fri, Mar 17, 2006 at 03:31:06AM +0900, Tejun Heo wrote:
> Wouldn't it be better to have an option to tell make to assume the old
>
>>behavior? I only skimmed the original thread but it didn't seem terribly
>>complex thing to do. A LOT of people will be doing things on pre-2.6.17
>>kernel for quite some time and they will be cursing a lot if they have
>>to rebuild everything everytime.
>
>
> If Paul planned for a new make relase this year - then yes.
> But I assume it will take another year (almost) before next make realse
> after the current 3.81. And then it should not matter much.
>

Fair enough for me. :-)

--
tejun