>From ee63a191e50a33f6c47a00dc8f69b009a5406890 Mon Sep 17 00:00:00 2001
From: John Kacur <[email protected]>
Date: Wed, 6 Jan 2010 16:18:02 +0100
Subject: [PATCH] Don't mkdir -p include/config in the sourcetree when using O=outputdir option
Problem: When using the option to put output files in a separate directory, make oldconfig (or related steps like silentoldconfig) create the include/config dierectory in the source tree. This causes the prepare3 check to fail, requiring the user to run make mrproper in the source tree to continue with the make.
Solution: Add the $(objtree) directory to the %config target.
The following is an example of how the Makefile currently can fail.
[jkacur@tycho rt.linux.git]$ make mrproper
[jkacur@tycho rt.linux.git]$ mkdir /bld/2.6.33-rc3
[jkacur@tycho rt.linux.git]$ cp /bld/config /bld/2.6.33-rc3/.config
[jkacur@tycho rt.linux.git]$ make O=/bld/2.6.33-rc3/ oldconfig
HOSTCC scripts/basic/fixdep
HOSTCC scripts/basic/docproc
HOSTCC scripts/basic/hash
GEN /bld/2.6.33-rc3/Makefile
HOSTCC scripts/kconfig/conf.o
HOSTCC scripts/kconfig/kxgettext.o
HOSTCC scripts/kconfig/zconf.tab.o
HOSTLD scripts/kconfig/conf
scripts/kconfig/conf -o arch/x86/Kconfig
[jkacur@tycho rt.linux.git]$ make O=/bld/2.6.33-rc3/
GEN /bld/2.6.33-rc3/Makefile
scripts/kconfig/conf -s arch/x86/Kconfig
Using /home/jkacur/rt.linux.git as source for kernel
/home/jkacur/rt.linux.git is not clean, please run 'make mrproper'
in the '/home/jkacur/rt.linux.git' directory.
make[1]: *** [prepare3] Error 1
make: *** [sub-make] Error 2
[jkacur@tycho rt.linux.git]$ ls -ld include/config/
drwxrwxr-x 2 jkacur jkacur 6 2010-01-06 15:58 include/config/
Signed-off-by: John Kacur <[email protected]>
---
Makefile | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/Makefile b/Makefile
index 1231094..7d8d722 100644
--- a/Makefile
+++ b/Makefile
@@ -458,7 +458,7 @@ config: scripts_basic outputmakefile FORCE
$(Q)$(MAKE) $(build)=scripts/kconfig $@
%config: scripts_basic outputmakefile FORCE
- $(Q)mkdir -p include/linux include/config
+ $(Q)mkdir -p $(objtree)/include/linux include/config
$(Q)$(MAKE) $(build)=scripts/kconfig $@
else
--
1.6.0.6
John Kacur napsal(a):
> The following is an example of how the Makefile currently can fail.
> [jkacur@tycho rt.linux.git]$ make mrproper
> [jkacur@tycho rt.linux.git]$ mkdir /bld/2.6.33-rc3
> [jkacur@tycho rt.linux.git]$ cp /bld/config /bld/2.6.33-rc3/.config
> [jkacur@tycho rt.linux.git]$ make O=/bld/2.6.33-rc3/ oldconfig
> HOSTCC scripts/basic/fixdep
> HOSTCC scripts/basic/docproc
> HOSTCC scripts/basic/hash
> GEN /bld/2.6.33-rc3/Makefile
> HOSTCC scripts/kconfig/conf.o
> HOSTCC scripts/kconfig/kxgettext.o
> HOSTCC scripts/kconfig/zconf.tab.o
> HOSTLD scripts/kconfig/conf
> scripts/kconfig/conf -o arch/x86/Kconfig
> [jkacur@tycho rt.linux.git]$ make O=/bld/2.6.33-rc3/
> GEN /bld/2.6.33-rc3/Makefile
> scripts/kconfig/conf -s arch/x86/Kconfig
> Using /home/jkacur/rt.linux.git as source for kernel
> /home/jkacur/rt.linux.git is not clean, please run 'make mrproper'
> in the '/home/jkacur/rt.linux.git' directory.
> make[1]: *** [prepare3] Error 1
> make: *** [sub-make] Error 2
> [jkacur@tycho rt.linux.git]$ ls -ld include/config/
> drwxrwxr-x 2 jkacur jkacur 6 2010-01-06 15:58 include/config/
>
> Signed-off-by: John Kacur <[email protected]>
> ---
> Makefile | 2 +-
> 1 files changed, 1 insertions(+), 1 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index 1231094..7d8d722 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -458,7 +458,7 @@ config: scripts_basic outputmakefile FORCE
> $(Q)$(MAKE) $(build)=scripts/kconfig $@
>
> %config: scripts_basic outputmakefile FORCE
> - $(Q)mkdir -p include/linux include/config
> + $(Q)mkdir -p $(objtree)/include/linux include/config
> $(Q)$(MAKE) $(build)=scripts/kconfig $@
Can you double-check that this patch really fixes your issue and that
without this patch it fails again? Because 1) I can't reproduce this
behaviour and I also don't see how it could f, 2) your problem is a
include/config file somehow created in the source tree, but you patch
the creation of include/linux. And if there really needs to be a fix,
the same should be applied to the identical block above.
thanks,
Michal
On Wed, Jan 6, 2010 at 8:34 PM, Michal Marek <[email protected]> wrote:
> John Kacur napsal(a):
>> The following is an example of how the Makefile currently can fail.
>> [jkacur@tycho rt.linux.git]$ make mrproper
>> [jkacur@tycho rt.linux.git]$ mkdir /bld/2.6.33-rc3
>> [jkacur@tycho rt.linux.git]$ cp /bld/config /bld/2.6.33-rc3/.config
>> [jkacur@tycho rt.linux.git]$ make O=/bld/2.6.33-rc3/ oldconfig
>> ? HOSTCC ?scripts/basic/fixdep
>> ? HOSTCC ?scripts/basic/docproc
>> ? HOSTCC ?scripts/basic/hash
>> ? GEN ? ? /bld/2.6.33-rc3/Makefile
>> ? HOSTCC ?scripts/kconfig/conf.o
>> ? HOSTCC ?scripts/kconfig/kxgettext.o
>> ? HOSTCC ?scripts/kconfig/zconf.tab.o
>> ? HOSTLD ?scripts/kconfig/conf
>> scripts/kconfig/conf -o arch/x86/Kconfig
>> [jkacur@tycho rt.linux.git]$ make O=/bld/2.6.33-rc3/
>> ? GEN ? ? /bld/2.6.33-rc3/Makefile
>> scripts/kconfig/conf -s arch/x86/Kconfig
>> ? Using /home/jkacur/rt.linux.git as source for kernel
>> ? /home/jkacur/rt.linux.git is not clean, please run 'make mrproper'
>> ? in the '/home/jkacur/rt.linux.git' directory.
>> make[1]: *** [prepare3] Error 1
>> make: *** [sub-make] Error 2
>> [jkacur@tycho rt.linux.git]$ ls -ld include/config/
>> drwxrwxr-x 2 jkacur jkacur 6 2010-01-06 15:58 include/config/
>>
>> Signed-off-by: John Kacur <[email protected]>
>> ---
>> ?Makefile | ? ?2 +-
>> ?1 files changed, 1 insertions(+), 1 deletions(-)
>>
>> diff --git a/Makefile b/Makefile
>> index 1231094..7d8d722 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -458,7 +458,7 @@ config: scripts_basic outputmakefile FORCE
>> ? ? ? $(Q)$(MAKE) $(build)=scripts/kconfig $@
>>
>> ?%config: scripts_basic outputmakefile FORCE
>> - ? ? $(Q)mkdir -p include/linux include/config
>> + ? ? $(Q)mkdir -p $(objtree)/include/linux include/config
>> ? ? ? $(Q)$(MAKE) $(build)=scripts/kconfig $@
>
> Can you double-check that this patch really fixes your issue and that
> without this patch it fails again? Because 1) I can't reproduce this
> behaviour and I also don't see how it could f, 2) your problem is a
> include/config file somehow created in the source tree, but you patch
> the creation of include/linux. And if there really needs to be a fix,
> the same should be applied to the identical block above.
>
> thanks,
Hi Michal
Yes, I doubled checked that the problem exists, and it is quite easy for me to
reproduce. However, my proposed solution is also broken, please don't
apply, I'm still investigating.
I am surprised that you are not able to reproduce the problem though.
Did you follow the steps I that I outlined?
Thanks, John
John Kacur napsal(a):
> Yes, I doubled checked that the problem exists, and it is quite easy for me to
> reproduce. However, my proposed solution is also broken, please don't
> apply, I'm still investigating.
>
> I am surprised that you are not able to reproduce the problem though.
> Did you follow the steps I that I outlined?
I did
$ git checkout v2.6.33-rc3
$ mkdir /bld/2.6.33-rc3
$ zcat /proc/config.gz >/bld/2.6.33-rc3/.config
$ make O=/bld/2.6.33-rc3/ oldconfig
...
hit Enter several times
...
$ ls -ld include/config /bld/2.6.33-rc3/include/config/
ls: cannot access include/config: No such file or directory
drwxr-xr-x 2 mmarek users 4096 2010-01-06 21:35
/bld/2.6.33-rc3/include/config//
$ make O=/bld/2.6.33-rc3/
GEN /bld/2.6.33-rc3/Makefile
scripts/kconfig/conf -s arch/x86/Kconfig
Using /home/mmarek/linux-2.6 as source for kernel
GEN /bld/2.6.33-rc3/Makefile
CHK include/linux/version.h
UPD include/linux/version.h
CHK include/generated/utsrelease.h
UPD include/generated/utsrelease.h
CC kernel/bounds.s
... continues building ...
The mkdir command has been in the Makefile since 2006 (c955cca), so I
think that your problem is caused by something else. Try a make V=1
build with a read-only source tree to see which mkdir command fails.
Michal
On Wed, Jan 6, 2010 at 9:45 PM, Michal Marek <[email protected]> wrote:
> John Kacur napsal(a):
>> Yes, I doubled checked that the problem exists, and it is quite easy for me to
>> reproduce. However, my proposed solution is also broken, please don't
>> apply, I'm still investigating.
>>
>> I am surprised that you are not able to reproduce the problem though.
>> Did you follow the steps I that I outlined?
>
> I did
>
> $ git checkout v2.6.33-rc3
> $ mkdir /bld/2.6.33-rc3
> $ zcat /proc/config.gz >/bld/2.6.33-rc3/.config
> $ make O=/bld/2.6.33-rc3/ oldconfig
> ...
> hit Enter several times
> ...
> $ ls -ld include/config /bld/2.6.33-rc3/include/config/
> ls: cannot access include/config: No such file or directory
> drwxr-xr-x 2 mmarek users 4096 2010-01-06 21:35
> /bld/2.6.33-rc3/include/config//
> $ make O=/bld/2.6.33-rc3/
> ?GEN ? ? /bld/2.6.33-rc3/Makefile
> scripts/kconfig/conf -s arch/x86/Kconfig
> ?Using /home/mmarek/linux-2.6 as source for kernel
> ?GEN ? ? /bld/2.6.33-rc3/Makefile
> ?CHK ? ? include/linux/version.h
> ?UPD ? ? include/linux/version.h
> ?CHK ? ? include/generated/utsrelease.h
> ?UPD ? ? include/generated/utsrelease.h
> ?CC ? ? ?kernel/bounds.s
> ... continues building ...
>
>
> The mkdir command has been in the Makefile since 2006 (c955cca), so I
> think that your problem is caused by something else. Try a make V=1
> build with a read-only source tree to see which mkdir command fails.
>
Michal: The length of time that a bug exists in a piece of code is no
argument for
it's correctness!
I tried to debug this using
make -d V=1 O=/bld/2.6.33-rc3
and discovered that the bug did not always occur, but did sometimes.
This leads me to think that it is timing sensitive, and indeed the
problem shows up
more easily on a machine with multiple processors.
Is it possible that the step with mkdir -p can sometimes occur in a
Makefile parallel process
before the cd KBUILD_OUTPUT occurs? That would explain why it doesn't
reliably reproduce.
In any case, the idea behind my original patch is good, just specify
the full path
using the objtree variable and the problem goes away. The first round of my
implementation was simply lacking.
Following up this note with a patch.
John
On Wed, 6 Jan 2010, John Kacur wrote:
> On Wed, Jan 6, 2010 at 9:45 PM, Michal Marek <[email protected]> wrote:
> > John Kacur napsal(a):
> >> Yes, I doubled checked that the problem exists, and it is quite easy for me to
> >> reproduce. However, my proposed solution is also broken, please don't
> >> apply, I'm still investigating.
> >>
> >> I am surprised that you are not able to reproduce the problem though.
> >> Did you follow the steps I that I outlined?
> >
> > I did
> >
> > $ git checkout v2.6.33-rc3
> > $ mkdir /bld/2.6.33-rc3
> > $ zcat /proc/config.gz >/bld/2.6.33-rc3/.config
> > $ make O=/bld/2.6.33-rc3/ oldconfig
> > ...
> > hit Enter several times
> > ...
> > $ ls -ld include/config /bld/2.6.33-rc3/include/config/
> > ls: cannot access include/config: No such file or directory
> > drwxr-xr-x 2 mmarek users 4096 2010-01-06 21:35
> > /bld/2.6.33-rc3/include/config//
> > $ make O=/bld/2.6.33-rc3/
> > ?GEN ? ? /bld/2.6.33-rc3/Makefile
> > scripts/kconfig/conf -s arch/x86/Kconfig
> > ?Using /home/mmarek/linux-2.6 as source for kernel
> > ?GEN ? ? /bld/2.6.33-rc3/Makefile
> > ?CHK ? ? include/linux/version.h
> > ?UPD ? ? include/linux/version.h
> > ?CHK ? ? include/generated/utsrelease.h
> > ?UPD ? ? include/generated/utsrelease.h
> > ?CC ? ? ?kernel/bounds.s
> > ... continues building ...
> >
> >
> > The mkdir command has been in the Makefile since 2006 (c955cca), so I
> > think that your problem is caused by something else. Try a make V=1
> > build with a read-only source tree to see which mkdir command fails.
> >
>
> Michal: The length of time that a bug exists in a piece of code is no
> argument for
> it's correctness!
>
> I tried to debug this using
> make -d V=1 O=/bld/2.6.33-rc3
> and discovered that the bug did not always occur, but did sometimes.
> This leads me to think that it is timing sensitive, and indeed the
> problem shows up
> more easily on a machine with multiple processors.
>
> Is it possible that the step with mkdir -p can sometimes occur in a
> Makefile parallel process
> before the cd KBUILD_OUTPUT occurs? That would explain why it doesn't
> reliably reproduce.
>
> In any case, the idea behind my original patch is good, just specify
> the full path
> using the objtree variable and the problem goes away. The first round of my
> implementation was simply lacking.
>
> Following up this note with a patch.
>
> John
>
>From b9db45e49b36e589de410f1c34fe77087fa36796 Mon Sep 17 00:00:00 2001
From: John Kacur <[email protected]>
Date: Wed, 6 Jan 2010 23:16:38 +0100
Subject: [PATCH] Kbuild: Use $(objtree) for mkdir -p include/linux include/config
This fixes a problem where make sometimes creates the directories
include/linux include/config
in the local source tree, even though O=/outputdir was specified.
The solution is to give an absolute path using $(objtree)
Signed-off-by: John Kacur <[email protected]>
---
Makefile | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/Makefile b/Makefile
index 1231094..2f3ba92 100644
--- a/Makefile
+++ b/Makefile
@@ -454,11 +454,11 @@ include $(srctree)/arch/$(SRCARCH)/Makefile
export KBUILD_DEFCONFIG KBUILD_KCONFIG
config: scripts_basic outputmakefile FORCE
- $(Q)mkdir -p include/linux include/config
+ $(Q)mkdir -p $(objtree)/include/linux $(objtree)/include/config
$(Q)$(MAKE) $(build)=scripts/kconfig $@
%config: scripts_basic outputmakefile FORCE
- $(Q)mkdir -p include/linux include/config
+ $(Q)mkdir -p $(objtree)/include/linux $(objtree)/include/config
$(Q)$(MAKE) $(build)=scripts/kconfig $@
else
--
1.6.0.6
On 6.1.2010 23:30, John Kacur wrote:
> On Wed, Jan 6, 2010 at 9:45 PM, Michal Marek <[email protected]> wrote:
> Michal: The length of time that a bug exists in a piece of code is no
> argument for
> it's correctness!
>From your original post I though it fails reliably for you. If it's a
race, then you're right, the time it was introduced is irrelevant :-).
> I tried to debug this using
> make -d V=1 O=/bld/2.6.33-rc3
> and discovered that the bug did not always occur, but did sometimes.
> This leads me to think that it is timing sensitive, and indeed the
> problem shows up
> more easily on a machine with multiple processors.
>
> Is it possible that the step with mkdir -p can sometimes occur in a
> Makefile parallel process
> before the cd KBUILD_OUTPUT occurs? That would explain why it doesn't
> reliably reproduce.
OK, I still don't see how it fails, but adding $(objtree) there is a
valid change and if it fixes your problem, let's apply it.
Michal