2018-06-23 13:59:13

by Masahiro Yamada

[permalink] [raw]
Subject: [PATCH v2 1/3] kbuild: use 'include' directive to load auto.conf from top Makefile

When you build targets that require the kernel configuration, dot-config
is set to 1, then the top-level Makefile includes auto.conf. However,
Make considers its inclusion is optional because the '-include' directive
is used.

If a necessary configuration file is missing for the external module
building, the following error message is displayed:

ERROR: Kernel configuration is invalid.
include/generated/autoconf.h or include/config/auto.conf are missing.
Run 'make oldconfig && make prepare' on kernel src to fix it.

However, Make still continues building; /bin/false let the creation of
'include/config/auto.config' fail, but Make can ignore the error since
it is included by the '-include' directive.

Use the 'include' directive instead. This allows the /bin/false
to correctly terminate the whole build process.

Signed-off-by: Masahiro Yamada <[email protected]>
---

Changes in v2:
- New patch

Makefile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index ca2af1a..413c2ed 100644
--- a/Makefile
+++ b/Makefile
@@ -589,7 +589,7 @@ virt-y := virt/
endif # KBUILD_EXTMOD

ifeq ($(dot-config),1)
--include include/config/auto.conf
+include include/config/auto.conf
endif

# The all: target is the default when no target is given on the
--
2.7.4



2018-06-23 13:59:30

by Masahiro Yamada

[permalink] [raw]
Subject: [PATCH v2 2/3] kbuild: do not update config when running install targets

"make syncconfig" is automatically invoked when any of the following
happens:

- .config is updated
- any of Kconfig files is updated
- any of environment variables referenced in Kconfig is changed

Then, it updates configuration files such as include/config/auto.conf
include/generated/autoconf.h, etc.

Even install targets (install, modules_install, etc.) are no exception.
However, they should never ever modify the source tree. Install
targets are often run with root privileges. Once those configuration
files are owned by root, "make mrproper" would end up with permission
error.

Install targets should just copy things blindly. They should not care
whether the configuration is up-to-date or not. This makes more sense
because we are interested in the configuration that was used in the
previous kernel building.

This issue has existed since before, but rarely happened. I expect
more chance where people are hit by this; with the new Kconfig syntax
extension, the .config now contains the compiler information. If you
cross-compile the kernel with CROSS_COMPILE, but forget to pass it
for "make install", you meet "any of environment variables referenced
in Kconfig is changed" because $(CC) is referenced in Kconfig.
Another scenario is the compiler upgrade before the installation.

Install targets need the configuration. "make modules_install" refer
to CONFIG_MODULES etc. "make dtbs_install" also needs CONFIG_ARCH_*
to decide which dtb files to install. However, the auto-update of
the configuration files should be avoided. We already do this for
external modules.

Now, Make targets are categorized into 3 groups:

[1] Do not need the kernel configuration at all

help, coccicheck, headers_install etc.

[2] Need the latest kernel configuration

If new config options are added, Kconfig will show prompt to
ask user's selection.

Build targets such as vmlinux, in-kernel modules are the cases.

[3] Need the kernel configuration, but do not want to update it

Install targets except headers_install, and external modules
are the cases.

Signed-off-by: Masahiro Yamada <[email protected]>
---

Changes in v2: None

Makefile | 22 +++++++++++++++++-----
1 file changed, 17 insertions(+), 5 deletions(-)

diff --git a/Makefile b/Makefile
index 413c2ed..f1e06fba 100644
--- a/Makefile
+++ b/Makefile
@@ -225,10 +225,12 @@ no-dot-config-targets := $(clean-targets) \
cscope gtags TAGS tags help% %docs check% coccicheck \
$(version_h) headers_% archheaders archscripts \
kernelversion %src-pkg
+no-sync-config-targets := $(no-dot-config-targets) install %install

-config-targets := 0
-mixed-targets := 0
-dot-config := 1
+config-targets := 0
+mixed-targets := 0
+dot-config := 1
+may-sync-config := 1

ifneq ($(filter $(no-dot-config-targets), $(MAKECMDGOALS)),)
ifeq ($(filter-out $(no-dot-config-targets), $(MAKECMDGOALS)),)
@@ -236,6 +238,16 @@ ifneq ($(filter $(no-dot-config-targets), $(MAKECMDGOALS)),)
endif
endif

+ifneq ($(filter $(no-sync-config-targets), $(MAKECMDGOALS)),)
+ ifeq ($(filter-out $(no-sync-config-targets), $(MAKECMDGOALS)),)
+ may-sync-config := 0
+ endif
+endif
+
+ifneq ($(KBUILD_EXTMOD),)
+ may-sync-config := 0
+endif
+
ifeq ($(KBUILD_EXTMOD),)
ifneq ($(filter config %config,$(MAKECMDGOALS)),)
config-targets := 1
@@ -611,7 +623,7 @@ ARCH_CFLAGS :=
include arch/$(SRCARCH)/Makefile

ifeq ($(dot-config),1)
-ifeq ($(KBUILD_EXTMOD),)
+ifeq ($(may-sync-config),1)
# Read in dependencies to all Kconfig* files, make sure to run syncconfig if
# changes are detected. This should be included after arch/$(SRCARCH)/Makefile
# because some architectures define CROSS_COMPILE there.
@@ -639,7 +651,7 @@ include/config/auto.conf:
echo >&2 ; \
/bin/false)

-endif # KBUILD_EXTMOD
+endif # may-sync-config

else
# Dummy target needed, because used as prerequisite
--
2.7.4


2018-06-23 14:00:23

by Masahiro Yamada

[permalink] [raw]
Subject: [PATCH v2 3/3] kbuild: do not update config for 'make kernelrelease'

'make kernelrelease' depends on CONFIG_LOCALVERSION(_AUTO), but
for the same reason as install targets, we do not want to update
the configuration just for printing the kernelrelease string.

This is likely to happen when you compiled the kernel with
CROSS_COMPILE, but forget to pass it to 'make kernelrelease'.

Signed-off-by: Masahiro Yamada <[email protected]>
---

Changes in v2: None

Makefile | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index f1e06fba..bb7394a 100644
--- a/Makefile
+++ b/Makefile
@@ -225,7 +225,8 @@ no-dot-config-targets := $(clean-targets) \
cscope gtags TAGS tags help% %docs check% coccicheck \
$(version_h) headers_% archheaders archscripts \
kernelversion %src-pkg
-no-sync-config-targets := $(no-dot-config-targets) install %install
+no-sync-config-targets := $(no-dot-config-targets) install %install \
+ kernelrelease

config-targets := 0
mixed-targets := 0
--
2.7.4


2018-06-23 19:00:47

by kernel test robot

[permalink] [raw]
Subject: Re: [PATCH v2 1/3] kbuild: use 'include' directive to load auto.conf from top Makefile

Hi Masahiro,

I love your patch! Yet something to improve:

[auto build test ERROR on kbuild/for-next]
[also build test ERROR on v4.18-rc1 next-20180622]
[cannot apply to mmarek/for-next mmarek/rc-fixes]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url: https://github.com/0day-ci/linux/commits/Masahiro-Yamada/kbuild-use-include-directive-to-load-auto-conf-from-top-Makefile/20180623-220114
base: https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git for-next
config: xtensa-nommu_kc705_defconfig (attached as .config)
compiler: xtensa-de212-elf-gcc (crosstool-NG crosstool-ng-1.23.0-307-g452ee331) 7.3.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
GCC_VERSION=7.3.0 make.cross ARCH=xtensa

All errors (new ones prefixed by >>):

>> Makefile:592: include/config/auto.conf: No such file or directory
drivers/staging/mt7621-dts/Kconfig:4:warning: 'BUILTIN_DTB' has wrong type. 'select' only accept arguments of bool and tristate type
<stdin>:1332:2: warning: #warning syscall io_pgetevents not implemented [-Wcpp]
<stdin>:1335:2: warning: #warning syscall rseq not implemented [-Wcpp]

vim +592 Makefile

590
591 ifeq ($(dot-config),1)
> 592 include include/config/auto.conf
593 endif
594

---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation


Attachments:
(No filename) (1.62 kB)
.config.gz (10.22 kB)
Download all attachments

2018-06-24 08:58:40

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH v2 1/3] kbuild: use 'include' directive to load auto.conf from top Makefile

Hi.

2018-06-24 3:59 GMT+09:00 kbuild test robot <[email protected]>:
> Hi Masahiro,
>
> I love your patch! Yet something to improve:
>
> [auto build test ERROR on kbuild/for-next]
> [also build test ERROR on v4.18-rc1 next-20180622]
> [cannot apply to mmarek/for-next mmarek/rc-fixes]
> [if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
>
> url: https://github.com/0day-ci/linux/commits/Masahiro-Yamada/kbuild-use-include-directive-to-load-auto-conf-from-top-Makefile/20180623-220114
> base: https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git for-next
> config: xtensa-nommu_kc705_defconfig (attached as .config)
> compiler: xtensa-de212-elf-gcc (crosstool-NG crosstool-ng-1.23.0-307-g452ee331) 7.3.0
> reproduce:
> wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
> chmod +x ~/bin/make.cross
> # save the attached .config to linux build tree
> GCC_VERSION=7.3.0 make.cross ARCH=xtensa

This did not work for me
because the make.cross script tries to download GCC 7.3.0 xtensa-linux
but it is missing in the web site.

$ GCC_VERSION=7.3.0 make.cross ARCH=xtensa
Cannot find xtensa-linux under
https://download.01.org/0day-ci/cross-package check
/tmp/crosstool-files


I installed xtensa-de212-elf-gcc by hand,
but I could not reproduce the error.


GCC_VERSION=7.2.0 make.cross ARCH=xtensa
worked, but I could not reproduce the issue either.
(the build terminated with a different error.)


Where can I see the full build log?


> All errors (new ones prefixed by >>):
>
>>> Makefile:592: include/config/auto.conf: No such file or directory
> drivers/staging/mt7621-dts/Kconfig:4:warning: 'BUILTIN_DTB' has wrong type. 'select' only accept arguments of bool and tristate type
> <stdin>:1332:2: warning: #warning syscall io_pgetevents not implemented [-Wcpp]
> <stdin>:1335:2: warning: #warning syscall rseq not implemented [-Wcpp]
>
> vim +592 Makefile
>
> 590
> 591 ifeq ($(dot-config),1)
> > 592 include include/config/auto.conf
> 593 endif
> 594
>
> ---
> 0-DAY kernel test infrastructure Open Source Technology Center
> https://lists.01.org/pipermail/kbuild-all Intel Corporation



--
Best Regards
Masahiro Yamada

2018-06-24 10:25:48

by Philip Li

[permalink] [raw]
Subject: Re: [kbuild-all] [PATCH v2 1/3] kbuild: use 'include' directive to load auto.conf from top Makefile

On Sun, Jun 24, 2018 at 05:56:24PM +0900, Masahiro Yamada wrote:
> Hi.
>
> 2018-06-24 3:59 GMT+09:00 kbuild test robot <[email protected]>:
> > Hi Masahiro,
> >
> > I love your patch! Yet something to improve:
> >
> > [auto build test ERROR on kbuild/for-next]
> > [also build test ERROR on v4.18-rc1 next-20180622]
> > [cannot apply to mmarek/for-next mmarek/rc-fixes]
> > [if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
> >
> > url: https://github.com/0day-ci/linux/commits/Masahiro-Yamada/kbuild-use-include-directive-to-load-auto-conf-from-top-Makefile/20180623-220114
> > base: https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git for-next
> > config: xtensa-nommu_kc705_defconfig (attached as .config)
> > compiler: xtensa-de212-elf-gcc (crosstool-NG crosstool-ng-1.23.0-307-g452ee331) 7.3.0
> > reproduce:
> > wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
> > chmod +x ~/bin/make.cross
> > # save the attached .config to linux build tree
> > GCC_VERSION=7.3.0 make.cross ARCH=xtensa
>
> This did not work for me
> because the make.cross script tries to download GCC 7.3.0 xtensa-linux
> but it is missing in the web site.
>
> $ GCC_VERSION=7.3.0 make.cross ARCH=xtensa
> Cannot find xtensa-linux under
> https://download.01.org/0day-ci/cross-package check
> /tmp/crosstool-files
thanks for info, we will fix it as early as possible, and
reproduce the issue to provide feedback later.

>
>
> I installed xtensa-de212-elf-gcc by hand,
> but I could not reproduce the error.
>
>
> GCC_VERSION=7.2.0 make.cross ARCH=xtensa
> worked, but I could not reproduce the issue either.
> (the build terminated with a different error.)
>
>
> Where can I see the full build log?
>
>
> > All errors (new ones prefixed by >>):
> >
> >>> Makefile:592: include/config/auto.conf: No such file or directory
> > drivers/staging/mt7621-dts/Kconfig:4:warning: 'BUILTIN_DTB' has wrong type. 'select' only accept arguments of bool and tristate type
> > <stdin>:1332:2: warning: #warning syscall io_pgetevents not implemented [-Wcpp]
> > <stdin>:1335:2: warning: #warning syscall rseq not implemented [-Wcpp]
> >
> > vim +592 Makefile
> >
> > 590
> > 591 ifeq ($(dot-config),1)
> > > 592 include include/config/auto.conf
> > 593 endif
> > 594
> >
> > ---
> > 0-DAY kernel test infrastructure Open Source Technology Center
> > https://lists.01.org/pipermail/kbuild-all Intel Corporation
>
>
>
> --
> Best Regards
> Masahiro Yamada
> _______________________________________________
> kbuild-all mailing list
> [email protected]
> https://lists.01.org/mailman/listinfo/kbuild-all

2018-06-28 13:42:14

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH v2 1/3] kbuild: use 'include' directive to load auto.conf from top Makefile

2018-06-24 3:59 GMT+09:00 kbuild test robot <[email protected]>:
> Hi Masahiro,
>
> I love your patch! Yet something to improve:
>
> [auto build test ERROR on kbuild/for-next]
> [also build test ERROR on v4.18-rc1 next-20180622]
> [cannot apply to mmarek/for-next mmarek/rc-fixes]
> [if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
>
> url: https://github.com/0day-ci/linux/commits/Masahiro-Yamada/kbuild-use-include-directive-to-load-auto-conf-from-top-Makefile/20180623-220114
> base: https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git for-next
> config: xtensa-nommu_kc705_defconfig (attached as .config)
> compiler: xtensa-de212-elf-gcc (crosstool-NG crosstool-ng-1.23.0-307-g452ee331) 7.3.0
> reproduce:
> wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
> chmod +x ~/bin/make.cross
> # save the attached .config to linux build tree
> GCC_VERSION=7.3.0 make.cross ARCH=xtensa
>
> All errors (new ones prefixed by >>):
>
>>> Makefile:592: include/config/auto.conf: No such file or directory


I was able to reproduce this.


This depends on the version of GNU Make.

The warning is displayed if you use GNU Make 4.1 or older.
No more annoying warning is displayed for GNU Make 4.2 or later.

Probably, this commit in GNU Make:

commit 87a5f98d248fe63fe6e3e53ee3e1b1b1fa5e49dc
Author: Paul Smith <[email protected]>
Date: Sat Apr 9 19:49:27 2016 -0400

[SV 102] Don't show unnecessary include file errors.


However, 4.2 is quite new. I will do something with it.


Thanks.






> drivers/staging/mt7621-dts/Kconfig:4:warning: 'BUILTIN_DTB' has wrong type. 'select' only accept arguments of bool and tristate type
> <stdin>:1332:2: warning: #warning syscall io_pgetevents not implemented [-Wcpp]
> <stdin>:1335:2: warning: #warning syscall rseq not implemented [-Wcpp]
>
> vim +592 Makefile
>
> 590
> 591 ifeq ($(dot-config),1)
> > 592 include include/config/auto.conf
> 593 endif
> 594
>
> ---
> 0-DAY kernel test infrastructure Open Source Technology Center
> https://lists.01.org/pipermail/kbuild-all Intel Corporation



--
Best Regards
Masahiro Yamada