2019-10-04 08:33:08

by Harald Arnesen

[permalink] [raw]
Subject: BISECTED: Compile error on 5.4-rc1

I just tried to compile kernel 5.4-rc1 on my ThinkPad, which runs Devuan
Beowulf. Got the following:

$ make bindeb-pkg
UPD include/config/kernel.release
sh ./scripts/package/mkdebian
dpkg-buildpackage -r"fakeroot -u" -a$(cat debian/arch) -b -nc -uc
dpkg-buildpackage: info: source package linux-5.4.0-rc1-00014-gcc3a7bfe62b9
dpkg-buildpackage: info: source version 5.4.0-rc1-00014-gcc3a7bfe62b9-1
dpkg-buildpackage: info: source distribution beowulf
dpkg-buildpackage: info: source changed by Harald Arnesen
<[email protected]>
dpkg-architecture: warning: specified GNU system type x86_64-linux-gnu
does not match CC system type x86_64-pc-linux-gnu, try setting a correct
CC environment variable
dpkg-buildpackage: info: host architecture amd64
dpkg-source --before-build .
dpkg-source: warning: can't parse dependency -n libelf-dev
dpkg-source: error: error occurred while parsing Build-Depends
dpkg-buildpackage: error: dpkg-source --before-build . subprocess
returned exit status 255
make[1]: *** [scripts/Makefile.package:83: bindeb-pkg] Error 255
make: *** [Makefile:1448: bindeb-pkg] Error 2


Bisecting gives me

858805b336be1cabb3d9033adaa3676574d12e37 is the first bad commit
commit 858805b336be1cabb3d9033adaa3676574d12e37
Author: Masahiro Yamada <[email protected]>
Date: Sun Aug 25 22:28:37 2019 +0900
...

By reverting commit 858805b336be1cabb3d9033adaa3676574d12e37 I could
compile the kernel.
--
Hilsen Harald


2019-10-05 03:33:13

by Masahiro Yamada

[permalink] [raw]
Subject: Re: BISECTED: Compile error on 5.4-rc1

On Fri, Oct 4, 2019 at 5:26 PM Harald Arnesen <[email protected]> wrote:
>
> I just tried to compile kernel 5.4-rc1 on my ThinkPad, which runs Devuan
> Beowulf. Got the following:
>
> $ make bindeb-pkg
> UPD include/config/kernel.release
> sh ./scripts/package/mkdebian
> dpkg-buildpackage -r"fakeroot -u" -a$(cat debian/arch) -b -nc -uc
> dpkg-buildpackage: info: source package linux-5.4.0-rc1-00014-gcc3a7bfe62b9
> dpkg-buildpackage: info: source version 5.4.0-rc1-00014-gcc3a7bfe62b9-1
> dpkg-buildpackage: info: source distribution beowulf
> dpkg-buildpackage: info: source changed by Harald Arnesen
> <[email protected]>
> dpkg-architecture: warning: specified GNU system type x86_64-linux-gnu
> does not match CC system type x86_64-pc-linux-gnu, try setting a correct
> CC environment variable
> dpkg-buildpackage: info: host architecture amd64
> dpkg-source --before-build .
> dpkg-source: warning: can't parse dependency -n libelf-dev
> dpkg-source: error: error occurred while parsing Build-Depends
> dpkg-buildpackage: error: dpkg-source --before-build . subprocess
> returned exit status 255
> make[1]: *** [scripts/Makefile.package:83: bindeb-pkg] Error 255
> make: *** [Makefile:1448: bindeb-pkg] Error 2
>
>
> Bisecting gives me
>
> 858805b336be1cabb3d9033adaa3676574d12e37 is the first bad commit
> commit 858805b336be1cabb3d9033adaa3676574d12e37
> Author: Masahiro Yamada <[email protected]>
> Date: Sun Aug 25 22:28:37 2019 +0900
> ...
>
> By reverting commit 858805b336be1cabb3d9033adaa3676574d12e37 I could
> compile the kernel.
> --
> Hilsen Harald
>


I cannot reproduce it.

I tested bindeb-pkg for the latest Linus tree successfully.



masahiro@grover:~/ref/linux$ git log --oneline -1
4ea655343ce4 (HEAD -> master, origin/master, origin/HEAD) Merge tag
'mips_fixes_5.4_1' of
git://git.kernel.org/pub/scm/linux/kernel/git/mips/linux
masahiro@grover:~/ref/linux$ make bindeb-pkg
sh ./scripts/package/mkdebian
dpkg-buildpackage -r"fakeroot -u" -a$(cat debian/arch) -b -nc -uc
dpkg-buildpackage: info: source package linux-5.4.0-rc1+
dpkg-buildpackage: info: source version 5.4.0-rc1+-4
dpkg-buildpackage: info: source distribution bionic
dpkg-buildpackage: info: source changed by Masahiro Yamada
<[email protected]>
dpkg-buildpackage: info: host architecture amd64
dpkg-source --before-build linux
debian/rules build
make KERNELRELEASE=5.4.0-rc1+ ARCH=x86 KBUILD_BUILD_VERSION=4 -f ./Makefile
CALL scripts/checksyscalls.sh
CALL scripts/atomic/check-atomics.sh
DESCEND objtool
CHK include/generated/compile.h
UPD include/generated/compile.h
CC init/version.o
AR init/built-in.a
GEN .version
CHK include/generated/compile.h
LD vmlinux.o
MODPOST vmlinux.o
MODINFO modules.builtin.modinfo
LD .tmp_vmlinux1
KSYM .tmp_kallsyms1.o
LD .tmp_vmlinux2
KSYM .tmp_kallsyms2.o
LD vmlinux
SORTEX vmlinux
SYSMAP System.map
VOFFSET arch/x86/boot/compressed/../voffset.h
CC arch/x86/boot/compressed/misc.o
OBJCOPY arch/x86/boot/compressed/vmlinux.bin
RELOCS arch/x86/boot/compressed/vmlinux.relocs
GZIP arch/x86/boot/compressed/vmlinux.bin.gz
MKPIGGY arch/x86/boot/compressed/piggy.S
AS arch/x86/boot/compressed/piggy.o
CC arch/x86/boot/compressed/kaslr.o
LD arch/x86/boot/compressed/vmlinux
ZOFFSET arch/x86/boot/zoffset.h
AS arch/x86/boot/header.o
CC arch/x86/boot/version.o
LD arch/x86/boot/setup.elf
OBJCOPY arch/x86/boot/setup.bin
OBJCOPY arch/x86/boot/vmlinux.bin
BUILD arch/x86/boot/bzImage
Setup is 16124 bytes (padded to 16384 bytes).
System is 8661 kB
CRC fab8ebd9
Kernel: arch/x86/boot/bzImage is ready (#5)
Building modules, stage 2.
MODPOST 12 modules
fakeroot -u debian/rules binary
make KERNELRELEASE=5.4.0-rc1+ ARCH=x86 KBUILD_BUILD_VERSION=4 -f
./Makefile intdeb-pkg
sh ./scripts/package/builddeb
INSTALL drivers/thermal/intel/x86_pkg_temp_thermal.ko
INSTALL fs/efivarfs/efivarfs.ko
INSTALL net/ipv4/netfilter/iptable_nat.ko
INSTALL net/ipv4/netfilter/nf_log_arp.ko
INSTALL net/ipv4/netfilter/nf_log_ipv4.ko
INSTALL net/ipv6/netfilter/nf_log_ipv6.ko
INSTALL net/netfilter/nf_log_common.ko
INSTALL net/netfilter/xt_LOG.ko
INSTALL net/netfilter/xt_MASQUERADE.ko
INSTALL net/netfilter/xt_addrtype.ko
INSTALL net/netfilter/xt_mark.ko
INSTALL net/netfilter/xt_nat.ko
DEPMOD 5.4.0-rc1+
INSTALL ./debian/headertmp/usr/include
dpkg-deb: building package 'linux-headers-5.4.0-rc1+' in
'../linux-headers-5.4.0-rc1+_5.4.0-rc1+-4_amd64.deb'.
dpkg-deb: building package 'linux-libc-dev' in
'../linux-libc-dev_5.4.0-rc1+-4_amd64.deb'.
dpkg-deb: building package 'linux-image-5.4.0-rc1+' in
'../linux-image-5.4.0-rc1+_5.4.0-rc1+-4_amd64.deb'.
dpkg-genbuildinfo --build=binary
dpkg-genchanges --build=binary >../linux-5.4.0-rc1+_5.4.0-rc1+-4_amd64.changes
dpkg-genchanges: warning: package linux-image-5.4.0-rc1+-dbg in
control file but not in files list
dpkg-genchanges: info: binary-only upload (no source code included)
dpkg-source --after-build linux
dpkg-buildpackage: info: binary-only upload (no source included)



--
Best Regards
Masahiro Yamada

2019-10-05 09:09:28

by Harald Arnesen

[permalink] [raw]
Subject: Re: BISECTED: Compile error on 5.4-rc1

Masahiro Yamada [05.10.2019 05:24]:

> I cannot reproduce it.
>
> I tested bindeb-pkg for the latest Linus tree successfully.

Strange, I have now tried another machine running the same distro
(Devuan Beowulf), and I get the same result there. Will check further.
--
Hilsen Harald

2019-10-05 09:44:11

by Harald Arnesen

[permalink] [raw]
Subject: Re: BISECTED: Compile error on 5.4-rc1

Harald Arnesen [05.10.2019 11:03]:

> Masahiro Yamada [05.10.2019 05:24]:
>
>> I cannot reproduce it.
>>
>> I tested bindeb-pkg for the latest Linus tree successfully.
>
> Strange, I have now tried another machine running the same distro
> (Devuan Beowulf), and I get the same result there. Will check further.

I found out what was wrong.

Both machines have been used for dvd burning, and I have used Jörg
Schilling's cdrecord - and I had installed all of the "Schily Tools",
which unfortunately includes a shell command. Now, I had (by mistake)
/opt/schily/bin early in my path, and his OpenSolaris-derived shell
didn't work as expected.

Moving /opt/schily/bin to the end of the path fixes the problem.

But shouldn't it work even if "sh" is not equal to "/bin/sh"?
--
Hilsen Harald

2019-10-05 10:23:28

by Masahiro Yamada

[permalink] [raw]
Subject: Re: BISECTED: Compile error on 5.4-rc1

On Sat, Oct 5, 2019 at 6:36 PM Harald Arnesen <[email protected]> wrote:
>
> Harald Arnesen [05.10.2019 11:03]:
>
> > Masahiro Yamada [05.10.2019 05:24]:
> >
> >> I cannot reproduce it.
> >>
> >> I tested bindeb-pkg for the latest Linus tree successfully.
> >
> > Strange, I have now tried another machine running the same distro
> > (Devuan Beowulf), and I get the same result there. Will check further.
>
> I found out what was wrong.
>
> Both machines have been used for dvd burning, and I have used Jörg
> Schilling's cdrecord - and I had installed all of the "Schily Tools",
> which unfortunately includes a shell command. Now, I had (by mistake)
> /opt/schily/bin early in my path, and his OpenSolaris-derived shell
> didn't work as expected.
>
> Moving /opt/schily/bin to the end of the path fixes the problem.
>
> But shouldn't it work even if "sh" is not equal to "/bin/sh"?


CONFIG_SHELL previously fell back to 'sh' when bash is not installed,
so I just kept it as it was.

If we had used the exact absolute path /bin/sh,
it would have worked irrespective of the PATH environment.

But, there is a counter option like this:


commit 16f8259ca77d04f95e5ca90be1b1894ed45816c0
Author: Bjørn Forsman <[email protected]>
Date: Sun Nov 5 10:44:16 2017 +0100

kbuild: /bin/pwd -> pwd

Most places use pwd and rely on $PATH lookup. Moving the remaining
absolute path /bin/pwd users over for consistency.

Also, a reason for doing /bin/pwd -> pwd instead of the other way around
is because I believe build systems should make little assumptions on
host filesystem layout. Case in point, we do this kind of patching
already in NixOS.

Ref. commit 028568d84da3cfca49f5f846eeeef01441d70451
("kbuild: revert $(realpath ...) to $(shell cd ... && /bin/pwd)").

Signed-off-by: Bjørn Forsman <[email protected]>
Signed-off-by: Masahiro Yamada <[email protected]>



I cannot find a way to satisfy everybody.



--
Best Regards
Masahiro Yamada

2019-10-05 10:55:42

by Harald Arnesen

[permalink] [raw]
Subject: Re: BISECTED: Compile error on 5.4-rc1

Masahiro Yamada [05.10.2019 12:19]:

> CONFIG_SHELL previously fell back to 'sh' when bash is not installed,
> so I just kept it as it was.
>
> If we had used the exact absolute path /bin/sh,
> it would have worked irrespective of the PATH environment.
>
> But, there is a counter option like this:
>
>
> commit 16f8259ca77d04f95e5ca90be1b1894ed45816c0
> Author: Bjørn Forsman <[email protected]>
> Date: Sun Nov 5 10:44:16 2017 +0100
>
> kbuild: /bin/pwd -> pwd
>
> Most places use pwd and rely on $PATH lookup. Moving the remaining
> absolute path /bin/pwd users over for consistency.
>
> Also, a reason for doing /bin/pwd -> pwd instead of the other way around
> is because I believe build systems should make little assumptions on
> host filesystem layout. Case in point, we do this kind of patching
> already in NixOS.
>
> Ref. commit 028568d84da3cfca49f5f846eeeef01441d70451
> ("kbuild: revert $(realpath ...) to $(shell cd ... && /bin/pwd)").
>
> Signed-off-by: Bjørn Forsman <[email protected]>
> Signed-off-by: Masahiro Yamada <[email protected]>
>
>
>
> I cannot find a way to satisfy everybody.
>

I'm totally fine with the way it is now, now that I know how it works.
However, doesn't Posix dictate that there is a /bin/sh?
--
Hilsen Harald

2019-10-05 13:19:53

by Masahiro Yamada

[permalink] [raw]
Subject: Re: BISECTED: Compile error on 5.4-rc1

On Sat, Oct 5, 2019 at 7:50 PM Harald Arnesen <[email protected]> wrote:
>
> Masahiro Yamada [05.10.2019 12:19]:
>
> > CONFIG_SHELL previously fell back to 'sh' when bash is not installed,
> > so I just kept it as it was.
> >
> > If we had used the exact absolute path /bin/sh,
> > it would have worked irrespective of the PATH environment.
> >
> > But, there is a counter option like this:
> >
> >
> > commit 16f8259ca77d04f95e5ca90be1b1894ed45816c0
> > Author: Bjørn Forsman <[email protected]>
> > Date: Sun Nov 5 10:44:16 2017 +0100
> >
> > kbuild: /bin/pwd -> pwd
> >
> > Most places use pwd and rely on $PATH lookup. Moving the remaining
> > absolute path /bin/pwd users over for consistency.
> >
> > Also, a reason for doing /bin/pwd -> pwd instead of the other way around
> > is because I believe build systems should make little assumptions on
> > host filesystem layout. Case in point, we do this kind of patching
> > already in NixOS.
> >
> > Ref. commit 028568d84da3cfca49f5f846eeeef01441d70451
> > ("kbuild: revert $(realpath ...) to $(shell cd ... && /bin/pwd)").
> >
> > Signed-off-by: Bjørn Forsman <[email protected]>
> > Signed-off-by: Masahiro Yamada <[email protected]>
> >
> >
> >
> > I cannot find a way to satisfy everybody.
> >
>
> I'm totally fine with the way it is now, now that I know how it works.
> However, doesn't Posix dictate that there is a /bin/sh?


As for POSIX, I found this:

---------------------->8----------------------------
Applications should note that the standard PATH to the shell cannot
be assumed to be either /bin/sh or /usr/bin/sh, and should be determined
by interrogation of the PATH returned by getconf PATH , ensuring that
the returned pathname is an absolute pathname and not a shell built-in.
---------------------->8----------------------------

https://pubs.opengroup.org/onlinepubs/009695399/utilities/sh.html


--
Best Regards
Masahiro Yamada

2019-10-05 14:11:46

by Harald Arnesen

[permalink] [raw]
Subject: Re: BISECTED: Compile error on 5.4-rc1

Masahiro Yamada [05.10.2019 15:17]:

> As for POSIX, I found this:
>
> ---------------------->8----------------------------
> Applications should note that the standard PATH to the shell cannot
> be assumed to be either /bin/sh or /usr/bin/sh, and should be determined
> by interrogation of the PATH returned by getconf PATH , ensuring that
> the returned pathname is an absolute pathname and not a shell built-in.
> ---------------------->8----------------------------
>
> https://pubs.opengroup.org/onlinepubs/009695399/utilities/sh.html

Okay. Then it's the Schilling/OpenSolaris shell that is the problem, not
that I use it anyway.
--
Hilsen Harald