% uname -m
x86_64
% git log -1 | head
commit 100b33c8bd8a3235fd0b7948338d6cbb3db3c63d
% grep XZ .config
CONFIG_HAVE_KERNEL_XZ=y
CONFIG_KERNEL_XZ=y
CONFIG_RD_XZ=y
CONFIG_SQUASHFS_XZ=y
CONFIG_XZ_DEC=y
CONFIG_XZ_DEC_X86=y
CONFIG_XZ_DEC_POWERPC=y
CONFIG_XZ_DEC_IA64=y
CONFIG_XZ_DEC_ARM=y
CONFIG_XZ_DEC_ARMTHUMB=y
CONFIG_XZ_DEC_SPARC=y
CONFIG_XZ_DEC_BCJ=y
CONFIG_XZ_DEC_TEST=y
CONFIG_DECOMPRESS_XZ=y
% make -j4
...
CC arch/x86/boot/tty.o
CC arch/x86/oprofile/oprofile.mod.o
XZKERN arch/x86/boot/compressed/vmlinux.bin.xz
exec: 23: xz: not found
CC arch/x86/boot/video.o
CC crypto/crc32c.mod.o
CC crypto/pcrypt.mod.o
CC arch/x86/boot/video-mode.o
CC drivers/acpi/acpi_memhotplug.mod.o
MKPIGGY arch/x86/boot/compressed/piggy.S
arch/x86/boot/compressed/vmlinux.bin.xz: No such file or directory
CC arch/x86/boot/version.o
LD arch/x86/boot/compressed/vmlinux
... [hundreds of lines of output snipped]
LD [M] net/sched/act_csum.ko
IHEX firmware/bnx2/bnx2-mips-09-6.2.1.fw
IHEX firmware/bnx2/bnx2-mips-06-6.2.1.fw
% echo $?
2
% ls -l arch/x86/boot/bzImage
-rw-r--r-- 1 adi adi 1991504 Feb 1 18:41 arch/x86/boot/bzImage
The failure is obscured by trailing make output, and since a broken
bzImage is created, "make install" succeeds, and the first visible
failure is after reboot when I get the lovely
Decompressing Linux...
Input is not in the XZ format (wrong magic bytes)
-- System halted
Sometimes I run "make -j4 && make install" which would have caught this,
but the error should be more prominent and we shouldn't create bzImage
if the dependencies are not present.
-andy
09.02.2011 22:52, Andy Isaacson пишет:
> % uname -m
> x86_64
> % git log -1 | head
> commit 100b33c8bd8a3235fd0b7948338d6cbb3db3c63d
> % grep XZ .config
> CONFIG_HAVE_KERNEL_XZ=y
> CONFIG_KERNEL_XZ=y
> CONFIG_RD_XZ=y
> CONFIG_SQUASHFS_XZ=y
> CONFIG_XZ_DEC=y
> CONFIG_XZ_DEC_X86=y
> CONFIG_XZ_DEC_POWERPC=y
> CONFIG_XZ_DEC_IA64=y
> CONFIG_XZ_DEC_ARM=y
> CONFIG_XZ_DEC_ARMTHUMB=y
> CONFIG_XZ_DEC_SPARC=y
> CONFIG_XZ_DEC_BCJ=y
> CONFIG_XZ_DEC_TEST=y
> CONFIG_DECOMPRESS_XZ=y
> % make -j4
> ...
> CC arch/x86/boot/tty.o
> CC arch/x86/oprofile/oprofile.mod.o
> XZKERN arch/x86/boot/compressed/vmlinux.bin.xz
> exec: 23: xz: not found
> CC arch/x86/boot/video.o
> CC crypto/crc32c.mod.o
> CC crypto/pcrypt.mod.o
> CC arch/x86/boot/video-mode.o
> CC drivers/acpi/acpi_memhotplug.mod.o
> MKPIGGY arch/x86/boot/compressed/piggy.S
> arch/x86/boot/compressed/vmlinux.bin.xz: No such file or directory
> CC arch/x86/boot/version.o
> LD arch/x86/boot/compressed/vmlinux
> ... [hundreds of lines of output snipped]
> LD [M] net/sched/act_csum.ko
> IHEX firmware/bnx2/bnx2-mips-09-6.2.1.fw
> IHEX firmware/bnx2/bnx2-mips-06-6.2.1.fw
> % echo $?
> 2
> % ls -l arch/x86/boot/bzImage
> -rw-r--r-- 1 adi adi 1991504 Feb 1 18:41 arch/x86/boot/bzImage
>
> The failure is obscured by trailing make output, and since a broken
> bzImage is created, "make install" succeeds, and the first visible
> failure is after reboot when I get the lovely
>
> Decompressing Linux...
>
> Input is not in the XZ format (wrong magic bytes)
>
> -- System halted
>
> Sometimes I run "make -j4 && make install" which would have caught this,
> but the error should be more prominent and we shouldn't create bzImage
> if the dependencies are not present.
:)
Signed-off-by: Pavel Vasilyev <[email protected]>
---
diff --git a/scripts/xz_wrap.sh b/scripts/xz_wrap.sh
index 17a5798..5ec9d36 100644
--- a/scripts/xz_wrap.sh
+++ b/scripts/xz_wrap.sh
@@ -20,4 +20,4 @@ case $ARCH in
sparc) BCJ=--sparc ;;
esac
-exec xz --check=crc32 $BCJ --lzma2=$LZMA2OPTS,dict=32MiB
+exec xz --check=crc32 $BCJ --lzma2=$LZMA2OPTS,dict=32MiB || exit 1
--
Pavel.
On Wed, Feb 09, 2011 at 11:07:32PM +0300, Pavel Vasilyev wrote:
> > The failure is obscured by trailing make output, and since a broken
> > bzImage is created, "make install" succeeds, and the first visible
[snip]
> :)
>
> Signed-off-by: Pavel Vasilyev <[email protected]>
> ---
> diff --git a/scripts/xz_wrap.sh b/scripts/xz_wrap.sh
> index 17a5798..5ec9d36 100644
> --- a/scripts/xz_wrap.sh
> +++ b/scripts/xz_wrap.sh
> @@ -20,4 +20,4 @@ case $ARCH in
> sparc) BCJ=--sparc ;;
> esac
>
> -exec xz --check=crc32 $BCJ --lzma2=$LZMA2OPTS,dict=32MiB
> +exec xz --check=crc32 $BCJ --lzma2=$LZMA2OPTS,dict=32MiB || exit 1
>
I get the same behavior with and without your patch, tested with both
bash and dash. (FWIW, /bin/sh on my systems is a symlink to /bin/dash.)
% printf '#!/bin/bash\nexec /no/such/file\n' > foo.sh
% chmod +x foo.sh
% ./foo.sh; echo $?
./foo.sh: line 2: /no/such/file: No such file or directory
./foo.sh: line 2: exec: /no/such/file: cannot execute: No such file or directory
126
% printf '#!/bin/bash\nexec /no/such/file\n' > foo.sh
% ./foo.sh; echo $?
./foo.sh: line 2: /no/such/file: No such file or directory
./foo.sh: line 2: exec: /no/such/file: cannot execute: No such file or
directory
126
% printf '#!/bin/dash\nexec /no/such/file\n' > foo.sh
% ./foo.sh; echo $?
exec: 2: /no/such/file: not found
2
% printf '#!/bin/dash\nexec /no/such/file || exit 1\n' > foo.sh
% ./foo.sh; echo $?
exec: 2: /no/such/file: not found
2
-andy
09.02.2011 23:38, Andy Isaacson пишет:
> On Wed, Feb 09, 2011 at 11:07:32PM +0300, Pavel Vasilyev wrote:
>>> The failure is obscured by trailing make output, and since a broken
>>> bzImage is created, "make install" succeeds, and the first visible
> [snip]
>> :)
>>
>> Signed-off-by: Pavel Vasilyev <[email protected]>
>> ---
>> diff --git a/scripts/xz_wrap.sh b/scripts/xz_wrap.sh
>> index 17a5798..5ec9d36 100644
>> --- a/scripts/xz_wrap.sh
>> +++ b/scripts/xz_wrap.sh
>> @@ -20,4 +20,4 @@ case $ARCH in
>> sparc) BCJ=--sparc ;;
>> esac
>>
>> -exec xz --check=crc32 $BCJ --lzma2=$LZMA2OPTS,dict=32MiB
>> +exec xz --check=crc32 $BCJ --lzma2=$LZMA2OPTS,dict=32MiB || exit 1
>>
>
> I get the same behavior with and without your patch, tested with both
> bash and dash. (FWIW, /bin/sh on my systems is a symlink to /bin/dash.)
>
# mv /usr/bin/xz /
# rm arch/x86/boot/compressed/vmlinux.bin*
# rm arch/x86/boot/bzImage
# make
# make
CHK include/linux/version.h
CHK include/generated/utsrelease.h
CALL scripts/checksyscalls.sh
CHK include/generated/compile.h
CHK include/linux/version.h
make[2]: `scripts/unifdef' is up to date.
XZKERN arch/x86/boot/compressed/vmlinux.bin.xz
/media/kernel/linux-2.6/scripts/xz_wrap.sh: line 23: exec: xz: not found
make[2]: *** [arch/x86/boot/compressed/vmlinux.bin.xz] Error 1
make[1]: *** [arch/x86/boot/compressed/vmlinux] Error 2
make: *** [bzImage] Error 2
# make install
sh /media/kernel/linux-2.6/arch/x86/boot/install.sh 2.6.38-rc4+
arch/x86/boot/bzImage \
System.map "/boot"
*** Missing file: arch/x86/boot/bzImage
*** You need to run "make" before "make install".
make[1]: *** [install] Error 1
make: *** [install] Error 2
All work, without and with patch. :-/
# sh --version
GNU bash, version 4.1.9(1)-release (x86_64-suse-linux-gnu)
--
Pavel.
On 02/09/2011 11:52 AM, Andy Isaacson wrote:
>
> The failure is obscured by trailing make output, and since a broken
> bzImage is created, "make install" succeeds, and the first visible
> failure is after reboot when I get the lovely
>
> Decompressing Linux...
>
> Input is not in the XZ format (wrong magic bytes)
>
> -- System halted
>
> Sometimes I run "make -j4 && make install" which would have caught this,
> but the error should be more prominent and we shouldn't create bzImage
> if the dependencies are not present.
>
Agreed that the bzImage should not have been created. That should make
the error prominent enough (your complaint that it gets obscured with
make -j is "just how things work, sorry.")
-hpa
On 2011-02-09 Andy Isaacson wrote:
> % make -j4
> ...
> CC arch/x86/boot/tty.o
> CC arch/x86/oprofile/oprofile.mod.o
> XZKERN arch/x86/boot/compressed/vmlinux.bin.xz
> exec: 23: xz: not found
This may have something to do with the shell that "make" uses to run the
commands. The compression command isn't seen as failed with dash for
some reason.
It doesn't matter what shell is used to run xz_wrap.sh. The same problem
exists with other compressors if they are missing.
> commit 100b33c8bd8a3235fd0b7948338d6cbb3db3c63d
[...]
> % ls -l arch/x86/boot/bzImage
> -rw-r--r-- 1 adi adi 1991504 Feb 1 18:41 arch/x86/boot/bzImage
"Feb 1" was about a week ago. The file is a few days older than the
commit ID, assuming that your clock is set correctly.
--
Lasse Collin | IRC: Larhzu @ IRCnet & Freenode
On Thu, Feb 10, 2011 at 12:01:42AM +0200, Lasse Collin wrote:
> On 2011-02-09 Andy Isaacson wrote:
> > % make -j4
> > ...
> > CC arch/x86/boot/tty.o
> > CC arch/x86/oprofile/oprofile.mod.o
> > XZKERN arch/x86/boot/compressed/vmlinux.bin.xz
> > exec: 23: xz: not found
>
> This may have something to do with the shell that "make" uses to run the
> commands. The compression command isn't seen as failed with dash for
> some reason.
>
> It doesn't matter what shell is used to run xz_wrap.sh. The same problem
> exists with other compressors if they are missing.
>
> > commit 100b33c8bd8a3235fd0b7948338d6cbb3db3c63d
> [...]
> > % ls -l arch/x86/boot/bzImage
> > -rw-r--r-- 1 adi adi 1991504 Feb 1 18:41 arch/x86/boot/bzImage
>
> "Feb 1" was about a week ago. The file is a few days older than the
> commit ID, assuming that your clock is set correctly.
Ah, yes, this is my regular build tree; I routinely pull and build
there.
So the build failed, but the output file remains because it failed on an
intermediate step. (This testing was from me reproducing it on a
different system, with 100b33).
Rewinding to the original system which failed to boot...
How did "make install" create a broken
/boot/vmlinuz-2.6.38-rc3-0312-gcb5520f that caused the boot-time error
message?
If I re-run "make -j4" in the tree that caused the problem, make fails
with exit code 2.
The following seems to reproduce the silent error:
rm .config
make defconfig
make -j4
edit .config and set CONFIG_KERNEL_XZ=y, CONFIG_KERNEL_GZIP=n
% make -j4
...
Kernel: arch/x86/boot/bzImage is ready (#64)
% echo $?
0
The resulting bzImage fails to boot with the aforementioned "Input is
not in the XZ format" error.
FWIW, the machine that's showing the problem is an Ubuntu Karmic x86_64
desktop install with all updates through mid-January installed.
-andy
On 2011-02-10 Andy Isaacson wrote:
> On Thu, Feb 10, 2011 at 12:01:42AM +0200, Lasse Collin wrote:
> > On 2011-02-09 Andy Isaacson wrote:
> > > % ls -l arch/x86/boot/bzImage
> > > -rw-r--r-- 1 adi adi 1991504 Feb 1 18:41 arch/x86/boot/bzImage
> >
> > "Feb 1" was about a week ago. The file is a few days older than the
> > commit ID, assuming that your clock is set correctly.
>
> Ah, yes, this is my regular build tree; I routinely pull and build
> there.
>
> So the build failed, but the output file remains because it failed on
> an intermediate step. (This testing was from me reproducing it on a
> different system, with 100b33).
>
> Rewinding to the original system which failed to boot...
>
> How did "make install" create a broken
> /boot/vmlinuz-2.6.38-rc3-0312-gcb5520f that caused the boot-time
> error message?
>
> If I re-run "make -j4" in the tree that caused the problem, make
> fails with exit code 2.
>
> The following seems to reproduce the silent error:
>
> rm .config
> make defconfig
> make -j4
> edit .config and set CONFIG_KERNEL_XZ=y, CONFIG_KERNEL_GZIP=n
>
> % make -j4
> ...
> Kernel: arch/x86/boot/bzImage is ready (#64)
> % echo $?
> 0
It looks a like a bug in dash. The following with dash prints foo bar
and returns with exit status 0, but other shells print only foo and
return with non-zero exit status:
sh -c 'set -e ; echo foo ; ( false ) ; echo bar'
That is, "set -e" has no effect on a subshell returning a non-zero exit
status. Without a subshell it works correctly:
sh -c 'set -e ; echo foo ; { false ; } ; echo bar'
Kernel compression and mkpiggy use command lines that hit this bug. Here
is a quick hack to avoid the unneeded subshell for these command lines,
which also works around the dash bug:
sed -i 's|( *\( *rm -f $@ ; false\) *)|{ \1 ; }|' \
scripts/Makefile.lib arch/x86/boot/compressed/Makefile
There are similar "set -e" + subshell constructs in other makefiles and
scripts. I don't know if it makes sense to accommodate a buggy but
common /bin/sh by modifying makefiles and scripts to avoid the
problematic construct. Maybe it would be better to have a test to catch
the buggy shell and tell the user to use e.g. "make SHELL=/bin/bash".
--
Lasse Collin | IRC: Larhzu @ IRCnet & Freenode
On 2011-02-10 Lasse Collin wrote:
> It looks a like a bug in dash. The following with dash prints foo bar
> and returns with exit status 0, but other shells print only foo and
> return with non-zero exit status:
>
> sh -c 'set -e ; echo foo ; ( false ) ; echo bar'
This dash bug has been fixed on 2010-07-06.
http://git.kernel.org/?p=utils/dash/dash.git;a=commitdiff;h=a92255d6fa8b8efb0f9fb093b77ac601fe3e9ede
--
Lasse Collin | IRC: Larhzu @ IRCnet & Freenode