2014-02-15 23:15:12

by Daniel M. Weeks

[permalink] [raw]
Subject: [PATCH] Fix flags for initramfs LZ4 compression

LZ4 as implemented in the kernel differs from the default method now
used by the reference implementation of LZ4. Until the in-kernel method
is updated to support the new default, passing the legacy flag (-l) to
the compressor is necessary. Without this flag the kernel-generated,
LZ4-compressed initramfs is junk.

Signed-off-by: Daniel M. Weeks <[email protected]>
---
scripts/gen_initramfs_list.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/gen_initramfs_list.sh b/scripts/gen_initramfs_list.sh
index ef47409..17fa901 100644
--- a/scripts/gen_initramfs_list.sh
+++ b/scripts/gen_initramfs_list.sh
@@ -257,7 +257,7 @@ case "$arg" in
&& compr="lzop -9 -f"
echo "$output_file" | grep -q "\.lz4$" \
&& [ -x "`which lz4 2> /dev/null`" ] \
- && compr="lz4 -9 -f"
+ && compr="lz4 -l -9 -f"
echo "$output_file" | grep -q "\.cpio$" && compr="cat"
shift
;;
--
Daniel M. Weeks


2014-02-19 00:07:00

by Andrew Morton

[permalink] [raw]
Subject: Re: [PATCH] Fix flags for initramfs LZ4 compression

On Sat, 15 Feb 2014 18:14:57 -0500 "Daniel M. Weeks" <[email protected]> wrote:

> LZ4 as implemented in the kernel differs from the default method now
> used by the reference implementation of LZ4. Until the in-kernel method
> is updated to support the new default, passing the legacy flag (-l) to
> the compressor is necessary. Without this flag the kernel-generated,
> LZ4-compressed initramfs is junk.

Well that's bad. Kyungsik, cuold you please comment?

> --- a/scripts/gen_initramfs_list.sh
> +++ b/scripts/gen_initramfs_list.sh
> @@ -257,7 +257,7 @@ case "$arg" in
> && compr="lzop -9 -f"
> echo "$output_file" | grep -q "\.lz4$" \
> && [ -x "`which lz4 2> /dev/null`" ] \
> - && compr="lz4 -9 -f"
> + && compr="lz4 -l -9 -f"
> echo "$output_file" | grep -q "\.cpio$" && compr="cat"
> shift
> ;;

2014-02-19 00:08:58

by Andrew Morton

[permalink] [raw]
Subject: Re: [PATCH] Fix flags for initramfs LZ4 compression

On Sat, 15 Feb 2014 18:14:57 -0500 "Daniel M. Weeks" <[email protected]> wrote:

> LZ4 as implemented in the kernel differs from the default method now
> used by the reference implementation of LZ4. Until the in-kernel method
> is updated to support the new default, passing the legacy flag (-l) to
> the compressor is necessary. Without this flag the kernel-generated,
> LZ4-compressed initramfs is junk.
>
> ...
>
> --- a/scripts/gen_initramfs_list.sh
> +++ b/scripts/gen_initramfs_list.sh
> @@ -257,7 +257,7 @@ case "$arg" in
> && compr="lzop -9 -f"
> echo "$output_file" | grep -q "\.lz4$" \
> && [ -x "`which lz4 2> /dev/null`" ] \
> - && compr="lz4 -9 -f"
> + && compr="lz4 -l -9 -f"
> echo "$output_file" | grep -q "\.cpio$" && compr="cat"
> shift
> ;;

What happens is the user is running an old version of /bin/lz4? A
version which predates this switch to a new format? Do those earlier
versions accept -l, even though they don't need it? Or will the kernel
build fail?

2014-02-19 07:51:41

by Kyungsik Lee

[permalink] [raw]
Subject: Re: [PATCH] Fix flags for initramfs LZ4 compression

Hello,

On Tue, Feb 18, 2014 at 04:08:56PM -0800, Andrew Morton wrote:
> On Sat, 15 Feb 2014 18:14:57 -0500 "Daniel M. Weeks" <[email protected]> wrote:
>
> > LZ4 as implemented in the kernel differs from the default method now
> > used by the reference implementation of LZ4. Until the in-kernel method
> > is updated to support the new default, passing the legacy flag (-l) to
> > the compressor is necessary. Without this flag the kernel-generated,
> > LZ4-compressed initramfs is junk.
> >
> > ...
> >
> > --- a/scripts/gen_initramfs_list.sh
> > +++ b/scripts/gen_initramfs_list.sh
> > @@ -257,7 +257,7 @@ case "$arg" in
> > && compr="lzop -9 -f"
> > echo "$output_file" | grep -q "\.lz4$" \
> > && [ -x "`which lz4 2> /dev/null`" ] \
> > - && compr="lz4 -9 -f"
> > + && compr="lz4 -l -9 -f"
> > echo "$output_file" | grep -q "\.cpio$" && compr="cat"
> > shift
> > ;;
>
> What happens is the user is running an old version of /bin/lz4? A
> version which predates this switch to a new format? Do those earlier
> versions accept -l, even though they don't need it? Or will the kernel
> build fail?

It seems that lz4 supports legacy format with the same option as lz4c
does. Just looking at the first few bytes of lz4 compressed image, we
can see whether it is new format or not.

It shows new format magic number without this patch. New format magic
number is 0x184d2204.

$ hexdump -C ./initramfs_data.cpio.lz4 |more
00000000 04 22 4d 18 64 70 b9 69 (Little Endian)
...

Currently Kernel supports legacy format only.

Acked-by: Kyungsik Lee <[email protected]>

Thanks,
Kyungsik