[ CC: Nick Desaulniers ]
Hi Nick,
I have tested your patchset v3 on top of Linux v5.6-rc7+
compiled/linked with Clang/LLD v9.0.1.
My Linux system is Debian/testing (will be "bullseye" aka version 11) AMD64.
I used zstd package version 1.4.4+dfsg-3.
I have no big numbers for you expect a comparison of initrd.img files
compressed with GZIP (default by Debian) and ZSTD:
root# du -k /boot/initrd.img-$(uname -r).{gz,zst}
29272 /boot/initrd.img-5.6.0-rc7-1-amd64-clang.gz
23540 /boot/initrd.img-5.6.0-rc7-1-amd64-clang.zst
I needed to do some modifications to stuff from
initramfs-tools/initramfs-tools-core (see end-titles).
Do you plan to push this for Linux v5.7?
Feel free to add credits for the whole series:
Tested-by: Sedat Dilek <[email protected]>
I have attached my kernel-config and dmesg-output.
Thanks.
Regards,
- Sedat -
P.S.: For posterity / ensuing ages - README_zstd-v3.txt (file is
attached seperately)
[ PREREQS ]
LINK: https://lore.kernel.org/lkml/[email protected]/
("[PATCH v3 0/8] Add support for ZSTD-compressed kernel and initramfs")
LINK: https://github.com/terrelln/linux/commits/zstd-v3
zstd compression support for initramfs requires additional patches for
the Linux kernel (see above Link).
NOTE-1: Tested zstd-v3 patchset with Linux v5.6-rc7 on x86 64-bit (dileks)
NOTE-2: My Linux OS is Debian/testing AMD64.
NOTE-3: Install zstd Debian package (here: version 1.4.4+dfsg-3).
[ EDITS ]
[ EDIT: /etc/initramfs-tools/initramfs.conf ]
-# COMPRESS: [ gzip | bzip2 | lz4 | lzma | lzop | xz ]
+# COMPRESS: [ gzip | bzip2 | lz4 | lzma | lzop | xz | zstd ]
-COMPRESS=gzip
+COMPRESS=zstd
[ EDIT: /usr/sbin/mkinitramfs ]
case "${compress}" in
+zstd) compress="zstd -19 -v" ;;
bzip2|lzma|lzop)
esac
NOTE: In my kernel build-log I have seen 'zstd -19' to compress the
vmlinux file ('-v' for verbose output).
$ grep zstd build-log_5.6.0-rc7-1-amd64-clang.txt | grep vmlinux
(cat arch/x86/boot/compressed/vmlinux.bin
arch/x86/boot/compressed/vmlinux.relocs | zstd -19 && printf
\220\016\300\001) > arch/x86/boot/compressed/vmlinux.bin.zst || (rm -f
arch/x86/boot/compressed/vmlinux.bin.zst ; false)
[ EDIT: /usr/bin/unmkinitramfs ]
# Extract a compressed cpio archive
xcpio()
{
if gzip -t "$archive" >/dev/null 2>&1 ; then
gzip -c -d "$archive"
+ elif zstdcat -t "$archive" >/dev/null 2>&1 ; then
+ zstdcat "$archive"
elif xzcat -t "$archive" >/dev/null 2>&1 ; then
}
[ UPDATE-INITRAMFS ]
root# update-initramfs -c -k $(uname -r) -v 2>&1 | tee log_update-initramfs.txt
...
Building cpio /boot/initrd.img-5.6.0-rc7-1-amd64-clang.new initramfs
*** zstd command line interface 64-bits v1.4.4, by Yann Collet ***
/*stdin*\ : 23.58% (89630208 => 21134850 bytes, /*stdout*\)
[ INITRD-IMAGE SIZES ]
root# du -k /boot/initrd.img-$(uname -r).{gz,zst}
29272 /boot/initrd.img-5.6.0-rc7-1-amd64-clang.gz
23540 /boot/initrd.img-5.6.0-rc7-1-amd64-clang.zst
[ INSPECT AND EXTRACT INITRD-IMAGE ]
root# lsinitramfs -l /boot/initrd.img-5.6.0-rc7-1-amd64-clang
root# unmkinitramfs -v /boot/initrd.img-5.6.0-rc7-1-amd64-clang
/path/to/unpacked_dir
[ CHECK VMLINUX ]
user$ file vmlinux
vmlinux: ELF 64-bit LSB executable, x86-64, version 1 (SYSV),
statically linked, BuildID[xxHash]=e04120f126f447e9, with debug_info,
not stripped
user$ file arch/x86/boot/compressed/vmlinux{,.bin*}
arch/x86/boot/compressed/vmlinux: ELF 64-bit LSB executable,
x86-64, version 1 (SYSV), statically linked, not stripped
arch/x86/boot/compressed/vmlinux.bin: ELF 64-bit LSB executable,
x86-64, version 1 (SYSV), statically linked,
BuildID[xxHash]=e04120f126f447e9, stripped
arch/x86/boot/compressed/vmlinux.bin.zst: Zstandard compressed data
(v0.8+), Dictionary ID: None
user$ du -k arch/x86/boot/compressed/vmlinux{,.bin*}
6536 arch/x86/boot/compressed/vmlinux
21448 arch/x86/boot/compressed/vmlinux.bin
6432 arch/x86/boot/compressed/vmlinux.bin.zst
-dileks // 29-MAR-2020: Add more Links; Add new "CHECK VMLINUX"
section; Add note about zstd package requirements
-dileks // 28-MAR-2020: Initial release
- EOT -
> On Mar 29, 2020, at 8:45 AM, Sedat Dilek <[email protected]> wrote:
>
> [ CC: Nick Desaulniers ]
>
> Hi Nick,
>
> I have tested your patchset v3 on top of Linux v5.6-rc7+
> compiled/linked with Clang/LLD v9.0.1.
>
> My Linux system is Debian/testing (will be "bullseye" aka version 11) AMD64.
> I used zstd package version 1.4.4+dfsg-3.
>
> I have no big numbers for you expect a comparison of initrd.img files
> compressed with GZIP (default by Debian) and ZSTD:
>
> root# du -k /boot/initrd.img-$(uname -r).{gz,zst}
> 29272 /boot/initrd.img-5.6.0-rc7-1-amd64-clang.gz
> 23540 /boot/initrd.img-5.6.0-rc7-1-amd64-clang.zst
>
> I needed to do some modifications to stuff from
> initramfs-tools/initramfs-tools-core (see end-titles).
>
> Do you plan to push this for Linux v5.7?
I hope that it will be accepted. From my point of view this patch set is
ready for merge, except for the maximum window size increase
requested by Petr.
> Feel free to add credits for the whole series:
>
> Tested-by: Sedat Dilek <[email protected]>
Thanks for testing the patch set Sedat!
-Nick
On Tue, Mar 31, 2020 at 6:15 AM Nick Terrell <[email protected]> wrote:
[...]
> > Do you plan to push this for Linux v5.7?
>
> I hope that it will be accepted. From my point of view this patch set is
> ready for merge, except for the maximum window size increase
> requested by Petr.
>
Hi Nick,
thanks for your answer.
Did you send out a pull-request already?
Can you point me to that request or patch of Petr?
Is it relevant or optional for the pull-request?
> > Feel free to add credits for the whole series:
> >
> > Tested-by: Sedat Dilek <[email protected]>
>
> Thanks for testing the patch set Sedat!
>
I have re-tested zstd-v3 patchset with Linux version 5.6 final and
Clang/LLD version 10.0.0 final (from Debian/unstable repository).
Is it possible to mention that there might distro-specific changes
needed to initramfs-handling?
For Debian you are welcome to include below Link [1].
Not sure I will send/ask to/on the debian-kernel mailing list in this topic.
Thanks and bonne chance.
Regards,
- Sedat -
[1] LINK: https://lore.kernel.org/lkml/CA+icZUXCn2an9aNDrm+-eneSAOyGibz0W1xYhwkA5k3B3U-5vQ@mail.gmail.com/
> On Mar 31, 2020, at 8:11 AM, Sedat Dilek <[email protected]> wrote:
>
> On Tue, Mar 31, 2020 at 6:15 AM Nick Terrell <[email protected]> wrote:
>
> [...]
>
>>> Do you plan to push this for Linux v5.7?
>>
>> I hope that it will be accepted. From my point of view this patch set is
>> ready for merge, except for the maximum window size increase
>> requested by Petr.
>>
>
> Hi Nick,
>
> thanks for your answer.
>
> Did you send out a pull-request already?
I haven’t sent out a pull request. If that is something that I need to
do, or if it makes the process easier for a maintainer to merge then
I can do that.
> Can you point me to that request or patch of Petr?
> Is it relevant or optional for the pull-request?
The patches from Petr are a parallel implementation of initramfs
decompression and are available here [0]. They support initramfs
compression well, but are missing some pieces for correct and
performant zstd kernel decompression. Namely the equivalent of
patch 6 in this series to increase ZO_z_extra_bytes, the
performance fix in patch 1 that switches memcpy() to
__builtin_memcpy(), and the decompress_single() function in patch 3.
Without those changes a very compressible kernel may fail to
decompress successfully, and will be corrupted. And the performance
is about 4x worse on x86_64. My patches took 70 ms to decompress vs
318 ms with Petr’s. See the detailed comparison here [1].
>>> Feel free to add credits for the whole series:
>>>
>>> Tested-by: Sedat Dilek <[email protected]>
>>
>> Thanks for testing the patch set Sedat!
>>
>
> I have re-tested zstd-v3 patchset with Linux version 5.6 final and
> Clang/LLD version 10.0.0 final (from Debian/unstable repository).
>
> Is it possible to mention that there might distro-specific changes
> needed to initramfs-handling?
> For Debian you are welcome to include below Link [1].
> Not sure I will send/ask to/on the debian-kernel mailing list in this topic.
I don’t expect any distro specific changes are required to continue operating
as-is. However, if a distro wanted to switch to a zstd compressed initramfs
they would need to update their toolchain to compress with zstd.
> Thanks and bonne chance.
>
> Regards,
> - Sedat -
>
> [1] LINK: https://lore.kernel.org/lkml/CA+icZUXCn2an9aNDrm+-eneSAOyGibz0W1xYhwkA5k3B3U-5vQ@mail.gmail.com/
[0] https://lkml.org/lkml/2020/3/16/461
[1] https://lkml.org/lkml/2020/3/31/1573
On Tue, Mar 31, 2020 at 05:11:35PM +0200, Sedat Dilek wrote:
> I have re-tested zstd-v3 patchset with Linux version 5.6 final and
> Clang/LLD version 10.0.0 final (from Debian/unstable repository).
So did I and a bunch of other folks. I for one run v1 since 2017 on
a bunch of boxes (amd64/BIOS, amd64/EFI, arm64, armhf), without a glitch
(not counting trying to boot zstd initrd _without_ the patchset :p).
I've tried v2 on all of the above configurations, v3 on all but armhf, v4 on
amd64/EFI -- all is fine.
Back in the days, folks reported success on IIRC ppc64, sparc64 and more.
(Obviously, initrd only on !x86.)
> Is it possible to mention that there might distro-specific changes
> needed to initramfs-handling?
> For Debian you are welcome to include below Link [1].
> [1] LINK: https://lore.kernel.org/lkml/CA+icZUXCn2an9aNDrm+-eneSAOyGibz0W1xYhwkA5k3B3U-5vQ@mail.gmail.com/
+zstd) compress="zstd -19 -v" ;;
I'm not sure if hard-coding -19 is always right. It's good for production
machines but slows down dev cycles.
But that's a matter for userland, not a problem with Nick's patchset.
Meow!
--
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ in the beginning was the boot and root floppies and they were good.
⢿⡄⠘⠷⠚⠋⠀ -- <willmore> on #linux-sunxi
⠈⠳⣄⠀⠀⠀⠀
On Wed, Apr 01, 2020 at 02:52:06AM +0000, Nick Terrell wrote:
> > On Mar 31, 2020, at 8:11 AM, Sedat Dilek <[email protected]> wrote:
> > On Tue, Mar 31, 2020 at 6:15 AM Nick Terrell <[email protected]> wrote:
> >>> Do you plan to push this for Linux v5.7?
> >>
> >> I hope that it will be accepted. From my point of view this patch set is
> >> ready for merge, except for the maximum window size increase
> >> requested by Petr.
If you target 5.7, remember that the merge window for 5.7 is already going
on, and Linus is extremely cranky about pulling anything that hasn't stewed
in next for a while.
Your patchset hasn't been in next yet, so the timing is very very late.
You'd need to bother Stephen Rothwell or someone who has a tree in next
about right now to have any chances.
> > Is it possible to mention that there might distro-specific changes
> > needed to initramfs-handling?
> > For Debian you are welcome to include below Link [1].
> > Not sure I will send/ask to/on the debian-kernel mailing list in this topic.
>
> I don’t expect any distro specific changes are required to continue operating
> as-is. However, if a distro wanted to switch to a zstd compressed initramfs
> they would need to update their toolchain to compress with zstd.
Just setting COMPRESS=zstd is enough -- it'll say:
W: Unknown compression command zstd
but will do everything right.
Also, just minutes ago someone filed https://bugs.debian.org/955469
Meow!
--
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ in the beginning was the boot and root floppies and they were good.
⢿⡄⠘⠷⠚⠋⠀ -- <willmore> on #linux-sunxi
⠈⠳⣄⠀⠀⠀⠀
On Wed, Apr 1, 2020 at 4:52 AM Nick Terrell <[email protected]> wrote:
>
>
>
> > On Mar 31, 2020, at 8:11 AM, Sedat Dilek <[email protected]> wrote:
> >
> > On Tue, Mar 31, 2020 at 6:15 AM Nick Terrell <[email protected]> wrote:
> >
> > [...]
> >
> >>> Do you plan to push this for Linux v5.7?
> >>
> >> I hope that it will be accepted. From my point of view this patch set is
> >> ready for merge, except for the maximum window size increase
> >> requested by Petr.
> >>
> >
> > Hi Nick,
> >
> > thanks for your answer.
> >
> > Did you send out a pull-request already?
>
> I haven’t sent out a pull request. If that is something that I need to
> do, or if it makes the process easier for a maintainer to merge then
> I can do that.
>
[ CC Stephen Rothwell (maintainer of Linux-next) ]
Hi Nick,
as Adam pointed out it might be good to ask "Stephen Rothwell"
<[email protected]>
to include your Git tree into linux-next. Stephen will give you some
informations about that.
> > Can you point me to that request or patch of Petr?
> > Is it relevant or optional for the pull-request?
>
> The patches from Petr are a parallel implementation of initramfs
> decompression and are available here [0]. They support initramfs
> compression well, but are missing some pieces for correct and
> performant zstd kernel decompression. Namely the equivalent of
> patch 6 in this series to increase ZO_z_extra_bytes, the
> performance fix in patch 1 that switches memcpy() to
> __builtin_memcpy(), and the decompress_single() function in patch 3.
>
> Without those changes a very compressible kernel may fail to
> decompress successfully, and will be corrupted. And the performance
> is about 4x worse on x86_64. My patches took 70 ms to decompress vs
> 318 ms with Petr’s. See the detailed comparison here [1].
>
Thanks for version 4 of your patchset and these clarifications.
I will try zstd-v4.
Regards,
- Sedat -
> >>> Feel free to add credits for the whole series:
> >>>
> >>> Tested-by: Sedat Dilek <[email protected]>
> >>
> >> Thanks for testing the patch set Sedat!
> >>
> >
> > I have re-tested zstd-v3 patchset with Linux version 5.6 final and
> > Clang/LLD version 10.0.0 final (from Debian/unstable repository).
> >
> > Is it possible to mention that there might distro-specific changes
> > needed to initramfs-handling?
> > For Debian you are welcome to include below Link [1].
> > Not sure I will send/ask to/on the debian-kernel mailing list in this topic.
>
> I don’t expect any distro specific changes are required to continue operating
> as-is. However, if a distro wanted to switch to a zstd compressed initramfs
> they would need to update their toolchain to compress with zstd.
>
> > Thanks and bonne chance.
> >
> > Regards,
> > - Sedat -
> >
> > [1] LINK: https://lore.kernel.org/lkml/CA+icZUXCn2an9aNDrm+-eneSAOyGibz0W1xYhwkA5k3B3U-5vQ@mail.gmail.com/
>
> [0] https://lkml.org/lkml/2020/3/16/461
> [1] https://lkml.org/lkml/2020/3/31/1573
>
On Wed, Apr 1, 2020 at 9:27 AM Adam Borowski <[email protected]> wrote:
>
> On Wed, Apr 01, 2020 at 02:52:06AM +0000, Nick Terrell wrote:
> > > On Mar 31, 2020, at 8:11 AM, Sedat Dilek <[email protected]> wrote:
> > > On Tue, Mar 31, 2020 at 6:15 AM Nick Terrell <[email protected]> wrote:
[...]
> > I don’t expect any distro specific changes are required to continue operating
> > as-is. However, if a distro wanted to switch to a zstd compressed initramfs
> > they would need to update their toolchain to compress with zstd.
>
> Just setting COMPRESS=zstd is enough -- it'll say:
> W: Unknown compression command zstd
> but will do everything right.
>
> Also, just minutes ago someone filed https://bugs.debian.org/955469
>
Hi Adam,
thanks for your feedback and the pointer to the Debian bug #955469.
With "distro-specific changes" I meant the ones to userland - for
Debian - namely initramfs-tools{,-core} packages.
For generating, inspecting and unpacking an initramfs the Debian way
you need a bit more.
In the Bug-BR a patch called "enable_zstd.patch" is attached - seems
to have all what I mentioned in [1].
Regards,
- Sedat -
[1] LINK: https://lore.kernel.org/lkml/CA+icZUXCn2an9aNDrm+-eneSAOyGibz0W1xYhwkA5k3B3U-5vQ@mail.gmail.com/
> On Apr 1, 2020, at 2:50 AM, Sedat Dilek <[email protected]> wrote:
>
> On Wed, Apr 1, 2020 at 4:52 AM Nick Terrell <[email protected]> wrote:
>>
>>
>>
>>> On Mar 31, 2020, at 8:11 AM, Sedat Dilek <[email protected]> wrote:
>>>
>>> On Tue, Mar 31, 2020 at 6:15 AM Nick Terrell <[email protected]> wrote:
>>>
>>> [...]
>>>
>>>>> Do you plan to push this for Linux v5.7?
>>>>
>>>> I hope that it will be accepted. From my point of view this patch set is
>>>> ready for merge, except for the maximum window size increase
>>>> requested by Petr.
>>>>
>>>
>>> Hi Nick,
>>>
>>> thanks for your answer.
>>>
>>> Did you send out a pull-request already?
>>
>> I haven’t sent out a pull request. If that is something that I need to
>> do, or if it makes the process easier for a maintainer to merge then
>> I can do that.
>>
>
> [ CC Stephen Rothwell (maintainer of Linux-next) ]
>
> Hi Nick,
>
> as Adam pointed out it might be good to ask "Stephen Rothwell"
> <[email protected]>
> to include your Git tree into linux-next. Stephen will give you some
> informations about that.
Thanks for the pointers Sedat and Adam! I will reach out and ask
about the best way forward. I don’t particularly care if it makes it in
5.7 or 5.8, I just want to ensure that it does get merged this time.
Best,
Nick
On Wed, Apr 1, 2020 at 9:00 PM Nick Terrell <[email protected]> wrote:
>
> > On Apr 1, 2020, at 2:50 AM, Sedat Dilek <[email protected]> wrote:
> >
> > On Wed, Apr 1, 2020 at 4:52 AM Nick Terrell <[email protected]> wrote:
> >>
> >>
> >>
> >>> On Mar 31, 2020, at 8:11 AM, Sedat Dilek <[email protected]> wrote:
> >>>
> >>> On Tue, Mar 31, 2020 at 6:15 AM Nick Terrell <[email protected]> wrote:
> >>>
> >>> [...]
> >>>
> >>>>> Do you plan to push this for Linux v5.7?
> >>>>
> >>>> I hope that it will be accepted. From my point of view this patch set is
> >>>> ready for merge, except for the maximum window size increase
> >>>> requested by Petr.
> >>>>
> >>>
> >>> Hi Nick,
> >>>
> >>> thanks for your answer.
> >>>
> >>> Did you send out a pull-request already?
> >>
> >> I haven’t sent out a pull request. If that is something that I need to
> >> do, or if it makes the process easier for a maintainer to merge then
> >> I can do that.
> >>
> >
> > [ CC Stephen Rothwell (maintainer of Linux-next) ]
> >
> > Hi Nick,
> >
> > as Adam pointed out it might be good to ask "Stephen Rothwell"
> > <[email protected]>
> > to include your Git tree into linux-next. Stephen will give you some
> > informations about that.
>
> Thanks for the pointers Sedat and Adam! I will reach out and ask
> about the best way forward. I don’t particularly care if it makes it in
> 5.7 or 5.8, I just want to ensure that it does get merged this time.
>
Hi Nick,
just in case you did not know:
The Documentation directory in linux-Git has a subfolder called
"process" (RST format) which gives a lot of hints.
All stuff is also available as HTML if you prefer this format.
See [0] "A guide to the Kernel Development Process".
Hope that helps you.
Regards,
- Sedat -
[0] https://www.kernel.org/doc/html/latest/process/index.html
[1] https://www.kernel.org/doc/html/latest/process/2.Process.html#next-trees
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/
[3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/2.Process.rst
[4] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/submitting-patches.rst