2021-06-15 11:43:00

by Dimitri John Ledkov

[permalink] [raw]
Subject: [PATCH] s390/boot: add zstd support

Enable ztsd support in s390/boot, to enable booting with zstd
compressed kernel when configured with CONFIG_KERNEL_ZSTD=y.

BugLink: https://bugs.launchpad.net/bugs/1931725
Signed-off-by: Dimitri John Ledkov <[email protected]>
cc: Heiko Carstens <[email protected]>
cc: Vasily Gorbik <[email protected]>
cc: Christian Borntraeger <[email protected]>
cc: [email protected]
---
arch/s390/Kconfig | 1 +
arch/s390/boot/compressed/Makefile | 4 ++++
arch/s390/boot/compressed/decompressor.c | 4 ++++
3 files changed, 9 insertions(+)

diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index b4c7c34069f8..fdbf584e13eb 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -172,6 +172,7 @@ config S390
select HAVE_KERNEL_LZO
select HAVE_KERNEL_UNCOMPRESSED
select HAVE_KERNEL_XZ
+ select HAVE_KERNEL_ZSTD
select HAVE_KPROBES
select HAVE_KPROBES_ON_FTRACE
select HAVE_KRETPROBES
diff --git a/arch/s390/boot/compressed/Makefile b/arch/s390/boot/compressed/Makefile
index de18dab518bb..75660aaaa7a3 100644
--- a/arch/s390/boot/compressed/Makefile
+++ b/arch/s390/boot/compressed/Makefile
@@ -14,6 +14,7 @@ obj-y := $(if $(CONFIG_KERNEL_UNCOMPRESSED),,decompressor.o) info.o
obj-all := $(obj-y) piggy.o syms.o
targets := vmlinux.lds vmlinux vmlinux.bin vmlinux.bin.gz vmlinux.bin.bz2
targets += vmlinux.bin.xz vmlinux.bin.lzma vmlinux.bin.lzo vmlinux.bin.lz4
+targets += vmlinux.bin.zst
targets += info.bin syms.bin vmlinux.syms $(obj-all)

KBUILD_AFLAGS := $(KBUILD_AFLAGS_DECOMPRESSOR)
@@ -63,6 +64,7 @@ suffix-$(CONFIG_KERNEL_LZ4) := .lz4
suffix-$(CONFIG_KERNEL_LZMA) := .lzma
suffix-$(CONFIG_KERNEL_LZO) := .lzo
suffix-$(CONFIG_KERNEL_XZ) := .xz
+suffix-$(CONFIG_KERNEL_ZSTD) := .zst

$(obj)/vmlinux.bin.gz: $(vmlinux.bin.all-y) FORCE
$(call if_changed,gzip)
@@ -76,6 +78,8 @@ $(obj)/vmlinux.bin.lzo: $(vmlinux.bin.all-y) FORCE
$(call if_changed,lzo)
$(obj)/vmlinux.bin.xz: $(vmlinux.bin.all-y) FORCE
$(call if_changed,xzkern)
+$(obj)/vmlinux.bin.zst: $(vmlinux.bin.all-y) FORCE
+ $(call if_changed,zstd22)

OBJCOPYFLAGS_piggy.o := -I binary -O elf64-s390 -B s390:64-bit --rename-section .data=.vmlinux.bin.compressed
$(obj)/piggy.o: $(obj)/vmlinux.bin$(suffix-y) FORCE
diff --git a/arch/s390/boot/compressed/decompressor.c b/arch/s390/boot/compressed/decompressor.c
index 3061b11c4d27..02fd0d245c7e 100644
--- a/arch/s390/boot/compressed/decompressor.c
+++ b/arch/s390/boot/compressed/decompressor.c
@@ -61,6 +61,10 @@ static unsigned long free_mem_end_ptr = (unsigned long) _end + BOOT_HEAP_SIZE;
#include "../../../../lib/decompress_unxz.c"
#endif

+#ifdef CONFIG_KERNEL_ZSTD
+#include "../../../../lib/decompress_unzstd.c"
+#endif
+
#define decompress_offset ALIGN((unsigned long)_end + BOOT_HEAP_SIZE, PAGE_SIZE)

unsigned long mem_safe_offset(void)
--
2.27.0


2021-06-15 23:21:48

by Vasily Gorbik

[permalink] [raw]
Subject: Re: [PATCH] s390/boot: add zstd support

On Tue, Jun 15, 2021 at 12:41:50PM +0100, Dimitri John Ledkov wrote:
> Enable ztsd support in s390/boot, to enable booting with zstd
> compressed kernel when configured with CONFIG_KERNEL_ZSTD=y.
>
> BugLink: https://bugs.launchpad.net/bugs/1931725
> Signed-off-by: Dimitri John Ledkov <[email protected]>
> cc: Heiko Carstens <[email protected]>
> cc: Vasily Gorbik <[email protected]>
> cc: Christian Borntraeger <[email protected]>
> cc: [email protected]
> ---
> arch/s390/Kconfig | 1 +
> arch/s390/boot/compressed/Makefile | 4 ++++
> arch/s390/boot/compressed/decompressor.c | 4 ++++
> 3 files changed, 9 insertions(+)

Reviewing your patch I noticed that we use wrong condition to
define BOOT_HEAP_SIZE. So I made a tiny fix:

diff --git a/arch/s390/boot/compressed/decompressor.c b/arch/s390/boot/compressed/decompressor.c
index 3061b11c4d27..cf2571050c68 100644
--- a/arch/s390/boot/compressed/decompressor.c
+++ b/arch/s390/boot/compressed/decompressor.c
@@ -29,5 +29,5 @@ extern unsigned char _compressed_start[];
extern unsigned char _compressed_end[];

-#ifdef CONFIG_HAVE_KERNEL_BZIP2
+#ifdef CONFIG_KERNEL_BZIP2
#define BOOT_HEAP_SIZE 0x400000
#else

And applied your patch with the following changes:
Added to the commit message:
"""
BOOT_HEAP_SIZE is defined to 0x30000 in this case. Actual decompressor
memory usage with allyesconfig is currently 0x26150.
"""

diff --git a/arch/s390/boot/compressed/decompressor.c b/arch/s390/boot/compressed/decompressor.c
index cf2571050c68..37a4a8d33c6c 100644
--- a/arch/s390/boot/compressed/decompressor.c
+++ b/arch/s390/boot/compressed/decompressor.c
@@ -31,4 +31,6 @@ extern unsigned char _compressed_end[];
#ifdef CONFIG_KERNEL_BZIP2
#define BOOT_HEAP_SIZE 0x400000
+#elif CONFIG_KERNEL_ZSTD
+#define BOOT_HEAP_SIZE 0x30000
#else
#define BOOT_HEAP_SIZE 0x10000

I hope you are ok with that, thanks!

2021-06-16 07:39:04

by Dimitri John Ledkov

[permalink] [raw]
Subject: Re: [PATCH] s390/boot: add zstd support

On Wed, Jun 16, 2021 at 12:20 AM Vasily Gorbik <[email protected]> wrote:
>
> On Tue, Jun 15, 2021 at 12:41:50PM +0100, Dimitri John Ledkov wrote:
> > Enable ztsd support in s390/boot, to enable booting with zstd
> > compressed kernel when configured with CONFIG_KERNEL_ZSTD=y.
> >
> > BugLink: https://bugs.launchpad.net/bugs/1931725
> > Signed-off-by: Dimitri John Ledkov <[email protected]>
> > cc: Heiko Carstens <[email protected]>
> > cc: Vasily Gorbik <[email protected]>
> > cc: Christian Borntraeger <[email protected]>
> > cc: [email protected]
> > ---
> > arch/s390/Kconfig | 1 +
> > arch/s390/boot/compressed/Makefile | 4 ++++
> > arch/s390/boot/compressed/decompressor.c | 4 ++++
> > 3 files changed, 9 insertions(+)
>
> Reviewing your patch I noticed that we use wrong condition to
> define BOOT_HEAP_SIZE. So I made a tiny fix:
>
> diff --git a/arch/s390/boot/compressed/decompressor.c b/arch/s390/boot/compressed/decompressor.c
> index 3061b11c4d27..cf2571050c68 100644
> --- a/arch/s390/boot/compressed/decompressor.c
> +++ b/arch/s390/boot/compressed/decompressor.c
> @@ -29,5 +29,5 @@ extern unsigned char _compressed_start[];
> extern unsigned char _compressed_end[];
>
> -#ifdef CONFIG_HAVE_KERNEL_BZIP2
> +#ifdef CONFIG_KERNEL_BZIP2
> #define BOOT_HEAP_SIZE 0x400000
> #else
>

Nice. I guess it means all kernels were always built with large
HEAP_SIZE, and that's why my boot tests worked too.

Note this bug is also present in arch/sh/boot/compressed/misc.c.


> And applied your patch with the following changes:
> Added to the commit message:
> """
> BOOT_HEAP_SIZE is defined to 0x30000 in this case. Actual decompressor
> memory usage with allyesconfig is currently 0x26150.
> """
>

That's appropriate.

> diff --git a/arch/s390/boot/compressed/decompressor.c b/arch/s390/boot/compressed/decompressor.c
> index cf2571050c68..37a4a8d33c6c 100644
> --- a/arch/s390/boot/compressed/decompressor.c
> +++ b/arch/s390/boot/compressed/decompressor.c
> @@ -31,4 +31,6 @@ extern unsigned char _compressed_end[];
> #ifdef CONFIG_KERNEL_BZIP2
> #define BOOT_HEAP_SIZE 0x400000
> +#elif CONFIG_KERNEL_ZSTD
> +#define BOOT_HEAP_SIZE 0x30000
> #else
> #define BOOT_HEAP_SIZE 0x10000
>
> I hope you are ok with that, thanks!

Thank you!

--
Regards,

Dimitri.