Received: by 2002:a25:6193:0:0:0:0:0 with SMTP id v141csp1030060ybb; Wed, 8 Apr 2020 15:02:17 -0700 (PDT) X-Google-Smtp-Source: APiQypLKGyWrd6ug+dATmBggojt1SFVy4exjK1bgRSAENDK1CW+H7tTwiTa0TmXHuaX3WhTXwyTj X-Received: by 2002:a9d:4f0:: with SMTP id 103mr7351663otm.336.1586383337844; Wed, 08 Apr 2020 15:02:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1586383337; cv=none; d=google.com; s=arc-20160816; b=MfX2n1MztR7LFMQD0CQQQJGgY43TvAisfGmYea5TIluch1rZdpsqkWoJoaPXCwOjSW hBcQ3Qx+BbZ9TUvSqL+TOt0s6q/ow7fXpafgosfNI570ReedhHWpEnNyDcvHAa3mkaVa aTHEUnnRWrs3vyYJIGvHCrEJqK56iKW8CHvpGoNfTnkG+vU7/ojhCsZtFaTXnso8s7Hh pAs1DOO+yizMntlc5brDdMfdonNbl1hturj5tGuajgKGHQ2AfODMDPmVtxwFQAWPJuOM +XVwm2lVGa3a+UdBrlGfA8GjPoWPEchTn7MeA5HMmgHP+DGkWvGalAbiFmdltJmR9jWD du0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=KBq0FbpbyyOegkK860IedQejNmDq0nqvjMAmUchq/dk=; b=FmW45F/I633Xqiy1wegZM5kJP5mX0vgoV2gQd2e1yHrisjmKI1kowKurz+F49Ei8lb xhWAJqMfEtfcEi+1wC0qJAfE2tVGUcA2k/eIB6htYXHi/r86TQpC9mVl7YRbyine6O9A wKGH0ZSqBW5agVNtggudcu/CMGJPoq8dNHS0cVJos8BhQaFAXiy51YBe8jFRr200A1bZ rwUMhix/H1eO/17e+FyTvGGRfVVsIHozFkcKSmuNg8pewfYP+n3ITGJ1An/kJhp+4rxC 5Jc/d4fFGyR9ctM75SB/DpWnoG70VgqLVl7wiitwM3kopCCnUxcmIFbJjOBMB3rcpY9s I81Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=sHYfy96k; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id v8si1419839ooq.63.2020.04.08.15.02.05; Wed, 08 Apr 2020 15:02:17 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=sHYfy96k; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726950AbgDHV4Y (ORCPT + 99 others); Wed, 8 Apr 2020 17:56:24 -0400 Received: from mail-pj1-f43.google.com ([209.85.216.43]:51614 "EHLO mail-pj1-f43.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726799AbgDHV4W (ORCPT ); Wed, 8 Apr 2020 17:56:22 -0400 Received: by mail-pj1-f43.google.com with SMTP id n4so406630pjp.1; Wed, 08 Apr 2020 14:56:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KBq0FbpbyyOegkK860IedQejNmDq0nqvjMAmUchq/dk=; b=sHYfy96kGprsMzhsjsLaxtRatkCOOoSKSaNdlhZyV54ZncKVqe54BMQlZXJhdh1hSd NYPONFOXMA+V2wauUZl/9/2IwFk3BeynJnTfS/I1k/slc4hTIcSrcOwJ5xGJPZ6+MHFx DUJUO5gA2nvl7cTnyHHX9r5JHHODPQLxoj+/wWJFlMMtsG42lOoMhKojEbYcuYuU4WCw J38kcz7ZvKKJrlz86X+5cWR/ESdzyLPe3mVPKW/dMsfRZiTboQEl2fULFW+lB+L0Dgat rINJzbWV8xzksPFC77vNGDmy8gPuE/BT9c9r5vxyeqSF2jChH0K83GAuvVkOSMpm9J8V O7jQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KBq0FbpbyyOegkK860IedQejNmDq0nqvjMAmUchq/dk=; b=YTVjo6OpjLgCfo/nXnKDYdy7dzZE4j+HiRpJaYVmuhcbsKNWAl31uKJovtr3TBCm0c vqqy59AZrndvShdwZnTcRjuZNITrYiDNRzbmiGuShoBF+xeGeHrd2Rw4vgCSPtmiNxLQ D6wV2azrwIzLWDWGHjAH6vrJv6vEQDqFzGXRlK53xHALREneQqgyJoJUCLpMmaHC91gH K1737/0f7GhiRr7DFqqtTINOJe0pRgsLzsz+P0jioT1oS30VKsqq84mQZ9FBsfIA7fN1 GYPKwjU0fAx/CcY5tD1BeTbZJRYmsNyO25FCakZEcC2zEwXJh4TNr5fzqRl9XAy9NJqx n9VA== X-Gm-Message-State: AGi0Pub+q2+tDHdUAWIR16LE4zHw/T/z/R/2YvPRPZsxrPU/Qq4oc/VL poLXE7+xamEiPTcUk3LfPOM= X-Received: by 2002:a17:90a:8c18:: with SMTP id a24mr8109135pjo.43.1586382980092; Wed, 08 Apr 2020 14:56:20 -0700 (PDT) Received: from nickserv.localdomain (c-73-222-55-223.hsd1.ca.comcast.net. [73.222.55.223]) by smtp.gmail.com with ESMTPSA id ci13sm459604pjb.16.2020.04.08.14.56.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Apr 2020 14:56:19 -0700 (PDT) From: Nick Terrell To: Nick Terrell Cc: linux-kernel@vger.kernel.org, Chris Mason , linux-kbuild@vger.kernel.org, x86@kernel.org, gregkh@linuxfoundation.org, Petr Malat , Kees Cook , Kernel Team , Nick Terrell , Adam Borowski , Patrick Williams , rmikey@fb.com, mingo@kernel.org, Patrick Williams , Sedat Dilek Subject: [PATCH v5 7/8] x86: Add support for ZSTD compressed kernel Date: Wed, 8 Apr 2020 14:57:10 -0700 Message-Id: <20200408215711.137639-8-nickrterrell@gmail.com> X-Mailer: git-send-email 2.26.0 In-Reply-To: <20200408215711.137639-1-nickrterrell@gmail.com> References: <20200408215711.137639-1-nickrterrell@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Nick Terrell * Add support for zstd compressed kernel * Bump the heap size for zstd. * Update the documentation. Integrates the ZSTD decompression code to the x86 pre-boot code. Zstandard requires slightly more memory during the kernel decompression on x86 (192 KB vs 64 KB), and the memory usage is independent of the window size. This patch has been boot tested with both a zstd and gzip compressed kernel on i386 and x86_64 using buildroot and QEMU. Additionally, this has been tested in production on x86_64 devices. We saw a 2 second boot time reduction by switching kernel compression from xz to zstd. Reviewed-by: Kees Cook Tested-by: Sedat Dilek Signed-off-by: Nick Terrell --- Documentation/x86/boot.rst | 6 +++--- arch/x86/Kconfig | 1 + arch/x86/boot/compressed/Makefile | 5 ++++- arch/x86/boot/compressed/misc.c | 4 ++++ arch/x86/include/asm/boot.h | 6 ++++-- 5 files changed, 16 insertions(+), 6 deletions(-) diff --git a/Documentation/x86/boot.rst b/Documentation/x86/boot.rst index fa7ddc0428c8..0404e99dc1d4 100644 --- a/Documentation/x86/boot.rst +++ b/Documentation/x86/boot.rst @@ -782,9 +782,9 @@ Protocol: 2.08+ uncompressed data should be determined using the standard magic numbers. The currently supported compression formats are gzip (magic numbers 1F 8B or 1F 9E), bzip2 (magic number 42 5A), LZMA - (magic number 5D 00), XZ (magic number FD 37), and LZ4 (magic number - 02 21). The uncompressed payload is currently always ELF (magic - number 7F 45 4C 46). + (magic number 5D 00), XZ (magic number FD 37), LZ4 (magic number + 02 21) and ZSTD (magic number 28 B5). The uncompressed payload is + currently always ELF (magic number 7F 45 4C 46). ============ ============== Field name: payload_length diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 886fa8368256..912f783bc01a 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -185,6 +185,7 @@ config X86 select HAVE_KERNEL_LZMA select HAVE_KERNEL_LZO select HAVE_KERNEL_XZ + select HAVE_KERNEL_ZSTD select HAVE_KPROBES select HAVE_KPROBES_ON_FTRACE select HAVE_FUNCTION_ERROR_INJECTION diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile index 7619742f91c9..471e61400a2e 100644 --- a/arch/x86/boot/compressed/Makefile +++ b/arch/x86/boot/compressed/Makefile @@ -26,7 +26,7 @@ OBJECT_FILES_NON_STANDARD := y KCOV_INSTRUMENT := n targets := vmlinux vmlinux.bin vmlinux.bin.gz vmlinux.bin.bz2 vmlinux.bin.lzma \ - vmlinux.bin.xz vmlinux.bin.lzo vmlinux.bin.lz4 + vmlinux.bin.xz vmlinux.bin.lzo vmlinux.bin.lz4 vmlinux.bin.zst KBUILD_CFLAGS := -m$(BITS) -O2 KBUILD_CFLAGS += -fno-strict-aliasing $(call cc-option, -fPIE, -fPIC) @@ -145,6 +145,8 @@ $(obj)/vmlinux.bin.lzo: $(vmlinux.bin.all-y) FORCE $(call if_changed,lzo) $(obj)/vmlinux.bin.lz4: $(vmlinux.bin.all-y) FORCE $(call if_changed,lz4) +$(obj)/vmlinux.bin.zst: $(vmlinux.bin.all-y) FORCE + $(call if_changed,zstd) suffix-$(CONFIG_KERNEL_GZIP) := gz suffix-$(CONFIG_KERNEL_BZIP2) := bz2 @@ -152,6 +154,7 @@ suffix-$(CONFIG_KERNEL_LZMA) := lzma suffix-$(CONFIG_KERNEL_XZ) := xz suffix-$(CONFIG_KERNEL_LZO) := lzo suffix-$(CONFIG_KERNEL_LZ4) := lz4 +suffix-$(CONFIG_KERNEL_ZSTD) := zst quiet_cmd_mkpiggy = MKPIGGY $@ cmd_mkpiggy = $(obj)/mkpiggy $< > $@ diff --git a/arch/x86/boot/compressed/misc.c b/arch/x86/boot/compressed/misc.c index 9652d5c2afda..39e592d0e0b4 100644 --- a/arch/x86/boot/compressed/misc.c +++ b/arch/x86/boot/compressed/misc.c @@ -77,6 +77,10 @@ static int lines, cols; #ifdef CONFIG_KERNEL_LZ4 #include "../../../../lib/decompress_unlz4.c" #endif + +#ifdef CONFIG_KERNEL_ZSTD +#include "../../../../lib/decompress_unzstd.c" +#endif /* * NOTE: When adding a new decompressor, please update the analysis in * ../header.S. diff --git a/arch/x86/include/asm/boot.h b/arch/x86/include/asm/boot.h index 680c320363db..d6dd43d25d9f 100644 --- a/arch/x86/include/asm/boot.h +++ b/arch/x86/include/asm/boot.h @@ -24,9 +24,11 @@ # error "Invalid value for CONFIG_PHYSICAL_ALIGN" #endif -#ifdef CONFIG_KERNEL_BZIP2 +#if defined(CONFIG_KERNEL_BZIP2) # define BOOT_HEAP_SIZE 0x400000 -#else /* !CONFIG_KERNEL_BZIP2 */ +#elif defined(CONFIG_KERNEL_ZSTD) +# define BOOT_HEAP_SIZE 0x30000 +#else # define BOOT_HEAP_SIZE 0x10000 #endif -- 2.26.0