Received: by 2002:a05:7412:cfc7:b0:fc:a2b0:25d7 with SMTP id by7csp2304098rdb; Wed, 21 Feb 2024 03:39:39 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCX4N5lVxREvlN07e7vOJkFj2EX2SfrViyJ5ps0JdJO3hvNyWi/3W2fWMfLLnaU6jYtcTexEzKD2/fleDRyDeM134T4iTyE34+9zLlQp3g== X-Google-Smtp-Source: AGHT+IEaf4MbNsi/Q+EoJ9d45caZLV34zrLBtKqS5Vc4DMq6tYJ80en/sIOh9O3Q1vz0ruQLDVGL X-Received: by 2002:a05:620a:211b:b0:787:7e27:6d5f with SMTP id l27-20020a05620a211b00b007877e276d5fmr3667425qkl.46.1708515578911; Wed, 21 Feb 2024 03:39:38 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708515578; cv=pass; d=google.com; s=arc-20160816; b=uwzluOTgCvA6/QsIdzAVRZp9rJbT4aGi2WwBdH+huAA+vEnS4iEmOwdzAMEiz9mMTq HwcDN2ByI+Kh1LknjSxSmrdLwnSYUlcUs3uEqmJJL14CsM0sNThfxpx98+WkSsXonWIj 7Yj6YjIQ7yTC6dOGWs4pdt+1YyIeMMyXoYwwgaipBKpAFmR8EaAoE5dKXHGnH6UXJkXj xugtnMKpfs2hFhyGjH3kQDMvx/cipqUiaTYq10RGABZ0vrfymNxNWW7ukz+gnQbBJj9t VUGKSRbkufMSEjImfol6iHhH2wQ5X8N/58pFgpxIbFCGlN6Kze3anYbQUmb4c/DO5WAj NTMw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=xDNgWt2lMCvtDV/Eee3l1XVOGlo7HH2KTDyTgOKacbQ=; fh=fMhMPvo8duafGkM4xZHdH7fULbau3etefTGsKa+Qy7I=; b=nyG1pBon/TCBny92B7wejnKJPShud9YD4Ja7LSRZ7bFfW6EcvT6+NibvOeFY4YYzh6 7yHL+HPhJ2CktjZsDfXyrnmx9Z0uhn+Kklzg3evsk8nNzpztx1n1oqARZHl+qdMNY4gx x+wSixAojwei0uQY6/rlApYXrghnpPwEAsZdsOy94JH2tnZuK8U0S6CNhPXlVwSbZbGm pEHrGUKQiB059UiF84e4eex0jXKAU8S2fMwBQ/hjmfrpbx43BIa7br7eikSptJ/enOFg F04bmXhFJaS0IxbW3bFIVY07YseaTwwmxEb9b9XK4joJl+1wAQCf5Y5Bkpuv6EdoGSjo oxkg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=VzRFF5lF; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-74643-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74643-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id y18-20020a05620a44d200b00787389cfc39si12325734qkp.735.2024.02.21.03.39.38 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Feb 2024 03:39:38 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-74643-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=VzRFF5lF; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-74643-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74643-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 99D6A1C22E00 for ; Wed, 21 Feb 2024 11:39:38 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 94B834F618; Wed, 21 Feb 2024 11:36:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="VzRFF5lF" Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C10034654F for ; Wed, 21 Feb 2024 11:36:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708515368; cv=none; b=HViae1dUNttAvW3zrxe84P3Arhthl5R06bvVuk1WYtqvopPUiQ6jLaCT5mIk4Lxzoni51ApkPDt58KiNGG8kicHgb73ZFEpsGk+i4m8YCtC5Tle53Ia6w4ai1AxGPik0chOFm6cuRCY3F73U7puYhsjP7bxUw6yYniVtGawZJ7o= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708515368; c=relaxed/simple; bh=S4NGlxmVOhrQxFAGepaTMi748KWyZv6XTKqvhleJQH8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=FJE6LI/C/1sikmKRhot1ic9JJN0NKQ2e57LQU3nR6HiRQUdT8YAepNpXc6AKwvc5xGu6wYQNTiYEpyRSVn77zCzM/R9xldd5Goy9cddpIWX7SPkr382f4rQS6hE61sbHqmE78NlGRzsJR9cVZ2QNLLUeqSeQsr/JgUd9+89PIZg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=VzRFF5lF; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-dccc49ef73eso7569175276.2 for ; Wed, 21 Feb 2024 03:36:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1708515366; x=1709120166; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=xDNgWt2lMCvtDV/Eee3l1XVOGlo7HH2KTDyTgOKacbQ=; b=VzRFF5lFzKAg9Vc2zjiOzfHdwKTen7ie8RZIuA97pLx1vi/taaK5ogLKf8DIXdNVgN hocoTWaRZaAZAU1W+4aQW6SsUndmmFLDyQMrDTOXhdkKlulr2jm8edFAVtQqHpcD4H8x 8OCefRoiEz6rFOEk1LkssoCak/D5/J4kfwUHVwsyQi//MDgcVSMakenlK/Y79p5QlETM o1tbaXsmw7X1S0LJlh0YMSpZPnQOPg6Af551M/rog1YFa0tMvURRUo08+1RAc+IV8emt 8UgYUGNnRv4vOPTWrjgjxPh9dGz9tOg45ys/oiDOKn8RbHwCwYcJyh4bJj0PmuF8CpBy nvSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708515366; x=1709120166; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xDNgWt2lMCvtDV/Eee3l1XVOGlo7HH2KTDyTgOKacbQ=; b=U1Lp/8XdSPETx8Gzbl3rNeEpIMnrbk/n1XIpRY1tNbflsTDZUEtcandbZbIj1R2LeZ CIphtXaOE4ELIO4xmjGh+UXfpueOflcCE4wi+cys3FjKVuhMfIt9WFO/+ej3KAokTh70 vHgSWls/VtDcp3XIkOiS5nMXkREyIJkCGWxN3GOwjPJsbbfQ65ZYxhkuaU3vdOmm0dK8 JxyxcgDic0yT2t3gEdopXPdMfsecWnG0pb/EYfp4nHLHiM7aovR88OvhouA+SRhefqI9 YRXe5p6rDLr+XraJOmmjpVEEO0kAwrnF+MlZSgS5bqd66orkHOABb+n7OSCJLs0gr8yl DHtw== X-Gm-Message-State: AOJu0Yzz+IVYwvG34mi6EwSBLc41o49vv+9PoPN8/HXWd9l/3FsOUTnd RKmUS2mqlHrly418xR+EJULM2uKgCMJd2Cja3EUXuhrL/AnVMilb6vN+GBwyDhXt7LVeZyfCJOL agabrhETqgYttKQb3fLszX729EGtFvQ4j1i/S0VvWFtIr8lW94W+R2R1CxMMxkHktEsCkz/2XUJ qLqSih9MKOYC/T64o1QADauJBTFrK1nA== X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:6902:1887:b0:dc2:2ace:860 with SMTP id cj7-20020a056902188700b00dc22ace0860mr1032065ybb.2.1708515365485; Wed, 21 Feb 2024 03:36:05 -0800 (PST) Date: Wed, 21 Feb 2024 12:35:20 +0100 In-Reply-To: <20240221113506.2565718-18-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240221113506.2565718-18-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=7014; i=ardb@kernel.org; h=from:subject; bh=oEjbaEakolZKtY3/OEd7gfiAzx2qVYvdQU90/PQTOX8=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIfXq/S+hVVb/nl/wOGHDrbkl6L3ogmRx+xl7VtflzZl9w MeTPedVRykLgxgHg6yYIovA7L/vdp6eKFXrPEsWZg4rE8gQBi5OAZjI08+MDM2+pjLOKhaPn23+ vvn/+rBYbvHl1ddYdgu4yZ2MOD1fZC7Db1bvuq+H0+5u/XZwi+wn3u/3lEuvn9p5VYfP68zHM77 ZPcwA X-Mailer: git-send-email 2.44.0.rc0.258.g7320e95886-goog Message-ID: <20240221113506.2565718-31-ardb+git@google.com> Subject: [PATCH v5 13/16] x86/boot: Move mem_encrypt= parsing to the decompressor From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Kevin Loughlin , Tom Lendacky , Dionna Glaze , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Nathan Chancellor , Nick Desaulniers , Justin Stitt , Kees Cook , Brian Gerst , linux-arch@vger.kernel.org, llvm@lists.linux.dev Content-Type: text/plain; charset="UTF-8" From: Ard Biesheuvel The early SME/SEV code parses the command line very early, in order to decide whether or not memory encryption should be enabled, which needs to occur even before the initial page tables are created. This is problematic for a number of reasons: - this early code runs from the 1:1 mapping provided by the decompressor or firmware, which uses a different translation than the one assumed by the linker, and so the code needs to be built in a special way; - parsing external input while the entire kernel image is still mapped writable is a bad idea in general, and really does not belong in security minded code; - the current code ignores the built-in command line entirely (although this appears to be the case for the entire decompressor) Given that the decompressor/EFI stub is an intrinsic part of the x86 bootable kernel image, move the command line parsing there and out of the core kernel. This removes the need to build lib/cmdline.o in a special way, or to use RIP-relative LEA instructions in inline asm blocks. This involves a new xloadflag in the setup header to indicate that mem_encrypt=on appeared on the kernel command line. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/misc.c | 15 +++++++++ arch/x86/include/uapi/asm/bootparam.h | 1 + arch/x86/lib/Makefile | 13 -------- arch/x86/mm/mem_encrypt_identity.c | 32 ++------------------ drivers/firmware/efi/libstub/x86-stub.c | 3 ++ 5 files changed, 22 insertions(+), 42 deletions(-) diff --git a/arch/x86/boot/compressed/misc.c b/arch/x86/boot/compressed/misc.c index b99e08e6815b..6c5c190a4d86 100644 --- a/arch/x86/boot/compressed/misc.c +++ b/arch/x86/boot/compressed/misc.c @@ -357,6 +357,19 @@ unsigned long decompress_kernel(unsigned char *outbuf, unsigned long virt_addr, return entry; } +/* + * Set the memory encryption xloadflag based on the mem_encrypt= command line + * parameter, if provided. + */ +static void parse_mem_encrypt(struct setup_header *hdr) +{ + int on = cmdline_find_option_bool("mem_encrypt=on"); + int off = cmdline_find_option_bool("mem_encrypt=off"); + + if (on > off) + hdr->xloadflags |= XLF_MEM_ENCRYPTION; +} + /* * The compressed kernel image (ZO), has been moved so that its position * is against the end of the buffer used to hold the uncompressed kernel @@ -387,6 +400,8 @@ asmlinkage __visible void *extract_kernel(void *rmode, unsigned char *output) /* Clear flags intended for solely in-kernel use. */ boot_params_ptr->hdr.loadflags &= ~KASLR_FLAG; + parse_mem_encrypt(&boot_params_ptr->hdr); + sanitize_boot_params(boot_params_ptr); if (boot_params_ptr->screen_info.orig_video_mode == 7) { diff --git a/arch/x86/include/uapi/asm/bootparam.h b/arch/x86/include/uapi/asm/bootparam.h index 01d19fc22346..eeea058cf602 100644 --- a/arch/x86/include/uapi/asm/bootparam.h +++ b/arch/x86/include/uapi/asm/bootparam.h @@ -38,6 +38,7 @@ #define XLF_EFI_KEXEC (1<<4) #define XLF_5LEVEL (1<<5) #define XLF_5LEVEL_ENABLED (1<<6) +#define XLF_MEM_ENCRYPTION (1<<7) #ifndef __ASSEMBLY__ diff --git a/arch/x86/lib/Makefile b/arch/x86/lib/Makefile index ea3a28e7b613..f0dae4fb6d07 100644 --- a/arch/x86/lib/Makefile +++ b/arch/x86/lib/Makefile @@ -14,19 +14,6 @@ ifdef CONFIG_KCSAN CFLAGS_REMOVE_delay.o = $(CC_FLAGS_FTRACE) endif -# Early boot use of cmdline; don't instrument it -ifdef CONFIG_AMD_MEM_ENCRYPT -KCOV_INSTRUMENT_cmdline.o := n -KASAN_SANITIZE_cmdline.o := n -KCSAN_SANITIZE_cmdline.o := n - -ifdef CONFIG_FUNCTION_TRACER -CFLAGS_REMOVE_cmdline.o = -pg -endif - -CFLAGS_cmdline.o := -fno-stack-protector -fno-jump-tables -endif - inat_tables_script = $(srctree)/arch/x86/tools/gen-insn-attr-x86.awk inat_tables_maps = $(srctree)/arch/x86/lib/x86-opcode-map.txt quiet_cmd_inat_tables = GEN $@ diff --git a/arch/x86/mm/mem_encrypt_identity.c b/arch/x86/mm/mem_encrypt_identity.c index 7ddcf960e92a..0180fbbcc940 100644 --- a/arch/x86/mm/mem_encrypt_identity.c +++ b/arch/x86/mm/mem_encrypt_identity.c @@ -43,7 +43,6 @@ #include #include -#include #include #include #include @@ -96,9 +95,6 @@ struct sme_populate_pgd_data { */ static char sme_workarea[2 * PMD_SIZE] __section(".init.scratch"); -static char sme_cmdline_arg[] __initdata = "mem_encrypt"; -static char sme_cmdline_on[] __initdata = "on"; - static void __init sme_clear_pgd(struct sme_populate_pgd_data *ppd) { unsigned long pgd_start, pgd_end, pgd_size; @@ -505,11 +501,9 @@ void __init sme_encrypt_kernel(struct boot_params *bp) void __head sme_enable(struct boot_params *bp) { - const char *cmdline_ptr, *cmdline_arg, *cmdline_on; unsigned int eax, ebx, ecx, edx; unsigned long feature_mask; unsigned long me_mask; - char buffer[16]; u64 msr; /* Check for the SME/SEV support leaf */ @@ -549,6 +543,9 @@ void __head sme_enable(struct boot_params *bp) /* Check if memory encryption is enabled */ if (feature_mask == AMD_SME_BIT) { + if (!(bp->hdr.xloadflags & XLF_MEM_ENCRYPTION)) + return; + /* * No SME if Hypervisor bit is set. This check is here to * prevent a guest from trying to enable SME. For running as a @@ -568,31 +565,8 @@ void __head sme_enable(struct boot_params *bp) msr = __rdmsr(MSR_AMD64_SYSCFG); if (!(msr & MSR_AMD64_SYSCFG_MEM_ENCRYPT)) return; - } else { - /* SEV state cannot be controlled by a command line option */ - goto out; } - /* - * Fixups have not been applied to phys_base yet and we're running - * identity mapped, so we must obtain the address to the SME command - * line argument data using rip-relative addressing. - */ - asm ("lea sme_cmdline_arg(%%rip), %0" - : "=r" (cmdline_arg) - : "p" (sme_cmdline_arg)); - asm ("lea sme_cmdline_on(%%rip), %0" - : "=r" (cmdline_on) - : "p" (sme_cmdline_on)); - - cmdline_ptr = (const char *)((u64)bp->hdr.cmd_line_ptr | - ((u64)bp->ext_cmd_line_ptr << 32)); - - if (cmdline_find_option(cmdline_ptr, cmdline_arg, buffer, sizeof(buffer)) < 0 || - strncmp(buffer, cmdline_on, sizeof(buffer))) - return; - -out: RIP_REL_REF(sme_me_mask) = me_mask; physical_mask &= ~me_mask; cc_vendor = CC_VENDOR_AMD; diff --git a/drivers/firmware/efi/libstub/x86-stub.c b/drivers/firmware/efi/libstub/x86-stub.c index 0d510c9a06a4..9a25ec16b344 100644 --- a/drivers/firmware/efi/libstub/x86-stub.c +++ b/drivers/firmware/efi/libstub/x86-stub.c @@ -879,6 +879,9 @@ void __noreturn efi_stub_entry(efi_handle_t handle, } } + if (efi_mem_encrypt > 0) + hdr->xloadflags |= XLF_MEM_ENCRYPTION; + status = efi_decompress_kernel(&kernel_entry); if (status != EFI_SUCCESS) { efi_err("Failed to decompress kernel\n"); -- 2.44.0.rc0.258.g7320e95886-goog