Received: by 2002:a25:e74b:0:0:0:0:0 with SMTP id e72csp1701807ybh; Tue, 14 Jul 2020 05:13:43 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyk3vhEToJ3vS1Gxy2kDw5SC6fJVZiY5XOeoPisAVpRLgQvEuTeImBKrlJqhM0ZL2IBHzfS X-Received: by 2002:a17:906:d217:: with SMTP id w23mr4346399ejz.292.1594728823653; Tue, 14 Jul 2020 05:13:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594728823; cv=none; d=google.com; s=arc-20160816; b=Ie29SFb70bLD/eWG96HKK4VxVtD0ZEIh/lNRI2lYEj6Zq7m4awmJE6tpS9YPwzOWsO 7LCW6oRZXcRqLJRzeFI03NxgQT/C+IvF8xDOH1Q9pPuPrMrU4HfUaTM3EToHV/y+a2uH LIeiUrE3EnZuMx/BlZ/hkhpdESYu8IrmFSgW9N7f/xV9VL2BQCPFPruXsJjbcVqjMNmb jlgIVpMjQswsIT7DoFp8o6SdZObuz8KaVEqB7jazIsAz2lNo0oNiQg/kGVy3AMsVyACa ZdJakyQRRyrYfnJiNBz5fOeEqVzElG1PpfhsgHFhViCAjMZ3htrgiu78ZsBXx37qJLpE 125Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=biEI0jYRCS60CKRsoRwC2bTeDTcc0M3EErEql7LlDHk=; b=SygSsK06YDGJwU1SyTMVH8Yz6tpxPL85JCO6a8nyMae2bdXOmHU+0+ITTz3pq4gDcv wbQNc+DxCFm/ZWb+apXNuhFGRmyMXkeNpKEErFvFEv81zrftf31vXFJ0ardKEIE9JIzh ljWGxbf4f1Vvd8GyTEy3FZa3A89LyP1CRXg/PpuXarY/13AllMvRgDNbAFdAZxh2S9Dh 882tSVQFUFM1nSs8A6pvLAh/b/ErjZMHXGQDN4jwBa7i6j7nFuvQnKgMNC3SaDbCKfHw 2nQMivvmX18dwnL6MYJUfHicOQCddJcRcmGbGB/JMPokH9dx3FN2CVspZZuPNXTkWXLm ejNg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=8bytes.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id j17si11352533eds.349.2020.07.14.05.13.19; Tue, 14 Jul 2020 05:13:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=8bytes.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728767AbgGNMMR (ORCPT + 99 others); Tue, 14 Jul 2020 08:12:17 -0400 Received: from 8bytes.org ([81.169.241.247]:52886 "EHLO theia.8bytes.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728576AbgGNMLT (ORCPT ); Tue, 14 Jul 2020 08:11:19 -0400 Received: from cap.home.8bytes.org (p5b006776.dip0.t-ipconnect.de [91.0.103.118]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by theia.8bytes.org (Postfix) with ESMTPSA id A05E9FE9; Tue, 14 Jul 2020 14:11:10 +0200 (CEST) From: Joerg Roedel To: x86@kernel.org Cc: Joerg Roedel , Joerg Roedel , Tom Lendacky , hpa@zytor.com, Andy Lutomirski , Dave Hansen , Peter Zijlstra , Jiri Slaby , Dan Williams , Juergen Gross , Kees Cook , David Rientjes , Cfir Cohen , Erdem Aktas , Masami Hiramatsu , Mike Stunes , Sean Christopherson , Martin Radev , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, virtualization@lists.linux-foundation.org Subject: [PATCH v4 68/75] x86/realmode: Setup AP jump table Date: Tue, 14 Jul 2020 14:09:10 +0200 Message-Id: <20200714120917.11253-69-joro@8bytes.org> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200714120917.11253-1-joro@8bytes.org> References: <20200714120917.11253-1-joro@8bytes.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Tom Lendacky As part of the GHCB specification, the booting of APs under SEV-ES requires an AP jump table when transitioning from one layer of code to another (e.g. when going from UEFI to the OS). As a result, each layer that parks an AP must provide the physical address of an AP jump table to the next layer via the hypervisor. Upon booting of the kernel, read the AP jump table address from the hypervisor. Under SEV-ES, APs are started using the INIT-SIPI-SIPI sequence. Before issuing the first SIPI request for an AP, the start CS and IP is programmed into the AP jump table. Upon issuing the SIPI request, the AP will awaken and jump to that start CS:IP address. Signed-off-by: Tom Lendacky [ jroedel@suse.de: - Adapted to different code base - Moved AP table setup from SIPI sending path to real-mode setup code - Fix sparse warnings ] Co-developed-by: Joerg Roedel Signed-off-by: Joerg Roedel --- arch/x86/include/asm/sev-es.h | 5 +++ arch/x86/include/uapi/asm/svm.h | 3 ++ arch/x86/kernel/sev-es.c | 68 +++++++++++++++++++++++++++++++++ arch/x86/realmode/init.c | 18 ++++++++- 4 files changed, 92 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/sev-es.h b/arch/x86/include/asm/sev-es.h index 330140a189be..25ab44f977db 100644 --- a/arch/x86/include/asm/sev-es.h +++ b/arch/x86/include/asm/sev-es.h @@ -73,6 +73,9 @@ static inline u64 lower_bits(u64 val, unsigned int bits) return (val & mask); } +struct real_mode_header; +enum stack_type; + /* Early IDT entry points for #VC handler */ extern void vc_no_ghcb(void); extern bool handle_vc_boot_ghcb(struct pt_regs *regs); @@ -80,9 +83,11 @@ extern bool handle_vc_boot_ghcb(struct pt_regs *regs); #ifdef CONFIG_AMD_MEM_ENCRYPT extern void sev_es_ist_enter(struct pt_regs *regs); extern void sev_es_ist_exit(void); +extern int sev_es_setup_ap_jump_table(struct real_mode_header *rmh); #else static inline void sev_es_ist_enter(struct pt_regs *regs) { } static inline void sev_es_ist_exit(void) { } +static inline int sev_es_setup_ap_jump_table(struct real_mode_header *rmh) { return 0; } #endif #endif diff --git a/arch/x86/include/uapi/asm/svm.h b/arch/x86/include/uapi/asm/svm.h index 8f36ae021a7f..a19ce9681ec2 100644 --- a/arch/x86/include/uapi/asm/svm.h +++ b/arch/x86/include/uapi/asm/svm.h @@ -84,6 +84,9 @@ /* SEV-ES software-defined VMGEXIT events */ #define SVM_VMGEXIT_MMIO_READ 0x80000001 #define SVM_VMGEXIT_MMIO_WRITE 0x80000002 +#define SVM_VMGEXIT_AP_JUMP_TABLE 0x80000005 +#define SVM_VMGEXIT_SET_AP_JUMP_TABLE 0 +#define SVM_VMGEXIT_GET_AP_JUMP_TABLE 1 #define SVM_VMGEXIT_UNSUPPORTED_EVENT 0x8000ffff #define SVM_EXIT_ERR -1 diff --git a/arch/x86/kernel/sev-es.c b/arch/x86/kernel/sev-es.c index 76104c71fc85..370cf9bf2c88 100644 --- a/arch/x86/kernel/sev-es.c +++ b/arch/x86/kernel/sev-es.c @@ -21,6 +21,8 @@ #include #include +#include +#include #include #include #include @@ -229,6 +231,9 @@ static __always_inline void sev_es_put_ghcb(struct ghcb_state *state) } } +/* Needed in vc_early_vc_forward_exception */ +void do_early_exception(struct pt_regs *regs, int trapnr); + static inline u64 sev_es_rd_ghcb_msr(void) { return native_read_msr(MSR_AMD64_SEV_ES_GHCB); @@ -416,6 +421,69 @@ static bool vc_slow_virt_to_phys(struct ghcb *ghcb, struct es_em_ctxt *ctxt, /* Include code shared with pre-decompression boot stage */ #include "sev-es-shared.c" +static u64 sev_es_get_jump_table_addr(void) +{ + struct ghcb_state state; + unsigned long flags; + struct ghcb *ghcb; + u64 ret; + + local_irq_save(flags); + + ghcb = sev_es_get_ghcb(&state); + + vc_ghcb_invalidate(ghcb); + ghcb_set_sw_exit_code(ghcb, SVM_VMGEXIT_AP_JUMP_TABLE); + ghcb_set_sw_exit_info_1(ghcb, SVM_VMGEXIT_GET_AP_JUMP_TABLE); + ghcb_set_sw_exit_info_2(ghcb, 0); + + sev_es_wr_ghcb_msr(__pa(ghcb)); + VMGEXIT(); + + if (!ghcb_is_valid_sw_exit_info_1(ghcb) || + !ghcb_is_valid_sw_exit_info_2(ghcb)) + ret = 0; + + ret = ghcb->save.sw_exit_info_2; + + sev_es_put_ghcb(&state); + + local_irq_restore(flags); + + return ret; +} + +int sev_es_setup_ap_jump_table(struct real_mode_header *rmh) +{ + u16 startup_cs, startup_ip; + phys_addr_t jump_table_pa; + u64 jump_table_addr; + u16 __iomem *jump_table; + + jump_table_addr = sev_es_get_jump_table_addr(); + + /* Check if AP Jump Table is non-zero and page-aligned */ + if (!jump_table_addr || jump_table_addr & ~PAGE_MASK) + return 0; + + jump_table_pa = jump_table_addr & PAGE_MASK; + + startup_cs = (u16)(rmh->trampoline_start >> 4); + startup_ip = (u16)(rmh->sev_es_trampoline_start - + rmh->trampoline_start); + + jump_table = ioremap_encrypted(jump_table_pa, PAGE_SIZE); + if (!jump_table) + return -EIO; + + writew(startup_ip, &jump_table[0]); + writew(startup_cs, &jump_table[1]); + + iounmap(jump_table); + + return 0; +} + static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt) { struct pt_regs *regs = ctxt->regs; diff --git a/arch/x86/realmode/init.c b/arch/x86/realmode/init.c index 1ed1208931e0..61a52b925d15 100644 --- a/arch/x86/realmode/init.c +++ b/arch/x86/realmode/init.c @@ -9,6 +9,7 @@ #include #include #include +#include struct real_mode_header *real_mode_header; u32 *trampoline_cr4_features; @@ -38,6 +39,19 @@ void __init reserve_real_mode(void) crash_reserve_low_1M(); } +static void sme_sev_setup_real_mode(struct trampoline_header *th) +{ +#ifdef CONFIG_AMD_MEM_ENCRYPT + if (sme_active()) + th->flags |= TH_FLAGS_SME_ACTIVE; + + if (sev_es_active()) { + if (sev_es_setup_ap_jump_table(real_mode_header)) + panic("Failed to update SEV-ES AP Jump Table"); + } +#endif +} + static void __init setup_real_mode(void) { u16 real_mode_seg; @@ -104,13 +118,13 @@ static void __init setup_real_mode(void) *trampoline_cr4_features = mmu_cr4_features; trampoline_header->flags = 0; - if (sme_active()) - trampoline_header->flags |= TH_FLAGS_SME_ACTIVE; trampoline_pgd = (u64 *) __va(real_mode_header->trampoline_pgd); trampoline_pgd[0] = trampoline_pgd_entry.pgd; trampoline_pgd[511] = init_top_pgt[511].pgd; #endif + + sme_sev_setup_real_mode(trampoline_header); } /* -- 2.27.0