Received: by 2002:a05:6359:c8b:b0:c7:702f:21d4 with SMTP id go11csp1329014rwb; Tue, 27 Sep 2022 11:25:12 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7ln6jAyNTgOgoF+xyjzjWtU8Wqs/w8nBXJSIrWJCMPwB+GFK7EdAM5CcRFr5aRfSG+MnlO X-Received: by 2002:a17:902:e847:b0:179:f707:2f18 with SMTP id t7-20020a170902e84700b00179f7072f18mr2498893plg.18.1664303112289; Tue, 27 Sep 2022 11:25:12 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1664303112; cv=pass; d=google.com; s=arc-20160816; b=TroxCz5ZC7mIMajAml9Zb4zLUcYySP3L35tfzPiKFW8GahzPKO19DLt7k2w09Hfgug WHLzCnBRlvuM/W0kPFb/d4G+hM1maGo9ugv4ieSI3LkYLeaBH6GUV7GcNQR7QJkHkPYN DeU4/JNJgJKIzzRIbJ7Dl4SgxhD2JCMAy89knu7jtXGrZbPslllxQwE19eMgXXyZ5EhK OoGWIPKm+WiUZlYaQLTbjHQtQY+3P1bCc2n9E7KkOV7xaYPv1b4KrLSqZjpcPS2+0rbC 7oEsg2C08boiILpoTqb3IBKrNJbpkzWbGWGRGYcmO4/3wlmlCSp6gqoJH2w3k7H/0Wdj USZg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=PG+74JEGHuI7t+JgAlWrAixxM9d/8RP4B2qZ1NqkSz4=; b=JqDDgL3TEs30JpHlk+7ta1TqOvMdIV0Rc3m/amUHSm7jRPDsyFW6bqmuzy/g4oMLvF P/3zL6WVp7sZc8SMul36u5ZkinGhQ+ZystWoHGCXpEfeE9X4mesQhCBHtcgijXtsNIwD iWn5uzmduLTS8xlnXS7Pkoh/Jc1mbsLi22uuzGWW2Idf4G1Umb3/SIXVIjsB0hw3QmVN 4/5O1JmbvFPueiAQJVmZg4eDxkp7rSEN5JkXiCnA2SqSjOV8k7/RpZVsNRfuPREGP/y+ aBJCUyvMOk0uzV7xg1K+tv0tnwIR1MEIF5/sGGdoQOhL9gj6Q879jg0j2J1OZFOInDq8 aagw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=iQk4RTqF; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u7-20020a17090341c700b001755d722578si2870898ple.524.2022.09.27.11.25.00; Tue, 27 Sep 2022 11:25:12 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=iQk4RTqF; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231934AbiI0RGL (ORCPT + 99 others); Tue, 27 Sep 2022 13:06:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59582 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232207AbiI0RFi (ORCPT ); Tue, 27 Sep 2022 13:05:38 -0400 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2061a.outbound.protection.outlook.com [IPv6:2a01:111:f400:7e89::61a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 131B660507 for ; Tue, 27 Sep 2022 10:05:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fMmtNq2eACTfi899dXiQtHIVRjOIjHCrMbTzhSXNxp7vlm4K1YtOmxTZsfQjl222IKGhpHYaN/qNUYb1YNa44YCTNN7qst3IexrVkD6M7b5u1MLoJO3M8mijBQzx6rgl9fnSFiBBR9hlM0bjW3eHU+a5p196oSHbBMryC110JcDTDncsqpj4fR8XkConxBl8hu+ytbtI5/JxSwUrUXrl1PlB3v98nDBi/lXNSCdH7Xz7kJli/FwOQKV3f9T0LPxK8EHmMkoGs7DoElj5d5ZotYdLi/nqgI4njlTneFikpTwnGBgftzkcjIsS1wxyoaTwZoSBzycc1985IirnmfIBnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PG+74JEGHuI7t+JgAlWrAixxM9d/8RP4B2qZ1NqkSz4=; b=BmizuB11aaN/84Zdc5rXvVZF/QYgXadzFUX/8MFpYhH+KU60y7gloSLvXVjkanp9OGj0DJScyfMBbO6GLiQmTbMgm2STUbwm2ntdDe7WbgTV9RO6NnSQkQ3ArwgLB37taegZEtq3/VOSqm4t/SWda4sNK/wlYVsvRtWDwCzZyaAnoZpy1Vhoyq2T0f/NaU8wZD/bVd68amwobzR1mNcIB4B25R4Td8iLvmrWgC0rFbUEO2kkHyXKJNMxhO1ql0NeLQ/80z5URgdno9BXDboCS7ho74O4V0i1yuPhD4vyRb4NOJpYNRd2NOW/WjJxWOq11TdjPZAGrAXh9mYK9d+sqw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PG+74JEGHuI7t+JgAlWrAixxM9d/8RP4B2qZ1NqkSz4=; b=iQk4RTqFvgViH7ibixgtHXxYugkTUuMyLj//hD+XNmIY8duBUhA8USiVlxdtFeltQ49JwZPNa+dx7afiAPGba49ThH1PB4zf2kkWqeQJ8GSRGL9CmUEnnTLzttDmIpF9NsfAzTU92cIsO1hf9w+lzlORwqrspvv2ySQB9vECkpY= Received: from BN9PR03CA0230.namprd03.prod.outlook.com (2603:10b6:408:f8::25) by MW4PR12MB7287.namprd12.prod.outlook.com (2603:10b6:303:22c::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.16; Tue, 27 Sep 2022 17:05:21 +0000 Received: from BN8NAM11FT071.eop-nam11.prod.protection.outlook.com (2603:10b6:408:f8:cafe::a8) by BN9PR03CA0230.outlook.office365.com (2603:10b6:408:f8::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5654.23 via Frontend Transport; Tue, 27 Sep 2022 17:05:21 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT071.mail.protection.outlook.com (10.13.177.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5654.14 via Frontend Transport; Tue, 27 Sep 2022 17:05:21 +0000 Received: from tlendack-t1.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Tue, 27 Sep 2022 12:05:20 -0500 From: Tom Lendacky To: , CC: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "Kirill A. Shutemov" , "H. Peter Anvin" , Michael Roth , Joerg Roedel , Andy Lutomirski , Peter Zijlstra Subject: [PATCH v5 6/6] x86/sev: Add SNP-specific unaccepted memory support Date: Tue, 27 Sep 2022 12:04:21 -0500 Message-ID: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: <20220614120231.48165-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT071:EE_|MW4PR12MB7287:EE_ X-MS-Office365-Filtering-Correlation-Id: 192c1abb-dd42-419c-b34d-08daa0aa765b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: oy4I7HWqCIg4F+xnefPKEYyjQ40ieCFXX3gqhid0Br4u4P8ESThbRQlyDoDrsXwFG5veaO3ILWs2iq3OReQGfT8tdJh/9BcjgbFmH5mEsptkwARZN/5OqiKe0vvzPYkAUzocI8Ca8JSkS69kwFrtZ5xVWBvQ2ZkWrWGTp35F+z3N8spUVLU1UjFjpcyK9CulyPK5VH+qSF7ik/naaIexKc7bUKCj59RZTpc/Da1kghTqZZxp3qw5ZTaFTyFyOAqZekH3LHBxeAzCHiH/PDkWBpHvepqjNuN5I7+BpVyqvWwJdnZr4l3pDUsT2K4zrPO9uk3Pa4mwYyx6jlR84jdUVBOAyL25bwHTnBMVE1iy/N//LYV7ZYApR2pJdHl3krQBqzWZK0IJjwZKNJ506RVQp+wrQRVwzAs5q4iZZIQFf0cwKpwiSTygxZhNfxr85r456OQ8nebVRvm7fJaFUd+MRadLiXy+tOMj9YpVw/1yZUcbYv32tfAvElpPI5riDGRYsvzaB81f7Zl3Nae8OHdQaRL2eJEvl2BV6/YRSgZuJwECEKvc9uIW5v0P0s7dbFmqY1Qf2ypAVMo3n6fHK3RF9vTZ/QP2Lu1XfnQ/qqr3FhTE9I999XwSPEjj+n4N00DAFel2G07bd5KEXs1YYnlDgh7LkriyGfm+udB6R/i9HnT8gvxB0/OuM2XfTh53FkB/smQiPJOBJbK9EK2f1FJwu/EZygcWvBrmuFKCAylOYizSU/HBqm9rx8YtLpAqsXDwEs6O/W0QjHm5+qVfkBXIcemAUIiaw79ABjvxukE3/6YmjGtEpjP88j2i26QbqOA0 X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(39860400002)(396003)(346002)(136003)(376002)(451199015)(36840700001)(40470700004)(46966006)(356005)(36860700001)(82740400003)(5660300002)(30864003)(40480700001)(83380400001)(81166007)(82310400005)(41300700001)(16526019)(47076005)(40460700003)(36756003)(2906002)(8936002)(26005)(7416002)(7696005)(70206006)(70586007)(336012)(8676002)(2616005)(426003)(186003)(4326008)(54906003)(316002)(86362001)(110136005)(478600001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2022 17:05:21.6327 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 192c1abb-dd42-419c-b34d-08daa0aa765b X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT071.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB7287 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_PASS,SPF_PASS, T_FILL_THIS_FORM_SHORT autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Add SNP-specific hooks to the unaccepted memory support in the boot path (__accept_memory()) and the core kernel (accept_memory()) in order to support booting SNP guests when unaccepted memory is present. Without this support, SNP guests will fail to boot and/or panic() when unaccepted memory is present in the EFI memory map. The process of accepting memory under SNP involves invoking the hypervisor to perform a page state change for the page to private memory and then issuing a PVALIDATE instruction to accept the page. Since the boot path and the core kernel paths perform similar operations, move the pvalidate_pages() and vmgexit_psc() functions into sev-shared.c to avoid code duplication. Create the new header file arch/x86/boot/compressed/sev.h because adding the function declaration to any of the existing SEV related header files pulls in too many other header files, causing the build to fail. Signed-off-by: Tom Lendacky --- arch/x86/Kconfig | 1 + arch/x86/boot/compressed/mem.c | 3 + arch/x86/boot/compressed/sev.c | 54 ++++++++++++++- arch/x86/boot/compressed/sev.h | 23 +++++++ arch/x86/include/asm/sev.h | 3 + arch/x86/kernel/sev-shared.c | 104 +++++++++++++++++++++++++++++ arch/x86/kernel/sev.c | 112 ++++---------------------------- arch/x86/mm/unaccepted_memory.c | 4 ++ 8 files changed, 205 insertions(+), 99 deletions(-) create mode 100644 arch/x86/boot/compressed/sev.h diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 34146ecc5bdd..0ad53c3533c2 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -1553,6 +1553,7 @@ config AMD_MEM_ENCRYPT select INSTRUCTION_DECODER select ARCH_HAS_CC_PLATFORM select X86_MEM_ENCRYPT + select UNACCEPTED_MEMORY help Say yes to enable support for the encryption of system memory. This requires an AMD processor that supports Secure Memory diff --git a/arch/x86/boot/compressed/mem.c b/arch/x86/boot/compressed/mem.c index 48e36e640da1..3e19dc0da0d7 100644 --- a/arch/x86/boot/compressed/mem.c +++ b/arch/x86/boot/compressed/mem.c @@ -6,6 +6,7 @@ #include "find.h" #include "math.h" #include "tdx.h" +#include "sev.h" #include #define PMD_SHIFT 21 @@ -39,6 +40,8 @@ static inline void __accept_memory(phys_addr_t start, phys_addr_t end) /* Platform-specific memory-acceptance call goes here */ if (is_tdx_guest()) tdx_accept_memory(start, end); + else if (sev_snp_enabled()) + snp_accept_memory(start, end); else error("Cannot accept memory: unknown platform\n"); } diff --git a/arch/x86/boot/compressed/sev.c b/arch/x86/boot/compressed/sev.c index 730c4677e9db..22da65c96b47 100644 --- a/arch/x86/boot/compressed/sev.c +++ b/arch/x86/boot/compressed/sev.c @@ -115,7 +115,7 @@ static enum es_result vc_read_mem(struct es_em_ctxt *ctxt, /* Include code for early handlers */ #include "../../kernel/sev-shared.c" -static inline bool sev_snp_enabled(void) +bool sev_snp_enabled(void) { return sev_status & MSR_AMD64_SEV_SNP_ENABLED; } @@ -181,6 +181,58 @@ static bool early_setup_ghcb(void) return true; } +static phys_addr_t __snp_accept_memory(struct snp_psc_desc *desc, + phys_addr_t pa, phys_addr_t pa_end) +{ + struct psc_hdr *hdr; + struct psc_entry *e; + unsigned int i; + + hdr = &desc->hdr; + memset(hdr, 0, sizeof(*hdr)); + + e = desc->entries; + + i = 0; + while (pa < pa_end && i < VMGEXIT_PSC_MAX_ENTRY) { + hdr->end_entry = i; + + e->gfn = pa >> PAGE_SHIFT; + e->operation = SNP_PAGE_STATE_PRIVATE; + if (IS_ALIGNED(pa, PMD_PAGE_SIZE) && (pa_end - pa) >= PMD_PAGE_SIZE) { + e->pagesize = RMP_PG_SIZE_2M; + pa += PMD_PAGE_SIZE; + } else { + e->pagesize = RMP_PG_SIZE_4K; + pa += PAGE_SIZE; + } + + e++; + i++; + } + + if (vmgexit_psc(boot_ghcb, desc)) + sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PSC); + + pvalidate_pages(desc); + + return pa; +} + +void snp_accept_memory(phys_addr_t start, phys_addr_t end) +{ + struct snp_psc_desc desc = {}; + unsigned int i; + phys_addr_t pa; + + if (!boot_ghcb && !early_setup_ghcb()) + sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PSC); + + pa = start; + while (pa < end) + pa = __snp_accept_memory(&desc, pa, end); +} + void sev_es_shutdown_ghcb(void) { if (!boot_ghcb) diff --git a/arch/x86/boot/compressed/sev.h b/arch/x86/boot/compressed/sev.h new file mode 100644 index 000000000000..fc725a981b09 --- /dev/null +++ b/arch/x86/boot/compressed/sev.h @@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * AMD SEV header for early boot related functions. + * + * Author: Tom Lendacky + */ + +#ifndef BOOT_COMPRESSED_SEV_H +#define BOOT_COMPRESSED_SEV_H + +#ifdef CONFIG_AMD_MEM_ENCRYPT + +bool sev_snp_enabled(void); +void snp_accept_memory(phys_addr_t start, phys_addr_t end); + +#else + +static inline bool sev_snp_enabled(void) { return false; } +static inline void snp_accept_memory(phys_addr_t start, phys_addr_t end) { } + +#endif + +#endif diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h index 0007ab04ac5f..9297aab0c79e 100644 --- a/arch/x86/include/asm/sev.h +++ b/arch/x86/include/asm/sev.h @@ -206,6 +206,7 @@ void snp_set_wakeup_secondary_cpu(void); bool snp_init(struct boot_params *bp); void snp_abort(void); int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, unsigned long *fw_err); +void snp_accept_memory(phys_addr_t start, phys_addr_t end); #else static inline void sev_es_ist_enter(struct pt_regs *regs) { } static inline void sev_es_ist_exit(void) { } @@ -230,6 +231,8 @@ static inline int snp_issue_guest_request(u64 exit_code, struct snp_req_data *in { return -ENOTTY; } + +static inline void snp_accept_memory(phys_addr_t start, phys_addr_t end) { } #endif #endif diff --git a/arch/x86/kernel/sev-shared.c b/arch/x86/kernel/sev-shared.c index b478edf43bec..7ac7857da2b8 100644 --- a/arch/x86/kernel/sev-shared.c +++ b/arch/x86/kernel/sev-shared.c @@ -12,6 +12,9 @@ #ifndef __BOOT_COMPRESSED #define error(v) pr_err(v) #define has_cpuflag(f) boot_cpu_has(f) +#else +#undef WARN +#define WARN(condition...) #endif /* I/O parameters for CPUID-related helpers */ @@ -998,3 +1001,104 @@ static void __init setup_cpuid_table(const struct cc_blob_sev_info *cc_info) cpuid_ext_range_max = fn->eax; } } + +static void pvalidate_pages(struct snp_psc_desc *desc) +{ + struct psc_entry *e; + unsigned long vaddr; + unsigned int size; + unsigned int i; + bool validate; + int rc; + + for (i = 0; i <= desc->hdr.end_entry; i++) { + e = &desc->entries[i]; + + vaddr = (unsigned long)pfn_to_kaddr(e->gfn); + size = e->pagesize ? RMP_PG_SIZE_2M : RMP_PG_SIZE_4K; + validate = (e->operation == SNP_PAGE_STATE_PRIVATE) ? true : false; + + rc = pvalidate(vaddr, size, validate); + if (rc == PVALIDATE_FAIL_SIZEMISMATCH && size == RMP_PG_SIZE_2M) { + unsigned long vaddr_end = vaddr + PMD_PAGE_SIZE; + + for (; vaddr < vaddr_end; vaddr += PAGE_SIZE) { + rc = pvalidate(vaddr, RMP_PG_SIZE_4K, validate); + if (rc) + break; + } + } + + if (rc) { + WARN(1, "Failed to validate address 0x%lx ret %d", vaddr, rc); + sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PVALIDATE); + } + } +} + +static int vmgexit_psc(struct ghcb *ghcb, struct snp_psc_desc *desc) +{ + int cur_entry, end_entry, ret = 0; + struct snp_psc_desc *data; + struct es_em_ctxt ctxt; + + vc_ghcb_invalidate(ghcb); + + /* Copy the input desc into GHCB shared buffer */ + data = (struct snp_psc_desc *)ghcb->shared_buffer; + memcpy(ghcb->shared_buffer, desc, min_t(int, GHCB_SHARED_BUF_SIZE, sizeof(*desc))); + + /* + * As per the GHCB specification, the hypervisor can resume the guest + * before processing all the entries. Check whether all the entries + * are processed. If not, then keep retrying. Note, the hypervisor + * will update the data memory directly to indicate the status, so + * reference the data->hdr everywhere. + * + * The strategy here is to wait for the hypervisor to change the page + * state in the RMP table before guest accesses the memory pages. If the + * page state change was not successful, then later memory access will + * result in a crash. + */ + cur_entry = data->hdr.cur_entry; + end_entry = data->hdr.end_entry; + + while (data->hdr.cur_entry <= data->hdr.end_entry) { + ghcb_set_sw_scratch(ghcb, (u64)__pa(data)); + + /* This will advance the shared buffer data points to. */ + ret = sev_es_ghcb_hv_call(ghcb, true, &ctxt, SVM_VMGEXIT_PSC, 0, 0); + + /* + * Page State Change VMGEXIT can pass error code through + * exit_info_2. + */ + if (ret || ghcb->save.sw_exit_info_2) { + WARN(1, "SNP: PSC failed ret=%d exit_info_2=%llx\n", + ret, ghcb->save.sw_exit_info_2); + ret = 1; + goto out; + } + + /* Verify that reserved bit is not set */ + if (data->hdr.reserved) { + WARN(1, "Reserved bit is set in the PSC header\n"); + ret = 1; + goto out; + } + + /* + * Sanity check that entry processing is not going backwards. + * This will happen only if hypervisor is tricking us. + */ + if (data->hdr.end_entry > end_entry || cur_entry > data->hdr.cur_entry) { + WARN(1, "SNP: PSC processing going backward, end_entry %d (got %d) cur_entry %d (got %d)\n", + end_entry, data->hdr.end_entry, cur_entry, data->hdr.cur_entry); + ret = 1; + goto out; + } + } + +out: + return ret; +} diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index eabb8dd5be5b..48440933bde2 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -655,38 +655,6 @@ static u64 __init get_jump_table_addr(void) return ret; } -static void pvalidate_pages(struct snp_psc_desc *desc) -{ - struct psc_entry *e; - unsigned long vaddr; - unsigned int size; - unsigned int i; - bool validate; - int rc; - - for (i = 0; i <= desc->hdr.end_entry; i++) { - e = &desc->entries[i]; - - vaddr = (unsigned long)pfn_to_kaddr(e->gfn); - size = e->pagesize ? RMP_PG_SIZE_2M : RMP_PG_SIZE_4K; - validate = (e->operation == SNP_PAGE_STATE_PRIVATE) ? true : false; - - rc = pvalidate(vaddr, size, validate); - if (rc == PVALIDATE_FAIL_SIZEMISMATCH && size == RMP_PG_SIZE_2M) { - unsigned long vaddr_end = vaddr + PMD_PAGE_SIZE; - - for (; vaddr < vaddr_end; vaddr += PAGE_SIZE) { - rc = pvalidate(vaddr, RMP_PG_SIZE_4K, validate); - if (rc) - break; - } - } - - if (WARN(rc, "Failed to validate address 0x%lx ret %d", vaddr, rc)) - sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PVALIDATE); - } -} - static void early_set_pages_state(unsigned long vaddr, unsigned long paddr, unsigned int npages, enum psc_op op) { @@ -782,72 +750,6 @@ void __init snp_prep_memory(unsigned long paddr, unsigned int sz, enum psc_op op WARN(1, "invalid memory op %d\n", op); } -static int vmgexit_psc(struct ghcb *ghcb, struct snp_psc_desc *desc) -{ - int cur_entry, end_entry, ret = 0; - struct snp_psc_desc *data; - struct es_em_ctxt ctxt; - - vc_ghcb_invalidate(ghcb); - - /* Copy the input desc into GHCB shared buffer */ - data = (struct snp_psc_desc *)ghcb->shared_buffer; - memcpy(ghcb->shared_buffer, desc, min_t(int, GHCB_SHARED_BUF_SIZE, sizeof(*desc))); - - /* - * As per the GHCB specification, the hypervisor can resume the guest - * before processing all the entries. Check whether all the entries - * are processed. If not, then keep retrying. Note, the hypervisor - * will update the data memory directly to indicate the status, so - * reference the data->hdr everywhere. - * - * The strategy here is to wait for the hypervisor to change the page - * state in the RMP table before guest accesses the memory pages. If the - * page state change was not successful, then later memory access will - * result in a crash. - */ - cur_entry = data->hdr.cur_entry; - end_entry = data->hdr.end_entry; - - while (data->hdr.cur_entry <= data->hdr.end_entry) { - ghcb_set_sw_scratch(ghcb, (u64)__pa(data)); - - /* This will advance the shared buffer data points to. */ - ret = sev_es_ghcb_hv_call(ghcb, true, &ctxt, SVM_VMGEXIT_PSC, 0, 0); - - /* - * Page State Change VMGEXIT can pass error code through - * exit_info_2. - */ - if (WARN(ret || ghcb->save.sw_exit_info_2, - "SNP: PSC failed ret=%d exit_info_2=%llx\n", - ret, ghcb->save.sw_exit_info_2)) { - ret = 1; - goto out; - } - - /* Verify that reserved bit is not set */ - if (WARN(data->hdr.reserved, "Reserved bit is set in the PSC header\n")) { - ret = 1; - goto out; - } - - /* - * Sanity check that entry processing is not going backwards. - * This will happen only if hypervisor is tricking us. - */ - if (WARN(data->hdr.end_entry > end_entry || cur_entry > data->hdr.cur_entry, -"SNP: PSC processing going backward, end_entry %d (got %d) cur_entry %d (got %d)\n", - end_entry, data->hdr.end_entry, cur_entry, data->hdr.cur_entry)) { - ret = 1; - goto out; - } - } - -out: - return ret; -} - static unsigned long __set_pages_state(struct snp_psc_desc *data, unsigned long vaddr, unsigned long vaddr_end, int op) { @@ -952,6 +854,20 @@ void snp_set_memory_private(unsigned long vaddr, unsigned int npages) set_pages_state(vaddr, npages, SNP_PAGE_STATE_PRIVATE); } +void snp_accept_memory(phys_addr_t start, phys_addr_t end) +{ + unsigned long vaddr; + unsigned int npages; + + if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) + return; + + vaddr = (unsigned long)__va(start); + npages = (end - start) >> PAGE_SHIFT; + + set_pages_state(vaddr, npages, SNP_PAGE_STATE_PRIVATE); +} + static int snp_set_vmsa(void *va, bool vmsa) { u64 attrs; diff --git a/arch/x86/mm/unaccepted_memory.c b/arch/x86/mm/unaccepted_memory.c index 9ec2304272dc..b86ad6a8ddf5 100644 --- a/arch/x86/mm/unaccepted_memory.c +++ b/arch/x86/mm/unaccepted_memory.c @@ -9,6 +9,7 @@ #include #include #include +#include /* Protects unaccepted memory bitmap */ static DEFINE_SPINLOCK(unaccepted_memory_lock); @@ -66,6 +67,9 @@ void accept_memory(phys_addr_t start, phys_addr_t end) if (cpu_feature_enabled(X86_FEATURE_TDX_GUEST)) { tdx_accept_memory(range_start * PMD_SIZE, range_end * PMD_SIZE); + } else if (cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) { + snp_accept_memory(range_start * PMD_SIZE, + range_end * PMD_SIZE); } else { panic("Cannot accept memory: unknown platform\n"); } -- 2.37.3