X-Received: by 2002:a17:902:930b:b0:14d:b0c0:1f71 with SMTP id bc11-20020a170902930b00b0014db0c01f71mr818692plb.113.1645640357095; Wed, 23 Feb 2022 10:19:17 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1645640357; cv=none; d=google.com; s=arc-20160816; b=zvT+RyT4GTAqR3e+vFh+cFC67a2ku+IdoOyS7aQFAhWV17Bzi6Tf95ceQVABeWndXN TW/NOiKWV+wYnbwj6lGU0hhJltCAyt/0CbjWV7SCwNB2qq4/Ppjb2ept6ohMNZvyradK uQrEAee32Ye9zIrrTFISTa/8gNVxaExNjj1Jq4IepCJwPlXjKYTChG6i4f461VPdUc6e b4Mxh/QwII5CTWELzSefEKGDuBodKOeVioT/hcNyw3sblYqP5QX1OG7HydOl4R8ekgL2 KIWyR2o4mR64TbdlmuD/DiUVsemAxUwxcp9kPmhuqxvlvrCwII/a5Al9m3cugvmwjC2D UbiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=CLZk7ee4rtdycG1tAcBRvIEaodSetX5UhW72UlpjIhM=; b=pKNbo8M9W2lXwvcxo81aFXRvu58IKsJTbe1uJ3J2EqhI4Gr16YkADVj//He1b/wCNy LlR9zpUkQM/6uOpagO0qAGIq4TMXPDKg1Dcnc2uz/spG7+FEbyaKKdJgp37vQr/JH5zY wwkRxX5ShT/g1Ic4S2yxuAkNOGCnJlYPVq7IHdEjo97Mr2Lizj3yird7WTpTKXG27D5x HJqn7n5ZanebabWFZQqszsWxIIR/hCRTLJHK1qJUT7E5LHM+1hZX/g9tvE1Q232mkmMU oPzIPem3qm5wbW/wGHrz0/FORSy7GnLNlOHtv5lrP9vY1CaaiudjeW9fylWzk1leJ5zY B+rw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=CyqtsY+K; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w24si331454pgi.85.2022.02.23.10.19.00; Wed, 23 Feb 2022 10:19:17 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=CyqtsY+K; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234967AbiBVS6G (ORCPT + 99 others); Tue, 22 Feb 2022 13:58:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38126 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234867AbiBVS6F (ORCPT ); Tue, 22 Feb 2022 13:58:05 -0500 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A111BBE3E for ; Tue, 22 Feb 2022 10:57:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1645556259; x=1677092259; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=AuodqAdtsYft9lvEIxVYtPnZkgg8T2osK8Sb22u/TqE=; b=CyqtsY+KCpLwGoRvlBSDLr0pvMbhbTcryU4XxdJg9FlocgLpy9lZc/5v aPUH1YAaXfv8kgsS2js547URg6R5ApAwVobm3VeprqmWCJADanRs60nZ1 cVu9oLFSBGhY0zVsWYZLWGEo+2wnR/kJPj/dBdhmRvNirTnrfKdEUwpXV UXN6pE6Z7Wflw/VwKfL2e6BR4LvXFHjn9tdBVkmpiB7WY4HB9IWlwgsSE x4x0T9w+WzQGlXkwUhirj4Uy/3YT69uWV+XKZ7rJBRSY3JdyY2mZNaaoe 3gTg4IfnbqtEz0aEmmO017kUBFlVEqs4qfozrbaboMbYNNBvi4S5yG9JG Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10266"; a="232406837" X-IronPort-AV: E=Sophos;i="5.88,387,1635231600"; d="scan'208";a="232406837" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Feb 2022 10:57:39 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,387,1635231600"; d="scan'208";a="591415361" Received: from black.fi.intel.com ([10.237.72.28]) by fmsmga008.fm.intel.com with ESMTP; 22 Feb 2022 10:57:34 -0800 Received: by black.fi.intel.com (Postfix, from userid 1000) id 69DE840D; Tue, 22 Feb 2022 20:57:51 +0200 (EET) From: "Kirill A. Shutemov" To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@intel.com, luto@kernel.org, peterz@infradead.org Cc: sathyanarayanan.kuppuswamy@linux.intel.com, aarcange@redhat.com, ak@linux.intel.com, dan.j.williams@intel.com, david@redhat.com, hpa@zytor.com, jmattson@google.com, seanjc@google.com, thomas.lendacky@amd.com, brijesh.singh@amd.com, x86@kernel.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" Subject: [PATCH 4/4] x86/coco: Add API to handle encryption mask Date: Tue, 22 Feb 2022 21:57:40 +0300 Message-Id: <20220222185740.26228-5-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220222185740.26228-1-kirill.shutemov@linux.intel.com> References: <20220222185740.26228-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org AMD SME/SEV uses a bit in the page table entries to indicate that the page is encrypted and not accessible to the VMM. TDX uses a similar approach, but the polarity of the mask is opposite to AMD: if the bit is set the page is accessible to VMM. Provide vendor-neutral API to deal with the mask: cc_mkenc() and cc_mkdec() modify given address to make it encrypted/decrypted. It can be applied to phys_addr_t, pgprotval_t or page table entry value. pgprot_encrypted() and pgprot_decrypted() reimplemented using new helpers. The implementation will be extended to cover TDX. pgprot_decrypted() is used by drivers (i915, virtio_gpu, vfio). cc_mkdec() called by pgprot_decrypted(). Export cc_mkdec(). Signed-off-by: Kirill A. Shutemov --- arch/x86/coco/core.c | 27 +++++++++++++++++++++++++++ arch/x86/include/asm/coco.h | 18 ++++++++++++++++++ arch/x86/include/asm/pgtable.h | 13 +++++++------ arch/x86/mm/mem_encrypt_identity.c | 1 + arch/x86/mm/pat/set_memory.c | 5 +++-- 5 files changed, 56 insertions(+), 8 deletions(-) diff --git a/arch/x86/coco/core.c b/arch/x86/coco/core.c index 476dcd198af5..fc1365dd927e 100644 --- a/arch/x86/coco/core.c +++ b/arch/x86/coco/core.c @@ -14,6 +14,7 @@ #include static enum cc_vendor vendor __ro_after_init; +static u64 cc_mask __ro_after_init; static bool intel_cc_platform_has(enum cc_attr attr) { @@ -84,7 +85,33 @@ bool cc_platform_has(enum cc_attr attr) } EXPORT_SYMBOL_GPL(cc_platform_has); +u64 cc_mkenc(u64 val) +{ + switch (vendor) { + case CC_VENDOR_AMD: + return val | cc_mask; + default: + return val; + } +} + +u64 cc_mkdec(u64 val) +{ + switch (vendor) { + case CC_VENDOR_AMD: + return val & ~cc_mask; + default: + return val; + } +} +EXPORT_SYMBOL_GPL(cc_mkdec); + __init void cc_set_vendor(enum cc_vendor v) { vendor = v; } + +__init void cc_set_mask(u64 mask) +{ + cc_mask = mask; +} diff --git a/arch/x86/include/asm/coco.h b/arch/x86/include/asm/coco.h index e49f9ddb6ae6..3d98c3a60d34 100644 --- a/arch/x86/include/asm/coco.h +++ b/arch/x86/include/asm/coco.h @@ -2,6 +2,8 @@ #ifndef _ASM_X86_COCO_H #define _ASM_X86_COCO_H +#include + enum cc_vendor { CC_VENDOR_NONE, CC_VENDOR_AMD, @@ -10,5 +12,21 @@ enum cc_vendor { }; void cc_set_vendor(enum cc_vendor v); +void cc_set_mask(u64 mask); + +#ifdef CONFIG_ARCH_HAS_CC_PLATFORM +u64 cc_mkenc(u64 val); +u64 cc_mkdec(u64 val); +#else +static inline u64 cc_mkenc(u64 val) +{ + return val; +} + +static inline u64 cc_mkdec(u64 val) +{ + return val; +} +#endif #endif /* _ASM_X86_COCO_H */ diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 8a9432fb3802..62ab07e24aef 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -15,17 +15,12 @@ cachemode2protval(_PAGE_CACHE_MODE_UC_MINUS))) \ : (prot)) -/* - * Macros to add or remove encryption attribute - */ -#define pgprot_encrypted(prot) __pgprot(__sme_set(pgprot_val(prot))) -#define pgprot_decrypted(prot) __pgprot(__sme_clr(pgprot_val(prot))) - #ifndef __ASSEMBLY__ #include #include #include #include +#include #include #include @@ -38,6 +33,12 @@ void ptdump_walk_pgd_level_debugfs(struct seq_file *m, struct mm_struct *mm, void ptdump_walk_pgd_level_checkwx(void); void ptdump_walk_user_pgd_level_checkwx(void); +/* + * Macros to add or remove encryption attribute + */ +#define pgprot_encrypted(prot) __pgprot(cc_mkenc(pgprot_val(prot))) +#define pgprot_decrypted(prot) __pgprot(cc_mkdec(pgprot_val(prot))) + #ifdef CONFIG_DEBUG_WX #define debug_checkwx() ptdump_walk_pgd_level_checkwx() #define debug_checkwx_user() ptdump_walk_user_pgd_level_checkwx() diff --git a/arch/x86/mm/mem_encrypt_identity.c b/arch/x86/mm/mem_encrypt_identity.c index 06314ae3998e..b43bc24d2bb6 100644 --- a/arch/x86/mm/mem_encrypt_identity.c +++ b/arch/x86/mm/mem_encrypt_identity.c @@ -604,5 +604,6 @@ void __init sme_enable(struct boot_params *bp) if (sme_me_mask) { physical_mask &= ~sme_me_mask; cc_set_vendor(CC_VENDOR_AMD); + cc_set_mask(sme_me_mask); } } diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c index b4072115c8ef..af77dbfd143c 100644 --- a/arch/x86/mm/pat/set_memory.c +++ b/arch/x86/mm/pat/set_memory.c @@ -1990,6 +1990,7 @@ int set_memory_global(unsigned long addr, int numpages) static int __set_memory_enc_pgtable(unsigned long addr, int numpages, bool enc) { struct cpa_data cpa; + pgprot_t empty = __pgprot(0); int ret; /* Should not be working on unaligned addresses */ @@ -1999,8 +2000,8 @@ static int __set_memory_enc_pgtable(unsigned long addr, int numpages, bool enc) memset(&cpa, 0, sizeof(cpa)); cpa.vaddr = &addr; cpa.numpages = numpages; - cpa.mask_set = enc ? __pgprot(_PAGE_ENC) : __pgprot(0); - cpa.mask_clr = enc ? __pgprot(0) : __pgprot(_PAGE_ENC); + cpa.mask_set = enc ? pgprot_encrypted(empty) : pgprot_decrypted(empty); + cpa.mask_clr = enc ? pgprot_decrypted(empty) : pgprot_encrypted(empty); cpa.pgd = init_mm.pgd; /* Must avoid aliasing mappings in the highmem code */ -- 2.34.1