2024-03-27 12:27:34

by Sasha Levin

[permalink] [raw]
Subject: FAILED: Patch "x86/sev: Fix position dependent variable references in startup code" failed to apply to 6.8-stable tree

The patch below does not apply to the 6.8-stable tree.
If someone wants it applied there, or to any other stable or longterm
tree, then please email the backport, including the original git commit
id to <[email protected]>.

Thanks,
Sasha

------------------ original commit in Linus's tree ------------------

From 1c811d403afd73f04bde82b83b24c754011bd0e8 Mon Sep 17 00:00:00 2001
From: Ard Biesheuvel <[email protected]>
Date: Sat, 3 Feb 2024 13:53:06 +0100
Subject: [PATCH] x86/sev: Fix position dependent variable references in
startup code

The early startup code executes from a 1:1 mapping of memory, which
differs from the mapping that the code was linked and/or relocated to
run at. The latter mapping is not active yet at this point, and so
symbol references that rely on it will fault.

Given that the core kernel is built without -fPIC, symbol references are
typically emitted as absolute, and so any such references occuring in
the early startup code will therefore crash the kernel.

While an attempt was made to work around this for the early SEV/SME
startup code, by forcing RIP-relative addressing for certain global
SEV/SME variables via inline assembly (see snp_cpuid_get_table() for
example), RIP-relative addressing must be pervasively enforced for
SEV/SME global variables when accessed prior to page table fixups.

__startup_64() already handles this issue for select non-SEV/SME global
variables using fixup_pointer(), which adjusts the pointer relative to a
`physaddr` argument. To avoid having to pass around this `physaddr`
argument across all functions needing to apply pointer fixups, introduce
a macro RIP_RELATIVE_REF() which generates a RIP-relative reference to
a given global variable. It is used where necessary to force
RIP-relative accesses to global variables.

For backporting purposes, this patch makes no attempt at cleaning up
other occurrences of this pattern, involving either inline asm or
fixup_pointer(). Those will be addressed later.

[ bp: Call it "rip_rel_ref" everywhere like other code shortens
"rIP-relative reference" and make the asm wrapper __always_inline. ]

Co-developed-by: Kevin Loughlin <[email protected]>
Signed-off-by: Kevin Loughlin <[email protected]>
Signed-off-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Borislav Petkov (AMD) <[email protected]>
Cc: <[email protected]>
Link: https://lore.kernel.org/all/[email protected]
---
arch/x86/coco/core.c | 7 +------
arch/x86/include/asm/asm.h | 14 ++++++++++++++
arch/x86/include/asm/coco.h | 8 +++++++-
arch/x86/include/asm/mem_encrypt.h | 15 +++++++++------
arch/x86/kernel/sev-shared.c | 12 ++++++------
arch/x86/kernel/sev.c | 4 ++--
arch/x86/mm/mem_encrypt_identity.c | 27 ++++++++++++---------------
7 files changed, 51 insertions(+), 36 deletions(-)

diff --git a/arch/x86/coco/core.c b/arch/x86/coco/core.c
index eeec9986570ed..d07be9d05cd03 100644
--- a/arch/x86/coco/core.c
+++ b/arch/x86/coco/core.c
@@ -14,7 +14,7 @@
#include <asm/processor.h>

enum cc_vendor cc_vendor __ro_after_init = CC_VENDOR_NONE;
-static u64 cc_mask __ro_after_init;
+u64 cc_mask __ro_after_init;

static bool noinstr intel_cc_platform_has(enum cc_attr attr)
{
@@ -148,8 +148,3 @@ u64 cc_mkdec(u64 val)
}
}
EXPORT_SYMBOL_GPL(cc_mkdec);
-
-__init void cc_set_mask(u64 mask)
-{
- cc_mask = mask;
-}
diff --git a/arch/x86/include/asm/asm.h b/arch/x86/include/asm/asm.h
index fbcfec4dc4ccd..ca8eed1d496ab 100644
--- a/arch/x86/include/asm/asm.h
+++ b/arch/x86/include/asm/asm.h
@@ -113,6 +113,20 @@

#endif

+#ifndef __ASSEMBLY__
+#ifndef __pic__
+static __always_inline __pure void *rip_rel_ptr(void *p)
+{
+ asm("leaq %c1(%%rip), %0" : "=r"(p) : "i"(p));
+
+ return p;
+}
+#define RIP_REL_REF(var) (*(typeof(&(var)))rip_rel_ptr(&(var)))
+#else
+#define RIP_REL_REF(var) (var)
+#endif
+#endif
+
/*
* Macros to generate condition code outputs from inline assembly,
* The output operand must be type "bool".
diff --git a/arch/x86/include/asm/coco.h b/arch/x86/include/asm/coco.h
index 6ae2d16a7613b..21940ef8d2904 100644
--- a/arch/x86/include/asm/coco.h
+++ b/arch/x86/include/asm/coco.h
@@ -2,6 +2,7 @@
#ifndef _ASM_X86_COCO_H
#define _ASM_X86_COCO_H

+#include <asm/asm.h>
#include <asm/types.h>

enum cc_vendor {
@@ -11,9 +12,14 @@ enum cc_vendor {
};

extern enum cc_vendor cc_vendor;
+extern u64 cc_mask;

#ifdef CONFIG_ARCH_HAS_CC_PLATFORM
-void cc_set_mask(u64 mask);
+static inline void cc_set_mask(u64 mask)
+{
+ RIP_REL_REF(cc_mask) = mask;
+}
+
u64 cc_mkenc(u64 val);
u64 cc_mkdec(u64 val);
#else
diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h
index 359ada486fa92..b31eb9fd59544 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -15,7 +15,8 @@
#include <linux/init.h>
#include <linux/cc_platform.h>

-#include <asm/bootparam.h>
+#include <asm/asm.h>
+struct boot_params;

#ifdef CONFIG_X86_MEM_ENCRYPT
void __init mem_encrypt_init(void);
@@ -58,6 +59,11 @@ void __init mem_encrypt_free_decrypted_mem(void);

void __init sev_es_init_vc_handling(void);

+static inline u64 sme_get_me_mask(void)
+{
+ return RIP_REL_REF(sme_me_mask);
+}
+
#define __bss_decrypted __section(".bss..decrypted")

#else /* !CONFIG_AMD_MEM_ENCRYPT */
@@ -89,6 +95,8 @@ early_set_mem_enc_dec_hypercall(unsigned long vaddr, unsigned long size, bool en

static inline void mem_encrypt_free_decrypted_mem(void) { }

+static inline u64 sme_get_me_mask(void) { return 0; }
+
#define __bss_decrypted

#endif /* CONFIG_AMD_MEM_ENCRYPT */
@@ -106,11 +114,6 @@ void add_encrypt_protection_map(void);

extern char __start_bss_decrypted[], __end_bss_decrypted[], __start_bss_decrypted_unused[];

-static inline u64 sme_get_me_mask(void)
-{
- return sme_me_mask;
-}
-
#endif /* __ASSEMBLY__ */

#endif /* __X86_MEM_ENCRYPT_H__ */
diff --git a/arch/x86/kernel/sev-shared.c b/arch/x86/kernel/sev-shared.c
index 5db24d0fc557c..ae79f9505298d 100644
--- a/arch/x86/kernel/sev-shared.c
+++ b/arch/x86/kernel/sev-shared.c
@@ -560,9 +560,9 @@ static int snp_cpuid(struct ghcb *ghcb, struct es_em_ctxt *ctxt, struct cpuid_le
leaf->eax = leaf->ebx = leaf->ecx = leaf->edx = 0;

/* Skip post-processing for out-of-range zero leafs. */
- if (!(leaf->fn <= cpuid_std_range_max ||
- (leaf->fn >= 0x40000000 && leaf->fn <= cpuid_hyp_range_max) ||
- (leaf->fn >= 0x80000000 && leaf->fn <= cpuid_ext_range_max)))
+ if (!(leaf->fn <= RIP_REL_REF(cpuid_std_range_max) ||
+ (leaf->fn >= 0x40000000 && leaf->fn <= RIP_REL_REF(cpuid_hyp_range_max)) ||
+ (leaf->fn >= 0x80000000 && leaf->fn <= RIP_REL_REF(cpuid_ext_range_max))))
return 0;
}

@@ -1072,11 +1072,11 @@ static void __init setup_cpuid_table(const struct cc_blob_sev_info *cc_info)
const struct snp_cpuid_fn *fn = &cpuid_table->fn[i];

if (fn->eax_in == 0x0)
- cpuid_std_range_max = fn->eax;
+ RIP_REL_REF(cpuid_std_range_max) = fn->eax;
else if (fn->eax_in == 0x40000000)
- cpuid_hyp_range_max = fn->eax;
+ RIP_REL_REF(cpuid_hyp_range_max) = fn->eax;
else if (fn->eax_in == 0x80000000)
- cpuid_ext_range_max = fn->eax;
+ RIP_REL_REF(cpuid_ext_range_max) = fn->eax;
}
}

diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c
index 002af6c30601b..1ef7ae806a01b 100644
--- a/arch/x86/kernel/sev.c
+++ b/arch/x86/kernel/sev.c
@@ -748,7 +748,7 @@ void __init early_snp_set_memory_private(unsigned long vaddr, unsigned long padd
* This eliminates worries about jump tables or checking boot_cpu_data
* in the cc_platform_has() function.
*/
- if (!(sev_status & MSR_AMD64_SEV_SNP_ENABLED))
+ if (!(RIP_REL_REF(sev_status) & MSR_AMD64_SEV_SNP_ENABLED))
return;

/*
@@ -767,7 +767,7 @@ void __init early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr
* This eliminates worries about jump tables or checking boot_cpu_data
* in the cc_platform_has() function.
*/
- if (!(sev_status & MSR_AMD64_SEV_SNP_ENABLED))
+ if (!(RIP_REL_REF(sev_status) & MSR_AMD64_SEV_SNP_ENABLED))
return;

/* Ask hypervisor to mark the memory pages shared in the RMP table. */
diff --git a/arch/x86/mm/mem_encrypt_identity.c b/arch/x86/mm/mem_encrypt_identity.c
index efe9f217fcf99..0166ab1780ccb 100644
--- a/arch/x86/mm/mem_encrypt_identity.c
+++ b/arch/x86/mm/mem_encrypt_identity.c
@@ -304,7 +304,8 @@ void __init sme_encrypt_kernel(struct boot_params *bp)
* instrumentation or checking boot_cpu_data in the cc_platform_has()
* function.
*/
- if (!sme_get_me_mask() || sev_status & MSR_AMD64_SEV_ENABLED)
+ if (!sme_get_me_mask() ||
+ RIP_REL_REF(sev_status) & MSR_AMD64_SEV_ENABLED)
return;

/*
@@ -541,11 +542,11 @@ void __init sme_enable(struct boot_params *bp)
me_mask = 1UL << (ebx & 0x3f);

/* Check the SEV MSR whether SEV or SME is enabled */
- sev_status = __rdmsr(MSR_AMD64_SEV);
- feature_mask = (sev_status & MSR_AMD64_SEV_ENABLED) ? AMD_SEV_BIT : AMD_SME_BIT;
+ RIP_REL_REF(sev_status) = msr = __rdmsr(MSR_AMD64_SEV);
+ feature_mask = (msr & MSR_AMD64_SEV_ENABLED) ? AMD_SEV_BIT : AMD_SME_BIT;

/* The SEV-SNP CC blob should never be present unless SEV-SNP is enabled. */
- if (snp && !(sev_status & MSR_AMD64_SEV_SNP_ENABLED))
+ if (snp && !(msr & MSR_AMD64_SEV_SNP_ENABLED))
snp_abort();

/* Check if memory encryption is enabled */
@@ -571,7 +572,6 @@ void __init sme_enable(struct boot_params *bp)
return;
} else {
/* SEV state cannot be controlled by a command line option */
- sme_me_mask = me_mask;
goto out;
}

@@ -590,16 +590,13 @@ void __init sme_enable(struct boot_params *bp)
cmdline_ptr = (const char *)((u64)bp->hdr.cmd_line_ptr |
((u64)bp->ext_cmd_line_ptr << 32));

- if (cmdline_find_option(cmdline_ptr, cmdline_arg, buffer, sizeof(buffer)) < 0)
- goto out;
-
- if (!strncmp(buffer, cmdline_on, sizeof(buffer)))
- sme_me_mask = me_mask;
+ if (cmdline_find_option(cmdline_ptr, cmdline_arg, buffer, sizeof(buffer)) < 0 ||
+ strncmp(buffer, cmdline_on, sizeof(buffer)))
+ return;

out:
- if (sme_me_mask) {
- physical_mask &= ~sme_me_mask;
- cc_vendor = CC_VENDOR_AMD;
- cc_set_mask(sme_me_mask);
- }
+ RIP_REL_REF(sme_me_mask) = me_mask;
+ physical_mask &= ~me_mask;
+ cc_vendor = CC_VENDOR_AMD;
+ cc_set_mask(me_mask);
}
--
2.43.0






2024-03-27 15:15:33

by Ard Biesheuvel

[permalink] [raw]
Subject: Re: FAILED: Patch "x86/sev: Fix position dependent variable references in startup code" failed to apply to 6.8-stable tree

On Wed, 27 Mar 2024 at 14:08, Sasha Levin <[email protected]> wrote:
>
> The patch below does not apply to the 6.8-stable tree.
> If someone wants it applied there, or to any other stable or longterm
> tree, then please email the backport, including the original git commit
> id to <[email protected]>.
>

I will send the v6.8 backport separately right away.

v6.7 and v6.6 can take this patch unmodified but will need to take
29956748339aa8757a7e2f927a8679dd08f24bb6 as a prerequisite first.

(v6.8 no longer applies because of the way Linus fixed up a merge
conflict in the 6.8 cycle)

v6.1 and older need separate backports so i will send those out later.

2024-03-29 11:57:18

by Greg Kroah-Hartman

[permalink] [raw]
Subject: Re: FAILED: Patch "x86/sev: Fix position dependent variable references in startup code" failed to apply to 6.8-stable tree

On Wed, Mar 27, 2024 at 04:39:20PM +0200, Ard Biesheuvel wrote:
> On Wed, 27 Mar 2024 at 14:08, Sasha Levin <[email protected]> wrote:
> >
> > The patch below does not apply to the 6.8-stable tree.
> > If someone wants it applied there, or to any other stable or longterm
> > tree, then please email the backport, including the original git commit
> > id to <[email protected]>.
> >
>
> I will send the v6.8 backport separately right away.
>
> v6.7 and v6.6 can take this patch unmodified but will need to take
> 29956748339aa8757a7e2f927a8679dd08f24bb6 as a prerequisite first.

Now done, thanks.

> (v6.8 no longer applies because of the way Linus fixed up a merge
> conflict in the 6.8 cycle)

I fixed this up by hand, hopefully got it right :)

> v6.1 and older need separate backports so i will send those out later.

Wonderful, thanks!

greg k-h

2024-03-29 16:00:52

by Ard Biesheuvel

[permalink] [raw]
Subject: Re: FAILED: Patch "x86/sev: Fix position dependent variable references in startup code" failed to apply to 6.8-stable tree

On Fri, 29 Mar 2024 at 13:56, Greg KH <[email protected]> wrote:
>
> On Wed, Mar 27, 2024 at 04:39:20PM +0200, Ard Biesheuvel wrote:
> > On Wed, 27 Mar 2024 at 14:08, Sasha Levin <[email protected]> wrote:
> > >
> > > The patch below does not apply to the 6.8-stable tree.
> > > If someone wants it applied there, or to any other stable or longterm
> > > tree, then please email the backport, including the original git commit
> > > id to <[email protected]>.
> > >
> >
> > I will send the v6.8 backport separately right away.
> >
> > v6.7 and v6.6 can take this patch unmodified but will need to take
> > 29956748339aa8757a7e2f927a8679dd08f24bb6 as a prerequisite first.
>
> Now done, thanks.
>
> > (v6.8 no longer applies because of the way Linus fixed up a merge
> > conflict in the 6.8 cycle)
>
> I fixed this up by hand, hopefully got it right :)
>

Thanks - I will double check once the patches are sent out.

> > v6.1 and older need separate backports so i will send those out later.
>
> Wonderful, thanks!
>

I sent this out on Wednesday but I don't see them in lore. Please let
me know if I should resend these.

[PATCH -stable-6.1 1/4] x86/coco: Export cc_vendor
[PATCH -stable-6.1 2/4] x86/coco: Get rid of accessor functions
[PATCH -stable-6.1 3/4] x86/Kconfig: Remove
CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT
[PATCH -stable-6.1 4/4] x86/sev: Fix position dependent variable
references in startup code

2024-03-29 17:56:21

by Greg Kroah-Hartman

[permalink] [raw]
Subject: Re: FAILED: Patch "x86/sev: Fix position dependent variable references in startup code" failed to apply to 6.8-stable tree

On Fri, Mar 29, 2024 at 05:25:35PM +0200, Ard Biesheuvel wrote:
> On Fri, 29 Mar 2024 at 13:56, Greg KH <[email protected]> wrote:
> >
> > On Wed, Mar 27, 2024 at 04:39:20PM +0200, Ard Biesheuvel wrote:
> > > On Wed, 27 Mar 2024 at 14:08, Sasha Levin <[email protected]> wrote:
> > > >
> > > > The patch below does not apply to the 6.8-stable tree.
> > > > If someone wants it applied there, or to any other stable or longterm
> > > > tree, then please email the backport, including the original git commit
> > > > id to <[email protected]>.
> > > >
> > >
> > > I will send the v6.8 backport separately right away.
> > >
> > > v6.7 and v6.6 can take this patch unmodified but will need to take
> > > 29956748339aa8757a7e2f927a8679dd08f24bb6 as a prerequisite first.
> >
> > Now done, thanks.
> >
> > > (v6.8 no longer applies because of the way Linus fixed up a merge
> > > conflict in the 6.8 cycle)
> >
> > I fixed this up by hand, hopefully got it right :)
> >
>
> Thanks - I will double check once the patches are sent out.
>
> > > v6.1 and older need separate backports so i will send those out later.
> >
> > Wonderful, thanks!
> >
>
> I sent this out on Wednesday but I don't see them in lore. Please let
> me know if I should resend these.
>
> [PATCH -stable-6.1 1/4] x86/coco: Export cc_vendor
> [PATCH -stable-6.1 2/4] x86/coco: Get rid of accessor functions
> [PATCH -stable-6.1 3/4] x86/Kconfig: Remove
> CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT
> [PATCH -stable-6.1 4/4] x86/sev: Fix position dependent variable
> references in startup code

I can't see those anywhere, so yes, please resend as they are not in my
queue, or archive, at all.

thanks,

greg k-h