Control-flow Enforcement (CET) is a new Intel processor feature that blocks
return/jump-oriented programming attacks. Details are in "Intel 64 and
IA-32 Architectures Software Developer's Manual" [1].
This is the second part of CET and enables Indirect Branch Tracking (IBT).
It is built on top of the shadow stack series.
Changes in v14:
- Drop vsyscall fixup patch, as this needs more discussion and testing.
I will send this out separately.
[1] Intel 64 and IA-32 Architectures Software Developer's Manual:
https://software.intel.com/en-us/download/intel-64-and-ia-32-
architectures-sdm-combined-volumes-1-2a-2b-2c-2d-3a-3b-3c-3d-and-4
[2] Indirect Branch Tracking patches v13.
https://lkml.kernel.org/r/[email protected]/
H.J. Lu (3):
x86/cet/ibt: Update arch_prctl functions for Indirect Branch Tracking
x86/vdso/32: Add ENDBR32 to __kernel_vsyscall entry point
x86/vdso: Insert endbr32/endbr64 to vDSO
Yu-cheng Yu (4):
x86/cet/ibt: Add Kconfig option for user-mode Indirect Branch Tracking
x86/cet/ibt: User-mode Indirect Branch Tracking support
x86/cet/ibt: Handle signals for Indirect Branch Tracking
x86/cet/ibt: ELF header parsing for Indirect Branch Tracking
arch/x86/Kconfig | 21 +++++++
arch/x86/entry/vdso/Makefile | 4 ++
arch/x86/entry/vdso/vdso32/system_call.S | 3 +
arch/x86/include/asm/cet.h | 3 +
arch/x86/include/asm/disabled-features.h | 8 ++-
arch/x86/kernel/cet.c | 60 ++++++++++++++++++-
arch/x86/kernel/cet_prctl.c | 8 ++-
arch/x86/kernel/cpu/common.c | 17 ++++++
arch/x86/kernel/fpu/signal.c | 8 ++-
arch/x86/kernel/process_64.c | 8 +++
.../arch/x86/include/asm/disabled-features.h | 8 ++-
11 files changed, 140 insertions(+), 8 deletions(-)
--
2.21.0
From: "H.J. Lu" <[email protected]>
Add ENDBR32 to __kernel_vsyscall entry point.
Signed-off-by: H.J. Lu <[email protected]>
Signed-off-by: Yu-cheng Yu <[email protected]>
Acked-by: Andy Lutomirski <[email protected]>
---
arch/x86/entry/vdso/vdso32/system_call.S | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/x86/entry/vdso/vdso32/system_call.S b/arch/x86/entry/vdso/vdso32/system_call.S
index de1fff7188aa..e331fcdebd95 100644
--- a/arch/x86/entry/vdso/vdso32/system_call.S
+++ b/arch/x86/entry/vdso/vdso32/system_call.S
@@ -14,6 +14,9 @@
ALIGN
__kernel_vsyscall:
CFI_STARTPROC
+#ifdef CONFIG_X86_BRANCH_TRACKING_USER
+ endbr32
+#endif
/*
* Reshuffle regs so that all of any of the entry instructions
* will preserve enough state.
--
2.21.0
Introduce Kconfig option X86_BRANCH_TRACKING_USER.
Indirect Branch Tracking (IBT) provides protection against CALL-/JMP-
oriented programming attacks. It is active when the kernel has this
feature enabled, and the processor and the application support it.
When this feature is enabled, legacy non-IBT applications continue to
work, but without IBT protection.
Signed-off-by: Yu-cheng Yu <[email protected]>
---
arch/x86/Kconfig | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 4b28a0ce4594..15c7f2606c9d 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1966,6 +1966,25 @@ config X86_SHADOW_STACK_USER
If unsure, say N.
+config X86_BRANCH_TRACKING_USER
+ prompt "Intel Indirect Branch Tracking for user-mode"
+ def_bool n
+ depends on CPU_SUP_INTEL && X86_64
+ depends on $(cc-option,-fcf-protection)
+ select X86_CET
+ help
+ Indirect Branch Tracking (IBT) provides protection against
+ CALL-/JMP-oriented programming attacks. It is active when
+ the kernel has this feature enabled, and the processor and
+ the application support it. When this feature is enabled,
+ legacy non-IBT applications continue to work, but without
+ IBT protection.
+ Support for this feature is only known to be present on
+ processors released in 2020 or later. CET features are also
+ known to increase kernel text size by 3.7 KB.
+
+ If unsure, say N.
+
config EFI
bool "EFI runtime service support"
depends on ACPI
--
2.21.0
An indirect CALL/JMP moves the indirect branch tracking (IBT) state machine
to WAIT_ENDBR status until the instruction reaches an ENDBR opcode. If the
CALL/JMP does not reach an ENDBR opcode, the processor raises a control-
protection fault. WAIT_ENDBR status can be read from MSR_IA32_U_CET.
WAIT_ENDBR is cleared for signal handling, and restored for sigreturn.
IBT state machine is described in Intel SDM Vol. 1, Sec. 18.3.
Signed-off-by: Yu-cheng Yu <[email protected]>
---
arch/x86/kernel/cet.c | 27 +++++++++++++++++++++++++--
arch/x86/kernel/fpu/signal.c | 8 +++++---
2 files changed, 30 insertions(+), 5 deletions(-)
diff --git a/arch/x86/kernel/cet.c b/arch/x86/kernel/cet.c
index 1dcd16aa4f90..0edcbd70f408 100644
--- a/arch/x86/kernel/cet.c
+++ b/arch/x86/kernel/cet.c
@@ -295,6 +295,13 @@ void cet_restore_signal(struct sc_ext *sc_ext)
msr_val |= CET_SHSTK_EN;
}
+ if (cet->ibt_enabled) {
+ msr_val |= (CET_ENDBR_EN | CET_NO_TRACK_EN);
+
+ if (sc_ext->wait_endbr)
+ msr_val |= CET_WAIT_ENDBR;
+ }
+
if (test_thread_flag(TIF_NEED_FPU_LOAD))
cet_user_state->user_cet = msr_val;
else
@@ -335,9 +342,25 @@ int cet_setup_signal(bool ia32, unsigned long rstor_addr, struct sc_ext *sc_ext)
sc_ext->ssp = new_ssp;
}
- if (ssp) {
+ if (ssp || cet->ibt_enabled) {
+
start_update_msrs();
- wrmsrl(MSR_IA32_PL3_SSP, ssp);
+
+ if (ssp)
+ wrmsrl(MSR_IA32_PL3_SSP, ssp);
+
+ if (cet->ibt_enabled) {
+ u64 r;
+
+ rdmsrl(MSR_IA32_U_CET, r);
+
+ if (r & CET_WAIT_ENDBR) {
+ sc_ext->wait_endbr = 1;
+ r &= ~CET_WAIT_ENDBR;
+ wrmsrl(MSR_IA32_U_CET, r);
+ }
+ }
+
end_update_msrs();
}
diff --git a/arch/x86/kernel/fpu/signal.c b/arch/x86/kernel/fpu/signal.c
index c0c2141cb4b3..077853ef6f48 100644
--- a/arch/x86/kernel/fpu/signal.c
+++ b/arch/x86/kernel/fpu/signal.c
@@ -57,7 +57,8 @@ int save_cet_to_sigframe(int ia32, void __user *fp, unsigned long restorer)
{
int err = 0;
- if (!current->thread.cet.shstk_size)
+ if (!current->thread.cet.shstk_size &&
+ !current->thread.cet.ibt_enabled)
return 0;
if (fp) {
@@ -89,7 +90,8 @@ static int get_cet_from_sigframe(int ia32, void __user *fp, struct sc_ext *ext)
memset(ext, 0, sizeof(*ext));
- if (!current->thread.cet.shstk_size)
+ if (!current->thread.cet.shstk_size &&
+ !current->thread.cet.ibt_enabled)
return 0;
if (fp) {
@@ -577,7 +579,7 @@ static unsigned long fpu__alloc_sigcontext_ext(unsigned long sp)
* sigcontext_ext is at: fpu + fpu_user_xstate_size +
* FP_XSTATE_MAGIC2_SIZE, then aligned to 8.
*/
- if (cet->shstk_size)
+ if (cet->shstk_size || cet->ibt_enabled)
sp -= (sizeof(struct sc_ext) + 8);
return sp;
--
2.21.0
From: "H.J. Lu" <[email protected]>
When Indirect Branch Tracking (IBT) is enabled, vDSO functions may be
called indirectly, and must have ENDBR32 or ENDBR64 as the first
instruction. The compiler must support -fcf-protection=branch so that it
can be used to compile vDSO.
Signed-off-by: H.J. Lu <[email protected]>
Signed-off-by: Yu-cheng Yu <[email protected]>
Acked-by: Andy Lutomirski <[email protected]>
---
arch/x86/entry/vdso/Makefile | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/arch/x86/entry/vdso/Makefile b/arch/x86/entry/vdso/Makefile
index 215376d975a2..1f1b6893068a 100644
--- a/arch/x86/entry/vdso/Makefile
+++ b/arch/x86/entry/vdso/Makefile
@@ -94,6 +94,10 @@ endif
$(vobjs): KBUILD_CFLAGS := $(filter-out $(GCC_PLUGINS_CFLAGS) $(RETPOLINE_CFLAGS),$(KBUILD_CFLAGS)) $(CFL)
+ifdef CONFIG_X86_BRANCH_TRACKING_USER
+$(vobjs) $(vobjs32): KBUILD_CFLAGS += -fcf-protection=branch
+endif
+
#
# vDSO code runs in userspace and -pg doesn't help with profiling anyway.
#
--
2.21.0
Introduce user-mode Indirect Branch Tracking (IBT) support. Update setup
routines to include IBT.
Signed-off-by: Yu-cheng Yu <[email protected]>
---
arch/x86/include/asm/cet.h | 3 ++
arch/x86/include/asm/disabled-features.h | 8 ++++-
arch/x86/kernel/cet.c | 33 +++++++++++++++++++
arch/x86/kernel/cpu/common.c | 17 ++++++++++
.../arch/x86/include/asm/disabled-features.h | 8 ++++-
5 files changed, 67 insertions(+), 2 deletions(-)
diff --git a/arch/x86/include/asm/cet.h b/arch/x86/include/asm/cet.h
index 16870e5bc8eb..3a1cba579cb2 100644
--- a/arch/x86/include/asm/cet.h
+++ b/arch/x86/include/asm/cet.h
@@ -15,6 +15,7 @@ struct cet_status {
unsigned long shstk_base;
unsigned long shstk_size;
unsigned int locked:1;
+ unsigned int ibt_enabled:1;
};
#ifdef CONFIG_X86_CET
@@ -26,6 +27,8 @@ void cet_free_shstk(struct task_struct *p);
int cet_verify_rstor_token(bool ia32, unsigned long ssp, unsigned long *new_ssp);
void cet_restore_signal(struct sc_ext *sc);
int cet_setup_signal(bool ia32, unsigned long rstor, struct sc_ext *sc);
+int cet_setup_ibt(void);
+void cet_disable_ibt(void);
#else
static inline int prctl_cet(int option, u64 arg2) { return -EINVAL; }
static inline int cet_setup_thread_shstk(struct task_struct *p,
diff --git a/arch/x86/include/asm/disabled-features.h b/arch/x86/include/asm/disabled-features.h
index edac76ed75e7..e7096a1e2698 100644
--- a/arch/x86/include/asm/disabled-features.h
+++ b/arch/x86/include/asm/disabled-features.h
@@ -62,6 +62,12 @@
#define DISABLE_SHSTK (1<<(X86_FEATURE_SHSTK & 31))
#endif
+#ifdef CONFIG_X86_BRANCH_TRACKING_USER
+#define DISABLE_IBT 0
+#else
+#define DISABLE_IBT (1<<(X86_FEATURE_IBT & 31))
+#endif
+
/*
* Make sure to add features to the correct mask
*/
@@ -83,7 +89,7 @@
#define DISABLED_MASK15 0
#define DISABLED_MASK16 (DISABLE_PKU|DISABLE_OSPKE|DISABLE_LA57|DISABLE_UMIP|DISABLE_SHSTK)
#define DISABLED_MASK17 0
-#define DISABLED_MASK18 0
+#define DISABLED_MASK18 (DISABLE_IBT)
#define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 19)
#endif /* _ASM_X86_DISABLED_FEATURES_H */
diff --git a/arch/x86/kernel/cet.c b/arch/x86/kernel/cet.c
index d57f3a433af9..1dcd16aa4f90 100644
--- a/arch/x86/kernel/cet.c
+++ b/arch/x86/kernel/cet.c
@@ -13,6 +13,8 @@
#include <linux/uaccess.h>
#include <linux/sched/signal.h>
#include <linux/compat.h>
+#include <linux/vmalloc.h>
+#include <linux/bitops.h>
#include <asm/msr.h>
#include <asm/user.h>
#include <asm/fpu/internal.h>
@@ -341,3 +343,34 @@ int cet_setup_signal(bool ia32, unsigned long rstor_addr, struct sc_ext *sc_ext)
return 0;
}
+
+int cet_setup_ibt(void)
+{
+ u64 msr_val;
+
+ if (!static_cpu_has(X86_FEATURE_IBT))
+ return -EOPNOTSUPP;
+
+ start_update_msrs();
+ rdmsrl(MSR_IA32_U_CET, msr_val);
+ msr_val |= (CET_ENDBR_EN | CET_NO_TRACK_EN);
+ wrmsrl(MSR_IA32_U_CET, msr_val);
+ end_update_msrs();
+ current->thread.cet.ibt_enabled = 1;
+ return 0;
+}
+
+void cet_disable_ibt(void)
+{
+ u64 msr_val;
+
+ if (!static_cpu_has(X86_FEATURE_IBT))
+ return;
+
+ start_update_msrs();
+ rdmsrl(MSR_IA32_U_CET, msr_val);
+ msr_val &= CET_SHSTK_EN;
+ wrmsrl(MSR_IA32_U_CET, msr_val);
+ end_update_msrs();
+ current->thread.cet.ibt_enabled = 0;
+}
diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index 084480f975aa..909b4160a2d2 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -536,6 +536,23 @@ static __init int setup_disable_shstk(char *s)
__setup("no_user_shstk", setup_disable_shstk);
#endif
+#ifdef CONFIG_X86_BRANCH_TRACKING_USER
+static __init int setup_disable_ibt(char *s)
+{
+ /* require an exact match without trailing characters */
+ if (s[0] != '\0')
+ return 0;
+
+ if (!boot_cpu_has(X86_FEATURE_IBT))
+ return 1;
+
+ setup_clear_cpu_cap(X86_FEATURE_IBT);
+ pr_info("x86: 'no_user_ibt' specified, disabling user Branch Tracking\n");
+ return 1;
+}
+__setup("no_user_ibt", setup_disable_ibt);
+#endif
+
/*
* Some CPU features depend on higher CPUID levels, which may not always
* be available due to CPUID level capping or broken virtualization
diff --git a/tools/arch/x86/include/asm/disabled-features.h b/tools/arch/x86/include/asm/disabled-features.h
index edac76ed75e7..e7096a1e2698 100644
--- a/tools/arch/x86/include/asm/disabled-features.h
+++ b/tools/arch/x86/include/asm/disabled-features.h
@@ -62,6 +62,12 @@
#define DISABLE_SHSTK (1<<(X86_FEATURE_SHSTK & 31))
#endif
+#ifdef CONFIG_X86_BRANCH_TRACKING_USER
+#define DISABLE_IBT 0
+#else
+#define DISABLE_IBT (1<<(X86_FEATURE_IBT & 31))
+#endif
+
/*
* Make sure to add features to the correct mask
*/
@@ -83,7 +89,7 @@
#define DISABLED_MASK15 0
#define DISABLED_MASK16 (DISABLE_PKU|DISABLE_OSPKE|DISABLE_LA57|DISABLE_UMIP|DISABLE_SHSTK)
#define DISABLED_MASK17 0
-#define DISABLED_MASK18 0
+#define DISABLED_MASK18 (DISABLE_IBT)
#define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 19)
#endif /* _ASM_X86_DISABLED_FEATURES_H */
--
2.21.0
On Mon, Oct 12, 2020 at 08:45:24AM -0700, Yu-cheng Yu wrote:
...
> + the application support it. When this feature is enabled,
> + legacy non-IBT applications continue to work, but without
> + IBT protection.
> + Support for this feature is only known to be present on
> + processors released in 2020 or later. CET features are also
> + known to increase kernel text size by 3.7 KB.
It seems the last sentence is redundant - new features always bloat
the kernel code and precise size may differ depending on compiler
and options. Surely this can be patched on top.
From: "H.J. Lu" <[email protected]>
Update ARCH_X86_CET_STATUS and ARCH_X86_CET_DISABLE for Indirect Branch
Tracking.
Signed-off-by: H.J. Lu <[email protected]>
Signed-off-by: Yu-cheng Yu <[email protected]>
---
arch/x86/kernel/cet_prctl.c | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kernel/cet_prctl.c b/arch/x86/kernel/cet_prctl.c
index bd5ad11763e4..0af1ec5d028f 100644
--- a/arch/x86/kernel/cet_prctl.c
+++ b/arch/x86/kernel/cet_prctl.c
@@ -22,6 +22,9 @@ static int copy_status_to_user(struct cet_status *cet, u64 arg2)
buf[2] = (u64)cet->shstk_size;
}
+ if (cet->ibt_enabled)
+ buf[0] |= GNU_PROPERTY_X86_FEATURE_1_IBT;
+
return copy_to_user((u64 __user *)arg2, buf, sizeof(buf));
}
@@ -42,7 +45,8 @@ int prctl_cet(int option, u64 arg2)
if (option == ARCH_X86_CET_STATUS)
return copy_status_to_user(cet, arg2);
- if (!static_cpu_has(X86_FEATURE_SHSTK))
+ if (!static_cpu_has(X86_FEATURE_SHSTK) &&
+ !static_cpu_has(X86_FEATURE_IBT))
return -EOPNOTSUPP;
switch (option) {
@@ -56,6 +60,8 @@ int prctl_cet(int option, u64 arg2)
return -EINVAL;
if (features & GNU_PROPERTY_X86_FEATURE_1_SHSTK)
cet_disable_shstk();
+ if (features & GNU_PROPERTY_X86_FEATURE_1_IBT)
+ cet_disable_ibt();
return 0;
case ARCH_X86_CET_LOCK:
--
2.21.0
On 10/12/2020 12:15 PM, Cyrill Gorcunov wrote:
> On Mon, Oct 12, 2020 at 08:45:24AM -0700, Yu-cheng Yu wrote:
> ...
>> + the application support it. When this feature is enabled,
>> + legacy non-IBT applications continue to work, but without
>> + IBT protection.
>> + Support for this feature is only known to be present on
>> + processors released in 2020 or later. CET features are also
>> + known to increase kernel text size by 3.7 KB.
>
> It seems the last sentence is redundant - new features always bloat
> the kernel code and precise size may differ depending on compiler
> and options. Surely this can be patched on top.
>
This was added after some discussion in v12 about kernel text size [1].
I think these few extra words can help people who have older machines
and want to save some space. Yes, like you said, if later we don't want
this sentence, I will be happy to add a patch to remove it. Thanks for
the feedback.
Yu-cheng
[1]
https://lore.kernel.org/linux-api/[email protected]/