2018-02-07 07:01:35

by Martin Schwidefsky

[permalink] [raw]
Subject: [PATCH 0/6] s390: improve speculative execution handling v3

Version 3 of the speculative execution improvements for s390.

Changes to v2:

* Dropped the prctl to introduce the PR_ISOLATE_BP control and simply
added two exported functions s390_isolate_bp and s390_isolate_bp_guest.
There is currently no caller for these functions, for now an out-of-tree
module can be used until an acceptable upstream solution for the user
space interface is found.

* Added an optimized version for the the array_index_mask_nospec
function based on subtract with borrow for the spectre v1 defense.

* Introduce "expoline", the s390 version of a retpoline. As s390 does
not have a return instruction and the associate return stack we use
an execute-type instruction on an indirect branch to get unpredicatable
branches. This requires gcc support for -mindirect-branch=thunk /
-mfunction-return=thunk. To be able to disable expolines there is
another gcc option -mindirect-branch-table to keep a list of PC relative
locations of calls to the execute thunks. With spectre_v2=off the call
will be replaced with the original indirect branch and a nop.

Martin Schwidefsky (6):
s390: scrub registers on kernel entry and KVM exit
s390: add optimized array_index_mask_nospec
s390/alternative: use a copy of the facility bit mask
s390: add options to change branch prediction behaviour for the kernel
s390: run user space and KVM guests with modified branch prediction
s390: introduce execute-trampolines for branches

arch/s390/Kconfig | 45 ++++++
arch/s390/Makefile | 12 ++
arch/s390/include/asm/barrier.h | 24 ++++
arch/s390/include/asm/facility.h | 18 +++
arch/s390/include/asm/lowcore.h | 9 +-
arch/s390/include/asm/nospec-branch.h | 18 +++
arch/s390/include/asm/processor.h | 4 +
arch/s390/include/asm/thread_info.h | 4 +
arch/s390/kernel/Makefile | 4 +
arch/s390/kernel/alternative.c | 26 +++-
arch/s390/kernel/early.c | 5 +
arch/s390/kernel/entry.S | 249 ++++++++++++++++++++++++++++++----
arch/s390/kernel/ipl.c | 1 +
arch/s390/kernel/module.c | 62 +++++++--
arch/s390/kernel/nospec-branch.c | 100 ++++++++++++++
arch/s390/kernel/processor.c | 18 +++
arch/s390/kernel/setup.c | 8 +-
arch/s390/kernel/smp.c | 7 +-
arch/s390/kernel/vmlinux.lds.S | 14 ++
drivers/s390/char/Makefile | 2 +
20 files changed, 591 insertions(+), 39 deletions(-)
create mode 100644 arch/s390/include/asm/nospec-branch.h
create mode 100644 arch/s390/kernel/nospec-branch.c

--
2.7.4



2018-02-07 07:01:48

by Martin Schwidefsky

[permalink] [raw]
Subject: [PATCH 4/6] s390: add options to change branch prediction behaviour for the kernel

Add the PPA instruction to the system entry and exit path to switch
the kernel to a different branch prediction behaviour. The instructions
are added via CPU alternatives and can be disabled with the "nospec"
or the "nobp=0" kernel parameter. If the default behaviour selected
with CONFIG_KERNEL_NOBP is set to "n" then the "nobp=1" parameter can be
used to enable the changed kernel branch prediction.

Acked-by: Cornelia Huck <[email protected]>
Signed-off-by: Martin Schwidefsky <[email protected]>
---
arch/s390/Kconfig | 17 ++++++++++++++
arch/s390/include/asm/processor.h | 1 +
arch/s390/kernel/alternative.c | 23 +++++++++++++++++++
arch/s390/kernel/early.c | 2 ++
arch/s390/kernel/entry.S | 48 +++++++++++++++++++++++++++++++++++++++
arch/s390/kernel/ipl.c | 1 +
arch/s390/kernel/smp.c | 2 ++
7 files changed, 94 insertions(+)

diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index 0105ce2..d514e25 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -540,6 +540,23 @@ config ARCH_RANDOM

If unsure, say Y.

+config KERNEL_NOBP
+ def_bool n
+ prompt "Enable modified branch prediction for the kernel by default"
+ help
+ If this option is selected the kernel will switch to a modified
+ branch prediction mode if the firmware interface is available.
+ The modified branch prediction mode improves the behaviour in
+ regard to speculative execution.
+
+ With the option enabled the kernel parameter "nobp=0" or "nospec"
+ can be used to run the kernel in the normal branch prediction mode.
+
+ With the option disabled the modified branch prediction mode is
+ enabled with the "nobp=1" kernel parameter.
+
+ If unsure, say N.
+
endmenu

menu "Memory setup"
diff --git a/arch/s390/include/asm/processor.h b/arch/s390/include/asm/processor.h
index bfbfad4..5f37f9c 100644
--- a/arch/s390/include/asm/processor.h
+++ b/arch/s390/include/asm/processor.h
@@ -91,6 +91,7 @@ void cpu_detect_mhz_feature(void);
extern const struct seq_operations cpuinfo_op;
extern int sysctl_ieee_emulation_warnings;
extern void execve_tail(void);
+extern void __bpon(void);

/*
* User space process size: 2GB for 31 bit, 4TB or 8PT for 64 bit.
diff --git a/arch/s390/kernel/alternative.c b/arch/s390/kernel/alternative.c
index 1abf4f3..2247613 100644
--- a/arch/s390/kernel/alternative.c
+++ b/arch/s390/kernel/alternative.c
@@ -15,6 +15,29 @@ static int __init disable_alternative_instructions(char *str)

early_param("noaltinstr", disable_alternative_instructions);

+static int __init nobp_setup_early(char *str)
+{
+ bool enabled;
+ int rc;
+
+ rc = kstrtobool(str, &enabled);
+ if (rc)
+ return rc;
+ if (enabled && test_facility(82))
+ __set_facility(82, S390_lowcore.alt_stfle_fac_list);
+ else
+ __clear_facility(82, S390_lowcore.alt_stfle_fac_list);
+ return 0;
+}
+early_param("nobp", nobp_setup_early);
+
+static int __init nospec_setup_early(char *str)
+{
+ __clear_facility(82, S390_lowcore.alt_stfle_fac_list);
+ return 0;
+}
+early_param("nospec", nospec_setup_early);
+
struct brcl_insn {
u16 opc;
s32 disp;
diff --git a/arch/s390/kernel/early.c b/arch/s390/kernel/early.c
index 510f218..ac707a9 100644
--- a/arch/s390/kernel/early.c
+++ b/arch/s390/kernel/early.c
@@ -196,6 +196,8 @@ static noinline __init void setup_facility_list(void)
memcpy(S390_lowcore.alt_stfle_fac_list,
S390_lowcore.stfle_fac_list,
sizeof(S390_lowcore.alt_stfle_fac_list));
+ if (!IS_ENABLED(CONFIG_KERNEL_NOBP))
+ __clear_facility(82, S390_lowcore.alt_stfle_fac_list);
}

static __init void detect_diag9c(void)
diff --git a/arch/s390/kernel/entry.S b/arch/s390/kernel/entry.S
index 5d87eda..e6d7550 100644
--- a/arch/s390/kernel/entry.S
+++ b/arch/s390/kernel/entry.S
@@ -159,6 +159,34 @@ _PIF_WORK = (_PIF_PER_TRAP | _PIF_SYSCALL_RESTART)
tm off+\addr, \mask
.endm

+ .macro BPOFF
+ .pushsection .altinstr_replacement, "ax"
+660: .long 0xb2e8c000
+ .popsection
+661: .long 0x47000000
+ .pushsection .altinstructions, "a"
+ .long 661b - .
+ .long 660b - .
+ .word 82
+ .byte 4
+ .byte 4
+ .popsection
+ .endm
+
+ .macro BPON
+ .pushsection .altinstr_replacement, "ax"
+662: .long 0xb2e8d000
+ .popsection
+663: .long 0x47000000
+ .pushsection .altinstructions, "a"
+ .long 663b - .
+ .long 662b - .
+ .word 82
+ .byte 4
+ .byte 4
+ .popsection
+ .endm
+
.section .kprobes.text, "ax"
.Ldummy:
/*
@@ -171,6 +199,11 @@ _PIF_WORK = (_PIF_PER_TRAP | _PIF_SYSCALL_RESTART)
*/
nop 0

+ENTRY(__bpon)
+ .globl __bpon
+ BPON
+ br %r14
+
/*
* Scheduler resume function, called by switch_to
* gpr2 = (task_struct *) prev
@@ -226,8 +259,11 @@ ENTRY(sie64a)
jnz .Lsie_skip
TSTMSK __LC_CPU_FLAGS,_CIF_FPU
jo .Lsie_skip # exit if fp/vx regs changed
+ BPON
.Lsie_entry:
sie 0(%r14)
+.Lsie_exit:
+ BPOFF
.Lsie_skip:
ni __SIE_PROG0C+3(%r14),0xfe # no longer in SIE
lctlg %c1,%c1,__LC_USER_ASCE # load primary asce
@@ -279,6 +315,7 @@ ENTRY(system_call)
stpt __LC_SYNC_ENTER_TIMER
.Lsysc_stmg:
stmg %r8,%r15,__LC_SAVE_AREA_SYNC
+ BPOFF
lg %r12,__LC_CURRENT
lghi %r13,__TASK_thread
lghi %r14,_PIF_SYSCALL
@@ -325,6 +362,7 @@ ENTRY(system_call)
jnz .Lsysc_work # check for work
TSTMSK __LC_CPU_FLAGS,_CIF_WORK
jnz .Lsysc_work
+ BPON
.Lsysc_restore:
lg %r14,__LC_VDSO_PER_CPU
lmg %r0,%r10,__PT_R0(%r11)
@@ -530,6 +568,7 @@ ENTRY(kernel_thread_starter)

ENTRY(pgm_check_handler)
stpt __LC_SYNC_ENTER_TIMER
+ BPOFF
stmg %r8,%r15,__LC_SAVE_AREA_SYNC
lg %r10,__LC_LAST_BREAK
lg %r12,__LC_CURRENT
@@ -637,6 +676,7 @@ ENTRY(pgm_check_handler)
ENTRY(io_int_handler)
STCK __LC_INT_CLOCK
stpt __LC_ASYNC_ENTER_TIMER
+ BPOFF
stmg %r8,%r15,__LC_SAVE_AREA_ASYNC
lg %r12,__LC_CURRENT
larl %r13,cleanup_critical
@@ -687,9 +727,13 @@ ENTRY(io_int_handler)
lg %r14,__LC_VDSO_PER_CPU
lmg %r0,%r10,__PT_R0(%r11)
mvc __LC_RETURN_PSW(16),__PT_PSW(%r11)
+ tm __PT_PSW+1(%r11),0x01 # returning to user ?
+ jno .Lio_exit_kernel
+ BPON
.Lio_exit_timer:
stpt __LC_EXIT_TIMER
mvc __VDSO_ECTG_BASE(16,%r14),__LC_EXIT_TIMER
+.Lio_exit_kernel:
lmg %r11,%r15,__PT_R11(%r11)
lpswe __LC_RETURN_PSW
.Lio_done:
@@ -860,6 +904,7 @@ ENTRY(io_int_handler)
ENTRY(ext_int_handler)
STCK __LC_INT_CLOCK
stpt __LC_ASYNC_ENTER_TIMER
+ BPOFF
stmg %r8,%r15,__LC_SAVE_AREA_ASYNC
lg %r12,__LC_CURRENT
larl %r13,cleanup_critical
@@ -908,6 +953,7 @@ ENTRY(psw_idle)
.Lpsw_idle_stcctm:
#endif
oi __LC_CPU_FLAGS+7,_CIF_ENABLED_WAIT
+ BPON
STCK __CLOCK_IDLE_ENTER(%r2)
stpt __TIMER_IDLE_ENTER(%r2)
.Lpsw_idle_lpsw:
@@ -1008,6 +1054,7 @@ load_fpu_regs:
*/
ENTRY(mcck_int_handler)
STCK __LC_MCCK_CLOCK
+ BPOFF
la %r1,4095 # validate r1
spt __LC_CPU_TIMER_SAVE_AREA-4095(%r1) # validate cpu timer
sckc __LC_CLOCK_COMPARATOR # validate comparator
@@ -1118,6 +1165,7 @@ ENTRY(mcck_int_handler)
mvc __LC_RETURN_MCCK_PSW(16),__PT_PSW(%r11) # move return PSW
tm __LC_RETURN_MCCK_PSW+1,0x01 # returning to user ?
jno 0f
+ BPON
stpt __LC_EXIT_TIMER
mvc __VDSO_ECTG_BASE(16,%r14),__LC_EXIT_TIMER
0: lmg %r11,%r15,__PT_R11(%r11)
diff --git a/arch/s390/kernel/ipl.c b/arch/s390/kernel/ipl.c
index da5cc3b..34477c1 100644
--- a/arch/s390/kernel/ipl.c
+++ b/arch/s390/kernel/ipl.c
@@ -543,6 +543,7 @@ static struct kset *ipl_kset;

static void __ipl_run(void *unused)
{
+ __bpon();
diag308(DIAG308_LOAD_CLEAR, NULL);
if (MACHINE_IS_VM)
__cpcmd("IPL", NULL, 0, NULL);
diff --git a/arch/s390/kernel/smp.c b/arch/s390/kernel/smp.c
index fc28c95..2fd7d60 100644
--- a/arch/s390/kernel/smp.c
+++ b/arch/s390/kernel/smp.c
@@ -319,6 +319,7 @@ static void pcpu_delegate(struct pcpu *pcpu, void (*func)(void *),
mem_assign_absolute(lc->restart_fn, (unsigned long) func);
mem_assign_absolute(lc->restart_data, (unsigned long) data);
mem_assign_absolute(lc->restart_source, source_cpu);
+ __bpon();
asm volatile(
"0: sigp 0,%0,%2 # sigp restart to target cpu\n"
" brc 2,0b # busy, try again\n"
@@ -903,6 +904,7 @@ void __cpu_die(unsigned int cpu)
void __noreturn cpu_die(void)
{
idle_task_exit();
+ __bpon();
pcpu_sigp_retry(pcpu_devices + smp_processor_id(), SIGP_STOP, 0);
for (;;) ;
}
--
2.7.4


2018-02-07 07:02:14

by Martin Schwidefsky

[permalink] [raw]
Subject: [PATCH 5/6] s390: run user space and KVM guests with modified branch prediction

Define TIF_ISOLATE_BP and TIF_ISOLATE_BP_GUEST and add the necessary
plumbing in entry.S to be able to run user space and KVM guests with
limited branch prediction.

To switch a user space process to limited branch prediction the
s390_isolate_bp() function has to be call, and to run a vCPU of a KVM
guest associated with the current task with limited branch prediction
call s390_isolate_bp_guest().

Signed-off-by: Martin Schwidefsky <[email protected]>
---
arch/s390/include/asm/processor.h | 3 +++
arch/s390/include/asm/thread_info.h | 4 +++
arch/s390/kernel/entry.S | 51 +++++++++++++++++++++++++++++++++----
arch/s390/kernel/processor.c | 18 +++++++++++++
4 files changed, 71 insertions(+), 5 deletions(-)

diff --git a/arch/s390/include/asm/processor.h b/arch/s390/include/asm/processor.h
index 5f37f9c..7f2953c 100644
--- a/arch/s390/include/asm/processor.h
+++ b/arch/s390/include/asm/processor.h
@@ -378,6 +378,9 @@ extern void memcpy_absolute(void *, void *, size_t);
memcpy_absolute(&(dest), &__tmp, sizeof(__tmp)); \
} while (0)

+extern int s390_isolate_bp(void);
+extern int s390_isolate_bp_guest(void);
+
#endif /* __ASSEMBLY__ */

#endif /* __ASM_S390_PROCESSOR_H */
diff --git a/arch/s390/include/asm/thread_info.h b/arch/s390/include/asm/thread_info.h
index 25d6ec3..83ba575 100644
--- a/arch/s390/include/asm/thread_info.h
+++ b/arch/s390/include/asm/thread_info.h
@@ -58,6 +58,8 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src);
#define TIF_GUARDED_STORAGE 4 /* load guarded storage control block */
#define TIF_PATCH_PENDING 5 /* pending live patching update */
#define TIF_PGSTE 6 /* New mm's will use 4K page tables */
+#define TIF_ISOLATE_BP 8 /* Run process with isolated BP */
+#define TIF_ISOLATE_BP_GUEST 9 /* Run KVM guests with isolated BP */

#define TIF_31BIT 16 /* 32bit process */
#define TIF_MEMDIE 17 /* is terminating due to OOM killer */
@@ -78,6 +80,8 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src);
#define _TIF_UPROBE _BITUL(TIF_UPROBE)
#define _TIF_GUARDED_STORAGE _BITUL(TIF_GUARDED_STORAGE)
#define _TIF_PATCH_PENDING _BITUL(TIF_PATCH_PENDING)
+#define _TIF_ISOLATE_BP _BITUL(TIF_ISOLATE_BP)
+#define _TIF_ISOLATE_BP_GUEST _BITUL(TIF_ISOLATE_BP_GUEST)

#define _TIF_31BIT _BITUL(TIF_31BIT)
#define _TIF_SINGLE_STEP _BITUL(TIF_SINGLE_STEP)
diff --git a/arch/s390/kernel/entry.S b/arch/s390/kernel/entry.S
index e6d7550..53145b5 100644
--- a/arch/s390/kernel/entry.S
+++ b/arch/s390/kernel/entry.S
@@ -107,6 +107,7 @@ _PIF_WORK = (_PIF_PER_TRAP | _PIF_SYSCALL_RESTART)
aghi %r15,-(STACK_FRAME_OVERHEAD + __PT_SIZE)
j 3f
1: UPDATE_VTIME %r14,%r15,\timer
+ BPENTER __TI_flags(%r12),_TIF_ISOLATE_BP
2: lg %r15,__LC_ASYNC_STACK # load async stack
3: la %r11,STACK_FRAME_OVERHEAD(%r15)
.endm
@@ -187,6 +188,40 @@ _PIF_WORK = (_PIF_PER_TRAP | _PIF_SYSCALL_RESTART)
.popsection
.endm

+ .macro BPENTER tif_ptr,tif_mask
+ .pushsection .altinstr_replacement, "ax"
+662: .word 0xc004, 0x0000, 0x0000 # 6 byte nop
+ .word 0xc004, 0x0000, 0x0000 # 6 byte nop
+ .popsection
+664: TSTMSK \tif_ptr,\tif_mask
+ jz . + 8
+ .long 0xb2e8d000
+ .pushsection .altinstructions, "a"
+ .long 664b - .
+ .long 662b - .
+ .word 82
+ .byte 12
+ .byte 12
+ .popsection
+ .endm
+
+ .macro BPEXIT tif_ptr,tif_mask
+ TSTMSK \tif_ptr,\tif_mask
+ .pushsection .altinstr_replacement, "ax"
+662: jnz . + 8
+ .long 0xb2e8d000
+ .popsection
+664: jz . + 8
+ .long 0xb2e8c000
+ .pushsection .altinstructions, "a"
+ .long 664b - .
+ .long 662b - .
+ .word 82
+ .byte 8
+ .byte 8
+ .popsection
+ .endm
+
.section .kprobes.text, "ax"
.Ldummy:
/*
@@ -240,9 +275,11 @@ ENTRY(__switch_to)
*/
ENTRY(sie64a)
stmg %r6,%r14,__SF_GPRS(%r15) # save kernel registers
+ lg %r12,__LC_CURRENT
stg %r2,__SF_EMPTY(%r15) # save control block pointer
stg %r3,__SF_EMPTY+8(%r15) # save guest register save area
xc __SF_EMPTY+16(8,%r15),__SF_EMPTY+16(%r15) # reason code = 0
+ mvc __SF_EMPTY+24(8,%r15),__TI_flags(%r12) # copy thread flags
TSTMSK __LC_CPU_FLAGS,_CIF_FPU # load guest fp/vx registers ?
jno .Lsie_load_guest_gprs
brasl %r14,load_fpu_regs # load guest fp/vx regs
@@ -259,11 +296,12 @@ ENTRY(sie64a)
jnz .Lsie_skip
TSTMSK __LC_CPU_FLAGS,_CIF_FPU
jo .Lsie_skip # exit if fp/vx regs changed
- BPON
+ BPEXIT __SF_EMPTY+24(%r15),(_TIF_ISOLATE_BP|_TIF_ISOLATE_BP_GUEST)
.Lsie_entry:
sie 0(%r14)
.Lsie_exit:
BPOFF
+ BPENTER __SF_EMPTY+24(%r15),(_TIF_ISOLATE_BP|_TIF_ISOLATE_BP_GUEST)
.Lsie_skip:
ni __SIE_PROG0C+3(%r14),0xfe # no longer in SIE
lctlg %c1,%c1,__LC_USER_ASCE # load primary asce
@@ -324,6 +362,7 @@ ENTRY(system_call)
la %r11,STACK_FRAME_OVERHEAD(%r15) # pointer to pt_regs
.Lsysc_vtime:
UPDATE_VTIME %r8,%r9,__LC_SYNC_ENTER_TIMER
+ BPENTER __TI_flags(%r12),_TIF_ISOLATE_BP
stmg %r0,%r7,__PT_R0(%r11)
# clear user controlled register to prevent speculative use
xgr %r0,%r0
@@ -362,7 +401,7 @@ ENTRY(system_call)
jnz .Lsysc_work # check for work
TSTMSK __LC_CPU_FLAGS,_CIF_WORK
jnz .Lsysc_work
- BPON
+ BPEXIT __TI_flags(%r12),_TIF_ISOLATE_BP
.Lsysc_restore:
lg %r14,__LC_VDSO_PER_CPU
lmg %r0,%r10,__PT_R0(%r11)
@@ -597,6 +636,7 @@ ENTRY(pgm_check_handler)
aghi %r15,-(STACK_FRAME_OVERHEAD + __PT_SIZE)
j 4f
2: UPDATE_VTIME %r14,%r15,__LC_SYNC_ENTER_TIMER
+ BPENTER __TI_flags(%r12),_TIF_ISOLATE_BP
lg %r15,__LC_KERNEL_STACK
lgr %r14,%r12
aghi %r14,__TASK_thread # pointer to thread_struct
@@ -729,7 +769,7 @@ ENTRY(io_int_handler)
mvc __LC_RETURN_PSW(16),__PT_PSW(%r11)
tm __PT_PSW+1(%r11),0x01 # returning to user ?
jno .Lio_exit_kernel
- BPON
+ BPEXIT __TI_flags(%r12),_TIF_ISOLATE_BP
.Lio_exit_timer:
stpt __LC_EXIT_TIMER
mvc __VDSO_ECTG_BASE(16,%r14),__LC_EXIT_TIMER
@@ -1165,7 +1205,7 @@ ENTRY(mcck_int_handler)
mvc __LC_RETURN_MCCK_PSW(16),__PT_PSW(%r11) # move return PSW
tm __LC_RETURN_MCCK_PSW+1,0x01 # returning to user ?
jno 0f
- BPON
+ BPEXIT __TI_flags(%r12),_TIF_ISOLATE_BP
stpt __LC_EXIT_TIMER
mvc __VDSO_ECTG_BASE(16,%r14),__LC_EXIT_TIMER
0: lmg %r11,%r15,__PT_R11(%r11)
@@ -1292,7 +1332,8 @@ cleanup_critical:
clg %r9,BASED(.Lsie_crit_mcck_length)
jh 1f
oi __LC_CPU_FLAGS+7, _CIF_MCCK_GUEST
-1: lg %r9,__SF_EMPTY(%r15) # get control block pointer
+1: BPENTER __SF_EMPTY+24(%r15),(_TIF_ISOLATE_BP|_TIF_ISOLATE_BP_GUEST)
+ lg %r9,__SF_EMPTY(%r15) # get control block pointer
ni __SIE_PROG0C+3(%r9),0xfe # no longer in SIE
lctlg %c1,%c1,__LC_USER_ASCE # load primary asce
larl %r9,sie_exit # skip forward to sie_exit
diff --git a/arch/s390/kernel/processor.c b/arch/s390/kernel/processor.c
index 5362fd8..6fe2e18 100644
--- a/arch/s390/kernel/processor.c
+++ b/arch/s390/kernel/processor.c
@@ -197,3 +197,21 @@ const struct seq_operations cpuinfo_op = {
.stop = c_stop,
.show = show_cpuinfo,
};
+
+int s390_isolate_bp(void)
+{
+ if (!test_facility(82))
+ return -EOPNOTSUPP;
+ set_thread_flag(TIF_ISOLATE_BP);
+ return 0;
+}
+EXPORT_SYMBOL(s390_isolate_bp);
+
+int s390_isolate_bp_guest(void)
+{
+ if (!test_facility(82))
+ return -EOPNOTSUPP;
+ set_thread_flag(TIF_ISOLATE_BP_GUEST);
+ return 0;
+}
+EXPORT_SYMBOL(s390_isolate_bp_guest);
--
2.7.4


2018-02-07 07:02:32

by Martin Schwidefsky

[permalink] [raw]
Subject: [PATCH 6/6] s390: introduce execute-trampolines for branches

Add CONFIG_EXPOLINE to enable the use of the new -mindirect-branch= and
-mfunction_return= compiler options to create a kernel fortified against
the specte v2 attack.

With CONFIG_EXPOLINE=y all indirect branches will be issued with an
execute type instruction. For z10 or newer the EXRL instruction will
be used, for older machines the EX instruction. The typical indirect
call

basr %r14,%r1

is replaced with a PC relative call to a new thunk

brasl %r14,__s390x_indirect_jump_r1

The thunk contains the EXRL/EX instruction to the indirect branch

__s390x_indirect_jump_r1:
exrl 0,0f
j .
0: br %r1

The detour via the execute type instruction has a performance impact.
To get rid of the detour the new kernel parameter "nospectre_v2" and
"spectre_v2=[on,off,auto]" can be used. If the parameter is specified
the kernel and module code will be patched at runtime.

Signed-off-by: Martin Schwidefsky <[email protected]>
---
arch/s390/Kconfig | 28 +++++++++
arch/s390/Makefile | 12 ++++
arch/s390/include/asm/lowcore.h | 6 +-
arch/s390/include/asm/nospec-branch.h | 18 ++++++
arch/s390/kernel/Makefile | 4 ++
arch/s390/kernel/entry.S | 113 ++++++++++++++++++++++++++--------
arch/s390/kernel/module.c | 62 ++++++++++++++++---
arch/s390/kernel/nospec-branch.c | 100 ++++++++++++++++++++++++++++++
arch/s390/kernel/setup.c | 4 ++
arch/s390/kernel/smp.c | 1 +
arch/s390/kernel/vmlinux.lds.S | 14 +++++
drivers/s390/char/Makefile | 2 +
12 files changed, 329 insertions(+), 35 deletions(-)
create mode 100644 arch/s390/include/asm/nospec-branch.h
create mode 100644 arch/s390/kernel/nospec-branch.c

diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index d514e25..d4a65bf 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -557,6 +557,34 @@ config KERNEL_NOBP

If unsure, say N.

+config EXPOLINE
+ def_bool n
+ prompt "Avoid speculative indirect branches in the kernel"
+ help
+ Compile the kernel with the expoline compiler options to guard
+ against kernel-to-user data leaks by avoiding speculative indirect
+ branches.
+ Requires a compiler with -mindirect-branch=thunk support for full
+ protection. The kernel may run slower.
+
+ If unsure, say N.
+
+choice
+ prompt "Expoline default"
+ depends on EXPOLINE
+ default EXPOLINE_FULL
+
+config EXPOLINE_OFF
+ bool "spectre_v2=off"
+
+config EXPOLINE_MEDIUM
+ bool "spectre_v2=auto"
+
+config EXPOLINE_FULL
+ bool "spectre_v2=on"
+
+endchoice
+
endmenu

menu "Memory setup"
diff --git a/arch/s390/Makefile b/arch/s390/Makefile
index fd691c4..2f925ef 100644
--- a/arch/s390/Makefile
+++ b/arch/s390/Makefile
@@ -78,6 +78,18 @@ ifeq ($(call cc-option-yn,-mwarn-dynamicstack),y)
cflags-$(CONFIG_WARN_DYNAMIC_STACK) += -mwarn-dynamicstack
endif

+ifdef CONFIG_EXPOLINE
+ ifeq ($(call cc-option-yn,$(CC_FLAGS_MARCH) -mindirect-branch=thunk),y)
+ CC_FLAGS_EXPOLINE := -mindirect-branch=thunk
+ CC_FLAGS_EXPOLINE += -mfunction-return=thunk
+ CC_FLAGS_EXPOLINE += -mindirect-branch-table
+ export CC_FLAGS_EXPOLINE
+ cflags-y += $(CC_FLAGS_EXPOLINE)
+ else
+ $(warning "Your gcc lacks the -mindirect-branch= option")
+ endif
+endif
+
ifdef CONFIG_FUNCTION_TRACER
# make use of hotpatch feature if the compiler supports it
cc_hotpatch := -mhotpatch=0,3
diff --git a/arch/s390/include/asm/lowcore.h b/arch/s390/include/asm/lowcore.h
index c63986a..5bc8888 100644
--- a/arch/s390/include/asm/lowcore.h
+++ b/arch/s390/include/asm/lowcore.h
@@ -136,7 +136,11 @@ struct lowcore {
__u64 vdso_per_cpu_data; /* 0x03b8 */
__u64 machine_flags; /* 0x03c0 */
__u64 gmap; /* 0x03c8 */
- __u8 pad_0x03d0[0x0e00-0x03d0]; /* 0x03d0 */
+ __u8 pad_0x03d0[0x0400-0x03d0]; /* 0x03d0 */
+
+ /* br %r1 trampoline */
+ __u16 br_r1_trampoline; /* 0x0400 */
+ __u8 pad_0x0402[0x0e00-0x0402]; /* 0x0402 */

/*
* 0xe00 contains the address of the IPL Parameter Information
diff --git a/arch/s390/include/asm/nospec-branch.h b/arch/s390/include/asm/nospec-branch.h
new file mode 100644
index 0000000..7df48e5
--- /dev/null
+++ b/arch/s390/include/asm/nospec-branch.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_S390_EXPOLINE_H
+#define _ASM_S390_EXPOLINE_H
+
+#ifndef __ASSEMBLY__
+
+#include <linux/types.h>
+
+extern int nospec_call_disable;
+extern int nospec_return_disable;
+
+void nospec_init_branches(void);
+void nospec_call_revert(s32 *start, s32 *end);
+void nospec_return_revert(s32 *start, s32 *end);
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* _ASM_S390_EXPOLINE_H */
diff --git a/arch/s390/kernel/Makefile b/arch/s390/kernel/Makefile
index 909bce6..7f27e3d 100644
--- a/arch/s390/kernel/Makefile
+++ b/arch/s390/kernel/Makefile
@@ -29,6 +29,7 @@ UBSAN_SANITIZE_early.o := n
#
ifneq ($(CC_FLAGS_MARCH),-march=z900)
CFLAGS_REMOVE_als.o += $(CC_FLAGS_MARCH)
+CFLAGS_REMOVE_als.o += $(CC_FLAGS_EXPOLINE)
CFLAGS_als.o += -march=z900
AFLAGS_REMOVE_head.o += $(CC_FLAGS_MARCH)
AFLAGS_head.o += -march=z900
@@ -63,6 +64,9 @@ obj-y += entry.o reipl.o relocate_kernel.o kdebugfs.o alternative.o

extra-y += head.o head64.o vmlinux.lds

+obj-$(CONFIG_EXPOLINE) += nospec-branch.o
+CFLAGS_REMOVE_expoline.o += $(CC_FLAGS_EXPOLINE)
+
obj-$(CONFIG_MODULES) += module.o
obj-$(CONFIG_SMP) += smp.o
obj-$(CONFIG_SCHED_TOPOLOGY) += topology.o
diff --git a/arch/s390/kernel/entry.S b/arch/s390/kernel/entry.S
index 53145b5..13a133a 100644
--- a/arch/s390/kernel/entry.S
+++ b/arch/s390/kernel/entry.S
@@ -222,6 +222,68 @@ _PIF_WORK = (_PIF_PER_TRAP | _PIF_SYSCALL_RESTART)
.popsection
.endm

+#ifdef CONFIG_EXPOLINE
+
+ .macro GEN_BR_THUNK name,reg,tmp
+ .section .text.\name,"axG",@progbits,\name,comdat
+ .globl \name
+ .hidden \name
+ .type \name,@function
+\name:
+ .cfi_startproc
+#ifdef CONFIG_HAVE_MARCH_Z10_FEATURES
+ exrl 0,0f
+#else
+ larl \tmp,0f
+ ex 0,0(\tmp)
+#endif
+ j .
+0: br \reg
+ .cfi_endproc
+ .endm
+
+ GEN_BR_THUNK __s390x_indirect_jump_r1use_r9,%r9,%r1
+ GEN_BR_THUNK __s390x_indirect_jump_r1use_r14,%r14,%r1
+ GEN_BR_THUNK __s390x_indirect_jump_r11use_r14,%r14,%r11
+
+ .macro BASR_R14_R9
+0: brasl %r14,__s390x_indirect_jump_r1use_r9
+ .pushsection .s390_indirect_branches,"a",@progbits
+ .long 0b-.
+ .popsection
+ .endm
+
+ .macro BR_R1USE_R14
+0: jg __s390x_indirect_jump_r1use_r14
+ .pushsection .s390_indirect_branches,"a",@progbits
+ .long 0b-.
+ .popsection
+ .endm
+
+ .macro BR_R11USE_R14
+0: jg __s390x_indirect_jump_r11use_r14
+ .pushsection .s390_indirect_branches,"a",@progbits
+ .long 0b-.
+ .popsection
+ .endm
+
+#else /* CONFIG_EXPOLINE */
+
+ .macro BASR_R14_R9
+ basr %r14,%r9
+ .endm
+
+ .macro BR_R1USE_R14
+ br %r14
+ .endm
+
+ .macro BR_R11USE_R14
+ br %r14
+ .endm
+
+#endif /* CONFIG_EXPOLINE */
+
+
.section .kprobes.text, "ax"
.Ldummy:
/*
@@ -237,7 +299,7 @@ _PIF_WORK = (_PIF_PER_TRAP | _PIF_SYSCALL_RESTART)
ENTRY(__bpon)
.globl __bpon
BPON
- br %r14
+ BR_R1USE_R14

/*
* Scheduler resume function, called by switch_to
@@ -261,9 +323,9 @@ ENTRY(__switch_to)
mvc __LC_CURRENT_PID(4,%r0),0(%r3) # store pid of next
lmg %r6,%r15,__SF_GPRS(%r15) # load gprs of next task
TSTMSK __LC_MACHINE_FLAGS,MACHINE_FLAG_LPP
- bzr %r14
+ jz 0f
.insn s,0xb2800000,__LC_LPP # set program parameter
- br %r14
+0: BR_R1USE_R14

.L__critical_start:

@@ -330,7 +392,7 @@ sie_exit:
xgr %r5,%r5
lmg %r6,%r14,__SF_GPRS(%r15) # restore kernel registers
lg %r2,__SF_EMPTY+16(%r15) # return exit reason code
- br %r14
+ BR_R1USE_R14
.Lsie_fault:
lghi %r14,-EFAULT
stg %r14,__SF_EMPTY+16(%r15) # set exit reason code
@@ -389,7 +451,7 @@ ENTRY(system_call)
lgf %r9,0(%r8,%r10) # get system call add.
TSTMSK __TI_flags(%r12),_TIF_TRACE
jnz .Lsysc_tracesys
- basr %r14,%r9 # call sys_xxxx
+ BASR_R14_R9 # call sys_xxxx
stg %r2,__PT_R2(%r11) # store return value

.Lsysc_return:
@@ -574,7 +636,7 @@ ENTRY(system_call)
lmg %r3,%r7,__PT_R3(%r11)
stg %r7,STACK_FRAME_OVERHEAD(%r15)
lg %r2,__PT_ORIG_GPR2(%r11)
- basr %r14,%r9 # call sys_xxx
+ BASR_R14_R9 # call sys_xxx
stg %r2,__PT_R2(%r11) # store return value
.Lsysc_tracenogo:
TSTMSK __TI_flags(%r12),_TIF_TRACE
@@ -598,7 +660,7 @@ ENTRY(ret_from_fork)
lmg %r9,%r10,__PT_R9(%r11) # load gprs
ENTRY(kernel_thread_starter)
la %r2,0(%r10)
- basr %r14,%r9
+ BASR_R14_R9
j .Lsysc_tracenogo

/*
@@ -678,9 +740,9 @@ ENTRY(pgm_check_handler)
nill %r10,0x007f
sll %r10,2
je .Lpgm_return
- lgf %r1,0(%r10,%r1) # load address of handler routine
+ lgf %r9,0(%r10,%r1) # load address of handler routine
lgr %r2,%r11 # pass pointer to pt_regs
- basr %r14,%r1 # branch to interrupt-handler
+ BASR_R14_R9 # branch to interrupt-handler
.Lpgm_return:
LOCKDEP_SYS_EXIT
tm __PT_PSW+1(%r11),0x01 # returning to user ?
@@ -998,7 +1060,7 @@ ENTRY(psw_idle)
stpt __TIMER_IDLE_ENTER(%r2)
.Lpsw_idle_lpsw:
lpswe __SF_EMPTY(%r15)
- br %r14
+ BR_R1USE_R14
.Lpsw_idle_end:

/*
@@ -1012,7 +1074,7 @@ ENTRY(save_fpu_regs)
lg %r2,__LC_CURRENT
aghi %r2,__TASK_thread
TSTMSK __LC_CPU_FLAGS,_CIF_FPU
- bor %r14
+ jo .Lsave_fpu_regs_exit
stfpc __THREAD_FPU_fpc(%r2)
lg %r3,__THREAD_FPU_regs(%r2)
TSTMSK __LC_MACHINE_FLAGS,MACHINE_FLAG_VX
@@ -1039,7 +1101,8 @@ ENTRY(save_fpu_regs)
std 15,120(%r3)
.Lsave_fpu_regs_done:
oi __LC_CPU_FLAGS+7,_CIF_FPU
- br %r14
+.Lsave_fpu_regs_exit:
+ BR_R1USE_R14
.Lsave_fpu_regs_end:
EXPORT_SYMBOL(save_fpu_regs)

@@ -1057,7 +1120,7 @@ load_fpu_regs:
lg %r4,__LC_CURRENT
aghi %r4,__TASK_thread
TSTMSK __LC_CPU_FLAGS,_CIF_FPU
- bnor %r14
+ jno .Lload_fpu_regs_exit
lfpc __THREAD_FPU_fpc(%r4)
TSTMSK __LC_MACHINE_FLAGS,MACHINE_FLAG_VX
lg %r4,__THREAD_FPU_regs(%r4) # %r4 <- reg save area
@@ -1084,7 +1147,8 @@ load_fpu_regs:
ld 15,120(%r4)
.Lload_fpu_regs_done:
ni __LC_CPU_FLAGS+7,255-_CIF_FPU
- br %r14
+.Lload_fpu_regs_exit:
+ BR_R1USE_R14
.Lload_fpu_regs_end:

.L__critical_end:
@@ -1301,7 +1365,7 @@ cleanup_critical:
jl 0f
clg %r9,BASED(.Lcleanup_table+104) # .Lload_fpu_regs_end
jl .Lcleanup_load_fpu_regs
-0: br %r14
+0: BR_R11USE_R14

.align 8
.Lcleanup_table:
@@ -1337,7 +1401,7 @@ cleanup_critical:
ni __SIE_PROG0C+3(%r9),0xfe # no longer in SIE
lctlg %c1,%c1,__LC_USER_ASCE # load primary asce
larl %r9,sie_exit # skip forward to sie_exit
- br %r14
+ BR_R11USE_R14
#endif

.Lcleanup_system_call:
@@ -1390,7 +1454,7 @@ cleanup_critical:
stg %r15,56(%r11) # r15 stack pointer
# set new psw address and exit
larl %r9,.Lsysc_do_svc
- br %r14
+ BR_R11USE_R14
.Lcleanup_system_call_insn:
.quad system_call
.quad .Lsysc_stmg
@@ -1402,7 +1466,7 @@ cleanup_critical:

.Lcleanup_sysc_tif:
larl %r9,.Lsysc_tif
- br %r14
+ BR_R11USE_R14

.Lcleanup_sysc_restore:
# check if stpt has been executed
@@ -1419,14 +1483,14 @@ cleanup_critical:
mvc 0(64,%r11),__PT_R8(%r9)
lmg %r0,%r7,__PT_R0(%r9)
1: lmg %r8,%r9,__LC_RETURN_PSW
- br %r14
+ BR_R11USE_R14
.Lcleanup_sysc_restore_insn:
.quad .Lsysc_exit_timer
.quad .Lsysc_done - 4

.Lcleanup_io_tif:
larl %r9,.Lio_tif
- br %r14
+ BR_R11USE_R14

.Lcleanup_io_restore:
# check if stpt has been executed
@@ -1440,7 +1504,7 @@ cleanup_critical:
mvc 0(64,%r11),__PT_R8(%r9)
lmg %r0,%r7,__PT_R0(%r9)
1: lmg %r8,%r9,__LC_RETURN_PSW
- br %r14
+ BR_R11USE_R14
.Lcleanup_io_restore_insn:
.quad .Lio_exit_timer
.quad .Lio_done - 4
@@ -1493,17 +1557,17 @@ cleanup_critical:
# prepare return psw
nihh %r8,0xfcfd # clear irq & wait state bits
lg %r9,48(%r11) # return from psw_idle
- br %r14
+ BR_R11USE_R14
.Lcleanup_idle_insn:
.quad .Lpsw_idle_lpsw

.Lcleanup_save_fpu_regs:
larl %r9,save_fpu_regs
- br %r14
+ BR_R11USE_R14

.Lcleanup_load_fpu_regs:
larl %r9,load_fpu_regs
- br %r14
+ BR_R11USE_R14

/*
* Integer constants
@@ -1523,7 +1587,6 @@ cleanup_critical:
.Lsie_crit_mcck_length:
.quad .Lsie_skip - .Lsie_entry
#endif
-
.section .rodata, "a"
#define SYSCALL(esame,emu) .long esame
.globl sys_call_table
diff --git a/arch/s390/kernel/module.c b/arch/s390/kernel/module.c
index b7abfad..1fc6d1f 100644
--- a/arch/s390/kernel/module.c
+++ b/arch/s390/kernel/module.c
@@ -19,6 +19,8 @@
#include <linux/moduleloader.h>
#include <linux/bug.h>
#include <asm/alternative.h>
+#include <asm/nospec-branch.h>
+#include <asm/facility.h>

#if 0
#define DEBUGP printk
@@ -156,7 +158,11 @@ int module_frob_arch_sections(Elf_Ehdr *hdr, Elf_Shdr *sechdrs,
me->arch.got_offset = me->core_layout.size;
me->core_layout.size += me->arch.got_size;
me->arch.plt_offset = me->core_layout.size;
- me->core_layout.size += me->arch.plt_size;
+ if (me->arch.plt_size) {
+ if (IS_ENABLED(CONFIG_EXPOLINE) && !nospec_call_disable)
+ me->arch.plt_size += PLT_ENTRY_SIZE;
+ me->core_layout.size += me->arch.plt_size;
+ }
return 0;
}

@@ -310,9 +316,21 @@ static int apply_rela(Elf_Rela *rela, Elf_Addr base, Elf_Sym *symtab,
unsigned int *ip;
ip = me->core_layout.base + me->arch.plt_offset +
info->plt_offset;
- ip[0] = 0x0d10e310; /* basr 1,0; lg 1,10(1); br 1 */
- ip[1] = 0x100a0004;
- ip[2] = 0x07f10000;
+ ip[0] = 0x0d10e310; /* basr 1,0 */
+ ip[1] = 0x100a0004; /* lg 1,10(1) */
+ if (IS_ENABLED(CONFIG_EXPOLINE) &&
+ !nospec_call_disable) {
+ unsigned int *ij;
+ ij = me->core_layout.base +
+ me->arch.plt_offset +
+ me->arch.plt_size - PLT_ENTRY_SIZE;
+ ip[2] = 0xa7f40000 + /* j __jump_r1 */
+ (unsigned int)(u16)
+ (((unsigned long) ij - 8 -
+ (unsigned long) ip) / 2);
+ } else {
+ ip[2] = 0x07f10000; /* br %r1 */
+ }
ip[3] = (unsigned int) (val >> 32);
ip[4] = (unsigned int) val;
info->plt_initialized = 1;
@@ -418,16 +436,42 @@ int module_finalize(const Elf_Ehdr *hdr,
struct module *me)
{
const Elf_Shdr *s;
- char *secstrings;
+ char *secstrings, *secname;
+ void *aseg;
+
+ if (IS_ENABLED(CONFIG_EXPOLINE) &&
+ !nospec_call_disable && me->arch.plt_size) {
+ unsigned int *ij;
+
+ ij = me->core_layout.base + me->arch.plt_offset +
+ me->arch.plt_size - PLT_ENTRY_SIZE;
+ if (test_facility(35)) {
+ ij[0] = 0xc6000000; /* exrl %r0,.+10 */
+ ij[1] = 0x0005a7f4; /* j . */
+ ij[2] = 0x000007f1; /* br %r1 */
+ } else {
+ ij[0] = 0x44000000 | (unsigned int)
+ offsetof(struct lowcore, br_r1_trampoline);
+ ij[1] = 0xa7f40000; /* j . */
+ }
+ }

secstrings = (void *)hdr + sechdrs[hdr->e_shstrndx].sh_offset;
for (s = sechdrs; s < sechdrs + hdr->e_shnum; s++) {
- if (!strcmp(".altinstructions", secstrings + s->sh_name)) {
- /* patch .altinstructions */
- void *aseg = (void *)s->sh_addr;
+ aseg = (void *) s->sh_addr;
+ secname = secstrings + s->sh_name;

+ if (!strcmp(".altinstructions", secname))
+ /* patch .altinstructions */
apply_alternatives(aseg, aseg + s->sh_size);
- }
+
+ if (IS_ENABLED(CONFIG_EXPOLINE) &&
+ (!strcmp(".nospec_call_table", secname)))
+ nospec_call_revert(aseg, aseg + s->sh_size);
+
+ if (IS_ENABLED(CONFIG_EXPOLINE) &&
+ (!strcmp(".nospec_return_table", secname)))
+ nospec_return_revert(aseg, aseg + s->sh_size);
}

jump_label_apply_nops(me);
diff --git a/arch/s390/kernel/nospec-branch.c b/arch/s390/kernel/nospec-branch.c
new file mode 100644
index 0000000..69d7fcf
--- /dev/null
+++ b/arch/s390/kernel/nospec-branch.c
@@ -0,0 +1,100 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/module.h>
+#include <asm/nospec-branch.h>
+
+int nospec_call_disable = IS_ENABLED(EXPOLINE_OFF);
+int nospec_return_disable = !IS_ENABLED(EXPOLINE_FULL);
+
+static int __init nospectre_v2_setup_early(char *str)
+{
+ nospec_call_disable = 1;
+ nospec_return_disable = 1;
+ return 0;
+}
+early_param("nospectre_v2", nospectre_v2_setup_early);
+
+static int __init spectre_v2_setup_early(char *str)
+{
+ if (str && !strncmp(str, "on", 2)) {
+ nospec_call_disable = 0;
+ nospec_return_disable = 0;
+ }
+ if (str && !strncmp(str, "off", 3)) {
+ nospec_call_disable = 1;
+ nospec_return_disable = 1;
+ }
+ if (str && !strncmp(str, "auto", 4)) {
+ nospec_call_disable = 0;
+ nospec_return_disable = 1;
+ }
+ return 0;
+}
+early_param("spectre_v2", spectre_v2_setup_early);
+
+static void __init_or_module __nospec_revert(s32 *start, s32 *end)
+{
+ enum { BRCL_EXPOLINE, BRASL_EXPOLINE } type;
+ u8 *instr, *thunk, *br;
+ u8 insnbuf[6];
+ s32 *epo;
+
+ /* Second part of the instruction replace is always a nop */
+ memcpy(insnbuf + 2, (char[]) { 0x47, 0x00, 0x00, 0x00 }, 4);
+ for (epo = start; epo < end; epo++) {
+ instr = (u8 *) epo + *epo;
+ if (instr[0] == 0xc0 && (instr[1] & 0x0f) == 0x04)
+ type = BRCL_EXPOLINE; /* brcl instruction */
+ else if (instr[0] == 0xc0 && (instr[1] & 0x0f) == 0x05)
+ type = BRASL_EXPOLINE; /* brasl instruction */
+ else
+ continue;
+ thunk = instr + (*(int *)(instr + 2)) * 2;
+ if (thunk[0] == 0xc6 && thunk[1] == 0x00)
+ /* exrl %r0,<target-br> */
+ br = thunk + (*(int *)(thunk + 2)) * 2;
+ else if (thunk[0] == 0xc0 && (thunk[1] & 0x0f) == 0x00 &&
+ thunk[6] == 0x44 && thunk[7] == 0x00 &&
+ (thunk[8] & 0x0f) == 0x00 && thunk[9] == 0x00 &&
+ (thunk[1] & 0xf0) == (thunk[8] & 0xf0))
+ /* larl %rx,<target br> + ex %r0,0(%rx) */
+ br = thunk + (*(int *)(thunk + 2)) * 2;
+ else
+ continue;
+ if (br[0] != 0x07 || (br[1] & 0xf0) != 0xf0)
+ continue;
+ switch (type) {
+ case BRCL_EXPOLINE:
+ /* brcl to thunk, replace with br + nop */
+ insnbuf[0] = br[0];
+ insnbuf[1] = (instr[1] & 0xf0) | (br[1] & 0x0f);
+ break;
+ case BRASL_EXPOLINE:
+ /* brasl to thunk, replace with basr + nop */
+ insnbuf[0] = 0x0d;
+ insnbuf[1] = (instr[1] & 0xf0) | (br[1] & 0x0f);
+ break;
+ }
+
+ s390_kernel_write(instr, insnbuf, 6);
+ }
+}
+
+void __init_or_module nospec_call_revert(s32 *start, s32 *end)
+{
+ if (nospec_call_disable)
+ __nospec_revert(start, end);
+}
+
+void __init_or_module nospec_return_revert(s32 *start, s32 *end)
+{
+ if (nospec_return_disable)
+ __nospec_revert(start, end);
+}
+
+extern s32 __nospec_call_start[], __nospec_call_end[];
+extern s32 __nospec_return_start[], __nospec_return_end[];
+void __init nospec_init_branches(void)
+{
+ nospec_call_revert(__nospec_call_start, __nospec_call_end);
+ nospec_return_revert(__nospec_return_start, __nospec_return_end);
+}
diff --git a/arch/s390/kernel/setup.c b/arch/s390/kernel/setup.c
index bcd2a4a..a6a91f0 100644
--- a/arch/s390/kernel/setup.c
+++ b/arch/s390/kernel/setup.c
@@ -68,6 +68,7 @@
#include <asm/sysinfo.h>
#include <asm/numa.h>
#include <asm/alternative.h>
+#include <asm/nospec-branch.h>
#include "entry.h"

/*
@@ -379,6 +380,7 @@ static void __init setup_lowcore(void)
lc->spinlock_index = 0;
arch_spin_lock_setup(0);
#endif
+ lc->br_r1_trampoline = 0x07f1; /* br %r1 */

set_prefix((u32)(unsigned long) lc);
lowcore_ptr[0] = lc;
@@ -954,6 +956,8 @@ void __init setup_arch(char **cmdline_p)
set_preferred_console();

apply_alternative_instructions();
+ if (IS_ENABLED(CONFIG_EXPOLINE))
+ nospec_init_branches();

/* Setup zfcpdump support */
setup_zfcpdump();
diff --git a/arch/s390/kernel/smp.c b/arch/s390/kernel/smp.c
index 2fd7d60..a4a9fe1 100644
--- a/arch/s390/kernel/smp.c
+++ b/arch/s390/kernel/smp.c
@@ -214,6 +214,7 @@ static int pcpu_alloc_lowcore(struct pcpu *pcpu, int cpu)
lc->cpu_nr = cpu;
lc->spinlock_lockval = arch_spin_lockval(cpu);
lc->spinlock_index = 0;
+ lc->br_r1_trampoline = 0x07f1; /* br %r1 */
if (nmi_alloc_per_cpu(lc))
goto out;
if (vdso_alloc_per_cpu(lc))
diff --git a/arch/s390/kernel/vmlinux.lds.S b/arch/s390/kernel/vmlinux.lds.S
index 608cf29..08d12cf 100644
--- a/arch/s390/kernel/vmlinux.lds.S
+++ b/arch/s390/kernel/vmlinux.lds.S
@@ -123,6 +123,20 @@ SECTIONS
*(.altinstr_replacement)
}

+ /*
+ * Table with the patch locations to undo expolines
+ */
+ .nospec_call_table : {
+ __nospec_call_start = . ;
+ *(.s390_indirect*)
+ __nospec_call_end = . ;
+ }
+ .nospec_return_table : {
+ __nospec_return_start = . ;
+ *(.s390_return*)
+ __nospec_return_end = . ;
+ }
+
/* early.c uses stsi, which requires page aligned data. */
. = ALIGN(PAGE_SIZE);
INIT_DATA_SECTION(0x100)
diff --git a/drivers/s390/char/Makefile b/drivers/s390/char/Makefile
index 614b44e..a2b33a2 100644
--- a/drivers/s390/char/Makefile
+++ b/drivers/s390/char/Makefile
@@ -19,6 +19,8 @@ endif

CFLAGS_sclp_early_core.o += -D__NO_FORTIFY

+CFLAGS_REMOVE_sclp_early_core.o += $(CC_FLAGS_EXPOLINE)
+
obj-y += ctrlchar.o keyboard.o defkeymap.o sclp.o sclp_rw.o sclp_quiesce.o \
sclp_cmd.o sclp_config.o sclp_cpi_sys.o sclp_ocf.o sclp_ctl.o \
sclp_early.o sclp_early_core.o
--
2.7.4


2018-02-07 07:02:59

by Martin Schwidefsky

[permalink] [raw]
Subject: [PATCH 1/6] s390: scrub registers on kernel entry and KVM exit

Clear all user space registers on entry to the kernel and all KVM guest
registers on KVM guest exit if the register does not contain either a
parameter or a result value.

Reviewed-by: Christian Borntraeger <[email protected]>
Signed-off-by: Martin Schwidefsky <[email protected]>
---
arch/s390/kernel/entry.S | 47 +++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 47 insertions(+)

diff --git a/arch/s390/kernel/entry.S b/arch/s390/kernel/entry.S
index 6cd444d..5d87eda 100644
--- a/arch/s390/kernel/entry.S
+++ b/arch/s390/kernel/entry.S
@@ -248,6 +248,12 @@ ENTRY(sie64a)
sie_exit:
lg %r14,__SF_EMPTY+8(%r15) # load guest register save area
stmg %r0,%r13,0(%r14) # save guest gprs 0-13
+ xgr %r0,%r0 # clear guest registers to
+ xgr %r1,%r1 # prevent speculative use
+ xgr %r2,%r2
+ xgr %r3,%r3
+ xgr %r4,%r4
+ xgr %r5,%r5
lmg %r6,%r14,__SF_GPRS(%r15) # restore kernel registers
lg %r2,__SF_EMPTY+16(%r15) # return exit reason code
br %r14
@@ -282,6 +288,8 @@ ENTRY(system_call)
.Lsysc_vtime:
UPDATE_VTIME %r8,%r9,__LC_SYNC_ENTER_TIMER
stmg %r0,%r7,__PT_R0(%r11)
+ # clear user controlled register to prevent speculative use
+ xgr %r0,%r0
mvc __PT_R8(64,%r11),__LC_SAVE_AREA_SYNC
mvc __PT_PSW(16,%r11),__LC_SVC_OLD_PSW
mvc __PT_INT_CODE(4,%r11),__LC_SVC_ILC
@@ -561,6 +569,15 @@ ENTRY(pgm_check_handler)
4: lgr %r13,%r11
la %r11,STACK_FRAME_OVERHEAD(%r15)
stmg %r0,%r7,__PT_R0(%r11)
+ # clear user controlled registers to prevent speculative use
+ xgr %r0,%r0
+ xgr %r1,%r1
+ xgr %r2,%r2
+ xgr %r3,%r3
+ xgr %r4,%r4
+ xgr %r5,%r5
+ xgr %r6,%r6
+ xgr %r7,%r7
mvc __PT_R8(64,%r11),__LC_SAVE_AREA_SYNC
stmg %r8,%r9,__PT_PSW(%r11)
mvc __PT_INT_CODE(4,%r11),__LC_PGM_ILC
@@ -626,6 +643,16 @@ ENTRY(io_int_handler)
lmg %r8,%r9,__LC_IO_OLD_PSW
SWITCH_ASYNC __LC_SAVE_AREA_ASYNC,__LC_ASYNC_ENTER_TIMER
stmg %r0,%r7,__PT_R0(%r11)
+ # clear user controlled registers to prevent speculative use
+ xgr %r0,%r0
+ xgr %r1,%r1
+ xgr %r2,%r2
+ xgr %r3,%r3
+ xgr %r4,%r4
+ xgr %r5,%r5
+ xgr %r6,%r6
+ xgr %r7,%r7
+ xgr %r10,%r10
mvc __PT_R8(64,%r11),__LC_SAVE_AREA_ASYNC
stmg %r8,%r9,__PT_PSW(%r11)
mvc __PT_INT_CODE(12,%r11),__LC_SUBCHANNEL_ID
@@ -839,6 +866,16 @@ ENTRY(ext_int_handler)
lmg %r8,%r9,__LC_EXT_OLD_PSW
SWITCH_ASYNC __LC_SAVE_AREA_ASYNC,__LC_ASYNC_ENTER_TIMER
stmg %r0,%r7,__PT_R0(%r11)
+ # clear user controlled registers to prevent speculative use
+ xgr %r0,%r0
+ xgr %r1,%r1
+ xgr %r2,%r2
+ xgr %r3,%r3
+ xgr %r4,%r4
+ xgr %r5,%r5
+ xgr %r6,%r6
+ xgr %r7,%r7
+ xgr %r10,%r10
mvc __PT_R8(64,%r11),__LC_SAVE_AREA_ASYNC
stmg %r8,%r9,__PT_PSW(%r11)
lghi %r1,__LC_EXT_PARAMS2
@@ -1046,6 +1083,16 @@ ENTRY(mcck_int_handler)
.Lmcck_skip:
lghi %r14,__LC_GPREGS_SAVE_AREA+64
stmg %r0,%r7,__PT_R0(%r11)
+ # clear user controlled registers to prevent speculative use
+ xgr %r0,%r0
+ xgr %r1,%r1
+ xgr %r2,%r2
+ xgr %r3,%r3
+ xgr %r4,%r4
+ xgr %r5,%r5
+ xgr %r6,%r6
+ xgr %r7,%r7
+ xgr %r10,%r10
mvc __PT_R8(64,%r11),0(%r14)
stmg %r8,%r9,__PT_PSW(%r11)
xc __PT_FLAGS(8,%r11),__PT_FLAGS(%r11)
--
2.7.4


2018-02-07 07:03:12

by Martin Schwidefsky

[permalink] [raw]
Subject: [PATCH 3/6] s390/alternative: use a copy of the facility bit mask

To be able to switch off specific CPU alternatives with kernel parameters
make a copy of the facility bit mask provided by STFLE and use the copy
for the decision to apply an alternative.

Reviewed-by: David Hildenbrand <[email protected]>
Reviewed-by: Cornelia Huck <[email protected]>
Signed-off-by: Martin Schwidefsky <[email protected]>
---
arch/s390/include/asm/facility.h | 18 ++++++++++++++++++
arch/s390/include/asm/lowcore.h | 3 ++-
arch/s390/kernel/alternative.c | 3 ++-
arch/s390/kernel/early.c | 3 +++
arch/s390/kernel/setup.c | 4 +++-
arch/s390/kernel/smp.c | 4 +++-
6 files changed, 31 insertions(+), 4 deletions(-)

diff --git a/arch/s390/include/asm/facility.h b/arch/s390/include/asm/facility.h
index fbe0c4b..99c8ce3 100644
--- a/arch/s390/include/asm/facility.h
+++ b/arch/s390/include/asm/facility.h
@@ -15,6 +15,24 @@

#define MAX_FACILITY_BIT (sizeof(((struct lowcore *)0)->stfle_fac_list) * 8)

+static inline void __set_facility(unsigned long nr, void *facilities)
+{
+ unsigned char *ptr = (unsigned char *) facilities;
+
+ if (nr >= MAX_FACILITY_BIT)
+ return;
+ ptr[nr >> 3] |= 0x80 >> (nr & 7);
+}
+
+static inline void __clear_facility(unsigned long nr, void *facilities)
+{
+ unsigned char *ptr = (unsigned char *) facilities;
+
+ if (nr >= MAX_FACILITY_BIT)
+ return;
+ ptr[nr >> 3] &= ~(0x80 >> (nr & 7));
+}
+
static inline int __test_facility(unsigned long nr, void *facilities)
{
unsigned char *ptr;
diff --git a/arch/s390/include/asm/lowcore.h b/arch/s390/include/asm/lowcore.h
index ec6592e..c63986a 100644
--- a/arch/s390/include/asm/lowcore.h
+++ b/arch/s390/include/asm/lowcore.h
@@ -151,7 +151,8 @@ struct lowcore {
__u8 pad_0x0e20[0x0f00-0x0e20]; /* 0x0e20 */

/* Extended facility list */
- __u64 stfle_fac_list[32]; /* 0x0f00 */
+ __u64 stfle_fac_list[16]; /* 0x0f00 */
+ __u64 alt_stfle_fac_list[16]; /* 0x0f80 */
__u8 pad_0x1000[0x11b0-0x1000]; /* 0x1000 */

/* Pointer to the machine check extended save area */
diff --git a/arch/s390/kernel/alternative.c b/arch/s390/kernel/alternative.c
index 574e776..1abf4f3 100644
--- a/arch/s390/kernel/alternative.c
+++ b/arch/s390/kernel/alternative.c
@@ -75,7 +75,8 @@ static void __init_or_module __apply_alternatives(struct alt_instr *start,
instr = (u8 *)&a->instr_offset + a->instr_offset;
replacement = (u8 *)&a->repl_offset + a->repl_offset;

- if (!test_facility(a->facility))
+ if (!__test_facility(a->facility,
+ S390_lowcore.alt_stfle_fac_list))
continue;

if (unlikely(a->instrlen % 2 || a->replacementlen % 2)) {
diff --git a/arch/s390/kernel/early.c b/arch/s390/kernel/early.c
index 497a920..510f218 100644
--- a/arch/s390/kernel/early.c
+++ b/arch/s390/kernel/early.c
@@ -193,6 +193,9 @@ static noinline __init void setup_facility_list(void)
{
stfle(S390_lowcore.stfle_fac_list,
ARRAY_SIZE(S390_lowcore.stfle_fac_list));
+ memcpy(S390_lowcore.alt_stfle_fac_list,
+ S390_lowcore.stfle_fac_list,
+ sizeof(S390_lowcore.alt_stfle_fac_list));
}

static __init void detect_diag9c(void)
diff --git a/arch/s390/kernel/setup.c b/arch/s390/kernel/setup.c
index 793da97..bcd2a4a 100644
--- a/arch/s390/kernel/setup.c
+++ b/arch/s390/kernel/setup.c
@@ -340,7 +340,9 @@ static void __init setup_lowcore(void)
lc->preempt_count = S390_lowcore.preempt_count;
lc->stfl_fac_list = S390_lowcore.stfl_fac_list;
memcpy(lc->stfle_fac_list, S390_lowcore.stfle_fac_list,
- MAX_FACILITY_BIT/8);
+ sizeof(lc->stfle_fac_list));
+ memcpy(lc->alt_stfle_fac_list, S390_lowcore.alt_stfle_fac_list,
+ sizeof(lc->alt_stfle_fac_list));
nmi_alloc_boot_cpu(lc);
vdso_alloc_boot_cpu(lc);
lc->sync_enter_timer = S390_lowcore.sync_enter_timer;
diff --git a/arch/s390/kernel/smp.c b/arch/s390/kernel/smp.c
index a919b2f..fc28c95 100644
--- a/arch/s390/kernel/smp.c
+++ b/arch/s390/kernel/smp.c
@@ -266,7 +266,9 @@ static void pcpu_prepare_secondary(struct pcpu *pcpu, int cpu)
__ctl_store(lc->cregs_save_area, 0, 15);
save_access_regs((unsigned int *) lc->access_regs_save_area);
memcpy(lc->stfle_fac_list, S390_lowcore.stfle_fac_list,
- MAX_FACILITY_BIT/8);
+ sizeof(lc->stfle_fac_list));
+ memcpy(lc->alt_stfle_fac_list, S390_lowcore.alt_stfle_fac_list,
+ sizeof(lc->alt_stfle_fac_list));
arch_spin_lock_setup(cpu);
}

--
2.7.4


2018-02-07 07:04:00

by Martin Schwidefsky

[permalink] [raw]
Subject: [PATCH 2/6] s390: add optimized array_index_mask_nospec

Add an optimized version of the array_index_mask_nospec function for
s390 based on a compare and a subtract with borrow.

Signed-off-by: Martin Schwidefsky <[email protected]>
---
arch/s390/include/asm/barrier.h | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)

diff --git a/arch/s390/include/asm/barrier.h b/arch/s390/include/asm/barrier.h
index 1043260..f9eddbc 100644
--- a/arch/s390/include/asm/barrier.h
+++ b/arch/s390/include/asm/barrier.h
@@ -49,6 +49,30 @@ do { \
#define __smp_mb__before_atomic() barrier()
#define __smp_mb__after_atomic() barrier()

+/**
+ * array_index_mask_nospec - generate a mask for array_idx() that is
+ * ~0UL when the bounds check succeeds and 0 otherwise
+ * @index: array element index
+ * @size: number of elements in array
+ */
+#define array_index_mask_nospec array_index_mask_nospec
+static inline unsigned long array_index_mask_nospec(unsigned long index,
+ unsigned long size)
+{
+ unsigned long mask;
+
+ if (__builtin_constant_p(size) && size > 0) {
+ asm(" clgr %2,%1\n"
+ " slbgr %0,%0\n"
+ :"=d" (mask) : "d" (size-1), "d" (index) :"cc");
+ return mask;
+ }
+ asm(" clgr %1,%2\n"
+ " slbgr %0,%0\n"
+ :"=d" (mask) : "d" (size), "d" (index) :"cc");
+ return ~mask;
+}
+
#include <asm-generic/barrier.h>

#endif /* __ASM_BARRIER_H */
--
2.7.4


2018-02-07 10:03:54

by Pavel Machek

[permalink] [raw]
Subject: Re: [PATCH 4/6] s390: add options to change branch prediction behaviour for the kernel

On Wed 2018-02-07 08:00:09, Martin Schwidefsky wrote:
> Add the PPA instruction to the system entry and exit path to switch
> the kernel to a different branch prediction behaviour. The instructions
> are added via CPU alternatives and can be disabled with the "nospec"
> or the "nobp=0" kernel parameter. If the default behaviour selected
> with CONFIG_KERNEL_NOBP is set to "n" then the "nobp=1" parameter can be
> used to enable the changed kernel branch prediction.
>
> Acked-by: Cornelia Huck <[email protected]>
> Signed-off-by: Martin Schwidefsky <[email protected]>
> ---
> arch/s390/Kconfig | 17 ++++++++++++++
> arch/s390/include/asm/processor.h | 1 +
> arch/s390/kernel/alternative.c | 23 +++++++++++++++++++
> arch/s390/kernel/early.c | 2 ++
> arch/s390/kernel/entry.S | 48 +++++++++++++++++++++++++++++++++++++++
> arch/s390/kernel/ipl.c | 1 +
> arch/s390/kernel/smp.c | 2 ++
> 7 files changed, 94 insertions(+)
>
> diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
> index 0105ce2..d514e25 100644
> --- a/arch/s390/Kconfig
> +++ b/arch/s390/Kconfig
> @@ -540,6 +540,23 @@ config ARCH_RANDOM
>
> If unsure, say Y.
>
> +config KERNEL_NOBP
> + def_bool n
> + prompt "Enable modified branch prediction for the kernel by default"
> + help
> + If this option is selected the kernel will switch to a modified
> + branch prediction mode if the firmware interface is available.
> + The modified branch prediction mode improves the behaviour in
> + regard to speculative execution.
> +
> + With the option enabled the kernel parameter "nobp=0" or "nospec"
> + can be used to run the kernel in the normal branch prediction mode.
> +
> + With the option disabled the modified branch prediction mode is
> + enabled with the "nobp=1" kernel parameter.
> +
> + If unsure, say N.
> +

This could use some improvement.

Afaict the config option only changes the default behaviour? Do we
need the option in such case? (CONFIG_CMDLINE_APPEND can be useful to
avoid some options).

Plus, it is not clear what "modified branch prediction mode is" and if
"improves behaviour" means "faster" or "safer".

Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html


Attachments:
(No filename) (2.35 kB)
signature.asc (188.00 B)
Digital signature
Download all attachments

2018-02-07 10:08:40

by Pavel Machek

[permalink] [raw]
Subject: Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

On Wed 2018-02-07 08:00:11, Martin Schwidefsky wrote:
> Add CONFIG_EXPOLINE to enable the use of the new -mindirect-branch= and
> -mfunction_return= compiler options to create a kernel fortified against
> the specte v2 attack.
>
> With CONFIG_EXPOLINE=y all indirect branches will be issued with an
> execute type instruction. For z10 or newer the EXRL instruction will
> be used, for older machines the EX instruction. The typical indirect
> call
>
> basr %r14,%r1
>
> is replaced with a PC relative call to a new thunk
>
> brasl %r14,__s390x_indirect_jump_r1
>
> The thunk contains the EXRL/EX instruction to the indirect branch
>
> __s390x_indirect_jump_r1:
> exrl 0,0f
> j .
> 0: br %r1
>
> The detour via the execute type instruction has a performance impact.
> To get rid of the detour the new kernel parameter "nospectre_v2" and
> "spectre_v2=[on,off,auto]" can be used. If the parameter is specified
> the kernel and module code will be patched at runtime.

This is really unfortunate naming of kernel option.

spectre_v2=off sounds like we are turning the "bug" off, but i somehow
suspect you are turning the bug _workaround_ off.

Pavel

> Signed-off-by: Martin Schwidefsky <[email protected]>
> ---
> arch/s390/Kconfig | 28 +++++++++
> arch/s390/Makefile | 12 ++++
> arch/s390/include/asm/lowcore.h | 6 +-
> arch/s390/include/asm/nospec-branch.h | 18 ++++++
> arch/s390/kernel/Makefile | 4 ++
> arch/s390/kernel/entry.S | 113 ++++++++++++++++++++++++++--------
> arch/s390/kernel/module.c | 62 ++++++++++++++++---
> arch/s390/kernel/nospec-branch.c | 100 ++++++++++++++++++++++++++++++
> arch/s390/kernel/setup.c | 4 ++
> arch/s390/kernel/smp.c | 1 +
> arch/s390/kernel/vmlinux.lds.S | 14 +++++
> drivers/s390/char/Makefile | 2 +
> 12 files changed, 329 insertions(+), 35 deletions(-)
> create mode 100644 arch/s390/include/asm/nospec-branch.h
> create mode 100644 arch/s390/kernel/nospec-branch.c
>
> diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
> index d514e25..d4a65bf 100644
> --- a/arch/s390/Kconfig
> +++ b/arch/s390/Kconfig
> @@ -557,6 +557,34 @@ config KERNEL_NOBP
>
> If unsure, say N.
>
> +config EXPOLINE
> + def_bool n
> + prompt "Avoid speculative indirect branches in the kernel"
> + help
> + Compile the kernel with the expoline compiler options to guard
> + against kernel-to-user data leaks by avoiding speculative indirect
> + branches.
> + Requires a compiler with -mindirect-branch=thunk support for full
> + protection. The kernel may run slower.
> +
> + If unsure, say N.
> +
> +choice
> + prompt "Expoline default"
> + depends on EXPOLINE
> + default EXPOLINE_FULL
> +
> +config EXPOLINE_OFF
> + bool "spectre_v2=off"
> +
> +config EXPOLINE_MEDIUM
> + bool "spectre_v2=auto"
> +
> +config EXPOLINE_FULL
> + bool "spectre_v2=on"
> +
> +endchoice
> +
> endmenu
>
> menu "Memory setup"
> diff --git a/arch/s390/Makefile b/arch/s390/Makefile
> index fd691c4..2f925ef 100644
> --- a/arch/s390/Makefile
> +++ b/arch/s390/Makefile
> @@ -78,6 +78,18 @@ ifeq ($(call cc-option-yn,-mwarn-dynamicstack),y)
> cflags-$(CONFIG_WARN_DYNAMIC_STACK) += -mwarn-dynamicstack
> endif
>
> +ifdef CONFIG_EXPOLINE
> + ifeq ($(call cc-option-yn,$(CC_FLAGS_MARCH) -mindirect-branch=thunk),y)
> + CC_FLAGS_EXPOLINE := -mindirect-branch=thunk
> + CC_FLAGS_EXPOLINE += -mfunction-return=thunk
> + CC_FLAGS_EXPOLINE += -mindirect-branch-table
> + export CC_FLAGS_EXPOLINE
> + cflags-y += $(CC_FLAGS_EXPOLINE)
> + else
> + $(warning "Your gcc lacks the -mindirect-branch= option")
> + endif
> +endif
> +
> ifdef CONFIG_FUNCTION_TRACER
> # make use of hotpatch feature if the compiler supports it
> cc_hotpatch := -mhotpatch=0,3
> diff --git a/arch/s390/include/asm/lowcore.h b/arch/s390/include/asm/lowcore.h
> index c63986a..5bc8888 100644
> --- a/arch/s390/include/asm/lowcore.h
> +++ b/arch/s390/include/asm/lowcore.h
> @@ -136,7 +136,11 @@ struct lowcore {
> __u64 vdso_per_cpu_data; /* 0x03b8 */
> __u64 machine_flags; /* 0x03c0 */
> __u64 gmap; /* 0x03c8 */
> - __u8 pad_0x03d0[0x0e00-0x03d0]; /* 0x03d0 */
> + __u8 pad_0x03d0[0x0400-0x03d0]; /* 0x03d0 */
> +
> + /* br %r1 trampoline */
> + __u16 br_r1_trampoline; /* 0x0400 */
> + __u8 pad_0x0402[0x0e00-0x0402]; /* 0x0402 */
>
> /*
> * 0xe00 contains the address of the IPL Parameter Information
> diff --git a/arch/s390/include/asm/nospec-branch.h b/arch/s390/include/asm/nospec-branch.h
> new file mode 100644
> index 0000000..7df48e5
> --- /dev/null
> +++ b/arch/s390/include/asm/nospec-branch.h
> @@ -0,0 +1,18 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef _ASM_S390_EXPOLINE_H
> +#define _ASM_S390_EXPOLINE_H
> +
> +#ifndef __ASSEMBLY__
> +
> +#include <linux/types.h>
> +
> +extern int nospec_call_disable;
> +extern int nospec_return_disable;
> +
> +void nospec_init_branches(void);
> +void nospec_call_revert(s32 *start, s32 *end);
> +void nospec_return_revert(s32 *start, s32 *end);
> +
> +#endif /* __ASSEMBLY__ */
> +
> +#endif /* _ASM_S390_EXPOLINE_H */
> diff --git a/arch/s390/kernel/Makefile b/arch/s390/kernel/Makefile
> index 909bce6..7f27e3d 100644
> --- a/arch/s390/kernel/Makefile
> +++ b/arch/s390/kernel/Makefile
> @@ -29,6 +29,7 @@ UBSAN_SANITIZE_early.o := n
> #
> ifneq ($(CC_FLAGS_MARCH),-march=z900)
> CFLAGS_REMOVE_als.o += $(CC_FLAGS_MARCH)
> +CFLAGS_REMOVE_als.o += $(CC_FLAGS_EXPOLINE)
> CFLAGS_als.o += -march=z900
> AFLAGS_REMOVE_head.o += $(CC_FLAGS_MARCH)
> AFLAGS_head.o += -march=z900
> @@ -63,6 +64,9 @@ obj-y += entry.o reipl.o relocate_kernel.o kdebugfs.o alternative.o
>
> extra-y += head.o head64.o vmlinux.lds
>
> +obj-$(CONFIG_EXPOLINE) += nospec-branch.o
> +CFLAGS_REMOVE_expoline.o += $(CC_FLAGS_EXPOLINE)
> +
> obj-$(CONFIG_MODULES) += module.o
> obj-$(CONFIG_SMP) += smp.o
> obj-$(CONFIG_SCHED_TOPOLOGY) += topology.o
> diff --git a/arch/s390/kernel/entry.S b/arch/s390/kernel/entry.S
> index 53145b5..13a133a 100644
> --- a/arch/s390/kernel/entry.S
> +++ b/arch/s390/kernel/entry.S
> @@ -222,6 +222,68 @@ _PIF_WORK = (_PIF_PER_TRAP | _PIF_SYSCALL_RESTART)
> .popsection
> .endm
>
> +#ifdef CONFIG_EXPOLINE
> +
> + .macro GEN_BR_THUNK name,reg,tmp
> + .section .text.\name,"axG",@progbits,\name,comdat
> + .globl \name
> + .hidden \name
> + .type \name,@function
> +\name:
> + .cfi_startproc
> +#ifdef CONFIG_HAVE_MARCH_Z10_FEATURES
> + exrl 0,0f
> +#else
> + larl \tmp,0f
> + ex 0,0(\tmp)
> +#endif
> + j .
> +0: br \reg
> + .cfi_endproc
> + .endm
> +
> + GEN_BR_THUNK __s390x_indirect_jump_r1use_r9,%r9,%r1
> + GEN_BR_THUNK __s390x_indirect_jump_r1use_r14,%r14,%r1
> + GEN_BR_THUNK __s390x_indirect_jump_r11use_r14,%r14,%r11
> +
> + .macro BASR_R14_R9
> +0: brasl %r14,__s390x_indirect_jump_r1use_r9
> + .pushsection .s390_indirect_branches,"a",@progbits
> + .long 0b-.
> + .popsection
> + .endm
> +
> + .macro BR_R1USE_R14
> +0: jg __s390x_indirect_jump_r1use_r14
> + .pushsection .s390_indirect_branches,"a",@progbits
> + .long 0b-.
> + .popsection
> + .endm
> +
> + .macro BR_R11USE_R14
> +0: jg __s390x_indirect_jump_r11use_r14
> + .pushsection .s390_indirect_branches,"a",@progbits
> + .long 0b-.
> + .popsection
> + .endm
> +
> +#else /* CONFIG_EXPOLINE */
> +
> + .macro BASR_R14_R9
> + basr %r14,%r9
> + .endm
> +
> + .macro BR_R1USE_R14
> + br %r14
> + .endm
> +
> + .macro BR_R11USE_R14
> + br %r14
> + .endm
> +
> +#endif /* CONFIG_EXPOLINE */
> +
> +
> .section .kprobes.text, "ax"
> .Ldummy:
> /*
> @@ -237,7 +299,7 @@ _PIF_WORK = (_PIF_PER_TRAP | _PIF_SYSCALL_RESTART)
> ENTRY(__bpon)
> .globl __bpon
> BPON
> - br %r14
> + BR_R1USE_R14
>
> /*
> * Scheduler resume function, called by switch_to
> @@ -261,9 +323,9 @@ ENTRY(__switch_to)
> mvc __LC_CURRENT_PID(4,%r0),0(%r3) # store pid of next
> lmg %r6,%r15,__SF_GPRS(%r15) # load gprs of next task
> TSTMSK __LC_MACHINE_FLAGS,MACHINE_FLAG_LPP
> - bzr %r14
> + jz 0f
> .insn s,0xb2800000,__LC_LPP # set program parameter
> - br %r14
> +0: BR_R1USE_R14
>
> .L__critical_start:
>
> @@ -330,7 +392,7 @@ sie_exit:
> xgr %r5,%r5
> lmg %r6,%r14,__SF_GPRS(%r15) # restore kernel registers
> lg %r2,__SF_EMPTY+16(%r15) # return exit reason code
> - br %r14
> + BR_R1USE_R14
> .Lsie_fault:
> lghi %r14,-EFAULT
> stg %r14,__SF_EMPTY+16(%r15) # set exit reason code
> @@ -389,7 +451,7 @@ ENTRY(system_call)
> lgf %r9,0(%r8,%r10) # get system call add.
> TSTMSK __TI_flags(%r12),_TIF_TRACE
> jnz .Lsysc_tracesys
> - basr %r14,%r9 # call sys_xxxx
> + BASR_R14_R9 # call sys_xxxx
> stg %r2,__PT_R2(%r11) # store return value
>
> .Lsysc_return:
> @@ -574,7 +636,7 @@ ENTRY(system_call)
> lmg %r3,%r7,__PT_R3(%r11)
> stg %r7,STACK_FRAME_OVERHEAD(%r15)
> lg %r2,__PT_ORIG_GPR2(%r11)
> - basr %r14,%r9 # call sys_xxx
> + BASR_R14_R9 # call sys_xxx
> stg %r2,__PT_R2(%r11) # store return value
> .Lsysc_tracenogo:
> TSTMSK __TI_flags(%r12),_TIF_TRACE
> @@ -598,7 +660,7 @@ ENTRY(ret_from_fork)
> lmg %r9,%r10,__PT_R9(%r11) # load gprs
> ENTRY(kernel_thread_starter)
> la %r2,0(%r10)
> - basr %r14,%r9
> + BASR_R14_R9
> j .Lsysc_tracenogo
>
> /*
> @@ -678,9 +740,9 @@ ENTRY(pgm_check_handler)
> nill %r10,0x007f
> sll %r10,2
> je .Lpgm_return
> - lgf %r1,0(%r10,%r1) # load address of handler routine
> + lgf %r9,0(%r10,%r1) # load address of handler routine
> lgr %r2,%r11 # pass pointer to pt_regs
> - basr %r14,%r1 # branch to interrupt-handler
> + BASR_R14_R9 # branch to interrupt-handler
> .Lpgm_return:
> LOCKDEP_SYS_EXIT
> tm __PT_PSW+1(%r11),0x01 # returning to user ?
> @@ -998,7 +1060,7 @@ ENTRY(psw_idle)
> stpt __TIMER_IDLE_ENTER(%r2)
> .Lpsw_idle_lpsw:
> lpswe __SF_EMPTY(%r15)
> - br %r14
> + BR_R1USE_R14
> .Lpsw_idle_end:
>
> /*
> @@ -1012,7 +1074,7 @@ ENTRY(save_fpu_regs)
> lg %r2,__LC_CURRENT
> aghi %r2,__TASK_thread
> TSTMSK __LC_CPU_FLAGS,_CIF_FPU
> - bor %r14
> + jo .Lsave_fpu_regs_exit
> stfpc __THREAD_FPU_fpc(%r2)
> lg %r3,__THREAD_FPU_regs(%r2)
> TSTMSK __LC_MACHINE_FLAGS,MACHINE_FLAG_VX
> @@ -1039,7 +1101,8 @@ ENTRY(save_fpu_regs)
> std 15,120(%r3)
> .Lsave_fpu_regs_done:
> oi __LC_CPU_FLAGS+7,_CIF_FPU
> - br %r14
> +.Lsave_fpu_regs_exit:
> + BR_R1USE_R14
> .Lsave_fpu_regs_end:
> EXPORT_SYMBOL(save_fpu_regs)
>
> @@ -1057,7 +1120,7 @@ load_fpu_regs:
> lg %r4,__LC_CURRENT
> aghi %r4,__TASK_thread
> TSTMSK __LC_CPU_FLAGS,_CIF_FPU
> - bnor %r14
> + jno .Lload_fpu_regs_exit
> lfpc __THREAD_FPU_fpc(%r4)
> TSTMSK __LC_MACHINE_FLAGS,MACHINE_FLAG_VX
> lg %r4,__THREAD_FPU_regs(%r4) # %r4 <- reg save area
> @@ -1084,7 +1147,8 @@ load_fpu_regs:
> ld 15,120(%r4)
> .Lload_fpu_regs_done:
> ni __LC_CPU_FLAGS+7,255-_CIF_FPU
> - br %r14
> +.Lload_fpu_regs_exit:
> + BR_R1USE_R14
> .Lload_fpu_regs_end:
>
> .L__critical_end:
> @@ -1301,7 +1365,7 @@ cleanup_critical:
> jl 0f
> clg %r9,BASED(.Lcleanup_table+104) # .Lload_fpu_regs_end
> jl .Lcleanup_load_fpu_regs
> -0: br %r14
> +0: BR_R11USE_R14
>
> .align 8
> .Lcleanup_table:
> @@ -1337,7 +1401,7 @@ cleanup_critical:
> ni __SIE_PROG0C+3(%r9),0xfe # no longer in SIE
> lctlg %c1,%c1,__LC_USER_ASCE # load primary asce
> larl %r9,sie_exit # skip forward to sie_exit
> - br %r14
> + BR_R11USE_R14
> #endif
>
> .Lcleanup_system_call:
> @@ -1390,7 +1454,7 @@ cleanup_critical:
> stg %r15,56(%r11) # r15 stack pointer
> # set new psw address and exit
> larl %r9,.Lsysc_do_svc
> - br %r14
> + BR_R11USE_R14
> .Lcleanup_system_call_insn:
> .quad system_call
> .quad .Lsysc_stmg
> @@ -1402,7 +1466,7 @@ cleanup_critical:
>
> .Lcleanup_sysc_tif:
> larl %r9,.Lsysc_tif
> - br %r14
> + BR_R11USE_R14
>
> .Lcleanup_sysc_restore:
> # check if stpt has been executed
> @@ -1419,14 +1483,14 @@ cleanup_critical:
> mvc 0(64,%r11),__PT_R8(%r9)
> lmg %r0,%r7,__PT_R0(%r9)
> 1: lmg %r8,%r9,__LC_RETURN_PSW
> - br %r14
> + BR_R11USE_R14
> .Lcleanup_sysc_restore_insn:
> .quad .Lsysc_exit_timer
> .quad .Lsysc_done - 4
>
> .Lcleanup_io_tif:
> larl %r9,.Lio_tif
> - br %r14
> + BR_R11USE_R14
>
> .Lcleanup_io_restore:
> # check if stpt has been executed
> @@ -1440,7 +1504,7 @@ cleanup_critical:
> mvc 0(64,%r11),__PT_R8(%r9)
> lmg %r0,%r7,__PT_R0(%r9)
> 1: lmg %r8,%r9,__LC_RETURN_PSW
> - br %r14
> + BR_R11USE_R14
> .Lcleanup_io_restore_insn:
> .quad .Lio_exit_timer
> .quad .Lio_done - 4
> @@ -1493,17 +1557,17 @@ cleanup_critical:
> # prepare return psw
> nihh %r8,0xfcfd # clear irq & wait state bits
> lg %r9,48(%r11) # return from psw_idle
> - br %r14
> + BR_R11USE_R14
> .Lcleanup_idle_insn:
> .quad .Lpsw_idle_lpsw
>
> .Lcleanup_save_fpu_regs:
> larl %r9,save_fpu_regs
> - br %r14
> + BR_R11USE_R14
>
> .Lcleanup_load_fpu_regs:
> larl %r9,load_fpu_regs
> - br %r14
> + BR_R11USE_R14
>
> /*
> * Integer constants
> @@ -1523,7 +1587,6 @@ cleanup_critical:
> .Lsie_crit_mcck_length:
> .quad .Lsie_skip - .Lsie_entry
> #endif
> -
> .section .rodata, "a"
> #define SYSCALL(esame,emu) .long esame
> .globl sys_call_table
> diff --git a/arch/s390/kernel/module.c b/arch/s390/kernel/module.c
> index b7abfad..1fc6d1f 100644
> --- a/arch/s390/kernel/module.c
> +++ b/arch/s390/kernel/module.c
> @@ -19,6 +19,8 @@
> #include <linux/moduleloader.h>
> #include <linux/bug.h>
> #include <asm/alternative.h>
> +#include <asm/nospec-branch.h>
> +#include <asm/facility.h>
>
> #if 0
> #define DEBUGP printk
> @@ -156,7 +158,11 @@ int module_frob_arch_sections(Elf_Ehdr *hdr, Elf_Shdr *sechdrs,
> me->arch.got_offset = me->core_layout.size;
> me->core_layout.size += me->arch.got_size;
> me->arch.plt_offset = me->core_layout.size;
> - me->core_layout.size += me->arch.plt_size;
> + if (me->arch.plt_size) {
> + if (IS_ENABLED(CONFIG_EXPOLINE) && !nospec_call_disable)
> + me->arch.plt_size += PLT_ENTRY_SIZE;
> + me->core_layout.size += me->arch.plt_size;
> + }
> return 0;
> }
>
> @@ -310,9 +316,21 @@ static int apply_rela(Elf_Rela *rela, Elf_Addr base, Elf_Sym *symtab,
> unsigned int *ip;
> ip = me->core_layout.base + me->arch.plt_offset +
> info->plt_offset;
> - ip[0] = 0x0d10e310; /* basr 1,0; lg 1,10(1); br 1 */
> - ip[1] = 0x100a0004;
> - ip[2] = 0x07f10000;
> + ip[0] = 0x0d10e310; /* basr 1,0 */
> + ip[1] = 0x100a0004; /* lg 1,10(1) */
> + if (IS_ENABLED(CONFIG_EXPOLINE) &&
> + !nospec_call_disable) {
> + unsigned int *ij;
> + ij = me->core_layout.base +
> + me->arch.plt_offset +
> + me->arch.plt_size - PLT_ENTRY_SIZE;
> + ip[2] = 0xa7f40000 + /* j __jump_r1 */
> + (unsigned int)(u16)
> + (((unsigned long) ij - 8 -
> + (unsigned long) ip) / 2);
> + } else {
> + ip[2] = 0x07f10000; /* br %r1 */
> + }
> ip[3] = (unsigned int) (val >> 32);
> ip[4] = (unsigned int) val;
> info->plt_initialized = 1;
> @@ -418,16 +436,42 @@ int module_finalize(const Elf_Ehdr *hdr,
> struct module *me)
> {
> const Elf_Shdr *s;
> - char *secstrings;
> + char *secstrings, *secname;
> + void *aseg;
> +
> + if (IS_ENABLED(CONFIG_EXPOLINE) &&
> + !nospec_call_disable && me->arch.plt_size) {
> + unsigned int *ij;
> +
> + ij = me->core_layout.base + me->arch.plt_offset +
> + me->arch.plt_size - PLT_ENTRY_SIZE;
> + if (test_facility(35)) {
> + ij[0] = 0xc6000000; /* exrl %r0,.+10 */
> + ij[1] = 0x0005a7f4; /* j . */
> + ij[2] = 0x000007f1; /* br %r1 */
> + } else {
> + ij[0] = 0x44000000 | (unsigned int)
> + offsetof(struct lowcore, br_r1_trampoline);
> + ij[1] = 0xa7f40000; /* j . */
> + }
> + }
>
> secstrings = (void *)hdr + sechdrs[hdr->e_shstrndx].sh_offset;
> for (s = sechdrs; s < sechdrs + hdr->e_shnum; s++) {
> - if (!strcmp(".altinstructions", secstrings + s->sh_name)) {
> - /* patch .altinstructions */
> - void *aseg = (void *)s->sh_addr;
> + aseg = (void *) s->sh_addr;
> + secname = secstrings + s->sh_name;
>
> + if (!strcmp(".altinstructions", secname))
> + /* patch .altinstructions */
> apply_alternatives(aseg, aseg + s->sh_size);
> - }
> +
> + if (IS_ENABLED(CONFIG_EXPOLINE) &&
> + (!strcmp(".nospec_call_table", secname)))
> + nospec_call_revert(aseg, aseg + s->sh_size);
> +
> + if (IS_ENABLED(CONFIG_EXPOLINE) &&
> + (!strcmp(".nospec_return_table", secname)))
> + nospec_return_revert(aseg, aseg + s->sh_size);
> }
>
> jump_label_apply_nops(me);
> diff --git a/arch/s390/kernel/nospec-branch.c b/arch/s390/kernel/nospec-branch.c
> new file mode 100644
> index 0000000..69d7fcf
> --- /dev/null
> +++ b/arch/s390/kernel/nospec-branch.c
> @@ -0,0 +1,100 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <linux/module.h>
> +#include <asm/nospec-branch.h>
> +
> +int nospec_call_disable = IS_ENABLED(EXPOLINE_OFF);
> +int nospec_return_disable = !IS_ENABLED(EXPOLINE_FULL);
> +
> +static int __init nospectre_v2_setup_early(char *str)
> +{
> + nospec_call_disable = 1;
> + nospec_return_disable = 1;
> + return 0;
> +}
> +early_param("nospectre_v2", nospectre_v2_setup_early);
> +
> +static int __init spectre_v2_setup_early(char *str)
> +{
> + if (str && !strncmp(str, "on", 2)) {
> + nospec_call_disable = 0;
> + nospec_return_disable = 0;
> + }
> + if (str && !strncmp(str, "off", 3)) {
> + nospec_call_disable = 1;
> + nospec_return_disable = 1;
> + }
> + if (str && !strncmp(str, "auto", 4)) {
> + nospec_call_disable = 0;
> + nospec_return_disable = 1;
> + }
> + return 0;
> +}
> +early_param("spectre_v2", spectre_v2_setup_early);
> +
> +static void __init_or_module __nospec_revert(s32 *start, s32 *end)
> +{
> + enum { BRCL_EXPOLINE, BRASL_EXPOLINE } type;
> + u8 *instr, *thunk, *br;
> + u8 insnbuf[6];
> + s32 *epo;
> +
> + /* Second part of the instruction replace is always a nop */
> + memcpy(insnbuf + 2, (char[]) { 0x47, 0x00, 0x00, 0x00 }, 4);
> + for (epo = start; epo < end; epo++) {
> + instr = (u8 *) epo + *epo;
> + if (instr[0] == 0xc0 && (instr[1] & 0x0f) == 0x04)
> + type = BRCL_EXPOLINE; /* brcl instruction */
> + else if (instr[0] == 0xc0 && (instr[1] & 0x0f) == 0x05)
> + type = BRASL_EXPOLINE; /* brasl instruction */
> + else
> + continue;
> + thunk = instr + (*(int *)(instr + 2)) * 2;
> + if (thunk[0] == 0xc6 && thunk[1] == 0x00)
> + /* exrl %r0,<target-br> */
> + br = thunk + (*(int *)(thunk + 2)) * 2;
> + else if (thunk[0] == 0xc0 && (thunk[1] & 0x0f) == 0x00 &&
> + thunk[6] == 0x44 && thunk[7] == 0x00 &&
> + (thunk[8] & 0x0f) == 0x00 && thunk[9] == 0x00 &&
> + (thunk[1] & 0xf0) == (thunk[8] & 0xf0))
> + /* larl %rx,<target br> + ex %r0,0(%rx) */
> + br = thunk + (*(int *)(thunk + 2)) * 2;
> + else
> + continue;
> + if (br[0] != 0x07 || (br[1] & 0xf0) != 0xf0)
> + continue;
> + switch (type) {
> + case BRCL_EXPOLINE:
> + /* brcl to thunk, replace with br + nop */
> + insnbuf[0] = br[0];
> + insnbuf[1] = (instr[1] & 0xf0) | (br[1] & 0x0f);
> + break;
> + case BRASL_EXPOLINE:
> + /* brasl to thunk, replace with basr + nop */
> + insnbuf[0] = 0x0d;
> + insnbuf[1] = (instr[1] & 0xf0) | (br[1] & 0x0f);
> + break;
> + }
> +
> + s390_kernel_write(instr, insnbuf, 6);
> + }
> +}
> +
> +void __init_or_module nospec_call_revert(s32 *start, s32 *end)
> +{
> + if (nospec_call_disable)
> + __nospec_revert(start, end);
> +}
> +
> +void __init_or_module nospec_return_revert(s32 *start, s32 *end)
> +{
> + if (nospec_return_disable)
> + __nospec_revert(start, end);
> +}
> +
> +extern s32 __nospec_call_start[], __nospec_call_end[];
> +extern s32 __nospec_return_start[], __nospec_return_end[];
> +void __init nospec_init_branches(void)
> +{
> + nospec_call_revert(__nospec_call_start, __nospec_call_end);
> + nospec_return_revert(__nospec_return_start, __nospec_return_end);
> +}
> diff --git a/arch/s390/kernel/setup.c b/arch/s390/kernel/setup.c
> index bcd2a4a..a6a91f0 100644
> --- a/arch/s390/kernel/setup.c
> +++ b/arch/s390/kernel/setup.c
> @@ -68,6 +68,7 @@
> #include <asm/sysinfo.h>
> #include <asm/numa.h>
> #include <asm/alternative.h>
> +#include <asm/nospec-branch.h>
> #include "entry.h"
>
> /*
> @@ -379,6 +380,7 @@ static void __init setup_lowcore(void)
> lc->spinlock_index = 0;
> arch_spin_lock_setup(0);
> #endif
> + lc->br_r1_trampoline = 0x07f1; /* br %r1 */
>
> set_prefix((u32)(unsigned long) lc);
> lowcore_ptr[0] = lc;
> @@ -954,6 +956,8 @@ void __init setup_arch(char **cmdline_p)
> set_preferred_console();
>
> apply_alternative_instructions();
> + if (IS_ENABLED(CONFIG_EXPOLINE))
> + nospec_init_branches();
>
> /* Setup zfcpdump support */
> setup_zfcpdump();
> diff --git a/arch/s390/kernel/smp.c b/arch/s390/kernel/smp.c
> index 2fd7d60..a4a9fe1 100644
> --- a/arch/s390/kernel/smp.c
> +++ b/arch/s390/kernel/smp.c
> @@ -214,6 +214,7 @@ static int pcpu_alloc_lowcore(struct pcpu *pcpu, int cpu)
> lc->cpu_nr = cpu;
> lc->spinlock_lockval = arch_spin_lockval(cpu);
> lc->spinlock_index = 0;
> + lc->br_r1_trampoline = 0x07f1; /* br %r1 */
> if (nmi_alloc_per_cpu(lc))
> goto out;
> if (vdso_alloc_per_cpu(lc))
> diff --git a/arch/s390/kernel/vmlinux.lds.S b/arch/s390/kernel/vmlinux.lds.S
> index 608cf29..08d12cf 100644
> --- a/arch/s390/kernel/vmlinux.lds.S
> +++ b/arch/s390/kernel/vmlinux.lds.S
> @@ -123,6 +123,20 @@ SECTIONS
> *(.altinstr_replacement)
> }
>
> + /*
> + * Table with the patch locations to undo expolines
> + */
> + .nospec_call_table : {
> + __nospec_call_start = . ;
> + *(.s390_indirect*)
> + __nospec_call_end = . ;
> + }
> + .nospec_return_table : {
> + __nospec_return_start = . ;
> + *(.s390_return*)
> + __nospec_return_end = . ;
> + }
> +
> /* early.c uses stsi, which requires page aligned data. */
> . = ALIGN(PAGE_SIZE);
> INIT_DATA_SECTION(0x100)
> diff --git a/drivers/s390/char/Makefile b/drivers/s390/char/Makefile
> index 614b44e..a2b33a2 100644
> --- a/drivers/s390/char/Makefile
> +++ b/drivers/s390/char/Makefile
> @@ -19,6 +19,8 @@ endif
>
> CFLAGS_sclp_early_core.o += -D__NO_FORTIFY
>
> +CFLAGS_REMOVE_sclp_early_core.o += $(CC_FLAGS_EXPOLINE)
> +
> obj-y += ctrlchar.o keyboard.o defkeymap.o sclp.o sclp_rw.o sclp_quiesce.o \
> sclp_cmd.o sclp_config.o sclp_cpi_sys.o sclp_ocf.o sclp_ctl.o \
> sclp_early.o sclp_early_core.o

--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html


Attachments:
(No filename) (22.72 kB)
signature.asc (188.00 B)
Digital signature
Download all attachments

2018-02-07 11:51:15

by Martin Schwidefsky

[permalink] [raw]
Subject: Re: [PATCH 4/6] s390: add options to change branch prediction behaviour for the kernel

On Wed, 7 Feb 2018 11:02:52 +0100
Pavel Machek <[email protected]> wrote:

> On Wed 2018-02-07 08:00:09, Martin Schwidefsky wrote:
> > Add the PPA instruction to the system entry and exit path to switch
> > the kernel to a different branch prediction behaviour. The instructions
> > are added via CPU alternatives and can be disabled with the "nospec"
> > or the "nobp=0" kernel parameter. If the default behaviour selected
> > with CONFIG_KERNEL_NOBP is set to "n" then the "nobp=1" parameter can be
> > used to enable the changed kernel branch prediction.
> >
> > Acked-by: Cornelia Huck <[email protected]>
> > Signed-off-by: Martin Schwidefsky <[email protected]>
> > ---
> > arch/s390/Kconfig | 17 ++++++++++++++
> > arch/s390/include/asm/processor.h | 1 +
> > arch/s390/kernel/alternative.c | 23 +++++++++++++++++++
> > arch/s390/kernel/early.c | 2 ++
> > arch/s390/kernel/entry.S | 48 +++++++++++++++++++++++++++++++++++++++
> > arch/s390/kernel/ipl.c | 1 +
> > arch/s390/kernel/smp.c | 2 ++
> > 7 files changed, 94 insertions(+)
> >
> > diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
> > index 0105ce2..d514e25 100644
> > --- a/arch/s390/Kconfig
> > +++ b/arch/s390/Kconfig
> > @@ -540,6 +540,23 @@ config ARCH_RANDOM
> >
> > If unsure, say Y.
> >
> > +config KERNEL_NOBP
> > + def_bool n
> > + prompt "Enable modified branch prediction for the kernel by default"
> > + help
> > + If this option is selected the kernel will switch to a modified
> > + branch prediction mode if the firmware interface is available.
> > + The modified branch prediction mode improves the behaviour in
> > + regard to speculative execution.
> > +
> > + With the option enabled the kernel parameter "nobp=0" or "nospec"
> > + can be used to run the kernel in the normal branch prediction mode.
> > +
> > + With the option disabled the modified branch prediction mode is
> > + enabled with the "nobp=1" kernel parameter.
> > +
> > + If unsure, say N.
> > +
>
> This could use some improvement.
>
> Afaict the config option only changes the default behaviour? Do we
> need the option in such case? (CONFIG_CMDLINE_APPEND can be useful to
> avoid some options).

# git grep CMDLINE_APPEND

returns nothing. What are you referring to?

The idea of this config option is that a distributor can decide what
default behaviour the system should have if no command line parameter
is specified.

> Plus, it is not clear what "modified branch prediction mode is" and if
> "improves behaviour" means "faster" or "safer".

Naturally "improves behaviour in regard to speculative execution" is
indicating that the system is safer. In general that means slower as well
but as always it depends. What exactly is done is up to the machine.

--
blue skies,
Martin.

"Reality continues to ruin my life." - Calvin.


2018-02-07 11:53:05

by Martin Schwidefsky

[permalink] [raw]
Subject: Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

On Wed, 7 Feb 2018 11:07:26 +0100
Pavel Machek <[email protected]> wrote:

> On Wed 2018-02-07 08:00:11, Martin Schwidefsky wrote:
> > Add CONFIG_EXPOLINE to enable the use of the new -mindirect-branch= and
> > -mfunction_return= compiler options to create a kernel fortified against
> > the specte v2 attack.
> >
> > With CONFIG_EXPOLINE=y all indirect branches will be issued with an
> > execute type instruction. For z10 or newer the EXRL instruction will
> > be used, for older machines the EX instruction. The typical indirect
> > call
> >
> > basr %r14,%r1
> >
> > is replaced with a PC relative call to a new thunk
> >
> > brasl %r14,__s390x_indirect_jump_r1
> >
> > The thunk contains the EXRL/EX instruction to the indirect branch
> >
> > __s390x_indirect_jump_r1:
> > exrl 0,0f
> > j .
> > 0: br %r1
> >
> > The detour via the execute type instruction has a performance impact.
> > To get rid of the detour the new kernel parameter "nospectre_v2" and
> > "spectre_v2=[on,off,auto]" can be used. If the parameter is specified
> > the kernel and module code will be patched at runtime.
>
> This is really unfortunate naming of kernel option.
>
> spectre_v2=off sounds like we are turning the "bug" off, but i somehow
> suspect you are turning the bug _workaround_ off.

Well, that is the 1:1 copy of the x86 option. Do you want to change that
one as well?

--
blue skies,
Martin.

"Reality continues to ruin my life." - Calvin.


2018-02-07 12:09:01

by David Woodhouse

[permalink] [raw]
Subject: Re: [PATCH 6/6] s390: introduce execute-trampolines for branches



On Wed, 2018-02-07 at 11:07 +0100, Pavel Machek wrote:
> This is really unfortunate naming of kernel option.
>
> spectre_v2=off sounds like we are turning the "bug" off, but i somehow
> suspect you are turning the bug _workaround_ off.

That's consistent with what we have on x86.

> > +ifdef CONFIG_EXPOLINE
> > +  ifeq ($(call cc-option-yn,$(CC_FLAGS_MARCH) -mindirect-branch=thunk),y)
> > +    CC_FLAGS_EXPOLINE := -mindirect-branch=thunk
> > +    CC_FLAGS_EXPOLINE += -mfunction-return=thunk
> > +    CC_FLAGS_EXPOLINE += -mindirect-branch-table
> > +    export CC_FLAGS_EXPOLINE
> > +    cflags-y += $(CC_FLAGS_EXPOLINE)
> > +  else
> > +    $(warning "Your gcc lacks the -mindirect-branch= option")
> > +  endif
> > +endif

That isn't, though. Linus asked us to drop the $(warning) part.

... and then spent a week building with a non-retpoline compiler and
not noticing, so he might have changed his mind ;)


Attachments:
smime.p7s (5.09 kB)

2018-02-07 12:18:57

by Martin Schwidefsky

[permalink] [raw]
Subject: Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

On Wed, 07 Feb 2018 12:07:55 +0000
David Woodhouse <[email protected]> wrote:

> On Wed, 2018-02-07 at 11:07 +0100, Pavel Machek wrote:
> > This is really unfortunate naming of kernel option.
> >
> > spectre_v2=off sounds like we are turning the "bug" off, but i somehow
> > suspect you are turning the bug _workaround_ off.
>
> That's consistent with what we have on x86.
>
> > > +ifdef CONFIG_EXPOLINE
> > > +  ifeq ($(call cc-option-yn,$(CC_FLAGS_MARCH) -mindirect-branch=thunk),y)
> > > +    CC_FLAGS_EXPOLINE := -mindirect-branch=thunk
> > > +    CC_FLAGS_EXPOLINE += -mfunction-return=thunk
> > > +    CC_FLAGS_EXPOLINE += -mindirect-branch-table
> > > +    export CC_FLAGS_EXPOLINE
> > > +    cflags-y += $(CC_FLAGS_EXPOLINE)
> > > +  else
> > > +    $(warning "Your gcc lacks the -mindirect-branch= option")
> > > +  endif
> > > +endif
>
> That isn't, though. Linus asked us to drop the $(warning) part.
>
> ... and then spent a week building with a non-retpoline compiler and
> not noticing, so he might have changed his mind ;)

I found the warning to have some value, it helps for the case where my
fingers are faster than my brain and I type "make" instead of "smake"
which uses the alternative compiler with the required support.

@Linus: do you want a warning or prefer not to have one ?

--
blue skies,
Martin.

"Reality continues to ruin my life." - Calvin.


2018-02-07 12:21:06

by David Woodhouse

[permalink] [raw]
Subject: Re: [PATCH 6/6] s390: introduce execute-trampolines for branches



On Wed, 2018-02-07 at 13:17 +0100, Martin Schwidefsky wrote:
> On Wed, 07 Feb 2018 12:07:55 +0000
> David Woodhouse <[email protected]> wrote:
>
> >
> > On Wed, 2018-02-07 at 11:07 +0100, Pavel Machek wrote:
> > >
> > > This is really unfortunate naming of kernel option.
> > >
> > > spectre_v2=off sounds like we are turning the "bug" off, but i somehow
> > > suspect you are turning the bug _workaround_ off.  
> > That's consistent with what we have on x86.
> >
> > >
> > > >
> > > > +ifdef CONFIG_EXPOLINE
> > > > +  ifeq ($(call cc-option-yn,$(CC_FLAGS_MARCH) -mindirect-branch=thunk),y)
> > > > +    CC_FLAGS_EXPOLINE := -mindirect-branch=thunk
> > > > +    CC_FLAGS_EXPOLINE += -mfunction-return=thunk
> > > > +    CC_FLAGS_EXPOLINE += -mindirect-branch-table
> > > > +    export CC_FLAGS_EXPOLINE
> > > > +    cflags-y += $(CC_FLAGS_EXPOLINE)
> > > > +  else
> > > > +    $(warning "Your gcc lacks the -mindirect-branch= option")
> > > > +  endif
> > > > +endif  
> > That isn't, though. Linus asked us to drop the $(warning) part.
> >
> > ... and then spent a week building with a non-retpoline compiler and
> > not noticing, so he might have changed his mind ;)
>
> I found the warning to have some value, it helps for the case where my
> fingers are faster than my brain and I type "make" instead of "smake"
> which uses the alternative compiler with the required support.
>
> @Linus: do you want a warning or prefer not to have one ?

FWIW I agreed to drop it when the plan in my head was "we'll just turn
on IBRS instead if the compiler doesn't do full retpoline support".

Now that Linus has expressed a disinclination to take IBRS support in
that form, I might be more inclined to defend the $(warning) too.


Attachments:
smime.p7s (5.09 kB)

2018-02-07 17:56:52

by Linus Torvalds

[permalink] [raw]
Subject: Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

On Wed, Feb 7, 2018 at 4:17 AM, Martin Schwidefsky
<[email protected]> wrote:
>> That isn't, though. Linus asked us to drop the $(warning) part.
>>
>> ... and then spent a week building with a non-retpoline compiler and
>> not noticing, so he might have changed his mind ;)
>
> I found the warning to have some value, it helps for the case where my
> fingers are faster than my brain and I type "make" instead of "smake"
> which uses the alternative compiler with the required support.
>
> @Linus: do you want a warning or prefer not to have one ?

Honestly, I think I'd be much happier with the warning as part of the
"make config" phase.

What really annoyed me was that it showed up at every build.

What I would really want - and this is entirely unrelated to this
particular case - is to have those damn compiler option tests as part
of the config phase in general. We now have about a million of these
crazy things, where we have config options that simply depend on which
compiler we have, and we have no sane way to show them at
configuration time.

Though Andrew's tree I got yet another ugly hack
(CONFIG_CC_STACKPROTECTOR_AUTO) that handles just _one_ special case
by turning it into a special magic Kconfig entry in the main Makefile.
See commit 44c6dc940b19 ("Makefile: introduce
CONFIG_CC_STACKPROTECTOR_AUTO"). I wasn't sure if I really wanted it,
and honestly, I'm still thinking of just reverting it, because it's
_so_ ugly and _so_ wrong.

What we need is an extension to the Kconfig language itself so that we can do

config CC_HAS_RETPOLINE
cc_option "-mindirect-branch=thunk -mindirect-branch-table"

or something. And then we can make sane _conditional_ dependencies at
Kconfig time, and our makefiles would be much cleaner too when you
could just do

cflags-$(USE_RETPOLINE) += -mfunction-return=thunk -mindirect-branch-table

because the validity of the C compiler flag has been tested when configuring.

And then we could add that warning at configure time (or just disable
the option there thanks to "depends on CC_HAS_xyz" logic).

All our compiler option handling right now is just nasty nasty nasty crud.

Adding more people in the hopes that somebody gets motivated.. I've
talked about this before, so far we haven't made any progress.

Linus

2018-02-07 23:46:37

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

2018-02-08 2:55 GMT+09:00 Linus Torvalds <[email protected]>:
> On Wed, Feb 7, 2018 at 4:17 AM, Martin Schwidefsky
> <[email protected]> wrote:
>>> That isn't, though. Linus asked us to drop the $(warning) part.
>>>
>>> ... and then spent a week building with a non-retpoline compiler and
>>> not noticing, so he might have changed his mind ;)
>>
>> I found the warning to have some value, it helps for the case where my
>> fingers are faster than my brain and I type "make" instead of "smake"
>> which uses the alternative compiler with the required support.
>>
>> @Linus: do you want a warning or prefer not to have one ?
>
> Honestly, I think I'd be much happier with the warning as part of the
> "make config" phase.
>
> What really annoyed me was that it showed up at every build.
>
> What I would really want - and this is entirely unrelated to this
> particular case - is to have those damn compiler option tests as part
> of the config phase in general. We now have about a million of these
> crazy things, where we have config options that simply depend on which
> compiler we have, and we have no sane way to show them at
> configuration time.
>
> Though Andrew's tree I got yet another ugly hack
> (CONFIG_CC_STACKPROTECTOR_AUTO) that handles just _one_ special case
> by turning it into a special magic Kconfig entry in the main Makefile.
> See commit 44c6dc940b19 ("Makefile: introduce
> CONFIG_CC_STACKPROTECTOR_AUTO"). I wasn't sure if I really wanted it,
> and honestly, I'm still thinking of just reverting it, because it's
> _so_ ugly and _so_ wrong.
>
> What we need is an extension to the Kconfig language itself so that we can do
>
> config CC_HAS_RETPOLINE
> cc_option "-mindirect-branch=thunk -mindirect-branch-table"
>
> or something. And then we can make sane _conditional_ dependencies at
> Kconfig time, and our makefiles would be much cleaner too when you
> could just do
>
> cflags-$(USE_RETPOLINE) += -mfunction-return=thunk -mindirect-branch-table
>
> because the validity of the C compiler flag has been tested when configuring.
>
> And then we could add that warning at configure time (or just disable
> the option there thanks to "depends on CC_HAS_xyz" logic).
>
> All our compiler option handling right now is just nasty nasty nasty crud.
>
> Adding more people in the hopes that somebody gets motivated.. I've
> talked about this before, so far we haven't made any progress.


Sorry for slow progress.

I agreed this before, and still motivated.
(because I also motivated to remove kbuild cache.
This turned out not so clever as I first thought)

I was trying to do this, but in this development cycle
I spent most of my time to flush out lots of piled up Kconfig patches.
Sorry.

Unless somebody is working, I will.



--
Best Regards
Masahiro Yamada

2018-02-08 00:06:57

by Linus Torvalds

[permalink] [raw]
Subject: Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

On Wed, Feb 7, 2018 at 3:44 PM, Masahiro Yamada
<[email protected]> wrote:
>
> I agreed this before, and still motivated.
> (because I also motivated to remove kbuild cache.

I actually wish I still had my old "run shell script" thing. I had
some very preliminary patches that actually worked for simple things,
and you could do something like

config SOME_NAME
bool
option shell "true"

and it would act basically like "option env", except it didn't do
"getenv()", it did "system()" and checked the return value. So the
above would make SOME_NAME have the value 'y', because when you
executed "true" it was successful. I have this dim memory of allowing
it to set strings too (filling in the default value with the stdout
output from the shell execution).

So I had some experimental patch like that, and it kind of worked, but
I never finished it.

But the reason I never completed it was that for the compiler option
case, it really wanted more than a shell command, it needed to get the
whole $(CC) etc from the make environment.

I don't remember the exact syntax I used, but I think it was based on
that "option env" syntax, just replacing "env" with "shell". But
searching my mail archives I can't find anything, so I may never have
sent anything out. And so the patch is long gone.

Maybe I'll get frustrated enough and try to recreate it. I don't think
the patch was that big (but as mentioned, it really wasn't in a form
where it was _useful_ yet).

Linus

2018-02-08 00:59:30

by Kees Cook

[permalink] [raw]
Subject: Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

On Thu, Feb 8, 2018 at 10:44 AM, Masahiro Yamada
<[email protected]> wrote:
> 2018-02-08 2:55 GMT+09:00 Linus Torvalds <[email protected]>:
>> What I would really want - and this is entirely unrelated to this
>> particular case - is to have those damn compiler option tests as part
>> of the config phase in general. We now have about a million of these
>> crazy things, where we have config options that simply depend on which
>> compiler we have, and we have no sane way to show them at
>> configuration time.
>>
>> Though Andrew's tree I got yet another ugly hack
>> (CONFIG_CC_STACKPROTECTOR_AUTO) that handles just _one_ special case
>> by turning it into a special magic Kconfig entry in the main Makefile.
>> See commit 44c6dc940b19 ("Makefile: introduce
>> CONFIG_CC_STACKPROTECTOR_AUTO"). I wasn't sure if I really wanted it,
>> and honestly, I'm still thinking of just reverting it, because it's
>> _so_ ugly and _so_ wrong.
>>
>> What we need is an extension to the Kconfig language itself so that we can do
>>
>> config CC_HAS_RETPOLINE
>> cc_option "-mindirect-branch=thunk -mindirect-branch-table"
>>
>> or something. And then we can make sane _conditional_ dependencies at
>> Kconfig time, and our makefiles would be much cleaner too when you
>> could just do
>>
>> cflags-$(USE_RETPOLINE) += -mfunction-return=thunk -mindirect-branch-table
>>
>> because the validity of the C compiler flag has been tested when configuring.
>>
>> And then we could add that warning at configure time (or just disable
>> the option there thanks to "depends on CC_HAS_xyz" logic).
>>
>> All our compiler option handling right now is just nasty nasty nasty crud.
>>
>> Adding more people in the hopes that somebody gets motivated.. I've
>> talked about this before, so far we haven't made any progress.
>
>
> Sorry for slow progress.
>
> I agreed this before, and still motivated.
> (because I also motivated to remove kbuild cache.
> This turned out not so clever as I first thought)
>
> I was trying to do this, but in this development cycle
> I spent most of my time to flush out lots of piled up Kconfig patches.
> Sorry.
>
> Unless somebody is working, I will.

FWIW, I did try to do this before I went with the STACKPROTECTOR_AUTO
solution, and it defeated me at every turn. Between the circular
dependency of the Makefile setting up KBUILD flags and Kconfig wanting
to know about the compiler, I got stuck. And it also seemed like the
cache was a problem too, as I couldn't find a way to re-evaluate the
script-controlled CONFIG items once it got cached.

And ultimately, a lot of the operational logic ended up sticking
around in the Makefile anyway (to provide fallback decisions, warns,
and errors). To correctly deal with the complex corner-cases for
stack-protector, I end up with three pieces:

the user config:

- do you want auto, weak, strong, or off?

the compiler tests:

- which of weak, strong, or off are supported?
- does the flag _actually produce working code_ for this
architecture/compiler version/etc?

and the build behavior:

- for auto, choose best available flag and warn if none
- for user-selected weak or strong, stop if unavailable/non-functional
- for off (or auto-none) use the "off" flag, if it is supported (to
force-disable it on ssp-by-default distro compilers)

And all of this needs to bypass the Kconfig cache, since if it gets
cached, it won't get re-evaluated when a selection is changed. If we
had a working "option shell $(CC) ..." then I could do the "compiler
tests" part in Kconfig, and I could probably do the bulk of the "build
behavior" logic in Kconfig too, though any intermediate configs would
need to avoid getting cached too.

The ssp handling has always been extremely complex due to all its gory
details, but I've tried _really_ hard to improve it, keep it
documented, and in one place (e.g. the compiler tests used to be
stuffed in per-arch Makefile).

-Kees

--
Kees Cook
Pixel Security

2018-02-08 17:13:50

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH 6/6] s390: introduce execute-trampolines for branches

2018-02-08 9:57 GMT+09:00 Kees Cook <[email protected]>:
> On Thu, Feb 8, 2018 at 10:44 AM, Masahiro Yamada
> <[email protected]> wrote:
>> 2018-02-08 2:55 GMT+09:00 Linus Torvalds <[email protected]>:
>>> What I would really want - and this is entirely unrelated to this
>>> particular case - is to have those damn compiler option tests as part
>>> of the config phase in general. We now have about a million of these
>>> crazy things, where we have config options that simply depend on which
>>> compiler we have, and we have no sane way to show them at
>>> configuration time.
>>>
>>> Though Andrew's tree I got yet another ugly hack
>>> (CONFIG_CC_STACKPROTECTOR_AUTO) that handles just _one_ special case
>>> by turning it into a special magic Kconfig entry in the main Makefile.
>>> See commit 44c6dc940b19 ("Makefile: introduce
>>> CONFIG_CC_STACKPROTECTOR_AUTO"). I wasn't sure if I really wanted it,
>>> and honestly, I'm still thinking of just reverting it, because it's
>>> _so_ ugly and _so_ wrong.
>>>
>>> What we need is an extension to the Kconfig language itself so that we can do
>>>
>>> config CC_HAS_RETPOLINE
>>> cc_option "-mindirect-branch=thunk -mindirect-branch-table"
>>>
>>> or something. And then we can make sane _conditional_ dependencies at
>>> Kconfig time, and our makefiles would be much cleaner too when you
>>> could just do
>>>
>>> cflags-$(USE_RETPOLINE) += -mfunction-return=thunk -mindirect-branch-table
>>>
>>> because the validity of the C compiler flag has been tested when configuring.
>>>
>>> And then we could add that warning at configure time (or just disable
>>> the option there thanks to "depends on CC_HAS_xyz" logic).
>>>
>>> All our compiler option handling right now is just nasty nasty nasty crud.
>>>
>>> Adding more people in the hopes that somebody gets motivated.. I've
>>> talked about this before, so far we haven't made any progress.
>>
>>
>> Sorry for slow progress.
>>
>> I agreed this before, and still motivated.
>> (because I also motivated to remove kbuild cache.
>> This turned out not so clever as I first thought)
>>
>> I was trying to do this, but in this development cycle
>> I spent most of my time to flush out lots of piled up Kconfig patches.
>> Sorry.
>>
>> Unless somebody is working, I will.

I sent the first draft.

For people interested, I pushed the series in the following too.

git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
kconfig-opt-shell-rfc



--
Best Regards
Masahiro Yamada