2022-05-02 20:20:20

by Peter Zijlstra

[permalink] [raw]
Subject: [PATCH] objtool: Fix SLS checks


Fix the SLS validation; not having a next instruction is also a fail
when the next instruction should be INSN_TRAP.

Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
---
tools/objtool/check.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index 3f6785415894..3354101ffe34 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -3380,7 +3380,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,

case INSN_RETURN:
if (sls && !insn->retpoline_safe &&
- next_insn && next_insn->type != INSN_TRAP) {
+ (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
WARN_FUNC("missing int3 after ret",
insn->sec, insn->offset);
}
@@ -3428,7 +3428,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,

case INSN_JUMP_DYNAMIC:
if (sls && !insn->retpoline_safe &&
- next_insn && next_insn->type != INSN_TRAP) {
+ (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
WARN_FUNC("missing int3 after indirect jump",
insn->sec, insn->offset);
}


2022-05-03 00:30:36

by Josh Poimboeuf

[permalink] [raw]
Subject: Re: [PATCH] objtool: Fix SLS checks

On Sat, Apr 30, 2022 at 12:50:02PM +0200, Peter Zijlstra wrote:
>
> Fix the SLS validation; not having a next instruction is also a fail
> when the next instruction should be INSN_TRAP.
>
> Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
> ---
> tools/objtool/check.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/tools/objtool/check.c b/tools/objtool/check.c
> index 3f6785415894..3354101ffe34 100644
> --- a/tools/objtool/check.c
> +++ b/tools/objtool/check.c
> @@ -3380,7 +3380,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
>
> case INSN_RETURN:
> if (sls && !insn->retpoline_safe &&
> - next_insn && next_insn->type != INSN_TRAP) {
> + (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
> WARN_FUNC("missing int3 after ret",
> insn->sec, insn->offset);
> }
> @@ -3428,7 +3428,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
>
> case INSN_JUMP_DYNAMIC:
> if (sls && !insn->retpoline_safe &&
> - next_insn && next_insn->type != INSN_TRAP) {
> + (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
> WARN_FUNC("missing int3 after indirect jump",
> insn->sec, insn->offset);
> }

My SLS rewrite in tip/objtool/core already fixed this, FWIW. But this
could be good for -urgent.

Acked-by: Josh Poimboeuf <[email protected]>

Here's another SLS improvement I mentioned to you the other day, do you
agree we should do this as well?


From: Josh Poimboeuf <[email protected]>
Subject: [PATCH] x86/speculation: Mitigate SLS for JMP_NOSPEC with retpolines disabled

Having disabled retpolines doesn't necessarily mean the user doesn't
care about straight-line speculation. For example, retpolines are
disabled when eIBRS is used.

If CONFIG_SLS is enabled, properly mitigate SLS for JMP_NOSPEC for the
retpolines disabled cases.

Signed-off-by: Josh Poimboeuf <[email protected]>
---
arch/x86/include/asm/nospec-branch.h | 13 ++++++++++---
tools/objtool/check.c | 3 ---
2 files changed, 10 insertions(+), 6 deletions(-)

diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
index acbaeaf83b61..0648746bf60b 100644
--- a/arch/x86/include/asm/nospec-branch.h
+++ b/arch/x86/include/asm/nospec-branch.h
@@ -75,6 +75,13 @@
.popsection
.endm

+.macro INDIRECT_JMP reg
+ jmp *%\reg
+#ifdef CONFIG_SLS
+ int3
+#endif
+.endm
+
/*
* JMP_NOSPEC and CALL_NOSPEC macros can be used instead of a simple
* indirect jmp/call which may be susceptible to the Spectre variant 2
@@ -82,11 +89,11 @@
*/
.macro JMP_NOSPEC reg:req
#ifdef CONFIG_RETPOLINE
- ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), \
+ ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), \
__stringify(jmp __x86_indirect_thunk_\reg), X86_FEATURE_RETPOLINE, \
- __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), X86_FEATURE_RETPOLINE_LFENCE
+ __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), X86_FEATURE_RETPOLINE_LFENCE
#else
- jmp *%\reg
+ INDIRECT_JMP \reg
#endif
.endm

diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index e7983c3e2408..3cf3ad0b5db5 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -3842,9 +3842,6 @@ static int validate_sls(struct objtool_file *file)
for_each_insn(file, insn) {
next_insn = next_insn_same_sec(file, insn);

- if (insn->retpoline_safe)
- continue;
-
switch (insn->type) {
case INSN_RETURN:
if (!next_insn || next_insn->type != INSN_TRAP) {
--
2.34.1

2022-05-03 00:34:25

by Peter Zijlstra

[permalink] [raw]
Subject: Re: [PATCH] objtool: Fix SLS checks

On Mon, May 02, 2022 at 11:15:47AM -0700, Josh Poimboeuf wrote:
> On Sat, Apr 30, 2022 at 12:50:02PM +0200, Peter Zijlstra wrote:
> >
> > Fix the SLS validation; not having a next instruction is also a fail
> > when the next instruction should be INSN_TRAP.
> >
> > Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
> > ---
> > tools/objtool/check.c | 4 ++--
> > 1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/tools/objtool/check.c b/tools/objtool/check.c
> > index 3f6785415894..3354101ffe34 100644
> > --- a/tools/objtool/check.c
> > +++ b/tools/objtool/check.c
> > @@ -3380,7 +3380,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
> >
> > case INSN_RETURN:
> > if (sls && !insn->retpoline_safe &&
> > - next_insn && next_insn->type != INSN_TRAP) {
> > + (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
> > WARN_FUNC("missing int3 after ret",
> > insn->sec, insn->offset);
> > }
> > @@ -3428,7 +3428,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
> >
> > case INSN_JUMP_DYNAMIC:
> > if (sls && !insn->retpoline_safe &&
> > - next_insn && next_insn->type != INSN_TRAP) {
> > + (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
> > WARN_FUNC("missing int3 after indirect jump",
> > insn->sec, insn->offset);
> > }
>
> My SLS rewrite in tip/objtool/core already fixed this, FWIW. But this
> could be good for -urgent.

Urgh, I should've looked at that indeed. This didn't find any new sites
though; so I don't think this needs to go through urgent.

2022-05-03 00:40:35

by Peter Zijlstra

[permalink] [raw]
Subject: Re: [PATCH] objtool: Fix SLS checks

On Mon, May 02, 2022 at 11:15:47AM -0700, Josh Poimboeuf wrote:

> From: Josh Poimboeuf <[email protected]>
> Subject: [PATCH] x86/speculation: Mitigate SLS for JMP_NOSPEC with retpolines disabled
>
> Having disabled retpolines doesn't necessarily mean the user doesn't
> care about straight-line speculation. For example, retpolines are
> disabled when eIBRS is used.
>
> If CONFIG_SLS is enabled, properly mitigate SLS for JMP_NOSPEC for the
> retpolines disabled cases.
>
> Signed-off-by: Josh Poimboeuf <[email protected]>
> ---
> arch/x86/include/asm/nospec-branch.h | 13 ++++++++++---
> tools/objtool/check.c | 3 ---
> 2 files changed, 10 insertions(+), 6 deletions(-)
>
> diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
> index acbaeaf83b61..0648746bf60b 100644
> --- a/arch/x86/include/asm/nospec-branch.h
> +++ b/arch/x86/include/asm/nospec-branch.h
> @@ -75,6 +75,13 @@
> .popsection
> .endm
>
> +.macro INDIRECT_JMP reg
> + jmp *%\reg
> +#ifdef CONFIG_SLS
> + int3
> +#endif
> +.endm
> +
> /*
> * JMP_NOSPEC and CALL_NOSPEC macros can be used instead of a simple
> * indirect jmp/call which may be susceptible to the Spectre variant 2
> @@ -82,11 +89,11 @@
> */
> .macro JMP_NOSPEC reg:req
> #ifdef CONFIG_RETPOLINE
> - ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), \
> + ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), \
> __stringify(jmp __x86_indirect_thunk_\reg), X86_FEATURE_RETPOLINE, \
> - __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), X86_FEATURE_RETPOLINE_LFENCE
> + __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), X86_FEATURE_RETPOLINE_LFENCE
> #else
> - jmp *%\reg
> + INDIRECT_JMP \reg
> #endif
> .endm
>
> diff --git a/tools/objtool/check.c b/tools/objtool/check.c
> index e7983c3e2408..3cf3ad0b5db5 100644
> --- a/tools/objtool/check.c
> +++ b/tools/objtool/check.c
> @@ -3842,9 +3842,6 @@ static int validate_sls(struct objtool_file *file)
> for_each_insn(file, insn) {
> next_insn = next_insn_same_sec(file, insn);
>
> - if (insn->retpoline_safe)
> - continue;
> -
> switch (insn->type) {
> case INSN_RETURN:
> if (!next_insn || next_insn->type != INSN_TRAP) {

Yes, agreed. But perhaps with something like this on top?

---
diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c
index 3c66073e7645..84beeb5297d5 100644
--- a/arch/x86/kernel/alternative.c
+++ b/arch/x86/kernel/alternative.c
@@ -452,6 +452,17 @@ static int patch_retpoline(void *addr, struct insn *insn, u8 *bytes)
return ret;
i += ret;

+#ifdef CONFIG_SLS
+ /*
+ * Ideally this would be unconditional, except in case of
+ * RETPOLINE_LFENCE we don't have sufficient space. Additionally,
+ * -mharden-sls=all should be extended to emit INT3 after
+ * direct jumps too, which will then cover that case.
+ */
+ if (i < insn->length)
+ bytes[i++] = INT3_INSN_OPCODE;
+#endif
+
for (; i < insn->length;)
bytes[i++] = BYTES_NOP1;

diff --git a/arch/x86/lib/retpoline.S b/arch/x86/lib/retpoline.S
index b2b2366885a2..26e742da3129 100644
--- a/arch/x86/lib/retpoline.S
+++ b/arch/x86/lib/retpoline.S
@@ -33,9 +33,9 @@ SYM_INNER_LABEL(__x86_indirect_thunk_\reg, SYM_L_GLOBAL)
UNWIND_HINT_EMPTY
ANNOTATE_NOENDBR

- ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; jmp *%\reg), \
+ ALTERNATIVE_2 __stringify(ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), \
__stringify(RETPOLINE \reg), X86_FEATURE_RETPOLINE, \
- __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; jmp *%\reg; int3), X86_FEATURE_RETPOLINE_LFENCE
+ __stringify(lfence; ANNOTATE_RETPOLINE_SAFE; INDIRECT_JMP \reg), X86_FEATURE_RETPOLINE_LFENCE

.endm

diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 16b6efacf7c6..6929856e7f6d 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -418,6 +418,10 @@ static void emit_indirect_jump(u8 **pprog, int reg, u8 *ip)
#endif
EMIT2(0xFF, 0xE0 + reg);

+#ifdef CONFIG_SLS
+ EMIT1(0xCC);
+#endif
+
*pprog = prog;
}

2022-05-03 00:48:25

by Josh Poimboeuf

[permalink] [raw]
Subject: Re: [PATCH] objtool: Fix SLS checks

On Mon, May 02, 2022 at 10:01:53PM +0200, Peter Zijlstra wrote:
> On Mon, May 02, 2022 at 11:15:47AM -0700, Josh Poimboeuf wrote:
> > On Sat, Apr 30, 2022 at 12:50:02PM +0200, Peter Zijlstra wrote:
> > >
> > > Fix the SLS validation; not having a next instruction is also a fail
> > > when the next instruction should be INSN_TRAP.
> > >
> > > Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
> > > ---
> > > tools/objtool/check.c | 4 ++--
> > > 1 file changed, 2 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/tools/objtool/check.c b/tools/objtool/check.c
> > > index 3f6785415894..3354101ffe34 100644
> > > --- a/tools/objtool/check.c
> > > +++ b/tools/objtool/check.c
> > > @@ -3380,7 +3380,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
> > >
> > > case INSN_RETURN:
> > > if (sls && !insn->retpoline_safe &&
> > > - next_insn && next_insn->type != INSN_TRAP) {
> > > + (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
> > > WARN_FUNC("missing int3 after ret",
> > > insn->sec, insn->offset);
> > > }
> > > @@ -3428,7 +3428,7 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,
> > >
> > > case INSN_JUMP_DYNAMIC:
> > > if (sls && !insn->retpoline_safe &&
> > > - next_insn && next_insn->type != INSN_TRAP) {
> > > + (!next_insn || (next_insn && next_insn->type != INSN_TRAP))) {
> > > WARN_FUNC("missing int3 after indirect jump",
> > > insn->sec, insn->offset);
> > > }
> >
> > My SLS rewrite in tip/objtool/core already fixed this, FWIW. But this
> > could be good for -urgent.
>
> Urgh, I should've looked at that indeed. This didn't find any new sites
> though; so I don't think this needs to go through urgent.

Well to be fair, it was easy to miss since I snuck it in with rewrite.

--
Josh

2022-05-04 04:17:49

by Josh Poimboeuf

[permalink] [raw]
Subject: Re: [PATCH] objtool: Fix SLS checks

On Mon, May 02, 2022 at 10:17:39PM +0200, Peter Zijlstra wrote:
> > +++ b/tools/objtool/check.c
> > @@ -3842,9 +3842,6 @@ static int validate_sls(struct objtool_file *file)
> > for_each_insn(file, insn) {
> > next_insn = next_insn_same_sec(file, insn);
> >
> > - if (insn->retpoline_safe)
> > - continue;
> > -
> > switch (insn->type) {
> > case INSN_RETURN:
> > if (!next_insn || next_insn->type != INSN_TRAP) {
>
> Yes, agreed. But perhaps with something like this on top?

Yup, I missed those... Looks good. Just one comment:

> +++ b/arch/x86/kernel/alternative.c
> @@ -452,6 +452,17 @@ static int patch_retpoline(void *addr, struct insn *insn, u8 *bytes)
> return ret;
> i += ret;
>
> +#ifdef CONFIG_SLS
> + /*
> + * Ideally this would be unconditional, except in case of
> + * RETPOLINE_LFENCE we don't have sufficient space. Additionally,
> + * -mharden-sls=all should be extended to emit INT3 after
> + * direct jumps too, which will then cover that case.
> + */

I don't quite follow this 2nd sentence and how it's related here, since
this function doesn't actually deal with direct jumps.

Speaking of, I guess we'll eventually need to hack this SLS mess into
jump labels :-/

> + if (i < insn->length)
> + bytes[i++] = INT3_INSN_OPCODE;
> +#endif

--
Josh