Lfence is preferred than general retpoline on AMD, add this option
in C / inline asm just as the ASM code does.
For x86_64, it still help to have minimal retpoline for kernel even
if gcc doesn't support it, change the inline asm for x86 so that it
could also be used by x86_64.
Add ANNOTATE_NOSPEC_ALTERNATIVE for i386 to avoid below warning:
"warning: objtool: .altinstr_replacement+0x10: unsupported
intra-function call"
"warning: objtool: If this is a retpoline, please patch it
in with alternatives and annotate it with ANNOTATE_NOSPEC_ALTERNATIVE."
Signed-off-by: Zhenzhong Duan <[email protected]>
---
arch/x86/include/asm/nospec-branch.h | 23 ++++++++++++++++-------
1 files changed, 16 insertions(+), 7 deletions(-)
diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
index fd2a8c1..2d49eab 100644
--- a/arch/x86/include/asm/nospec-branch.h
+++ b/arch/x86/include/asm/nospec-branch.h
@@ -170,21 +170,26 @@
*/
# define CALL_NOSPEC \
ANNOTATE_NOSPEC_ALTERNATIVE \
- ALTERNATIVE( \
+ ALTERNATIVE_2( \
ANNOTATE_RETPOLINE_SAFE \
"call *%[thunk_target]\n", \
"call __x86_indirect_thunk_%V[thunk_target]\n", \
- X86_FEATURE_RETPOLINE)
+ X86_FEATURE_RETPOLINE, \
+ "lfence;\n" \
+ ANNOTATE_RETPOLINE_SAFE \
+ "call *%[thunk_target]\n", \
+ X86_FEATURE_RETPOLINE_AMD)
# define THUNK_TARGET(addr) [thunk_target] "r" (addr)
-#elif defined(CONFIG_X86_32) && defined(CONFIG_RETPOLINE)
+#elif defined(CONFIG_RETPOLINE)
/*
* For i386 we use the original ret-equivalent retpoline, because
* otherwise we'll run out of registers. We don't care about CET
* here, anyway.
*/
# define CALL_NOSPEC \
- ALTERNATIVE( \
+ ANNOTATE_NOSPEC_ALTERNATIVE \
+ ALTERNATIVE_2( \
ANNOTATE_RETPOLINE_SAFE \
"call *%[thunk_target]\n", \
" jmp 904f;\n" \
@@ -194,12 +199,16 @@
" lfence;\n" \
" jmp 902b;\n" \
" .align 16\n" \
- "903: addl $4, %%esp;\n" \
- " pushl %[thunk_target];\n" \
+ "903: add $4, %%" _ASM_SP ";\n" \
+ " push %[thunk_target];\n" \
" ret;\n" \
" .align 16\n" \
"904: call 901b;\n", \
- X86_FEATURE_RETPOLINE)
+ X86_FEATURE_RETPOLINE, \
+ "lfence;\n" \
+ ANNOTATE_RETPOLINE_SAFE \
+ "call *%[thunk_target]\n", \
+ X86_FEATURE_RETPOLINE_AMD)
# define THUNK_TARGET(addr) [thunk_target] "rm" (addr)
#else /* No retpoline for C / inline asm */
--
1.7.3
On Mon, Sep 17, 2018 at 10:17:30PM -0700, Zhenzhong Duan wrote:
> Lfence is preferred than general retpoline on AMD, add this option
> in C / inline asm just as the ASM code does.
>
> For x86_64, it still help to have minimal retpoline for kernel even
> if gcc doesn't support it, change the inline asm for x86 so that it
> could also be used by x86_64.
> Add ANNOTATE_NOSPEC_ALTERNATIVE for i386 to avoid below warning:
> "warning: objtool: .altinstr_replacement+0x10: unsupported
> intra-function call"
> "warning: objtool: If this is a retpoline, please patch it
> in with alternatives and annotate it with ANNOTATE_NOSPEC_ALTERNATIVE."
This Changelog is almost unreadable, please rewrite.
Reverse engineering the patch you add RETPOLINE_AMD support to the
inline-asm CALL_NOSPEC so that they match the asm CALL_NOSPEC.
> Signed-off-by: Zhenzhong Duan <[email protected]>
> ---
> arch/x86/include/asm/nospec-branch.h | 23 ++++++++++++++++-------
> 1 files changed, 16 insertions(+), 7 deletions(-)
>
> diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
> index fd2a8c1..2d49eab 100644
> --- a/arch/x86/include/asm/nospec-branch.h
> +++ b/arch/x86/include/asm/nospec-branch.h
> @@ -170,21 +170,26 @@
> */
> # define CALL_NOSPEC \
> ANNOTATE_NOSPEC_ALTERNATIVE \
> - ALTERNATIVE( \
> + ALTERNATIVE_2( \
> ANNOTATE_RETPOLINE_SAFE \
> "call *%[thunk_target]\n", \
> "call __x86_indirect_thunk_%V[thunk_target]\n", \
> - X86_FEATURE_RETPOLINE)
> + X86_FEATURE_RETPOLINE, \
> + "lfence;\n" \
> + ANNOTATE_RETPOLINE_SAFE \
> + "call *%[thunk_target]\n", \
> + X86_FEATURE_RETPOLINE_AMD)
> # define THUNK_TARGET(addr) [thunk_target] "r" (addr)
That's OK.
>
> -#elif defined(CONFIG_X86_32) && defined(CONFIG_RETPOLINE)
> +#elif defined(CONFIG_RETPOLINE)
This doesn't make any sense..
> /*
> * For i386 we use the original ret-equivalent retpoline, because
> * otherwise we'll run out of registers. We don't care about CET
> * here, anyway.
> */
> # define CALL_NOSPEC \
> - ALTERNATIVE( \
> + ANNOTATE_NOSPEC_ALTERNATIVE \
> + ALTERNATIVE_2( \
> ANNOTATE_RETPOLINE_SAFE \
> "call *%[thunk_target]\n", \
> " jmp 904f;\n" \
> @@ -194,12 +199,16 @@
> " lfence;\n" \
> " jmp 902b;\n" \
> " .align 16\n" \
> - "903: addl $4, %%esp;\n" \
> - " pushl %[thunk_target];\n" \
> + "903: add $4, %%" _ASM_SP ";\n" \
> + " push %[thunk_target];\n" \
Yeah, don't do that.
> " ret;\n" \
> " .align 16\n" \
> "904: call 901b;\n", \
> - X86_FEATURE_RETPOLINE)
> + X86_FEATURE_RETPOLINE, \
> + "lfence;\n" \
> + ANNOTATE_RETPOLINE_SAFE \
> + "call *%[thunk_target]\n", \
> + X86_FEATURE_RETPOLINE_AMD)
And that's OK again.
On 2018/9/18 17:50, Peter Zijlstra wrote:
> On Mon, Sep 17, 2018 at 10:17:30PM -0700, Zhenzhong Duan wrote:
>> Lfence is preferred than general retpoline on AMD, add this option
>> in C / inline asm just as the ASM code does.
>>
>> For x86_64, it still help to have minimal retpoline for kernel even
>> if gcc doesn't support it, change the inline asm for x86 so that it
>> could also be used by x86_64.
>> Add ANNOTATE_NOSPEC_ALTERNATIVE for i386 to avoid below warning:
>> "warning: objtool: .altinstr_replacement+0x10: unsupported
>> intra-function call"
>> "warning: objtool: If this is a retpoline, please patch it
>> in with alternatives and annotate it with ANNOTATE_NOSPEC_ALTERNATIVE."
>
> This Changelog is almost unreadable, please rewrite.
Sorry, I'll rewrite it.
>
> Reverse engineering the patch you add RETPOLINE_AMD support to the
> inline-asm CALL_NOSPEC so that they match the asm CALL_NOSPEC.
>
>> Signed-off-by: Zhenzhong Duan <[email protected]>
>> ---
>> arch/x86/include/asm/nospec-branch.h | 23 ++++++++++++++++-------
>> 1 files changed, 16 insertions(+), 7 deletions(-)
>>
>> diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
>> index fd2a8c1..2d49eab 100644
>> --- a/arch/x86/include/asm/nospec-branch.h
>> +++ b/arch/x86/include/asm/nospec-branch.h
>> @@ -170,21 +170,26 @@
>> */
>> # define CALL_NOSPEC \
>> ANNOTATE_NOSPEC_ALTERNATIVE \
>> - ALTERNATIVE( \
>> + ALTERNATIVE_2( \
>> ANNOTATE_RETPOLINE_SAFE \
>> "call *%[thunk_target]\n", \
>> "call __x86_indirect_thunk_%V[thunk_target]\n", \
>> - X86_FEATURE_RETPOLINE)
>> + X86_FEATURE_RETPOLINE, \
>> + "lfence;\n" \
>> + ANNOTATE_RETPOLINE_SAFE \
>> + "call *%[thunk_target]\n", \
>> + X86_FEATURE_RETPOLINE_AMD)
>> # define THUNK_TARGET(addr) [thunk_target] "r" (addr)
>
> That's OK.
>
>>
>> -#elif defined(CONFIG_X86_32) && defined(CONFIG_RETPOLINE)
>> +#elif defined(CONFIG_RETPOLINE)
>
> This doesn't make any sense..
This change is used for x86_64 to have minimal Retpoline support when
CONFIG_RETPOLINE is defined but RETPOLINE isn't defined, or I missed
something?
>
>> /*
>> * For i386 we use the original ret-equivalent retpoline, because
>> * otherwise we'll run out of registers. We don't care about CET
>> * here, anyway.
>> */
>> # define CALL_NOSPEC \
>> - ALTERNATIVE( \
>> + ANNOTATE_NOSPEC_ALTERNATIVE \
>> + ALTERNATIVE_2( \
>> ANNOTATE_RETPOLINE_SAFE \
>> "call *%[thunk_target]\n", \
>> " jmp 904f;\n" \
>> @@ -194,12 +199,16 @@
>> " lfence;\n" \
>> " jmp 902b;\n" \
>> " .align 16\n" \
>> - "903: addl $4, %%esp;\n" \
>> - " pushl %[thunk_target];\n" \
>> + "903: add $4, %%" _ASM_SP ";\n" \
>> + " push %[thunk_target];\n" \
>
> Yeah, don't do that.
This is the change for above reason.
Thanks
Zhenzhong
On Tue, Sep 18, 2018 at 06:31:07PM +0800, Zhenzhong Duan wrote:
> On 2018/9/18 17:50, Peter Zijlstra wrote:
> > On Mon, Sep 17, 2018 at 10:17:30PM -0700, Zhenzhong Duan wrote:
> > > -#elif defined(CONFIG_X86_32) && defined(CONFIG_RETPOLINE)
> > > +#elif defined(CONFIG_RETPOLINE)
> >
> > This doesn't make any sense..
> This change is used for x86_64 to have minimal Retpoline support when
> CONFIG_RETPOLINE is defined but RETPOLINE isn't defined, or I missed
> something?
No it doesn't.
#if defined(X86_64) && defined(RETPOLINE)
/* x86_64 retpoline goes here */
#elif defined(RETPOLINE)
/* !x86_64 retpoline goes here */
#else
/* !retpoline goes here
#endif
On 2018/9/18 18:59, Peter Zijlstra wrote:
> On Tue, Sep 18, 2018 at 06:31:07PM +0800, Zhenzhong Duan wrote:
>> On 2018/9/18 17:50, Peter Zijlstra wrote:
>>> On Mon, Sep 17, 2018 at 10:17:30PM -0700, Zhenzhong Duan wrote:
>>>> -#elif defined(CONFIG_X86_32) && defined(CONFIG_RETPOLINE)
>>>> +#elif defined(CONFIG_RETPOLINE)
>>>
>>> This doesn't make any sense..
>> This change is used for x86_64 to have minimal Retpoline support when
>> CONFIG_RETPOLINE is defined but RETPOLINE isn't defined, or I missed
>> something?
>
> No it doesn't.
>
> #if defined(X86_64) && defined(RETPOLINE)
>
> /* x86_64 retpoline goes here */
>
> #elif defined(RETPOLINE)
>
> /* !x86_64 retpoline goes here */
>
> #else
>
> /* !retpoline goes here
>
> #endif
Sorry, but I am confused.
So where is 'if defined(x86_64) && !defined(RETPOLINE) &&
defined(CONFIG_RETPOLINE)' go?
In original code, it will go to "call *%[thunk_target]\n" while
we have set SPECTRE_V2_RETPOLINE_MINIMAL or
SPECTRE_V2_RETPOLINE_MINIMAL_AMD. Is this expected?
Thanks
Zhenzhong
On Tue, Sep 18, 2018 at 08:04:44PM +0800, Zhenzhong Duan wrote:
> On 2018/9/18 18:59, Peter Zijlstra wrote:
> > On Tue, Sep 18, 2018 at 06:31:07PM +0800, Zhenzhong Duan wrote:
> > > On 2018/9/18 17:50, Peter Zijlstra wrote:
> > > > On Mon, Sep 17, 2018 at 10:17:30PM -0700, Zhenzhong Duan wrote:
> > > > > -#elif defined(CONFIG_X86_32) && defined(CONFIG_RETPOLINE)
> > > > > +#elif defined(CONFIG_RETPOLINE)
> > > >
> > > > This doesn't make any sense..
> > > This change is used for x86_64 to have minimal Retpoline support when
> > > CONFIG_RETPOLINE is defined but RETPOLINE isn't defined, or I missed
> > > something?
> >
> > No it doesn't.
> >
> > #if defined(X86_64) && defined(RETPOLINE)
> >
> > /* x86_64 retpoline goes here */
> >
> > #elif defined(RETPOLINE)
> >
> > /* !x86_64 retpoline goes here */
> >
> > #else
> >
> > /* !retpoline goes here
> >
> > #endif
>
> Sorry, but I am confused.
> So where is 'if defined(x86_64) && !defined(RETPOLINE) &&
> defined(CONFIG_RETPOLINE)' go?
Argh, CONFIG_RETPOLINE vs RETPOLINE :/
The thing is, the one you modify has a comment on that explains why it
is i386 only. CET and retpolines don't like one another much.
And the x86_64 version uses %V which requires new GCC.
So I'm all for fixing the RETPOLINE_AMD thing, but at this point nobody
should use the minimal stuff, that's just delusional.
> In original code, it will go to "call *%[thunk_target]\n" while
> we have set SPECTRE_V2_RETPOLINE_MINIMAL or
> SPECTRE_V2_RETPOLINE_MINIMAL_AMD. Is this expected?
Yes, that is exactly right -- it does that with or without your change
though.
> > In original code, it will go to "call *%[thunk_target]\n" while
> > we have set SPECTRE_V2_RETPOLINE_MINIMAL or
> > SPECTRE_V2_RETPOLINE_MINIMAL_AMD. Is this expected?
>
> Yes, that is exactly right -- it does that with or without your change
> though.
Soryr, I wrote that before I noticed the CONFIG_RETPOLINE vs RETPOLINE
thing.
In any case, since retpoline capable compilers are widely available, I'm
all for making CONFIG_RETPOLINE hard depend on it. That minimal stuff
really is quite silly.
On Tue, 2018-09-18 at 15:03 +0200, Peter Zijlstra wrote:
> > > In original code, it will go to "call *%[thunk_target]\n" while
> > > we have set SPECTRE_V2_RETPOLINE_MINIMAL or
> > > SPECTRE_V2_RETPOLINE_MINIMAL_AMD. Is this expected?
> >
> > Yes, that is exactly right -- it does that with or without your change
> > though.
>
> Soryr, I wrote that before I noticed the CONFIG_RETPOLINE vs RETPOLINE
> thing.
>
> In any case, since retpoline capable compilers are widely available, I'm
> all for making CONFIG_RETPOLINE hard depend on it. That minimal stuff
> really is quite silly.
Yep, now the the compiler support is widespread that makes a lot of
sense.
On 2018/9/18 21:00, Peter Zijlstra wrote:
> On Tue, Sep 18, 2018 at 08:04:44PM +0800, Zhenzhong Duan wrote:
>> On 2018/9/18 18:59, Peter Zijlstra wrote:
>>> On Tue, Sep 18, 2018 at 06:31:07PM +0800, Zhenzhong Duan wrote:
>>>> On 2018/9/18 17:50, Peter Zijlstra wrote:
>>>>> On Mon, Sep 17, 2018 at 10:17:30PM -0700, Zhenzhong Duan wrote:
>>>>>> -#elif defined(CONFIG_X86_32) && defined(CONFIG_RETPOLINE)
>>>>>> +#elif defined(CONFIG_RETPOLINE)
>>>>> This doesn't make any sense..
>>>> This change is used for x86_64 to have minimal Retpoline support when
>>>> CONFIG_RETPOLINE is defined but RETPOLINE isn't defined, or I missed
>>>> something?
>>> No it doesn't.
>>>
>>> #if defined(X86_64) && defined(RETPOLINE)
>>>
>>> /* x86_64 retpoline goes here */
>>>
>>> #elif defined(RETPOLINE)
>>>
>>> /* !x86_64 retpoline goes here */
>>>
>>> #else
>>>
>>> /* !retpoline goes here
>>>
>>> #endif
>> Sorry, but I am confused.
>> So where is 'if defined(x86_64) && !defined(RETPOLINE) &&
>> defined(CONFIG_RETPOLINE)' go?
> Argh, CONFIG_RETPOLINE vs RETPOLINE :/
>
> The thing is, the one you modify has a comment on that explains why it
> is i386 only. CET and retpolines don't like one another much.
>
> And the x86_64 version uses %V which requires new GCC.
>
> So I'm all for fixing the RETPOLINE_AMD thing, but at this point nobody
> should use the minimal stuff, that's just delusional.
>
Clear, thanks for your explanation.
Zhenzhong