LinuxLists.cc - [PATCH v2 1/2] x86/speculation: Allow enabling STIBP with legacy IBRS

[permalink] [raw]

Subject: [PATCH v2 2/2] Documentation/hw-vuln: Document the interaction between IBRS and STIBP

Explain why STIBP is needed with legacy IBRS as currently implemented
(KERNEL_IBRS) and why STIBP is not needed when enhanced IBRS is enabled.

Signed-off-by: KP Singh <[email protected]>
---
Documentation/admin-guide/hw-vuln/spectre.rst | 22 ++++++++++++++-----
1 file changed, 17 insertions(+), 5 deletions(-)

diff --git a/Documentation/admin-guide/hw-vuln/spectre.rst b/Documentation/admin-guide/hw-vuln/spectre.rst
index c4dcdb3d0d45..e193ee13dc9a 100644
--- a/Documentation/admin-guide/hw-vuln/spectre.rst
+++ b/Documentation/admin-guide/hw-vuln/spectre.rst
@@ -479,8 +479,17 @@ Spectre variant 2
On Intel Skylake-era systems the mitigation covers most, but not all,
cases. See :ref:`[3] <spec_ref3>` for more details.

- On CPUs with hardware mitigation for Spectre variant 2 (e.g. Enhanced
- IBRS on x86), retpoline is automatically disabled at run time.
+ On CPUs with hardware mitigation for Spectre variant 2 (e.g. IBRS
+ or enhanced IBRS on x86), retpoline is automatically disabled at run time.
+
+ Setting the IBRS bit implicitly enables STIBP which guards against
+ cross-thread branch target injection on SMT systems. On systems with enhanced
+ IBRS, the kernel sets the bit once, which keeps cross-thread protections
+ always enabled, obviating the need for an explicit STIBP. On CPUs with legacy
+ IBRS, the kernel clears the IBRS bit on returning to user-space, thus also
+ disabling the implicit STIBP. Consequently, STIBP needs to be explicitly
+ enabled to guard against cross-thread attacks in userspace.
+

The retpoline mitigation is turned on by default on vulnerable
CPUs. It can be forced on or off by the administrator
@@ -504,9 +513,12 @@ Spectre variant 2
For Spectre variant 2 mitigation, individual user programs
can be compiled with return trampolines for indirect branches.
This protects them from consuming poisoned entries in the branch
- target buffer left by malicious software. Alternatively, the
- programs can disable their indirect branch speculation via prctl()
- (See :ref:`Documentation/userspace-api/spec_ctrl.rst <set_spec_ctrl>`).
+ target buffer left by malicious software.
+
+ On legacy IBRS systems, at return to userspace, implicit STIBP is disabled
+ because the kernel clears the IBRS bit. In this case, the userspace programs
+ can disable indirect branch speculation via prctl() (See
+ :ref:`Documentation/userspace-api/spec_ctrl.rst <set_spec_ctrl>`).
On x86, this will turn on STIBP to guard against attacks from the
sibling thread when the user program is running, and use IBPB to
flush the branch target buffer when switching to/from the program.
--
2.39.2.637.g21b0678d19-goog

2023-02-21 19:29:20

[permalink] [raw]

Subject: Re: [PATCH v2 1/2] x86/speculation: Allow enabling STIBP with legacy IBRS

2023-02-21 19:30:00

[permalink] [raw]

Subject: Re: [PATCH v2 1/2] x86/speculation: Allow enabling STIBP with legacy IBRS

2023-02-21 19:35:52

[permalink] [raw]

Subject: Re: [PATCH v2 1/2] x86/speculation: Allow enabling STIBP with legacy IBRS

On Tue, Feb 21, 2023 at 11:29 AM Greg KH <[email protected]> wrote:
>
> On Tue, Feb 21, 2023 at 07:49:07PM +0100, KP Singh wrote:
> > Setting the IBRS bit implicitly enables STIBP to protect against
> > cross-thread branch target injection. With enhanced IBRS, the bit it set
> > once and is not cleared again. However, on CPUs with just legacy IBRS,
> > IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a
> > KERNEL_IBRS). Clearing this bit also disables the implicitly enabled
> > STIBP, thus requiring some form of cross-thread protection in userspace.
> >
> > Enable STIBP, either opt-in via prctl or seccomp, or always on depending
> > on the choice of mitigation selected via spectre_v2_user.
> >
> > Reported-by: José Oliveira <[email protected]>
> > Reported-by: Rodrigo Branco <[email protected]>
> > Reviewed-by: Alexandra Sandulescu <[email protected]>
> > Fixes: 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")
> > Cc: [email protected]
> > Signed-off-by: KP Singh <[email protected]>
> > ---
> > arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++-----------
> > 1 file changed, 22 insertions(+), 11 deletions(-)
>
> Why isn't patch 2/2 for stable as well?

It should be. I actually forgot to remove stable from the first one as
there are still ongoing discussions and people kept having to "drop
stable". I can send a v3 with stable Cc'ed. Should it have a fixes
tag too?

>
> thanks,
>
> greg k-h

2023-02-21 19:47:48

[permalink] [raw]

Subject: Re: [PATCH v2 1/2] x86/speculation: Allow enabling STIBP with legacy IBRS

On Tue, Feb 21, 2023 at 11:35:29AM -0800, KP Singh wrote:
> On Tue, Feb 21, 2023 at 11:29 AM Greg KH <[email protected]> wrote:
> >
> > On Tue, Feb 21, 2023 at 07:49:07PM +0100, KP Singh wrote:
> > > Setting the IBRS bit implicitly enables STIBP to protect against
> > > cross-thread branch target injection. With enhanced IBRS, the bit it set
> > > once and is not cleared again. However, on CPUs with just legacy IBRS,
> > > IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a
> > > KERNEL_IBRS). Clearing this bit also disables the implicitly enabled
> > > STIBP, thus requiring some form of cross-thread protection in userspace.
> > >
> > > Enable STIBP, either opt-in via prctl or seccomp, or always on depending
> > > on the choice of mitigation selected via spectre_v2_user.
> > >
> > > Reported-by: Jos? Oliveira <[email protected]>
> > > Reported-by: Rodrigo Branco <[email protected]>
> > > Reviewed-by: Alexandra Sandulescu <[email protected]>
> > > Fixes: 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")
> > > Cc: [email protected]
> > > Signed-off-by: KP Singh <[email protected]>
> > > ---
> > > arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++-----------
> > > 1 file changed, 22 insertions(+), 11 deletions(-)
> >
> > Why isn't patch 2/2 for stable as well?
>
> It should be. I actually forgot to remove stable from the first one as
> there are still ongoing discussions and people kept having to "drop
> stable". I can send a v3 with stable Cc'ed. Should it have a fixes
> tag too?

Why does anyone need to "drop stable" from a patch discussion? That's
not a problem, we _WANT_ to see the patch review and discussion also
copied there to be aware of what is coming down the pipeline. So
whomever said that is not correct, sorry.

And yes, a fixes: tag would be nice.

thanks,

greg k-h

2023-02-21 19:57:37

[permalink] [raw]

Subject: Re: [PATCH v2 1/2] x86/speculation: Allow enabling STIBP with legacy IBRS

On Tue, Feb 21, 2023 at 08:47:38PM +0100, Greg KH wrote:
> Why does anyone need to "drop stable" from a patch discussion? That's
> not a problem, we _WANT_ to see the patch review and discussion also
> copied there to be aware of what is coming down the pipeline. So
> whomever said that is not correct, sorry.

Someone dropped stable because you used to send automated formletter
mails that this is not how one should submit a patch to stable. I guess
that is not needed anymore so I'll stop dropping stable.

--
Regards/Gruss,
Boris.

https://people.kernel.org/tglx/notes-about-netiquette

2023-02-21 20:10:09

[permalink] [raw]

Subject: Re: [PATCH v2 1/2] x86/speculation: Allow enabling STIBP with legacy IBRS

On Tue, Feb 21, 2023 at 08:57:26PM +0100, Borislav Petkov wrote:
> On Tue, Feb 21, 2023 at 08:47:38PM +0100, Greg KH wrote:
> > Why does anyone need to "drop stable" from a patch discussion? That's
> > not a problem, we _WANT_ to see the patch review and discussion also
> > copied there to be aware of what is coming down the pipeline. So
> > whomever said that is not correct, sorry.
>
> Someone dropped stable because you used to send automated formletter
> mails that this is not how one should submit a patch to stable. I guess
> that is not needed anymore so I'll stop dropping stable.

I still send them, and so does 0-day, IF you send the emails
incorrectly. So far, that's not been the case for this series at all
(hint, there needs to be a cc: stable in the signed-off-by area of the
patch, that's all.)

thanks,

greg k-h

2023-02-21 20:23:49

[permalink] [raw]

Subject: Re: [PATCH v2 1/2] x86/speculation: Allow enabling STIBP with legacy IBRS

On Tue, Feb 21, 2023 at 09:09:32PM +0100, Greg KH wrote:
> I still send them, and so does 0-day, IF you send the emails
> incorrectly. So far, that's not been the case for this series at all
> (hint, there needs to be a cc: stable in the signed-off-by area of the
> patch, that's all.)

Aha, you send it when there's no Cc: stable in the SOB area but stable
is still CCed.

Bah, forget what I said.

--
Regards/Gruss,
Boris.

https://people.kernel.org/tglx/notes-about-netiquette

2023-02-22 03:09:17

by Pawan Gupta

[permalink] [raw]

Subject: Re: [PATCH v2 1/2] x86/speculation: Allow enabling STIBP with legacy IBRS

On Tue, Feb 21, 2023 at 07:49:07PM +0100, KP Singh wrote:
> Setting the IBRS bit implicitly enables STIBP to protect against
> cross-thread branch target injection. With enhanced IBRS, the bit it set
> once and is not cleared again. However, on CPUs with just legacy IBRS,
> IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a
> KERNEL_IBRS). Clearing this bit also disables the implicitly enabled
> STIBP, thus requiring some form of cross-thread protection in userspace.
>
> Enable STIBP, either opt-in via prctl or seccomp, or always on depending
> on the choice of mitigation selected via spectre_v2_user.
>
> Reported-by: José Oliveira <[email protected]>
> Reported-by: Rodrigo Branco <[email protected]>
> Reviewed-by: Alexandra Sandulescu <[email protected]>
> Fixes: 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")
> Cc: [email protected]
> Signed-off-by: KP Singh <[email protected]>
> ---
> arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++-----------
> 1 file changed, 22 insertions(+), 11 deletions(-)
>
> diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
> index 85168740f76a..5be6075d8e36 100644
> --- a/arch/x86/kernel/cpu/bugs.c
> +++ b/arch/x86/kernel/cpu/bugs.c
> @@ -1124,14 +1124,30 @@ spectre_v2_parse_user_cmdline(void)
> return SPECTRE_V2_USER_CMD_AUTO;
> }
>
> -static inline bool spectre_v2_in_ibrs_mode(enum spectre_v2_mitigation mode)
> +static inline bool spectre_v2_in_eibrs_mode(enum spectre_v2_mitigation mode)
> {
> - return mode == SPECTRE_V2_IBRS ||
> - mode == SPECTRE_V2_EIBRS ||
> + return mode == SPECTRE_V2_EIBRS ||
> mode == SPECTRE_V2_EIBRS_RETPOLINE ||
> mode == SPECTRE_V2_EIBRS_LFENCE;
> }
>
> +static inline bool spectre_v2_in_ibrs_mode(enum spectre_v2_mitigation mode)
> +{
> + return spectre_v2_in_eibrs_mode(mode) || mode == SPECTRE_V2_IBRS;
> +}
> +
> +static inline bool spectre_v2_user_needs_stibp(enum spectre_v2_mitigation mode)
> +{
> + /*
> + * enhanced IBRS also protects against user-mode attacks as the IBRS bit

Maybe:
* Enhanced IBRS mode also protects against cross-thread user-to-user
* attacks as the IBRS bit

> + * remains always set which implicitly enables cross-thread protections.
> + * However, In legacy IBRS mode, the IBRS bit is set only in kernel
> + * and cleared on return to userspace. This disables the implicit
> + * cross-thread protections and STIBP is needed.
> + */
> + return !spectre_v2_in_eibrs_mode(mode);
> +}
> +
> static void __init
> spectre_v2_user_select_mitigation(void)
> {
> @@ -1193,13 +1209,8 @@ spectre_v2_user_select_mitigation(void)
> "always-on" : "conditional");
> }
>
> - /*
> - * If no STIBP, IBRS or enhanced IBRS is enabled, or SMT impossible,
> - * STIBP is not required.
> - */
> - if (!boot_cpu_has(X86_FEATURE_STIBP) ||
> - !smt_possible ||
> - spectre_v2_in_ibrs_mode(spectre_v2_enabled))
> + if (!boot_cpu_has(X86_FEATURE_STIBP) || !smt_possible ||
> + !spectre_v2_user_needs_stibp(spectre_v2_enabled))

As pointed out in other discussions, it will be great if can get rid of
eIBRS check, and do what the user asked for; or atleast print a warning
about not setting STIBP bit explicitly.

> return;
>
> /*
> @@ -2327,7 +2338,7 @@ static ssize_t mmio_stale_data_show_state(char *buf)
>
> static char *stibp_state(void)
> {
> - if (spectre_v2_in_ibrs_mode(spectre_v2_enabled))
> + if (!spectre_v2_user_needs_stibp(spectre_v2_enabled))

Decoupling STIBP and eIBRS will also get rid of this check.

> return "";

2023-02-22 05:51:02

[permalink] [raw]

Subject: Re: [PATCH v2 1/2] x86/speculation: Allow enabling STIBP with legacy IBRS

On Tue, Feb 21, 2023 at 7:07 PM Pawan Gupta
<[email protected]> wrote:
>
> On Tue, Feb 21, 2023 at 07:49:07PM +0100, KP Singh wrote:
> > Setting the IBRS bit implicitly enables STIBP to protect against
> > cross-thread branch target injection. With enhanced IBRS, the bit it set
> > once and is not cleared again. However, on CPUs with just legacy IBRS,
> > IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a
> > KERNEL_IBRS). Clearing this bit also disables the implicitly enabled
> > STIBP, thus requiring some form of cross-thread protection in userspace.
> >
> > Enable STIBP, either opt-in via prctl or seccomp, or always on depending
> > on the choice of mitigation selected via spectre_v2_user.
> >
> > Reported-by: José Oliveira <[email protected]>
> > Reported-by: Rodrigo Branco <[email protected]>
> > Reviewed-by: Alexandra Sandulescu <[email protected]>
> > Fixes: 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")
> > Cc: [email protected]
> > Signed-off-by: KP Singh <[email protected]>
> > ---
> > arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++-----------
> > 1 file changed, 22 insertions(+), 11 deletions(-)
> >
> > diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
> > index 85168740f76a..5be6075d8e36 100644
> > --- a/arch/x86/kernel/cpu/bugs.c
> > +++ b/arch/x86/kernel/cpu/bugs.c
> > @@ -1124,14 +1124,30 @@ spectre_v2_parse_user_cmdline(void)
> > return SPECTRE_V2_USER_CMD_AUTO;
> > }
> >
> > -static inline bool spectre_v2_in_ibrs_mode(enum spectre_v2_mitigation mode)
> > +static inline bool spectre_v2_in_eibrs_mode(enum spectre_v2_mitigation mode)
> > {
> > - return mode == SPECTRE_V2_IBRS ||
> > - mode == SPECTRE_V2_EIBRS ||
> > + return mode == SPECTRE_V2_EIBRS ||
> > mode == SPECTRE_V2_EIBRS_RETPOLINE ||
> > mode == SPECTRE_V2_EIBRS_LFENCE;
> > }
> >
> > +static inline bool spectre_v2_in_ibrs_mode(enum spectre_v2_mitigation mode)
> > +{
> > + return spectre_v2_in_eibrs_mode(mode) || mode == SPECTRE_V2_IBRS;
> > +}
> > +
> > +static inline bool spectre_v2_user_needs_stibp(enum spectre_v2_mitigation mode)
> > +{
> > + /*
> > + * enhanced IBRS also protects against user-mode attacks as the IBRS bit
>
> Maybe:
> * Enhanced IBRS mode also protects against cross-thread user-to-user
> * attacks as the IBRS bit

updated, thanks!

>
> > + * remains always set which implicitly enables cross-thread protections.
> > + * However, In legacy IBRS mode, the IBRS bit is set only in kernel
> > + * and cleared on return to userspace. This disables the implicit
> > + * cross-thread protections and STIBP is needed.
> > + */
> > + return !spectre_v2_in_eibrs_mode(mode);
> > +}
> > +
> > static void __init
> > spectre_v2_user_select_mitigation(void)
> > {
> > @@ -1193,13 +1209,8 @@ spectre_v2_user_select_mitigation(void)
> > "always-on" : "conditional");
> > }
> >
> > - /*
> > - * If no STIBP, IBRS or enhanced IBRS is enabled, or SMT impossible,
> > - * STIBP is not required.
> > - */
> > - if (!boot_cpu_has(X86_FEATURE_STIBP) ||
> > - !smt_possible ||
> > - spectre_v2_in_ibrs_mode(spectre_v2_enabled))
> > + if (!boot_cpu_has(X86_FEATURE_STIBP) || !smt_possible ||
> > + !spectre_v2_user_needs_stibp(spectre_v2_enabled))
>
> As pointed out in other discussions, it will be great if can get rid of
> eIBRS check, and do what the user asked for; or atleast print a warning

I think I will keep it as pr_info as, with eIBRS, the user does not
really need STIBP and the mitigation is still effective.

> about not setting STIBP bit explicitly.

That is a bit more complicated as, for now, the user is not really
exposed to STIBP explicitly yet.

{ "auto", SPECTRE_V2_USER_CMD_AUTO, false },
{ "off", SPECTRE_V2_USER_CMD_NONE, false },
{ "on", SPECTRE_V2_USER_CMD_FORCE, true },
{ "prctl", SPECTRE_V2_USER_CMD_PRCTL, false },
{ "prctl,ibpb", SPECTRE_V2_USER_CMD_PRCTL_IBPB, false },
{ "seccomp", SPECTRE_V2_USER_CMD_SECCOMP, false },
{ "seccomp,ibpb", SPECTRE_V2_USER_CMD_SECCOMP_IBPB, false },

I would prefer to do it as a follow up and fix this bug first.

It's a bit gnarly and I think we really need to think about the
options that are exposed to the user [especially in light of Intel /
AMD subtelties].

With the current patch the userspace is still getting working V2
mitigations on both dimensions time (Process A followed by Process B
where A does BTI on the subsequent B that are flushed via an IBPB) and
space (i.e. cross-thread branch target injection) whenever necessary.

>
> > return;
> >
> > /*
> > @@ -2327,7 +2338,7 @@ static ssize_t mmio_stale_data_show_state(char *buf)
> >
> > static char *stibp_state(void)
> > {
> > - if (spectre_v2_in_ibrs_mode(spectre_v2_enabled))
> > + if (!spectre_v2_user_needs_stibp(spectre_v2_enabled))
>
> Decoupling STIBP and eIBRS will also get rid of this check.
>
> > return "";

2023-02-22 08:25:59

by Pawan Gupta

[permalink] [raw]

Subject: Re: [PATCH v2 1/2] x86/speculation: Allow enabling STIBP with legacy IBRS

On Tue, Feb 21, 2023 at 09:49:57PM -0800, KP Singh wrote:
> > > @@ -1193,13 +1209,8 @@ spectre_v2_user_select_mitigation(void)
> > > "always-on" : "conditional");
> > > }
> > >
> > > - /*
> > > - * If no STIBP, IBRS or enhanced IBRS is enabled, or SMT impossible,
> > > - * STIBP is not required.
> > > - */
> > > - if (!boot_cpu_has(X86_FEATURE_STIBP) ||
> > > - !smt_possible ||
> > > - spectre_v2_in_ibrs_mode(spectre_v2_enabled))
> > > + if (!boot_cpu_has(X86_FEATURE_STIBP) || !smt_possible ||
> > > + !spectre_v2_user_needs_stibp(spectre_v2_enabled))
> >
> > As pointed out in other discussions, it will be great if can get rid of
> > eIBRS check, and do what the user asked for; or atleast print a warning
>
> I think I will keep it as pr_info as, with eIBRS, the user does not
> really need STIBP and the mitigation is still effective.

Thats fair.

> > about not setting STIBP bit explicitly.
>
> That is a bit more complicated as, for now, the user is not really
> exposed to STIBP explicitly yet.

> I would prefer to do it as a follow up and fix this bug first.

On a second thought, STIBP bit being explicitly set or not shouldn't
matter as long as user is getting the STIBP protection that it asked
for.

A print may just help catch some bugs sooner than later.

> It's a bit gnarly and I think we really need to think about the
> options that are exposed to the user [especially in light of Intel /
> AMD subtelties].

With AMD's AutoIBRS support landing in mainline, and both (AutoIBRS and
eIBRS) sharing the same =eibrs mitigation mode, those subtelties becomes
more important.

Following up on Andrew's comment in the other thread, I hope AutoIBRS
does not require setting STIBP explicitly?:

/sigh so we're still talking about 3 different things then.

1) Intel's legacy IBRS
2) AMD's regular IBRS
3) AMD's AutoIBRS

which all have different relevant behaviours for userspace. Just so
it's written out coherently in at least one place...
[...]
For any AMD configuration setting STIBP, there must be an IBPB after
having set STIBP. Setting STIBP alone does not evict previously
created shared predictions. This one can go subtly wrong for anyone
who assumes that Intel STIBP and AMD STIBP have the same behaviour.

2023-02-22 12:25:54

[permalink] [raw]

Subject: Re: [PATCH v2 1/2] x86/speculation: Allow enabling STIBP with legacy IBRS

On Tue, Feb 21, 2023 at 07:49:07PM +0100, KP Singh wrote:
> Setting the IBRS bit implicitly enables STIBP to protect against
> cross-thread branch target injection. With enhanced IBRS, the bit it set
> once and is not cleared again. However, on CPUs with just legacy IBRS,
> IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a
> KERNEL_IBRS). Clearing this bit also disables the implicitly enabled
> STIBP, thus requiring some form of cross-thread protection in userspace.
>
> Enable STIBP, either opt-in via prctl or seccomp, or always on depending
> on the choice of mitigation selected via spectre_v2_user.
>
> Reported-by: José Oliveira <[email protected]>
> Reported-by: Rodrigo Branco <[email protected]>
> Reviewed-by: Alexandra Sandulescu <[email protected]>
> Fixes: 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")
> Cc: [email protected]
> Signed-off-by: KP Singh <[email protected]>
> ---
> arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++-----------
> 1 file changed, 22 insertions(+), 11 deletions(-)

Below is what I'd like to see. Holler if something's wrong.

It is totally untested ofc.

---
From: KP Singh <[email protected]>
Date: Tue, 21 Feb 2023 19:49:07 +0100
Subject: [PATCH] x86/speculation: Allow enabling STIBP with legacy IBRS
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

When plain IBRS is enabled (not enhanced IBRS), the logic in
spectre_v2_user_select_mitigation() determines that STIBP is not needed.

However, on return to userspace, the IBRS bit is cleared for performance
reasons. That leaves userspace threads vulnerable to cross-thread
predictions influence against which STIBP protects.

Exclude IBRS from the spectre_v2_in_ibrs_mode() check to allow for
enabling STIBP through seccomp/prctl().

[ bp: Rewrite commit message and massage. ]

Fixes: 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")
Reported-by: José Oliveira <[email protected]>
Reported-by: Rodrigo Branco <[email protected]>
Signed-off-by: KP Singh <[email protected]>
Signed-off-by: Borislav Petkov (AMD) <[email protected]>
Cc: [email protected]
Link: https://lore.kernel.org/r/[email protected]
---
arch/x86/kernel/cpu/bugs.c | 25 ++++++++++++++++++-------
1 file changed, 18 insertions(+), 7 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index cf81848b72f4..9a969ab0e62a 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -1133,14 +1133,18 @@ spectre_v2_parse_user_cmdline(void)
return SPECTRE_V2_USER_CMD_AUTO;
}

-static inline bool spectre_v2_in_ibrs_mode(enum spectre_v2_mitigation mode)
+static inline bool spectre_v2_in_eibrs_mode(enum spectre_v2_mitigation mode)
{
- return mode == SPECTRE_V2_IBRS ||
- mode == SPECTRE_V2_EIBRS ||
+ return mode == SPECTRE_V2_EIBRS ||
mode == SPECTRE_V2_EIBRS_RETPOLINE ||
mode == SPECTRE_V2_EIBRS_LFENCE;
}

+static inline bool spectre_v2_in_ibrs_mode(enum spectre_v2_mitigation mode)
+{
+ return spectre_v2_in_eibrs_mode(mode) || mode == SPECTRE_V2_IBRS;
+}
+
static void __init
spectre_v2_user_select_mitigation(void)
{
@@ -1203,12 +1207,19 @@ spectre_v2_user_select_mitigation(void)
}

/*
- * If no STIBP, IBRS or enhanced IBRS is enabled, or SMT impossible,
- * STIBP is not required.
+ * If no STIBP, enhanced IBRS is enabled, or SMT impossible, STIBP
+ * is not required.
+ *
+ * Enhanced IBRS protects also against user-mode attacks as the IBRS bit
+ * remains always set which implicitly enables cross-thread protections.
+ * However, in legacy IBRS mode, the IBRS bit is set only on kernel
+ * entry and cleared on return to userspace. This disables the implicit
+ * cross-thread protections so allow for STIBP to be selected in that
+ * case.
*/
if (!boot_cpu_has(X86_FEATURE_STIBP) ||
!smt_possible ||
- spectre_v2_in_ibrs_mode(spectre_v2_enabled))
+ spectre_v2_in_eibrs_mode(spectre_v2_enabled))
return;

/*
@@ -2340,7 +2351,7 @@ static ssize_t mmio_stale_data_show_state(char *buf)

static char *stibp_state(void)
{
- if (spectre_v2_in_ibrs_mode(spectre_v2_enabled))
+ if (spectre_v2_in_eibrs_mode(spectre_v2_enabled))
return "";

switch (spectre_v2_user_stibp) {
--
2.35.1

--
Regards/Gruss,
Boris.

https://people.kernel.org/tglx/notes-about-netiquette

2023-02-22 12:32:46

[permalink] [raw]

Subject: Re: [PATCH v2 1/2] x86/speculation: Allow enabling STIBP with legacy IBRS

On Tue, Feb 21, 2023 at 09:49:57PM -0800, KP Singh wrote:
> That is a bit more complicated as, for now, the user is not really
> exposed to STIBP explicitly yet.

Remember that we're exposing the normal user to a gazillion switches
already. And not every user has done a PhD in hw vulns like we were
forced to in the last, at least 5, years.

So whatever you do, you should try to make it work automatic - if
possible - and DTRT - i.e., sane defaults.

Every new functionality added to that madness needs a proper
justification.

Thx.

--
Regards/Gruss,
Boris.

https://people.kernel.org/tglx/notes-about-netiquette

2023-02-22 13:56:51

by David Laight

[permalink] [raw]

Subject: RE: [PATCH v2 1/2] x86/speculation: Allow enabling STIBP with legacy IBRS

From: Borislav Petkov
> Sent: 22 February 2023 12:33
>
> On Tue, Feb 21, 2023 at 09:49:57PM -0800, KP Singh wrote:
> > That is a bit more complicated as, for now, the user is not really
> > exposed to STIBP explicitly yet.
>
> Remember that we're exposing the normal user to a gazillion switches
> already. And not every user has done a PhD in hw vulns like we were
> forced to in the last, at least 5, years.

It is also worth explaining the acronyms and what the mitigations
actually do.
There are also the big family where disabling hyperthreading
is probably a better way to avoid the vulnerability.

David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)

2023-02-22 17:16:40

[permalink] [raw]

Subject: Re: [PATCH v2 1/2] x86/speculation: Allow enabling STIBP with legacy IBRS

On Wed, Feb 22, 2023 at 4:24 AM Borislav Petkov <[email protected]> wrote:
>
> On Tue, Feb 21, 2023 at 07:49:07PM +0100, KP Singh wrote:
> > Setting the IBRS bit implicitly enables STIBP to protect against
> > cross-thread branch target injection. With enhanced IBRS, the bit it set
> > once and is not cleared again. However, on CPUs with just legacy IBRS,
> > IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a
> > KERNEL_IBRS). Clearing this bit also disables the implicitly enabled
> > STIBP, thus requiring some form of cross-thread protection in userspace.
> >
> > Enable STIBP, either opt-in via prctl or seccomp, or always on depending
> > on the choice of mitigation selected via spectre_v2_user.
> >
> > Reported-by: José Oliveira <[email protected]>
> > Reported-by: Rodrigo Branco <[email protected]>
> > Reviewed-by: Alexandra Sandulescu <[email protected]>
> > Fixes: 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")
> > Cc: [email protected]
> > Signed-off-by: KP Singh <[email protected]>
> > ---
> > arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++-----------
> > 1 file changed, 22 insertions(+), 11 deletions(-)
>
> Below is what I'd like to see. Holler if something's wrong.

Thanks for iterating. I think your commit description and rewrite
omits a few key subtleties which I have tried to reinforce in both the
commit log and the comments.

Q: What does STIBP have to do with IBRS?
A: Setting the IBRS bit implicitly enables STIBP / some form of cross
thread protection.

Q: Why does it work with eIBRS?
A: Because we set the IBRS bit once and leave it set when using eIBRS

I think this subtlety should be reinforced in the commit description
and code comments so that we don't get it wrong again. Your commit
does answer this one (thanks!)

Q: Why does it not work with the way the kernel currently implements
legacy IBRS?
A: Because the kernel clears the bit on returning to user space.

>
> It is totally untested ofc.
>
> ---
> From: KP Singh <[email protected]>
> Date: Tue, 21 Feb 2023 19:49:07 +0100
> Subject: [PATCH] x86/speculation: Allow enabling STIBP with legacy IBRS
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
>
> When plain IBRS is enabled (not enhanced IBRS), the logic in
> spectre_v2_user_select_mitigation() determines that STIBP is not needed.
>
> However, on return to userspace, the IBRS bit is cleared for performance
> reasons. That leaves userspace threads vulnerable to cross-thread
> predictions influence against which STIBP protects.
>
> Exclude IBRS from the spectre_v2_in_ibrs_mode() check to allow for
> enabling STIBP through seccomp/prctl().
>
> [ bp: Rewrite commit message and massage. ]
>
> Fixes: 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")
> Reported-by: José Oliveira <[email protected]>
> Reported-by: Rodrigo Branco <[email protected]>
> Signed-off-by: KP Singh <[email protected]>
> Signed-off-by: Borislav Petkov (AMD) <[email protected]>
> Cc: [email protected]
> Link: https://lore.kernel.org/r/[email protected]
> ---
> arch/x86/kernel/cpu/bugs.c | 25 ++++++++++++++++++-------
> 1 file changed, 18 insertions(+), 7 deletions(-)
>
> diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
> index cf81848b72f4..9a969ab0e62a 100644
> --- a/arch/x86/kernel/cpu/bugs.c
> +++ b/arch/x86/kernel/cpu/bugs.c
> @@ -1133,14 +1133,18 @@ spectre_v2_parse_user_cmdline(void)
> return SPECTRE_V2_USER_CMD_AUTO;
> }
>
> -static inline bool spectre_v2_in_ibrs_mode(enum spectre_v2_mitigation mode)
> +static inline bool spectre_v2_in_eibrs_mode(enum spectre_v2_mitigation mode)
> {
> - return mode == SPECTRE_V2_IBRS ||
> - mode == SPECTRE_V2_EIBRS ||
> + return mode == SPECTRE_V2_EIBRS ||
> mode == SPECTRE_V2_EIBRS_RETPOLINE ||
> mode == SPECTRE_V2_EIBRS_LFENCE;
> }
>
> +static inline bool spectre_v2_in_ibrs_mode(enum spectre_v2_mitigation mode)
> +{
> + return spectre_v2_in_eibrs_mode(mode) || mode == SPECTRE_V2_IBRS;
> +}
> +
> static void __init
> spectre_v2_user_select_mitigation(void)
> {
> @@ -1203,12 +1207,19 @@ spectre_v2_user_select_mitigation(void)
> }
>
> /*
> - * If no STIBP, IBRS or enhanced IBRS is enabled, or SMT impossible,
> - * STIBP is not required.
> + * If no STIBP, enhanced IBRS is enabled, or SMT impossible, STIBP
> + * is not required.
> + *
> + * Enhanced IBRS protects also against user-mode attacks as the IBRS bit
> + * remains always set which implicitly enables cross-thread protections.
> + * However, in legacy IBRS mode, the IBRS bit is set only on kernel
> + * entry and cleared on return to userspace. This disables the implicit
> + * cross-thread protections so allow for STIBP to be selected in that
> + * case.
> */
> if (!boot_cpu_has(X86_FEATURE_STIBP) ||
> !smt_possible ||
> - spectre_v2_in_ibrs_mode(spectre_v2_enabled))
> + spectre_v2_in_eibrs_mode(spectre_v2_enabled))
> return;
>
> /*
> @@ -2340,7 +2351,7 @@ static ssize_t mmio_stale_data_show_state(char *buf)
>
> static char *stibp_state(void)
> {
> - if (spectre_v2_in_ibrs_mode(spectre_v2_enabled))
> + if (spectre_v2_in_eibrs_mode(spectre_v2_enabled))

The reason why I refactored this into a separate helper was to
document the subtleties I mentioned above and anchor them to one place
as the function is used in 2 places. But this is a maintainer's
choice, so it's your call :)

I do agree with Pawan that it's worth adding a pr_info about what the
kernel is doing about STIBP.

- KP

> return "";
>
> switch (spectre_v2_user_stibp) {
> --
> 2.35.1
>
> --
> Regards/Gruss,
> Boris.
>
> https://people.kernel.org/tglx/notes-about-netiquette

2023-02-22 17:48:55

[permalink] [raw]

Subject: Re: [PATCH v2 1/2] x86/speculation: Allow enabling STIBP with legacy IBRS

On Wed, Feb 22, 2023 at 09:16:21AM -0800, KP Singh wrote:
> Thanks for iterating. I think your commit description and rewrite
> omits a few key subtleties which I have tried to reinforce in both the
> commit log and the comments.
>
> Q: What does STIBP have to do with IBRS?
> A: Setting the IBRS bit implicitly enables STIBP / some form of cross
> thread protection.

That belongs in the docs, if you want to explain this properly.

> Q: Why does it work with eIBRS?
> A: Because we set the IBRS bit once and leave it set when using eIBRS

Also docs.

> I think this subtlety should be reinforced in the commit description
> and code comments so that we don't get it wrong again. Your commit
> does answer this one (thanks!)

Commit messages are fine when explaining *why* a change is being done.
What is even finer is when you put a lenghtier explanation in our
documentation so that people can actually find it. Finding text in
commit messages is harder...

> Q: Why does it not work with the way the kernel currently implements
> legacy IBRS?
> A: Because the kernel clears the bit on returning to user space.

From the commit message:

However, on return to userspace, the IBRS bit is cleared for performance
reasons. That leaves userspace threads vulnerable to cross-thread
predictions influence against which STIBP protects.

> The reason why I refactored this into a separate helper was to
> document the subtleties I mentioned above and anchor them to one place
> as the function is used in 2 places. But this is a maintainer's
> choice, so it's your call :)

The less code gets added in that thing, the better. Not yet another
helper pls.

> I do agree with Pawan that it's worth adding a pr_info about what the
> kernel is doing about STIBP.

STIBP status gets dumped through stibp_state().

--
Regards/Gruss,
Boris.

https://people.kernel.org/tglx/notes-about-netiquette

2023-02-22 19:42:19

[permalink] [raw]

Subject: Re: [PATCH v2 1/2] x86/speculation: Allow enabling STIBP with legacy IBRS

On Wed, Feb 22, 2023 at 9:48 AM Borislav Petkov <[email protected]> wrote:
>
> On Wed, Feb 22, 2023 at 09:16:21AM -0800, KP Singh wrote:
> > Thanks for iterating. I think your commit description and rewrite
> > omits a few key subtleties which I have tried to reinforce in both the
> > commit log and the comments.
> >
> > Q: What does STIBP have to do with IBRS?
> > A: Setting the IBRS bit implicitly enables STIBP / some form of cross
> > thread protection.
>
> That belongs in the docs, if you want to explain this properly.
>
> > Q: Why does it work with eIBRS?
> > A: Because we set the IBRS bit once and leave it set when using eIBRS
>
> Also docs.
>
> > I think this subtlety should be reinforced in the commit description
> > and code comments so that we don't get it wrong again. Your commit
> > does answer this one (thanks!)
>
> Commit messages are fine when explaining *why* a change is being done.
> What is even finer is when you put a lenghtier explanation in our
> documentation so that people can actually find it. Finding text in
> commit messages is harder...

Sure, I think the docs do already cover it, but I sort of disagree
with your statement around the commit message. I feel the more context
you can add in the commit message, the better it is. When I am looking
at the change log, it would be helpful to have the information that I
mentioned in the Q&A. Small things like, "eIBRS needs the IBRS bit set
which also enables cross-thread protections" is a very important
context for this patch IMHO. Without this one is just left head
scratching and scrambling to read lengthy docs and processor manuals.

But again, totally your call. Others, feel free to chime in.

>
> > Q: Why does it not work with the way the kernel currently implements
> > legacy IBRS?
> > A: Because the kernel clears the bit on returning to user space.
>
> From the commit message:
>
> However, on return to userspace, the IBRS bit is cleared for performance
> reasons. That leaves userspace threads vulnerable to cross-thread
> predictions influence against which STIBP protects.

This sort of loosely implies that the IBRS bit also enables
cross-thread protections. Can you atleast add this one explicitly?

"Setting the IBRS bit also enables cross thread protections"

>
> > The reason why I refactored this into a separate helper was to
> > document the subtleties I mentioned above and anchor them to one place
> > as the function is used in 2 places. But this is a maintainer's
> > choice, so it's your call :)
>
> The less code gets added in that thing, the better. Not yet another
> helper pls.

Sure, your call.

>
> > I do agree with Pawan that it's worth adding a pr_info about what the
> > kernel is doing about STIBP.
>
> STIBP status gets dumped through stibp_state().

Not at the stage when the kernel decides to drop the STIBP protection
when eIBRS is enabled. If we had this information when we had a
positive POC, it would have been much easier to figure out what's
going on here.

- KP

>
> --
> Regards/Gruss,
> Boris.
>
> https://people.kernel.org/tglx/notes-about-netiquette

2023-02-23 12:45:10

[permalink] [raw]

Subject: Re: [PATCH v2 1/2] x86/speculation: Allow enabling STIBP with legacy IBRS

On Wed, Feb 22, 2023 at 11:41:59AM -0800, KP Singh wrote:
> Sure, I think the docs do already cover it,

I mean *our docs*. The stuff you're adding in your patch 2.

> but I sort of disagree with your statement around the commit message.
> I feel the more context you can add in the commit message, the better
> it is.

That's ofc wrong. And you'll find that out when you do git archeology
and you come across a huuuge wall of text explaining the world and some
more.

No, commit messages should be to the point with a structure similar to
something like this:

1. Prepare the context for the explanation briefly.

2. Explain the problem at hand.

3. "It happens because of <...>"

4. "Fix it by doing X"

5. "(Potentially do Y)."

concentrating on *why* the fix is being done.

> When I am looking at the change log, it would be helpful to have the
> information that I mentioned in the Q&A. Small things like, "eIBRS
> needs the IBRS bit set which also enables cross-thread protections" is
> a very important context for this patch IMHO. Without this one is just
> left head scratching and scrambling to read lengthy docs and processor
> manuals.

Yes, that's why you say in the commit message: "For more details, see
Documentation/admin-guide/hw-vuln/spectre.rst." where:

1. you can explain in a lot more detail

2. put it in place where people can find it *easily*

> This sort of loosely implies that the IBRS bit also enables
> cross-thread protections. Can you atleast add this one explicitly?
>
> "Setting the IBRS bit also enables cross thread protections"

Ok.

> Not at the stage when the kernel decides to drop the STIBP protection
> when eIBRS is enabled.

We can't dump every possible interaction between the mitigations. It is
a huge mess already. But I'm open to looking at improvements of the
situation *and* documenting stuff as we go.

Thx.

--
Regards/Gruss,
Boris.

https://people.kernel.org/tglx/notes-about-netiquette

2023-02-23 14:52:35

[permalink] [raw]

Subject: Re: [PATCH v2 2/2] Documentation/hw-vuln: Document the interaction between IBRS and STIBP

On Tue, Feb 21, 2023 at 07:49:08PM +0100, KP Singh wrote:
> ... Consequently, STIBP needs to be explicitly
> + enabled to guard against cross-thread attacks in userspace.

needs?

That sounds like something the user needs to do. But we do it by
default. Let's rephrase:

"Systems which support enhanced IBRS (eIBRS) enable IBRS protections once at
boot and they're automatically protected against Spectre v2 variant
attacks, including cross-thread branch target injections on SMT systems
(STIBP). IOW, eIBRS enables STIBP too.

Legacy IBRS systems clear the IBRS bit on exit to userspace and
therefore explicitly enable STIBP for that."

Simple.

--
Regards/Gruss,
Boris.

https://people.kernel.org/tglx/notes-about-netiquette

2023-02-24 03:30:43

[permalink] [raw]

Subject: Re: [PATCH v2 2/2] Documentation/hw-vuln: Document the interaction between IBRS and STIBP

2023-02-26 01:42:58

[permalink] [raw]

Subject: Re: [PATCH v2 2/2] Documentation/hw-vuln: Document the interaction between IBRS and STIBP

On Thu, Feb 23, 2023 at 9:52 AM Borislav Petkov <[email protected]> wrote:
>
> On Tue, Feb 21, 2023 at 07:49:08PM +0100, KP Singh wrote:
> > ... Consequently, STIBP needs to be explicitly
> > + enabled to guard against cross-thread attacks in userspace.
>
> needs?
>
> That sounds like something the user needs to do. But we do it by
> default. Let's rephrase:
>
> "Systems which support enhanced IBRS (eIBRS) enable IBRS protections once at
> boot and they're automatically protected against Spectre v2 variant
> attacks, including cross-thread branch target injections on SMT systems
> (STIBP). IOW, eIBRS enables STIBP too.
>
> Legacy IBRS systems clear the IBRS bit on exit to userspace and
> therefore explicitly enable STIBP for that."

+ Systems which support enhanced IBRS (eIBRS) enable IBRS protection once at
+ boot, by setting the IBRS bit, and they're automatically protected against
+ Spectre v2 variant attacks, including cross-thread branch target injections
+ on SMT systems (STIBP). In other words, eIBRS enables STIBP too.
+
+ Legacy IBRS systems clear the IBRS bit on exit to userspace and
+ therefore explicitly enable STIBP for that

I did add one phrase, we really need to stress on the IBRS bit here.
Had we been enabling KERNEL_IBRS accidentally with eIBRS, it would
still mess things up as the bit being set is important.

This is why my original patch felt "obtuse" as it focused on
KERNEL_IBRS instead of IBRS or eIBRS :).

>
> Simple.
>
> --
> Regards/Gruss,
> Boris.
>
> https://people.kernel.org/tglx/notes-about-netiquette

2023-02-26 01:50:44