From: Ramakrishna Saripalli <[email protected]>
Predictive Store Forwarding:
AMD Zen3 processors feature a new technology called
Predictive Store Forwarding (PSF).
<TODO:Insert link to AMD PSF whitepaper>
PSF is a hardware-based micro-architectural optimization designed
to improve the performance of code execution by predicting address
dependencies between loads and stores.
How PSF works:
It is very common for a CPU to execute a load instruction to an address
that was recently written by a store. Modern CPUs implement a technique
known as Store-To-Load-Forwarding (STLF) to improve performance in such
cases. With STLF, data from the store is forwarded directly to the load
without having to wait for it to be written to memory. In a typical CPU,
STLF occurs after the address of both the load and store are calculated
and determined to match.
PSF expands on this by speculating on the relationship between loads and
stores without waiting for the address calculation to complete. With PSF,
the CPU learns over time the relationship between loads and stores.
If STLF typically occurs between a particular store and load, the CPU will
remember this.
In typical code, PSF provides a performance benefit by speculating on
the load result and allowing later instructions to begin execution
sooner than they otherwise would be able to.
Causes of Incorrect PSF:
Incorrect PSF predictions can occur due to two reasons.
First, it is possible that the store/load pair had a dependency for a
while but later stops having a dependency. This can occur if the address
of either the store or load changes during the execution of the program.
The second source of incorrect PSF predictions can occur if there is an
alias in the PSF predictor structure. The PSF predictor tracks
store-load pairs based on portions of their RIP. It is possible that a
store-load pair which does have a dependency may alias in the predictor
with another store-load pair which does not.
This can result in incorrect speculation when the second store/load pair
is executed.
Security Analysis:
Previous research has shown that when CPUs speculate on non-architectural
paths it can lead to the potential of side channel attacks.
In particular, programs that implement isolation, also known as
‘sandboxing’, entirely in software may need to be concerned with incorrect
CPU speculation as they can occur due to bad PSF predictions.
Because PSF speculation is limited to the current program context,
the impact of bad PSF speculation is very similar to that of
Speculative Store Bypass (Spectre v4)
Predictive Store Forwarding controls:
There are two hardware control bits which influence the PSF feature:
- MSR 48h bit 2 – Speculative Store Bypass (SSBD)
- MSR 48h bit 7 – Predictive Store Forwarding Disable (PSFD)
The PSF feature is disabled if either of these bits are set. These bits
are controllable on a per-thread basis in an SMT system. By default, both
SSBD and PSFD are 0 meaning that the speculation features are enabled.
While the SSBD bit disables PSF and speculative store bypass, PSFD only
disables PSF.
PSFD may be desirable for software which is concerned with the
speculative behavior of PSF but desires a smaller performance impact than
setting SSBD.
Support for PSFD is indicated in CPUID Fn8000_0008 EBX[28].
All processors that support PSF will also support PSFD.
Ramakrishna Saripalli (5):
x86/cpufeatures: Define feature bits to support mitigation of PSF
x86/speculation: Implement support for PSFD detection and reporting
x86/speculation: Introduce SPEC_CTRL_MSR bit for PSFD
x86/speculation: Implement PSF mitigation support
x86/speculation: Add PSF mitigation kernel parameters
.../admin-guide/kernel-parameters.txt | 45 +++++
arch/x86/include/asm/cpufeatures.h | 4 +-
arch/x86/include/asm/msr-index.h | 2 +
arch/x86/include/asm/nospec-branch.h | 8 +
arch/x86/include/asm/spec-ctrl.h | 12 ++
arch/x86/include/asm/thread_info.h | 2 +
arch/x86/kernel/cpu/bugs.c | 191 ++++++++++++++++++
arch/x86/kernel/cpu/common.c | 6 +
arch/x86/kernel/process.c | 23 +++
include/linux/sched.h | 15 ++
include/uapi/linux/prctl.h | 2 +
11 files changed, 309 insertions(+), 1 deletion(-)
base-commit: 0e16f466004d7f04296b9676a712a32a12367d1f
--
2.25.1
From: Ramakrishna Saripalli <[email protected]>
All AMD processors that support Predictive Store Forwarding (PSF)
provide a CPUID bit to detect support for mitigation of this
feature. This bit is referred to as PSFD (Predictive Store Forwarding
Disable)
If CPU advertises PSFD:
- Advertise a generic feature (X86_FEATURE_PSFD) in Linux capability
flags to indicate PSF mitigation is available.
- Advertise X86_BUG_PSF in Linux bug bits to indicate that CPU is
affected by PSF security risks.
Signed-off-by: Ramakrishna Saripalli<[email protected]>
---
arch/x86/kernel/cpu/common.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index ab640abe26b6..4e604d58f3b3 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -893,6 +893,9 @@ static void init_speculation_control(struct cpuinfo_x86 *c)
set_cpu_cap(c, X86_FEATURE_MSR_SPEC_CTRL);
clear_cpu_cap(c, X86_FEATURE_VIRT_SSBD);
}
+
+ if (cpu_has(c, X86_FEATURE_AMD_PSFD))
+ set_cpu_cap(c, X86_FEATURE_PSFD);
}
void get_cpu_cap(struct cpuinfo_x86 *c)
@@ -1154,6 +1157,9 @@ static void __init cpu_set_bug_bits(struct cpuinfo_x86 *c)
!cpu_has(c, X86_FEATURE_AMD_SSB_NO))
setup_force_cpu_bug(X86_BUG_SPEC_STORE_BYPASS);
+ if (cpu_has(c, X86_FEATURE_AMD_PSFD))
+ setup_force_cpu_bug(X86_BUG_PSF);
+
if (ia32_cap & ARCH_CAP_IBRS_ALL)
setup_force_cpu_cap(X86_FEATURE_IBRS_ENHANCED);
--
2.25.1
From: Ramakrishna Saripalli <[email protected]>
PSF mitigation introduces new kernel parameters.
The kernel parameters for PSF mitigation are modeled
after spec_store_bypass_disable.
Signed-off-by: Ramakrishna Saripalli<[email protected]>
---
.../admin-guide/kernel-parameters.txt | 45 +++++++++++++++++++
1 file changed, 45 insertions(+)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 04545725f187..68dfde77a87d 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2876,6 +2876,7 @@
nospectre_v2 [X86,PPC,S390,ARM64]
spectre_v2_user=off [X86]
spec_store_bypass_disable=off [X86,PPC]
+ psfd=off [X86]
ssbd=force-off [ARM64]
l1tf=off [X86]
mds=off [X86]
@@ -3243,6 +3244,8 @@
nohugeiomap [KNL,X86,PPC,ARM64] Disable kernel huge I/O mappings.
+ nopsfd [HW,X86] Disable mitigation for Predictive Store Forwarding.
+
nosmt [KNL,S390] Disable symmetric multithreading (SMT).
Equivalent to smt=1.
@@ -4002,6 +4005,48 @@
that).
Format: <bool>
+ psfd= [HW,X86]
+ Predictive Store Forwarding Disable control
+
+ Certain AMD processors feature a new technology called Predictive
+ Store Forwarding. This feature is designed to improve the
+ performance of code execution by predicting dependencies
+ between loads and stores.
+
+ Modern processors implement techniques to optimize the
+ execution of a load instruction to an address that was
+ recently written by a store instruction.
+
+ PSF expands on the above by speculating on the relationship
+ between loads and stores without waiting for address
+ calculation to complete. With PSF, CPU learns over time the
+ relationship between loads and stores.
+
+ Incorrect PSF predictions can occur for various reasons.
+ Please see the AMD PSF whitepaper for more information.
+
+ All AMD processors that implement PSF also provide ability
+ to control mitigation of PSF.
+
+ Following options are provided to control PSF mitigation.
+
+ The options are:
+ on - Unconditionally disable Speculative Store Bypass
+ off - Unconditionally enable Speculative Store Bypass
+ auto - Kernel detects whether the CPU is vulnerable.
+ If the CPU is not vulnerable, off is selected.
+ If the CPU is vulnerable, default mitigation is
+ KConfig dependent.
+ prctl - Control Predictive Store Forwarding per thread
+ via prctl. Predictive Store Forwarding is enabled
+ per process by default. The state of the control
+ is inherited on fork.
+ seccomp - Same as prctl above but all seccomp threads will
+ disable PSF unless they opt out.
+
+ Default mitigations:
+ [X86] If CONFIG_SECCOMP=y "seccomp" else "prctl"
+
psi= [KNL] Enable or disable pressure stall information
tracking.
Format: <bool>
--
2.25.1
From: Ramakrishna Saripalli <[email protected]>
All AMD processors that support PSF implement a bit in
SPEC_CTRL MSR (0x48) to disable or enable Predictive Store
Forwarding.
Signed-off-by: Ramakrishna Saripalli<[email protected]>
---
arch/x86/include/asm/msr-index.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index 546d6ecf0a35..f569918c8754 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -51,6 +51,8 @@
#define SPEC_CTRL_STIBP BIT(SPEC_CTRL_STIBP_SHIFT) /* STIBP mask */
#define SPEC_CTRL_SSBD_SHIFT 2 /* Speculative Store Bypass Disable bit */
#define SPEC_CTRL_SSBD BIT(SPEC_CTRL_SSBD_SHIFT) /* Speculative Store Bypass Disable */
+#define SPEC_CTRL_PSFD_SHIFT 7
+#define SPEC_CTRL_PSFD BIT(SPEC_CTRL_PSFD_SHIFT) /* Predictive Store Forwarding Disable */
#define MSR_IA32_PRED_CMD 0x00000049 /* Prediction Command */
#define PRED_CMD_IBPB BIT(0) /* Indirect Branch Prediction Barrier */
--
2.25.1
On Tue, Apr 06, 2021 at 10:49:59AM -0500, Ramakrishna Saripalli wrote:
> Because PSF speculation is limited to the current program context,
> the impact of bad PSF speculation is very similar to that of
> Speculative Store Bypass (Spectre v4)
>
> Predictive Store Forwarding controls:
> There are two hardware control bits which influence the PSF feature:
> - MSR 48h bit 2 – Speculative Store Bypass (SSBD)
> - MSR 48h bit 7 – Predictive Store Forwarding Disable (PSFD)
>
> The PSF feature is disabled if either of these bits are set. These bits
> are controllable on a per-thread basis in an SMT system. By default, both
> SSBD and PSFD are 0 meaning that the speculation features are enabled.
>
> While the SSBD bit disables PSF and speculative store bypass, PSFD only
> disables PSF.
>
> PSFD may be desirable for software which is concerned with the
> speculative behavior of PSF but desires a smaller performance impact than
> setting SSBD.
Hi Ramakrishna,
Is there a realistic scenario where an application would want to disable
PSF, but not disable SSB?
Maybe I'm missing something, but I'd presume an application would either
care about this class of attacks, or not.
--
Josh
Josh, thank you for taking the time to review the patches.
On 4/7/2021 5:39 PM, Josh Poimboeuf wrote:
> On Tue, Apr 06, 2021 at 10:49:59AM -0500, Ramakrishna Saripalli wrote:
>> Because PSF speculation is limited to the current program context,
>> the impact of bad PSF speculation is very similar to that of
>> Speculative Store Bypass (Spectre v4)
>>
>> Predictive Store Forwarding controls:
>> There are two hardware control bits which influence the PSF feature:
>> - MSR 48h bit 2 – Speculative Store Bypass (SSBD)
>> - MSR 48h bit 7 – Predictive Store Forwarding Disable (PSFD)
>>
>> The PSF feature is disabled if either of these bits are set. These bits
>> are controllable on a per-thread basis in an SMT system. By default, both
>> SSBD and PSFD are 0 meaning that the speculation features are enabled.
>>
>> While the SSBD bit disables PSF and speculative store bypass, PSFD only
>> disables PSF.
>>
>> PSFD may be desirable for software which is concerned with the
>> speculative behavior of PSF but desires a smaller performance impact than
>> setting SSBD.
>
> Hi Ramakrishna,
>
> Is there a realistic scenario where an application would want to disable
> PSF, but not disable SSB?
It is possible most applications have been reviewed and scrubbed for SSB-type attacks but PSF-type issues may not have been looked at yet.
This may be one of the cases where SSB is enabled but PSF is disabled until the application(s) are scrubbed for the same.
In certain cases, disabling PSF may have less impact performance-wise than disabling SSB.
>
> Maybe I'm missing something, but I'd presume an application would either
> care about this class of attacks, or not.
>
On Thu, Apr 08, 2021 at 09:56:47AM -0500, Saripalli, RK wrote:
> It is possible most applications have been reviewed and scrubbed for
> SSB-type attacks but PSF-type issues may not have been looked at yet.
> This may be one of the cases where SSB is enabled but PSF is disabled
> until the application(s) are scrubbed for the same.
Right, and for that I think we could do a slimmer version of the psfd=
toggle - no prctl and seccomp stuff - just the cmdline disable thing to
keep this simpler.
Btw "psfd=" is maybe too short and cryptic. It would probably be more
user-friendly if it were called:
predict_store_fwd={on,off,...}
or so.
Thx.
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
On Thu, Apr 08, 2021 at 09:56:47AM -0500, Saripalli, RK wrote:
> Josh, thank you for taking the time to review the patches.
>
> On 4/7/2021 5:39 PM, Josh Poimboeuf wrote:
> > On Tue, Apr 06, 2021 at 10:49:59AM -0500, Ramakrishna Saripalli wrote:
> >> Because PSF speculation is limited to the current program context,
> >> the impact of bad PSF speculation is very similar to that of
> >> Speculative Store Bypass (Spectre v4)
> >>
> >> Predictive Store Forwarding controls:
> >> There are two hardware control bits which influence the PSF feature:
> >> - MSR 48h bit 2 – Speculative Store Bypass (SSBD)
> >> - MSR 48h bit 7 – Predictive Store Forwarding Disable (PSFD)
> >>
> >> The PSF feature is disabled if either of these bits are set. These bits
> >> are controllable on a per-thread basis in an SMT system. By default, both
> >> SSBD and PSFD are 0 meaning that the speculation features are enabled.
> >>
> >> While the SSBD bit disables PSF and speculative store bypass, PSFD only
> >> disables PSF.
> >>
> >> PSFD may be desirable for software which is concerned with the
> >> speculative behavior of PSF but desires a smaller performance impact than
> >> setting SSBD.
> >
> > Hi Ramakrishna,
> >
> > Is there a realistic scenario where an application would want to disable
> > PSF, but not disable SSB?
>
> It is possible most applications have been reviewed and scrubbed for
> SSB-type attacks but PSF-type issues may not have been looked at yet.
It's "possible", but is it realistic? As far as I know, SSB is
impractical to scrub an application for.
Do we know of any real-world cases where this option is needed?
--
Josh
Josh, PSF being new, may be in the future someone will find something new.
This is really extra precaution.
On 4/9/2021 11:45 AM, Josh Poimboeuf wrote:
> On Thu, Apr 08, 2021 at 09:56:47AM -0500, Saripalli, RK wrote:
>> Josh, thank you for taking the time to review the patches.
>>
>> On 4/7/2021 5:39 PM, Josh Poimboeuf wrote:
>>> On Tue, Apr 06, 2021 at 10:49:59AM -0500, Ramakrishna Saripalli wrote:
>>>> Because PSF speculation is limited to the current program context,
>>>> the impact of bad PSF speculation is very similar to that of
>>>> Speculative Store Bypass (Spectre v4)
>>>>
>>>> Predictive Store Forwarding controls:
>>>> There are two hardware control bits which influence the PSF feature:
>>>> - MSR 48h bit 2 – Speculative Store Bypass (SSBD)
>>>> - MSR 48h bit 7 – Predictive Store Forwarding Disable (PSFD)
>>>>
>>>> The PSF feature is disabled if either of these bits are set. These bits
>>>> are controllable on a per-thread basis in an SMT system. By default, both
>>>> SSBD and PSFD are 0 meaning that the speculation features are enabled.
>>>>
>>>> While the SSBD bit disables PSF and speculative store bypass, PSFD only
>>>> disables PSF.
>>>>
>>>> PSFD may be desirable for software which is concerned with the
>>>> speculative behavior of PSF but desires a smaller performance impact than
>>>> setting SSBD.
>>>
>>> Hi Ramakrishna,
>>>
>>> Is there a realistic scenario where an application would want to disable
>>> PSF, but not disable SSB?
>>
>> It is possible most applications have been reviewed and scrubbed for
>> SSB-type attacks but PSF-type issues may not have been looked at yet.
>
> It's "possible", but is it realistic? As far as I know, SSB is
> impractical to scrub an application for.
>
> Do we know of any real-world cases where this option is needed?
>