2021-01-25 18:59:56

by Greg Kroah-Hartman

Subject: [PATCH 5.4 55/86] x86/mmx: Use KFPU_387 for MMX string operations

From: Andy Lutomirski <[email protected]>

commit 67de8dca50c027ca0fa3b62a488ee5035036a0da upstream.

The default kernel_fpu_begin() doesn't work on systems that support XMM but
haven't yet enabled CR4.OSFXSR. This causes crashes when _mmx_memcpy() is
called too early because LDMXCSR generates #UD when the aforementioned bit
is clear.

Fix it by using kernel_fpu_begin_mask(KFPU_387) explicitly.

Fixes: 7ad816762f9b ("x86/fpu: Reset MXCSR to default in kernel_fpu_begin()")
Reported-by: Krzysztof Mazur <[email protected]>
Signed-off-by: Andy Lutomirski <[email protected]>
Signed-off-by: Borislav Petkov <[email protected]>
Tested-by: Krzysztof Piotr Olędzki <[email protected]>
Tested-by: Krzysztof Mazur <[email protected]>
Cc: <[email protected]>
Link: https://lkml.kernel.org/r/e7bf21855fe99e5f3baa27446e32623358f69e8d.1611205691.git.luto@kernel.org
Signed-off-by: Greg Kroah-Hartman <[email protected]>

---
arch/x86/lib/mmx_32.c | 20 +++++++++++++++-----
1 file changed, 15 insertions(+), 5 deletions(-)

--- a/arch/x86/lib/mmx_32.c
+++ b/arch/x86/lib/mmx_32.c
@@ -26,6 +26,16 @@
#include <asm/fpu/api.h>
#include <asm/asm.h>

+/*
+ * Use KFPU_387. MMX instructions are not affected by MXCSR,
+ * but both AMD and Intel documentation states that even integer MMX
+ * operations will result in #MF if an exception is pending in FCW.
+ *
+ * EMMS is not needed afterwards because, after calling kernel_fpu_end(),
+ * any subsequent user of the 387 stack will reinitialize it using
+ * KFPU_387.
+ */
+
void *_mmx_memcpy(void *to, const void *from, size_t len)
{
void *p;
@@ -37,7 +47,7 @@ void *_mmx_memcpy(void *to, const void *
p = to;
i = len >> 6; /* len/64 */

- kernel_fpu_begin();
+ kernel_fpu_begin_mask(KFPU_387);

__asm__ __volatile__ (
"1: prefetch (%0)\n" /* This set is 28 bytes */
@@ -127,7 +137,7 @@ static void fast_clear_page(void *page)
{
int i;

- kernel_fpu_begin();
+ kernel_fpu_begin_mask(KFPU_387);

__asm__ __volatile__ (
" pxor %%mm0, %%mm0\n" : :
@@ -160,7 +170,7 @@ static void fast_copy_page(void *to, voi
{
int i;

- kernel_fpu_begin();
+ kernel_fpu_begin_mask(KFPU_387);

/*
* maybe the prefetch stuff can go before the expensive fnsave...
@@ -247,7 +257,7 @@ static void fast_clear_page(void *page)
{
int i;

- kernel_fpu_begin();
+ kernel_fpu_begin_mask(KFPU_387);

__asm__ __volatile__ (
" pxor %%mm0, %%mm0\n" : :
@@ -282,7 +292,7 @@ static void fast_copy_page(void *to, voi
{
int i;

- kernel_fpu_begin();
+ kernel_fpu_begin_mask(KFPU_387);

__asm__ __volatile__ (
"1: prefetch (%0)\n"

2021-01-26 19:54:58

by Krzysztof Olędzki

Subject: Re: [PATCH 5.4 55/86] x86/mmx: Use KFPU_387 for MMX string operations

On 2021-01-25 at 10:39, Greg Kroah-Hartman wrote:
> From: Andy Lutomirski <[email protected]>
>
> commit 67de8dca50c027ca0fa3b62a488ee5035036a0da upstream.
>
> The default kernel_fpu_begin() doesn't work on systems that support XMM but
> haven't yet enabled CR4.OSFXSR. This causes crashes when _mmx_memcpy() is
> called too early because LDMXCSR generates #UD when the aforementioned bit
> is clear.
>
> Fix it by using kernel_fpu_begin_mask(KFPU_387) explicitly.
>
> Fixes: 7ad816762f9b ("x86/fpu: Reset MXCSR to default in kernel_fpu_begin()")
> Reported-by: Krzysztof Mazur <[email protected]>
> Signed-off-by: Andy Lutomirski <[email protected]>
> Signed-off-by: Borislav Petkov <[email protected]>
> Tested-by: Krzysztof Piotr Olędzki <[email protected]>
> Tested-by: Krzysztof Mazur <[email protected]>
> Cc: <[email protected]>
> Link: https://lkml.kernel.org/r/e7bf21855fe99e5f3baa27446e32623358f69e8d.1611205691.git.luto@kernel.org
> Signed-off-by: Greg Kroah-Hartman <[email protected]>

Similar to 5.10.11, we also need
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e45122893a9870813f9bd7b4add4f613e6f29008
in 5.4.93:

Otherwise, the kernel will fail to compile if CONFIG_X86_USE_3DNOW=y

arch/x86/lib/mmx_32.c: In function '_mmx_memcpy':
arch/x86/lib/mmx_32.c:50:2: error: implicit declaration of function
'kernel_fpu_begin_mask'; did you mean 'kernel_fpu_begin'?
[-Werror=implicit-function-declaration]
50 | kernel_fpu_begin_mask(KFPU_387);
| ^~~~~~~~~~~~~~~~~~~~~
| kernel_fpu_begin
arch/x86/lib/mmx_32.c:50:24: error: 'KFPU_387' undeclared (first use in
this function)
50 | kernel_fpu_begin_mask(KFPU_387);
| ^~~~~~~~
arch/x86/lib/mmx_32.c:50:24: note: each undeclared identifier is
reported only once for each function it appears in
arch/x86/lib/mmx_32.c: In function 'fast_clear_page':
arch/x86/lib/mmx_32.c:140:24: error: 'KFPU_387' undeclared (first use in
this function)
140 | kernel_fpu_begin_mask(KFPU_387);
| ^~~~~~~~
arch/x86/lib/mmx_32.c: In function 'fast_copy_page':
arch/x86/lib/mmx_32.c:173:24: error: 'KFPU_387' undeclared (first use in
this function)
173 | kernel_fpu_begin_mask(KFPU_387);
| ^~~~~~~~

Thanks,
Krzysztof