2014-01-08 07:48:40

by halfdog

[permalink] [raw]
Subject: Re: Sanitize CPU-state when switching tasks (was sanitize CPU-state when switching from virtual-8086 mode to other task)

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Update to the issue:

* Although first observed with virtual-8086 mode, the bug is not
specific to virtual-8086 mode, it can be triggered with normal x86
userspace code also, even with better reproducibility.

* Ben Hutchings looked at the Debian bug report [1], he failed to
reproduce on his hardware, so it might be specific to some CPU models
(currently my AMD E-350 is only machine known to be affected).

* When deactivating mmap_min_addr, the NULL-dereferences during
task-switch is exploitable, works both on native hardware and within
VirtualBox. See [2] for POC to gain root privileges.

* It seems, that when changing the FPU control word with "fstcw" just
before exit of the process, then another process could suffer when
doing __do_switch, probably related to the xsave instruction and a x86
processor bug workaround, see "noxsave" switch [3]: [BUGS=X86]
Disables x86 extended register state save and restore using xsave. The
kernel will fallback to enabling legacy floating-point and sse state.

hd

[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=733551
[2]
http://www.halfdog.net/Security/2013/Vm86SyscallTaskSwitchKernelPanic/
[3] https://www.kernel.org/doc/Documentation/kernel-parameters.txt

- --
http://www.halfdog.net/
PGP: 156A AE98 B91F 0114 FE88 2BD8 C459 9386 feed a bee
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlLNAjEACgkQxFmThv7tq+44FACfeDHQHK71+7tZawm9Ftjw7Hvp
j04AmwY04UwG9clERS3e1HisM2swbo1i
=KoQL
-----END PGP SIGNATURE-----


2014-01-08 17:43:28

by H. Peter Anvin

[permalink] [raw]
Subject: Re: Sanitize CPU-state when switching tasks (was sanitize CPU-state when switching from virtual-8086 mode to other task)

Adding Borislav.

Boris, do you happen to know of any erratum on AMD E-350 which may be
in play here?

-hpa


On 01/07/2014 11:45 PM, halfdog wrote:
> Update to the issue:
>
> * Although first observed with virtual-8086 mode, the bug is not
> specific to virtual-8086 mode, it can be triggered with normal x86
> userspace code also, even with better reproducibility.
>
> * Ben Hutchings looked at the Debian bug report [1], he failed to
> reproduce on his hardware, so it might be specific to some CPU
> models (currently my AMD E-350 is only machine known to be
> affected).
>
> * When deactivating mmap_min_addr, the NULL-dereferences during
> task-switch is exploitable, works both on native hardware and
> within VirtualBox. See [2] for POC to gain root privileges.
>
> * It seems, that when changing the FPU control word with "fstcw"
> just before exit of the process, then another process could suffer
> when doing __do_switch, probably related to the xsave instruction
> and a x86 processor bug workaround, see "noxsave" switch [3]:
> [BUGS=X86] Disables x86 extended register state save and restore
> using xsave. The kernel will fallback to enabling legacy
> floating-point and sse state.
>
> hd
>
> [1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=733551 [2]
> http://www.halfdog.net/Security/2013/Vm86SyscallTaskSwitchKernelPanic/
>
>
[3] https://www.kernel.org/doc/Documentation/kernel-parameters.txt
>
>

2014-01-08 19:37:05

by Borislav Petkov

[permalink] [raw]
Subject: Re: Sanitize CPU-state when switching tasks (was sanitize CPU-state when switching from virtual-8086 mode to other task)

On Wed, Jan 08, 2014 at 09:42:40AM -0800, H. Peter Anvin wrote:
> Adding Borislav.
>
> Boris, do you happen to know of any erratum on AMD E-350 which may be
> in play here?

Interesting. Well, nothing looks even remotely related from looking at the F14h
rev guide here:

http://developer.amd.com/wordpress/media/2012/10/47534_14h_Mod_00h-0Fh_Rev_Guide.pdf

Btw, hd (if that is your real name :-)), can you post /proc/cpuinfo? I
think I might have a E-350 here too and I could try to reproduce. Btw,
how exactly do you trigger?

You run FpuStateTaskSwitchShmemXattrHandlersOverwriteWithNullPage.c
first to modify shmem_xattr_handlers and then
ManipulatedXattrHandlerForPrivEscalation.c? You need a 32-bit kernel and
userspace, right? Anything else?

Thanks.

--
Regards/Gruss,
Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

2014-01-08 21:31:28

by halfdog

[permalink] [raw]
Subject: Re: Sanitize CPU-state when switching tasks (was sanitize CPU-state when switching from virtual-8086 mode to other task)

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Borislav Petkov wrote:
> On Wed, Jan 08, 2014 at 09:42:40AM -0800, H. Peter Anvin wrote:
>> Adding Borislav.
>>
>> Boris, do you happen to know of any erratum on AMD E-350 which
>> may be in play here?
>
> Interesting. Well, nothing looks even remotely related from looking
> at the F14h rev guide here:
>
> http://developer.amd.com/wordpress/media/2012/10/47534_14h_Mod_00h-0Fh_Rev_Guide.pdf
>
> Btw, hd (if that is your real name :-)), can you post
> /proc/cpuinfo?

Of course (you can also find it in the Debian bug report [1]):

processor : 0
vendor_id : AuthenticAMD
cpu family : 20
model : 1
model name : AMD E-350 Processor
stepping : 0
microcode : 0x5000028
cpu MHz : 1596.563
cache size : 512 KB
fdiv_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 6
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt
pdpe1gb rdtscp lm constant_tsc nonstop_tsc extd_apicid aperfmperf pni
monitor ssse3 cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy
abm sse4a misalignsse 3dnowprefetch ibs skinit wdt arat hw_pstate npt
lbrv svm_lock nrip_save pausefilter
bogomips : 3193.12
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate

> I think I might have a E-350 here too and I could try to reproduce.
> Btw, how exactly do you trigger?
>
> You run
> FpuStateTaskSwitchShmemXattrHandlersOverwriteWithNullPage.c first
> to modify shmem_xattr_handlers and then
> ManipulatedXattrHandlerForPrivEscalation.c? You need a 32-bit
> kernel and userspace, right? Anything else?

Yes: I used the standard Debian Sid 468 kernel (32bit), the first tool
might just trigger the OOPS to early, this seems to be harmless to the
kernel, so one can invoke it until the handler pointer was modified.
Since I hardcoded the Debian kernel addresses (copied from
System.map), this is very unlikly to give you root on another kernel,
but the math OOPS should be reproducible.


Does this sound fishy (from [2])?

"There is no need to save any active fpu state to the task structure
memory if the task is dead. Just drop the state instead."

My rogue process might interfere with that: change control registers,
cause exception and then exit quickly


Or could it be invalid CPU-features detection, perhaps related to [3]?

The math-restore/__do_switch combination occurred already in older bug
reports, e.g. [4] (very close), [5] (similar, poor info). )))OOPS "EIP
is at math_state_restore"((( seems to be suitable search expression.


[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=733551
[2] http://lkml.indiana.edu/hypermail/linux/kernel/1205.1/02182.html
[3] http://lkml.indiana.edu/hypermail/linux/kernel/0905.2/02599.html
[4] https://lkml.org/lkml/2008/6/16/146
[5] http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1536

- --
http://www.halfdog.net/
PGP: 156A AE98 B91F 0114 FE88 2BD8 C459 9386 feed a bee
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlLNww0ACgkQxFmThv7tq+4LngCeI/ZVFtzEy9RDpVP9Jk46tzGs
9h8Ani/YO9FsUOpcKxiXovJkTPiKuI4e
=InkM
-----END PGP SIGNATURE-----

2014-01-08 22:40:31

by H. Peter Anvin

[permalink] [raw]
Subject: Re: Sanitize CPU-state when switching tasks (was sanitize CPU-state when switching from virtual-8086 mode to other task)

It is obviously critical here that we get a handle on if this is a
CPU-specific problem that we might have to work around or a general
problem with the Linux code.

-hpa

2014-01-09 22:53:10

by halfdog

[permalink] [raw]
Subject: Re: Sanitize CPU-state when switching tasks (was sanitize CPU-state when switching from virtual-8086 mode to other task)

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

It took me some time to build me the Debian Sid testing environment
for amd64 with the same quality, I have vor i386, but now it is ready.
And it seems, that amd64 is also affected, but lockup is immediately
(makes exploitation harder)

Here is the OOPS from the serial console, again in __switch_to

[ 498.783577] fpu exception: 0000 [#1] SMP
[ 498.787054] Modules linked in: xt_multiport xt_hashlimit xt_tcpudp
ipt_ULOG xt_LOG xt_conntrack iptable_raw iptable_nat nf_conntrack_ipv4
nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle
iptable_filter ip_tables x_tables fuse snd_pcm snd_page_alloc
snd_timer snd soundcore i2c_piix4 psmouse pcspkr evdev serio_raw
i2c_core parport_pc parport battery button ac ext4 crc16 mbcache jbd2
sd_mod crc_t10dif crct10dif_common sg sr_mod cdrom ata_generic
virtio_net mptspi scsi_transport_spi ata_piix virtio_pci virtio_ring
virtio mptscsih mptbase libata scsi_mod
[ 498.787205] CPU: 0 PID: 1783 Comm: Test Not tainted 3.12-1-amd64 #1
Debian 3.12.6-2
[ 498.787205] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS
VirtualBox 12/01/2006
[ 498.787205] task: ffff88000cb18840 ti: ffff88000b454000 task.ti:
ffff88000b454000
[ 498.787205] RIP: 0010:[<ffffffff81011730>] [<ffffffff81011730>]
__switch_to+0x2d0/0x490
[ 498.787205] RSP: 0018:ffff88000e0c78b8 EFLAGS: 00010002
[ 498.787205] RAX: 0000000000000001 RBX: ffff88000e0b77c0 RCX:
00000000c0000100
[ 498.787205] RDX: 0000000000000000 RSI: 0000000051e3f800 RDI:
00000000c0000100
[ 498.787205] RBP: ffff88000cb18840 R08: 0000000000000000 R09:
0000000000003314
[ 498.787205] R10: 0000000000001746 R11: 000000000000000f R12:
0000000000000000
[ 498.787205] R13: 0000000000000000 R14: ffff88000fc11780 R15:
0000000000000000
[ 498.787205] FS: 00007fb651e3f800(0000) GS:ffff88000fc00000(0000)
knlGS:0000000000000000
[ 498.787205] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 498.787205] CR2: 00007f72ddfcc990 CR3: 000000000e22d000 CR4:
00000000000006f0
[ 498.787205] Stack:
[ 498.787205] ffff88000e0b7bc0 000000010fc14330 ffff88000b4efac0
ffff88000e0b77c0
[ 498.787205] ffff88000fc142c0 ffff88000b5d3b40 0000000000000000
ffff88000e0b77c0
[ 498.787205] ffffffff8148febe ffff88000e0b77c0 0000000000000086
00000000000142c0
[ 498.787205] Call Trace:
[ 498.787205] Code: ff 66 2e 0f 1f 84 00 00 00 00 00 bf 7d 00 00 00
e8 e6 00 01 00 84 c0 0f 85 d7 fd ff ff 0f 06 66 66 90 66 90 e9 cb fd
ff ff 66 90 <0f> 77 db 83 94 04 00 00 66 90 eb 74 b8 ff ff ff ff 48 8b
bb 98
[ 498.787205] RIP [<ffffffff81011730>] __switch_to+0x2d0/0x490
[ 498.787205] RSP <ffff88000e0c78b8>
[ 498.787205] ---[ end trace 3f873c38e16c8005 ]---
[ 498.787205] Fixing recursive fault but reboot is needed!

I'll try to go the same line as before: understand it, write a
local-root-exploit for it (I feel it somehow, that this might be
really hard on that kernel) and test it on the bare hardware afterwards.

- --
http://www.halfdog.net/
PGP: 156A AE98 B91F 0114 FE88 2BD8 C459 9386 feed a bee
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlLPJ7MACgkQxFmThv7tq+6CSACeK7/SBzJOVvLlVBas9NANZYFp
pEUAn21LoX0ewsnOag7fomqtvKqUzGyL
=pst0
-----END PGP SIGNATURE-----

2014-01-09 22:58:25

by Borislav Petkov

[permalink] [raw]
Subject: Re: Sanitize CPU-state when switching tasks (was sanitize CPU-state when switching from virtual-8086 mode to other task)

On Wed, Jan 08, 2014 at 02:39:42PM -0800, H. Peter Anvin wrote:
> It is obviously critical here that we get a handle on if this is a
> CPU-specific problem that we might have to work around or a general
> problem with the Linux code.

Ok, I was able to reproduce with

http://www.halfdog.net/Security/2013/Vm86SyscallTaskSwitchKernelPanic/FpuStateTaskSwitchShmemXattrHandlersOverwriteWithNullPage.c

here on the latest linus+tip, see OOPS below:

$ AFLAGS=--32 decodecode < ~/fpu.oops

...

Code is:

All code
========
0: 89 d8 mov %ebx,%eax
2: e8 8c 96 00 00 call 0x9693
7: 85 c0 test %eax,%eax
9: 0f 85 9c 00 00 00 jne 0xab
f: fa cli
10: e8 7e bd 08 00 call 0x8bd93
15: e9 6f 00 00 00 jmp 0x89
1a: c7 83 a0 02 00 00 01 movl $0x1,0x2a0(%ebx)
21: 00 00 00
24: 64 89 1d ac a7 8a c1 mov %ebx,%fs:0xc18aa7ac
2b:* 0f 77 emms <-- trapping instruction
2d: db 83 a0 02 00 00 fildl 0x2a0(%ebx)
33: 89 f6 mov %esi,%esi
35: 89 f6 mov %esi,%esi
37: eb 27 jmp 0x60
39: b8 ff ff ff ff mov $0xffffffff,%eax
3e: 8b .byte 0x8b
3f: bb .byte 0xbb

Code starting with the faulting instruction
===========================================
0: 0f 77 emms
2: db 83 a0 02 00 00 fildl 0x2a0(%ebx)
8: 89 f6 mov %esi,%esi
a: 89 f6 mov %esi,%esi
c: eb 27 jmp 0x35
e: b8 ff ff ff ff mov $0xffffffff,%eax
13: 8b .byte 0x8b
14: bb .byte 0xbb

which points at EMMS, which gets runtime-replaced in:

static inline int restore_fpu_checking(struct task_struct *tsk)
{
/* AMD K7/K8 CPUs don't save/restore FDP/FIP/FOP unless an exception
is pending. Clear the x87 state here by setting it to fixed
values. "m" is a random variable that should be in L1 */
alternative_input(
ASM_NOP8 ASM_NOP2,
"emms\n\t" /* clear stack tags */
"fildl %P[addr]", /* set F?P to defined value */
X86_FEATURE_FXSAVE_LEAK,
[addr] "m" (tsk->thread.fpu.has_fpu));

return fpu_restore_checking(&tsk->thread.fpu);
}

Now, judging by the exception type and if I'm not mistaken, we get an
#MF which, according to the EMMS documentation means we get an #MF
exception because an unmasked x87 floating-point exception was pending.

Now, we most likely have done FXSAVE before that and this one doesn't
check for pending unmasked x87 floating-point exceptions and we have
CR0.NE=1b which means in that case that a numeric exception gets
generated.

Yadda, yadda, all is fine but why do we have any pending x87 FPU
exceptions then where we shouldn't? I dunno - I've never warmed up to
the FPU diddling in the kernel. I'll try to wrap my head around this
later...

---
localhost vmunix: [ 364.177380] fpu exception: 0000 [#1] PREEMPT SMP
localhost vmunix: [ 364.185005] Modules linked in: ipv6 radeon snd_hda_codec_conexant rtl8192ce rtl_pci snd_hda_codec_hdmi rtlwifi snd_hda_intel mac80211 snd_hda_codec snd_hwdep usbhid snd_pcm cfg80211 rtsx_pci_sdmmc snd_page_alloc drm_kms_helper mmc_core snd_timer rtsx_pci rtl8192c_common ttm thinkpad_acpi nvram snd ohci_pci ehci_pci ohci_hcd mfd_core button ac video battery ehci_hcd pcspkr thermal k10temp soundcore
localhost vmunix: [ 364.209743] CPU: 1 PID: 1200 Comm: find Tainted: G W 3.13.0-rc7+ #4
localhost vmunix: [ 364.217947] Hardware name: LENOVO 30515QG/30515QG, BIOS 8RET30WW (1.12 ) 09/15/2011
localhost vmunix: [ 364.226231] task: f4104980 ti: f3800000 task.ti: f3800000
localhost vmunix: [ 364.234511] EIP: 0060:[<c10026b8>] EFLAGS: 00010002 CPU: 1
localhost vmunix: [ 364.242734] EIP is at math_state_restore+0x48/0x1a0
localhost vmunix: [ 364.250939] EAX: f3801fb4 EBX: f4104980 ECX: 0000007b EDX: ffffffff
localhost vmunix: [ 364.259188] ESI: 089c58f8 EDI: c1003510 EBP: f3801fa0 ESP: f3801f98
localhost vmunix: [ 364.267364] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
localhost vmunix: [ 364.275542] CR0: 80050033 CR2: b75d6f32 CR3: 331f9000 CR4: 000007d0
localhost vmunix: [ 364.283758] Stack:
localhost vmunix: [ 364.291834] f3801fb4 c1003510 f3801fac c1003535 089c39f8 bf8b8e88 c155c96b 089c39f8
localhost vmunix: [ 364.300213] 00000000 00000029 089c58f8 00000000 bf8b8e88 080681fc 0000007b 0000007b
localhost vmunix: [ 364.308631] 00000000 c1003510 ffffffff 0805e0f8 00000073 00010206 bf8b8e30 0000007b
localhost vmunix: [ 364.316985] Call Trace:
localhost vmunix: [ 364.325104] [<c1003510>] ? smp_thermal_interrupt+0x20/0x20
localhost vmunix: [ 364.333300] [<c1003535>] do_device_not_available+0x25/0x40
localhost vmunix: [ 364.341458] [<c155c96b>] error_code+0x5f/0x64
localhost vmunix: [ 364.349541] [<c1003510>] ? smp_thermal_interrupt+0x20/0x20
localhost vmunix: [ 364.357649] Code: 89 d8 e8 8c 96 00 00 85 c0 0f 85 9c 00 00 00 fa e8 7e bd 08 00 e9 6f 00 00 00 c7 83 a0 02 00 00 01 00 00 00 64 89 1d ac a7 8a c1 <0f> 77 db 83 a0 02 00 00 89 f6 89 f6 eb 27 b8 ff ff ff ff 8b bb
localhost vmunix: [ 364.375367] EIP: [<c10026b8>] math_state_restore+0x48/0x1a0 SS:ESP 0068:f3801f98
localhost vmunix: [ 364.383830] ---[ end trace c8241b0abe86a792 ]---

--
Regards/Gruss,
Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

2014-01-09 23:02:17

by Borislav Petkov

[permalink] [raw]
Subject: Re: Sanitize CPU-state when switching tasks (was sanitize CPU-state when switching from virtual-8086 mode to other task)

On Thu, Jan 09, 2014 at 10:50:28PM +0000, halfdog wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> It took me some time to build me the Debian Sid testing environment
> for amd64 with the same quality, I have vor i386, but now it is ready.
> And it seems, that amd64 is also affected, but lockup is immediately
> (makes exploitation harder)
>
> Here is the OOPS from the serial console, again in __switch_to
>
> [ 498.783577] fpu exception: 0000 [#1] SMP
> [ 498.787054] Modules linked in: xt_multiport xt_hashlimit xt_tcpudp
> ipt_ULOG xt_LOG xt_conntrack iptable_raw iptable_nat nf_conntrack_ipv4
> nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle
> iptable_filter ip_tables x_tables fuse snd_pcm snd_page_alloc
> snd_timer snd soundcore i2c_piix4 psmouse pcspkr evdev serio_raw
> i2c_core parport_pc parport battery button ac ext4 crc16 mbcache jbd2
> sd_mod crc_t10dif crct10dif_common sg sr_mod cdrom ata_generic
> virtio_net mptspi scsi_transport_spi ata_piix virtio_pci virtio_ring
> virtio mptscsih mptbase libata scsi_mod
> [ 498.787205] CPU: 0 PID: 1783 Comm: Test Not tainted 3.12-1-amd64 #1
> Debian 3.12.6-2
> [ 498.787205] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS
> VirtualBox 12/01/2006
> [ 498.787205] task: ffff88000cb18840 ti: ffff88000b454000 task.ti:
> ffff88000b454000
> [ 498.787205] RIP: 0010:[<ffffffff81011730>] [<ffffffff81011730>]
> __switch_to+0x2d0/0x490
> [ 498.787205] RSP: 0018:ffff88000e0c78b8 EFLAGS: 00010002
> [ 498.787205] RAX: 0000000000000001 RBX: ffff88000e0b77c0 RCX:
> 00000000c0000100
> [ 498.787205] RDX: 0000000000000000 RSI: 0000000051e3f800 RDI:
> 00000000c0000100
> [ 498.787205] RBP: ffff88000cb18840 R08: 0000000000000000 R09:
> 0000000000003314
> [ 498.787205] R10: 0000000000001746 R11: 000000000000000f R12:
> 0000000000000000
> [ 498.787205] R13: 0000000000000000 R14: ffff88000fc11780 R15:
> 0000000000000000
> [ 498.787205] FS: 00007fb651e3f800(0000) GS:ffff88000fc00000(0000)
> knlGS:0000000000000000
> [ 498.787205] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 498.787205] CR2: 00007f72ddfcc990 CR3: 000000000e22d000 CR4:
> 00000000000006f0
> [ 498.787205] Stack:
> [ 498.787205] ffff88000e0b7bc0 000000010fc14330 ffff88000b4efac0
> ffff88000e0b77c0
> [ 498.787205] ffff88000fc142c0 ffff88000b5d3b40 0000000000000000
> ffff88000e0b77c0
> [ 498.787205] ffffffff8148febe ffff88000e0b77c0 0000000000000086
> 00000000000142c0
> [ 498.787205] Call Trace:
> [ 498.787205] Code: ff 66 2e 0f 1f 84 00 00 00 00 00 bf 7d 00 00 00
> e8 e6 00 01 00 84 c0 0f 85 d7 fd ff ff 0f 06 66 66 90 66 90 e9 cb fd
> ff ff 66 90 <0f> 77 db 83 94 04 00 00 66 90 eb 74 b8 ff ff ff ff 48 8b

Yep, EMMS again: 0f 77 - unhandled x87 FPU exception, see my other mail
I just sent.

I'll try this on another AMD machine tomorrow to see whether it is
affected too.

--
Regards/Gruss,
Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

2014-01-10 00:42:37

by Linus Torvalds

[permalink] [raw]
Subject: Re: Sanitize CPU-state when switching tasks (was sanitize CPU-state when switching from virtual-8086 mode to other task)

On Fri, Jan 10, 2014 at 6:58 AM, Borislav Petkov <[email protected]> wrote:
>
> Ok, I was able to reproduce

Looking at this, I think this is just a bug in our
restore_fpu_checking() hackery for X86_FEATURE_FXSAVE_LEAK..

Which also explains why it only triggers on E-350 - it's only relevant
for those K7/K8 CPU's that use this.

Maybe just add a fcnlex to before the emms? Something like this
(TOTALLY UNTESTED!!) attached patch.

Linus


Attachments:
patch.diff (1.04 kB)

2014-01-10 02:14:10

by H. Peter Anvin

[permalink] [raw]
Subject: Re: Sanitize CPU-state when switching tasks (was sanitize CPU-state when switching from virtual-8086 mode to other task)

On 01/09/2014 04:42 PM, Linus Torvalds wrote:
> On Fri, Jan 10, 2014 at 6:58 AM, Borislav Petkov <[email protected]> wrote:
>>
>> Ok, I was able to reproduce
>
> Looking at this, I think this is just a bug in our
> restore_fpu_checking() hackery for X86_FEATURE_FXSAVE_LEAK..
>
> Which also explains why it only triggers on E-350 - it's only relevant
> for those K7/K8 CPU's that use this.
>
> Maybe just add a fcnlex to before the emms? Something like this
> (TOTALLY UNTESTED!!) attached patch.
>

OK, that sounds very reasonable. Boris, halfdog, does something like
this resolve your problem?

-hpa

2014-01-10 10:06:13

by Borislav Petkov

[permalink] [raw]
Subject: Re: Sanitize CPU-state when switching tasks (was sanitize CPU-state when switching from virtual-8086 mode to other task)

On Thu, Jan 09, 2014 at 06:13:19PM -0800, H. Peter Anvin wrote:
> OK, that sounds very reasonable. Boris, halfdog, does something like
> this resolve your problem?

Yeah, if in doubt, Linus to the rescue! :)

Tested-by: Borislav Petkov <[email protected]>

--
Regards/Gruss,
Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

2014-01-10 11:16:30

by Linus Torvalds

[permalink] [raw]
Subject: Re: Sanitize CPU-state when switching tasks (was sanitize CPU-state when switching from virtual-8086 mode to other task)

On Fri, Jan 10, 2014 at 6:06 PM, Borislav Petkov <[email protected]> wrote:
>
> Tested-by: Borislav Petkov <[email protected]>

Ok, good.

Peter, do you want to take it (feel free to add my sign-off), or
should I just commit it?

Also, is there a way to have a "likely not true" version of that
"static_cpu_has()"? There seems to be no way to make the non-K7/K8
case the fallthrough code.. Not that this is likely that
performance-critical, but..

Linus

2014-01-10 11:34:42

by Borislav Petkov

[permalink] [raw]
Subject: Re: Sanitize CPU-state when switching tasks (was sanitize CPU-state when switching from virtual-8086 mode to other task)

On Fri, Jan 10, 2014 at 07:16:24PM +0800, Linus Torvalds wrote:
> Also, is there a way to have a "likely not true" version of that
> "static_cpu_has()"? There seems to be no way to make the non-K7/K8
> case

FWIW, this is not only K7/K8 but actually all AMD from family 6 onwards,
which is - practically speaking - all AMD.

--
Regards/Gruss,
Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

2014-01-10 16:12:14

by H. Peter Anvin

[permalink] [raw]
Subject: Re: Sanitize CPU-state when switching tasks (was sanitize CPU-state when switching from virtual-8086 mode to other task)

On 01/10/2014 03:16 AM, Linus Torvalds wrote:
> On Fri, Jan 10, 2014 at 6:06 PM, Borislav Petkov <[email protected]> wrote:
>>
>> Tested-by: Borislav Petkov <[email protected]>
>
> Ok, good.
>
> Peter, do you want to take it (feel free to add my sign-off), or
> should I just commit it?
>
> Also, is there a way to have a "likely not true" version of that
> "static_cpu_has()"? There seems to be no way to make the non-K7/K8
> case the fallthrough code.. Not that this is likely that
> performance-critical, but..
>

I'll take it.

We don't have a "likely not true" version of static_cpu_has() at this
point... it would mean we couldn't do short jumps unfortunately (and
they would still take the false path until alternatives run.)

-hpa

Subject: [tip:x86/urgent] x86, fpu, amd: Clear exceptions in AMD FXSAVE workaround

Commit-ID: 26bef1318adc1b3a530ecc807ef99346db2aa8b0
Gitweb: http://git.kernel.org/tip/26bef1318adc1b3a530ecc807ef99346db2aa8b0
Author: Linus Torvalds <[email protected]>
AuthorDate: Sat, 11 Jan 2014 19:15:52 -0800
Committer: H. Peter Anvin <[email protected]>
CommitDate: Sat, 11 Jan 2014 19:15:52 -0800

x86, fpu, amd: Clear exceptions in AMD FXSAVE workaround

Before we do an EMMS in the AMD FXSAVE information leak workaround we
need to clear any pending exceptions, otherwise we trap with a
floating-point exception inside this code.

Reported-by: halfdog <[email protected]>
Tested-by: Borislav Petkov <[email protected]>
Link: http://lkml.kernel.org/r/CA%2B55aFxQnY_PCG_n4=0w-VG=YLXL-yr7oMxyy0WU2gCBAf3ydg@mail.gmail.com
Signed-off-by: H. Peter Anvin <[email protected]>
---
arch/x86/include/asm/fpu-internal.h | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/arch/x86/include/asm/fpu-internal.h b/arch/x86/include/asm/fpu-internal.h
index c49a613..cea1c76 100644
--- a/arch/x86/include/asm/fpu-internal.h
+++ b/arch/x86/include/asm/fpu-internal.h
@@ -293,12 +293,13 @@ static inline int restore_fpu_checking(struct task_struct *tsk)
/* AMD K7/K8 CPUs don't save/restore FDP/FIP/FOP unless an exception
is pending. Clear the x87 state here by setting it to fixed
values. "m" is a random variable that should be in L1 */
- alternative_input(
- ASM_NOP8 ASM_NOP2,
- "emms\n\t" /* clear stack tags */
- "fildl %P[addr]", /* set F?P to defined value */
- X86_FEATURE_FXSAVE_LEAK,
- [addr] "m" (tsk->thread.fpu.has_fpu));
+ if (unlikely(static_cpu_has(X86_FEATURE_FXSAVE_LEAK))) {
+ asm volatile(
+ "fnclex\n\t"
+ "emms\n\t"
+ "fildl %P[addr]" /* set F?P to defined value */
+ : : [addr] "m" (tsk->thread.fpu.has_fpu));
+ }

return fpu_restore_checking(&tsk->thread.fpu);
}