2017-12-06 11:59:26

by Wanpeng Li

Subject: [PATCH v2] KVM: X86: Fix load RFLAGS w/o the fixed bit

From: Wanpeng Li <[email protected]>

*** Guest State ***
CR0: actual=0x0000000000000030, shadow=0x0000000060000010, gh_mask=fffffffffffffff7
CR4: actual=0x0000000000002050, shadow=0x0000000000000000, gh_mask=ffffffffffffe871
CR3 = 0x00000000fffbc000
RSP = 0x0000000000000000 RIP = 0x0000000000000000
RFLAGS=0x00000000 DR7 = 0x0000000000000400
^^^^^^^^^^

The failed vmentry is triggered by the following testcase when ept=Y:

#include <unistd.h>
#include <sys/syscall.h>
#include <string.h>
#include <stdint.h>
#include <linux/kvm.h>
#include <fcntl.h>
#include <sys/ioctl.h>

long r[5];
int main()
{
r[2] = open("/dev/kvm", O_RDONLY);
r[3] = ioctl(r[2], KVM_CREATE_VM, 0);
r[4] = ioctl(r[3], KVM_CREATE_VCPU, 7);
struct kvm_regs regs = {
.rflags = 0,
};
ioctl(r[4], KVM_SET_REGS, &regs);
ioctl(r[4], KVM_RUN, 0);
}

X86 RFLAGS bit 1 is fixed set, userspace can simply clearing bit 1
of RFLAGS with KVM_SET_REGS ioctl which results in vmentry fails.
This patch fixes it by oring X86_EFLAGS_FIXED during ioctl.

Suggested-by: Jim Mattson <[email protected]>
Cc: Paolo Bonzini <[email protected]>
Cc: Radim Krčmář <[email protected]>
Cc: Jim Mattson <[email protected]>
Signed-off-by: Wanpeng Li <[email protected]>
---
v1 -> v2:
* Oring X86_EFLAGS_FIXED

virt/kvm/kvm_main.c | 1 +
1 file changed, 1 insertion(+)

diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index b55bad3..0f3f283 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -2602,6 +2602,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
r = PTR_ERR(kvm_regs);
goto out;
}
+ kvm_regs->rflags |= X86_EFLAGS_FIXED;
r = kvm_arch_vcpu_ioctl_set_regs(vcpu, kvm_regs);
kfree(kvm_regs);
break;
--
2.7.4

2017-12-06 14:41:06

by David Hildenbrand

Subject: Re: [PATCH v2] KVM: X86: Fix load RFLAGS w/o the fixed bit

On 06.12.2017 12:59, Wanpeng Li wrote:
> From: Wanpeng Li <[email protected]>
>
> *** Guest State ***
> CR0: actual=0x0000000000000030, shadow=0x0000000060000010, gh_mask=fffffffffffffff7
> CR4: actual=0x0000000000002050, shadow=0x0000000000000000, gh_mask=ffffffffffffe871
> CR3 = 0x00000000fffbc000
> RSP = 0x0000000000000000 RIP = 0x0000000000000000
> RFLAGS=0x00000000 DR7 = 0x0000000000000400
> ^^^^^^^^^^
>
> The failed vmentry is triggered by the following testcase when ept=Y:
>
> #include <unistd.h>
> #include <sys/syscall.h>
> #include <string.h>
> #include <stdint.h>
> #include <linux/kvm.h>
> #include <fcntl.h>
> #include <sys/ioctl.h>
>
> long r[5];
> int main()
> {
> r[2] = open("/dev/kvm", O_RDONLY);
> r[3] = ioctl(r[2], KVM_CREATE_VM, 0);
> r[4] = ioctl(r[3], KVM_CREATE_VCPU, 7);
> struct kvm_regs regs = {
> .rflags = 0,
> };
> ioctl(r[4], KVM_SET_REGS, &regs);
> ioctl(r[4], KVM_RUN, 0);
> }
>
> X86 RFLAGS bit 1 is fixed set, userspace can simply clearing bit 1
> of RFLAGS with KVM_SET_REGS ioctl which results in vmentry fails.
> This patch fixes it by oring X86_EFLAGS_FIXED during ioctl.
>
> Suggested-by: Jim Mattson <[email protected]>
> Cc: Paolo Bonzini <[email protected]>
> Cc: Radim Krčmář <[email protected]>
> Cc: Jim Mattson <[email protected]>
> Signed-off-by: Wanpeng Li <[email protected]>
> ---
> v1 -> v2:
> * Oring X86_EFLAGS_FIXED
>
> virt/kvm/kvm_main.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index b55bad3..0f3f283 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -2602,6 +2602,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
> r = PTR_ERR(kvm_regs);
> goto out;
> }
> + kvm_regs->rflags |= X86_EFLAGS_FIXED;
> r = kvm_arch_vcpu_ioctl_set_regs(vcpu, kvm_regs);
> kfree(kvm_regs);
> break;
>

Not sure if failing KVM_SET_REGS would be nicer, but maybe this has
already been discussed. So this should be fine.

Reviewed-by: David Hildenbrand <[email protected]>

--

Thanks,

David / dhildenb

2017-12-06 17:15:00

by Jim Mattson

Subject: Re: [PATCH v2] KVM: X86: Fix load RFLAGS w/o the fixed bit

What if I change the testcase so that regs.rflags is initialized to (1 << 3)?

On Wed, Dec 6, 2017 at 6:40 AM, David Hildenbrand <[email protected]> wrote:
> On 06.12.2017 12:59, Wanpeng Li wrote:
>> From: Wanpeng Li <[email protected]>
>>
>> *** Guest State ***
>> CR0: actual=0x0000000000000030, shadow=0x0000000060000010, gh_mask=fffffffffffffff7
>> CR4: actual=0x0000000000002050, shadow=0x0000000000000000, gh_mask=ffffffffffffe871
>> CR3 = 0x00000000fffbc000
>> RSP = 0x0000000000000000 RIP = 0x0000000000000000
>> RFLAGS=0x00000000 DR7 = 0x0000000000000400
>> ^^^^^^^^^^
>>
>> The failed vmentry is triggered by the following testcase when ept=Y:
>>
>> #include <unistd.h>
>> #include <sys/syscall.h>
>> #include <string.h>
>> #include <stdint.h>
>> #include <linux/kvm.h>
>> #include <fcntl.h>
>> #include <sys/ioctl.h>
>>
>> long r[5];
>> int main()
>> {
>> r[2] = open("/dev/kvm", O_RDONLY);
>> r[3] = ioctl(r[2], KVM_CREATE_VM, 0);
>> r[4] = ioctl(r[3], KVM_CREATE_VCPU, 7);
>> struct kvm_regs regs = {
>> .rflags = 0,
>> };
>> ioctl(r[4], KVM_SET_REGS, &regs);
>> ioctl(r[4], KVM_RUN, 0);
>> }
>>
>> X86 RFLAGS bit 1 is fixed set, userspace can simply clearing bit 1
>> of RFLAGS with KVM_SET_REGS ioctl which results in vmentry fails.
>> This patch fixes it by oring X86_EFLAGS_FIXED during ioctl.
>>
>> Suggested-by: Jim Mattson <[email protected]>
>> Cc: Paolo Bonzini <[email protected]>
>> Cc: Radim Krčmář <[email protected]>
>> Cc: Jim Mattson <[email protected]>
>> Signed-off-by: Wanpeng Li <[email protected]>
>> ---
>> v1 -> v2:
>> * Oring X86_EFLAGS_FIXED
>>
>> virt/kvm/kvm_main.c | 1 +
>> 1 file changed, 1 insertion(+)
>>
>> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
>> index b55bad3..0f3f283 100644
>> --- a/virt/kvm/kvm_main.c
>> +++ b/virt/kvm/kvm_main.c
>> @@ -2602,6 +2602,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
>> r = PTR_ERR(kvm_regs);
>> goto out;
>> }
>> + kvm_regs->rflags |= X86_EFLAGS_FIXED;
>> r = kvm_arch_vcpu_ioctl_set_regs(vcpu, kvm_regs);
>> kfree(kvm_regs);
>> break;
>>
>
> Not sure if failing KVM_SET_REGS would be nicer, but maybe this has
> already been discussed. So this should be fine.
>
> Reviewed-by: David Hildenbrand <[email protected]>
>
> --
>
> Thanks,
>
> David / dhildenb

2017-12-06 17:28:59

by David Hildenbrand

Subject: Re: [PATCH v2] KVM: X86: Fix load RFLAGS w/o the fixed bit

On 06.12.2017 15:40, David Hildenbrand wrote:
> On 06.12.2017 12:59, Wanpeng Li wrote:
>> From: Wanpeng Li <[email protected]>
>>
>> *** Guest State ***
>> CR0: actual=0x0000000000000030, shadow=0x0000000060000010, gh_mask=fffffffffffffff7
>> CR4: actual=0x0000000000002050, shadow=0x0000000000000000, gh_mask=ffffffffffffe871
>> CR3 = 0x00000000fffbc000
>> RSP = 0x0000000000000000 RIP = 0x0000000000000000
>> RFLAGS=0x00000000 DR7 = 0x0000000000000400
>> ^^^^^^^^^^
>>
>> The failed vmentry is triggered by the following testcase when ept=Y:
>>
>> #include <unistd.h>
>> #include <sys/syscall.h>
>> #include <string.h>
>> #include <stdint.h>
>> #include <linux/kvm.h>
>> #include <fcntl.h>
>> #include <sys/ioctl.h>
>>
>> long r[5];
>> int main()
>> {
>> r[2] = open("/dev/kvm", O_RDONLY);
>> r[3] = ioctl(r[2], KVM_CREATE_VM, 0);
>> r[4] = ioctl(r[3], KVM_CREATE_VCPU, 7);
>> struct kvm_regs regs = {
>> .rflags = 0,
>> };
>> ioctl(r[4], KVM_SET_REGS, &regs);
>> ioctl(r[4], KVM_RUN, 0);
>> }
>>
>> X86 RFLAGS bit 1 is fixed set, userspace can simply clearing bit 1
>> of RFLAGS with KVM_SET_REGS ioctl which results in vmentry fails.
>> This patch fixes it by oring X86_EFLAGS_FIXED during ioctl.
>>
>> Suggested-by: Jim Mattson <[email protected]>
>> Cc: Paolo Bonzini <[email protected]>
>> Cc: Radim Krčmář <[email protected]>
>> Cc: Jim Mattson <[email protected]>
>> Signed-off-by: Wanpeng Li <[email protected]>
>> ---
>> v1 -> v2:
>> * Oring X86_EFLAGS_FIXED
>>
>> virt/kvm/kvm_main.c | 1 +
>> 1 file changed, 1 insertion(+)
>>
>> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
>> index b55bad3..0f3f283 100644
>> --- a/virt/kvm/kvm_main.c
>> +++ b/virt/kvm/kvm_main.c
>> @@ -2602,6 +2602,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
>> r = PTR_ERR(kvm_regs);
>> goto out;
>> }
>> + kvm_regs->rflags |= X86_EFLAGS_FIXED;

Just realized that this should go into kvm_arch_vcpu_ioctl_set_regs().


>> r = kvm_arch_vcpu_ioctl_set_regs(vcpu, kvm_regs);
>> kfree(kvm_regs);
>> break;
>>
>
> Not sure if failing KVM_SET_REGS would be nicer, but maybe this has
> already been discussed. So this should be fine.
>
> Reviewed-by: David Hildenbrand <[email protected]>
>
--

Thanks,

David / dhildenb

2017-12-07 03:21:41

by Wanpeng Li

Subject: Re: [PATCH v2] KVM: X86: Fix load RFLAGS w/o the fixed bit

2017-12-07 1:14 GMT+08:00 Jim Mattson <[email protected]>:
> What if I change the testcase so that regs.rflags is initialized to (1 << 3)?

>From Paolo's pointing:

> I suspect somebody might be passing an all-zero regs struct to
> KVM_SET_REGS, so ORing X86_EFLAGS_FIXED is better.

I think the patch tries to fix a case when the userspace forgets to
set the fixed bit of rflags instead of set invalidate guest state
deliberately.

Regards,
Wanpeng Li

>
> On Wed, Dec 6, 2017 at 6:40 AM, David Hildenbrand <[email protected]> wrote:
>> On 06.12.2017 12:59, Wanpeng Li wrote:
>>> From: Wanpeng Li <[email protected]>
>>>
>>> *** Guest State ***
>>> CR0: actual=0x0000000000000030, shadow=0x0000000060000010, gh_mask=fffffffffffffff7
>>> CR4: actual=0x0000000000002050, shadow=0x0000000000000000, gh_mask=ffffffffffffe871
>>> CR3 = 0x00000000fffbc000
>>> RSP = 0x0000000000000000 RIP = 0x0000000000000000
>>> RFLAGS=0x00000000 DR7 = 0x0000000000000400
>>> ^^^^^^^^^^
>>>
>>> The failed vmentry is triggered by the following testcase when ept=Y:
>>>
>>> #include <unistd.h>
>>> #include <sys/syscall.h>
>>> #include <string.h>
>>> #include <stdint.h>
>>> #include <linux/kvm.h>
>>> #include <fcntl.h>
>>> #include <sys/ioctl.h>
>>>
>>> long r[5];
>>> int main()
>>> {
>>> r[2] = open("/dev/kvm", O_RDONLY);
>>> r[3] = ioctl(r[2], KVM_CREATE_VM, 0);
>>> r[4] = ioctl(r[3], KVM_CREATE_VCPU, 7);
>>> struct kvm_regs regs = {
>>> .rflags = 0,
>>> };
>>> ioctl(r[4], KVM_SET_REGS, &regs);
>>> ioctl(r[4], KVM_RUN, 0);
>>> }
>>>
>>> X86 RFLAGS bit 1 is fixed set, userspace can simply clearing bit 1
>>> of RFLAGS with KVM_SET_REGS ioctl which results in vmentry fails.
>>> This patch fixes it by oring X86_EFLAGS_FIXED during ioctl.
>>>
>>> Suggested-by: Jim Mattson <[email protected]>
>>> Cc: Paolo Bonzini <[email protected]>
>>> Cc: Radim Krčmář <[email protected]>
>>> Cc: Jim Mattson <[email protected]>
>>> Signed-off-by: Wanpeng Li <[email protected]>
>>> ---
>>> v1 -> v2:
>>> * Oring X86_EFLAGS_FIXED
>>>
>>> virt/kvm/kvm_main.c | 1 +
>>> 1 file changed, 1 insertion(+)
>>>
>>> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
>>> index b55bad3..0f3f283 100644
>>> --- a/virt/kvm/kvm_main.c
>>> +++ b/virt/kvm/kvm_main.c
>>> @@ -2602,6 +2602,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
>>> r = PTR_ERR(kvm_regs);
>>> goto out;
>>> }
>>> + kvm_regs->rflags |= X86_EFLAGS_FIXED;
>>> r = kvm_arch_vcpu_ioctl_set_regs(vcpu, kvm_regs);
>>> kfree(kvm_regs);
>>> break;
>>>
>>
>> Not sure if failing KVM_SET_REGS would be nicer, but maybe this has
>> already been discussed. So this should be fine.
>>
>> Reviewed-by: David Hildenbrand <[email protected]>
>>
>> --
>>
>> Thanks,
>>
>> David / dhildenb

2017-12-07 08:32:05

by Wanpeng Li

Subject: Re: [PATCH v2] KVM: X86: Fix load RFLAGS w/o the fixed bit

2017-12-07 1:28 GMT+08:00 David Hildenbrand <[email protected]>:
> On 06.12.2017 15:40, David Hildenbrand wrote:
>> On 06.12.2017 12:59, Wanpeng Li wrote:
>>> From: Wanpeng Li <[email protected]>
>>>
>>> *** Guest State ***
>>> CR0: actual=0x0000000000000030, shadow=0x0000000060000010, gh_mask=fffffffffffffff7
>>> CR4: actual=0x0000000000002050, shadow=0x0000000000000000, gh_mask=ffffffffffffe871
>>> CR3 = 0x00000000fffbc000
>>> RSP = 0x0000000000000000 RIP = 0x0000000000000000
>>> RFLAGS=0x00000000 DR7 = 0x0000000000000400
>>> ^^^^^^^^^^
>>>
>>> The failed vmentry is triggered by the following testcase when ept=Y:
>>>
>>> #include <unistd.h>
>>> #include <sys/syscall.h>
>>> #include <string.h>
>>> #include <stdint.h>
>>> #include <linux/kvm.h>
>>> #include <fcntl.h>
>>> #include <sys/ioctl.h>
>>>
>>> long r[5];
>>> int main()
>>> {
>>> r[2] = open("/dev/kvm", O_RDONLY);
>>> r[3] = ioctl(r[2], KVM_CREATE_VM, 0);
>>> r[4] = ioctl(r[3], KVM_CREATE_VCPU, 7);
>>> struct kvm_regs regs = {
>>> .rflags = 0,
>>> };
>>> ioctl(r[4], KVM_SET_REGS, &regs);
>>> ioctl(r[4], KVM_RUN, 0);
>>> }
>>>
>>> X86 RFLAGS bit 1 is fixed set, userspace can simply clearing bit 1
>>> of RFLAGS with KVM_SET_REGS ioctl which results in vmentry fails.
>>> This patch fixes it by oring X86_EFLAGS_FIXED during ioctl.
>>>
>>> Suggested-by: Jim Mattson <[email protected]>
>>> Cc: Paolo Bonzini <[email protected]>
>>> Cc: Radim Krčmář <[email protected]>
>>> Cc: Jim Mattson <[email protected]>
>>> Signed-off-by: Wanpeng Li <[email protected]>
>>> ---
>>> v1 -> v2:
>>> * Oring X86_EFLAGS_FIXED
>>>
>>> virt/kvm/kvm_main.c | 1 +
>>> 1 file changed, 1 insertion(+)
>>>
>>> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
>>> index b55bad3..0f3f283 100644
>>> --- a/virt/kvm/kvm_main.c
>>> +++ b/virt/kvm/kvm_main.c
>>> @@ -2602,6 +2602,7 @@ static long kvm_vcpu_ioctl(struct file *filp,
>>> r = PTR_ERR(kvm_regs);
>>> goto out;
>>> }
>>> + kvm_regs->rflags |= X86_EFLAGS_FIXED;
>
> Just realized that this should go into kvm_arch_vcpu_ioctl_set_regs().

Do it in v3, thanks for your review. :)

Regards,
Wanpeng Li

>
>
>>> r = kvm_arch_vcpu_ioctl_set_regs(vcpu, kvm_regs);
>>> kfree(kvm_regs);
>>> break;
>>>
>>
>> Not sure if failing KVM_SET_REGS would be nicer, but maybe this has
>> already been discussed. So this should be fine.
>>
>> Reviewed-by: David Hildenbrand <[email protected]>
>>
> --
>
> Thanks,
>
> David / dhildenb