Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S1757543AbbGGNu0 (ORCPT <rfc822;w@1wt.eu>);
	Tue, 7 Jul 2015 09:50:26 -0400
Received: from mx1.redhat.com ([209.132.183.28]:38721 "EHLO mx1.redhat.com"
	rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
	id S1756943AbbGGNuR (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
	Tue, 7 Jul 2015 09:50:17 -0400
Date: Tue, 7 Jul 2015 15:50:13 +0200
From: Radim =?utf-8?B?S3LEjW3DocWZ?= <rkrcmar@redhat.com>
To: Paolo Bonzini <pbonzini@redhat.com>
Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, stable@vger.kernel.org,
        Yang Zhang <yang.z.zhang@intel.com>, Liang Li <liang.z.li@intel.com>
Subject: Re: [PATCH] KVM: VMX: fix vmwrite to invalid VMCS
Message-ID: <20150707135012.GA26862@potion.brq.redhat.com>
References: <1435931368-27730-1-git-send-email-rkrcmar@redhat.com>
 <5596A679.1050202@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <5596A679.1050202@redhat.com>
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
Content-Length: 3463
Lines: 87

2015-07-03 17:12+0200, Paolo Bonzini:
> On 03/07/2015 15:49, Radim Krčmář wrote:
>> fpu_activate is called outside of vcpu_load(), which means it should not
>> touch VMCS, but fpu_activate needs to.  Avoid the call by moving it to a
>> point where we know that the guest needs eager FPU and VMCS is loaded.
>> 
>> This will get rid of the following trace
>> 
>>  vmwrite error: reg 6800 value 0 (err 1)
>>   [<ffffffff8162035b>] dump_stack+0x19/0x1b
>>   [<ffffffffa046c701>] vmwrite_error+0x2c/0x2e [kvm_intel]
>>   [<ffffffffa045f26f>] vmcs_writel+0x1f/0x30 [kvm_intel]
>>   [<ffffffffa04617e5>] vmx_fpu_activate.part.61+0x45/0xb0 [kvm_intel]
>>   [<ffffffffa0461865>] vmx_fpu_activate+0x15/0x20 [kvm_intel]
>>   [<ffffffffa0560b91>] kvm_arch_vcpu_create+0x51/0x70 [kvm]
>>   [<ffffffffa0548011>] kvm_vm_ioctl+0x1c1/0x760 [kvm]
>>   [<ffffffff8118b55a>] ? handle_mm_fault+0x49a/0xec0
>>   [<ffffffff811e47d5>] do_vfs_ioctl+0x2e5/0x4c0
>>   [<ffffffff8127abbe>] ? file_has_perm+0xae/0xc0
>>   [<ffffffff811e4a51>] SyS_ioctl+0xa1/0xc0
>>   [<ffffffff81630949>] system_call_fastpath+0x16/0x1b
>> 
>> (Note: we also unconditionally activate FPU in vmx_vcpu_reset(), so the
>>  removed code added nothing.)
>> 
>> Fixes: c447e76b4cab ("kvm/fpu: Enable eager restore kvm FPU for MPX")
>> Cc: <stable@vger.kernel.org>
>> Reported-by: Vlastimil Holer <vlastimil.holer@gmail.com>
>> Signed-off-by: Radim Krčmář <rkrcmar@redhat.com>
> 
> Andrey reported offlist that the bug went away by reverting 1cde293.  So
> the patch would at least need a new commit message. :)
> 
> I'm putting it on hold.

I think it's a different bug than the one Andrey reproduced
(https://bugzilla.kernel.org/show_bug.cgi?id=100671).
I'll send a v2 that cleans up the code and makes the commit message
clearer, unless you find the reasoning below unsound.

This bug is specific to 'kvm_arch_vcpu_create()' and Vlastimil Holer hit
it on RHEL 7.2 (278.el7) kernel that didn't have 1cde2930e154
("sched/preempt: Add static_key() to preempt_notifiers").

The commit message does not base on tracing (I haven't reproduced it),
but I couldn't make sense out of this bug otherwise.
I think it happens just because other VCPU preempted the new one between
vmx_vcpu_put()+put_cpu() and the end of kvm_x86_ops->fpu_activate(), so
vmwrite accessed different VMCS.  The code in kvm_vm_ioctl_create_vcpu()
that made me think so:

  vcpu = kvm_arch_vcpu_create(id) {
    vcpu = kvm_x86_ops->vcpu_create(kvm, id) {
      vmx = kmem_cache_zalloc(kvm_vcpu_cache, GFP_KERNEL);
      kvm_vcpu_init(&vmx->vcpu, kvm, id);
      vmx->loaded_vmcs = &vmx->vmcs01;
      vmx->loaded_vmcs->vmcs = alloc_vmcs();
      loaded_vmcs_init(vmx->loaded_vmcs);

      // disabling preemption and activating VMCS
      cpu = get_cpu();
      vmx_vcpu_load(&vmx->vcpu, cpu);

      vmx_vcpu_setup(vmx);

      // abandoning VMCS and enabling preemption
      vmx_vcpu_put(&vmx->vcpu);
      put_cpu();

      return &vmx->vcpu;
    }

    // enabled preemption and undefined current VMCS
    kvm_x86_ops->fpu_activate(vcpu);
    return vcpu;
  }

  preempt_notifier_init(&vcpu->preempt_notifier, &kvm_preempt_ops);
  kvm_arch_vcpu_setup(vcpu) {
    vcpu_load(vcpu);
    ...
  }
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/