Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754187AbdLUNDy (ORCPT ); Thu, 21 Dec 2017 08:03:54 -0500 Received: from mx1.redhat.com ([209.132.183.28]:34768 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753918AbdLUNDt (ORCPT ); Thu, 21 Dec 2017 08:03:49 -0500 From: Vitaly Kuznetsov To: Paolo Bonzini Cc: Jim Mattson , kvm list , the arch/x86 maintainers , Radim =?utf-8?B?S3LEjW3DocWZ?= , "K. Y. Srinivasan" , Haiyang Zhang , Stephen Hemminger , "Michael Kelley \(EOSG\)" , Mohammed Gamal , Cathy Avery , Bandan Das , Roman Kagan , LKML , devel@linuxdriverproject.org Subject: Re: [PATCH RFC 2/7] KVM: nVMX: modify vmcs12 fields to match Hyper-V enlightened VMCS References: <20171218171742.5765-1-vkuznets@redhat.com> <20171218171742.5765-3-vkuznets@redhat.com> <87zi6elxaw.fsf@vitty.brq.redhat.com> Date: Thu, 21 Dec 2017 14:02:05 +0100 In-Reply-To: (Paolo Bonzini's message of "Tue, 19 Dec 2017 13:37:49 +0100") Message-ID: <87r2row7xu.fsf@vitty.brq.redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Thu, 21 Dec 2017 13:03:49 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2540 Lines: 57 Paolo Bonzini writes: > On 19/12/2017 13:25, Vitaly Kuznetsov wrote: >> >>> At this point in time, I don't think you can just blithely change the >>> virtual VMCS layout and revision number. Existing VMs using the old >>> layout and revision number must continue to work on versions of kvm >>> past this point. You could tie the layout and revision number changes >>> to KVM_CAP_HYPERV_ENLIGHTENED_VMCS if you like, but kvm must be able >>> to continue to service VMs using the previous layout and revision >>> number in perpetuity. >>> >> I see what you mean. In case we need to keep migration of nested >> workloads working between KVMs of different versions we can't (ever) >> touch vmcs12. > > Actually we can, for two reasons. > > First, the active VMCS is stored in host RAM (not in guest RAM). This > means there are clear points where to do the translation, namely vmptrld > and the (not yet upstream) ioctl to set VMX state. > > Therefore you only need to keep an (offset, type) map from old to new > layout map; at those two points if you detect an old VMCS12_REVISION you > copy the fields one by one instead of doing a memcpy. The next vmclear > or vmptrld or get-VMX-state ioctl will automatically update to the new > VMCS12_REVISION. Of course, this is a one-way street unless you also > add support for writing old VMCS12_REVISIONs. > > But, second, VMX state migration is not upstream yet, so nested > hypervisors are currently not migratable: the active VMCS12 state will > not be migrated at all! So in upstream KVM we wouldn't even need to > upgrade the VMCS12_REVISION to make changes to vmcs12. > > That said... > >> The way to go in this case, I think, is to create a completely separate >> enlightened_vmcs12 struct and use it when appropriate. We can't possibly >> support migrating workloads which use enlightened VMCS to an old KVM >> which doesn't support it. > > ... this is probably a good idea as well. > One other thing I was thinking about is the shared definition of enlightened vmcs which we'll use for both KVM-on-Hyper-V and Hyper-V on KVM and for that purpose I'd like it to be placed outside of struct vmcs12. We can, of course, embed it at the beginning of vmcs12. Thinking long term (and having in mind that Microsoft will be updating enlightened VMCS on its own schedule) -- what would be the preferred way to go? It seems that personally I'm leaning towards untangling and keeping it separate from vmcs12 but I can't really find a convincing argument... -- Vitaly