Subject: Re: [PART1 RFC v3 02/12] KVM: x86: Introducing kvm_x86_ops VM
 init/uninit hooks
To: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>, rkrcmar@redhat.com,
        joro@8bytes.org, bp@alien8.de, gleb@kernel.org,
        alex.williamson@redhat.com
References: <1458281388-14452-1-git-send-email-Suravee.Suthikulpanit@amd.com>
 <1458281388-14452-3-git-send-email-Suravee.Suthikulpanit@amd.com>
 <56EBD438.5000702@redhat.com> <56FA1240.2000600@amd.com>
 <56FA5737.2090406@redhat.com> <56FA6B36.4030607@amd.com>
 <56FBA3A3.1060907@amd.com>
Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, wei@redhat.com,
        sherry.hurwitz@amd.com
From: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <56FBC18D.1040505@redhat.com>
Date: Wed, 30 Mar 2016 14:07:41 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
 Thunderbird/38.6.0
MIME-Version: 1.0
In-Reply-To: <56FBA3A3.1060907@amd.com>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 8bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2449
Lines: 75


On 30/03/2016 12:00, Suravee Suthikulpanit wrote:
> Hi Paolo,
> 
> On 3/29/16 18:47, Suravee Suthikulpanit wrote:
>> Hi,
>>
>> On 03/29/2016 05:21 PM, Paolo Bonzini wrote:
>>>
>>>
>>> On 29/03/2016 07:27, Suravee Suthikulpanit wrote:
>>>>>
>>>>>>> Adding function pointers in struct kvm_x86_ops for
>>>>>>> processor-specific
>>>>>>> layer to provide hooks for when KVM initialize and un-initialize VM.
>>>>> This is not the only thing your patch is doing, and the "other" change
>>>>> definitely needs a lot more explanation about why you did it and how
>>>>> you
>>>>> audited the code to ensure that it is safe.
>>>>>
>>>>> Paolo
>>>>>
>>>>
>>>> Sorry, for not mentioning this earlier. I am moving the
>>>> kvm_arch_init_vm() call mainly to go after mutex_init(&kvm->slots_lock)
>>>> since I am calling the x86_set_memory_region() (which uses slots_lock)
>>>> in the vm_init() hooks (for AVIC initialization).
>>>>
>>>> Lemme re-check if this would be safe for other code.
>>>
>>> No problem.  In the meanwhile a patch got in ("KVM: fix spin_lock_init
>>> order on x86") that should help you.
>>>
>>> Thanks,
>>>
>>> Paolo
>>>
>>
>> Right.... that's just what I need :) I'll re-base to the latest tip then.
> 
> Actually, in the file virt/kvm/kvm_main.c, I still need to move the
> kvm_arch_init_vm() to some place after the call to kvm_alloc_memslots()
> since I am calling x86_set_memory_region() in the vm_init hook.
> 
>         r = -ENOMEM;
>         for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++) {
>                 kvm->memslots[i] = kvm_alloc_memslots();
>                 if (!kvm->memslots[i])
>                         goto out_err_no_srcu;
>         }
> 
>         if (init_srcu_struct(&kvm->srcu))
>                 goto out_err_no_srcu;
>         if (init_srcu_struct(&kvm->irq_srcu))
>                 goto out_err_no_irq_srcu;
>         for (i = 0; i < KVM_NR_BUSES; i++) {
>                 kvm->buses[i] = kzalloc(sizeof(struct kvm_io_bus),
>                                         GFP_KERNEL);
>                 if (!kvm->buses[i])
>                         goto out_err;
>         }
> //HERE
>         r = kvm_arch_init_vm(kvm, type);
>         if (r)
>                 goto out_err;
> 
> Do you think that would be a problem?

Can you delay that to after the creation of the first VCPU?

Allocating AVIC data structures is not required if userspace has not
called KVM_CREATE_IRQCHIP or enabled KVM_CAP_SPLIT_IRQCHIP.

Paolo