Subject: Re: [PART1 RFC v3 02/12] KVM: x86: Introducing kvm_x86_ops VM
 init/uninit hooks
To: Paolo Bonzini <pbonzini@redhat.com>, <rkrcmar@redhat.com>,
        <joro@8bytes.org>, <bp@alien8.de>, <gleb@kernel.org>,
        <alex.williamson@redhat.com>
References: <1458281388-14452-1-git-send-email-Suravee.Suthikulpanit@amd.com>
 <1458281388-14452-3-git-send-email-Suravee.Suthikulpanit@amd.com>
 <56EBD438.5000702@redhat.com> <56FA1240.2000600@amd.com>
 <56FA5737.2090406@redhat.com> <56FA6B36.4030607@amd.com>
 <56FBA3A3.1060907@amd.com> <56FBC18D.1040505@redhat.com>
CC: <kvm@vger.kernel.org>, <linux-kernel@vger.kernel.org>, <wei@redhat.com>,
        <sherry.hurwitz@amd.com>
From: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
Message-ID: <56FBC405.2000208@amd.com>
Date: Wed, 30 Mar 2016 19:18:14 +0700
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:38.0)
 Gecko/20100101 Thunderbird/38.6.0
MIME-Version: 1.0
In-Reply-To: <56FBC18D.1040505@redhat.com>
Content-Type: text/plain; charset="windows-1252"; format=flowed
Content-Transfer-Encoding: 7bit
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Mar 2016 12:18:33.8825
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY1PR12MB0438
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
Content-Length: 2756
Lines: 87

Hi Paolo,

On 3/30/16 19:07, Paolo Bonzini wrote:
>
>
> On 30/03/2016 12:00, Suravee Suthikulpanit wrote:
>> Hi Paolo,
>>
>> On 3/29/16 18:47, Suravee Suthikulpanit wrote:
>>> Hi,
>>>
>>> On 03/29/2016 05:21 PM, Paolo Bonzini wrote:
>>>>
>>>>
>>>> On 29/03/2016 07:27, Suravee Suthikulpanit wrote:
>>>>>>
>>>>>>>> Adding function pointers in struct kvm_x86_ops for
>>>>>>>> processor-specific
>>>>>>>> layer to provide hooks for when KVM initialize and un-initialize VM.
>>>>>> This is not the only thing your patch is doing, and the "other" change
>>>>>> definitely needs a lot more explanation about why you did it and how
>>>>>> you
>>>>>> audited the code to ensure that it is safe.
>>>>>>
>>>>>> Paolo
>>>>>>
>>>>>
>>>>> Sorry, for not mentioning this earlier. I am moving the
>>>>> kvm_arch_init_vm() call mainly to go after mutex_init(&kvm->slots_lock)
>>>>> since I am calling the x86_set_memory_region() (which uses slots_lock)
>>>>> in the vm_init() hooks (for AVIC initialization).
>>>>>
>>>>> Lemme re-check if this would be safe for other code.
>>>>
>>>> No problem.  In the meanwhile a patch got in ("KVM: fix spin_lock_init
>>>> order on x86") that should help you.
>>>>
>>>> Thanks,
>>>>
>>>> Paolo
>>>>
>>>
>>> Right.... that's just what I need :) I'll re-base to the latest tip then.
>>
>> Actually, in the file virt/kvm/kvm_main.c, I still need to move the
>> kvm_arch_init_vm() to some place after the call to kvm_alloc_memslots()
>> since I am calling x86_set_memory_region() in the vm_init hook.
>>
>>          r = -ENOMEM;
>>          for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++) {
>>                  kvm->memslots[i] = kvm_alloc_memslots();
>>                  if (!kvm->memslots[i])
>>                          goto out_err_no_srcu;
>>          }
>>
>>          if (init_srcu_struct(&kvm->srcu))
>>                  goto out_err_no_srcu;
>>          if (init_srcu_struct(&kvm->irq_srcu))
>>                  goto out_err_no_irq_srcu;
>>          for (i = 0; i < KVM_NR_BUSES; i++) {
>>                  kvm->buses[i] = kzalloc(sizeof(struct kvm_io_bus),
>>                                          GFP_KERNEL);
>>                  if (!kvm->buses[i])
>>                          goto out_err;
>>          }
>> //HERE
>>          r = kvm_arch_init_vm(kvm, type);
>>          if (r)
>>                  goto out_err;
>>
>> Do you think that would be a problem?
>
> Can you delay that to after the creation of the first VCPU?

Sure, I can. That's what I was doing originally before we introduce the 
vm_init hook. I just thought that this would make a nice place.

> Allocating AVIC data structures is not required if userspace has not
> called KVM_CREATE_IRQCHIP or enabled KVM_CAP_SPLIT_IRQCHIP.
>
> Paolo
>

Okay.

Thanks,
Suravee