2014-04-03 15:18:48

by Eric Auger

[permalink] [raw]
Subject: [PATCH] ARM: KVM: Handle IPA unmapping on memory region deletion

Currently when a KVM region is removed using
kvm_vm_ioctl_set_memory_region (with memory region size equal to 0), the
corresponding intermediate physical memory is not unmapped.

This patch unmaps the region's IPA range in
kvm_arch_commit_memory_region using unmap_stage2_range.

The patch was tested on QEMU VFIO based use case where RAM memory region
creation/deletion frequently happens for IRQ handling.

Notes:
- the KVM_MR_MOVE case shall request some similar addition but I cannot test
this currently

Signed-off-by: Eric Auger <[email protected]>
---
arch/arm/include/asm/kvm_mmu.h | 2 ++
arch/arm/kvm/arm.c | 8 ++++++++
arch/arm/kvm/mmu.c | 2 +-
3 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h
index 2d122ad..a91c863 100644
--- a/arch/arm/include/asm/kvm_mmu.h
+++ b/arch/arm/include/asm/kvm_mmu.h
@@ -52,6 +52,8 @@ void kvm_free_stage2_pgd(struct kvm *kvm);
int kvm_phys_addr_ioremap(struct kvm *kvm, phys_addr_t guest_ipa,
phys_addr_t pa, unsigned long size);

+void unmap_stage2_range(struct kvm *kvm, phys_addr_t guest_ipa, u64 size);
+
int kvm_handle_guest_abort(struct kvm_vcpu *vcpu, struct kvm_run *run);

void kvm_mmu_free_memory_caches(struct kvm_vcpu *vcpu);
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index bd18bb8..9a4bc10 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -241,6 +241,14 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
const struct kvm_memory_slot *old,
enum kvm_mr_change change)
{
+ if (change == KVM_MR_DELETE) {
+ gpa_t gpa = old->base_gfn << PAGE_SHIFT;
+ u64 size = old->npages << PAGE_SHIFT;
+
+ spin_lock(&kvm->mmu_lock);
+ unmap_stage2_range(kvm, gpa, size);
+ spin_unlock(&kvm->mmu_lock);
+ }
}

void kvm_arch_flush_shadow_all(struct kvm *kvm)
diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
index 7789857..e8580e2 100644
--- a/arch/arm/kvm/mmu.c
+++ b/arch/arm/kvm/mmu.c
@@ -443,7 +443,7 @@ int kvm_alloc_stage2_pgd(struct kvm *kvm)
* destroying the VM), otherwise another faulting VCPU may come in and mess
* with things behind our backs.
*/
-static void unmap_stage2_range(struct kvm *kvm, phys_addr_t start, u64 size)
+void unmap_stage2_range(struct kvm *kvm, phys_addr_t start, u64 size)
{
unmap_range(kvm, kvm->arch.pgd, start, size);
}
--
1.9.1


2014-04-03 15:26:19

by Marc Zyngier

[permalink] [raw]
Subject: Re: [PATCH] ARM: KVM: Handle IPA unmapping on memory region deletion

Hi Eric,

On 03/04/14 16:17, Eric Auger wrote:
> Currently when a KVM region is removed using
> kvm_vm_ioctl_set_memory_region (with memory region size equal to 0), the
> corresponding intermediate physical memory is not unmapped.
>
> This patch unmaps the region's IPA range in
> kvm_arch_commit_memory_region using unmap_stage2_range.
>
> The patch was tested on QEMU VFIO based use case where RAM memory region
> creation/deletion frequently happens for IRQ handling.
>
> Notes:
> - the KVM_MR_MOVE case shall request some similar addition but I cannot test
> this currently

I think you should try to handle it anyway. I'm sure you could hack QEMU
to do this test it, but even if you don't, better plug that hole right now.

> Signed-off-by: Eric Auger <[email protected]>
> ---
> arch/arm/include/asm/kvm_mmu.h | 2 ++
> arch/arm/kvm/arm.c | 8 ++++++++
> arch/arm/kvm/mmu.c | 2 +-
> 3 files changed, 11 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h
> index 2d122ad..a91c863 100644
> --- a/arch/arm/include/asm/kvm_mmu.h
> +++ b/arch/arm/include/asm/kvm_mmu.h
> @@ -52,6 +52,8 @@ void kvm_free_stage2_pgd(struct kvm *kvm);
> int kvm_phys_addr_ioremap(struct kvm *kvm, phys_addr_t guest_ipa,
> phys_addr_t pa, unsigned long size);
>
> +void unmap_stage2_range(struct kvm *kvm, phys_addr_t guest_ipa, u64 size);
> +
> int kvm_handle_guest_abort(struct kvm_vcpu *vcpu, struct kvm_run *run);
>
> void kvm_mmu_free_memory_caches(struct kvm_vcpu *vcpu);
> diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
> index bd18bb8..9a4bc10 100644
> --- a/arch/arm/kvm/arm.c
> +++ b/arch/arm/kvm/arm.c
> @@ -241,6 +241,14 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
> const struct kvm_memory_slot *old,
> enum kvm_mr_change change)
> {
> + if (change == KVM_MR_DELETE) {
> + gpa_t gpa = old->base_gfn << PAGE_SHIFT;
> + u64 size = old->npages << PAGE_SHIFT;
> +
> + spin_lock(&kvm->mmu_lock);
> + unmap_stage2_range(kvm, gpa, size);
> + spin_unlock(&kvm->mmu_lock);
> + }
> }

Just move the whole function to mmu.c, as it makes more sense to have it
there. And while you're at it, how about moving the other
memslot/memory_region stubs?

> void kvm_arch_flush_shadow_all(struct kvm *kvm)
> diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
> index 7789857..e8580e2 100644
> --- a/arch/arm/kvm/mmu.c
> +++ b/arch/arm/kvm/mmu.c
> @@ -443,7 +443,7 @@ int kvm_alloc_stage2_pgd(struct kvm *kvm)
> * destroying the VM), otherwise another faulting VCPU may come in and mess
> * with things behind our backs.
> */
> -static void unmap_stage2_range(struct kvm *kvm, phys_addr_t start, u64 size)
> +void unmap_stage2_range(struct kvm *kvm, phys_addr_t start, u64 size)
> {
> unmap_range(kvm, kvm->arch.pgd, start, size);
> }
>

Looks sensible otherwise.

Thanks!

M.
--
Jazz is not dead. It just smells funny...

2014-04-03 16:32:26

by Eric Auger

[permalink] [raw]
Subject: Re: [PATCH] ARM: KVM: Handle IPA unmapping on memory region deletion

On 04/03/2014 05:26 PM, Marc Zyngier wrote:
> Hi Eric,
>
> On 03/04/14 16:17, Eric Auger wrote:
>> Currently when a KVM region is removed using
>> kvm_vm_ioctl_set_memory_region (with memory region size equal to 0), the
>> corresponding intermediate physical memory is not unmapped.
>>
>> This patch unmaps the region's IPA range in
>> kvm_arch_commit_memory_region using unmap_stage2_range.
>>
>> The patch was tested on QEMU VFIO based use case where RAM memory region
>> creation/deletion frequently happens for IRQ handling.
>>
>> Notes:
>> - the KVM_MR_MOVE case shall request some similar addition but I cannot test
>> this currently
>
> I think you should try to handle it anyway. I'm sure you could hack QEMU
> to do this test it, but even if you don't, better plug that hole right now.

Hi Marc,

OK I will proceed

>
>> Signed-off-by: Eric Auger <[email protected]>
>> ---
>> arch/arm/include/asm/kvm_mmu.h | 2 ++
>> arch/arm/kvm/arm.c | 8 ++++++++
>> arch/arm/kvm/mmu.c | 2 +-
>> 3 files changed, 11 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h
>> index 2d122ad..a91c863 100644
>> --- a/arch/arm/include/asm/kvm_mmu.h
>> +++ b/arch/arm/include/asm/kvm_mmu.h
>> @@ -52,6 +52,8 @@ void kvm_free_stage2_pgd(struct kvm *kvm);
>> int kvm_phys_addr_ioremap(struct kvm *kvm, phys_addr_t guest_ipa,
>> phys_addr_t pa, unsigned long size);
>>
>> +void unmap_stage2_range(struct kvm *kvm, phys_addr_t guest_ipa, u64 size);
>> +
>> int kvm_handle_guest_abort(struct kvm_vcpu *vcpu, struct kvm_run *run);
>>
>> void kvm_mmu_free_memory_caches(struct kvm_vcpu *vcpu);
>> diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
>> index bd18bb8..9a4bc10 100644
>> --- a/arch/arm/kvm/arm.c
>> +++ b/arch/arm/kvm/arm.c
>> @@ -241,6 +241,14 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
>> const struct kvm_memory_slot *old,
>> enum kvm_mr_change change)
>> {
>> + if (change == KVM_MR_DELETE) {
>> + gpa_t gpa = old->base_gfn << PAGE_SHIFT;
>> + u64 size = old->npages << PAGE_SHIFT;
>> +
>> + spin_lock(&kvm->mmu_lock);
>> + unmap_stage2_range(kvm, gpa, size);
>> + spin_unlock(&kvm->mmu_lock);
>> + }
>> }
>
> Just move the whole function to mmu.c, as it makes more sense to have it
> there. And while you're at it, how about moving the other
> memslot/memory_region stubs?

OK I will move:
kvm_arch_free_memslot,
kvm_arch_create_memslot,
kvm_arch_prepare_memory_region,
kvm_arch_commit_memory_region,
kvm_arch_shadow_all and
kvm_arch_shadow_memslot then.

Best Regards

Eric

>
>> void kvm_arch_flush_shadow_all(struct kvm *kvm)
>> diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
>> index 7789857..e8580e2 100644
>> --- a/arch/arm/kvm/mmu.c
>> +++ b/arch/arm/kvm/mmu.c
>> @@ -443,7 +443,7 @@ int kvm_alloc_stage2_pgd(struct kvm *kvm)
>> * destroying the VM), otherwise another faulting VCPU may come in and mess
>> * with things behind our backs.
>> */
>> -static void unmap_stage2_range(struct kvm *kvm, phys_addr_t start, u64 size)
>> +void unmap_stage2_range(struct kvm *kvm, phys_addr_t start, u64 size)
>> {
>> unmap_range(kvm, kvm->arch.pgd, start, size);
>> }
>>
>
> Looks sensible otherwise.
>
> Thanks!
>
> M.
>

2014-04-03 17:07:35

by Christoffer Dall

[permalink] [raw]
Subject: Re: [PATCH] ARM: KVM: Handle IPA unmapping on memory region deletion

On 3 April 2014 08:26, Marc Zyngier <[email protected]> wrote:
> Hi Eric,
>
> On 03/04/14 16:17, Eric Auger wrote:
>> Currently when a KVM region is removed using
>> kvm_vm_ioctl_set_memory_region (with memory region size equal to 0), the
>> corresponding intermediate physical memory is not unmapped.
>>
>> This patch unmaps the region's IPA range in
>> kvm_arch_commit_memory_region using unmap_stage2_range.
>>
>> The patch was tested on QEMU VFIO based use case where RAM memory region
>> creation/deletion frequently happens for IRQ handling.
>>
>> Notes:
>> - the KVM_MR_MOVE case shall request some similar addition but I cannot test
>> this currently
>
> I think you should try to handle it anyway. I'm sure you could hack QEMU
> to do this test it, but even if you don't, better plug that hole right now.
>
I don't think we want untested code in the kernel if we can avoid it,
so I would say, either test it somehow (kvm-unit-tests ?) or print a
warning...?

-Christoffer