2022-02-25 10:16:36

by Zeng Guang

Subject: [PATCH v6 8/9] KVM: x86: Allow userspace set maximum VCPU id for VM

Introduce new max_vcpu_id in KVM for x86 architecture. Userspace
can assign maximum possible vcpu id for current VM session using
KVM_CAP_MAX_VCPU_ID of KVM_ENABLE_CAP ioctl().

This is done for x86 only because the sole use case is to guide
memory allocation for PID-pointer table, a structure needed to
enable VMX IPI.

By default, max_vcpu_id set as KVM_MAX_VCPU_IDS.

Suggested-by: Sean Christopherson <[email protected]>
Signed-off-by: Zeng Guang <[email protected]>
---
No new KVM capability is added to advertise the support of
configurable maximum vCPU ID to user space because max_vcpu_id is
just a hint/commitment to allow KVM to reduce the size of PID-pointer
table. But I am not 100% sure if it is proper to do so.

arch/x86/include/asm/kvm_host.h | 6 ++++++
arch/x86/kvm/x86.c | 11 +++++++++++
2 files changed, 17 insertions(+)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 6dcccb304775..db16aebd946c 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1233,6 +1233,12 @@ struct kvm_arch {
hpa_t hv_root_tdp;
spinlock_t hv_root_tdp_lock;
#endif
+ /*
+ * VM-scope maximum vCPU ID. Used to determine the size of structures
+ * that increase along with the maximum vCPU ID, in which case, using
+ * the global KVM_MAX_VCPU_IDS may lead to significant memory waste.
+ */
+ u32 max_vcpu_id;
};

struct kvm_vm_stat {
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 4f6fe9974cb5..ca17cc452bd3 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -5994,6 +5994,13 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
kvm->arch.exit_on_emulation_error = cap->args[0];
r = 0;
break;
+ case KVM_CAP_MAX_VCPU_ID:
+ if (cap->args[0] <= KVM_MAX_VCPU_IDS) {
+ kvm->arch.max_vcpu_id = cap->args[0];
+ r = 0;
+ } else
+ r = -E2BIG;
+ break;
default:
r = -EINVAL;
break;
@@ -11067,6 +11074,9 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
struct page *page;
int r;

+ if (vcpu->vcpu_id >= vcpu->kvm->arch.max_vcpu_id)
+ return -E2BIG;
+
vcpu->arch.last_vmentry_cpu = -1;
vcpu->arch.regs_avail = ~0;
vcpu->arch.regs_dirty = ~0;
@@ -11589,6 +11599,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
spin_lock_init(&kvm->arch.hv_root_tdp_lock);
kvm->arch.hv_root_tdp = INVALID_PAGE;
#endif
+ kvm->arch.max_vcpu_id = KVM_MAX_VCPU_IDS;

INIT_DELAYED_WORK(&kvm->arch.kvmclock_update_work, kvmclock_update_fn);
INIT_DELAYED_WORK(&kvm->arch.kvmclock_sync_work, kvmclock_sync_fn);
--
2.27.0

2022-02-26 02:28:21

by Maxim Levitsky

Subject: Re: [PATCH v6 8/9] KVM: x86: Allow userspace set maximum VCPU id for VM

On Fri, 2022-02-25 at 16:22 +0800, Zeng Guang wrote:
> Introduce new max_vcpu_id in KVM for x86 architecture. Userspace
> can assign maximum possible vcpu id for current VM session using
> KVM_CAP_MAX_VCPU_ID of KVM_ENABLE_CAP ioctl().
>
> This is done for x86 only because the sole use case is to guide
> memory allocation for PID-pointer table, a structure needed to
> enable VMX IPI.
>
> By default, max_vcpu_id set as KVM_MAX_VCPU_IDS.
>
> Suggested-by: Sean Christopherson <[email protected]>
> Signed-off-by: Zeng Guang <[email protected]>
> ---
> No new KVM capability is added to advertise the support of
> configurable maximum vCPU ID to user space because max_vcpu_id is
> just a hint/commitment to allow KVM to reduce the size of PID-pointer
> table. But I am not 100% sure if it is proper to do so.
>
> arch/x86/include/asm/kvm_host.h | 6 ++++++
> arch/x86/kvm/x86.c | 11 +++++++++++
> 2 files changed, 17 insertions(+)
>
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index 6dcccb304775..db16aebd946c 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -1233,6 +1233,12 @@ struct kvm_arch {
> hpa_t hv_root_tdp;
> spinlock_t hv_root_tdp_lock;
> #endif
> + /*
> + * VM-scope maximum vCPU ID. Used to determine the size of structures
> + * that increase along with the maximum vCPU ID, in which case, using
> + * the global KVM_MAX_VCPU_IDS may lead to significant memory waste.
> + */
> + u32 max_vcpu_id;
> };
>
> struct kvm_vm_stat {
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 4f6fe9974cb5..ca17cc452bd3 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -5994,6 +5994,13 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
> kvm->arch.exit_on_emulation_error = cap->args[0];
> r = 0;
> break;
> + case KVM_CAP_MAX_VCPU_ID:
> + if (cap->args[0] <= KVM_MAX_VCPU_IDS) {
> + kvm->arch.max_vcpu_id = cap->args[0];
> + r = 0;
> + } else
> + r = -E2BIG;
> + break;
> default:
> r = -EINVAL;
> break;
> @@ -11067,6 +11074,9 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
> struct page *page;
> int r;
>
> + if (vcpu->vcpu_id >= vcpu->kvm->arch.max_vcpu_id)
> + return -E2BIG;
> +
> vcpu->arch.last_vmentry_cpu = -1;
> vcpu->arch.regs_avail = ~0;
> vcpu->arch.regs_dirty = ~0;
> @@ -11589,6 +11599,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
> spin_lock_init(&kvm->arch.hv_root_tdp_lock);
> kvm->arch.hv_root_tdp = INVALID_PAGE;
> #endif
> + kvm->arch.max_vcpu_id = KVM_MAX_VCPU_IDS;
>
> INIT_DELAYED_WORK(&kvm->arch.kvmclock_update_work, kvmclock_update_fn);
> INIT_DELAYED_WORK(&kvm->arch.kvmclock_sync_work, kvmclock_sync_fn);

Reviewed-by: Maxim Levitsky <[email protected]>

+ Reminder for myself to use this in AVIC code to as well to limit the size of the physid table.
(The max size of the table (for now...) is 1 page (or 1/2 of page for non x2avic),
but still no doubt it takes some effort for microcode to scan all the unused entries in it for nothing
(and soon my nested avic code will have to scan it as well).

Best regards,
Maxim Levitsky