Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 20 Apr 2022 11:22:51 +0800
From:   Chao Gao <chao.gao@intel.com>
To:     Zeng Guang <guang.zeng@intel.com>
Cc:     Paolo Bonzini <pbonzini@redhat.com>,
        Sean Christopherson <seanjc@google.com>,
        Vitaly Kuznetsov <vkuznets@redhat.com>,
        Wanpeng Li <wanpengli@tencent.com>,
        Jim Mattson <jmattson@google.com>,
        Joerg Roedel <joro@8bytes.org>, kvm@vger.kernel.org,
        Dave Hansen <dave.hansen@linux.intel.com>,
        Tony Luck <tony.luck@intel.com>,
        Kan Liang <kan.liang@linux.intel.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        "H. Peter Anvin" <hpa@zytor.com>,
        Kim Phillips <kim.phillips@amd.com>,
        Jarkko Sakkinen <jarkko@kernel.org>,
        Jethro Beekman <jethro@fortanix.com>,
        Kai Huang <kai.huang@intel.com>, x86@kernel.org,
        linux-kernel@vger.kernel.org, Robert Hu <robert.hu@intel.com>
Subject: Re: [PATCH v9 8/9] KVM: x86: Allow userspace set maximum VCPU id for
 VM
Message-ID: <20220420032245.GA14083@gao-cwp>
References: <20220419154444.11888-1-guang.zeng@intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20220419154444.11888-1-guang.zeng@intel.com>
User-Agent: Mutt/1.9.4 (2018-02-28)
Precedence: bulk

On Tue, Apr 19, 2022 at 11:44:44PM +0800, Zeng Guang wrote:
>Introduce new max_vcpu_ids in KVM for x86 architecture. Userspace
>can assign maximum possible vcpu id for current VM session using
>KVM_CAP_MAX_VCPU_ID of KVM_ENABLE_CAP ioctl().
>
>This is done for x86 only because the sole use case is to guide
>memory allocation for PID-pointer table, a structure needed to
>enable VMX IPI.
>
>By default, max_vcpu_ids set as KVM_MAX_VCPU_IDS.
>
>Suggested-by: Sean Christopherson <seanjc@google.com>
>Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
>Signed-off-by: Zeng Guang <guang.zeng@intel.com>
>---
> Documentation/virt/kvm/api.rst  | 18 ++++++++++++++++++
> arch/x86/include/asm/kvm_host.h |  6 ++++++
> arch/x86/kvm/x86.c              | 25 ++++++++++++++++++++++++-
> 3 files changed, 48 insertions(+), 1 deletion(-)
>
>diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
>index d13fa6600467..0c6ad2d8bea0 100644
>--- a/Documentation/virt/kvm/api.rst
>+++ b/Documentation/virt/kvm/api.rst
>@@ -7136,6 +7136,24 @@ The valid bits in cap.args[0] are:
>                                     IA32_MISC_ENABLE[bit 18] is cleared.
> =================================== ============================================
> 
>+7.32 KVM_CAP_MAX_VCPU_ID
>+------------------------
>+
>+:Architectures: x86
>+:Target: VM
>+:Parameters: args[0] - maximum APIC ID value set for current VM
>+:Returns: 0 on success, -EINVAL if args[0] is beyond KVM_MAX_VCPU_IDS
>+          supported in KVM or if it has been settled.
>+
>+Userspace is able to calculate the limit to APIC ID values from designated CPU
>+topology. This capability allows userspace to specify maximum possible APIC ID
>+assigned for current VM session prior to the creation of vCPUs. By design, it
>+can set only once and doesn't accept change any more. KVM will manage memory
>+allocation of VM-scope structures which depends on the value of APIC ID.
>+
>+Calling KVM_CHECK_EXTENSION for this capability returns the value of maximum APIC
>+ID that KVM supports at runtime. It sets as KVM_MAX_VCPU_IDS by default.
>+
> 8. Other capabilities.
> ======================
> 
>diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
>index d23e80a56eb8..cdd14033988d 100644
>--- a/arch/x86/include/asm/kvm_host.h
>+++ b/arch/x86/include/asm/kvm_host.h
>@@ -1238,6 +1238,12 @@ struct kvm_arch {
> 	hpa_t	hv_root_tdp;
> 	spinlock_t hv_root_tdp_lock;
> #endif
>+	/*
>+	 * VM-scope maximum vCPU ID. Used to determine the size of structures
>+	 * that increase along with the maximum vCPU ID, in which case, using
>+	 * the global KVM_MAX_VCPU_IDS may lead to significant memory waste.
>+	 */
>+	u32 max_vcpu_ids;
> };
> 
> struct kvm_vm_stat {
>diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>index 277a0da8c290..744e88a71b63 100644
>--- a/arch/x86/kvm/x86.c
>+++ b/arch/x86/kvm/x86.c
>@@ -4320,7 +4320,10 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
> 		r = KVM_MAX_VCPUS;
> 		break;
> 	case KVM_CAP_MAX_VCPU_ID:
>-		r = KVM_MAX_VCPU_IDS;
>+		if (!kvm->arch.max_vcpu_ids)
>+			r = KVM_MAX_VCPU_IDS;
>+		else
>+			r = kvm->arch.max_vcpu_ids;
> 		break;
> 	case KVM_CAP_PV_MMU:	/* obsolete */
> 		r = 0;
>@@ -6064,6 +6067,20 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
> 		}
> 		mutex_unlock(&kvm->lock);
> 		break;
>+	case KVM_CAP_MAX_VCPU_ID:
>+		r = -EINVAL;
>+		if (cap->args[0] > KVM_MAX_VCPU_IDS)
>+			break;
>+
>+		mutex_lock(&kvm->lock);
>+		if (kvm->arch.max_vcpu_ids == cap->args[0]) {
>+			r = 0;
>+		} else if (!kvm->arch.max_vcpu_ids) {
>+			kvm->arch.max_vcpu_ids = cap->args[0];
>+			r = 0;
>+		}
>+		mutex_unlock(&kvm->lock);
>+		break;

It would be better to have a kselftest to exercise this capability.
For example,
1. launch a VM.
2. set the max vCPU ID via KVM_CAP_MAX_VCPU_ID
3. read the max vCPU ID to check if the value written is returned.
4. create a vCPU which has apic id larger than the maximum.
5. try to change the max vCPU ID after set once.
...

This test can be the last patch of this series or posted separately.