Received-SPF: pass (google.com: domain of linux-kernel+bounces-110529-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Date: Thu, 21 Mar 2024 10:59:34 -0700
From: Isaku Yamahata <isaku.yamahata@intel.com>
To: Chao Gao <chao.gao@intel.com>
Cc: isaku.yamahata@intel.com, kvm@vger.kernel.org,
	linux-kernel@vger.kernel.org, isaku.yamahata@gmail.com,
	Paolo Bonzini <pbonzini@redhat.com>, erdemaktas@google.com,
	Sean Christopherson <seanjc@google.com>,
	Sagi Shahar <sagis@google.com>, Kai Huang <kai.huang@intel.com>,
	chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com,
	Binbin Wu <binbin.wu@linux.intel.com>,
	isaku.yamahata@linux.intel.com
Subject: Re: [PATCH v19 041/130] KVM: TDX: Refuse to unplug the last cpu on
 the package
Message-ID: <20240321175934.GO1994522@ls.amr.corp.intel.com>
References: <cover.1708933498.git.isaku.yamahata@intel.com>
 <15d4d08cf1fe16777f8f8d84ee940e41afbad406.1708933498.git.isaku.yamahata@intel.com>
 <ZfuIJpHzp4sEjCi/@chao-email>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <ZfuIJpHzp4sEjCi/@chao-email>

On Thu, Mar 21, 2024 at 09:06:46AM +0800,
Chao Gao <chao.gao@intel.com> wrote:

> >diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c
> >index 437c6d5e802e..d69dd474775b 100644
> >--- a/arch/x86/kvm/vmx/main.c
> >+++ b/arch/x86/kvm/vmx/main.c
> >@@ -110,6 +110,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
> > 	.check_processor_compatibility = vmx_check_processor_compat,
> > 
> > 	.hardware_unsetup = vt_hardware_unsetup,
> >+	.offline_cpu = tdx_offline_cpu,
> > 
> > 	/* TDX cpu enablement is done by tdx_hardware_setup(). */
> > 	.hardware_enable = vmx_hardware_enable,
> >diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
> >index b11f105db3cd..f2ee5abac14e 100644
> >--- a/arch/x86/kvm/vmx/tdx.c
> >+++ b/arch/x86/kvm/vmx/tdx.c
> >@@ -97,6 +97,7 @@ int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
> >  */
> > static DEFINE_MUTEX(tdx_lock);
> > static struct mutex *tdx_mng_key_config_lock;
> >+static atomic_t nr_configured_hkid;
> > 
> > static __always_inline hpa_t set_hkid_to_hpa(hpa_t pa, u16 hkid)
> > {
> >@@ -112,6 +113,7 @@ static inline void tdx_hkid_free(struct kvm_tdx *kvm_tdx)
> > {
> > 	tdx_guest_keyid_free(kvm_tdx->hkid);
> > 	kvm_tdx->hkid = -1;
> >+	atomic_dec(&nr_configured_hkid);
> 
> I may think it is better to extend IDA infrastructure e.g., add an API to check if
> any ID is allocated for a given range. No strong opinion on this.

Will use ida_is_empyt().


> > }
> > 
> > static inline bool is_hkid_assigned(struct kvm_tdx *kvm_tdx)
> >@@ -586,6 +588,7 @@ static int __tdx_td_init(struct kvm *kvm, struct td_params *td_params,
> > 	if (ret < 0)
> > 		return ret;
> > 	kvm_tdx->hkid = ret;
> >+	atomic_inc(&nr_configured_hkid);
> > 
> > 	va = __get_free_page(GFP_KERNEL_ACCOUNT);
> > 	if (!va)
> >@@ -1071,3 +1074,41 @@ void tdx_hardware_unsetup(void)
> > 	kfree(tdx_info);
> > 	kfree(tdx_mng_key_config_lock);
> > }
> >+
> >+int tdx_offline_cpu(void)
> >+{
> >+	int curr_cpu = smp_processor_id();
> >+	cpumask_var_t packages;
> >+	int ret = 0;
> >+	int i;
> >+
> >+	/* No TD is running.  Allow any cpu to be offline. */
> >+	if (!atomic_read(&nr_configured_hkid))
> >+		return 0;
> >+
> >+	/*
> >+	 * In order to reclaim TDX HKID, (i.e. when deleting guest TD), need to
> >+	 * call TDH.PHYMEM.PAGE.WBINVD on all packages to program all memory
> >+	 * controller with pconfig.  If we have active TDX HKID, refuse to
> >+	 * offline the last online cpu.
> >+	 */
> >+	if (!zalloc_cpumask_var(&packages, GFP_KERNEL))
> >+		return -ENOMEM;
> >+	for_each_online_cpu(i) {
> >+		if (i != curr_cpu)
> >+			cpumask_set_cpu(topology_physical_package_id(i), packages);
> >+	}
> 
> Just check if any other CPU is in the same package of the one about to go
> offline. This would obviate the need for the cpumask and allow us to break once
> one cpu in the same package is found.

Good idea. Will rewrite it so.
-- 
Isaku Yamahata <isaku.yamahata@intel.com>