Received-SPF: pass (google.com: domain of linux-kernel+bounces-104988-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Date: Fri, 15 Mar 2024 16:25:55 -0700
From: Isaku Yamahata <isaku.yamahata@intel.com>
To: Binbin Wu <binbin.wu@linux.intel.com>
Cc: Isaku Yamahata <isaku.yamahata@intel.com>, kvm@vger.kernel.org,
	linux-kernel@vger.kernel.org, isaku.yamahata@gmail.com,
	Paolo Bonzini <pbonzini@redhat.com>, erdemaktas@google.com,
	Sean Christopherson <seanjc@google.com>,
	Sagi Shahar <sagis@google.com>, Kai Huang <kai.huang@intel.com>,
	chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com,
	isaku.yamahata@linux.intel.com
Subject: Re: [PATCH v19 023/130] KVM: TDX: Initialize the TDX module when
 loading the KVM intel kernel module
Message-ID: <20240315232555.GK1258280@ls.amr.corp.intel.com>
References: <cover.1708933498.git.isaku.yamahata@intel.com>
 <f028d43abeadaa3134297d28fb99f283445c0333.1708933498.git.isaku.yamahata@intel.com>
 <f5da22e3-55fd-4e8b-8112-ccf1468012c8@linux.intel.com>
 <20240314162712.GO935089@ls.amr.corp.intel.com>
 <5470a429-cbbd-4946-b11a-ab86380d9b68@linux.intel.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <5470a429-cbbd-4946-b11a-ab86380d9b68@linux.intel.com>

On Fri, Mar 15, 2024 at 12:44:46PM +0800,
Binbin Wu <binbin.wu@linux.intel.com> wrote:

> On 3/15/2024 12:27 AM, Isaku Yamahata wrote:
> > On Thu, Mar 14, 2024 at 10:05:35AM +0800,
> > Binbin Wu <binbin.wu@linux.intel.com> wrote:
> > 
> > > > diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c
> > > > index 18cecf12c7c8..18aef6e23aab 100644
> > > > --- a/arch/x86/kvm/vmx/main.c
> > > > +++ b/arch/x86/kvm/vmx/main.c
> > > > @@ -6,6 +6,22 @@
> > > >    #include "nested.h"
> > > >    #include "pmu.h"
> > > > +static bool enable_tdx __ro_after_init;
> > > > +module_param_named(tdx, enable_tdx, bool, 0444);
> > > > +
> > > > +static __init int vt_hardware_setup(void)
> > > > +{
> > > > +	int ret;
> > > > +
> > > > +	ret = vmx_hardware_setup();
> > > > +	if (ret)
> > > > +		return ret;
> > > > +
> > > > +	enable_tdx = enable_tdx && !tdx_hardware_setup(&vt_x86_ops);
> > > > +
> > > > +	return 0;
> > > > +}
> > > > +
> > > >    #define VMX_REQUIRED_APICV_INHIBITS				\
> > > >    	(BIT(APICV_INHIBIT_REASON_DISABLE)|			\
> > > >    	 BIT(APICV_INHIBIT_REASON_ABSENT) |			\
> > > > @@ -22,6 +38,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
> > > >    	.hardware_unsetup = vmx_hardware_unsetup,
> > > > +	/* TDX cpu enablement is done by tdx_hardware_setup(). */
> > > How about if there are some LPs that are offline.
> > > In tdx_hardware_setup(), only online LPs are initialed for TDX, right?
> > Correct.
> > 
> > 
> > > Then when an offline LP becoming online, it doesn't have a chance to call
> > > tdx_cpu_enable()?
> > KVM registers kvm_online/offline_cpu() @ kvm_main.c as cpu hotplug callbacks.
> > Eventually x86 kvm hardware_enable() is called on online/offline event.
> 
> Yes, hardware_enable() will be called when online,
> but  hardware_enable() now is vmx_hardware_enable() right?
> It doens't call tdx_cpu_enable() during the online path.

TDX module requires TDH.SYS.LP.INIT() on all logical processors(LPs).  If we
successfully initialized TDX module, we don't need further action for TDX on cpu
online/offline.

If some of LPs are not online when loading kvm_intel.ko, KVM fails to initialize
TDX module. TDX support is disabled.  We don't bother to attempt it.  Leave it
to the admin of the machine.
-- 
Isaku Yamahata <isaku.yamahata@intel.com>