Received-SPF: pass (google.com: domain of linux-kernel+bounces-193275-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Date: Tue, 28 May 2024 17:55:19 -0700
From: Isaku Yamahata <isaku.yamahata@intel.com>
To: Chen Yu <yu.c.chen@intel.com>
Cc: Chao Gao <chao.gao@intel.com>, isaku.yamahata@intel.com,
	kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
	isaku.yamahata@gmail.com, Paolo Bonzini <pbonzini@redhat.com>,
	erdemaktas@google.com, Sean Christopherson <seanjc@google.com>,
	Sagi Shahar <sagis@google.com>, Kai Huang <kai.huang@intel.com>,
	chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com,
	Fengwei Yin <fengwei.yin@intel.com>, isaku.yamahata@linux.intel.com
Subject: Re: [PATCH v19 070/130] KVM: TDX: TDP MMU TDX support
Message-ID: <20240529005519.GA386318@ls.amr.corp.intel.com>
References: <cover.1708933498.git.isaku.yamahata@intel.com>
 <56cdb0da8bbf17dc293a2a6b4ff74f6e3e034bbd.1708933498.git.isaku.yamahata@intel.com>
 <ZgTgOVNmyVVgfvFM@chao-email>
 <ZlL2m3PKnYqc3uHr@chenyu5-mobl2>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <ZlL2m3PKnYqc3uHr@chenyu5-mobl2>

On Sun, May 26, 2024 at 04:45:15PM +0800,
Chen Yu <yu.c.chen@intel.com> wrote:

> On 2024-03-28 at 11:12:57 +0800, Chao Gao wrote:
> > >+#if IS_ENABLED(CONFIG_HYPERV)
> > >+static int vt_flush_remote_tlbs(struct kvm *kvm);
> > >+#endif
> > >+
> > > static __init int vt_hardware_setup(void)
> > > {
> > > 	int ret;
> > >@@ -49,11 +53,29 @@ static __init int vt_hardware_setup(void)
> > > 		pr_warn_ratelimited("TDX requires mmio caching.  Please enable mmio caching for TDX.\n");
> > > 	}
> > > 
> > >+#if IS_ENABLED(CONFIG_HYPERV)
> > >+	/*
> > >+	 * TDX KVM overrides flush_remote_tlbs method and assumes
> > >+	 * flush_remote_tlbs_range = NULL that falls back to
> > >+	 * flush_remote_tlbs.  Disable TDX if there are conflicts.
> > >+	 */
> > >+	if (vt_x86_ops.flush_remote_tlbs ||
> > >+	    vt_x86_ops.flush_remote_tlbs_range) {
> > >+		enable_tdx = false;
> > >+		pr_warn_ratelimited("TDX requires baremetal. Not Supported on VMM guest.\n");
> > >+	}
> > >+#endif
> > >+
> > > 	enable_tdx = enable_tdx && !tdx_hardware_setup(&vt_x86_ops);
> > > 	if (enable_tdx)
> > > 		vt_x86_ops.vm_size = max_t(unsigned int, vt_x86_ops.vm_size,
> > > 					   sizeof(struct kvm_tdx));
> > > 
> > >+#if IS_ENABLED(CONFIG_HYPERV)
> > >+	if (enable_tdx)
> > >+		vt_x86_ops.flush_remote_tlbs = vt_flush_remote_tlbs;
> > 
> > Is this hook necessary/beneficial to TDX?
> >
> 
> I think so.
> 
> We happended to encounter the following error and breaks the boot up:
> "SEAMCALL (0x000000000000000f) failed: 0xc0000b0800000001"
> 0xc0000b0800000001 indicates the TDX_TLB_TRACKING_NOT_DONE, and it is caused
> by page demotion but not yet doing a tlb shotdown by tlb track.
> 
> 
> It was found on my system the CONFIG_HYPERV is not set, and it makes
> kvm_arch_flush_remote_tlbs() not invoking tdx_track() before the
> tdh_mem_page_demote(), which caused the problem.
> 
> > if no, we can leave .flush_remote_tlbs as NULL. if yes, we should do:
> > 
> > struct kvm_x86_ops {
> > ...
> > #if IS_ENABLED(CONFIG_HYPERV) || IS_ENABLED(TDX...)
> > 	int  (*flush_remote_tlbs)(struct kvm *kvm);
> > 	int  (*flush_remote_tlbs_range)(struct kvm *kvm, gfn_t gfn,
> > 					gfn_t nr_pages);
> > #endif
> 
> If the flush_remote_tlbs implementation are both available in HYPERV and TDX,
> does it make sense to remove the config checks? I thought when commit 0277022a77a5
> was introduced, the only user of flush_remote_tlbs() is hyperv, and now
> there is TDX.

You don't like IS_ENABLED(CONFIG_HYPERV) || IS_ENABLED(CONFIG_TDX_HOST) in many
places?  Then, we can do something like the followings.  Although It would be
a bit ugly than the commit of 0277022a77a5, it's better to keep the intention
of it.

#if IS_ENABLED(CONFIG_HYPERV) || IS_ENABLED(CONFIG_TDX_HOST)
# define KVM_X86_WANT_FLUSH_REMOTE_TLBS
#endif

#if IS_DEFINED(KVM_X86_WANT_FLUSH_REMOTE_TLBS)
..
#endif

-- 
Isaku Yamahata <isaku.yamahata@intel.com>