Received-SPF: pass (google.com: domain of linux-kernel+bounces-189611-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Date: Sun, 26 May 2024 16:45:15 +0800
From: Chen Yu <yu.c.chen@intel.com>
To: Chao Gao <chao.gao@intel.com>
CC: <isaku.yamahata@intel.com>, <kvm@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>, <isaku.yamahata@gmail.com>, Paolo Bonzini
	<pbonzini@redhat.com>, <erdemaktas@google.com>, Sean Christopherson
	<seanjc@google.com>, Sagi Shahar <sagis@google.com>, Kai Huang
	<kai.huang@intel.com>, <chen.bo@intel.com>, <hang.yuan@intel.com>,
	<tina.zhang@intel.com>, Fengwei Yin <fengwei.yin@intel.com>
Subject: Re: [PATCH v19 070/130] KVM: TDX: TDP MMU TDX support
Message-ID: <ZlL2m3PKnYqc3uHr@chenyu5-mobl2>
References: <cover.1708933498.git.isaku.yamahata@intel.com>
 <56cdb0da8bbf17dc293a2a6b4ff74f6e3e034bbd.1708933498.git.isaku.yamahata@intel.com>
 <ZgTgOVNmyVVgfvFM@chao-email>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <ZgTgOVNmyVVgfvFM@chao-email>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?1TXViiINB49payY3Ypvvtc3u2OxSHvl3uG/Qq40gSX/h6WrM5RzRV3uZ2FoX?=
 =?us-ascii?Q?CQ0mQERTAGBEGLcq5JZxla8C3EtIdmC8jOVJ1OwD58LAsqYIkZADbLRmMR0K?=
 =?us-ascii?Q?fIV0JpmiCGn3TiZOfJg+cEa64WI02mbby6mqQgaBDQxxuzgNd9gic/rgoP+m?=
 =?us-ascii?Q?c5aYVx0SlMlC6OVFIaXhfrYPg2pNX05L8c8xb//NGa+wLT+8o9dvQVP7CKKs?=
 =?us-ascii?Q?vQmsa5df6O96IaTWJN3aGaDqCiMOWX7LRQi+19b3ANRK+U+GCJJ+ROAlDeGR?=
 =?us-ascii?Q?uKf5LaRihJO/OtYhGxSCLYX8ksbkEVbVTUZYEW00MFKL2QAxMYPEVVIPDtHg?=
 =?us-ascii?Q?0SZRJHUp3itym+MGCRCcb/wtYOkgUkVThZgDb/dbi47Jx3Ln2OlHsbBACP5V?=
 =?us-ascii?Q?bFoF8fdx0Z370vHmxydFm5D69rcmX7FiGuhTU8kxhRhy+fQGNADn4K1tp4Ve?=
 =?us-ascii?Q?maxBOPaugt+2ov/rGdmkhiNckSDveuJAl6qBfn/2Ah43DV5qBOTpNpEpWmM3?=
 =?us-ascii?Q?G301ytuS8sgZgJjHqIxOrqvONPQcjiGE5WWrswh4MvCFCu/WQ5th5950x1Ly?=
 =?us-ascii?Q?DyrvmwRgs1fjq+59Z2gDcgByZmnWsu0ulsuWaS5dqrZL2BVsIJYE4VWGZPvO?=
 =?us-ascii?Q?j8heALugUaXsmUc/GkO46sIzSOU/z14bzehaRp3geIp4FhQDhsq/augvVzRT?=
 =?us-ascii?Q?Nx5Sqr+W1ZneqFqOS2yH3PEnPSdJvg/D01oLX3gysYEI/P58Qky/VIzHxoA7?=
 =?us-ascii?Q?Pi5jJo5DBkSBxqZ+kH3M0BxckccMugQuMDhZwwRegpCF3s9oc/VPbd+Clw0K?=
 =?us-ascii?Q?p9HUVWSCIcDEBcPCzU4pBcD8yQGZbwyiKvuL3plOH0xfLE45hbSxpeZ7sdOL?=
 =?us-ascii?Q?jCcIaG02qCKdV3JJ0maYfYW6p1fXEI3gs30hQBE8APcBeFL0tOzTSQuFhPqB?=
 =?us-ascii?Q?RyOZp1mIB/ZAaO0mhzwpllRKitMOk5DM0GbdAPijJ4azRRCBnfX6W/9dP8hP?=
 =?us-ascii?Q?272a/m7tWRaMYWqNOwxUz7D0N0QMrIUddhg1nq1DNYNFMHunVoNLuO/TZaTC?=
 =?us-ascii?Q?OM1dbM+tnF12OtcRaEE5NeI0DKusl8u/Z8cmsGkxfQnhhpX+Q/jiCl1Qnpuf?=
 =?us-ascii?Q?HHrjOIel1HcSDcJPWvPN+SDVK9kCl/W6jSIrDJpeeTh/EWLqKQpawg25cTod?=
 =?us-ascii?Q?iFc/IFOhFgLEDXSgn5WPMT5TCXsnYnNSJoAm2gmOeVzj0J+3l8DC/lHGrdUF?=
 =?us-ascii?Q?2mR34S64nPZg0gaHJwQrdH4X5ZGK0+z/h0pU3YMJ3KFSlbtfIOlLyJ93R1OH?=
 =?us-ascii?Q?+tttfQa6jK4xgSAW67cafKVube+J2bnXLRP6qUWVsHZE+7JclE0ujRUd/eMv?=
 =?us-ascii?Q?rWxLO9i+3CdulFW0wmsB27EUFh9jnCTmA8OJFiZoc16LCmhO90Xb41IfBroX?=
 =?us-ascii?Q?y2yXxh2MHyMgw+BtvYm5X17QfoqCCnlgQkGqoYxhAP02yZvXfcRIZeKoLzj0?=
 =?us-ascii?Q?uuetmREb+udMqT4s7uV3R3tM/9Z3bBELRNufZxCRSC3Y7JE087sKasjacU5n?=
 =?us-ascii?Q?0FqxbO/4/+ROv+3dYAFP+EYhjTFKLb2wepk6wb1B?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 5c95d23e-e3be-4fe1-955e-08dc7d603318
X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB6020.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 May 2024 08:45:30.9973
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: br0LlDaWVsO+M8/Cjf0fGxmsBz/afCP0LMetKWgZan0NCGEW9zpad/rU0tz/E+eqwYs6RpsOAawSQGD+K65NRQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR11MB7736
X-OriginatorOrg: intel.com

On 2024-03-28 at 11:12:57 +0800, Chao Gao wrote:
> >+#if IS_ENABLED(CONFIG_HYPERV)
> >+static int vt_flush_remote_tlbs(struct kvm *kvm);
> >+#endif
> >+
> > static __init int vt_hardware_setup(void)
> > {
> > 	int ret;
> >@@ -49,11 +53,29 @@ static __init int vt_hardware_setup(void)
> > 		pr_warn_ratelimited("TDX requires mmio caching.  Please enable mmio caching for TDX.\n");
> > 	}
> > 
> >+#if IS_ENABLED(CONFIG_HYPERV)
> >+	/*
> >+	 * TDX KVM overrides flush_remote_tlbs method and assumes
> >+	 * flush_remote_tlbs_range = NULL that falls back to
> >+	 * flush_remote_tlbs.  Disable TDX if there are conflicts.
> >+	 */
> >+	if (vt_x86_ops.flush_remote_tlbs ||
> >+	    vt_x86_ops.flush_remote_tlbs_range) {
> >+		enable_tdx = false;
> >+		pr_warn_ratelimited("TDX requires baremetal. Not Supported on VMM guest.\n");
> >+	}
> >+#endif
> >+
> > 	enable_tdx = enable_tdx && !tdx_hardware_setup(&vt_x86_ops);
> > 	if (enable_tdx)
> > 		vt_x86_ops.vm_size = max_t(unsigned int, vt_x86_ops.vm_size,
> > 					   sizeof(struct kvm_tdx));
> > 
> >+#if IS_ENABLED(CONFIG_HYPERV)
> >+	if (enable_tdx)
> >+		vt_x86_ops.flush_remote_tlbs = vt_flush_remote_tlbs;
> 
> Is this hook necessary/beneficial to TDX?
>

I think so.

We happended to encounter the following error and breaks the boot up:
"SEAMCALL (0x000000000000000f) failed: 0xc0000b0800000001"
0xc0000b0800000001 indicates the TDX_TLB_TRACKING_NOT_DONE, and it is caused
by page demotion but not yet doing a tlb shotdown by tlb track.


It was found on my system the CONFIG_HYPERV is not set, and it makes
kvm_arch_flush_remote_tlbs() not invoking tdx_track() before the
tdh_mem_page_demote(), which caused the problem.

> if no, we can leave .flush_remote_tlbs as NULL. if yes, we should do:
> 
> struct kvm_x86_ops {
> ...
> #if IS_ENABLED(CONFIG_HYPERV) || IS_ENABLED(TDX...)
> 	int  (*flush_remote_tlbs)(struct kvm *kvm);
> 	int  (*flush_remote_tlbs_range)(struct kvm *kvm, gfn_t gfn,
> 					gfn_t nr_pages);
> #endif

If the flush_remote_tlbs implementation are both available in HYPERV and TDX,
does it make sense to remove the config checks? I thought when commit 0277022a77a5
was introduced, the only user of flush_remote_tlbs() is hyperv, and now
there is TDX.

thanks,
Chenyu