Received-SPF: pass (google.com: domain of linux-kernel+bounces-21742-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Date: Wed, 10 Jan 2024 14:16:54 +0800
From: Chao Gao <chao.gao@intel.com>
To: Sean Christopherson <seanjc@google.com>
CC: Paolo Bonzini <pbonzini@redhat.com>, Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>, Dave Hansen
	<dave.hansen@linux.intel.com>, <x86@kernel.org>, <kvm@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>, Yi Lai <yi1.lai@intel.com>, Tao Su
	<tao1.su@linux.intel.com>, Xudong Hao <xudong.hao@intel.com>
Subject: Re: [PATCH] x86/cpu: Add a VMX flag to enumerate 5-level EPT support
 to userspace
Message-ID: <ZZ42Vs3uAPwBmezn@chao-email>
References: <20240110002340.485595-1-seanjc@google.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20240110002340.485595-1-seanjc@google.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?HYkofkDEdOhbx03Crcc62xW3DFiC9tyhIm5HFqPNwRSu7VQ8VDhSBsUTaH3F?=
 =?us-ascii?Q?m7NEQq6rkViDsDsnK9NqKeocWMdW0QhRAULUK/r33sjlNNVuq3b0QbP62g0t?=
 =?us-ascii?Q?ZVIRxjqafzTuYKU7UnSIhZOuzxnKqdPtG122jvkTct51920InE+Vq6pHEewv?=
 =?us-ascii?Q?oGxh3G2DsCMCKBPnPyBdbYKmMlV/pictV5Rr083xcSwRYwDGme4vu2Gi3zkQ?=
 =?us-ascii?Q?q4/MOG7nEeZkgrK2Rf5g7AGSdyhp+3ihDXrycWV/x2EsTc1CFz+CFG2qWFX/?=
 =?us-ascii?Q?i2xbSeiPoG6poCKo4PsCJ8XxKl7ufnZs68TWR4mY9N3rfu1QbvhlTrwjNwOx?=
 =?us-ascii?Q?bzPYX+NsozcgeqLcEb75YKJurCt3iKZ8kS+moVUdS1oFz1F/qEIljxV2iVdo?=
 =?us-ascii?Q?nBNAoPnEMA/cNWSmr8BzuobcUGGERY8HIRtgvR1coNlN3EFcDeBGHLVNyfob?=
 =?us-ascii?Q?YwbR1E7+JRH1Ek812qSK4YCWlkK3u3lhj9MhyE9QAc0bLlPSrQZzPjQqMVwi?=
 =?us-ascii?Q?Vyq41Um08Df1M9kZFiRmmn87sn75xkrD2jbif4bqafdNX608Iu1bQVySxTyM?=
 =?us-ascii?Q?6k/PGjildBAW+5UM7w+NvaFRVV6FDXU1tsrzYV4QUx5iAEsIqqrOIqvFKSg5?=
 =?us-ascii?Q?fc4EaJtt0+TDaFwJoyzH1mjIR0FOrbRzOKTl9uuz9uds3sA3UCx3PyZWL0m1?=
 =?us-ascii?Q?8Jpcjs4+d4pivr6z2vPO3q89Jgy9i855K9DVVE9qLj728c2CrIQWBxvs4ar7?=
 =?us-ascii?Q?MQIdpYzOZ1CphyNQ2GWgsnhnWDCKsE0q+94WX+4fMNadkFdI/8+cDry98f1T?=
 =?us-ascii?Q?mH+5FU6PdTSJbC8PV3Ip/UJvH/hT2Ll5hYQQ3viof+wEmosKqsIGmy5wP4gf?=
 =?us-ascii?Q?Q97F7HtZnNYiqKO954sCchcjsEPFsrB7sSgkQ9hWd3qiS31PpiwJpp3FcOU6?=
 =?us-ascii?Q?yjgnoBsvXjQieghIhq/3FznwXSZs+RVKWlLXIQcdtao8nz/6NZCGmQ+T2kb6?=
 =?us-ascii?Q?rpbO2oGp6NXF5NCPljdc7bMQy1C9flag95lH52BSe+mA2sWg0zBmQ43VdW1t?=
 =?us-ascii?Q?pFosz7sr7F2YHk2eXJytAdFy2stkGAr6CPVAxENsq+58XRNexSvkE/BNBGeb?=
 =?us-ascii?Q?3/DNqNh8JMTLgOvSnVON2rufYmaZuEj0tP8SRKRS+OZkT4NZ6vr31iUHdya3?=
 =?us-ascii?Q?ViSOsknu0aZQfHz/jmov4mO55EZv3uor7Uk6/gZSADLxj0fWkfE5aHRz/bno?=
 =?us-ascii?Q?/G/b6W3LPsGc+CAKL06F5PH9d+jS8mcJME8l94Hb3ZOVo4/kgZGe32K0manH?=
 =?us-ascii?Q?xgC/imJeapgIsBWkPOS6wlYTIu1zGz+y49ptCflBbolgps0Po8fpgRHrzmBw?=
 =?us-ascii?Q?F3eyt/aLcv9WglRmslxocCIF/Z+STn7GE3UsEsh7fMf1g4KZF+iX/V1XYTft?=
 =?us-ascii?Q?IwpvAGjEmHBw865AMdO9AOEey34/bveOn67jrpz7kMp7VX+mcToyq33lUa/B?=
 =?us-ascii?Q?dIFBPmwPjMWx6zB9hVcOF54vtfAqQtth9r5cm8YCDMtCeehOQCfaYL9ckpLe?=
 =?us-ascii?Q?L6b1AbpsG6xKkd5HchiaZqu/tSmBUWwzeQ0pPx4+?=
X-MS-Exchange-CrossTenant-Network-Message-Id: bafe24ef-0c5f-4227-34ec-08dc11a3c3df
X-MS-Exchange-CrossTenant-AuthSource: CH3PR11MB8660.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Jan 2024 06:17:04.4626
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: LJA+KivsgImu2bESTlsRENXLhCHhv45R+q3kX+zX/nkBPWplcB1N4bhFS+9J/VMJWfbLqeE3r18lhaATTk4WAw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR11MB5681
X-OriginatorOrg: intel.com

On Tue, Jan 09, 2024 at 04:23:40PM -0800, Sean Christopherson wrote:
>Add a VMX flag in /proc/cpuinfo, ept_5level, so that userspace can query
>whether or not the CPU supports 5-level EPT paging.  EPT capabilities are
>enumerated via MSR, i.e. aren't accessible to userspace without help from
>the kernel, and knowing whether or not 5-level EPT is supported is sadly
>necessary for userspace to correctly configure KVM VMs.

This assumes procfs is enabled in Kconfig and userspace has permission to
access /proc/cpuinfo. But it isn't always true. So, I think it is better to
advertise max addressable GPA via KVM ioctls.

>
>When EPT is enabled, bits 51:49 of guest physical addresses are consumed
>if and only if 5-level EPT is enabled.  For CPUs with MAXPHYADDR > 48, KVM
>*can't* map all legal guest memory if 5-level EPT is unsupported, e.g.
>creating a VM with RAM (or anything that gets stuffed into KVM's memslots)
>above bit 48 will be completely broken.
>
>Having KVM enumerate guest.MAXPHYADDR=48 in this scenario doesn't work
>either, as architecturally guest accesses to illegal addresses generate
>RSVD #PF, i.e. advertising guest.MAXPHYADDR < host.MAXPHYADDR when EPT is
>enabled would also result in broken guests.  KVM does provide a knob,
>allow_smaller_maxphyaddr, to let userspace opt-in to such setups, but
>that support is firmly best-effort, i.e. not something KVM wants to force
>upon userspace.
>
>While it's decidedly odd for a CPU to support a 52-bit MAXPHYADDR but not
>5-level EPT, the combination is architecturally legal and such CPUs do
>exist (and can easily be "created" with nested virtualization).
>
>Reported-by: Yi Lai <yi1.lai@intel.com>
>Cc: Tao Su <tao1.su@linux.intel.com>
>Cc: Xudong Hao <xudong.hao@intel.com>
>Signed-off-by: Sean Christopherson <seanjc@google.com>
>---
>
>tip-tree folks, this is obviously not technically KVM code, but I'd like to
>take this through the KVM tree so that we can use the information to fix
>KVM selftests (hopefully this cycle).
>
> arch/x86/include/asm/vmxfeatures.h | 1 +
> arch/x86/kernel/cpu/feat_ctl.c     | 2 ++
> 2 files changed, 3 insertions(+)
>
>diff --git a/arch/x86/include/asm/vmxfeatures.h b/arch/x86/include/asm/vmxfeatures.h
>index c6a7eed03914..266daf5b5b84 100644
>--- a/arch/x86/include/asm/vmxfeatures.h
>+++ b/arch/x86/include/asm/vmxfeatures.h
>@@ -25,6 +25,7 @@
> #define VMX_FEATURE_EPT_EXECUTE_ONLY	( 0*32+ 17) /* "ept_x_only" EPT entries can be execute only */
> #define VMX_FEATURE_EPT_AD		( 0*32+ 18) /* EPT Accessed/Dirty bits */
> #define VMX_FEATURE_EPT_1GB		( 0*32+ 19) /* 1GB EPT pages */
>+#define VMX_FEATURE_EPT_5LEVEL		( 0*32+ 20) /* 5-level EPT paging */
> 
> /* Aggregated APIC features 24-27 */
> #define VMX_FEATURE_FLEXPRIORITY	( 0*32+ 24) /* TPR shadow + virt APIC */
>diff --git a/arch/x86/kernel/cpu/feat_ctl.c b/arch/x86/kernel/cpu/feat_ctl.c
>index 03851240c3e3..1640ae76548f 100644
>--- a/arch/x86/kernel/cpu/feat_ctl.c
>+++ b/arch/x86/kernel/cpu/feat_ctl.c
>@@ -72,6 +72,8 @@ static void init_vmx_capabilities(struct cpuinfo_x86 *c)
> 		c->vmx_capability[MISC_FEATURES] |= VMX_F(EPT_AD);
> 	if (ept & VMX_EPT_1GB_PAGE_BIT)
> 		c->vmx_capability[MISC_FEATURES] |= VMX_F(EPT_1GB);
>+	if (ept & VMX_EPT_PAGE_WALK_5_BIT)
>+		c->vmx_capability[MISC_FEATURES] |= VMX_F(EPT_5LEVEL);
> 
> 	/* Synthetic APIC features that are aggregates of multiple features. */
> 	if ((c->vmx_capability[PRIMARY_CTLS] & VMX_F(VIRTUAL_TPR)) &&
>
>base-commit: 1c6d984f523f67ecfad1083bb04c55d91977bb15
>-- 
>2.43.0.472.g3155946c3a-goog
>
>