Received-SPF: pass (google.com: domain of linux-kernel+bounces-209676-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Date: Tue, 11 Jun 2024 18:48:48 +0800
From: Chao Gao <chao.gao@intel.com>
To: Sean Christopherson <seanjc@google.com>
CC: <kvm@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
	<daniel.sneddon@linux.intel.com>, <pawan.kumar.gupta@linux.intel.com>, "Zhang
 Chen" <chen.zhang@intel.com>, Paolo Bonzini <pbonzini@redhat.com>, "Thomas
 Gleixner" <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, "Borislav
 Petkov" <bp@alien8.de>, Dave Hansen <dave.hansen@linux.intel.com>,
	<x86@kernel.org>, "H. Peter Anvin" <hpa@zytor.com>
Subject: Re: [RFC PATCH v3 09/10] KVM: VMX: Advertise
 MITI_CTRL_BHB_CLEAR_SEQ_S_SUPPORT
Message-ID: <ZmgrkMLuComwPl1X@chao-email>
References: <20240410143446.797262-1-chao.gao@intel.com>
 <20240410143446.797262-10-chao.gao@intel.com>
 <ZmepkZfLIvj_st5W@google.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <ZmepkZfLIvj_st5W@google.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?x0XMXqEPqbEiDx78DbbW0qCYhoos1/Rn3YsEO9WUVF0AVlfUpUsKAU/yi+xH?=
 =?us-ascii?Q?VmKqJbaGEJDrkFswbDvK/wzQxvmgzlNS4dOKxHqDZoPn9EHjl2gZJ727RQfX?=
 =?us-ascii?Q?pZO1RzKMFLUgqKdk1XfBBVE0ptE2dSwJvN/vZ0Omjt9ZucyAlLzN8OeSORDs?=
 =?us-ascii?Q?KBeJST6zFItlX6vLr8KW3hIdZovTMe8RhPbAou/eZPAYDfb0NiB443pujoXA?=
 =?us-ascii?Q?w6LY86u/1L/9sqa6VP101axpC/rfGbc5rNSBULwPK1GkKIV/QPRIKIrhy8Q/?=
 =?us-ascii?Q?Jqok8pBZ+FYy5o5+Kz17aFhC6tbMry/wOMm57eDfIbFMskUHYt2q3dcdB0BF?=
 =?us-ascii?Q?DKKJledEqMV2+xKXQ/si/JZoQSJfWCzsyJoopEPIKEiIFC71Liixz4GjyrE3?=
 =?us-ascii?Q?eSsQJaXkjWVxDOyvwcUvBR1CrQou6nJ2/ekIlWMS9u5S0m+TT+s8w+ELuE10?=
 =?us-ascii?Q?2/ZXJk0d0s6eAnj6RlHKyEpmSIRQVjL3RakFVk8RfRKupIqeLxafrpHiY2aM?=
 =?us-ascii?Q?Vg8Hz4dwlhnMCB0afFF08Ern4raJxD0BqSUYXvjShshXM6mfkl4HxE60WgVU?=
 =?us-ascii?Q?sEK8PrpT+wihLpLPpBCxXhiBC0Xa1bC/0lE665+H0qYDUBDReMZXKo5cMi9V?=
 =?us-ascii?Q?qaj8kBemBv2XTd/sHqqY45gH/DzUjo+svVbXtPeFGlJrrDetBIxDsbhE6XOs?=
 =?us-ascii?Q?TgMzskN1Flgq+n259134FeV+vaTxE1cGWCjf2W1vQXvo4RazTWz2phkLlzkd?=
 =?us-ascii?Q?+JfQDIjCZ2O1I4KxSsvlGaq32W3Pe3yxbdsgFwlkclJq1tf6j7VyiH1ZKNlU?=
 =?us-ascii?Q?ie0x8R87tWtz4ol+lq7Y874yBXSgJmly2QdDblruRUkAUcW8dFydofJLqHZa?=
 =?us-ascii?Q?DGlBOPsv+eZ7wJ8rADql5GUTT6qDLN3XYBw7jeGXE02s+TLbyIzSNvVeySlB?=
 =?us-ascii?Q?v8thhYPaB5L5rYaMlkpvM8JO+fD1B4M6aUrQhUw5ccnWVCj7aRqmPxx5PkT3?=
 =?us-ascii?Q?FjxqKEAY7jP8kQAOFUSgLuGHfErcmnXwF57Rc57lFgctaoJhCNawv26pj35+?=
 =?us-ascii?Q?Ijo7wHyDEAtnQNQ5ejUcGiW5mx7KfgP4xU+B/mXoMr6ZxD20qY2Bw3PrjemZ?=
 =?us-ascii?Q?1xVIkWlbxcIG60Gw90XyP9NUsxNJ3R6fo0kh3Gn8kFVmIWVsV/mw3jIMKG3E?=
 =?us-ascii?Q?mdMJbrwG6Pk0JTi4m2deG6xEf06rOAa8JSZbcI+4V++jEi8qw6SI6DPpn5+B?=
 =?us-ascii?Q?WRsrePQIxJ/u3fEV72F3quyATU/IQSldlcfl0BN+EmSKqAUYwjRDhp/x0s/o?=
 =?us-ascii?Q?Lzodgso5Xo5sjK6aYavN5pbJwriOFl5pNlo2al+zfWFMW1Yo9w1kUGW1auT/?=
 =?us-ascii?Q?0zGtx1+Z/zu2bHfWit2WreVAv0WVL313VDlBPnhmATttlm0x8T/BjUBjyH6d?=
 =?us-ascii?Q?7OkUS7/7sM0DmA3oVmLqDA6OZ+jt+tmosngDxy2piF6bN1AKOms8qGPBZAWP?=
 =?us-ascii?Q?cwIwa0SwumqOx8LKTX1DJi5Iofd7hE7xBzYboV3NnFR4Yk45Atq8njRb1SEf?=
 =?us-ascii?Q?nymiVGJUyWuboPrjMiQs9f/a1wskGZHhuRWKRexs?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 390e0efa-63e8-4d78-102a-08dc8a0418f4
X-MS-Exchange-CrossTenant-AuthSource: CH3PR11MB8660.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Jun 2024 10:48:58.6390
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: l3464oinAUriFL3XI2f4IqIlLDFJxynm6YUy0tKKQNLqLssXb/r67blwT40ur2WBQaknhf2fi6DJYrTXzbsrGw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR11MB8014
X-OriginatorOrg: intel.com

>> +		if (data & MITI_CTRL_BHB_CLEAR_SEQ_S_USED &&
>> +		    kvm_cpu_cap_has(X86_FEATURE_BHI_CTRL) &&
>> +		    !(host_arch_capabilities & ARCH_CAP_BHI_NO))
>> +			spec_ctrl_mask |= SPEC_CTRL_BHI_DIS_S;
>> +
>> +		/*
>> +		 * Intercept IA32_SPEC_CTRL to disallow guest from changing
>> +		 * certain bits if "virtualize IA32_SPEC_CTRL" isn't supported
>> +		 * e.g., in nested case.
>> +		 */
>> +		if (spec_ctrl_mask && !cpu_has_spec_ctrl_shadow())
>> +			vmx_enable_intercept_for_msr(vcpu, MSR_IA32_SPEC_CTRL, MSR_TYPE_RW);
>> +
>> +		/*
>> +		 * KVM_CAP_FORCE_SPEC_CTRL takes precedence over
>> +		 * MSR_VIRTUAL_MITIGATION_CTRL.
>> +		 */
>> +		spec_ctrl_mask &= ~vmx->vcpu.kvm->arch.force_spec_ctrl_mask;
>> +
>> +		vmx->force_spec_ctrl_mask = vmx->vcpu.kvm->arch.force_spec_ctrl_mask |
>> +					    spec_ctrl_mask;
>> +		vmx->force_spec_ctrl_value = vmx->vcpu.kvm->arch.force_spec_ctrl_value |
>> +					    spec_ctrl_mask;
>> +		vmx_set_spec_ctrl(&vmx->vcpu, vmx->spec_ctrl_shadow);
>> +
>>  		vmx->msr_virtual_mitigation_ctrl = data;
>>  		break;
>
>I continue find all of this unpalatable.  The guest tells KVM what software
>mitigations the guest is using, and then KVM is supposed to translate that into
>some hardware functionality?  And merge that with userspace's own overrides?

Yes. It is ugly. I will drop all Intel-defined stuff from KVM. Actually, I
wanted to punt to userspace ...

>
>Blech.
>
>With KVM_CAP_FORCE_SPEC_CTRL, I don't see any reason for KVM to support the
>Intel-defined virtual MSRs.  If the userspace VMM wants to play nice with the
>Intel-defined stuff, then userspace can advertise the MSRs and use an MSR filter
>to intercept and "emulate" the MSRs.  They should be set-and-forget MSRs, so
>there's no need for KVM to handle them for performance reasons.

... I had this idea of implementing policy-related stuff in userspace, and I wrote
in the cover-letter:

	"""
	1. the KVM<->userspace ABI defined in patch 1

	I am wondering if we can allow the userspace to configure the mask
	and the shadow value during guest's lifetime and do it on a vCPU basis.
	this way, in conjunction with "virtual MSRs" or any other interfaces,
	the usespace can adjust hardware mitigations applied to the guest during
	guest's lifetime e.g., for the best performance.
	"""

As said, this requires some tweaks to KVM_CAP_FORCE_SPEC_CTRL, such as making
the mask and shadow values adjustable and applicable on a per-vCPU basis. The
tweaks are not necessarily for Intel-defined virtual MSRs; if there were other
preferable interfaces, they could also benefit from these changes.

Any objections to these tweaks to KVM_CAP_FORCE_SPEC_CTRL?

>
>That way KVM doesn't need to deal with the the virtual MSRs, userspace can make
>an informed decision when deciding how to set KVM_CAP_FORCE_SPEC_CTRL, and as a
>bonus, rollouts for new mitigation thingies should be faster as updating userspace
>is typically easier than updating the kernel/KVM.

Good point!