Received-SPF: pass (google.com: domain of linux-kernel+bounces-181848-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Message-ID: <f80b1b68-fd29-4fbe-bcbd-782932432937@intel.com>
Date: Fri, 17 May 2024 16:04:38 +0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v10 24/27] KVM: x86: Enable CET virtualization for VMX and
 advertise to userspace
To: Sean Christopherson <seanjc@google.com>
CC: <pbonzini@redhat.com>, <dave.hansen@intel.com>, <x86@kernel.org>,
	<kvm@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
	<peterz@infradead.org>, <chao.gao@intel.com>, <rick.p.edgecombe@intel.com>,
	<mlevitsk@redhat.com>, <john.allen@amd.com>
References: <20240219074733.122080-1-weijiang.yang@intel.com>
 <20240219074733.122080-25-weijiang.yang@intel.com>
 <ZjLRnisdUgeYgg8i@google.com>
 <83bb5f3f-a374-4b0e-a26d-9a9d88561bbe@intel.com>
 <f77496b0-ee94-4690-803f-44650706640f@intel.com>
 <ZkYbdaW-2p9wHwEL@google.com>
Content-Language: en-US
From: "Yang, Weijiang" <weijiang.yang@intel.com>
In-Reply-To: <ZkYbdaW-2p9wHwEL@google.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?U04zT3JSN3NrRHBwak54KzQvZ3pxeUR1SjhSakFBYUhPRGhCQUJ0dmNHYklX?=
 =?utf-8?B?VWhRLzRKQldUKzF6bnV6dDYza3JGWVhLZmJFZUhFTytnSnR3Qm9rTVdNQlhk?=
 =?utf-8?B?UFI4RGdpZlRnS2NNeGx2VUhYcm44SW5LMnJ1RXVJOWM1Wkx3N1A0dEhocjc5?=
 =?utf-8?B?a1licElGaHh6eU8wWHpidklQYjFpRFVYWnpLb2t6WjhDdnZZaGUwLzVDTVov?=
 =?utf-8?B?dXBBQWJkK3hsS0dIa05rQ1Z4RldsWGViUnZLOFRlY29SaXZrTy9IT2Ywd1c3?=
 =?utf-8?B?OXRHRW1vbWtyc0RFME5UQThWYXhqdFRKdzZzVEdWK1ZTY1hnVFVob005dVll?=
 =?utf-8?B?SURZK2VBWGhLbTNoaXdRZlY2MC9NTDkxOVVOS3QvbVFHRmJDaUw3b2pFcnNp?=
 =?utf-8?B?ckhTVXRJZFBwaTh1VXdML3VzQmpDaXpiNUVtSTR6RkF2Q2FKeXpBVVFmUWRJ?=
 =?utf-8?B?VEZBdGxVNVpDb3p1T2xxd09FQnQ3UE43MXQyV2ZxcVExU2dSajQ4bUV4WGJW?=
 =?utf-8?B?OGIrVjdORFRlT29pR2J1c1dOdC9lSkE5N0luMkJ0VTJaT29oVWVEVVlFQmlZ?=
 =?utf-8?B?K1FvTlgzRnlrWEgxZE9JT0VoekRnM015a1NYcmRQZGthZFJlclVYKzVLT1F2?=
 =?utf-8?B?REluUkI5UUNXOUxFVE5LdE9tR3oyVncwRmpzL0VaM0sySXlrMHpsMnJkekNl?=
 =?utf-8?B?cTl4dGlHRUt6WlFmblBVcis5SjlOYmJJMjR2eHVodXdnb2ovcWRsMm5NSUph?=
 =?utf-8?B?aFhZUVNPeEFLbFZ5YmRoV1VVV1FWNTBoUFErVDZGa1BUaHovL1hkWGdxYk1p?=
 =?utf-8?B?U3MxR2pwdmNpaWZSdUdZK2U1UExFNDRVZXBjY3pwaS9ZU2YxclNMd1dCS2pX?=
 =?utf-8?B?b1ZsNFVYVXJnVm8xSUV0cUxiTkpoQ3JiMVhqME4rQ3hLUXVBemdob2RlRVRz?=
 =?utf-8?B?SnlwSm43T0g0SE9hTXMvYnVFaXV3Z2t3MFBVSi9vc3ZGWDBpSGhaOXpCYXkr?=
 =?utf-8?B?YTV3eEQ4d3VxNHBpd2VydW1adGhDVitvTkQ5VkgwNEUyZFFGZDRRaHoydGtl?=
 =?utf-8?B?NU1CMzBNOHUrYmY2ZXpzZExmNVVQczFWRWthWHpGQVo1K0orUDhKMlVhTHdO?=
 =?utf-8?B?VGo1WFl6eW5IbklndmhiUHpyQnVUOUpreWEyQi8ySE5obExlRzB5TWNkaUd2?=
 =?utf-8?B?MlFxakZGTTZ1ZkNRZlhWaG9QbHQ1UXhSdk1rMU5waUlBUlBkYUtVQnptOGlk?=
 =?utf-8?B?ZXhBdG5JQW9zeUtWSy9mK3JTWnhxWS9jUmlrTUwvUGVhYUlDQUtITXBMRFdV?=
 =?utf-8?B?eDh3ZGFCZW5yS2gvMm9CSDFHODlOa3VFYjJ3SEF4SUpOaWFoS2hQbGF0c0Jo?=
 =?utf-8?B?R0twODNwVHMrNUs5M3hSaXdicVoyenlOeVRodHBDZlF1RFBrbDVqckVVVTFh?=
 =?utf-8?B?OW4wa2NJa2YweUdJNXgvcUc1NzFITi9kbUszSFNsWGFNamllZFpnVk1tb1lM?=
 =?utf-8?B?eEJNWnNHamxoSVNRTVFadk9QVlViQW5QSGY1a0h4SWsxSTZRQVczUytOcWZ3?=
 =?utf-8?B?NWEvb1dWM1I5OTZyRVVYWWNNb1lwdWtjNGd2TzVhWXoyTHIyV2tiZEdpMjVt?=
 =?utf-8?B?ZDZwTHRHZElxcEVMK1pyeVg4UVJmL3NXVkpGSFlZbTh0VXBCdE1ESExqUEJN?=
 =?utf-8?B?VFd5OWN1ZU5WOHQwTnNXeEJMa0FVZmFPc2FybTVGN2tBUnU5enFpZUNoL1JU?=
 =?utf-8?B?YkpUUC96VUM1Tkhtb1Y0YTMwVjhDVm1DRVcwaElnZWtwcVJTbnJOanZEN1l1?=
 =?utf-8?B?OFRoWDdINnJtYkpoUUdvYnJPNUs3QmlTTUR1U2huRHR6SFdXalJ1UXc5cWJq?=
 =?utf-8?B?OVBBZE5JVjh3S3dOeHAwUHdDOTNhRUg3MnlYdEZjWkllWUxvK2xWbm1UUEd5?=
 =?utf-8?B?SmNYVHgzQUwvOUZ1S2w1TXRaYjk2OTVvdmhkZlcyOFR4Y0lhUkZnMndjcXUw?=
 =?utf-8?B?K1RDdmdQUFlpckloQUk3ZXlKWkg0OFlYUnhFY201RU80azBYRU82aE5hUU90?=
 =?utf-8?B?Y3h4TjlMektsVnNMS2sxejVwejdkZTJpYTE3cC9kbmpiTXVaZFZUb1NSRjlR?=
 =?utf-8?B?VmVjTG9wbmJsT1hhK3h6d3FneGNyU0ZVOVFiTVNHM1ZBenpaa1d6N0VmMkpv?=
 =?utf-8?B?SFE9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 8cbb0d20-9d4f-4c0b-a463-08dc764804d6
X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB4965.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 May 2024 08:04:47.4389
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 5j4VB/8CDTZE09ySQE7TdsvbzgOWC/CBlZRFQo7Z06X9MiOPb/+F9QDuwkH+HceWC6O3jIEQt27rIyn8SibzbQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW6PR11MB8365
X-OriginatorOrg: intel.com

On 5/16/2024 10:43 PM, Sean Christopherson wrote:
> On Thu, May 16, 2024, Weijiang Yang wrote:
>> On 5/6/2024 5:41 PM, Yang, Weijiang wrote:
>>> On 5/2/2024 7:34 AM, Sean Christopherson wrote:
>>>> On Sun, Feb 18, 2024, Yang Weijiang wrote:
>>>>> @@ -665,7 +665,7 @@ void kvm_set_cpu_caps(void)
>>>>>            F(AVX512_VPOPCNTDQ) | F(UMIP) | F(AVX512_VBMI2) | F(GFNI) |
>>>>>            F(VAES) | F(VPCLMULQDQ) | F(AVX512_VNNI) | F(AVX512_BITALG) |
>>>>>            F(CLDEMOTE) | F(MOVDIRI) | F(MOVDIR64B) | 0 /*WAITPKG*/ |
>>>>> -        F(SGX_LC) | F(BUS_LOCK_DETECT)
>>>>> +        F(SGX_LC) | F(BUS_LOCK_DETECT) | F(SHSTK)
>>>>>        );
>>>>>        /* Set LA57 based on hardware capability. */
>>>>>        if (cpuid_ecx(7) & F(LA57))
>>>>> @@ -683,7 +683,8 @@ void kvm_set_cpu_caps(void)
>>>>>            F(SPEC_CTRL_SSBD) | F(ARCH_CAPABILITIES) | F(INTEL_STIBP) |
>>>>>            F(MD_CLEAR) | F(AVX512_VP2INTERSECT) | F(FSRM) |
>>>>>            F(SERIALIZE) | F(TSXLDTRK) | F(AVX512_FP16) |
>>>>> -        F(AMX_TILE) | F(AMX_INT8) | F(AMX_BF16) | F(FLUSH_L1D)
>>>>> +        F(AMX_TILE) | F(AMX_INT8) | F(AMX_BF16) | F(FLUSH_L1D) |
>>>>> +        F(IBT)
>>>>>        );
>>>> ...
>>>>
>>>>> @@ -7977,6 +7993,18 @@ static __init void vmx_set_cpu_caps(void)
>>>>>          if (cpu_has_vmx_waitpkg())
>>>>>            kvm_cpu_cap_check_and_set(X86_FEATURE_WAITPKG);
>>>>> +
>>>>> +    /*
>>>>> +     * Disable CET if unrestricted_guest is unsupported as KVM doesn't
>>>>> +     * enforce CET HW behaviors in emulator. On platforms with
>>>>> +     * VMX_BASIC[bit56] == 0, inject #CP at VMX entry with error code
>>>>> +     * fails, so disable CET in this case too.
>>>>> +     */
>>>>> +    if (!cpu_has_load_cet_ctrl() || !enable_unrestricted_guest ||
>>>>> +        !cpu_has_vmx_basic_no_hw_errcode()) {
>>>>> +        kvm_cpu_cap_clear(X86_FEATURE_SHSTK);
>>>>> +        kvm_cpu_cap_clear(X86_FEATURE_IBT);
>>>>> +    }
>>>> Oh!  Almost missed it.  This patch should explicitly kvm_cpu_cap_clear()
>>>> X86_FEATURE_SHSTK and X86_FEATURE_IBT.  We *know* there are upcoming AMD CPUs
>>>> that support at least SHSTK, so enumerating support for common code would yield
>>>> a version of KVM that incorrectly advertises support for SHSTK.
>>>>
>>>> I hope to land both Intel and AMD virtualization in the same kernel release, but
>>>> there are no guarantees that will happen.  And explicitly clearing both SHSTK and
>>>> IBT would guard against IBT showing up in some future AMD CPU in advance of KVM
>>>> gaining full support.
>>> Let me be clear on this, you want me to disable SHSTK/IBT with
>>> kvm_cpu_cap_clear() unconditionally for now in this patch, and wait until
>>> both AMD's SVM patches and this series are ready for guest CET, then remove
>>> the disabling code in this patch for final merge, am I right?
> No, allow it to be enabled for VMX, but explicitly disable it for SVM, i.e.
>
> diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
> index 4aaffbf22531..b3df12af4ee6 100644
> --- a/arch/x86/kvm/svm/svm.c
> +++ b/arch/x86/kvm/svm/svm.c
> @@ -5125,6 +5125,10 @@ static __init void svm_set_cpu_caps(void)
>          kvm_caps.supported_perf_cap = 0;
>          kvm_caps.supported_xss = 0;
>   
> +       /* KVM doesn't yet support CET virtualization for SVM. */
> +       kvm_cpu_cap_clear(X86_FEATURE_SHSTK);
> +       kvm_cpu_cap_clear(X86_FEATURE_IBT);
> +
>          /* CPUID 0x80000001 and 0x8000000A (SVM features) */
>          if (nested) {
>                  kvm_cpu_cap_set(X86_FEATURE_SVM);
>
> Then the SVM series can simply delete those lines when all is ready.

Understood, thanks!