Received-SPF: pass (google.com: domain of linux-kernel+bounces-190285-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Message-ID: <e7742e9c-140e-4a36-8e5a-e6884bc24b76@intel.com>
Date: Mon, 27 May 2024 17:05:23 +0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v10 24/27] KVM: x86: Enable CET virtualization for VMX and
 advertise to userspace
From: "Yang, Weijiang" <weijiang.yang@intel.com>
To: Sean Christopherson <seanjc@google.com>, <dave.hansen@intel.com>,
	<rick.p.edgecombe@intel.com>
CC: Thomas Gleixner <tglx@linutronix.de>, <pbonzini@redhat.com>,
	<x86@kernel.org>, <kvm@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
	<peterz@infradead.org>, <chao.gao@intel.com>, <mlevitsk@redhat.com>,
	<john.allen@amd.com>
References: <20240219074733.122080-1-weijiang.yang@intel.com>
 <20240219074733.122080-25-weijiang.yang@intel.com>
 <ZjLNEPwXwPFJ5HJ3@google.com>
 <39b95ac6-f163-4461-93f3-eaa653ab1355@intel.com>
 <ZkYauRJBhaw9P1A_@google.com> <87r0e0ke8w.ffs@tglx>
 <ZkdpKiSyOwB3NwRD@google.com>
 <a170e420-efc3-47f9-b825-43229c999c0d@intel.com>
 <ZkuD1uglu5WFSoxR@google.com>
 <07fc9f6f-721a-4b89-baa5-986664ad5430@intel.com>
Content-Language: en-US
In-Reply-To: <07fc9f6f-721a-4b89-baa5-986664ad5430@intel.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?U05KN1hyK0QrYTZFQnQ3MTlUUk82aXdKVkJtU2tMZGdSZ085cktySnBHVzJX?=
 =?utf-8?B?T3VxeG5CRmhsUkMvYTJQb0RaOVlNL1VGOVdYdG95YjNnMHJtVGdmZmkxREkv?=
 =?utf-8?B?Q0hQb1NNbGk3eU01YTFBMFM0Y0tPeXU4SmlqN2hOYWZUbHFZU0tTOUdFTXpB?=
 =?utf-8?B?RGVQdmI3MHJWTXZ1dk5GcE5IUC9zWEt5M1dqRTlGaExNeThDNXgyRGVJSUpz?=
 =?utf-8?B?WFF0NEF3VWh0SUFIby93Q01mVW1PYVRJOWZrRmg4MHVpSTVyRnFudTJCdTBn?=
 =?utf-8?B?NG42T1AvVzFrYWJUQkJzaVlNUGNjd0tWZkpHRCtNWXhQNmVNWGxody9zL1R2?=
 =?utf-8?B?Z0NZcVd4a20xVmZaY2xXTWdWUG03WFdpVDNURmRaMmNYUitlMXNBcnEzYTdn?=
 =?utf-8?B?UUdqSUIxNGtMSXRhUXlLVUZ4czNwM2t4Q2ZKYTdRTUNLSGlrMFM0eWR3YnpG?=
 =?utf-8?B?RzVPVFBQMWJoK3RjNTRXWnVDeVB0UER0TGNaaHpUOFRTWlZrQkVpcmJQSVZS?=
 =?utf-8?B?RXF4c1FCa3VBVjRHMkJHbUFRTnNPWXN6OGUyVEtDRDU4T0hQbmpBK2JzUGFl?=
 =?utf-8?B?Qm9tL1VPNHo5ZGdObXdmaHJSS0F2QzlJOTFQaEFWNXd2aHFjT2pYZFVCV0Zu?=
 =?utf-8?B?dmVNUFBvMDU5R0tyK0ZlTEVXNjZCY25KMHpzVUtMaC9JSnZnQWFWVE13ekVP?=
 =?utf-8?B?OENQOCtJMmQrcmlaaHBjeFNvS3JpeXUrckN3ZU80aGJhWUY4dktIdHB2a2lW?=
 =?utf-8?B?bVNQUWpVVFpLK3cyb3BmTFJ6NkMzNnQrS2w5Rjh6WE1BekdrVGRXOFV4N29h?=
 =?utf-8?B?TENuNzJvczZWQi84WlNZYjEzbnFEbDFseGFPeHFjbklOTWhnL1VmeFBKVjIw?=
 =?utf-8?B?QXUrU3QvNnNUVFgxN2l5TWc5ejZPVElwRngvaXl2dE9MSVFYUHpwUVR1R0Z1?=
 =?utf-8?B?SnBhYzl5RE5RRmN1ckNGRHhRaDBqc1hXMUpHTlAxdCtoQXN3djlwSG00QXRD?=
 =?utf-8?B?YzB6M1pRMHRJMWpmLzh0aDVwRXRLeTlKeUFHTU15S0xZc0t6c2orZm8yNWIy?=
 =?utf-8?B?RjZsMVVZbUVhQ0UvSXRxUjNTVWFKL0xncmExejBtR2ZSYk9EY21UZDZMMG01?=
 =?utf-8?B?V3dJTUY5QmVWdWljREQ1R3FqRXhmQUVJY2RZeWFMME1XRXpiUk92YmgvaCti?=
 =?utf-8?B?bG9qT2taOWxMbzExNEUvQ3ZXaGtCc1FyaWh4d3RsV2VOOHd5dS9CVUVlaGxN?=
 =?utf-8?B?MGJjN3V2Wkh2ZDdBaUhOWkdNM3ZKL1VRNW5HbW81cnQxTmxja1k0R2piR0V0?=
 =?utf-8?B?RUppcUduakNHcGhib1N4UzhDaTFSK1ZFR1BuNEEvS0I0N3dmNUh2eW03Q1pC?=
 =?utf-8?B?T0dSOWtveW5ScEVjZTg1L3ZmWEJDTHlncTB4UUdvdmwrNEJmUE82WFJNbXdY?=
 =?utf-8?B?ZGJSOXFnR2FjalpsemkzQmlBaWpBQUNrTVpYZ3h0RDkwb1J2dEJES3F2WlRG?=
 =?utf-8?B?MlhKK0VHMmpyZjJGUU1UeXBOL1JpOWd6RkZwbDZqem5MaEhBZi95U0FzUjNF?=
 =?utf-8?B?dnlpT2VMalFRUUxNd1VKUEpPOW9JZjdxRUxNZ2Y1bDZ4WWgzYmVCbGc1R1pL?=
 =?utf-8?B?UEluSk9TRmdXUnpNWkYvc1NvZ3ZRNmlUcHhXU0J1NzNySkdGWXdZcGdqb2N0?=
 =?utf-8?B?MmdYd09udCtQQ1hKd25Sc1liQVRyR05QM2syZE5hR2djSk9MSlBhcG9rOUZG?=
 =?utf-8?B?WTJ1d0dWand4ZDhZSmVJMWkxQ3hkaGR4czZZUW9HSGxveHI3RXpBUDNKQ20r?=
 =?utf-8?B?Z3hPLzYxRndGcDBvTUVZMTVhTWhLRkpJT3ZJanNxajVzRUR1ZHFRMU90OWZ2?=
 =?utf-8?B?WjQ2UXlwd2JMQ2J2dG1MYW9pYXAyS0ZJUXNHaFR4bTVkaDRhWE9PbnNTaDNZ?=
 =?utf-8?B?cmE3dkozUndHUUdteis1M0dyUUlKQzM2Y2dDNXFtVTlNTTR2ZzF1OTJJVkYz?=
 =?utf-8?B?RHJZU2pZSVVkTmkwTHc3REFyVlplZmtjN1A3Zy9UbnZTMjNFYVhCNnc3UXdQ?=
 =?utf-8?B?aWhiYUF5eHFWeWNFS0tteVNTa01ScmxwbXduRVNsdmhwdkJVaERCL1luY1RG?=
 =?utf-8?B?UWdZSW5abkdkY016cHVjQ2gxWXQ4ZDd3NURaSmFzWE9oWWVvZVoxMjBHaWpV?=
 =?utf-8?B?bHc9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: e6d14f40-c885-4be9-15a1-08dc7e2c29a0
X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB4965.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 May 2024 09:05:32.5208
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: WDlofNSXodiC3aga0MhqXR8ZmBDF7OZ/YMKE3V1LRhuvW4/EMrpSQ7vAYG3qQpnFA59BwVhRBCyS2xV9Ia94ag==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR11MB7258
X-OriginatorOrg: intel.com

On 5/22/2024 4:41 PM, Yang, Weijiang wrote:
> On 5/21/2024 1:09 AM, Sean Christopherson wrote:
>> On Mon, May 20, 2024, Weijiang Yang wrote:
>>> On 5/17/2024 10:26 PM, Sean Christopherson wrote:
>>>> On Fri, May 17, 2024, Thomas Gleixner wrote:
>>>>> On Thu, May 16 2024 at 07:39, Sean Christopherson wrote:
>>>>>> On Thu, May 16, 2024, Weijiang Yang wrote:
>>>>>>> We synced the issue internally, and got conclusion that KVM should honor host
>>>>>>> IBT config.  In this case IBT bit in boot_cpu_data should be honored.  With
>>>>>>> this policy, it can avoid CPUID confusion to guest side due to host ibt=off
>>>>>>> config.
>>>>>> What was the reasoning?  CPUID confusion is a weak justification, e.g. it's not
>>>>>> like the guest has visibility into the host kernel, and raw CPUID will still show
>>>>>> IBT support in the host.
>>>>>>
>>>>>> On the other hand, I can definitely see folks wanting to expose IBT to guests
>>>>>> when running non-complaint host kernels, especially when live migration is in
>>>>>> play, i.e. when hiding IBT from the guest will actively cause problems.
>>>>> I have to disagree here violently.
>>>>>
>>>>> If the exposure of a CPUID bit to a guest requires host side support,
>>>>> e.g. in xstate handling, then exposing it to a guest is simply not
>>>>> possible.
>>>> Ya, I don't disagree, I just didn't realize that CET_USER would be cleared in the
>>>> supported xfeatures mask.
>>> For host side support, fortunately,  this patch already has some checks for
>>> that. But for userspace CPUID config, it allows IBT to be exposed alone.
>>>
>>> IIUC, this series tries to tie IBT to SHSTK feature, i.e., IBT cannot be
>>> exposed as an independent feature to guest without exposing SHSTK at the same
>>> time. If it is, then below patch is not needed anymore:
>>> https://lore.kernel.org/all/20240219074733.122080-3-weijiang.yang@intel.com/
>> That's a question for the x86 maintainers.  Specifically, do they want to allow
>> enabling XFEATURE_CET_USER even if userspace shadow stack support is disabled.
>>
>> I don't think it impacts KVM, at least not directly.  Regardless of what decision
>> the kernel makes, KVM needs to disable IBT and SHSTK if CET_USER _or_ CET_KERNEL
>> is missing, which KVM already does via:
>>
>>     if ((kvm_caps.supported_xss & (XFEATURE_MASK_CET_USER |
>>          XFEATURE_MASK_CET_KERNEL)) !=
>>         (XFEATURE_MASK_CET_USER | XFEATURE_MASK_CET_KERNEL)) {
>>         kvm_cpu_cap_clear(X86_FEATURE_SHSTK);
>>         kvm_cpu_cap_clear(X86_FEATURE_IBT);
>>         kvm_caps.supported_xss &= ~(XFEATURE_MASK_CET_USER |
>>                         XFEATURE_MASK_CET_KERNEL);
>>     }
>>
>>> I'd check and clear IBT bit from CPUID when userspace enables only IBT via
>>> KVM_SET_CPUID2.
>> No.  It is userspace's responsibility to provide a sane CPUID model for the guest.
>> KVM needs to ensure that *KVM* doesn't treat IBT as supported if the kernel doesn't
>> allow XFEATURE_CET_USER, but userspace can advertise whatever it wants to the guest
>> (and gets to keep the pieces if it does something funky).
>
> OK, I think we can go ahead to keep KVM patches as-is given the fact user IBT is not enabled in Linux.
> I only hope other OSes can enforce both SHSTK and IBT dependency on XFEATURE_CET_USER so
> that user IBT can work well there.
>
> Then IBT can be exposed to guest alone because guest *kernel* IBT only relies on S_CET MSR  which is
> VMCS auto-saved/restored.
>
> What's your thoughts?

If there's objection I'll do below changes for this series:
1) Remove patch :
https://lore.kernel.org/all/20240219074733.122080-3-weijiang.yang@intel.com/
2) Remove reference to raw CPUID for KVM IBT CPUID, handling it the same as SHSTK.

Meanwhile still allow userspace to enable IBT feature alone because user IBT is not enabled
in kernel now, leave enforcement of user IBT dependency on XFEATURE_CET_USER in future.

>
>