Received-SPF: pass (google.com: domain of linux-kernel+bounces-185944-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Message-ID: <07fc9f6f-721a-4b89-baa5-986664ad5430@intel.com>
Date: Wed, 22 May 2024 16:41:35 +0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v10 24/27] KVM: x86: Enable CET virtualization for VMX and
 advertise to userspace
To: Sean Christopherson <seanjc@google.com>
CC: Thomas Gleixner <tglx@linutronix.de>, <rick.p.edgecombe@intel.com>,
	<pbonzini@redhat.com>, <dave.hansen@intel.com>, <x86@kernel.org>,
	<kvm@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
	<peterz@infradead.org>, <chao.gao@intel.com>, <mlevitsk@redhat.com>,
	<john.allen@amd.com>
References: <20240219074733.122080-1-weijiang.yang@intel.com>
 <20240219074733.122080-25-weijiang.yang@intel.com>
 <ZjLNEPwXwPFJ5HJ3@google.com>
 <39b95ac6-f163-4461-93f3-eaa653ab1355@intel.com>
 <ZkYauRJBhaw9P1A_@google.com> <87r0e0ke8w.ffs@tglx>
 <ZkdpKiSyOwB3NwRD@google.com>
 <a170e420-efc3-47f9-b825-43229c999c0d@intel.com>
 <ZkuD1uglu5WFSoxR@google.com>
Content-Language: en-US
From: "Yang, Weijiang" <weijiang.yang@intel.com>
In-Reply-To: <ZkuD1uglu5WFSoxR@google.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?U0ZtdFJzam1yRjM5YXRHS1dYYmxPMU10UkJFRG0xRmk3dEdKRFh3M3l5R2xr?=
 =?utf-8?B?V2xRZEh3azh3R3JFVGMrTWo4T2IrK0MrWWNPRUtCMVVuK3VqMStJVk02L1dN?=
 =?utf-8?B?a3EvTkVrNU40RjJVUDhYNTNEOEtzSlJVYWZmRmtmVXZ0UTE0em01ZGs4c2JC?=
 =?utf-8?B?RkZheS9ETkwzOVM1Vzc0ZlowYkhDcVUwQXluOHczZzR1Z1NtQ2RoM1M5eVJo?=
 =?utf-8?B?R3hwUXcrOXhNVlc4ZFJLUzk4MUVrdk11RGIrRFVJU09JTDJxZlFMY2RIZFNt?=
 =?utf-8?B?VEVta1p6ckpDb3ppbmpraStnTXRHWG4rQ2pFVnZBR0o1Sk1sMGFCOU0rbGV5?=
 =?utf-8?B?Q3BWWDZJUEJXdHRkR3JkMWh0NGN2Z0hoUjM0L05FUEpsWHNqcCtFc2dZOUYx?=
 =?utf-8?B?UE5WS1lWU3FtL3E4L2lFQVY5aDZSbDF1SzdNNXhvQi9LOGRTOG5Gb0hDMVZH?=
 =?utf-8?B?TEFUb0JaOVY2eTJla0hwQi9WYTJWa2lVWExWMmk4azJkdTlObnROT0U5cTBt?=
 =?utf-8?B?cjRBV01odGljOFV2TitwV2toWkpaZ2Joa1ovSThSNERvZnY5QnhJdUhlMUUr?=
 =?utf-8?B?Nm1McFMweUpYNnVrMTN4Q20xNzF6amNRWE0zNC9XU2FjSnhUeEpaM0NIR29O?=
 =?utf-8?B?eHpPUVZqcHZvekI4ZE5JNWhNZ1BaaE5tc051Q0dmR093Qm5TOUNIWVNVbWoz?=
 =?utf-8?B?MW9DektQTS9jZmZZQTZNS0tCK1p3Q2F2NENPb0VZRmJQYmxlZHM1b0lrbVJM?=
 =?utf-8?B?TjNzeWZJYzEyNXVFMCtHdjhOZDdKWHovTkU5TWVaWldDZlNBTmdqVnJrNVpD?=
 =?utf-8?B?NVptMlhmcVBWY01lbXRUZU83enJiRzZHYWZoN0dlSEVTbzFRSU9VVXF2OC92?=
 =?utf-8?B?d3oxRUF0L3NoMm92aGxObG9OZWxmREs2QWs1WmdETXFhK3ozZnZrNi9RS1Y4?=
 =?utf-8?B?WStwanBkblhMeExxcUppeWZXVFUvWjBtUGIxa0h1aHRmc3Z2aTFObHpESVUy?=
 =?utf-8?B?MnF5VnM5QWJydWJSQzFmVFNPY3J1a2twTXNSRVY4U1VWdjN0b1hKejV0UFRU?=
 =?utf-8?B?NXgxUzVVQnZsdGVuMVNXb3ZucVJoVC9TQVhBaUZYeFV2L3RKTWgxZEJXL0Zw?=
 =?utf-8?B?WUdFajhZUmptQ1NQTHd4YW9kRkRCL3FqUStyNDB1WDVDQmtGTldEUGFUQXlO?=
 =?utf-8?B?R0JnbENYUFNWNnBkUWRnYkJ6Z1VsdCsrcEhwTXRuSGlCRVhZa2xyeVRzbDNz?=
 =?utf-8?B?NDFFT2p3eHhyM1loSldVUkhTajVZOEplRlVHdmI2ODBBK3k0TmUvS0NDNVFm?=
 =?utf-8?B?Y2c1U1VqVTZ3ZHJyVFZmOTZjUGtqTnFwNFI3QVZFTHhjSDBOYS9pUnJQTW0r?=
 =?utf-8?B?amhLR1ZKelJzNkJudXFDSFd5UHVHOE5TVitoRXBsU1M0dkhyKzBDS3UwMG5z?=
 =?utf-8?B?QkM2NmhMaGJHYjMzVmkweXJhaTQ4NGZ3TlBSYmpUUDduOHZnSE9aT3c3S0kv?=
 =?utf-8?B?QjUrb0ZwYWhLMnFTcUEweE9SemJmTVJrN0dCdnNZQlQ1blRnQUppTjJtRy96?=
 =?utf-8?B?RkZZcERZc2M3d0pNWWMzcUdOdmNhL2JFMkJUMmtCUTVZUGRYQWZOemlIOFNx?=
 =?utf-8?B?eGpCK2U5REpoZkd5ajl4VEE4RGJIZzJZUG1XMUk1VlhRdWdid09qLytGNklD?=
 =?utf-8?B?QU9NbDlYeDFoV244T3JrV245MGNqN1k3aWZ0bVN1RjBZOEtaeXJOeWhrSkx5?=
 =?utf-8?B?TkYrUEtxVFZkMVVEU0kyVktINEFldGxKdmdyb0FjVWhTSG9sS21OSFJKTXpY?=
 =?utf-8?B?NHp3b2d4RVpZUVdtME1HY3VQUzNMSzgvZmNHa2dzdzI0eTVabkpGVW5QT2NY?=
 =?utf-8?B?Y2hscXVtRGxMYWdOR0F6bFFpa3M1K0RGOE9oSDFRSkNKUmY4ajhRYmN1YldL?=
 =?utf-8?B?UHVvRkJ2WExLTVA3Z3g0TmtsQi9INk1PK0U3bTdzMU5FWGZjV1FzbTF3Z3pC?=
 =?utf-8?B?TTRlL29FRXNSWUNrZnpyUGhGNjJ5bU1zOHdaRmYrM3U0M3VsNFVJT1Q0ZmxT?=
 =?utf-8?B?a3NSYjh6UEFkcGNkTVA3N3hpR05VMS8zak16OWgwdWJQS1JTbFRieHFMRnJW?=
 =?utf-8?B?d0E1MFZOVmxRejNVaTR4K3ZQUy9wWXJvbXliUXlycHExQVJHd3EvbjhGWVBj?=
 =?utf-8?B?S3c9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 450f1c82-6452-4ec3-e16c-08dc7a3b0160
X-MS-Exchange-CrossTenant-AuthSource: SA2PR11MB4972.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 08:41:43.2488
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: doBewC2Kf8/KvzOCvXU+4WX7P9HoN19F6Qa40sULqKfxcIWBw0pvpKvV1dhDKBTI4g5M9jY1O6cI8m++pYmfdA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB4983
X-OriginatorOrg: intel.com

On 5/21/2024 1:09 AM, Sean Christopherson wrote:
> On Mon, May 20, 2024, Weijiang Yang wrote:
>> On 5/17/2024 10:26 PM, Sean Christopherson wrote:
>>> On Fri, May 17, 2024, Thomas Gleixner wrote:
>>>> On Thu, May 16 2024 at 07:39, Sean Christopherson wrote:
>>>>> On Thu, May 16, 2024, Weijiang Yang wrote:
>>>>>> We synced the issue internally, and got conclusion that KVM should honor host
>>>>>> IBT config.  In this case IBT bit in boot_cpu_data should be honored.  With
>>>>>> this policy, it can avoid CPUID confusion to guest side due to host ibt=off
>>>>>> config.
>>>>> What was the reasoning?  CPUID confusion is a weak justification, e.g. it's not
>>>>> like the guest has visibility into the host kernel, and raw CPUID will still show
>>>>> IBT support in the host.
>>>>>
>>>>> On the other hand, I can definitely see folks wanting to expose IBT to guests
>>>>> when running non-complaint host kernels, especially when live migration is in
>>>>> play, i.e. when hiding IBT from the guest will actively cause problems.
>>>> I have to disagree here violently.
>>>>
>>>> If the exposure of a CPUID bit to a guest requires host side support,
>>>> e.g. in xstate handling, then exposing it to a guest is simply not
>>>> possible.
>>> Ya, I don't disagree, I just didn't realize that CET_USER would be cleared in the
>>> supported xfeatures mask.
>> For host side support, fortunately,  this patch already has some checks for
>> that. But for userspace CPUID config, it allows IBT to be exposed alone.
>>
>> IIUC, this series tries to tie IBT to SHSTK feature, i.e., IBT cannot be
>> exposed as an independent feature to guest without exposing SHSTK at the same
>> time. If it is, then below patch is not needed anymore:
>> https://lore.kernel.org/all/20240219074733.122080-3-weijiang.yang@intel.com/
> That's a question for the x86 maintainers.  Specifically, do they want to allow
> enabling XFEATURE_CET_USER even if userspace shadow stack support is disabled.
>
> I don't think it impacts KVM, at least not directly.  Regardless of what decision
> the kernel makes, KVM needs to disable IBT and SHSTK if CET_USER _or_ CET_KERNEL
> is missing, which KVM already does via:
>
> 	if ((kvm_caps.supported_xss & (XFEATURE_MASK_CET_USER |
> 	     XFEATURE_MASK_CET_KERNEL)) !=
> 	    (XFEATURE_MASK_CET_USER | XFEATURE_MASK_CET_KERNEL)) {
> 		kvm_cpu_cap_clear(X86_FEATURE_SHSTK);
> 		kvm_cpu_cap_clear(X86_FEATURE_IBT);
> 		kvm_caps.supported_xss &= ~(XFEATURE_MASK_CET_USER |
> 					    XFEATURE_MASK_CET_KERNEL);
> 	}
>
>> I'd check and clear IBT bit from CPUID when userspace enables only IBT via
>> KVM_SET_CPUID2.
> No.  It is userspace's responsibility to provide a sane CPUID model for the guest.
> KVM needs to ensure that *KVM* doesn't treat IBT as supported if the kernel doesn't
> allow XFEATURE_CET_USER, but userspace can advertise whatever it wants to the guest
> (and gets to keep the pieces if it does something funky).

OK, I think we can go ahead to keep KVM patches as-is given the fact user IBT is not enabled in Linux.
I only hope other OSes can enforce both SHSTK and IBT dependency on XFEATURE_CET_USER so
that user IBT can work well there.

Then IBT can be exposed to guest alone because guest *kernel* IBT only relies on S_CET MSR  which is
VMCS auto-saved/restored.

What's your thoughts?