Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2;
Message-ID: <d1166177-c0ab-a8a5-94a6-e4e7ebdeb1c0@intel.com>
Date:   Fri, 3 Nov 2023 16:18:25 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.15.1
Subject: Re: [PATCH v6 19/25] KVM: VMX: Emulate read and write to CET MSRs
To:     Sean Christopherson <seanjc@google.com>
CC:     <pbonzini@redhat.com>, <kvm@vger.kernel.org>,
        <linux-kernel@vger.kernel.org>, <dave.hansen@intel.com>,
        <peterz@infradead.org>, <chao.gao@intel.com>,
        <rick.p.edgecombe@intel.com>, <john.allen@amd.com>,
        Maxim Levitsky <mlevitsk@redhat.com>
References: <20230914063325.85503-1-weijiang.yang@intel.com>
 <20230914063325.85503-20-weijiang.yang@intel.com>
 <d67fe0ca19f7aef855aa376ada0fc96a66ca0d4f.camel@redhat.com>
 <ZUJ9fDuQUNe9BLUA@google.com>
Content-Language: en-US
From:   "Yang, Weijiang" <weijiang.yang@intel.com>
In-Reply-To: <ZUJ9fDuQUNe9BLUA@google.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RU9aY3k1NjZBS1RYYjJjaHhJZmtZVGhBQ0RrNlVBbStyUjJzT2FSZXBxNDU1?=
 =?utf-8?B?MlZQbVg4M0FYWEJ0Y2VHRjVuMnphc0dVQVBrVFhUSnVlRVJjekFDOHFvbVVl?=
 =?utf-8?B?TXJCUGtCVHNCaHN1R1ptQkk3YU5PcGNvRXg0dWZzbXYvZkRxN3hKV1BUTmlP?=
 =?utf-8?B?eTdkTGRiNjJtd0M0RkdTRkNNTVJ1MUFWTy9MdURpSGpvOWlzRlFxQ1JabWth?=
 =?utf-8?B?OE1yZUVpYS9hbGtVaVFmOUNaNnFjRENiQjA3Q01KZStsN0VhcW1GOFdmMEs4?=
 =?utf-8?B?RGRyUFIrbURWbSs3SCs0N21UQ3RZU0Y0UG1Fd21vN3UySUQwNEo5UG94RkJU?=
 =?utf-8?B?Q0x0NDIzSTFwdUNNaTdPOFFlTTgzUkpVczFHSEZ4RUxHWU8ybnoxcnVycjI1?=
 =?utf-8?B?bWxlaldqekk5WEpHUzNnOU41enNvV0h0Z1JrVGZJaFpDaHNxY3l5TG1qZmlW?=
 =?utf-8?B?TGtxWnhFZ1FGVDRsNGVhRG1oSXRzV3U0eGNsVVdiV2lmeisxOUFPMDd1UFVz?=
 =?utf-8?B?Z294QXdKZDh1cDNzZGlROUY0bVBkQmZUMGpMRDlZR0NrZ2h6bkxDSEZLNXRY?=
 =?utf-8?B?VmZxdC93MElOd3V6ck85T1NBMmVzbjArU2RSdmhnSTdyVXJVdGFGRHpwbEVo?=
 =?utf-8?B?b1BueUNVUGlUUHplaWJadDgyM2Ixc1FldVNFUmRPZDhGdm56dERENWtjR0JC?=
 =?utf-8?B?K241ZUtpMGNJUkUyNTliMlFMalRwdUduYTJ1dHVVRkZQM0llRWNtYmswU0ZQ?=
 =?utf-8?B?SFBWeWlmV09tdHdwN25LOGlWQWhEUHRyL0djT3dhV0xCRFRJZy9od0x2TW0r?=
 =?utf-8?B?Ujk5NlM1VlVaM2J0THExUmQ4Y0NUNzdOSmZIc0xONGhYNFFVSFNHVTN6czcx?=
 =?utf-8?B?YnVPVTZMdFFlaWxFbFRWQkE0dVVVcWVsYXFQcm1kYWwyQVFWbVcrL0JHek5I?=
 =?utf-8?B?MU1DblQxNDQ3ZGU0WW9XUHQyaDdISGpneEFMd2tHNno0UXVYM3h0NWxvQlBZ?=
 =?utf-8?B?bXRMV1N3TGRjYWNVZXRuSmcwT0tEQU54cWlyMUJMSzNiTzFoamxDTlpmOGdX?=
 =?utf-8?B?SVUxRS9FOU9OYmdDcHRaQm54MytjQmRSRUhrYmhrVTNvU2xYUHMxVjBWVGRB?=
 =?utf-8?B?YlBGdnl6TTdXK2U1TXQweVRoU3JDVDBPM2VWNVp2VGpJaVZ2bzl2SmppRHpl?=
 =?utf-8?B?ZDNselJrbUxNYmtNalArTldPTjRNYkp6TDlua2xXR3NwMHR5MXJJTHJybito?=
 =?utf-8?B?Z2J6MElmbFZEZ1ZDSWdueVZ2U1lBaUdHSG82SDZ0aW9FdUJyNVlhZGtTbnpJ?=
 =?utf-8?B?NE1wbHZ6U2QrWFhvR2IxWUVodnlJRS9sa0JMTHVKZHIwb1ZwTWlRMzFQcnE3?=
 =?utf-8?B?cHE1akI4VklmRU1NSWFoMWtvMHRKendFbDhjZGcyTUtSZEhYejFlMlBrdmhG?=
 =?utf-8?B?RExUZWFiZHVZa29leHREaFhvRGZMdktRcy95RHhaUzgwcFQxZHk3Yk9vNWlR?=
 =?utf-8?B?aGtYZkRVeHY4bW9rb3FTays3b0FVbkFIRG9lbjFybnJVaXFHU3BQWGNNVnVj?=
 =?utf-8?B?alpjakhVV2o4VHk4aVJTYS93VnFQak85ZSsyc1lHZTQ1Z3ZJSTdjSmVnRmVP?=
 =?utf-8?B?bnNOTVIybys1OUgwTEc0Q0g5T2dwUy9ETmFlTHdMVDhXNnNvTWpTYVVOOFJv?=
 =?utf-8?B?bmQvblFaak9mZmRvZmJKOTZZQWVGdlZJd3BaaWlpQmRnUENQaENqRmlzT1o4?=
 =?utf-8?B?Zk1ReFl1OXpjMDA1MUducWpkbDc2UjR6T3g5czQzcUorMmx2N3p3NlNsUm1T?=
 =?utf-8?B?dGo2WlQ3Y29ZRlBGWXZJeUkxQTdBZVBkRjNqM2lBZmdjeDhsNFl0VDBUd21T?=
 =?utf-8?B?cU9nYU5pTGJ4Z3hHTkxpMWx1bXZZdmoxK3o3dHBWSmJ5TlMrNDdLRzJjYkRZ?=
 =?utf-8?B?eUkyZWZEM3N4UDdSeUZ1cEU2MUlrdkxCWFNGSW9LVElKQmp6ZkpwS1VQZnJS?=
 =?utf-8?B?WDM4dmxNWDE0bE5xdER6aTVsbFlzT3hJUElGamp0QUwzb1dxT24vREZ2M09D?=
 =?utf-8?B?Y04yWGJvU25HMGJ6WW1ZY2dDRlkvWFJseGtESU5qTnB5VFNoUUpTNklETWIr?=
 =?utf-8?B?ZkNLYVNhNUd3VENsRng3ak8xODhwUGV0Ym1ORHpYUjhYNitvdXFqN2dVNmNB?=
 =?utf-8?B?TXc9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 0faa447d-ab4b-4358-d905-08dbdc457907
X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB4965.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2023 08:18:34.6285
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: xtPbpZcZQznjpg52B3M5wrXUrRKzzP1+DHSzemILS6a9u7DM4UL6wOqqS371J3QckZQPQ2/7CpO50f5B/XDEzw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB6546
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-5.3 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,
	T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no
	version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Fri, 03 Nov 2023 01:19:09 -0700 (PDT)

On 11/2/2023 12:31 AM, Sean Christopherson wrote:
> On Tue, Oct 31, 2023, Maxim Levitsky wrote:
>> On Thu, 2023-09-14 at 02:33 -0400, Yang Weijiang wrote:
>>> Add emulation interface for CET MSR access. The emulation code is split
>>> into common part and vendor specific part. The former does common check
>>> for MSRs and reads/writes directly from/to XSAVE-managed MSRs via the
>>> helpers while the latter accesses the MSRs linked to VMCS fields.
>>>
>>> Signed-off-by: Yang Weijiang <weijiang.yang@intel.com>
>>> ---
> ...
>
>>> +	case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP:
>>> +	case MSR_KVM_SSP:
>>> +		if (host_msr_reset && kvm_cpu_cap_has(X86_FEATURE_SHSTK))
>>> +			break;
>>> +		if (!guest_can_use(vcpu, X86_FEATURE_SHSTK))
>>> +			return 1;
>>> +		if (index == MSR_KVM_SSP && !host_initiated)
>>> +			return 1;
>>> +		if (is_noncanonical_address(data, vcpu))
>>> +			return 1;
>>> +		if (index != MSR_IA32_INT_SSP_TAB && !IS_ALIGNED(data, 4))
>>> +			return 1;
>>> +		break;
>> Once again I'll prefer to have an ioctl for setting/getting SSP, this will
>> make the above code simpler (e.g there will be no need to check that write
>> comes from the host/etc).
> I don't think an ioctl() would be simpler overall, especially when factoring in
> userspace.  With a synthetic MSR, we get the following quite cheaply:
>
>   1. Enumerating support to userspace.
>   2. Save/restore of the value, e.g. for live migration.
>   3. Vendor hooks for propagating values to/from the VMCS/VMCB.
>
> For an ioctl(), #1 would require a capability, #2 (and #1 to some extent) would
> require new userspace flows, and #3 would require new kvm_x86_ops hooks.
>
> The synthetic MSR adds a small amount of messiness, as does bundling
> MSR_IA32_INT_SSP_TAB with the other shadow stack MSRs.  The bulk of the mess comes
> from the need to allow userspace to write '0' when KVM enumerated supported to
> userspace.
>
> If we isolate MSR_IA32_INT_SSP_TAB, that'll help with the synthetic MSR and with
> MSR_IA32_INT_SSP_TAB.  For the unfortunate "host reset" behavior, the best idea I
> came up with is to add a helper.  It's still a bit ugly, but the ugliness is
> contained in a helper and IMO makes it much easier to follow the case statements.

Frankly speaking, existing code is not hard to understand to me :-), the handling for MSR_KVM_SSP
and MSR_IA32_INT_SSP_TAB is straightforward if audiences read the related spec.
But I'll take your advice and enclose below changes. Thanks!
> get:
>
> 	case MSR_IA32_INT_SSP_TAB:
> 		if (!guest_can_use(vcpu, X86_FEATURE_SHSTK) ||
> 		    !guest_cpuid_has(vcpu, X86_FEATURE_LM))
> 			return 1;
> 		break;
> 	case MSR_KVM_SSP:
> 		if (!host_initiated)
> 			return 1;
> 		fallthrough;
> 	case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP:
> 		if (!guest_can_use(vcpu, X86_FEATURE_SHSTK))
> 			return 1;
> 		break;
>
> static bool is_set_cet_msr_allowed(struct kvm_vcpu *vcpu, u32 index, u64 data,
> 				   bool host_initiated)
> {
> 	bool any_cet = index == MSR_IA32_S_CET || index == MSR_IA32_U_CET;
>
> 	if (guest_can_use(vcpu, X86_FEATURE_SHSTK))
> 		return true;
>
> 	if (any_cet && guest_can_use(vcpu, X86_FEATURE_IBT))
> 		return true;
>
> 	/*
> 	 * If KVM supports the MSR, i.e. has enumerated the MSR existence to
> 	 * userspace, then userspace is allowed to write '0' irrespective of
> 	 * whether or not the MSR is exposed to the guest.
> 	 */
> 	if (!host_initiated || data)
> 		return false;
> 	if (kvm_cpu_cap_has(X86_FEATURE_SHSTK))
> 		return true;
>
> 	return any_cet && kvm_cpu_cap_has(X86_FEATURE_IBT);
> }
>
> set:
> 	case MSR_IA32_U_CET:
> 	case MSR_IA32_S_CET:
> 		if (!is_set_cet_msr_allowed(vcpu, index, data, host_initiated))
> 			return 1;
> 		if (data & CET_US_RESERVED_BITS)
> 			return 1;
> 		if (!guest_can_use(vcpu, X86_FEATURE_SHSTK) &&
> 		    (data & CET_US_SHSTK_MASK_BITS))
> 			return 1;
> 		if (!guest_can_use(vcpu, X86_FEATURE_IBT) &&
> 		    (data & CET_US_IBT_MASK_BITS))
> 			return 1;
> 		if (!IS_ALIGNED(CET_US_LEGACY_BITMAP_BASE(data), 4))
> 			return 1;
>
> 		/* IBT can be suppressed iff the TRACKER isn't WAIT_ENDBR. */
> 		if ((data & CET_SUPPRESS) && (data & CET_WAIT_ENDBR))
> 			return 1;
> 		break;
> 	case MSR_IA32_INT_SSP_TAB:
> 		if (!guest_cpuid_has(vcpu, X86_FEATURE_LM))
> 			return 1;
>
> 		if (is_noncanonical_address(data, vcpu))
> 			return 1;
> 		break;
> 	case MSR_KVM_SSP:
> 		if (!host_initiated)
> 			return 1;
> 		fallthrough;
> 	case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP:
> 		if (!is_set_cet_msr_allowed(vcpu, index, data, host_initiated))
> 			return 1;
> 		if (is_noncanonical_address(data, vcpu))
> 			return 1;
> 		if (!IS_ALIGNED(data, 4))
> 			return 1;
> 		break;
> 	}