Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5;
Message-ID: <e79d43ea-3ca0-44ca-9a55-b8e2c5094cf2@intel.com>
Date:   Fri, 1 Dec 2023 17:45:53 +0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v7 22/26] KVM: VMX: Set up interception for CET MSRs
Content-Language: en-US
To:     Maxim Levitsky <mlevitsk@redhat.com>
CC:     <pbonzini@redhat.com>, <seanjc@google.com>, <peterz@infradead.org>,
        <chao.gao@intel.com>, <rick.p.edgecombe@intel.com>,
        <john.allen@amd.com>, <linux-kernel@vger.kernel.org>,
        <kvm@vger.kernel.org>, <dave.hansen@intel.com>
References: <20231124055330.138870-1-weijiang.yang@intel.com>
 <20231124055330.138870-23-weijiang.yang@intel.com>
 <393d82243b7f44731439717be82b20fbeda45c77.camel@redhat.com>
From:   "Yang, Weijiang" <weijiang.yang@intel.com>
In-Reply-To: <393d82243b7f44731439717be82b20fbeda45c77.camel@redhat.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?d29SaDdRQmt2cW1FUTZyZ1RLZzdscWdrSTcwU0kzekdIV2sra0wzSUVaSmNp?=
 =?utf-8?B?eW9yUWRKbDFpWk84MFBsemhMVHlFdWJHT01iWmJkS0xSSlYveTZSWXYvSnh1?=
 =?utf-8?B?Vy84VFU1YVJ4NldvZDYzUjNUV09PVFpBeWYwMFNHMjFsNFJLdXlXMWdXZEhw?=
 =?utf-8?B?KzFPUTBBVGNrdVNaZVJPcWVNVW8yTXEvZ2pkQ3ZQT3ozUFA3SFNTWEtmakVL?=
 =?utf-8?B?MEx5eFJoc2pIdGxLaDJXUTdETXZvdG1qdThkanBtcjZEYnEyMG9zMnhoT1ln?=
 =?utf-8?B?akFwTWMzc3dYcFFTZ1kxTU9QcElQaWNRNEFibnlIdTRHb0ZXL0t1ajM4bnQy?=
 =?utf-8?B?aXRzMnF6aVNDZkFaZkNJWlRWSmZmKzkrL2ZSQmk1YnFNNkRnbW9UKzd5YzYz?=
 =?utf-8?B?elFqRW1EQmcrNlpSREtBNEpLSzBXTmJ1R1ViOG45NjFQTDJqOG1rSlA2OXln?=
 =?utf-8?B?ZEphRXM3eGFmcG9NOUZONVRZT2RrUmpyczRJdkJLRmZPaWpkTmRpQjd6ZERH?=
 =?utf-8?B?aWdkVGhKQks2SEs0Q3h0WTg0cDZtdHcrZVhaL28xRHM3K0o5YkR3RUFlNFZq?=
 =?utf-8?B?WGMxNjd3cHZ1ajlTcW9PaGpSMm5Beit2RXBVcEUyUFA2aHFGbnNpYUtTVGM0?=
 =?utf-8?B?QXRySzVuT2xBRmpqRTdMRDFHSjJxc3JOdk5DeFVkT2Q3NFZKUERIMXVERkpK?=
 =?utf-8?B?SmJ3SEpINzFoNENNelExSXNRLy9Ka1loL0YrYWs1cDlWdDJkeDdRNXNRQXVi?=
 =?utf-8?B?V0dvVlZjaC96TXJsY2laSFg5em40MzZjQ1llOXMzenE0cFE0MC9JemJOSzBO?=
 =?utf-8?B?QlE5ajRMVi9lVGRSdnBTRy9IaTFTck9sdkFmTk9TVXlQNlFnLzIvbW1vUnRw?=
 =?utf-8?B?V2RtNjZMZjU0N1BzOTBlUXVYTVIzUUF1OHZDTzJjNW42bW8wbFRPY1QxRXQx?=
 =?utf-8?B?c0NhR0ZBbDg0UFlUR2pQZ1BYWEVtSkFhUDNaS2pKa2Rhb0I3UktNU3N3Zmhu?=
 =?utf-8?B?RFJiSVRmR2tNSmZlVXYybFBIOC9BREV2Tmw0SnRDVDA5OGJZSFZJdVV2dUI0?=
 =?utf-8?B?RU56QVlXWGQ2T3ZiVnFsVWVVNVBPdHlmSzZCT1VvVm1mTWtkaG1YTnJaRVEr?=
 =?utf-8?B?bDFuSVM1dWpQZnByQjkvMjNtdlVSSFRXblZDNWFxV0lZRXlwVHl6OExUQ29K?=
 =?utf-8?B?WjZEL2dwMEVJZ3VoSGZTTEpvNVBhaDJqM3pzQXZGeEJCZ1lwMll3VTBuYnpl?=
 =?utf-8?B?ak90Q29QeU9vMHUvaWpMT2sySmJMVFVEUHhTWElxL2dSbUlNRFpZUGoxTTVX?=
 =?utf-8?B?RTlGeVFuc3ZJSk5UUVFoSzAyanFxOUZ2UHliMlZiTkJibEp4MzVSWEI3QXEr?=
 =?utf-8?B?TnB5UmpKeThsRlR5VUQ0aVBCMFo0Z0FFVHhqZGFrdWg5QTZ6QzN5dHQyVnFv?=
 =?utf-8?B?V2kxQUFPQzdSbDh6OGZ2M3Q2ZXZjeFM5d2crUzlDRlJXdVduQlVSUzYwZjBh?=
 =?utf-8?B?S0ZPN2pxZFhSbnlaN3F3MUxhMlNhazZyTXFlQUpiMmhLL1NKampnS3MwY2xK?=
 =?utf-8?B?dEFObDB3UWxPME9MTXFOMHQ2dzNvQ0hQaDRrRlgvbkJ2K1pJSlR0cWtOYndQ?=
 =?utf-8?B?TVdxSlpvN3kwOG5nWTRnOTJMVUpEQ1dyRks1R1dsaUJDSXBJVWl6TjFhTGR4?=
 =?utf-8?B?VWcyMTFiTGtGSFRYd3V6UjNrYkQvdURaQzdMcEJmR3NVTFNxeXYwanBvUWlZ?=
 =?utf-8?B?Q0ZyRkJwdkRyeEVHcndLTFIxb1NqZDlxMUt6U01Na2lGWHpUa095VFlPZlk5?=
 =?utf-8?B?eHdpcGFibzVtWThFeUR1VjFNa2F3OHFKNHl0bWZoS0pCeUw1MnlRMWN4M0FY?=
 =?utf-8?B?TlRQWkZVekRYOHUzMGNuU1ZsT1Q0NEJaSTlpcTRNM2xsL1hIaThuVkgrRXBG?=
 =?utf-8?B?NFBxRVppcWRrOSt4UkZNNmtaT1kwc0huWTNQOUgzUlFZVkNaYzkrTnErejA2?=
 =?utf-8?B?VEo1anB6bEtHL1RMR3FlcmlNbUpiZU9sUmVBWE5xNXZUaWorWEZnai9iczl3?=
 =?utf-8?B?UFd5ZHpsK3NJem04M2c2UUE1V29TZzFoZWZiM1hHMEVSRzJqOXRxdjZPVkxl?=
 =?utf-8?B?Q2w0VmFjWXEzRjdjelB0Z2d6OTRYVENCcFl1dlkvRk15TFVhN3E1VTZUYXcx?=
 =?utf-8?B?V0E9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 84bd25d1-04f7-45df-b07e-08dbf252551c
X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB4965.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Dec 2023 09:46:03.7208
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: MU4wOuUilxrDVvML+XBJetfL/xN5eWR3BhFYFxtpZ7QaNKsqzeC0wErKVXW6F1SFiJMsqRZ+0aPZZsme9JpEng==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR11MB6823
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Fri, 01 Dec 2023 01:46:29 -0800 (PST)

On 12/1/2023 1:44 AM, Maxim Levitsky wrote:
> On Fri, 2023-11-24 at 00:53 -0500, Yang Weijiang wrote:
>> Enable/disable CET MSRs interception per associated feature configuration.
>> Shadow Stack feature requires all CET MSRs passed through to guest to make
>> it supported in user and supervisor mode while IBT feature only depends on
>> MSR_IA32_{U,S}_CETS_CET to enable user and supervisor IBT.
>>
>> Note, this MSR design introduced an architectural limitation of SHSTK and
>> IBT control for guest, i.e., when SHSTK is exposed, IBT is also available
>> to guest from architectual perspective since IBT relies on subset of SHSTK
>> relevant MSRs.
>>
>> Signed-off-by: Yang Weijiang <weijiang.yang@intel.com>
>> ---
>>   arch/x86/kvm/vmx/vmx.c | 42 ++++++++++++++++++++++++++++++++++++++++++
>>   1 file changed, 42 insertions(+)
>>
>> diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
>> index 554f665e59c3..e484333eddb0 100644
>> --- a/arch/x86/kvm/vmx/vmx.c
>> +++ b/arch/x86/kvm/vmx/vmx.c
>> @@ -699,6 +699,10 @@ static bool is_valid_passthrough_msr(u32 msr)
>>   	case MSR_LBR_CORE_TO ... MSR_LBR_CORE_TO + 8:
>>   		/* LBR MSRs. These are handled in vmx_update_intercept_for_lbr_msrs() */
>>   		return true;
>> +	case MSR_IA32_U_CET:
>> +	case MSR_IA32_S_CET:
>> +	case MSR_IA32_PL0_SSP ... MSR_IA32_INT_SSP_TAB:
>> +		return true;
>>   	}
>>   
>>   	r = possible_passthrough_msr_slot(msr) != -ENOENT;
>> @@ -7766,6 +7770,42 @@ static void update_intel_pt_cfg(struct kvm_vcpu *vcpu)
>>   		vmx->pt_desc.ctl_bitmask &= ~(0xfULL << (32 + i * 4));
>>   }
>>   
>> +static void vmx_update_intercept_for_cet_msr(struct kvm_vcpu *vcpu)
>> +{
>> +	bool incpt;
>> +
>> +	if (kvm_cpu_cap_has(X86_FEATURE_SHSTK)) {
>> +		incpt = !guest_cpuid_has(vcpu, X86_FEATURE_SHSTK);
>> +
>> +		vmx_set_intercept_for_msr(vcpu, MSR_IA32_U_CET,
>> +					  MSR_TYPE_RW, incpt);
>> +		vmx_set_intercept_for_msr(vcpu, MSR_IA32_S_CET,
>> +					  MSR_TYPE_RW, incpt);
>> +		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PL0_SSP,
>> +					  MSR_TYPE_RW, incpt);
>> +		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PL1_SSP,
>> +					  MSR_TYPE_RW, incpt);
>> +		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PL2_SSP,
>> +					  MSR_TYPE_RW, incpt);
>> +		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PL3_SSP,
>> +					  MSR_TYPE_RW, incpt);
>> +		if (guest_cpuid_has(vcpu, X86_FEATURE_LM))
>> +			vmx_set_intercept_for_msr(vcpu, MSR_IA32_INT_SSP_TAB,
>> +						  MSR_TYPE_RW, incpt);
>> +		if (!incpt)
>> +			return;
>> +	}
>> +
>> +	if (kvm_cpu_cap_has(X86_FEATURE_IBT)) {
>> +		incpt = !guest_cpuid_has(vcpu, X86_FEATURE_IBT);
>> +
>> +		vmx_set_intercept_for_msr(vcpu, MSR_IA32_U_CET,
>> +					  MSR_TYPE_RW, incpt);
>> +		vmx_set_intercept_for_msr(vcpu, MSR_IA32_S_CET,
>> +					  MSR_TYPE_RW, incpt);
>> +	}
>> +}
>> +
>>   static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
>>   {
>>   	struct vcpu_vmx *vmx = to_vmx(vcpu);
>> @@ -7843,6 +7883,8 @@ static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
>>   
>>   	/* Refresh #PF interception to account for MAXPHYADDR changes. */
>>   	vmx_update_exception_bitmap(vcpu);
>> +
>> +	vmx_update_intercept_for_cet_msr(vcpu);
>>   }
>>   
>>   static u64 vmx_get_perf_capabilities(void)
> My review feedback from the previous patch still applies as well,
>
> I still think that we should either try a best effort approach to plug
> this virtualization hole, or we at least should fail guest creation
> if the virtualization hole is present as I said:
>
> "Another, much simpler option is to fail the guest creation if the shadow stack + indirect branch tracking
> state differs between host and the guest, unless both are disabled in the guest.
> (in essence don't let the guest be created if (2) or (3) happen)"
>
> Please at least tell me what do you think about this.

Oh, I thought I had replied this patch in v6 but I failed to send it out!
Let me explain it a bit, at early stage of this series, I thought of checking relevant host
feature enabling status before exposing guest CET features, but it's proved
unnecessary and user unfriendly.

E.g., we frequently disable host CET features due to whatever reasons on host,  then
the features cannot be used/tested in guest at all.  Technically, guest should be allowed
to run the features so long as the dependencies(i.e., xsave related support) are enabled
on host and there're no risks brought up by using of the features in guest.

I think cloud-computing should share the similar pain point when deploy CET into virtualization
usages.