Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <a8bae991-c9af-ef45-20de-b9369f4dbf09@amd.com>
Date:   Mon, 26 Jun 2023 11:45:56 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.11.0
Subject: Re: [RFC PATCH v2 6/6] KVM: SVM: Add CET features to supported_xss
Content-Language: en-US
To:     Sean Christopherson <seanjc@google.com>
Cc:     Rick P Edgecombe <rick.p.edgecombe@intel.com>,
        "john.allen@amd.com" <john.allen@amd.com>,
        Weijiang Yang <weijiang.yang@intel.com>,
        "bp@alien8.de" <bp@alien8.de>, "x86@kernel.org" <x86@kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "pbonzini@redhat.com" <pbonzini@redhat.com>,
        "andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>
References: <20230524155339.415820-1-john.allen@amd.com>
 <20230524155339.415820-7-john.allen@amd.com>
 <161174d013dff42ddfd2950fe33a8054f45c223e.camel@intel.com>
 <ZINGaJnNJ54+klsD@johallen-workstation.lan>
 <9ef2faeaa38e667bd4daa8ee338d4cade452c76c.camel@intel.com>
 <ZJYaNSzup+yuYxNy@google.com> <52ea8386-8652-dd91-23de-9d35781cb131@amd.com>
 <ZJm64a7IKuSDS9bz@google.com>
From:   Tom Lendacky <thomas.lendacky@amd.com>
In-Reply-To: <ZJm64a7IKuSDS9bz@google.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?WGFwenZTcG4ydmlGbGdkNWQyMVI2c2pacXZjYUoxL0JrUFJibzIyd1dZN0R1?=
 =?utf-8?B?ZUMvWmhuclpULzJoUzE3b3A3Nk1hbDV2ckU5ckdaSmdEdHk4SjZGQWZ1RzBn?=
 =?utf-8?B?TTlRV25Rc0pwTlc3RDBvaHdMZ2FYZG4yaGZNc1QvclVTNWpWWGViTlB0eW5E?=
 =?utf-8?B?bTdMS3R1SzZRcytwbHNPNElrQjZsbDhPQ1ZKL00vSHBXZEI2MU1FcTUyMFFs?=
 =?utf-8?B?WVhGZVdKd0NTdERzN1kyMlRVd1E1dFVCN3ZUVURwWE1Zak1sRVZaL2R1RUd4?=
 =?utf-8?B?azBCMlpxRHJyaGgxaFZwWUJmVGJ0UnhHM1ZqakxaSTEvdVJ3eHJDNWNIMDZB?=
 =?utf-8?B?TWhZQ1JGdjZGZ29aUmFudHJodVlZSTRMamI3TkR6TUZOU3cyR1lZbUg4RmEx?=
 =?utf-8?B?REYwM0V1NlVDeDVvQXFoWktwQmJmZUdsWU1CZmxWZS9OTFNrOGlYWTZjM0RW?=
 =?utf-8?B?dG41SWtodFZ1UW9pdEI5TlZFWDVZRlFaQlFVV1JkOHlZMVlNSTU4MDFnZ0Rh?=
 =?utf-8?B?TGVSOWdVUU9vZTRmdWZTZFJMTExzV1QvWldrL1A2MGVtNytlcFdoR1UrZ1ZJ?=
 =?utf-8?B?R20xbE8wRXlRdy9hL1VYMERrWGpzNTFUaEgyVDdtQXZkTHVJLy9HVW9VeUp2?=
 =?utf-8?B?eDI1M3ZEMjZLcjFrUHdhd0Z6OU1VL0VxMng2R1dSekV1UGxwR0RkQmNReTFk?=
 =?utf-8?B?QjEwejdzc1lVeDRqTEpuV3IwdVhYcHJwS1Vyc0hEQ1NpUnZiNnFSZy96SlAv?=
 =?utf-8?B?U0V1bGxlbS9mN0NpZ2thdXBFeFBidUJLazdGRDcweEJlREJwZTFMRG5sU0VN?=
 =?utf-8?B?T1RiV040SkJPYWhCRVBiMWJFMWJFSkk1M2VJMDBlaWJScGM2MWkyMnBidkhJ?=
 =?utf-8?B?NGU5ZGRNUDlqcU9qdzE3Y2Z2bVNRelNVTnJMRllTL3JPbSt0K09TVGdpOVRa?=
 =?utf-8?B?bDVCcTZNeHpzcVlpUVg3aTRNZTVaRmRDeXY0ZzdxcS93dklER0hLbkpyWVFh?=
 =?utf-8?B?YTBWODA4bnBYRUJzZFdZNUIzZ2ZwSkc2SzVsQWNUVDRZT0VUeHBoRHk2SkdM?=
 =?utf-8?B?UGdBaTFJS0cvN2xXY0xSTWxYcVE2bkUzdVEvd21hdkZIQ3dvbWJwRkl1c2Vx?=
 =?utf-8?B?YTVZNDMzM051N0NsalZrSjFRUThNWUoyMXFYR2FjaVYxalZuUGFsY1JJUElS?=
 =?utf-8?B?aHgxcXRqMU5Ic1NITFhUYWJZZ2VxNFJ5YWFSbmE2eDFRZDFXNlEzYUxBaXhB?=
 =?utf-8?B?eUNVR0hRMWtSdHFYS1FsS3BCdjh1ZW04L0p6eGoycWw0UHJ5SWlOZEJyY28w?=
 =?utf-8?B?MlRhT2dkMCtUWDdBNm0rQWVpYS9CRGlXTTl0RmpkZWcyZnZrUUp2dm12Vi8x?=
 =?utf-8?B?eHNDbzBtdFF6R1VOWGQzaFRHYlBBM0JZa0dhRXkxYnFXZmgzNWFHdnhuaGVy?=
 =?utf-8?B?bHBEd2VGSFpNMXZreUpmNU5leWhKZXhqaUI0TW9wSE9pK3kzQ3FuMUp2cHlq?=
 =?utf-8?B?Um4ydDY5NzVIbmxDTXU0eXJSVjZmQmRPOE4zMTFOZ1JrTFBmRjlvK2hKY2k2?=
 =?utf-8?B?ZHNkMTBSVG92RVlGWkJpSlJJU1lRekRtRVVvNkR3cktkTndIeFA2RldISUFS?=
 =?utf-8?B?OEl6OERDbmhIb0pmLzlHMlVGbXY0U3hXWTBBUGJpSjI2K1c2RVNBOFowSVhq?=
 =?utf-8?B?eE1TR1I4L3ZmcUxZQW5UZU9CUWdsM3dZcHBGUUczK2wrWWZZdFV0Q2ljR0FU?=
 =?utf-8?B?bFNYdGNaY3h5L1RiL29HMmpUVlFDYnU2VEdmSko1QXNhaFllakNGLzQ2cldG?=
 =?utf-8?B?TVFjcC9jTUdhRXhZVTdaVGJWK2VjZThjTXFXb1lmb1ROaTZDU2xLYlNHMFk0?=
 =?utf-8?B?dTRXNkdYRW01aGFHK0VNUnBuSXpMZzF2RGZpV252MDk3VWdaVEhDZU1kRDkr?=
 =?utf-8?B?QzlEUnB1QjZxQWNidUxMNHRyUUlQUXl4WndTUVdlb21rdnpoSUk2R0w2cndN?=
 =?utf-8?B?UFIvSGk0S2FDSVgxUzJZcHNQK25ySHpIR0pTdU95ZHUwQVZEK2dDVUdUMzBm?=
 =?utf-8?B?RUJjdVdrTk9PcjJJVU9scm0xRGRES05hNUpWTjhNQXNCWGlOWHpRaFY1TDNk?=
 =?utf-8?Q?bjZsvwDo1888hfOOmVyJRtaUc?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c5ed6f49-26de-4318-d677-08db7664d1eb
X-MS-Exchange-CrossTenant-AuthSource: DM4PR12MB5229.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Jun 2023 16:45:59.5098
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: pHMUER76sdzDAAdAm70oyVUSdFyo/2RxjSZqtWX2f+2V6TxDz8d5Vkb41KDeeA1BUziZbVqgMEpUUApLMfxFQQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB7458
X-Spam-Status: No, score=-1.2 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,NICE_REPLY_A,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 6/26/23 11:28, Sean Christopherson wrote:
> On Mon, Jun 26, 2023, Tom Lendacky wrote:
>> On 6/23/23 17:18, Sean Christopherson wrote:
>>> On Fri, Jun 09, 2023, Rick P Edgecombe wrote:
>>>> Also, since the host might have CR4.CET set for its own reasons, if the host
>>>> handled an exit with the the guests MSR_IA32_S_CET set it could suddenly be
>>>> subjected to CET enforcement that it doesn't expect. Waiting to restore it
>>>> until returning to the guest is too late.
>>>>
>>>> At least that's the reasoning on the VMX side as I understand it
>>>
>>> The APM doesn't come right out and say it, but I assume/hope that S_CET is saved
>>> on VMRUN and loaded on #VMEXIT, i.e. is the same as VMX for all intents and
>>> purposes.
>>>
>>> The host save state definitely has a field for S_CET, and VMRUN documents that the
>>> guest values are loaded, I just can't find anything in the APM that explicitly states
>>> how host S_CET and friends are handled.  E.g. in theory, they could have been
>>> shoved into VMSAVE+VMLOAD, though I very much doubt that's the case.
>>
>> Yes, the host value is saved/restored on VMRUN/#VMEXIT. Anything that is in
>> the VMCB Save Area (the non-SEV-ES save area) is fully virtualized (unless
>> noted otherwise) and doesn't require special processing to save/restore the
>> host values.
> 
> Would it makes sense to add a column in "Table B-2. VMCB Layout, State Save Area"
> to specify whether a field is handled by VMRUN+#VMEXIT vs. VMLOAD+VMSAVE?  I can't
> find anywhere in the APM where it explicitly states that VMRUN+#VMEXIT context
> switches everything in the Save Area except the fields listed in "15.5.2 VMSAVE
> and VMLOAD Instructions".
> 
> "15.5 VMRUN Instruction" kinda sorta covers that behavior, but the information is
> either incomplete or stale, e.g. for host state it says "at least the following"
> 
>    Saving Host State. To ensure that the host can resume operation after #VMEXIT,
>    VMRUN saves at least the following host state information:
> 
> but for guest state it says "the following"
> 
>    Loading Guest State. After saving host state, VMRUN loads the following guest
>    state from the VMCB:
> 
> and then both provide incomplete lists of state.  A pedantic reading of the guest
> case suggests that there's a large pile of state that *isn't* loaded, and the host
> case isn't all that helpful because it's way too handwavy.

I'll communicate this feedback to the folks that update the APM volumes 
and see what can be done.

Thanks,
Tom