Received-SPF: pass (google.com: domain of linux-kernel+bounces-95139-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Date: Thu, 7 Mar 2024 16:32:16 +0800
From: Chen Yu <yu.c.chen@intel.com>
To: <isaku.yamahata@intel.com>
CC: <kvm@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
	<isaku.yamahata@gmail.com>, Paolo Bonzini <pbonzini@redhat.com>,
	<erdemaktas@google.com>, Sean Christopherson <seanjc@google.com>, Sagi Shahar
	<sagis@google.com>, Kai Huang <kai.huang@intel.com>, <chen.bo@intel.com>,
	<hang.yuan@intel.com>, <tina.zhang@intel.com>
Subject: Re: [PATCH v19 080/130] KVM: TDX: restore host xsave state when exit
 from the guest TD
Message-ID: <Zel7kFS31SSVsSaJ@chenyu5-mobl2>
References: <cover.1708933498.git.isaku.yamahata@intel.com>
 <2894ed10014279f4b8caab582e3b7e7061b5dad3.1708933498.git.isaku.yamahata@intel.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <2894ed10014279f4b8caab582e3b7e7061b5dad3.1708933498.git.isaku.yamahata@intel.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?imLSun0Xw7GaSaabQHiXs+fT1mQJ70vOoNnimRjbqd99y4HWQZFp0W6ue1+e?=
 =?us-ascii?Q?WPIBoYdlEU5lMNK09pGGPbbMAsWFfMQwNPBZ8wZagp/NGrxClJNO0EKCxcr+?=
 =?us-ascii?Q?IK2bqDFYS5l0v+dZI3HT3dgbLeZQJpwuEXr4Lq0TNT57HrJdfgbk3sM+bD8Q?=
 =?us-ascii?Q?lESeNR9Ssc2/KKMaCSj7MJ/EqC83/tzrIdr2jkM0gC9VQLbc5iqWOvTkB4Gy?=
 =?us-ascii?Q?HOEj5yJN3K3ihDUuayngDqDvm5RFk1GE+5DmeqvN98Ff5GGIIkbWNStZh8ya?=
 =?us-ascii?Q?FrZQhy9mEx7B7siToq3oW13KLEYgF220BJSI2pV4iiLwIHEtaiHRYF8Oj3cr?=
 =?us-ascii?Q?02CXJjinWpCL3gRpVc/jxVAhnw72AhsujjXXx8j7KGl6KtbJq+ahDlUs9vVl?=
 =?us-ascii?Q?2J7Sx3mK2n+cAxzrZFgu8cQmE0rp6nrvz+9OVJZsWd1kw/b4YBKPDLXCk57h?=
 =?us-ascii?Q?78L9Cl59CvNDRvsPEV6/L7rfXgDXBZMv0caFYayPPQmTsAldUk3AZ76Sg4QA?=
 =?us-ascii?Q?rap/qf+/DtyKr62r3r0jCIR6n838E0hA/JTjD1kJKKLevMY0U8PRjur3UNar?=
 =?us-ascii?Q?omSgb1sZE9PSjuzKNsBzbHGKmmIunAFQ6BShlR+zDvZJWycAcNfuGu1MKVH2?=
 =?us-ascii?Q?Nv9lqr7Xh5xxXAA1OIk1+01u7k5WDITbxOQB3DmBIgpIiffTQBNWdNCW99O/?=
 =?us-ascii?Q?DFAUcaix3d2Acix+kfX4QsfHKSI+aJgP/iU7dH1G4juQivn009Sw0xufOLkf?=
 =?us-ascii?Q?BfDiMpVmbYlccMDXb8c0fFg1sL40g4f/LKp96joRqwmYKUfaRg70WpaiEGg3?=
 =?us-ascii?Q?GsKCTnGZnH4wtraidfzTVTp/l0EynJtFWouUkpPQxKN99IbezPk0BRThAz6d?=
 =?us-ascii?Q?nbXJi79ARazxlaQxZ1HYmUkV+EYcwd7AH0PFpWRAnsTi+3geu59nN81RG2Lu?=
 =?us-ascii?Q?AdeYr01AsVLanXAQ4dfZDzhg/DlcMfO32Xoi5b6gy2zSILKlSe3ahScQ61v9?=
 =?us-ascii?Q?28poPgs5fJU5djjQ5nRt800asx0hJPWxREUa6U7SRUZILmEAqPKwz4mIOLt/?=
 =?us-ascii?Q?YmcHkjY4K47VPOo1r09Tr2OhLVDNu4mv2qhwYytaKq3jZLFRkz8c38Qy38LT?=
 =?us-ascii?Q?aj+oCBxWhHGA+VgssfRnI/xot5pXANPrbcZigUbwuluprxQ4wjTncsfc/IWT?=
 =?us-ascii?Q?Hq5fnTXqxldlo1C6SyjuuC+AW20jo0xxU0iJbeWeG1/+2RroF/2fj22qgxOa?=
 =?us-ascii?Q?F2eUyNGRnQc0ESPYBcTBgQuPX1hrFYn4cp0FN2gYM0K+jSGLSvtOS6Nk21an?=
 =?us-ascii?Q?RpWKyMdWpZwhNV8FZtDvV4m/I5ks5BVXtttf7FixOeEf65OnGp1CgmPmwUgo?=
 =?us-ascii?Q?H7raagfq8rKoET04ENUkDBIMsxd3vLsW97NYmquy8G6bq5RoCq7qLkcjlCCR?=
 =?us-ascii?Q?lmDMOwZ38NGumqNAi4ltrYU98Q8NMMMSRkWNIzzczMLwEaWbx1st6PzjYSlB?=
 =?us-ascii?Q?DLpI6KifNOa0GF6DJiXfY1YMwPmQ03I+sO8cb0gY8e/QHpW9M85sl7G+x+mN?=
 =?us-ascii?Q?y/d4p29HnNuEVIZPgEG+XVonf+waaqwcTb5J7/op?=
X-MS-Exchange-CrossTenant-Network-Message-Id: d9714560-e2e5-4a12-d0d9-08dc3e81218c
X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB6020.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Mar 2024 08:32:31.6510
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: uNhv/2EfkxTnsDE+8fEn2pMaiGS4palgZ3jdvfT0SxXUakm2TjSPmH96AucXoCUhR9VBgZM+xqybQJ2HoWw6FQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR11MB4897
X-OriginatorOrg: intel.com

On 2024-02-26 at 00:26:22 -0800, isaku.yamahata@intel.com wrote:
> From: Isaku Yamahata <isaku.yamahata@intel.com>
> 
> On exiting from the guest TD, xsave state is clobbered.  Restore xsave
> state on TD exit.
> 
> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
> ---
> v19:
> - Add EXPORT_SYMBOL_GPL(host_xcr0)
> 
> v15 -> v16:
> - Added CET flag mask
> 
> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
> ---
>  arch/x86/kvm/vmx/tdx.c | 19 +++++++++++++++++++
>  arch/x86/kvm/x86.c     |  1 +
>  2 files changed, 20 insertions(+)
> 
> diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
> index 9616b1aab6ce..199226c6cf55 100644
> --- a/arch/x86/kvm/vmx/tdx.c
> +++ b/arch/x86/kvm/vmx/tdx.c
> @@ -2,6 +2,7 @@
>  #include <linux/cpu.h>
>  #include <linux/mmu_context.h>
>  
> +#include <asm/fpu/xcr.h>
>  #include <asm/tdx.h>
>  
>  #include "capabilities.h"
> @@ -534,6 +535,23 @@ void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)
>  	 */
>  }
>  
> +static void tdx_restore_host_xsave_state(struct kvm_vcpu *vcpu)
> +{
> +	struct kvm_tdx *kvm_tdx = to_kvm_tdx(vcpu->kvm);
> +
> +	if (static_cpu_has(X86_FEATURE_XSAVE) &&
> +	    host_xcr0 != (kvm_tdx->xfam & kvm_caps.supported_xcr0))
> +		xsetbv(XCR_XFEATURE_ENABLED_MASK, host_xcr0);
> +	if (static_cpu_has(X86_FEATURE_XSAVES) &&
> +	    /* PT can be exposed to TD guest regardless of KVM's XSS support */
> +	    host_xss != (kvm_tdx->xfam &
> +			 (kvm_caps.supported_xss | XFEATURE_MASK_PT | TDX_TD_XFAM_CET)))
> +		wrmsrl(MSR_IA32_XSS, host_xss);
> +	if (static_cpu_has(X86_FEATURE_PKU) &&
> +	    (kvm_tdx->xfam & XFEATURE_MASK_PKRU))
> +		write_pkru(vcpu->arch.host_pkru);
> +}

Maybe one minor question regarding the pkru restore. In the non-TDX version
kvm_load_host_xsave_state(), it first tries to read the current setting
vcpu->arch.pkru = rdpkru(); if this setting does not equal to host_pkru,
it trigger the write_pkru on host. Does it mean we can also leverage that mechanism
in TDX to avoid 1 pkru write(I guess pkru write is costly than a read pkru)?

thanks,
Chenyu