Received-SPF: pass (google.com: domain of linux-kernel+bounces-147377-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Message-ID: <7d19f693-d8e9-4a9d-8cfa-3ec9c388622f@intel.com>
Date: Tue, 16 Apr 2024 11:23:01 -0700
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v19 085/130] KVM: TDX: Complete interrupts after tdexit
To: <isaku.yamahata@intel.com>, <kvm@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>
CC: <isaku.yamahata@gmail.com>, Paolo Bonzini <pbonzini@redhat.com>,
	<erdemaktas@google.com>, Sean Christopherson <seanjc@google.com>, Sagi Shahar
	<sagis@google.com>, Kai Huang <kai.huang@intel.com>, <chen.bo@intel.com>,
	<hang.yuan@intel.com>, <tina.zhang@intel.com>, Binbin Wu
	<binbin.wu@linux.intel.com>
References: <cover.1708933498.git.isaku.yamahata@intel.com>
 <aa6a927214a5d29d5591a0079f4374b05a82a03f.1708933498.git.isaku.yamahata@intel.com>
Content-Language: en-US
From: Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <aa6a927214a5d29d5591a0079f4374b05a82a03f.1708933498.git.isaku.yamahata@intel.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?VXdoenZ2S1JzbkpZWDMzaFQrRnhQVDhrY0RyNjFzT29ydndISGlNSVVIQnlW?=
 =?utf-8?B?NGEyMk9LT0hBRmlWYWx1aExaL09NV1dTRDRaMTdkZnRuM3RxM3gxWVdQWjVm?=
 =?utf-8?B?S1dSaGdSWTM3Z0ZLc3FCTTlJcld2RWdxUjhNUnFzTnJaamxtUWsrYnZYYW1p?=
 =?utf-8?B?RVorQXRSY3BVU1FCR3hCdGd0b1dPdUxZd2krdDkwbmptZlAwNTJyOWxudnF0?=
 =?utf-8?B?WGtiVjNHeGJhWWM1T3pPM01SMEQyaytpYnB2ZzVDYklrNHQ0Y3FJOG53cjA0?=
 =?utf-8?B?MXM3WDhPS1hMS29KK3FWOEE0SytZenV5VU85WmdEOTkxZEdGMnRySU1YMFVM?=
 =?utf-8?B?LzhYWmt6aUdJSlBIU2tEU096dGNFVmJpTExReWZQM3hXaU5qbkRIRHhtZnBI?=
 =?utf-8?B?V2xkSkNLZzU5dkh4eldtV0FpWXBHbVVhL29tOW1FT0Y0QmpFdmZOa0l2aHV2?=
 =?utf-8?B?UFRibk8wQW1pNElGSEhkaEU3Tk5pRmNiRlVnZHpqV1AxS2kwTXVtVENkWlR6?=
 =?utf-8?B?OFpCMTNpa0EyeHVIaDdxajF2RmwvRTZ6V2c0b1RIUnlpM3lVV3VJZW1CL1Ry?=
 =?utf-8?B?M2xOc3JCTDY0VmZFbFQ2WVRtdmZuejRpTyszbnR6UWRkdml1aDBWTTZ5VXBB?=
 =?utf-8?B?UWNlMlNNMTdlcEtxVXM5bEdlQ2ZzeDlFSFJQS2FwN0Z3S2Q5bThYYURLWXUy?=
 =?utf-8?B?ejlzZXU3UXlZU1hPUEMwOEg2NG1yZ3BvTDVmdFBuN0ZTWTZqOExOVWprejRx?=
 =?utf-8?B?NStTczdrMFB0OUlJcnNIdnVPNUllcG93SFVPbmxURk5velcvb0FnTGloSWh4?=
 =?utf-8?B?UmhWeGZlQXBac3lEOXQyM0FRWlBTMGFkSld2RzR0YlZzQTdyZWY4aUpFanNN?=
 =?utf-8?B?b2lQbnExeGRXazVIWWFOQ2wyNzg1UnpuNnlXME1mU2NrYzRpTUVUQzNHV29C?=
 =?utf-8?B?d2hmbEhVUkNCR0c0c3U1V1k0VzN1ek5nVnZmRnZBb2QyZzJiUk9OUlQyZjRj?=
 =?utf-8?B?M0ZkYXVQQmZISStSR1FEWmpxVVJubXZUV3RjY0lrZ3BTYUlFaUF4d2h6RUd3?=
 =?utf-8?B?TXZNNTFBVXA2c1YwdkQzMC9SeEpwNXBQdEVhdDlqMnVsYUdTWEVCK1UxTW0y?=
 =?utf-8?B?YnpjbUd6YnZKdEhsb0VvYi9IbzV3STB1Q2doVnlVcjJkVHFBWmRTcEJBaUl3?=
 =?utf-8?B?VmNvT2p5K3hHQUpFMFBMMytJd0VRUUN3dlJPL2lTVEs4WTZ1L0RwQXMzUENq?=
 =?utf-8?B?cTNDMXhGcExSallTR0RzcVk0d1pHcllielBoTWllVGthSmNkQm5zZGNpMTBK?=
 =?utf-8?B?YWNmT2p0UElsaGZCaUFiK0xsbDZ4eXNqbXdXWXd0Y1pHS0kvbnBaM29LMVgy?=
 =?utf-8?B?OUdhMjlHTkJ1TWVvcGhMMUFsRmdLV25kWHJzbStZMlM2Lzd4NFdaNW54YTFP?=
 =?utf-8?B?OG1Ua043ZXNTQm9RenFFemhMdUxYMEFHSlg4bFpPYnd4a0Z3T0VKMDhYdjUx?=
 =?utf-8?B?VVcrdmRWRk1DWlhMMTA1NTNpVHMrM21DM3ZCQUR1bDE0YVZwbDJaUXNpRUNw?=
 =?utf-8?B?cStRejNxck9PWnhxbER0YzU5cTJQS3FFY0cvZU96RVBWck1DYXNBSkhJUmRs?=
 =?utf-8?B?Sy9uL24vUXVFRmNTblJCcHBnZWNjMW9kcGtqOHBvK0w2SlNRT0djNktpQnc2?=
 =?utf-8?B?Vjd1dFpReG83N2ZTckxBZkJ0RDZaZHl3SmlFcC96UWlxUmZPSGtXMHZIREdk?=
 =?utf-8?B?VzdrRjBwbnJyRmdpQ0NKc2d4ZnpXdXNZaHVrUEpVcFJmeUdxSHMyeTdrT2kr?=
 =?utf-8?B?bmJGZUdSNE9QRXRobDNVbkVMcUJkY3pWSUQ0emY2T2gxZzdnbnd4K1NBMzFr?=
 =?utf-8?B?S1U4ZmxORzEyLzB4UVc1eGU2ZkIva3NLWnlDN0VQVHc2TGtJbGllelc5S2Ra?=
 =?utf-8?B?ak9DSzhHMXNsSVBOYk0zNEtpTWw4a0JqSjBodUY1OEVTMjVldHZ0NW1MekFI?=
 =?utf-8?B?UUlxZVc5SGhIVXE3amNEb2RMWU9xaHc0d2hpbXZsZnFYK0FQTFFNS2UxSHAx?=
 =?utf-8?B?ZWNlWnBpWTl6c1ZYeFY4WlpHbDBHV1ZuTW1Ea1ZlUy9UNHhMMDZsdXBqOCtY?=
 =?utf-8?B?TUZEMTB4WFI2WUNMcFNSblY2N0xXY0wyQ1VIejQxZDkwVjBONEI5bTdzYnpr?=
 =?utf-8?B?SGc9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 0024c8d0-e597-4a44-648d-08dc5e424149
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB7573.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Apr 2024 18:23:03.8356
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: S+caTIqSlMclcWaeiypcuNqSQLga9kG6qBSmwLK4XbbuL/NFoA+bOn2gvOX1s5Sdw0I4FoKP3dl2MDWkwLU43r+INiWebJdQ5JOELHTXu8o=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN6PR11MB8101
X-OriginatorOrg: intel.com

Hi Isaku,

(In shortlog "tdexit" can be "TD exit" to be consistent with
documentation.)

On 2/26/2024 12:26 AM, isaku.yamahata@intel.com wrote:
> From: Isaku Yamahata <isaku.yamahata@intel.com>
> 
> This corresponds to VMX __vmx_complete_interrupts().  Because TDX
> virtualize vAPIC, KVM only needs to care NMI injection.

This seems to be the first appearance of NMI and the changelog
is very brief. How about expending it with:

"This corresponds to VMX __vmx_complete_interrupts().  Because TDX
 virtualize vAPIC, KVM only needs to care about NMI injection.

 KVM can request TDX to inject an NMI into a guest TD vCPU when the
 vCPU is not active. TDX will attempt to inject an NMI as soon as
 possible on TD entry. NMI injection is managed by writing to (to
 inject NMI) and reading from (to get status of NMI injection)
 the PEND_NMI field within the TDX vCPU scope metadata (Trust
 Domain Virtual Processor State (TDVPS)).

 Update KVM's NMI status on TD exit by checking whether a requested
 NMI has been injected into the TD. Reading the metadata via SEAMCALL
 is expensive so only perform the check if an NMI was injected.

 This is the first need to access vCPU scope metadata in the
 "management" class. Ensure that needed accessor is available. 
"

> 
> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
> ---
> v19:
> - move tdvps_management_check() to this patch
> - typo: complete -> Complete in short log
> ---
>  arch/x86/kvm/vmx/tdx.c | 10 ++++++++++
>  arch/x86/kvm/vmx/tdx.h |  4 ++++
>  2 files changed, 14 insertions(+)
> 
> diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
> index 83dcaf5b6fbd..b8b168f74dfe 100644
> --- a/arch/x86/kvm/vmx/tdx.c
> +++ b/arch/x86/kvm/vmx/tdx.c
> @@ -535,6 +535,14 @@ void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)
>  	 */
>  }
>  
> +static void tdx_complete_interrupts(struct kvm_vcpu *vcpu)
> +{
> +	/* Avoid costly SEAMCALL if no nmi was injected */

	/* Avoid costly SEAMCALL if no NMI was injected. */

> +	if (vcpu->arch.nmi_injected)
> +		vcpu->arch.nmi_injected = td_management_read8(to_tdx(vcpu),
> +							      TD_VCPU_PEND_NMI);
> +}
> +
>  struct tdx_uret_msr {
>  	u32 msr;
>  	unsigned int slot;
> @@ -663,6 +671,8 @@ fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu)
>  	vcpu->arch.regs_avail &= ~VMX_REGS_LAZY_LOAD_SET;
>  	trace_kvm_exit(vcpu, KVM_ISA_VMX);
>  
> +	tdx_complete_interrupts(vcpu);
> +
>  	return EXIT_FASTPATH_NONE;
>  }
>  
> diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h
> index 44eab734e702..0d8a98feb58e 100644
> --- a/arch/x86/kvm/vmx/tdx.h
> +++ b/arch/x86/kvm/vmx/tdx.h
> @@ -142,6 +142,8 @@ static __always_inline void tdvps_vmcs_check(u32 field, u8 bits)
>  			 "Invalid TD VMCS access for 16-bit field");
>  }
>  
> +static __always_inline void tdvps_management_check(u64 field, u8 bits) {}

Is this intended to be a stub or is it expected to be fleshed out with
some checks?

> +
>  #define TDX_BUILD_TDVPS_ACCESSORS(bits, uclass, lclass)				\
>  static __always_inline u##bits td_##lclass##_read##bits(struct vcpu_tdx *tdx,	\
>  							u32 field)		\
> @@ -200,6 +202,8 @@ TDX_BUILD_TDVPS_ACCESSORS(16, VMCS, vmcs);
>  TDX_BUILD_TDVPS_ACCESSORS(32, VMCS, vmcs);
>  TDX_BUILD_TDVPS_ACCESSORS(64, VMCS, vmcs);
>  
> +TDX_BUILD_TDVPS_ACCESSORS(8, MANAGEMENT, management);
> +
>  static __always_inline u64 td_tdcs_exec_read64(struct kvm_tdx *kvm_tdx, u32 field)
>  {
>  	struct tdx_module_args out;

Reinette