Received-SPF: pass (google.com: domain of linux-kernel+bounces-120253-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Message-ID: <b07d0749-8a72-4a0a-a0ad-808d7ea2b922@intel.com>
Date: Wed, 27 Mar 2024 11:08:46 +0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v19 046/130] KVM: x86/mmu: Add address conversion
 functions for TDX shared bit of GPA
To: <isaku.yamahata@intel.com>, <kvm@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>
CC: <isaku.yamahata@gmail.com>, Paolo Bonzini <pbonzini@redhat.com>,
	<erdemaktas@google.com>, Sean Christopherson <seanjc@google.com>, Sagi Shahar
	<sagis@google.com>, Kai Huang <kai.huang@intel.com>, <chen.bo@intel.com>,
	<hang.yuan@intel.com>, <tina.zhang@intel.com>, Rick Edgecombe
	<rick.p.edgecombe@intel.com>, Binbin Wu <binbin.wu@linux.intel.com>
References: <cover.1708933498.git.isaku.yamahata@intel.com>
 <973a3e06111fe84f2b1e971636cbaa3facf7b120.1708933498.git.isaku.yamahata@intel.com>
Content-Language: en-US
From: Chenyi Qiang <chenyi.qiang@intel.com>
In-Reply-To: <973a3e06111fe84f2b1e971636cbaa3facf7b120.1708933498.git.isaku.yamahata@intel.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?a2QrYVpWejBOVm8wUVFXN1l0Z0IvZkd3S2JEQmQ3NlU4aVhpMjNmem0rVlhx?=
 =?utf-8?B?U2crUFZoQXlzdm9PTUFRREFrQTgyZmx4QWQ5NnNWRzlGZjR2dVRsS0lNZGFa?=
 =?utf-8?B?cGZkZ2FDbVNmNFBYTllUaFlGdUlKNm80Sy9VbXM5YXU2WEFTaGhFSmNUd0Fm?=
 =?utf-8?B?djQ0cGx2a2F6aFJHVFpNZ1hzVEMxMVZrSkhxY2Rtbk5nVkJ3QjRMbCtsZ2R2?=
 =?utf-8?B?bFR1SVJhb0tTSjhKa2JNUGl2WEdyTTBzQVFJWEtzOS9YMlJVMnpSYUxnOEZK?=
 =?utf-8?B?clJCcG9ORVh1U094RU10dE9TbDRhdG9KTXVZMjdKTUMydCs0UUVjeDhGeUxv?=
 =?utf-8?B?N0ZaSElEZFZQWTIyUTNodlc0WE5YN3hVNnc2cmJFS2V1b1U3Y0VqbXc4dFl5?=
 =?utf-8?B?b05vc1BGQ3RnVXIwWERTNGhabDdXL1BMNGpFYS9mcHl1WHg1eW4zL2prdFZV?=
 =?utf-8?B?eTMxU3FXcHlpcjRwaDg3SEJxbitPWGJNSWlrd1dBT2JJZFk4R1dHem1HWDRH?=
 =?utf-8?B?U3plQmREMlFGVkJYSkorc0owZ3JYcEtkdkNJOFQ1eWhQZFc0Q2JKeUNqSFBr?=
 =?utf-8?B?akJRLy9kTDdMdVVhcndyRGREM1I2UUZBdjdwS2hIazg3L1QvbEtORmZuMEsv?=
 =?utf-8?B?OWhERWg0WWxqbzBONXl4WE9FWjZHZFdxeVpOOHYzNEF1QmFSdC9ZZEVkaS9O?=
 =?utf-8?B?WmFYa3h2RXBHOUFFc2FQdUt1dHU3cXlMcWFQM2JCRHplZkl5cTY4Skp4SVB0?=
 =?utf-8?B?ZkNwMTZ6THFBUHVPNy9QWUpvUTZHRXhNMVVMd0UwaG5xb3pVT21ZTUFsdytN?=
 =?utf-8?B?SEdkNmtnQUsvSkQ1UkVEUVM1NlFIY1dlY3cveDcvN2V1OXc5bkJvRjh3WnlG?=
 =?utf-8?B?T2VXcjBzVDhjdFRhaFJYeUtkSDBDWURRL0xMM0hpOFlXL1Z4a0lIYkxtUmNs?=
 =?utf-8?B?VHk3cTNpQTltaTFrODlkcWcrRDFOejd2eXRUZTlIajE3dHNIdjFDU3YvdkJM?=
 =?utf-8?B?Vkc4dDVTVVcySXpYUVduOGIvRUVMRjhLWE9McGY4eWVocnhSUmVmTXRaUitj?=
 =?utf-8?B?bjNUbDBGSG5JaE9uTEVVY2IwZSs1RDNOT3ErUTMvS2kweEJLc0hGZWgzNVZI?=
 =?utf-8?B?WmxKTVpUUEgyKzZQcUw3N1A5T2h6Sk1PeEd3NUt5eC9hQ2xkY09BL2pMcFc4?=
 =?utf-8?B?OEI0MldDSjZkRTVMOEdESXdjN3ZYODhZSmxYcGR1SUQ3ZFVqL1RtdXRLTTNN?=
 =?utf-8?B?V0UrWEp3U0pvcTVoNTFyb0Z6SEhqNDUvVHpJeTYrOFZJMjEybEFhcFdwUTR4?=
 =?utf-8?B?Q3VaNmpPcFY1UGNIVWM3UFAzZW0wRERWZGVRWmJQWHVrOWR5TWdOV3VvYlB1?=
 =?utf-8?B?TUk1WHVXZkE5Y2szbUtIS1JTWG9HTzRpc2hjZnBiUjZJYlZUaGpjUE9GeFp3?=
 =?utf-8?B?SisvM3ZRZEl5ckI4S2JKRG1BbWRzbG94eW1CdzhkeE9DcUg1NTJtWGQ1ZDNX?=
 =?utf-8?B?ajgvRG1TU1FUUnNnUDFnUjkzNWZvNEhrRmFpUWxtaDRWUTdNczhIc2VRdUhy?=
 =?utf-8?B?RldhYWdra2R4Z1U4ODNjZ3ZSdTlLUmFIR0lLcEsyRGhDRWRkeTNOQitKTlZs?=
 =?utf-8?B?MTlrbDVGdERSY3Zzd3NtRUtYdSswWFpDMzNrc1kzZUgxdnd5YzZ0MWZYUDg0?=
 =?utf-8?B?WWNOSEt0ZVFkZXg3bE9pV3lJb3ZSQnRXbmplaFMrUDRFaUd0RnZvMWJJLzZi?=
 =?utf-8?B?U0tQNFl6NVdENWhtRzRBSlFFeXprdWR3bFR2cVZoSzROWHRqcUsySGlUSTdv?=
 =?utf-8?B?Z2pOR0tZdTVTWjFUK2gxNG1yWUZSVjJTcFkrZ1oxdExxa3c2UFRZcHQ2U0pR?=
 =?utf-8?B?NHd6OUpSK25WTmxrOVNXSUFOclNmZmVvMzdJUU80M3VpYWlNRmdScTZ6UkRO?=
 =?utf-8?B?NUlSdXJuY0RZSms1UlRpZW9WTVlRMzFXNGM0QmRjcCtoc0ljbHd4ZWpvYUNk?=
 =?utf-8?B?RmtFdUNXbjU2RkdETzBTcGt3bTlLMFB4dVFiay9ad05HUFFJSVo0SHJOenJN?=
 =?utf-8?B?L3RtMjVPTDg3bE96eGpwSUJ3UjRJU0NIcVpRYzdxVGpqQzQ2SlljdG00bmVh?=
 =?utf-8?Q?cjYb+LngmrK9xApw/sM3IUosr?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 7a27c87b-f5b6-434f-5623-08dc4e0b3d4b
X-MS-Exchange-CrossTenant-AuthSource: DM3PR11MB8735.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Mar 2024 03:08:56.2951
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: GRUmEYi2tJZ5IbGOP7apioX6dmPMlts0U8dJjrrt3+p8UoEeuemYSWnVVW8OXEvJk3VKmnUPgZy+suk5TEJ8mQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR11MB5799
X-OriginatorOrg: intel.com


On 2/26/2024 4:25 PM, isaku.yamahata@intel.com wrote:
> From: Isaku Yamahata <isaku.yamahata@intel.com>
> 
> TDX repurposes one GPA bit (51 bit or 47 bit based on configuration) to
> indicate the GPA is private(if cleared) or shared (if set) with VMM.  If
> GPA.shared is set, GPA is covered by the existing conventional EPT pointed
> by EPTP.  If GPA.shared bit is cleared, GPA is covered by TDX module.
> VMM has to issue SEAMCALLs to operate.
> 
> Add a member to remember GPA shared bit for each guest TDs, add address
> conversion functions between private GPA and shared GPA and test if GPA
> is private.
> 
> Because struct kvm_arch (or struct kvm which includes struct kvm_arch. See
> kvm_arch_alloc_vm() that passes __GPF_ZERO) is zero-cleared when allocated,
> the new member to remember GPA shared bit is guaranteed to be zero with
> this patch unless it's initialized explicitly.
> 
> 			default or SEV-SNP	TDX: S = (47 or 51) - 12
> gfn_shared_mask		0			S bit
> kvm_is_private_gpa()	always false		true if GFN has S bit set

TDX: true if GFN has S bit clear?

> kvm_gfn_to_shared()	nop			set S bit
> kvm_gfn_to_private()	nop			clear S bit
> 
> fault.is_private means that host page should be gotten from guest_memfd
> is_private_gpa() means that KVM MMU should invoke private MMU hooks.
> 
> Co-developed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
> Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
> ---
> v19:
> - Add comment on default vm case.
> - Added behavior table in the commit message
> - drop CONFIG_KVM_MMU_PRIVATE
> 
> v18:
> - Added Reviewed-by Binbin
> 
> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
> ---
>  arch/x86/include/asm/kvm_host.h |  2 ++
>  arch/x86/kvm/mmu.h              | 33 +++++++++++++++++++++++++++++++++
>  arch/x86/kvm/vmx/tdx.c          |  5 +++++
>  3 files changed, 40 insertions(+)
> 
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index 5da3c211955d..de6dd42d226f 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -1505,6 +1505,8 @@ struct kvm_arch {
>  	 */
>  #define SPLIT_DESC_CACHE_MIN_NR_OBJECTS (SPTE_ENT_PER_PAGE + 1)
>  	struct kvm_mmu_memory_cache split_desc_cache;
> +
> +	gfn_t gfn_shared_mask;
>  };
>  
>  struct kvm_vm_stat {
> diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h
> index d96c93a25b3b..395b55684cb9 100644
> --- a/arch/x86/kvm/mmu.h
> +++ b/arch/x86/kvm/mmu.h
> @@ -322,4 +322,37 @@ static inline gpa_t kvm_translate_gpa(struct kvm_vcpu *vcpu,
>  		return gpa;
>  	return translate_nested_gpa(vcpu, gpa, access, exception);
>  }
> +
> +/*
> + *			default or SEV-SNP	TDX: where S = (47 or 51) - 12
> + * gfn_shared_mask	0			S bit
> + * is_private_gpa()	always false		if GPA has S bit set
> + * gfn_to_shared()	nop			set S bit
> + * gfn_to_private()	nop			clear S bit
> + *
> + * fault.is_private means that host page should be gotten from guest_memfd
> + * is_private_gpa() means that KVM MMU should invoke private MMU hooks.
> + */
> +static inline gfn_t kvm_gfn_shared_mask(const struct kvm *kvm)
> +{
> +	return kvm->arch.gfn_shared_mask;
> +}
> +
> +static inline gfn_t kvm_gfn_to_shared(const struct kvm *kvm, gfn_t gfn)
> +{
> +	return gfn | kvm_gfn_shared_mask(kvm);
> +}
> +
> +static inline gfn_t kvm_gfn_to_private(const struct kvm *kvm, gfn_t gfn)
> +{
> +	return gfn & ~kvm_gfn_shared_mask(kvm);
> +}
> +
> +static inline bool kvm_is_private_gpa(const struct kvm *kvm, gpa_t gpa)
> +{
> +	gfn_t mask = kvm_gfn_shared_mask(kvm);
> +
> +	return mask && !(gpa_to_gfn(gpa) & mask);
> +}
> +
>  #endif
> diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
> index aa1da51b8af7..54e0d4efa2bd 100644
> --- a/arch/x86/kvm/vmx/tdx.c
> +++ b/arch/x86/kvm/vmx/tdx.c
> @@ -906,6 +906,11 @@ static int tdx_td_init(struct kvm *kvm, struct kvm_tdx_cmd *cmd)
>  	kvm_tdx->attributes = td_params->attributes;
>  	kvm_tdx->xfam = td_params->xfam;
>  
> +	if (td_params->exec_controls & TDX_EXEC_CONTROL_MAX_GPAW)
> +		kvm->arch.gfn_shared_mask = gpa_to_gfn(BIT_ULL(51));
> +	else
> +		kvm->arch.gfn_shared_mask = gpa_to_gfn(BIT_ULL(47));
> +
>  out:
>  	/* kfree() accepts NULL. */
>  	kfree(init_vm);