Date:   Fri, 27 Jan 2023 23:36:52 +0200
From:   Zhi Wang <zhi.wang.linux@gmail.com>
To:     Sean Christopherson <seanjc@google.com>
Cc:     isaku.yamahata@intel.com, kvm@vger.kernel.org,
        linux-kernel@vger.kernel.org, isaku.yamahata@gmail.com,
        Paolo Bonzini <pbonzini@redhat.com>, erdemaktas@google.com,
        Sagi Shahar <sagis@google.com>,
        David Matlack <dmatlack@google.com>,
        Sean Christopherson <sean.j.christopherson@intel.com>,
        kai.huang@intel.com
Subject: Re: [PATCH v11 030/113] KVM: x86/mmu: Replace hardcoded value 0 for
 the initial value for SPTE
Message-ID: <20230127233513.0000367c@gmail.com>
In-Reply-To: <Y9Fj/vgPEzfU1eof@google.com>
References: <cover.1673539699.git.isaku.yamahata@intel.com>
        <dee30f0562d8be0102547d8eb9fc77736eae679d.1673539699.git.isaku.yamahata@intel.com>
        <20230125112434.0000512a@gmail.com>
        <Y9Fj/vgPEzfU1eof@google.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Precedence: bulk

On Wed, 25 Jan 2023 17:22:08 +0000
Sean Christopherson <seanjc@google.com> wrote:

> On Wed, Jan 25, 2023, Zhi Wang wrote:
> > On Thu, 12 Jan 2023 08:31:38 -0800
> > isaku.yamahata@intel.com wrote:
> > 
> > This refactor patch is quite hacky.
> > 
> > Why not change the purpose of vcpu->arch.mmu_shadow_page.gfp_zero and let the
> > callers respect that the initial value of spte can be configurable? It will be
> > generic and not TDX-specific, then kvm_init_shadow_page() is not required,
> > mmu_topup_shadow_page_cache() can be left un-touched as the refactor can cover
> > other architectures.
> > 
> > 1) Let it store the expected nonpresent value and rename it to nonpresent_spte.
> 
> 
> I agree that handling this in the common code would be cleaner, but repurposing
> gfp_zero gets kludgy because it would require a magic value to say "don't initialize
> the data", e.g. x86's mmu_shadowed_info_cache isn't pre-filled.
> 
> And supporting a custom 64-bit init value for kmem_cache-backed caches would require
> restricting such caches to be a multiple of 8 bytes in size.
> 
> How about this?  Lightly tested.
> 
> From: Sean Christopherson <seanjc@google.com>
> Date: Wed, 25 Jan 2023 16:55:01 +0000
> Subject: [PATCH] KVM: Allow page-sized MMU caches to be initialized with
>  custom 64-bit values
>

It looks good enough so far although it only supports 64bit init value. But
it can be extended in the future. 

Just want to make sure people are thinking the same:

1) Keep the changes of SHADOW_NONPRESENT_VALUE and REMOVED_SPTE in TDX patch.
init_value stays as a generic feature in the kvm mmu cache layer. It is *not*
going to replace SHADOW_NONPRESENT_VALUE.

2) TDX kvm_x86_vcpu_create sets the SHADOW_NONPRESENT value into init_value.

3) mmu cache topping up function initializes the page according to init_value
with Sean's patch. 

> Add support to MMU caches for initializing a page with a custom 64-bit
> value, e.g. to pre-fill an entire page table with non-zero PTE values.
> The functionality will be used by x86 to support Intel's TDX, which needs
> to set bit 63 in all non-present PTEs in order to prevent !PRESENT page
> faults from getting reflected into the guest (Intel's EPT Violation #VE
> architecture made the less than brilliant decision of having the per-PTE
> behavior be opt-out instead of opt-in).
> 
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
>  include/linux/kvm_types.h |  1 +
>  virt/kvm/kvm_main.c       | 16 ++++++++++++++--
>  2 files changed, 15 insertions(+), 2 deletions(-)
> 
> diff --git a/include/linux/kvm_types.h b/include/linux/kvm_types.h
> index 76de36e56cdf..67972db17b55 100644
> --- a/include/linux/kvm_types.h
> +++ b/include/linux/kvm_types.h
> @@ -94,6 +94,7 @@ struct kvm_mmu_memory_cache {
>  	int nobjs;
>  	gfp_t gfp_zero;
>  	gfp_t gfp_custom;
> +	u64 init_value;
>  	struct kmem_cache *kmem_cache;
>  	int capacity;
>  	void **objects;
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index d255964ec331..78f1e49179a7 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -380,12 +380,17 @@ static void kvm_flush_shadow_all(struct kvm *kvm)
>  static inline void *mmu_memory_cache_alloc_obj(struct kvm_mmu_memory_cache *mc,
>  					       gfp_t gfp_flags)
>  {
> +	void *page;
> +
>  	gfp_flags |= mc->gfp_zero;
>  
>  	if (mc->kmem_cache)
>  		return kmem_cache_alloc(mc->kmem_cache, gfp_flags);
> -	else
> -		return (void *)__get_free_page(gfp_flags);
> +
> +	page = (void *)__get_free_page(gfp_flags);
> +	if (page && mc->init_value)
> +		memset64(page, mc->init_value, PAGE_SIZE / sizeof(mc->init_value));
> +	return page;
>  }
>  
>  int __kvm_mmu_topup_memory_cache(struct kvm_mmu_memory_cache *mc, int capacity, int min)
> @@ -400,6 +405,13 @@ int __kvm_mmu_topup_memory_cache(struct kvm_mmu_memory_cache *mc, int capacity,
>  		if (WARN_ON_ONCE(!capacity))
>  			return -EIO;
>  
> +		/*
> +		 * Custom init values can be used only for page allocations,
> +		 * and obviously conflict with __GFP_ZERO.
> +		 */
> +		if (WARN_ON_ONCE(mc->init_value && (mc->kmem_cache || mc->gfp_zero)))
> +			return -EIO;
> +
>  		mc->objects = kvmalloc_array(sizeof(void *), capacity, gfp);
>  		if (!mc->objects)
>  			return -ENOMEM;
> 
> base-commit: 503f0315c97739d3f8e645c500d81757dfbf76be