Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 15 Dec 2022 21:26:42 -0800
From:   Isaku Yamahata <isaku.yamahata@gmail.com>
To:     "Huang, Kai" <kai.huang@intel.com>
Cc:     "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "Yamahata, Isaku" <isaku.yamahata@intel.com>,
        "sean.j.christopherson@intel.com" <sean.j.christopherson@intel.com>,
        "pbonzini@redhat.com" <pbonzini@redhat.com>,
        "Shahar, Sagi" <sagis@google.com>,
        "Aktas, Erdem" <erdemaktas@google.com>,
        "isaku.yamahata@gmail.com" <isaku.yamahata@gmail.com>,
        "dmatlack@google.com" <dmatlack@google.com>,
        "Christopherson,, Sean" <seanjc@google.com>
Subject: Re: [PATCH v10 105/108] KVM: TDX: Add methods to ignore accesses to
 CPU state
Message-ID: <20221216052642.GF527635@ls.amr.corp.intel.com>
References: <cover.1667110240.git.isaku.yamahata@intel.com>
 <282dd5a8edbee0aa87cdf035088ecd8558b0b999.1667110240.git.isaku.yamahata@intel.com>
 <5dc0263aa963534bb049b61adbfb18dec3f65f00.camel@intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <5dc0263aa963534bb049b61adbfb18dec3f65f00.camel@intel.com>
Precedence: bulk

On Wed, Dec 14, 2022 at 11:43:14AM +0000,
"Huang, Kai" <kai.huang@intel.com> wrote:

> On Sat, 2022-10-29 at 23:23 -0700, isaku.yamahata@intel.com wrote:
> > +static u8 vt_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio)
> > +{
> > +	if (is_td_vcpu(vcpu)) {
> > +		if (is_mmio)
> > +			return MTRR_TYPE_UNCACHABLE << VMX_EPT_MT_EPTE_SHIFT;
> > +		return  MTRR_TYPE_WRBACK << VMX_EPT_MT_EPTE_SHIFT;
> > +	}
> > +
> > +	return vmx_get_mt_mask(vcpu, gfn, is_mmio);
> > +}
> 
> So you are returning WB for _ALL_ guest memory, including shared.  Wouldn't this
> break MTRR handling for shared memory?  For instance, IIUC we can still support
> assigning a device to a TDX guest while the VT-d doesn't support coherent
> memory, in which case guest's MTRR/PAT are honored.  I think this should also
> apply to TDX guest's shared memory?

You're right. So here is the updated change.
--- a/arch/x86/kvm/vmx/main.c
+++ b/arch/x86/kvm/vmx/main.c
@@ -798,11 +798,8 @@ static int vt_set_identity_map_addr(struct kvm *kvm, u64 ident_addr)
 
 static u8 vt_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio)
 {
-       if (is_td_vcpu(vcpu)) {
-               if (is_mmio)
-                       return MTRR_TYPE_UNCACHABLE << VMX_EPT_MT_EPTE_SHIFT;
-               return  MTRR_TYPE_WRBACK << VMX_EPT_MT_EPTE_SHIFT;
-       }
+       if (is_td_vcpu(vcpu))
+               return tdx_get_mt_mask(vcpu, gfn, is_mmio);
 
        return vmx_get_mt_mask(vcpu, gfn, is_mmio);
 }
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index ac47b20e4e91..f1842eb32d6c 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -568,7 +568,31 @@ int tdx_vm_init(struct kvm *kvm)
        return 0;
 }
 
+u8 tdx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio)
+{
+       if (is_mmio)
+               return MTRR_TYPE_UNCACHABLE << VMX_EPT_MT_EPTE_SHIFT;
+
+       if (!kvm_arch_has_noncoherent_dma(vcpu->kvm))
+               return (MTRR_TYPE_WRBACK << VMX_EPT_MT_EPTE_SHIFT) | VMX_EPT_IPAT_BIT;
+
+       /* Guest CR0 is unknown.  Assume CR0.CD = 0. */
+
+       /* TDX private GPA is always WB. */
+       if (gfn & kvm_gfn_shared_mask(vcpu->kvm))
+               return  MTRR_TYPE_WRBACK << VMX_EPT_MT_EPTE_SHIFT;
+
+       /*
+        * Because the definition of MTRR MSR is unaware of shared-bit,
+        * clear shared-bit.
+        */
+       gfn = kvm_gfn_private(vcpu->kvm, gfn);
+       return kvm_mtrr_get_guest_memory_type(vcpu, gfn) << VMX_EPT_MT_EPTE_SHIFT;
+}
+
 int tdx_vcpu_create(struct kvm_vcpu *vcpu)
+
+
 {
        struct kvm_tdx *kvm_tdx = to_kvm_tdx(vcpu->kvm);
        struct vcpu_tdx *tdx = to_tdx(vcpu);
diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h
index 96f5546194d4..38fb888e1be9 100644
--- a/arch/x86/kvm/vmx/x86_ops.h
+++ b/arch/x86/kvm/vmx/x86_ops.h
@@ -168,6 +168,7 @@ int tdx_handle_exit(struct kvm_vcpu *vcpu,
 void tdx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode,
                           int trig_mode, int vector);
 void tdx_inject_nmi(struct kvm_vcpu *vcpu);
+u8 tdx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio);
 void tdx_get_exit_info(struct kvm_vcpu *vcpu, u32 *reason,
                u64 *info1, u64 *info2, u32 *intr_info, u32 *error_code);
 bool tdx_is_emulated_msr(u32 index, bool write);

-- 
Isaku Yamahata <isaku.yamahata@gmail.com>