Received-SPF: pass (google.com: domain of linux-kernel+bounces-151825-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Date: Fri, 19 Apr 2024 10:34:23 -0700
From: Isaku Yamahata <isaku.yamahata@intel.com>
To: Binbin Wu <binbin.wu@linux.intel.com>
Cc: Isaku Yamahata <isaku.yamahata@intel.com>,
	Sean Christopherson <sean.j.christopherson@intel.com>,
	kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
	isaku.yamahata@gmail.com, Paolo Bonzini <pbonzini@redhat.com>,
	erdemaktas@google.com, Sean Christopherson <seanjc@google.com>,
	Sagi Shahar <sagis@google.com>, Kai Huang <kai.huang@intel.com>,
	chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com,
	isaku.yamahata@linux.intel.com
Subject: Re: [PATCH v19 110/130] KVM: TDX: Handle TDX PV MMIO hypercall
Message-ID: <20240419173423.GD3596705@ls.amr.corp.intel.com>
References: <cover.1708933498.git.isaku.yamahata@intel.com>
 <a4421e0f2eafc17b4703c920936e32489d2382a3.1708933498.git.isaku.yamahata@intel.com>
 <e2400cf8-ee36-4e7f-ba1f-bb0c740b045c@linux.intel.com>
 <dac4aa8c-94d1-475e-ae97-20229bd9ade2@linux.intel.com>
 <20240418212214.GB3596705@ls.amr.corp.intel.com>
 <f83f6923-7aa2-4a10-8e83-3fa77400c446@linux.intel.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <f83f6923-7aa2-4a10-8e83-3fa77400c446@linux.intel.com>

On Fri, Apr 19, 2024 at 09:42:48AM +0800,
Binbin Wu <binbin.wu@linux.intel.com> wrote:

> 
> 
> On 4/19/2024 5:22 AM, Isaku Yamahata wrote:
> > On Thu, Apr 18, 2024 at 07:04:11PM +0800,
> > Binbin Wu <binbin.wu@linux.intel.com> wrote:
> > 
> > > 
> > > On 4/18/2024 5:29 PM, Binbin Wu wrote:
> > > > > +
> > > > > +static int tdx_emulate_mmio(struct kvm_vcpu *vcpu)
> > > > > +{
> > > > > +    struct kvm_memory_slot *slot;
> > > > > +    int size, write, r;
> > > > > +    unsigned long val;
> > > > > +    gpa_t gpa;
> > > > > +
> > > > > +    KVM_BUG_ON(vcpu->mmio_needed, vcpu->kvm);
> > > > > +
> > > > > +    size = tdvmcall_a0_read(vcpu);
> > > > > +    write = tdvmcall_a1_read(vcpu);
> > > > > +    gpa = tdvmcall_a2_read(vcpu);
> > > > > +    val = write ? tdvmcall_a3_read(vcpu) : 0;
> > > > > +
> > > > > +    if (size != 1 && size != 2 && size != 4 && size != 8)
> > > > > +        goto error;
> > > > > +    if (write != 0 && write != 1)
> > > > > +        goto error;
> > > > > +
> > > > > +    /* Strip the shared bit, allow MMIO with and without it set. */
> > > > Based on the discussion
> > > > https://lore.kernel.org/all/ZcUO5sFEAIH68JIA@google.com/
> > > > Do we still allow the MMIO without shared bit?
> > That's independent.  The part is how to work around guest accesses the
> > MMIO region with private GPA.  This part is,  the guest issues
> > TDG.VP.VMCALL<MMMIO> and KVM masks out the shared bit to make it friendly
> > to the user space VMM.
> It's similar.
> The tdvmcall from the guest for mmio can also be private GPA, which is not
> reasonable, right?
> According to the comment, kvm doens't care about if the TD guest issue the
> tdvmcall with private GPA or shared GPA.

I checked the GHCI spec. It clearly states this hypercall is for shared GPA.
We should return error for private GPA.

  This TDG.VP.VMCALL is used to help request the VMM perform
  emulated-MMIO-access operation. The VMM may emulate MMIO space in shared-GPA
  space. The VMM can induce a #VE on these shared-GPA accesses by mapping shared
  GPAs with the suppress-VE bit cleared in the EPT Entries corresponding to
  these mappings

So we'll have something as follows. Compile only tested.

diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index 3bf0d6e3cd21..0f696f3fbd86 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -1281,24 +1281,34 @@ static int tdx_emulate_mmio(struct kvm_vcpu *vcpu)
        if (write != 0 && write != 1)
                goto error;
 
-       /* Strip the shared bit, allow MMIO with and without it set. */
+       /*
+        * MMIO with TDG.VP.VMCALL<MMIO> allows only shared GPA because
+        * private GPA is for device assignment.
+        */
+       if (kvm_is_private_gpa(gpa))
+               goto error;
+
+       /*
+        * Strip the shared bit because device emulator is assigned to GPA
+        * without shared bit.  We'd like the existing code untouched.
+        */
        gpa = gpa & ~gfn_to_gpa(kvm_gfn_shared_mask(vcpu->kvm));
 
-       if (size > 8u || ((gpa + size - 1) ^ gpa) & PAGE_MASK)
+       /* Disallow MMIO crossing page boundary for simplicity. */
+       if (((gpa + size - 1) ^ gpa) & PAGE_MASK)
                goto error;
 
        slot = kvm_vcpu_gfn_to_memslot(vcpu, gpa_to_gfn(gpa));
        if (slot && !(slot->flags & KVM_MEMSLOT_INVALID))
                goto error;
 
-       if (!kvm_io_bus_write(vcpu, KVM_FAST_MMIO_BUS, gpa, 0, NULL)) {
-               trace_kvm_fast_mmio(gpa);
-               return 1;
-       }
-
-       if (write)
+       if (write) {
+               if (!kvm_io_bus_write(vcpu, KVM_FAST_MMIO_BUS, gpa, 0, NULL)) {
+                       trace_kvm_fast_mmio(gpa);
+                       return 1;
+               }
                r = tdx_mmio_write(vcpu, gpa, size, val);
-       else
+       } else
                r = tdx_mmio_read(vcpu, gpa, size);
        if (!r) {
                /* Kernel completed device emulation. */


-- 
Isaku Yamahata <isaku.yamahata@intel.com>