Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 28 Nov 2022 23:49:19 +0000
From:   Sean Christopherson <seanjc@google.com>
To:     Yan Zhao <yan.y.zhao@intel.com>
Cc:     "Huang, Kai" <kai.huang@intel.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "Shahar, Sagi" <sagis@google.com>,
        "isaku.yamahata@gmail.com" <isaku.yamahata@gmail.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "Aktas, Erdem" <erdemaktas@google.com>,
        "sean.j.christopherson@intel.com" <sean.j.christopherson@intel.com>,
        "dmatlack@google.com" <dmatlack@google.com>,
        "Yamahata, Isaku" <isaku.yamahata@intel.com>,
        "pbonzini@redhat.com" <pbonzini@redhat.com>
Subject: Re: [PATCH v10 035/108] KVM: x86/mmu: Track shadow MMIO value on a
 per-VM basis
Message-ID: <Y4VI//c9cYROYV4p@google.com>
References: <cover.1667110240.git.isaku.yamahata@intel.com>
 <f1c9996fa4f4bc71b3feee8407d247aeabb8968e.1667110240.git.isaku.yamahata@intel.com>
 <Y3wvhWyIKNzczFov@yzhao56-desk.sh.intel.com>
 <887a77acc9bf96f7c7bea519ab7ebdd27fb67985.camel@intel.com>
 <Y4AIYGGEU3BdrQgV@yzhao56-desk.sh.intel.com>
 <8e14a7732a2d873846d07c4ec467fb7c48b2307f.camel@intel.com>
 <Y4AOMo00B0vlQfIU@yzhao56-desk.sh.intel.com>
 <1ad20facd81dd346e00ff686ae4a0550123de851.camel@intel.com>
 <Y4AUrLJtk7pK082i@yzhao56-desk.sh.intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Y4AUrLJtk7pK082i@yzhao56-desk.sh.intel.com>
Precedence: bulk

On Fri, Nov 25, 2022, Yan Zhao wrote:
> On Fri, Nov 25, 2022 at 09:07:09AM +0800, Huang, Kai wrote:
> > On Fri, 2022-11-25 at 08:37 +0800, Yan Zhao wrote:
> > > On Fri, Nov 25, 2022 at 08:45:01AM +0800, Huang, Kai wrote:
> > > > On Fri, 2022-11-25 at 08:12 +0800, Yan Zhao wrote:
> > > > > On Fri, Nov 25, 2022 at 08:13:48AM +0800, Huang, Kai wrote:
> > > > > > On Tue, 2022-11-22 at 10:10 +0800, Yan Zhao wrote:
> > > > > > > Also make enable_mmio_caching to be a per-VM value?
> > > > > > > As if the shadow_mmio_value is 0, mmio_caching needs to be disabled.
> > > > > > 
> > > > > > If I recall correctly, Sean said we can disable TDX guests if mmio_caching is
> > > > > > disabled (we also will need to change to allow enable_mmio_caching to still be
> > > > > > true when mmio_value is 0).
> > > > > > 
> > > > > > SEV_ES has similar logic:
> > > > > > 
> > > > > > void __init sev_hardware_setup(void)
> > > > > > {
> > > > > > 
> > > > > > 	...
> > > > > > 
> > > > > >         /*
> > > > > >          * SEV-ES requires MMIO caching as KVM doesn't have access to the guest
> > > > > >          * instruction stream, i.e. can't emulate in response to a #NPF and
> > > > > >          * instead relies on #NPF(RSVD) being reflected into the guest as #VC
> > > > > >          * (the guest can then do a #VMGEXIT to request MMIO emulation).
> > > > > >          */
> > > > > >         if (!enable_mmio_caching)
> > > > > >                 goto out;
> > > > > > 
> > > > > 
> > > > > Would enabling mmio caching in per-VM basis be better?
> > > > > 
> > > > 
> > > > We need Paolo/Sean to decide.
> > > > 
> > > > The thing is TDX guests always require mmio_caching being enabled.  For VMX
> > > > guests, normally we will always enable mmio_caching too.  So I think per-VM
> > > > basis mmio_caching is not that useful.
> > > With per-VM basis enabling, I think we can get rid of the kvm_gfn_shared_mask(kvm)
> > > in below code and also in handle_abnormal_pfn()
> > > 
> > > static inline bool is_mmio_spte(struct kvm *kvm, u64 spte)
> > > {
> > >         return (spte & shadow_mmio_mask) == kvm->arch.shadow_mmio_value &&
> > >                likely(enable_mmio_caching || kvm_gfn_shared_mask(kvm));
> > > }
> > > 
> > 
> > It needs to go anyway regardless per-VM mmio_caching or not, as explained we
> > need to change to allow enable_mmio_caching to be true even mmio_value is 0.

Yes, the kvm_gfn_shared_mask() in is_mmio_spte() should not exist.

> Or it's better to check enable_mmio_caching is true in
> kvm_mmu_set_mmio_spte_value() as below. 
> 
> void kvm_mmu_set_mmio_spte_value(struct kvm *kvm, u64 mmio_value)
> {
>         WARN_ON(!enable_mmio_caching);
>         kvm->arch.shadow_mmio_value = mmio_value;

Eh, if you're going to bother with that assert, capture the more subtle aspects
as well, e.g. that EPT is enabled and that shadow_mmio_mask has been set to the
expected value.  I would also ditch the helper and just open code this in TDX
code, I doubt there's a use case outside of TDX+VMX coexistence.