Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <cover.1685333727.git.isaku.yamahata@intel.com>
 <a7fc71f5d30676e9360f3db040feb7e6ac18203f.1685333728.git.isaku.yamahata@intel.com>
 <CAAYXXYzR6JZq8OOD2qqC_vVGiCa3e5KmZZ+36YffGW6JFK4+Hw@mail.gmail.com>
In-Reply-To: <CAAYXXYzR6JZq8OOD2qqC_vVGiCa3e5KmZZ+36YffGW6JFK4+Hw@mail.gmail.com>
From:   Erdem Aktas <erdemaktas@google.com>
Date:   Thu, 8 Jun 2023 04:29:35 -0700
Message-ID: <CAAYXXYyUbqjmrezTw=p_yBkkB-o9H-Xg_J5rBCe1CXUpF5PM+A@mail.gmail.com>
Subject: Re: [PATCH v14 055/113] KVM: TDX: TDP MMU TDX support
To:     isaku.yamahata@intel.com
Cc:     kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
        isaku.yamahata@gmail.com, Paolo Bonzini <pbonzini@redhat.com>,
        Sean Christopherson <seanjc@google.com>,
        Sagi Shahar <sagis@google.com>,
        David Matlack <dmatlack@google.com>,
        Kai Huang <kai.huang@intel.com>,
        Zhi Wang <zhi.wang.linux@gmail.com>, chen.bo@intel.com,
        Vishal Annapurve <vannapurve@google.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Sun, May 28, 2023 at 9:21=E2=80=AFPM <isaku.yamahata@intel.com> wrote:
>
> From: Isaku Yamahata <isaku.yamahata@intel.com>
>
> +static int tdx_sept_drop_private_spte(struct kvm *kvm, gfn_t gfn,
> +                                      enum pg_level level, kvm_pfn_t pfn=
)
> +{
> +       int tdx_level =3D pg_level_to_tdx_sept_level(level);
> +       struct kvm_tdx *kvm_tdx =3D to_kvm_tdx(kvm);
> +       struct tdx_module_output out;
> +       gpa_t gpa =3D gfn_to_gpa(gfn);
> +       hpa_t hpa =3D pfn_to_hpa(pfn);
> +       hpa_t hpa_with_hkid;
> +       u64 err;
> +
> +       /* TODO: handle large pages. */
> +       if (KVM_BUG_ON(level !=3D PG_LEVEL_4K, kvm))
> +               return -EINVAL;
> +
> +       if (unlikely(!is_hkid_assigned(kvm_tdx))) {
> +               /*
> +                * The HKID assigned to this TD was already freed and cac=
he
> +                * was already flushed. We don't have to flush again.
> +                */
> +               err =3D tdx_reclaim_page(hpa, false, 0);
> +               if (KVM_BUG_ON(err, kvm))
> +                       return -EIO;
> +               tdx_unpin(kvm, pfn);
> +               return 0;
> +       }
> +
> +       do {
> +               /*
> +                * When zapping private page, write lock is held. So no r=
ace
> +                * condition with other vcpu sept operation.  Race only w=
ith
> +                * TDH.VP.ENTER.
> +                */
> +               err =3D tdh_mem_page_remove(kvm_tdx->tdr_pa, gpa, tdx_lev=
el, &out);
> +       } while (unlikely(err =3D=3D TDX_ERROR_SEPT_BUSY));
> +       if (KVM_BUG_ON(err, kvm)) {
> +               pr_tdx_error(TDH_MEM_PAGE_REMOVE, err, &out);
> +               return -EIO;
> +       }
> +
> +       hpa_with_hkid =3D set_hkid_to_hpa(hpa, (u16)kvm_tdx->hkid);
> +       do {
> +               /*
> +                * TDX_OPERAND_BUSY can happen on locking PAMT entry.  Be=
cause
> +                * this page was removed above, other thread shouldn't be
> +                * repeatedly operating on this page.  Just retry loop.
> +                */
> +               err =3D tdh_phymem_page_wbinvd(hpa_with_hkid);
> +       } while (unlikely(err =3D=3D (TDX_OPERAND_BUSY | TDX_OPERAND_ID_R=
CX)));
> +       if (KVM_BUG_ON(err, kvm)) {
> +               pr_tdx_error(TDH_PHYMEM_PAGE_WBINVD, err, NULL);
> +               return -EIO;
> +       }


It seems like when the TD is destroyed, all the TD private pages are
removed with tdx_reclaim_page which also clears the page with
movdir64b instruction. But when the page is removed while the TD is
alive (in this case), the page content is never cleared with movdir64b
which causes any poisoned cache line to be consumed by the host
resulting in #MC exceptions in the host context.

We should clear the page before returning it back to the free pool by
calling tdx_clear_page((unsigned long)__va(hpa)) here.

>
> +       tdx_unpin(kvm, pfn);
> +       return 0;
> +}
> +
>