Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <cover.1629726117.git.ashish.kalra@amd.com> <6fd25c749205dd0b1eb492c60d41b124760cc6ae.1629726117.git.ashish.kalra@amd.com>
In-Reply-To: <6fd25c749205dd0b1eb492c60d41b124760cc6ae.1629726117.git.ashish.kalra@amd.com>
From:   Steve Rutherford <srutherford@google.com>
Date:   Wed, 15 Sep 2021 18:15:41 -0700
Message-ID: <CABayD+fnZ+Ho4qoUjB6YfWW+tFGUuftpsVBF3d=-kcU0-CEu0g@mail.gmail.com>
Subject: Re: [PATCH v6 1/5] x86/kvm: Add AMD SEV specific Hypercall3
To:     Ashish Kalra <Ashish.Kalra@amd.com>, seanjc@google.com
Cc:     pbonzini@redhat.com, tglx@linutronix.de, mingo@redhat.com,
        hpa@zytor.com, joro@8bytes.org, bp@alien8.de,
        thomas.lendacky@amd.com, x86@kernel.org, kvm@vger.kernel.org,
        linux-kernel@vger.kernel.org, brijesh.singh@amd.com,
        dovmurik@linux.ibm.com, tobin@linux.ibm.com, jejb@linux.ibm.com,
        dgilbert@redhat.com
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Tue, Aug 24, 2021 at 4:04 AM Ashish Kalra <Ashish.Kalra@amd.com> wrote:
>
> From: Brijesh Singh <brijesh.singh@amd.com>
>
> KVM hypercall framework relies on alternative framework to patch the
> VMCALL -> VMMCALL on AMD platform. If a hypercall is made before
> apply_alternative() is called then it defaults to VMCALL. The approach
> works fine on non SEV guest. A VMCALL would causes #UD, and hypervisor
> will be able to decode the instruction and do the right things. But
> when SEV is active, guest memory is encrypted with guest key and
> hypervisor will not be able to decode the instruction bytes.
>
> To highlight the need to provide this interface, capturing the
> flow of apply_alternatives() :
> setup_arch() call init_hypervisor_platform() which detects
> the hypervisor platform the kernel is running under and then the
> hypervisor specific initialization code can make early hypercalls.
> For example, KVM specific initialization in case of SEV will try
> to mark the "__bss_decrypted" section's encryption state via early
> page encryption status hypercalls.
>
> Now, apply_alternatives() is called much later when setup_arch()
> calls check_bugs(), so we do need some kind of an early,
> pre-alternatives hypercall interface. Other cases of pre-alternatives
> hypercalls include marking per-cpu GHCB pages as decrypted on SEV-ES
> and per-cpu apf_reason, steal_time and kvm_apic_eoi as decrypted for
> SEV generally.
>
> Add SEV specific hypercall3, it unconditionally uses VMMCALL. The hypercall
> will be used by the SEV guest to notify encrypted pages to the hypervisor.
>
> This kvm_sev_hypercall3() function is abstracted and used as follows :
> All these early hypercalls are made through early_set_memory_XX() interfaces,
> which in turn invoke pv_ops (paravirt_ops).
>
> This early_set_memory_XX() -> pv_ops.mmu.notify_page_enc_status_changed()
> is a generic interface and can easily have SEV, TDX and any other
> future platform specific abstractions added to it.
>
> Currently, pv_ops.mmu.notify_page_enc_status_changed() callback is setup to
> invoke kvm_sev_hypercall3() in case of SEV.
>
> Similarly, in case of TDX, pv_ops.mmu.notify_page_enc_status_changed()
> can be setup to a TDX specific callback.
>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: "H. Peter Anvin" <hpa@zytor.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Joerg Roedel <joro@8bytes.org>
> Cc: Borislav Petkov <bp@suse.de>
> Cc: Tom Lendacky <thomas.lendacky@amd.com>
> Cc: x86@kernel.org
> Cc: kvm@vger.kernel.org
> Cc: linux-kernel@vger.kernel.org
> Reviewed-by: Steve Rutherford <srutherford@google.com>
> Reviewed-by: Venu Busireddy <venu.busireddy@oracle.com>
> Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
> Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
> ---
>  arch/x86/include/asm/kvm_para.h | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
>
> diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h
> index 69299878b200..56935ebb1dfe 100644
> --- a/arch/x86/include/asm/kvm_para.h
> +++ b/arch/x86/include/asm/kvm_para.h
> @@ -83,6 +83,18 @@ static inline long kvm_hypercall4(unsigned int nr, unsigned long p1,
>         return ret;
>  }
>
> +static inline long kvm_sev_hypercall3(unsigned int nr, unsigned long p1,
> +                                     unsigned long p2, unsigned long p3)
> +{
> +       long ret;
> +
> +       asm volatile("vmmcall"
> +                    : "=a"(ret)
> +                    : "a"(nr), "b"(p1), "c"(p2), "d"(p3)
> +                    : "memory");
> +       return ret;
> +}
> +
>  #ifdef CONFIG_KVM_GUEST
>  void kvmclock_init(void);
>  void kvmclock_disable(void);
> --
> 2.17.1
>

Looking at these threads, this patch either:
1) Needs review/approval from a maintainer that is interested or
2) Should flip back to using alternative (as suggested by Sean). In
particular: `ALTERNATIVE("vmmcall", "vmcall",
ALT_NOT(X86_FEATURE_VMMCALL))`. My understanding is that the advantage
of this is that (after calling apply alternatives) you get exactly the
same behavior as before. But before apply alternatives, you get the
desired flipped behavior. The previous patch changed the behavior
after apply alternatives in a very slight manner (if feature flags
were not set, you'd get a different instruction).

I personally don't have strong feelings on this decision, but this
decision does need to be made for this patch series to move forward.

I'd also be curious to hear Sean's opinion on this since he was vocal
about this previously.

Thanks,
Steve