Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20220408200349.1529080-1-kaleshsingh@google.com>
 <20220408200349.1529080-6-kaleshsingh@google.com> <87v8v6aek0.wl-maz@kernel.org>
In-Reply-To: <87v8v6aek0.wl-maz@kernel.org>
From:   Kalesh Singh <kaleshsingh@google.com>
Date:   Mon, 18 Apr 2022 19:41:00 -0700
Message-ID: <CAC_TJvd=ubTMM+BRdELFokbxiKqaePAprG-pq3fR1vjKV9L6aA@mail.gmail.com>
Subject: Re: [PATCH v7 5/6] KVM: arm64: Detect and handle hypervisor stack overflows
To:     Marc Zyngier <maz@kernel.org>
Cc:     Will Deacon <will@kernel.org>, Quentin Perret <qperret@google.com>,
        Fuad Tabba <tabba@google.com>,
        Suren Baghdasaryan <surenb@google.com>,
        "Cc: Android Kernel" <kernel-team@android.com>,
        James Morse <james.morse@arm.com>,
        Alexandru Elisei <alexandru.elisei@arm.com>,
        Suzuki K Poulose <suzuki.poulose@arm.com>,
        Catalin Marinas <catalin.marinas@arm.com>,
        Andrew Walbran <qwandor@google.com>,
        Mark Rutland <mark.rutland@arm.com>,
        Ard Biesheuvel <ardb@kernel.org>,
        Zenghui Yu <yuzenghui@huawei.com>,
        Andrew Jones <drjones@redhat.com>,
        Nathan Chancellor <nathan@kernel.org>,
        Masahiro Yamada <masahiroy@kernel.org>,
        "moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)" 
        <linux-arm-kernel@lists.infradead.org>,
        kvmarm <kvmarm@lists.cs.columbia.edu>,
        LKML <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Mon, Apr 18, 2022 at 3:09 AM Marc Zyngier <maz@kernel.org> wrote:
>
> On Fri, 08 Apr 2022 21:03:28 +0100,
> Kalesh Singh <kaleshsingh@google.com> wrote:
> >
> > The hypervisor stacks (for both nVHE Hyp mode and nVHE protected mode)
> > are aligned such  that any valid stack address has PAGE_SHIFT bit as 1.
> > This allows us to conveniently check for overflow in the exception entry
> > without corrupting any GPRs. We won't recover from a stack overflow so
> > panic the hypervisor.
> >
> > Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
> > Tested-by: Fuad Tabba <tabba@google.com>
> > Reviewed-by: Fuad Tabba <tabba@google.com>
> > ---
> >
> > Changes in v7:
> >   - Add Fuad's Reviewed-by and Tested-by tags.
> >
> > Changes in v5:
> >   - Valid stack addresses now have PAGE_SHIFT bit as 1 instead of 0
> >
> > Changes in v3:
> >   - Remove test_sp_overflow macro, per Mark
> >   - Add asmlinkage attribute for hyp_panic, hyp_panic_bad_stack, per Ard
> >
> >
> >  arch/arm64/kvm/hyp/nvhe/host.S   | 24 ++++++++++++++++++++++++
> >  arch/arm64/kvm/hyp/nvhe/switch.c |  7 ++++++-
> >  2 files changed, 30 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S
> > index 3d613e721a75..be6d844279b1 100644
> > --- a/arch/arm64/kvm/hyp/nvhe/host.S
> > +++ b/arch/arm64/kvm/hyp/nvhe/host.S
> > @@ -153,6 +153,18 @@ SYM_FUNC_END(__host_hvc)
> >
> >  .macro invalid_host_el2_vect
> >       .align 7
> > +
> > +     /*
> > +      * Test whether the SP has overflowed, without corrupting a GPR.
> > +      * nVHE hypervisor stacks are aligned so that the PAGE_SHIFT bit
> > +      * of SP should always be 1.
> > +      */
> > +     add     sp, sp, x0                      // sp' = sp + x0
> > +     sub     x0, sp, x0                      // x0' = sp' - x0 = (sp + x0) - x0 = sp
> > +     tbz     x0, #PAGE_SHIFT, .L__hyp_sp_overflow\@
> > +     sub     x0, sp, x0                      // x0'' = sp' - x0' = (sp + x0) - sp = x0
> > +     sub     sp, sp, x0                      // sp'' = sp' - x0 = (sp + x0) - x0 = sp
> > +
> >       /* If a guest is loaded, panic out of it. */
> >       stp     x0, x1, [sp, #-16]!
> >       get_loaded_vcpu x0, x1
> > @@ -165,6 +177,18 @@ SYM_FUNC_END(__host_hvc)
> >        * been partially clobbered by __host_enter.
> >        */
> >       b       hyp_panic
> > +
> > +.L__hyp_sp_overflow\@:
> > +     /*
> > +      * Reset SP to the top of the stack, to allow handling the hyp_panic.
> > +      * This corrupts the stack but is ok, since we won't be attempting
> > +      * any unwinding here.
> > +      */
> > +     ldr_this_cpu    x0, kvm_init_params + NVHE_INIT_STACK_HYP_VA, x1
> > +     mov     sp, x0
> > +
> > +     bl      hyp_panic_bad_stack
>
> Why bl? You clearly don't expect to return here, given that you have
> an ASM_BUG() right below, and that you are calling a __no_return
> function. I think we should be consistent with the rest of the code
> and just do a simple branch.

The idea was to use bl  to give the hyp_panic_bad_stack() frame in the
stack trace, which makes it easy to identify overflows. I can add a
comment and drop the redundant ASM_BUG()

Thanks,
Kalesh

>
> It also gives us a chance to preserve an extra register from the
> context.
>
> > +     ASM_BUG()
> >  .endm
> >
> >  .macro invalid_host_el1_vect
> > diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c
> > index 6410d21d8695..703a5d3f611b 100644
> > --- a/arch/arm64/kvm/hyp/nvhe/switch.c
> > +++ b/arch/arm64/kvm/hyp/nvhe/switch.c
> > @@ -347,7 +347,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu)
> >       return exit_code;
> >  }
> >
> > -void __noreturn hyp_panic(void)
> > +asmlinkage void __noreturn hyp_panic(void)
> >  {
> >       u64 spsr = read_sysreg_el2(SYS_SPSR);
> >       u64 elr = read_sysreg_el2(SYS_ELR);
> > @@ -369,6 +369,11 @@ void __noreturn hyp_panic(void)
> >       unreachable();
> >  }
> >
> > +asmlinkage void __noreturn hyp_panic_bad_stack(void)
> > +{
> > +     hyp_panic();
> > +}
> > +
> >  asmlinkage void kvm_unexpected_el2_exception(void)
> >  {
> >       return __kvm_unexpected_el2_exception();
> > --
> > 2.35.1.1178.g4f1659d476-goog
> >
> >
>
> Thanks,
>
>         M.
>
> --
> Without deviation from the norm, progress is not possible.