Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20220607165105.639716-1-kaleshsingh@google.com>
 <20220607165105.639716-4-kaleshsingh@google.com> <87k09rzk0o.wl-maz@kernel.org>
In-Reply-To: <87k09rzk0o.wl-maz@kernel.org>
From:   Kalesh Singh <kaleshsingh@google.com>
Date:   Wed, 8 Jun 2022 10:52:53 -0700
Message-ID: <CAC_TJvdCuGNEJC4M+bV6o48CSJRs_4GEUb3iiP_4ro79q=KesA@mail.gmail.com>
Subject: Re: [PATCH v3 3/5] KVM: arm64: Add hypervisor overflow stack
To:     Marc Zyngier <maz@kernel.org>
Cc:     Mark Rutland <mark.rutland@arm.com>,
        Mark Brown <broonie@kernel.org>, Will Deacon <will@kernel.org>,
        Quentin Perret <qperret@google.com>,
        Fuad Tabba <tabba@google.com>,
        Suren Baghdasaryan <surenb@google.com>,
        "T.J. Mercier" <tjmercier@google.com>,
        "Cc: Android Kernel" <kernel-team@android.com>,
        James Morse <james.morse@arm.com>,
        Alexandru Elisei <alexandru.elisei@arm.com>,
        Suzuki K Poulose <suzuki.poulose@arm.com>,
        Catalin Marinas <catalin.marinas@arm.com>,
        Masami Hiramatsu <mhiramat@kernel.org>,
        Alexei Starovoitov <ast@kernel.org>,
        "Madhavan T. Venkataraman" <madvenka@linux.microsoft.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Andrew Jones <drjones@redhat.com>,
        Zenghui Yu <yuzenghui@huawei.com>,
        Kefeng Wang <wangkefeng.wang@huawei.com>,
        Keir Fraser <keirf@google.com>,
        Ard Biesheuvel <ardb@kernel.org>,
        Oliver Upton <oupton@google.com>,
        "moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)" 
        <linux-arm-kernel@lists.infradead.org>,
        kvmarm <kvmarm@lists.cs.columbia.edu>,
        LKML <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Wed, Jun 8, 2022 at 12:34 AM Marc Zyngier <maz@kernel.org> wrote:
>
> On Tue, 07 Jun 2022 17:50:45 +0100,
> Kalesh Singh <kaleshsingh@google.com> wrote:
> >
> > Allocate and switch to 16-byte aligned secondary stack on overflow. This
> > provides us stack space to better handle overflows; and is used in
> > a subsequent patch to dump the hypervisor stacktrace.
> >
> > Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
> > ---
> >  arch/arm64/kernel/stacktrace.c | 3 +++
> >  arch/arm64/kvm/hyp/nvhe/host.S | 9 ++-------
> >  2 files changed, 5 insertions(+), 7 deletions(-)
> >
> > diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c
> > index a84e38d41d38..f346b4c66f1c 100644
> > --- a/arch/arm64/kernel/stacktrace.c
> > +++ b/arch/arm64/kernel/stacktrace.c
> > @@ -242,4 +242,7 @@ noinline notrace void arch_stack_walk(stack_trace_consume_fn consume_entry,
> >
> >       unwind(task, &state, consume_entry, cookie);
> >  }
> > +#else /* __KVM_NVHE_HYPERVISOR__ */
> > +DEFINE_PER_CPU(unsigned long [PAGE_SIZE/sizeof(long)], overflow_stack)
> > +     __aligned(16);
>
> Does this need to be a whole page? With 64kB pages, this is
> potentially a lot of memory for something that will hardly ever be
> used. The rest of the kernel limits this to 4kB, which seems more
> reasonable. There is no guard page anyway, so PAGE_SIZE doesn't
> provide any extra protection.

My oversight on the !4kB page sizes. I think this could be as small as:

    (STACK_SIZE - 1) / 2 + sizeof(long)

         '/ 2'                        : Min frame size (x29, x30)
         '+ sizeof(long)'      : To round up

since we only save the one address (PC) for each frame. WDYT?

Thanks,
Kalesh

>
> >  #endif /* !__KVM_NVHE_HYPERVISOR__ */
> > diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S
> > index ea6a397b64a6..4e3032a244e1 100644
> > --- a/arch/arm64/kvm/hyp/nvhe/host.S
> > +++ b/arch/arm64/kvm/hyp/nvhe/host.S
> > @@ -177,13 +177,8 @@ SYM_FUNC_END(__host_hvc)
> >       b       hyp_panic
> >
> >  .L__hyp_sp_overflow\@:
> > -     /*
> > -      * Reset SP to the top of the stack, to allow handling the hyp_panic.
> > -      * This corrupts the stack but is ok, since we won't be attempting
> > -      * any unwinding here.
> > -      */
> > -     ldr_this_cpu    x0, kvm_init_params + NVHE_INIT_STACK_HYP_VA, x1
> > -     mov     sp, x0
> > +     /* Switch to the overflow stack */
> > +     adr_this_cpu sp, overflow_stack + PAGE_SIZE, x0
> >
> >       b       hyp_panic_bad_stack
> >       ASM_BUG()
> > --
> > 2.36.1.255.ge46751e96f-goog
> >
> >
>
> Thanks,
>
>         M.
>
> --
> Without deviation from the norm, progress is not possible.