Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20200430005127.2205-1-luke.r.nels@gmail.com>
In-Reply-To: <20200430005127.2205-1-luke.r.nels@gmail.com>
From:   Xi Wang <xi.wang@gmail.com>
Date:   Wed, 29 Apr 2020 19:14:49 -0700
Message-ID: <CAKU6vybAuF-oziH8oOu1oCv+j8SLOMWq2UdM6_kVCbeggLvxSA@mail.gmail.com>
Subject: Re: [PATCH bpf-next] bpf, riscv: Fix stack layout of JITed code on RV32
To:     Luke Nelson <lukenels@cs.washington.edu>
Cc:     bpf@vger.kernel.org, Luke Nelson <luke.r.nels@gmail.com>,
        Alexei Starovoitov <ast@kernel.org>,
        Daniel Borkmann <daniel@iogearbox.net>,
        Martin KaFai Lau <kafai@fb.com>,
        Song Liu <songliubraving@fb.com>, Yonghong Song <yhs@fb.com>,
        Andrii Nakryiko <andriin@fb.com>,
        John Fastabend <john.fastabend@gmail.com>,
        KP Singh <kpsingh@chromium.org>,
        Paul Walmsley <paul.walmsley@sifive.com>,
        Palmer Dabbelt <palmer@dabbelt.com>,
        Albert Ou <aou@eecs.berkeley.edu>, netdev@vger.kernel.org,
        linux-riscv@lists.infradead.org,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, Apr 29, 2020 at 5:51 PM Luke Nelson <lukenels@cs.washington.edu> wrote:
>
> This patch fixes issues with stackframe unwinding and alignment in the
> current stack layout for BPF programs on RV32.
>
> In the current layout, RV32 fp points to the JIT scratch registers, rather
> than to the callee-saved registers. This breaks stackframe unwinding,
> which expects fp to point just above the saved ra and fp registers.
>
> This patch fixes the issue by moving the callee-saved registers to be
> stored on the top of the stack, pointed to by fp. This satisfies the
> assumptions of stackframe unwinding.
>
> This patch also fixes an issue with the old layout that the stack was
> not aligned to 16 bytes.
>
> Stacktrace from JITed code using the old stack layout:
>
>   [   12.196249 ] [<c0402200>] walk_stackframe+0x0/0x96
>
> Stacktrace using the new stack layout:
>
>   [   13.062888 ] [<c0402200>] walk_stackframe+0x0/0x96
>   [   13.063028 ] [<c04023c6>] show_stack+0x28/0x32
>   [   13.063253 ] [<a403e778>] bpf_prog_82b916b2dfa00464+0x80/0x908
>   [   13.063417 ] [<c09270b2>] bpf_test_run+0x124/0x39a
>   [   13.063553 ] [<c09276c0>] bpf_prog_test_run_skb+0x234/0x448
>   [   13.063704 ] [<c048510e>] __do_sys_bpf+0x766/0x13b4
>   [   13.063840 ] [<c0485d82>] sys_bpf+0xc/0x14
>   [   13.063961 ] [<c04010f0>] ret_from_syscall+0x0/0x2
>
> The new code is also simpler to understand and includes an ASCII diagram
> of the stack layout.
>
> Tested on riscv32 QEMU virt machine.
>
> Signed-off-by: Luke Nelson <luke.r.nels@gmail.com>

Thanks for the fix!

Acked-by: Xi Wang <xi.wang@gmail.com>