Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20230607125911.145345-1-imagedong@tencent.com>
 <20230607125911.145345-2-imagedong@tencent.com> <20230607200905.5tbosnupodvydezq@macbook-pro-8.dhcp.thefacebook.com>
 <CADxym3abYOZ5JVa4FP5R-Vi7HAk=n_0vTmMGveDH8xvFtuaBDw@mail.gmail.com> <2fb8c454-1ae7-27cd-a9fa-0d8dda18a900@meta.com>
In-Reply-To: <2fb8c454-1ae7-27cd-a9fa-0d8dda18a900@meta.com>
From:   Menglong Dong <menglong8.dong@gmail.com>
Date:   Fri, 9 Jun 2023 10:34:32 +0800
Message-ID: <CADxym3Y03+n+pDupWbdrFtXU96vsWR6h40PSrVC+jJzwov=A5Q@mail.gmail.com>
Subject: Re: [PATCH bpf-next v3 1/3] bpf, x86: allow function arguments up to
 12 for TRACING
To:     Yonghong Song <yhs@meta.com>
Cc:     Alexei Starovoitov <alexei.starovoitov@gmail.com>,
        davem@davemloft.net, dsahern@kernel.org, ast@kernel.org,
        daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev,
        song@kernel.org, yhs@fb.com, john.fastabend@gmail.com,
        kpsingh@kernel.org, sdf@google.com, x86@kernel.org,
        imagedong@tencent.com, benbjiang@tencent.com,
        netdev@vger.kernel.org, bpf@vger.kernel.org,
        linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Fri, Jun 9, 2023 at 5:12=E2=80=AFAM Yonghong Song <yhs@meta.com> wrote:
>
>
>
> On 6/7/23 8:17 PM, Menglong Dong wrote:
> > On Thu, Jun 8, 2023 at 4:09=E2=80=AFAM Alexei Starovoitov
> > <alexei.starovoitov@gmail.com> wrote:
> >>
> >> On Wed, Jun 07, 2023 at 08:59:09PM +0800, menglong8.dong@gmail.com wro=
te:
> >>> From: Menglong Dong <imagedong@tencent.com>
> >>>
> >>> For now, the BPF program of type BPF_PROG_TYPE_TRACING can only be us=
ed
> >>> on the kernel functions whose arguments count less than 6. This is no=
t
> >>> friendly at all, as too many functions have arguments count more than=
 6.
> >>>
> >>> Therefore, let's enhance it by increasing the function arguments coun=
t
> >>> allowed in arch_prepare_bpf_trampoline(), for now, only x86_64.
> >>>
> >>> For the case that we don't need to call origin function, which means
> >>> without BPF_TRAMP_F_CALL_ORIG, we need only copy the function argumen=
ts
> >>> that stored in the frame of the caller to current frame. The argument=
s
> >>> of arg6-argN are stored in "$rbp + 0x18", we need copy them to
> >>> "$rbp - regs_off + (6 * 8)".
> >>>
> >>> For the case with BPF_TRAMP_F_CALL_ORIG, we need prepare the argument=
s
> >>> in stack before call origin function, which means we need alloc extra
> >>> "8 * (arg_count - 6)" memory in the top of the stack. Note, there sho=
uld
> >>> not be any data be pushed to the stack before call the origin functio=
n.
> >>> Then, we have to store rbx with 'mov' instead of 'push'.
> >>
> >> x86-64 psABI requires stack to be 16-byte aligned when args are passed=
 on the stack.
> >> I don't see this logic in the patch.
> >
> > Yeah, it seems I missed this logic......:)
> >
> > I have not figure out the rule of the alignment, but after
> > observing the behavior of the compiler, the stack seems
> > should be like this:
> >
> > ------ stack frame begin
> > rbp
> >
> > xxx   -- this part should be aligned in 16-byte
> >
> > ------ end of arguments in stack
> > xxx
> > ------ begin of arguments in stack
> >
> > So the code should be:
> >
> > +       if (nr_regs > 6 && (flags & BPF_TRAMP_F_CALL_ORIG)) {
> > +                stack_size =3D ALIGN(stack_size, 16);
> > +                stack_size +=3D (nr_regs - 6) * 8;
> > +       }
> >
> > Am I right?
>
> This is the stack_size, you should ensure stack pointer is 16-byte aligne=
d.

Oh, I see. Considering the begin of the stack frame
should already be 16-byte aligned, what we should
do here is to make the size of the current stack frame
16-byte aligned. Then, rsp will be 16-byte aligned.

Am I right?

Which means the code should be:

+       if (nr_regs > 6 && (flags & BPF_TRAMP_F_CALL_ORIG)) {
+               stack_size +=3D (nr_regs - 6) * 8;
+               stack_size =3D ALIGN(stack_size, 16);
+       }

Then, the size of current stack frame will be:
stack_size + 8(rbp) + 8(rip)

This is the example that I refer to:
https://godbolt.org/z/7o9nh4nbc

>
> >
> > Thanks!
> > Menglong Dong