Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <34fb3a9841bf4977413be799f7cbef78560aaa20.camel@gmail.com>
Subject: Re: [PATCH] bpf, x86: allow function arguments up to 12 for TRACING
From:   Eduard Zingerman <eddyz87@gmail.com>
To:     Menglong Dong <menglong8.dong@gmail.com>,
        Jiri Olsa <olsajiri@gmail.com>
Cc:     dsahern@kernel.org, andrii@kernel.org, davem@davemloft.net,
        ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev,
        song@kernel.org, yhs@fb.com, john.fastabend@gmail.com,
        kpsingh@kernel.org, sdf@google.com, haoluo@google.com,
        tglx@linutronix.de, mingo@redhat.com, bp@alien8.de,
        dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com,
        netdev@vger.kernel.org, bpf@vger.kernel.org,
        linux-kernel@vger.kernel.org, Menglong Dong <imagedong@tencent.com>
Date:   Wed, 31 May 2023 15:02:29 +0300
In-Reply-To: <CADxym3biE8WcMxWf1wok+s4pBYEi6+fYQAbZJVxm7eBfzWLjLQ@mail.gmail.com>
References: <20230530044423.3897681-1-imagedong@tencent.com>
         <ZHb+ypjE4Ybg3O18@krava>
         <CADxym3biE8WcMxWf1wok+s4pBYEi6+fYQAbZJVxm7eBfzWLjLQ@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.44.4-0ubuntu1 
MIME-Version: 1.0
Precedence: bulk

On Wed, 2023-05-31 at 17:03 +0800, Menglong Dong wrote:
> On Wed, May 31, 2023 at 4:01=E2=80=AFPM Jiri Olsa <olsajiri@gmail.com> wr=
ote:
> >=20
> > On Tue, May 30, 2023 at 12:44:23PM +0800, menglong8.dong@gmail.com wrot=
e:
> > > From: Menglong Dong <imagedong@tencent.com>
> > >=20
> > > For now, the BPF program of type BPF_PROG_TYPE_TRACING can only be us=
ed
> > > on the kernel functions whose arguments count less than 6. This is no=
t
> > > friendly at all, as too many functions have arguments count more than=
 6.
> > >=20
> > > Therefore, let's enhance it by increasing the function arguments coun=
t
> > > allowed in arch_prepare_bpf_trampoline(), for now, only x86_64.
> > >=20
> > > For the case that we don't need to call origin function, which means
> > > without BPF_TRAMP_F_CALL_ORIG, we need only copy the function argumen=
ts
> > > that stored in the frame of the caller to current frame. The argument=
s
> > > of arg6-argN are stored in "$rbp + 0x18", we need copy them to
> > > "$rbp - regs_off + (6 * 8)".
> > >=20
> > > For the case with BPF_TRAMP_F_CALL_ORIG, we need prepare the argument=
s
> > > in stack before call origin function, which means we need alloc extra
> > > "8 * (arg_count - 6)" memory in the top of the stack. Note, there sho=
uld
> > > not be any data be pushed to the stack before call the origin functio=
n.
> > > Then, we have to store rbx with 'mov' instead of 'push'.
> > >=20
> > > It works well for the FENTRY and FEXIT, I'm not sure if there are oth=
er
> > > complicated cases.
> > >=20
> > > Signed-off-by: Menglong Dong <imagedong@tencent.com>
> > > ---
> > >  arch/x86/net/bpf_jit_comp.c | 88 ++++++++++++++++++++++++++++++++---=
--
> >=20
> > please add selftests for this.. I had to add one to be able to check
> > the generated trampoline
> >=20
>=20
> Okay!
>=20
> BTW, I failed to compile the latest selftests/bpf with
> the following errors:
>=20
> progs/verifier_and.c:58:16: error: invalid operand for instruction
>         asm volatile ("                                 \
>=20

These tests were moved to use inline assembly recently (2 month ago).
Discussion at the time was whether to use \n\ or \ terminators at the
end of each line. People opted for \ as easier to read.
Replacing \ with \n\ and compiling this test using clang 14 shows
more informative error message:

$ make -j14 `pwd`/verifier_and.bpf.o
  CLNG-BPF [test_maps] verifier_and.bpf.o
progs/verifier_and.c:68:1: error: invalid operand for instruction
        w1 %%=3D 2;                                       \n\
^
<inline asm>:11:5: note: instantiated into assembly here
        w1 %=3D 2;=20

My guess is that clang 14 does not know how to handle operations on
32-bit sub-registers w[0-9].

But using clang 14 I get some errors not related to inline assembly as well=
.
Also, I recall that there were runtime issues with clang 14 and
tests using enum64.

All-in-all, you need newer version of clang for tests nowadays,
sorry for inconvenience.

> The version of clang I used is:
>=20
> clang --version
> Debian clang version 14.0.6
> Target: x86_64-pc-linux-gnu
> Thread model: posix
> InstalledDir: /usr/bin
>=20
> Does anyone know the reason?
>=20
> Thanks!
> Menglong Dong
>=20
> > jirka
> >=20
> >=20
>=20