Received-SPF: pass (google.com: domain of linux-kernel+bounces-84387-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Precedence: bulk
MIME-Version: 1.0
References: <20240221-hid-bpf-sleepable-v3-0-1fb378ca6301@kernel.org>
 <20240221-hid-bpf-sleepable-v3-8-1fb378ca6301@kernel.org> <55177311ccdc24a74811d4a291ee1880044a5227.camel@gmail.com>
 <pocfd5n6lxriqg7r6usyhrlprgslclxs44jqoq63lw734fjl2g@5kv4hjaux2fp>
 <9a35a53a1887fb664fd540ec7e272cb3ea63f799.camel@gmail.com> <CAO-hwJ+TGiLrc4De7htvKaSsMfQnZahK-zONAMNgUMYHEQb-7g@mail.gmail.com>
In-Reply-To: <CAO-hwJ+TGiLrc4De7htvKaSsMfQnZahK-zONAMNgUMYHEQb-7g@mail.gmail.com>
From: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Date: Tue, 27 Feb 2024 17:49:11 -0800
Message-ID: <CAADnVQKrKzrvzu9NmcaDYGFYicqN--R5J6r--_J58gB0jic_NA@mail.gmail.com>
Subject: Re: [PATCH RFC bpf-next v3 08/16] bpf/verifier: do_misc_fixups for is_bpf_timer_set_sleepable_cb_kfunc
To: Benjamin Tissoires <benjamin.tissoires@redhat.com>
Cc: Eduard Zingerman <eddyz87@gmail.com>, Benjamin Tissoires <bentiss@kernel.org>, 
	Alexei Starovoitov <ast@kernel.org>, Daniel Borkmann <daniel@iogearbox.net>, 
	John Fastabend <john.fastabend@gmail.com>, Andrii Nakryiko <andrii@kernel.org>, 
	Martin KaFai Lau <martin.lau@linux.dev>, Song Liu <song@kernel.org>, 
	Yonghong Song <yonghong.song@linux.dev>, KP Singh <kpsingh@kernel.org>, 
	Stanislav Fomichev <sdf@google.com>, Hao Luo <haoluo@google.com>, Jiri Olsa <jolsa@kernel.org>, 
	Jiri Kosina <jikos@kernel.org>, Jonathan Corbet <corbet@lwn.net>, Shuah Khan <shuah@kernel.org>, 
	bpf <bpf@vger.kernel.org>, LKML <linux-kernel@vger.kernel.org>, 
	"open list:HID CORE LAYER" <linux-input@vger.kernel.org>, 
	"open list:DOCUMENTATION" <linux-doc@vger.kernel.org>, 
	"open list:KERNEL SELFTEST FRAMEWORK" <linux-kselftest@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Feb 27, 2024 at 8:51=E2=80=AFAM Benjamin Tissoires
<benjamin.tissoires@redhat.com> wrote:
>
> On Tue, Feb 27, 2024 at 5:36=E2=80=AFPM Eduard Zingerman <eddyz87@gmail.c=
om> wrote:
> >
> > On Tue, 2024-02-27 at 17:18 +0100, Benjamin Tissoires wrote:
> > [...]
> >
> > > Hmm, I must still be missing a piece of the puzzle:
> > > if I declare bpf_timer_set_sleepable_cb() to take a third "aux"
> > > argument, given that it is declared as kfunc, I also must declare it =
in
> > > my bpf program, or I get the following:
> > >
> > > # libbpf: extern (func ksym) 'bpf_timer_set_sleepable_cb': func_proto=
 [264] incompatible with vmlinux [18151]
> > >
> > > And if I declare it, then I don't know what to pass, given that this =
is
> > > purely added by the verifier:
> > >
> > > 43: (85) call bpf_timer_set_sleepable_cb#18152
> > > arg#2 pointer type STRUCT bpf_prog_aux must point to scalar, or struc=
t with scalar
> >
> > Right, something has to be done about number of arguments and we don't
> > have a convenient mechanism for this afaik.
> >
> > The simplest way would be to have two kfuncs:
> > - one with 2 arguments, used form bpf program;
> > - another with 3 arguments, used at runtime;
> > - replace former by latter during rewrite.
>
> It's hacky but seems interesting enough to be tested :)

Too hacky imo :)

Let's follow the existing pattern.
See:
__bpf_kfunc void *bpf_obj_new_impl(u64 local_type_id__k, void *meta__ign)

__ign suffix tells the verifier to ignore it.

Then we do:
#define bpf_obj_new(type) \
  ((type *)bpf_obj_new_impl(bpf_core_type_id_local(type), NULL))

and later the verifier replaces arg2 with the correct pointer.

> We also could use the suffix (like __uninit, __k, etc...), but it
> might introduce more headaches than the 2 kfuncs you are proposing.

Only one kfunc pls. Let's not make it more complex than necessary.

We cannot easily add a suffix to tell libbpf to ignore that arg,
since bpf_core_types_are_compat() compares types and there are
no argument names in the types.
So it will be a significant surgery for libbpf to find the arg name
in vmlinux BTF and strcmp the suffix.

>
> >
> > Could you please provide more details on what exactly it complains abou=
t?
> >
>
> Well, there is a simple reason: that code is never reached because, in
> that function, there is a `if (insn->src_reg =3D=3D
> BPF_PSEUDO_KFUNC_CALL)` above that unconditionally terminates with a
> `continue`. So basically this part of the code is never hit.
>
> I'll include that new third argument and the dual kfunc call in
> fixup_kfunc_call() and report if it works from here.

Something is wrong. fixup_kfunc_call() can rewrite args with whatever
it wants.
Are you sure you've added bpf_timer_set_sleepable_cb to special_kfunc_list =
?