Received-SPF: pass (google.com: domain of linux-kernel+bounces-157301-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Precedence: bulk
MIME-Version: 1.0
References: <20240420-bpf_wq-v2-0-6c986a5a741f@kernel.org> <20240420-bpf_wq-v2-11-6c986a5a741f@kernel.org>
 <CAADnVQJ9Qw6Lr644xRTU-n16UkBCyHoTAQs7QqGiniOdsOVAJg@mail.gmail.com> <CAADnVQKfzu1F=xZxyYhiocAn1iM=8f13Ca-2Jfht2dXsXuGu9A@mail.gmail.com>
In-Reply-To: <CAADnVQKfzu1F=xZxyYhiocAn1iM=8f13Ca-2Jfht2dXsXuGu9A@mail.gmail.com>
From: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Date: Wed, 24 Apr 2024 09:14:11 -0700
Message-ID: <CAADnVQJTyXS--chM61Ysk7sDNXoUqPctprXwZ8DEcJMHa9Uy3w@mail.gmail.com>
Subject: Re: [PATCH bpf-next v2 11/16] bpf: wq: add bpf_wq_init
To: Benjamin Tissoires <bentiss@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>, Daniel Borkmann <daniel@iogearbox.net>, 
	Andrii Nakryiko <andrii@kernel.org>, Martin KaFai Lau <martin.lau@linux.dev>, 
	Eduard Zingerman <eddyz87@gmail.com>, Song Liu <song@kernel.org>, 
	Yonghong Song <yonghong.song@linux.dev>, John Fastabend <john.fastabend@gmail.com>, 
	KP Singh <kpsingh@kernel.org>, Stanislav Fomichev <sdf@google.com>, Hao Luo <haoluo@google.com>, 
	Jiri Olsa <jolsa@kernel.org>, Mykola Lysenko <mykolal@fb.com>, Shuah Khan <shuah@kernel.org>, 
	bpf <bpf@vger.kernel.org>, LKML <linux-kernel@vger.kernel.org>, 
	"open list:KERNEL SELFTEST FRAMEWORK" <linux-kselftest@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, Apr 24, 2024 at 8:06=E2=80=AFAM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Tue, Apr 23, 2024 at 7:55=E2=80=AFPM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
> >
> > On Sat, Apr 20, 2024 at 2:10=E2=80=AFAM Benjamin Tissoires <bentiss@ker=
nel.org> wrote:
> > >
> > > We need to teach the verifier about the second argument which is decl=
ared
> > > as void * but which is of type KF_ARG_PTR_TO_MAP. We could have dropp=
ed
> > > this extra case if we declared the second argument as struct bpf_map =
*,
> > > but that means users will have to do extra casting to have their prog=
ram
> > > compile.
> > >
> > > We also need to duplicate the timer code for the checking if the map
> > > argument is matching the provided workqueue.
> > >
> > > Signed-off-by: Benjamin Tissoires <bentiss@kernel.org>
> > >
> > > ---
> > >
> > > FWIW, I still have one concern with this implementation:
> > > - bpf_wq_work() access ->prog without protection, but I think this mi=
ght
> > >   be racing with bpf_wq_set_callback(): if we have the following:
> > >
> > >   CPU 0                                     CPU 1
> > >   bpf_wq_set_callback()
> > >   bpf_start()
> > >                                             bpf_wq_work():
> > >                                               prog =3D cb->prog;
> > >
> > >   bpf_wq_set_callback()
> > >     cb->prog =3D prog;
> > >     bpf_prog_put(prev)
> > >     rcu_assign_ptr(cb->callback_fn,
> > >                    callback_fn);
> > >                                            callback =3D READ_ONCE(w->=
cb.callback_fn);
> > >
> > >   As I understand callback_fn is fine, prog might be, but we clearly
> > >   have an inconstency between "prog" and "callback_fn" as they can co=
me
> > >   from 2 different bpf_wq_set_callback() calls.
> > >
> > >   IMO we should protect this by the async->lock, but I'm not sure if
> > >   it's OK or not.
> >
> > I see the concern, but I think it's overkill.
> > Here 'prog' is used to pass it into __bpf_prog_enter_sleepable_recur()
> > to keep the standard pattern of calling into sleepable prog.
> > But it won't recurse.
> > We can open code migrate_disable,etc from there except this_cpu_inc_ret=
urn,
> > but it's an overkill.
> > The passed 'prog' is irrelevant.
> > If somebody tries really hard by having two progs sharing the same
> > map with bpf_wq and racing to set_callback... I can see how
> > prog won't match callback, but it won't make a difference.
> > prog is not going trigger recursion check (unless somebody
> > tries is obsessed) and not going to UAF.
> > I imagine it's possible to attach somewhere in core wq callback
> > invocation path with fentry, set_callback to the same prog,
> > and technically it's kinda sorta recursion, but different subprogs,
> > so not a safety issue.
> > The code as-is is fine. imo.
>
> After sleeping on it, I realized that the use of
> __bpf_prog_enter_sleepable_recur() here is very much incorrect :(
> The tests are passing only because we don't inc prog->active
> when we run the prog via prog_run cmd.
> Adding the following:
> diff --git a/net/bpf/test_run.c b/net/bpf/test_run.c
> index f6aad4ed2ab2..0732dfe22204 100644
> --- a/net/bpf/test_run.c
> +++ b/net/bpf/test_run.c
> @@ -1514,7 +1514,9 @@ int bpf_prog_test_run_syscall(struct bpf_prog *prog=
,
>         }
>
>         rcu_read_lock_trace();
> +       this_cpu_inc_return(*(prog->active));
>         retval =3D bpf_prog_run_pin_on_cpu(prog, ctx);
> +       this_cpu_dec(*(prog->active));
>         rcu_read_unlock_trace();
>
> makes the test fail sporadically.
> Or 100% fail when the kernel is booted with 1 cpu.
>
> Could you send a quick follow up to
> replace __bpf_prog_enter_sleepable_recur() with
>         rcu_read_lock_trace();
>         migrate_disable();
> ?
>
> Or I'll do it in an hour or so.

Considering two broken-build reports already
I've applied the following fix:
https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=
=3Ddc92febf7b93da5049fe177804e6b1961fcc6bd7

that addresses the build issue on !JIT and fixes this recursion problem.