Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <CAN+4W8iRH6kpDmmY8i5r1nKbckaYghmOCqRXe+4bDHE7vzVMMA@mail.gmail.com>
 <20230706153327.99298-1-kuniyu@amazon.com>
In-Reply-To: <20230706153327.99298-1-kuniyu@amazon.com>
From:   Alexei Starovoitov <alexei.starovoitov@gmail.com>
Date:   Mon, 10 Jul 2023 17:41:09 -0700
Message-ID: <CAADnVQK5gorOuM+GTANJjrwTNSfVcEO-cL_ESqXOjUBdpJLvGQ@mail.gmail.com>
Subject: Re: [PATCH bpf-next v4 6/7] bpf, net: Support SO_REUSEPORT sockets
 with bpf_sk_assign
To:     Kuniyuki Iwashima <kuniyu@amazon.com>
Cc:     Lorenz Bauer <lmb@isovalent.com>,
        Andrii Nakryiko <andrii@kernel.org>,
        Alexei Starovoitov <ast@kernel.org>, bpf <bpf@vger.kernel.org>,
        Daniel Borkmann <daniel@iogearbox.net>,
        "David S. Miller" <davem@davemloft.net>,
        David Ahern <dsahern@kernel.org>,
        Eric Dumazet <edumazet@google.com>,
        Hao Luo <haoluo@google.com>,
        Hemanth Malla <hemanthmalla@gmail.com>,
        Joe Stringer <joe@cilium.io>, Joe Stringer <joe@wand.net.nz>,
        John Fastabend <john.fastabend@gmail.com>,
        Jiri Olsa <jolsa@kernel.org>, KP Singh <kpsingh@kernel.org>,
        Jakub Kicinski <kuba@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        "open list:KERNEL SELFTEST FRAMEWORK" 
        <linux-kselftest@vger.kernel.org>,
        Martin KaFai Lau <martin.lau@linux.dev>,
        Mykola Lysenko <mykolal@fb.com>,
        Network Development <netdev@vger.kernel.org>,
        Paolo Abeni <pabeni@redhat.com>,
        Stanislav Fomichev <sdf@google.com>,
        Shuah Khan <shuah@kernel.org>, Song Liu <song@kernel.org>,
        Willem de Bruijn <willemdebruijn.kernel@gmail.com>,
        Yonghong Song <yhs@fb.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Thu, Jul 6, 2023 at 8:33=E2=80=AFAM Kuniyuki Iwashima <kuniyu@amazon.com=
> wrote:
>
> From: Lorenz Bauer <lmb@isovalent.com>
> Date: Thu, 6 Jul 2023 09:11:15 +0100
> > On Thu, Jul 6, 2023 at 1:41=E2=80=AFAM Kuniyuki Iwashima <kuniyu@amazon=
.com> wrote:
> > >
> > > Sorry for late reply.
> > >
> > > What we know about sk before inet6?_lookup_reuseport() are
> > >
> > >   (1) sk was full socket in bpf_sk_assign()
> > >   (2) sk had SOCK_RCU_FREE in bpf_sk_assign()
> > >   (3) sk was TCP_LISTEN here if TCP
> >
> > Are we looking at the same bpf_sk_assign? Confusingly there are two
> > very similarly named functions. The one we care about is:
> >
> > BPF_CALL_3(bpf_sk_assign, struct sk_buff *, skb, struct sock *, sk, u64=
, flags)
> > {
> >     if (!sk || flags !=3D 0)
> >         return -EINVAL;
> >     if (!skb_at_tc_ingress(skb))
> >         return -EOPNOTSUPP;
> >     if (unlikely(dev_net(skb->dev) !=3D sock_net(sk)))
> >         return -ENETUNREACH;
> >     if (sk_is_refcounted(sk) &&
> >         unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
> >         return -ENOENT;
> >
> >     skb_orphan(skb);
> >     skb->sk =3D sk;
> >     skb->destructor =3D sock_pfree;
> >
> >     return 0;
> > }
> >
> > From this we can't tell what state the socket is in or whether it is
> > RCU freed or not.
>
> But we can in inet6?_steal_sock() by calling sk_is_refcounted() again
> via skb_steal_sock().
>
> In inet6?_steal_sock(), we call inet6?_lookup_reuseport() only for
> sk that was a TCP listener or UDP non-connected socket until just before
> the sk_state checks.  Then, we know *refcounted should be false for such
> sockets even before inet6?_lookup_reuseport().
>
> After the checks, sk might be poped out of the reuseport group before
> inet6?_lookup_reuseport() and reuse_sk might be NULL, but it's not
> related because *refcounted is a value for sk, not for reuse_sk.

I was about to apply v5 before I noticed this discussion on v4.
Sounds like v6 will be needed.
Next time please continue discussion in the latest version.