Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Kuniyuki Iwashima <kuniyu@amazon.com>
To:     <lmb@isovalent.com>
CC:     <andrii@kernel.org>, <ast@kernel.org>, <bpf@vger.kernel.org>,
        <daniel@iogearbox.net>, <davem@davemloft.net>,
        <dsahern@kernel.org>, <edumazet@google.com>, <haoluo@google.com>,
        <hemanthmalla@gmail.com>, <joe@cilium.io>, <joe@wand.net.nz>,
        <john.fastabend@gmail.com>, <jolsa@kernel.org>,
        <kpsingh@kernel.org>, <kuba@kernel.org>, <kuniyu@amazon.com>,
        <linux-kernel@vger.kernel.org>, <linux-kselftest@vger.kernel.org>,
        <martin.lau@linux.dev>, <mykolal@fb.com>, <netdev@vger.kernel.org>,
        <pabeni@redhat.com>, <sdf@google.com>, <shuah@kernel.org>,
        <song@kernel.org>, <willemdebruijn.kernel@gmail.com>, <yhs@fb.com>
Subject: Re: [PATCH bpf-next v4 6/7] bpf, net: Support SO_REUSEPORT sockets with bpf_sk_assign
Date:   Thu, 6 Jul 2023 08:33:27 -0700
Message-ID: <20230706153327.99298-1-kuniyu@amazon.com>
In-Reply-To: <CAN+4W8iRH6kpDmmY8i5r1nKbckaYghmOCqRXe+4bDHE7vzVMMA@mail.gmail.com>
References: <CAN+4W8iRH6kpDmmY8i5r1nKbckaYghmOCqRXe+4bDHE7vzVMMA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
Precedence: bulk

From: Lorenz Bauer <lmb@isovalent.com>
Date: Thu, 6 Jul 2023 09:11:15 +0100
> On Thu, Jul 6, 2023 at 1:41 AM Kuniyuki Iwashima <kuniyu@amazon.com> wrote:
> >
> > Sorry for late reply.
> >
> > What we know about sk before inet6?_lookup_reuseport() are
> >
> >   (1) sk was full socket in bpf_sk_assign()
> >   (2) sk had SOCK_RCU_FREE in bpf_sk_assign()
> >   (3) sk was TCP_LISTEN here if TCP
> 
> Are we looking at the same bpf_sk_assign? Confusingly there are two
> very similarly named functions. The one we care about is:
> 
> BPF_CALL_3(bpf_sk_assign, struct sk_buff *, skb, struct sock *, sk, u64, flags)
> {
>     if (!sk || flags != 0)
>         return -EINVAL;
>     if (!skb_at_tc_ingress(skb))
>         return -EOPNOTSUPP;
>     if (unlikely(dev_net(skb->dev) != sock_net(sk)))
>         return -ENETUNREACH;
>     if (sk_is_refcounted(sk) &&
>         unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
>         return -ENOENT;
> 
>     skb_orphan(skb);
>     skb->sk = sk;
>     skb->destructor = sock_pfree;
> 
>     return 0;
> }
> 
> From this we can't tell what state the socket is in or whether it is
> RCU freed or not.

But we can in inet6?_steal_sock() by calling sk_is_refcounted() again
via skb_steal_sock().

In inet6?_steal_sock(), we call inet6?_lookup_reuseport() only for
sk that was a TCP listener or UDP non-connected socket until just before
the sk_state checks.  Then, we know *refcounted should be false for such
sockets even before inet6?_lookup_reuseport().

After the checks, sk might be poped out of the reuseport group before
inet6?_lookup_reuseport() and reuse_sk might be NULL, but it's not
related because *refcounted is a value for sk, not for reuse_sk.