Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Sat, 25 Apr 2020 14:17:26 -0700
From:   John Fastabend <john.fastabend@gmail.com>
To:     Jakub Sitnicki <jakub@cloudflare.com>,
        Xiyu Yang <xiyuyang19@fudan.edu.cn>
Cc:     John Fastabend <john.fastabend@gmail.com>,
        Daniel Borkmann <daniel@iogearbox.net>,
        Lorenz Bauer <lmb@cloudflare.com>,
        Eric Dumazet <edumazet@google.com>,
        "David S. Miller" <davem@davemloft.net>,
        Alexey Kuznetsov <kuznet@ms2.inr.ac.ru>,
        Hideaki YOSHIFUJI <yoshfuji@linux-ipv6.org>,
        Jakub Kicinski <kuba@kernel.org>,
        Alexei Starovoitov <ast@kernel.org>,
        Martin KaFai Lau <kafai@fb.com>,
        Song Liu <songliubraving@fb.com>, Yonghong Song <yhs@fb.com>,
        Andrii Nakryiko <andriin@fb.com>,
        KP Singh <kpsingh@chromium.org>, netdev@vger.kernel.org,
        bpf@vger.kernel.org, linux-kernel@vger.kernel.org,
        yuanxzhang@fudan.edu.cn, kjlu@umn.edu,
        Xin Tan <tanxin.ctf@gmail.com>
Message-ID: <5ea4a8e6c0482_144b2b0bbf3245c4b@john-XPS-13-9370.notmuch>
In-Reply-To: <87lfmjve1f.fsf@cloudflare.com>
References: <1587819040-38793-1-git-send-email-xiyuyang19@fudan.edu.cn>
 <87lfmjve1f.fsf@cloudflare.com>
Subject: Re: [PATCH] bpf: Fix sk_psock refcnt leak when receiving message
Mime-Version: 1.0
Content-Type: text/plain;
 charset=utf-8
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Jakub Sitnicki wrote:
> On Sat, Apr 25, 2020 at 02:50 PM CEST, Xiyu Yang wrote:
> > tcp_bpf_recvmsg() invokes sk_psock_get(), which returns a reference of
> > the specified sk_psock object to "psock" with increased refcnt.
> >
> > When tcp_bpf_recvmsg() returns, local variable "psock" becomes invalid,
> > so the refcount should be decreased to keep refcount balanced.
> >
> > The reference counting issue happens in several exception handling paths
> > of tcp_bpf_recvmsg(). When those error scenarios occur such as "flags"
> > includes MSG_ERRQUEUE, the function forgets to decrease the refcnt
> > increased by sk_psock_get(), causing a refcnt leak.
> >
> > Fix this issue by calling sk_psock_put() when those error scenarios
> > occur.
> >
> > Signed-off-by: Xiyu Yang <xiyuyang19@fudan.edu.cn>
> > Signed-off-by: Xin Tan <tanxin.ctf@gmail.com>
> > ---

Thanks Xiyu and Xin. Please add a Fixes tag,

Fixes: e7a5f1f1cd000 ("bpf/sockmap: Read psock ingress_msg before sk_receive_queue")

> >  net/ipv4/tcp_bpf.c | 8 ++++++--
> >  1 file changed, 6 insertions(+), 2 deletions(-)
> >
> > diff --git a/net/ipv4/tcp_bpf.c b/net/ipv4/tcp_bpf.c
> > index 5a05327f97c1..feb6b90672c1 100644
> > --- a/net/ipv4/tcp_bpf.c
> > +++ b/net/ipv4/tcp_bpf.c
> > @@ -265,11 +265,15 @@ static int tcp_bpf_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
> >  	psock = sk_psock_get(sk);
> >  	if (unlikely(!psock))
> >  		return tcp_recvmsg(sk, msg, len, nonblock, flags, addr_len);
> > -	if (unlikely(flags & MSG_ERRQUEUE))
> > +	if (unlikely(flags & MSG_ERRQUEUE)) {
> > +		sk_psock_put(sk, psock);
> >  		return inet_recv_error(sk, msg, len, addr_len);
> > +	}
> >  	if (!skb_queue_empty(&sk->sk_receive_queue) &&
> > -	    sk_psock_queue_empty(psock))
> > +	    sk_psock_queue_empty(psock)) {
> > +		sk_psock_put(sk, psock);
> >  		return tcp_recvmsg(sk, msg, len, nonblock, flags, addr_len);
> > +	}
> >  	lock_sock(sk);
> >  msg_bytes_ready:
> >  	copied = __tcp_bpf_recvmsg(sk, psock, msg, len, flags);
> 
> Thanks for the fix.
> 
> We can pull up the error queue read handling, that is the `flags &
> MSG_ERRQUEUE` branch, so that it happens before we grab a psock ref.
> 
> The effect is the same because now, if we hit the !psock branch,
> tcp_recvmsg will first check if user wants to read the error queue
> anyway.
> 
> That would translate to something like below, in addition to your
> changes.
> 
> WDYT?

Sure that seems slightly nicer to me. Xiyu do you mind sending a
v2 with that change.

Thanks again,
John

> 
> ---8<---
> 
> diff --git a/net/ipv4/tcp_bpf.c b/net/ipv4/tcp_bpf.c
> index 5a05327f97c1..99aa57bd1901 100644
> --- a/net/ipv4/tcp_bpf.c
> +++ b/net/ipv4/tcp_bpf.c
> @@ -262,14 +262,17 @@ static int tcp_bpf_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
>  	struct sk_psock *psock;
>  	int copied, ret;
> 
> +	if (unlikely(flags & MSG_ERRQUEUE))
> +		return inet_recv_error(sk, msg, len, addr_len);
> +
>  	psock = sk_psock_get(sk);
>  	if (unlikely(!psock))
>  		return tcp_recvmsg(sk, msg, len, nonblock, flags, addr_len);
> -	if (unlikely(flags & MSG_ERRQUEUE))
> -		return inet_recv_error(sk, msg, len, addr_len);
>  	if (!skb_queue_empty(&sk->sk_receive_queue) &&
>  	    sk_psock_queue_empty(psock))
> 		return tcp_recvmsg(sk, msg, len, nonblock, flags, addr_len);