Received: by 2002:a05:6a10:6744:0:0:0:0 with SMTP id w4csp4341405pxu; Mon, 12 Oct 2020 16:51:39 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz65loXjA6UEUxAVn1lKQp0ngOCa+vu0RjMROTsWrC9H3jf7LylzXrZxXqwhctV8UkZ51hn X-Received: by 2002:aa7:da12:: with SMTP id r18mr16540017eds.169.1602546699272; Mon, 12 Oct 2020 16:51:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1602546699; cv=none; d=google.com; s=arc-20160816; b=fFwXr/8vaiakQwX1eQ4KYNeAlAGnWWIBKay9xjnPwad+wvKmU9W1ntEgaYkNya2vfw 8Fp/YFulG6dY4c06ZdamkYKl8QoRW8b04krxCdPcUJfS6X7fKW7Cl+7Fr6B7E3Yij7Ro cpDqwz6kzYTLV3w3NLB5DMnjAgCwrc4F6NKoWp9AbBUTWuFX87WhxwhZ/FE6p1VT0w+I Y8OyuMseBIeHB5tC3aKHAABmKzfOLdnyrdBXGtJu1bVecIuEw+v6wPBXMCr/d/6RRlNL y3vfJtFDA0k/sDiKBdsvYCUxZioLl8h9W9cXuY4wk9HWhXCqmMwf9t4C5/ZJ1yewNsSv qb6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=gtD/xy61o/78XYjsXKHQwqUn4yafEd6pqVQdHakkzxs=; b=yOjR0jSYX3dbNaV85BIx+1GPYl2O4KOMb8iIZPG4Bt6oyY/VLW20cDZqz9B76qo0pQ K7/Qmv0i7XkGDWWAPakTNWdRBPFc/AvwY5NsYEuHs84vk+yN/4OigTIUKrqrf4YOVP5q nP5y9imfccFVVQUlVnp7AgplPB92WaxHVAcXVhJcSdroqPdUk1ziaMN8Haz0FF/ChBjE 3psvTWwrB/bl+fg05ScWUZeMyqrpzQM+uM2hpomwYo1cZ/YZinZFwxjvczPq1hxvbU2f TrqN939GNOjUi+fT5pZDGWKhc+UOeT4W1T2KHUIyo59d8RSk2cOyeP3fYlCptQ0s90Fz fpuQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=yLjCnRUx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id p15si13236694edy.474.2020.10.12.16.51.16; Mon, 12 Oct 2020 16:51:39 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=yLjCnRUx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2389297AbgJLNsp (ORCPT + 99 others); Mon, 12 Oct 2020 09:48:45 -0400 Received: from mail.kernel.org ([198.145.29.99]:53874 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731907AbgJLNsT (ORCPT ); Mon, 12 Oct 2020 09:48:19 -0400 Received: from localhost (83-86-74-64.cable.dynamic.v4.ziggo.nl [83.86.74.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id B655122272; Mon, 12 Oct 2020 13:47:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602510473; bh=0hs6sUWUUadl862mFBfbn80B9RAWItTP2g8QhSrKaWw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=yLjCnRUxkF78aqPeJcJc0tZnrpRA10nQelGagjxwNFUEv9Mcs9KHL3e+X5iIuqdvc /qJcHBHbkbyfg0Cm8tK/ypQ1lPYemhsJLjnNasVobZ136SNy/E6MMv3as3JrbJvzcS tMOHjw0EuqPqCVYJ/daJM1aNl1Wzt1EKZq5GXWeA= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Eric Dumazet , Alexandre Ferrieux , Soheil Hassas Yeganeh , Neal Cardwell , "David S. Miller" Subject: [PATCH 5.8 113/124] tcp: fix receive window update in tcp_add_backlog() Date: Mon, 12 Oct 2020 15:31:57 +0200 Message-Id: <20201012133152.323410412@linuxfoundation.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201012133146.834528783@linuxfoundation.org> References: <20201012133146.834528783@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Eric Dumazet commit 86bccd0367130f481ca99ba91de1c6a5aa1c78c1 upstream. We got reports from GKE customers flows being reset by netfilter conntrack unless nf_conntrack_tcp_be_liberal is set to 1. Traces seemed to suggest ACK packet being dropped by the packet capture, or more likely that ACK were received in the wrong order. wscale=7, SYN and SYNACK not shown here. This ACK allows the sender to send 1871*128 bytes from seq 51359321 : New right edge of the window -> 51359321+1871*128=51598809 09:17:23.389210 IP A > B: Flags [.], ack 51359321, win 1871, options [nop,nop,TS val 10 ecr 999], length 0 09:17:23.389212 IP B > A: Flags [.], seq 51422681:51424089, ack 1577, win 268, options [nop,nop,TS val 999 ecr 10], length 1408 09:17:23.389214 IP A > B: Flags [.], ack 51422681, win 1376, options [nop,nop,TS val 10 ecr 999], length 0 09:17:23.389253 IP B > A: Flags [.], seq 51424089:51488857, ack 1577, win 268, options [nop,nop,TS val 999 ecr 10], length 64768 09:17:23.389272 IP A > B: Flags [.], ack 51488857, win 859, options [nop,nop,TS val 10 ecr 999], length 0 09:17:23.389275 IP B > A: Flags [.], seq 51488857:51521241, ack 1577, win 268, options [nop,nop,TS val 999 ecr 10], length 32384 Receiver now allows to send 606*128=77568 from seq 51521241 : New right edge of the window -> 51521241+606*128=51598809 09:17:23.389296 IP A > B: Flags [.], ack 51521241, win 606, options [nop,nop,TS val 10 ecr 999], length 0 09:17:23.389308 IP B > A: Flags [.], seq 51521241:51553625, ack 1577, win 268, options [nop,nop,TS val 999 ecr 10], length 32384 It seems the sender exceeds RWIN allowance, since 51611353 > 51598809 09:17:23.389346 IP B > A: Flags [.], seq 51553625:51611353, ack 1577, win 268, options [nop,nop,TS val 999 ecr 10], length 57728 09:17:23.389356 IP B > A: Flags [.], seq 51611353:51618393, ack 1577, win 268, options [nop,nop,TS val 999 ecr 10], length 7040 09:17:23.389367 IP A > B: Flags [.], ack 51611353, win 0, options [nop,nop,TS val 10 ecr 999], length 0 netfilter conntrack is not happy and sends RST 09:17:23.389389 IP A > B: Flags [R], seq 92176528, win 0, length 0 09:17:23.389488 IP B > A: Flags [R], seq 174478967, win 0, length 0 Now imagine ACK were delivered out of order and tcp_add_backlog() sets window based on wrong packet. New right edge of the window -> 51521241+859*128=51631193 Normally TCP stack handles OOO packets just fine, but it turns out tcp_add_backlog() does not. It can update the window field of the aggregated packet even if the ACK sequence of the last received packet is too old. Many thanks to Alexandre Ferrieux for independently reporting the issue and suggesting a fix. Fixes: 4f693b55c3d2 ("tcp: implement coalescing on backlog queue") Signed-off-by: Eric Dumazet Reported-by: Alexandre Ferrieux Acked-by: Soheil Hassas Yeganeh Acked-by: Neal Cardwell Signed-off-by: David S. Miller Signed-off-by: Greg Kroah-Hartman --- net/ipv4/tcp_ipv4.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) --- a/net/ipv4/tcp_ipv4.c +++ b/net/ipv4/tcp_ipv4.c @@ -1787,12 +1787,12 @@ bool tcp_add_backlog(struct sock *sk, st __skb_pull(skb, hdrlen); if (skb_try_coalesce(tail, skb, &fragstolen, &delta)) { - thtail->window = th->window; - TCP_SKB_CB(tail)->end_seq = TCP_SKB_CB(skb)->end_seq; - if (after(TCP_SKB_CB(skb)->ack_seq, TCP_SKB_CB(tail)->ack_seq)) + if (likely(!before(TCP_SKB_CB(skb)->ack_seq, TCP_SKB_CB(tail)->ack_seq))) { TCP_SKB_CB(tail)->ack_seq = TCP_SKB_CB(skb)->ack_seq; + thtail->window = th->window; + } /* We have to update both TCP_SKB_CB(tail)->tcp_flags and * thtail->fin, so that the fast path in tcp_rcv_established()