Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 1 Apr 2022 14:09:58 +0200
From:   Florian Westphal <fw@strlen.de>
To:     Jaco Kroon <jaco@uls.co.za>
Cc:     Florian Westphal <fw@strlen.de>,
        Eric Dumazet <edumazet@google.com>,
        Neal Cardwell <ncardwell@google.com>,
        LKML <linux-kernel@vger.kernel.org>,
        Netdev <netdev@vger.kernel.org>,
        Yuchung Cheng <ycheng@google.com>
Subject: Re: linux 5.17.1 disregarding ACK values resulting in stalled TCP
 connections
Message-ID: <20220401120958.GA28321@breakpoint.cc>
References: <CANn89iKHbmVYoBdo2pCQWTzB4eFBjqAMdFbqL5EKSFqgg3uAJQ@mail.gmail.com>
 <10c1e561-8f01-784f-c4f4-a7c551de0644@uls.co.za>
 <CADVnQynf8f7SUtZ8iQi-fACYLpAyLqDKQVYKN-mkEgVtFUTVXQ@mail.gmail.com>
 <e0bc0c7f-5e47-ddb7-8e24-ad5fb750e876@uls.co.za>
 <CANn89i+Dqtrm-7oW+D6EY+nVPhRH07GXzDXt93WgzxZ1y9_tJA@mail.gmail.com>
 <CADVnQyn=VfcqGgWXO_9h6QTkMn5ZxPbNRTnMFAxwQzKpMRvH3A@mail.gmail.com>
 <5f1bbeb2-efe4-0b10-bc76-37eff30ea905@uls.co.za>
 <CANn89i+KsjGUppc3D8KLa4XUd-dzS3A+yDxbv2bRkDEkziS1qw@mail.gmail.com>
 <20220401001531.GB9545@breakpoint.cc>
 <7d08dcfd-6ba0-f972-cee3-4fa0eff8c855@uls.co.za>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <7d08dcfd-6ba0-f972-cee3-4fa0eff8c855@uls.co.za>
User-Agent: Mutt/1.10.1 (2018-07-13)
Precedence: bulk

Jaco Kroon <jaco@uls.co.za> wrote:
> > Eric Dumazet <edumazet@google.com> wrote:
> >> Next step would be to attempt removing _all_ firewalls, especially not
> >> common setups like yours.
> >>
> >> conntrack had a bug preventing TFO deployment for a while, because
> >> many boxes kept buggy kernel versions for years.
> >>
> >> 356d7d88e088687b6578ca64601b0a2c9d145296 netfilter: nf_conntrack: fix
> >> tcp_in_window for Fast Open
> > Jaco could also try with
> > net.netfilter.nf_conntrack_tcp_be_liberal=1
> >
> > and, if that helps, with liberal=0 and
> > sysctl net.netfilter.nf_conntrack_log_invalid=6
> >
> > (check dmesg/syslog/nflog).
> 
> Our core firewalls already had nf_conntrack_tcp_be_liberal for other
> reasons (asymmetric routing combined with conntrackd left-over if I
> recall), so maybe that's why it got through there ... don't exactly want
> to just flip that setting though, is there a way to log if it would have
> dropped anything, without actually dropping it (yet)?

This means conntrack doesn't tag packets as invalid EVEN if it would
consider sequence/ack out-of-window (e.g. due to a bug).

I have a hard time seeing how tcp liberal-mode conntrack would be to
blame here.

Only thing you could also check is if
net.netfilter.nf_conntrack_checksum=0 helps (but i doubt it).