MIME-Version: 1.0
References: <CABWYdi3bzd4P0nsyZe6P6coBCQ2jN=kVOJte62zKj=Q8iJCSOQ@mail.gmail.com>
 <CANn89i+mhqGaM2tuhgEmEPbbNu_59GGMhBMha4jnnzFE=UBNYg@mail.gmail.com>
In-Reply-To: <CANn89i+mhqGaM2tuhgEmEPbbNu_59GGMhBMha4jnnzFE=UBNYg@mail.gmail.com>
From:   Ivan Babrou <ivan@cloudflare.com>
Date:   Thu, 23 Dec 2021 14:52:55 -0800
Message-ID: <CABWYdi0qBQ57OHt4ZbRxMtdSzhubzkPaPKkYzdNfu4+cgPyXCA@mail.gmail.com>
Subject: Re: Initial TCP receive window is clamped to 64k by rcv_ssthresh
To:     Eric Dumazet <edumazet@google.com>
Cc:     Linux Kernel Network Developers <netdev@vger.kernel.org>,
        kernel-team <kernel-team@cloudflare.com>,
        linux-kernel <linux-kernel@vger.kernel.org>,
        "David S. Miller" <davem@davemloft.net>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Wed, Dec 22, 2021 at 10:10 AM Eric Dumazet <edumazet@google.com> wrote:
> Stack is conservative about RWIN increase, it wants to receive packets
> to have an idea
> of the skb->len/skb->truesize ratio to convert a memory budget to  RWIN.
>
> Some drivers have to allocate 16K buffers (or even 32K buffers) just
> to hold one segment
> (of less than 1500 bytes of payload), while others are able to pack
> memory more efficiently.
>
> I guess that you could use eBPF code to precisely tweak stack behavior
> to your needs.

Adding ebpf for this is certainly an option and it seems similar to
TCP_BPF_SNDCWND_CLAMP. I can certainly look into crafting a patch for
this.

Is it not possible to do anything automatically to pick a bigger
window without ebpf? When the scaled window is first advertised in the
very first ACK, the kernel already has the SYN ACK skb from the other
end of the connection. Could the skb->len / skb->truesize ratio be
looked up there?

Increasing tcp_rmem (the middle part specifically) is a lower entry
barrier than making ebpf involved, and it can really help with latency
even for human use cases like opening a website across the ocean.