On Tue, Aug 8, 2023 at 1:59 PM <[email protected]> wrote:
>
> From: Menglong Dong <[email protected]>
>
> In tcp_retransmit_timer(), a window shrunk connection will be regarded
> as timeout if 'tcp_jiffies32 - tp->rcv_tstamp > TCP_RTO_MAX'. This is not
> right all the time.
>
> The retransmits will become zero-window probes in tcp_retransmit_timer()
> if the 'snd_wnd==0'. Therefore, the icsk->icsk_rto will come up to
> TCP_RTO_MAX sooner or later.
>
> However, the timer can be delayed and be triggered after 122877ms, not
> TCP_RTO_MAX, as I tested.
>
> Therefore, 'tcp_jiffies32 - tp->rcv_tstamp > TCP_RTO_MAX' is always true
> once the RTO come up to TCP_RTO_MAX, and the socket will die.
>
> Fix this by replacing the 'tcp_jiffies32' with '(u32)icsk->icsk_timeout',
> which is exact the timestamp of the timeout. Meanwhile, using the later
> one of tp->retrans_stamp and tp->rcv_tstamp as the last updated timestamp
> in the receiving path, as "tp->rcv_tstamp" can restart from idle, then
> tp->rcv_tstamp could already be a long time (minutes or hours) in the
> past even on the first RTO.
>
> Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2")
> Link: https://lore.kernel.org/netdev/CADxym3YyMiO+zMD4zj03YPM3FBi-1LHi6gSD2XT8pyAMM096pg@mail.gmail.com/
> Signed-off-by: Menglong Dong <[email protected]>
> ---
> v3:
> - use after() instead of max() in tcp_rtx_probe0_timed_out()
> v2:
> - consider the case of the connection restart from idle, as Neal comment
> ---
> net/ipv4/tcp_timer.c | 14 +++++++++++++-
> 1 file changed, 13 insertions(+), 1 deletion(-)
>
> diff --git a/net/ipv4/tcp_timer.c b/net/ipv4/tcp_timer.c
> index d45c96c7f5a4..f30d1467771c 100644
> --- a/net/ipv4/tcp_timer.c
> +++ b/net/ipv4/tcp_timer.c
> @@ -454,6 +454,18 @@ static void tcp_fastopen_synack_timer(struct sock *sk, struct request_sock *req)
> req->timeout << req->num_timeout, TCP_RTO_MAX);
> }
>
> +static bool tcp_rtx_probe0_timed_out(struct sock *sk)
const struct sock *sk
> +{
> + struct tcp_sock *tp = tcp_sk(sk);
const struct tcp_sock *tp = tcp_sk(sk);
> + u32 timeout_ts, rtx_ts, rcv_ts;
> +
> + rtx_ts = tp->retrans_stamp;
> + rcv_ts = tp->rcv_tstamp;
> + timeout_ts = after(rtx_ts, rcv_ts) ? rtx_ts : rcv_ts;
> + timeout_ts += TCP_RTO_MAX;
If we are concerned with a socket dying too soon, I would suggest
adding 2*TCP_RTO_MAX instead of TCP_RTO_MAX
When a receiver is OOMing, it is possible the ACK RWIN 0 can not be sent all,
so tp->rcv_tstamp will not be refreshed. Or ACK could be lost in the
network path.
This also suggests the net_dbg_ratelimited("Peer %pI4:%u/%u
unexpectedly shrunk window %u:%u (repaired)\n"...) messages
are slightly wrong, because they could be printed even if we did not
receive a new ACK packet from the remote peer.
Perhaps we should change them to include delays (how long @skb stayed
in rtx queue, how old is the last ACK we received)
> +
> + return after(inet_csk(sk)->icsk_timeout, timeout_ts);
> +}
>
> /**
> * tcp_retransmit_timer() - The TCP retransmit timeout handler
> @@ -519,7 +531,7 @@ void tcp_retransmit_timer(struct sock *sk)
> tp->snd_una, tp->snd_nxt);
> }
> #endif
> - if (tcp_jiffies32 - tp->rcv_tstamp > TCP_RTO_MAX) {
> + if (tcp_rtx_probe0_timed_out(sk)) {
> tcp_write_err(sk);
> goto out;
> }
> --
> 2.40.1
>
> On Aug 8, 2023, at 20:49, Eric Dumazet <[email protected]> wrote:
>
> On Tue, Aug 8, 2023 at 1:59 PM <[email protected]> wrote:
>>
[……]
>>
>> diff --git a/net/ipv4/tcp_timer.c b/net/ipv4/tcp_timer.c
>> index d45c96c7f5a4..f30d1467771c 100644
>> --- a/net/ipv4/tcp_timer.c
>> +++ b/net/ipv4/tcp_timer.c
>> @@ -454,6 +454,18 @@ static void tcp_fastopen_synack_timer(struct sock *sk, struct request_sock *req)
>> req->timeout << req->num_timeout, TCP_RTO_MAX);
>> }
>>
>> +static bool tcp_rtx_probe0_timed_out(struct sock *sk)
>
> const struct sock *sk
>
>> +{
>> + struct tcp_sock *tp = tcp_sk(sk);
>
> const struct tcp_sock *tp = tcp_sk(sk);
>
>> + u32 timeout_ts, rtx_ts, rcv_ts;
>> +
>> + rtx_ts = tp->retrans_stamp;
>> + rcv_ts = tp->rcv_tstamp;
>> + timeout_ts = after(rtx_ts, rcv_ts) ? rtx_ts : rcv_ts;
>> + timeout_ts += TCP_RTO_MAX;
>
> If we are concerned with a socket dying too soon, I would suggest
> adding 2*TCP_RTO_MAX instead of TCP_RTO_MAX
>
> When a receiver is OOMing, it is possible the ACK RWIN 0 can not be sent all,
> so tp->rcv_tstamp will not be refreshed. Or ACK could be lost in the
> network path.
Yeah, I concern abort this too. I introduce the funtion “tcp_rtx_probe0_timed_out()”
here is to offer a more reliable way to check the timeout in the feature.
And for this time, we can fix the problem first, as you advised, adding
2*TCP_RTO_MAX instead of TCP_RTO_MAX.
> This also suggests the net_dbg_ratelimited("Peer %pI4:%u/%u
> unexpectedly shrunk window %u:%u (repaired)\n"...) messages
> are slightly wrong, because they could be printed even if we did not
> receive a new ACK packet from the remote peer.
>
> Perhaps we should change them to include delays (how long @skb stayed
> in rtx queue, how old is the last ACK we received)
Sounds great, which is more valuable. I’ll change them
in the next version.
Thanks!
Menglong Dong
>> +
>> + return after(inet_csk(sk)->icsk_timeout, timeout_ts);
>> +}
>>
>> /**
>> * tcp_retransmit_timer() - The TCP retransmit timeout handler
>> @@ -519,7 +531,7 @@ void tcp_retransmit_timer(struct sock *sk)
>> tp->snd_una, tp->snd_nxt);
>> }
>> #endif
>> - if (tcp_jiffies32 - tp->rcv_tstamp > TCP_RTO_MAX) {
>> + if (tcp_rtx_probe0_timed_out(sk)) {
>> tcp_write_err(sk);
>> goto out;
>> }
>> --
>> 2.40.1