Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20230529113804.GA20300@didi-ThinkCentre-M920t-N000>
 <CANn89iJw3Ehoj7GfYnc6Xv5N2wULqNuP3zNBwQx97i-YJD5avg@mail.gmail.com>
 <CAL+tcoCRgpoYgQXdM2Es0gjRtRZj1hCK-5M04WZ1sy3vt_5JjA@mail.gmail.com>
 <CANn89iKCKTg0ExMRAxXWK200Q6UX2DeuUQzZzbZevt5kST7qQA@mail.gmail.com> <CAL+tcoDd5M0=SzDVcuFnu_ntC7HASq9rrS_pZ5FXNyMa2aQrHQ@mail.gmail.com>
In-Reply-To: <CAL+tcoDd5M0=SzDVcuFnu_ntC7HASq9rrS_pZ5FXNyMa2aQrHQ@mail.gmail.com>
From:   Jason Xing <kerneljasonxing@gmail.com>
Date:   Tue, 30 May 2023 20:52:30 +0800
Message-ID: <CAL+tcoBYvV58rz9KcrH5QTdc1jPi=H1VXt1yUj-pYwA1FE9CEg@mail.gmail.com>
Subject: Re: [PATCH net] tcp: introduce a compack timer handler in sack compression
To:     Eric Dumazet <edumazet@google.com>
Cc:     toke@toke.dk, Yuchung Cheng <ycheng@google.com>,
        "David S. Miller" <davem@davemloft.net>,
        David Ahern <dsahern@kernel.org>,
        Jakub Kicinski <kuba@kernel.org>,
        Paolo Abeni <pabeni@redhat.com>,
        Neal Cardwell <ncardwell@google.com>, netdev@vger.kernel.org,
        zhangweiping <zhangweiping@didiglobal.com>,
        tiozhang <tiozhang@didiglobal.com>, linux-kernel@vger.kernel.org,
        bpf@vger.kernel.org, fuyuanli@didiglobal.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Tue, May 30, 2023 at 7:10=E2=80=AFPM Jason Xing <kerneljasonxing@gmail.c=
om> wrote:
>
> On Tue, May 30, 2023 at 6:01=E2=80=AFPM Eric Dumazet <edumazet@google.com=
> wrote:
> >
> > On Tue, May 30, 2023 at 9:45=E2=80=AFAM Jason Xing <kerneljasonxing@gma=
il.com> wrote:
> > >
> > > On Tue, May 30, 2023 at 3:12=E2=80=AFPM Eric Dumazet <edumazet@google=
.com> wrote:
> > > >
> > > > On Mon, May 29, 2023 at 1:38=E2=80=AFPM fuyuanli <fuyuanli@didiglob=
al.com> wrote:
> > > > >
> > > > > We've got some issues when sending a compressed ack is deferred t=
o
> > > > > release phrase due to the socket owned by another user:
> > > > > 1. a compressed ack would not be sent because of lack of ICSK_ACK=
_TIMER
> > > > > flag.
> > > >
> > > > Are you sure ? Just add it then, your patch will be a one-liner
> > > > instead of a complex one adding
> > > > more code in a fast path.
> > >
> > > Honestly, at the very beginning, we just added one line[1] to fix
> > > this. After I digged more into this part, I started to doubt if we
> > > should reuse the delayed ack logic.
> > >
> > > Because in the sack compression logic there is no need to do more
> > > things as delayed ack does in the tcp_delack_timer_handler() function=
.
> > >
> > > Besides, here are some things extra to be done if we defer to send an
> > > ack in sack compression:
> > > 1) decrease tp->compressed_ack. The same as "tp->compressed_ack--;" i=
n
> > > tcp_compressed_ack_kick().
> > > 2) initialize icsk->icsk_ack.timeout. Actually we don't need to do
> > > this because we don't modify the expiration time in the sack
> > > compression hrtimer.
> >
> > Yes, we do not need this, see my following comment.
> >
> > > 3) don't need to count the LINUX_MIB_DELAYEDACKS counter.
> > > 4) I wonder even if those checks about the ack schedule or ping pong
> > > mode in tcp_delack_timer_handler() for sack compression? I'm not sure
> > > about it.
> > >
> > > So one line cannot solve it perfectly. That's the reason why we
> > > introduce a new logic which can be clearer.
> > >
> > > I'm wondering if adding one check in the fast path is really that
> > > unacceptable (it may hurt performance?) because a new logic would be
> > > clearer for the whole sack compression.
> >
> > We definitely can solve the minor issue by not polluting the fast path.
> >
> > We also want simple/localised fixes, not something invasive (and risky)=
.
>
> Now I got it :)
>
> >
> > >
> > > [1]
> > > diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
> > > index cc072d2cfcd8..d9e76d761cc6 100644
> > > --- a/net/ipv4/tcp_input.c
> > > +++ b/net/ipv4/tcp_input.c
> > > @@ -5568,6 +5568,7 @@ static void __tcp_ack_snd_check(struct sock *sk=
,
> > > int ofo_possible)
> > >
> > > READ_ONCE(sock_net(sk)->ipv4.sysctl_tcp_comp_sack_delay_ns),
> > >                       rtt * (NSEC_PER_USEC >> 3)/20);
> > >         sock_hold(sk);
> > > +       inet_csk(sk)->icsk_ack.pending |=3D ICSK_ACK_TIMER;
> >
> > Why not simply use existing storage/variables (tp->compressed_ack),
> > instead of trying
> > to reuse something else or add another bit, then complain that this
> > does not work well ?
> >
> > Again, just fix tcp_delack_timer_handler(), it already can fetch existi=
ng state.
> >
> > As a bonus, no need to send one patch for net, and another in net-next,
> > trying to 'fix' issues that should have been fixed cleanly in a single =
patch.
> >
> > Something like:
> >
> > diff --git a/net/ipv4/tcp_timer.c b/net/ipv4/tcp_timer.c
> > index b839c2f91292f7346f33d6dcbf597594473a5aca..16bc4cedceb8a5e88f61f9a=
bc2c0a8cc9322676a
> > 100644
> > --- a/net/ipv4/tcp_timer.c
> > +++ b/net/ipv4/tcp_timer.c
> > @@ -290,10 +290,20 @@ static int tcp_write_timeout(struct sock *sk)
> >  void tcp_delack_timer_handler(struct sock *sk)
> >  {
> >         struct inet_connection_sock *icsk =3D inet_csk(sk);
> > +       struct tcp_sock *tp =3D tcp_sk(sk);
> >
> > -       if (((1 << sk->sk_state) & (TCPF_CLOSE | TCPF_LISTEN)) ||
> > -           !(icsk->icsk_ack.pending & ICSK_ACK_TIMER))
> > +       if ((1 << sk->sk_state) & (TCPF_CLOSE | TCPF_LISTEN))
> > +               return;
> > +
> > +       if (!(icsk->icsk_ack.pending & ICSK_ACK_TIMER) &&
> > +           !tp->compressed_ack)
> > +               return;
> > +
> > +       if (tp->compressed_ack) {
> > +               tcp_mstamp_refresh(tp);
> > +               tcp_sack_compress_send_ack(sk);
>
> I wonder if we could use this combination as below instead since the
> above function counts the snmp counter and clears the @compressed_ack,
> which is against what we normally do in tcp_compressed_ack_kick() if
> the socket is not owned?

I take it back. After I considered it for a while, I think it's
working because @compressed_ack and snmp counter are a pair which
means we can do both of them together.
It doesn't have any impact though it looks a bit different from before.

Thanks,
Jason

>
> "
> if (hrtimer_try_to_cancel(&tp->compressed_ack_timer) =3D=3D 1)
>     __sock_put(sk);
> tcp_send_ack(sk);
> "
>
> Above is extracted from tcp_sack_compress_send_ack()
>
> >                 return;
> > +       }
> >
> >         if (time_after(icsk->icsk_ack.timeout, jiffies)) {
> >                 sk_reset_timer(sk, &icsk->icsk_delack_timer,
> > icsk->icsk_ack.timeout);
> > @@ -312,7 +322,7 @@ void tcp_delack_timer_handler(struct sock *sk)
> >                         inet_csk_exit_pingpong_mode(sk);
> >                         icsk->icsk_ack.ato      =3D TCP_ATO_MIN;
> >                 }
> > -               tcp_mstamp_refresh(tcp_sk(sk));
> > +               tcp_mstamp_refresh(tp);
> >                 tcp_send_ack(sk);
> >                 __NET_INC_STATS(sock_net(sk), LINUX_MIB_DELAYEDACKS);
> >         }
>
> Thank you so much, Eric.
>
> I will let fuyuanli submit a v2 patch including that one without
> introducing a new logic/flag.
>
> Thanks again,
> Jason