Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35;
MIME-Version: 1.0
References: <20230918014752.1791518-1-guodongtai@kylinos.cn>
In-Reply-To: <20230918014752.1791518-1-guodongtai@kylinos.cn>
From:   Eric Dumazet <edumazet@google.com>
Date:   Mon, 18 Sep 2023 10:05:09 +0200
Message-ID: <CANn89i+WshtNwNSALCpbQbZFWN41xP85+c8GdHX2DabzQzx+6A@mail.gmail.com>
Subject: Re: [PATCH v1] tcp: enhancing timestamps random algo to address
 issues arising from NAT mapping
To:     George Guo <guodongtai@kylinos.cn>, Florian Westphal <fw@strlen.de>
Cc:     davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com,
        dsahern@kernel.org, netdev@vger.kernel.org,
        linux-kernel@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Mon, Sep 18, 2023 at 3:46=E2=80=AFAM George Guo <guodongtai@kylinos.cn> =
wrote:
>
> Tsval=3Dtsoffset+local_clock, here tsoffset is randomized with saddr and =
daddr parameters in func
> secure_tcp_ts_off. Most of time it is OK except for NAT mapping to the sa=
me port and daddr.
> Consider the following scenario:
>         ns1:                ns2:
>         +-----------+        +-----------+
>         |           |        |           |
>         |           |        |           |
>         |           |        |           |
>         | veth1     |        | vethb     |
>         |192.168.1.1|        |192.168.1.2|
>         +----+------+        +-----+-----+
>              |                     |
>              |                     |
>              | br0:192.168.1.254   |
>              +----------+----------+
>          veth0          |     vetha
>          192.168.1.3    |    192.168.1.4
>                         |
>                        nat(192.168.1.x -->172.30.60.199)
>                         |
>                         V
>                        eth0
>                  172.30.60.199
>                        |
>                        |
>                        +----> ... ...    ---->server: 172.30.60.191
>
> Let's say ns1 (192.168.1.1) generates a timestamp ts1, and ns2 (192.168.1=
.2) generates a timestamp
> ts2, with ts1 > ts2.
>
> If ns1 initiates a connection to a server, and then the server actively c=
loses the connection,
> entering the TIME_WAIT state, and ns2 attempts to connect to the server w=
hile port reuse is in
> progress, due to the presence of NAT, the server sees both connections as=
 originating from the
> same IP address (e.g., 172.30.60.199) and port. However, since ts2 is sma=
ller than ts1, the server
> will respond with the acknowledgment (ACK) for the fourth handshake.
>
>        SERVER                                                   CLIENT
>
>    1.  ESTABLISHED                                              ESTABLISH=
ED
>
>        (Close)
>    2.  FIN-WAIT-1  --> <SEQ=3D100><ACK=3D300><TSval=3D20><CTL=3DFIN,ACK> =
 --> CLOSE-WAIT
>
>    3.  FIN-WAIT-2  <-- <SEQ=3D300><ACK=3D101><TSval=3D40><CTL=3DACK>     =
 <-- CLOSE-WAIT
>
>                                                             (Close)
>    4.  TIME-WAIT   <-- <SEQ=3D300><ACK=3D101><TSval=3D41><CTL=3DFIN,ACK> =
 <-- LAST-ACK
>
>    5.  TIME-WAIT   --> <SEQ=3D101><ACK=3D301><TSval=3D25><CTL=3DACK>     =
 --> CLOSED
>
>   - - - - - - - - - - - - - port reused - - - - - - - - - - - - - - -
>
>    5.1. TIME-WAIT   <-- <SEQ=3D255><TSval=3D30><CTL=3DSYN>             <-=
- SYN-SENT
>
>    5.2. TIME-WAIT   --> <SEQ=3D101><ACK=3D301><TSval=3D35><CTL=3DACK>    =
--> SYN-SENT
>
>    5.3. CLOSED      <-- <SEQ=3D301><CTL=3DRST>                       <-- =
SYN-SENT
>
>    6.  SYN-RECV    <-- <SEQ=3D255><TSval=3D34><CTL=3DSYN>              <-=
- SYN-SENT
>
>    7.  SYN-RECV    --> <SEQ=3D400><ACK=3D301><TSval=3D40><CTL=3DSYN,ACK> =
--> ESTABLISHED
>
>    1.  ESTABLISH   <-- <SEQ=3D301><ACK=3D401><TSval=3D55><CTL=3DACK>     =
<-- ESTABLISHED
>
> This enhancement uses sport and daddr rather than saddr and daddr, which =
keep the timestamp
> monotonically increasing in the situation described above. Then the port =
reuse is like this:
>
>        SERVER                                                   CLIENT
>
>    1.  ESTABLISHED                                              ESTABLISH=
ED
>
>        (Close)
>    2.  FIN-WAIT-1  --> <SEQ=3D100><ACK=3D300><TSval=3D20><CTL=3DFIN,ACK> =
 --> CLOSE-WAIT
>
>    3.  FIN-WAIT-2  <-- <SEQ=3D300><ACK=3D101><TSval=3D40><CTL=3DACK>     =
 <-- CLOSE-WAIT
>
>                                                             (Close)
>    4.  TIME-WAIT   <-- <SEQ=3D300><ACK=3D101><TSval=3D41><CTL=3DFIN,ACK> =
 <-- LAST-ACK
>
>    5.  TIME-WAIT   --> <SEQ=3D101><ACK=3D301><TSval=3D25><CTL=3DACK>     =
 --> CLOSED
>
>   - - - - - - - - - - - - - port reused - - - - - - - - - - - - - - -
>
>    5.1. TIME-WAIT  <-- <SEQ=3D300><TSval=3D50><CTL=3DSYN>               <=
-- SYN-SENT
>
>    6.  SYN-RECV    --> <SEQ=3D400><ACK=3D301><TSval=3D40><CTL=3DSYN,ACK> =
 --> ESTABLISHED
>
>    1.  ESTABLISH   <-- <SEQ=3D301><ACK=3D401><TSval=3D55><CTL=3DACK>     =
 <-- ESTABLISHED
>
> The enhancement lets port reused more efficiently.
>
> Signed-off-by: George Guo <guodongtai@kylinos.cn>
>

CC Florian

I do not think we can 'fix' tcp timestamp vs NAT.
Unless the NAT device makes sure a port is dedicated for a peer,
and/or the NAT rewrites TS values
(which would be bad).

I personally prefer seeing the same timestamps from A to B regardless
of ports, it helps detect various issues.

Also, you seem to forget IPv6.