2022-02-28 18:12:07

by Wong Vee Khee

[permalink] [raw]
Subject: [PATCH net 1/1] net: stmmac: Resolve poor line rate after switching from TSO off to TSO on

From: Ling Pei Lee <[email protected]>

Sequential execution of these steps:
i) TSO ON – iperf3 execution,
ii) TSO OFF – iperf3 execution,
iii) TSO ON – iperf3 execution, it leads to iperf3 0 bytes transfer.

Example of mentioned Issue happened:
root@TGLA:~# iperf3 -c 169.254.168.191
Connecting to host 169.254.168.191, port 5201
[ 5] local 169.254.50.108 port 45846 connected to 169.254.168.191
port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 113 MBytes 947 Mbits/sec 0 378 KBytes
[ 5] 1.00-2.00 sec 111 MBytes 933 Mbits/sec 0 378 KBytes
[ 5] 2.00-3.00 sec 111 MBytes 933 Mbits/sec 0 378 KBytes
[ 5] 3.00-4.00 sec 111 MBytes 929 Mbits/sec 0 378 KBytes
[ 5] 4.00-5.00 sec 111 MBytes 934 Mbits/sec 0 378 KBytes
[ 5] 5.00-6.00 sec 111 MBytes 932 Mbits/sec 0 378 KBytes
[ 5] 6.00-7.00 sec 111 MBytes 932 Mbits/sec 0 378 KBytes
[ 5] 7.00-8.00 sec 111 MBytes 932 Mbits/sec 0 378 KBytes
[ 5] 8.00-9.00 sec 111 MBytes 931 Mbits/sec 0 378 KBytes
[ 5] 9.00-10.00 sec 111 MBytes 932 Mbits/sec 0 378 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 1.09 GBytes 933 Mbits/sec 0 sender
[ 5] 0.00-10.00 sec 1.09 GBytes 932 Mbits/sec receiver

iperf Done.
root@TGLA:~# ethtool -K enp0s30f4 tso off
root@TGLA:~# iperf3 -c 169.254.168.191
Connecting to host 169.254.168.191, port 5201
[ 5] local 169.254.50.108 port 45854 connected to 169.254.168.191
port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 104 MBytes 870 Mbits/sec 0 352 KBytes
[ 5] 1.00-2.00 sec 101 MBytes 850 Mbits/sec 0 369 KBytes
[ 5] 2.00-3.00 sec 102 MBytes 860 Mbits/sec 0 369 KBytes
[ 5] 3.00-4.00 sec 102 MBytes 853 Mbits/sec 0 369 KBytes
[ 5] 4.00-5.00 sec 102 MBytes 855 Mbits/sec 0 369 KBytes
[ 5] 5.00-6.00 sec 101 MBytes 849 Mbits/sec 0 369 KBytes
[ 5] 6.00-7.00 sec 102 MBytes 860 Mbits/sec 0 369 KBytes
[ 5] 7.00-8.00 sec 102 MBytes 853 Mbits/sec 0 369 KBytes
[ 5] 8.00-9.00 sec 101 MBytes 851 Mbits/sec 0 369 KBytes
[ 5] 9.00-10.00 sec 102 MBytes 856 Mbits/sec 0 369 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 1020 MBytes 856 Mbits/sec 0 sender
[ 5] 0.00-10.00 sec 1019 MBytes 854 Mbits/sec receiver

iperf Done.
root@TGLA:~# ethtool -K enp0s30f4 tso on
root@TGLA:~# iperf3 -c 169.254.168.191
Connecting to host 169.254.168.191, port 5201
[ 5] local 169.254.50.108 port 45860 connected to 169.254.168.191
port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 107 KBytes 879 Kbits/sec 0 1.41 KBytes
[ 5] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
[ 5] 2.00-3.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
[ 5] 3.00-4.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
[ 5] 4.00-5.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
[ 5] 5.00-6.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
[ 5] 6.00-7.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
[ 5] 7.00-8.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes

Clear mss in TDES and call stmmac_enable_tso() to indicate
a new TSO transmission when it is enabled from TSO off using
ethtool command

Fixes: f748be531d70 ("stmmac: support new GMAC4")
Signed-off-by: Ling Pei Lee <[email protected]>
Signed-off-by: Wong Vee Khee <[email protected]>
---
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c | 12 ++++++++++++
1 file changed, 12 insertions(+)

diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
index b745d624b2cb..9e2ea0e0bd68 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
@@ -5460,6 +5460,8 @@ static netdev_features_t stmmac_fix_features(struct net_device *dev,
netdev_features_t features)
{
struct stmmac_priv *priv = netdev_priv(dev);
+ u32 tx_cnt = priv->plat->tx_queues_to_use;
+ u32 chan;

if (priv->plat->rx_coe == STMMAC_RX_COE_NONE)
features &= ~NETIF_F_RXCSUM;
@@ -5483,6 +5485,16 @@ static netdev_features_t stmmac_fix_features(struct net_device *dev,
priv->tso = false;
}

+ for (chan = 0; chan < tx_cnt; chan++) {
+ struct stmmac_tx_queue *tx_q = &priv->tx_queue[chan];
+
+ /* TSO and TBS cannot co-exist */
+ if (tx_q->tbs & STMMAC_TBS_AVAIL)
+ continue;
+
+ tx_q->mss = 0;
+ stmmac_enable_tso(priv, priv->ioaddr, priv->tso, chan);
+ }
return features;
}

--
2.25.1


2022-03-03 07:27:56

by Jakub Kicinski

[permalink] [raw]
Subject: Re: [PATCH net 1/1] net: stmmac: Resolve poor line rate after switching from TSO off to TSO on

On Mon, 28 Feb 2022 19:15:58 +0800 Wong Vee Khee wrote:
> From: Ling Pei Lee <[email protected]>
>
> Sequential execution of these steps:
> i) TSO ON – iperf3 execution,
> ii) TSO OFF – iperf3 execution,
> iii) TSO ON – iperf3 execution, it leads to iperf3 0 bytes transfer.

IMHO the iperf output can be dropped from the commit message,
it doesn't add much beyond this description.

> Clear mss in TDES and call stmmac_enable_tso() to indicate
> a new TSO transmission when it is enabled from TSO off using
> ethtool command

How does the TSO get disabled I don't see any ...enable_tso(, 0, )
calls in the driver? And why call enable in fix_features rather
than set_features?

2022-07-15 13:06:43

by Wong Vee Khee

[permalink] [raw]
Subject: Re: [PATCH net 1/1] net: stmmac: Resolve poor line rate after switching from TSO off to TSO on

On Wed, Mar 02, 2022 at 10:32:48PM -0800, Jakub Kicinski wrote:
> On Mon, 28 Feb 2022 19:15:58 +0800 Wong Vee Khee wrote:
> > From: Ling Pei Lee <[email protected]>
> >
> > Sequential execution of these steps:
> > i) TSO ON – iperf3 execution,
> > ii) TSO OFF – iperf3 execution,
> > iii) TSO ON – iperf3 execution, it leads to iperf3 0 bytes transfer.
>
> IMHO the iperf output can be dropped from the commit message,
> it doesn't add much beyond this description.
>

Noted. Will drop those on next revision of pull request.

> > Clear mss in TDES and call stmmac_enable_tso() to indicate
> > a new TSO transmission when it is enabled from TSO off using
> > ethtool command
>
> How does the TSO get disabled I don't see any ...enable_tso(, 0, )
> calls in the driver? And why call enable in fix_features rather
> than set_features?

It is disable when 'priv->tso = 0' in this same function.
The reason I put this in fix_features rather than set_features is
because the commit f748be531d70("stmmac: support new GMAC4") has
already introduced the following codes in fix_features:-

+ /* Disable tso if asked by ethtool */
+ if ((priv->plat->tso_en) && (priv->dma_cap.tsoen)) {
+ if (features & NETIF_F_TSO)
+ priv->tso = true;
+ else
+ priv->tso = false;
+ }

BR,
Vee Khee