Date:   Tue, 14 Feb 2023 15:28:41 +0100
From:   Andrew Lunn <andrew@lunn.ch>
To:     Wei Fang <wei.fang@nxp.com>
Cc:     Alexander Lobakin <alexandr.lobakin@intel.com>,
        Shenwei Wang <shenwei.wang@nxp.com>,
        Clark Wang <xiaoning.wang@nxp.com>,
        "davem@davemloft.net" <davem@davemloft.net>,
        "edumazet@google.com" <edumazet@google.com>,
        "kuba@kernel.org" <kuba@kernel.org>,
        "pabeni@redhat.com" <pabeni@redhat.com>,
        "simon.horman@corigine.com" <simon.horman@corigine.com>,
        "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
        dl-linux-imx <linux-imx@nxp.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH V2 net-next] net: fec: add CBS offload support
Message-ID: <Y+uamTHJSaNHvTbU@lunn.ch>
References: <20230213092912.2314029-1-wei.fang@nxp.com>
 <ed27795a-f81f-a913-8275-b6f516b4f384@intel.com>
 <Y+pjl3vzi7TQcLKm@lunn.ch>
 <8b25bd1f-4265-33ea-bdb9-bc700eff0b0e@intel.com>
 <Y+p8WZCPKhp4/RIH@lunn.ch>
 <DB9PR04MB810669A4AC47DE2F0CBEA25B88A29@DB9PR04MB8106.eurprd04.prod.outlook.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <DB9PR04MB810669A4AC47DE2F0CBEA25B88A29@DB9PR04MB8106.eurprd04.prod.outlook.com>
Precedence: bulk

> Sorry, I'm not very familiar with the configuration of pure software implementation
> of CBS. I tried to configure the CBS like the following. The bandwidth of queue 1 was
> set to 30Mbps. And the queue 2 is set to 20Mbps. Then one stream were sent the
> queue 1 and the rate was 50Mbps, the link speed was 1Gbps. But the result seemed that
> the CBS did not take effective.

I'm not that familiar with CBS, but that is what i would expect. You
are over subscribing the queue by 20Mbps, so that 20Mbps gets
relegated to best effort. And since you have a 1G link, you have
plenty of best effort bandwidth.

As with most QoS queuing, it only really makes a different to packet
loss when you oversubscribe the link as a whole.

So with your 30Mbps + 20Mbps + BE configuration on a 1G link, send
50Mbps + 0Mbps + 1Gbps. 30Mbps of your 50Mbps stream should be
guaranteed to arrive at the destination. The remaining 20Mbps needs to
share the remaining 970Mbps of link capacity with the 1G of BE
traffic. So you would expect to see a few extra Kbps of queue #1
traffic arriving and around 969Mbps of best effort traffic.

However, that is not really the case i'm interested in. This
discussion started from the point that autoneg has resulted in a much
smaller link capacity. The link is now over subscribed by the CBS
configuration. Should the hardware just give up and go back to default
behaviour, or should it continue to do some CBS?

Set lets start with a 7Mbps queue 1 and 5Mbps queue 2, on a link which
auto negs to 100Mbps. Generate traffic of 8Mbps, 6Mpbs and 100Mbps
BE. You would expect ~7Mbps, ~5Mbps and 88Mbps to arrive at the link
peer. Your two CBS flows get there reserved bandwidth, plus a little
of the BE. BE gets whats remains of the link. Test that and make sure
that is what actually happens with software CBS, and with your TC
offload to hardware.

Now force the link down to 10Mbps. The CBS queues then over subscribe
the link. Keep with the traffic generator producing 8Mbps, 6Mpbs and
100Mbps BE. What i guess the software CBS will do is 7Mbps, 3Mbps and
0 BE. You should confirm this with testing.

What does this mean for TC offload? You should be aiming for the same
behaviour. So even when the link is over subscribed, you should still
be programming the hardware.

   Andrew