Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Sean Anderson <sean.anderson@seco.com>
To:     "David S . Miller" <davem@davemloft.net>,
        Jakub Kicinski <kuba@kernel.org>,
        Madalin Bucur <madalin.bucur@nxp.com>, netdev@vger.kernel.org
Cc:     Russell King <linux@armlinux.org.uk>,
        Paolo Abeni <pabeni@redhat.com>,
        linux-arm-kernel@lists.infradead.org,
        Eric Dumazet <edumazet@google.com>,
        linux-kernel@vger.kernel.org,
        Sean Anderson <sean.anderson@seco.com>,
        Li Yang <leoyang.li@nxp.com>, linuxppc-dev@lists.ozlabs.org
Subject: [PATCH net-next v2 28/35] net: dpaa: Adjust queue depth on rate change
Date:   Tue, 28 Jun 2022 18:13:57 -0400
Message-Id: <20220628221404.1444200-29-sean.anderson@seco.com>
In-Reply-To: <20220628221404.1444200-1-sean.anderson@seco.com>
References: <20220628221404.1444200-1-sean.anderson@seco.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?XDjnG+eKS7snCRpmAwy+mFNQYV97WGbFKW5GppoS6dacd5L7RrfiquQcfH2V?=
 =?us-ascii?Q?JtcjRUExvH54tULdVHq7nhUjYiPfwAAnttV0BcVoIKh1fCwCRQQN8BamGRqv?=
 =?us-ascii?Q?FJMZ5KB0R59uaiDWPZLQYuFfaJq35nCUdc4kimMVIPdw0QARdCBe8q1qFa4D?=
 =?us-ascii?Q?y1KgGCNWCmQdE+ryDs93W0WUpQd0/bB5czVLNP/2EW10ZUByU/Rvcfh0GLAT?=
 =?us-ascii?Q?9ZCeZXbHg0WI9ShMV9FdQFsqUFpCRDCMKtsOso0ZPeYFkTj3FmfpiUNCo7c2?=
 =?us-ascii?Q?svhd2eHeKMqZD0tughm4Ut4T/GAd+k/KARoMd3GC25kD3pcsBaGdN2MP+L2d?=
 =?us-ascii?Q?kGo1dknJWuvIFXezIsTN+J7JYn8KdV7JtqaA3xMxIu7NN8BwV7YpEttI6YEw?=
 =?us-ascii?Q?kZrj4CqkjfWkMlMqvOWM8T24TN+eZeO32aUO3ANmpl5vrvVViMUXxM3q5wG1?=
 =?us-ascii?Q?/4AwYGvIz+rnncwN4XOx4patjegU/hatlgBE3zSdRCwu7upKKQdrm4qSk6dV?=
 =?us-ascii?Q?MBvJkKV8lCC1F9wnnvfvNuNR5R7Y58KXGktoBLPFpZwWJrCdp+xXDVmSjzWW?=
 =?us-ascii?Q?ZOnyOL1cmTsb9/A2j0UODFMNYgO4BrjctXFJPmKIPYH0MpTE9tVI4AZgJaDg?=
 =?us-ascii?Q?7HW+4s/6e/M73drhpF5ip/3NF/37Kpa6f3agt48/yxzgO0+s6JXy9UhKMT2T?=
 =?us-ascii?Q?GxRdasS3BTEt+mP/+7sEj7b7VY9ikyeRcX+hbKJYD2PqYTbz+klMtCkfsF5+?=
 =?us-ascii?Q?byf5JpzZAE4GBMcxVvF/ni/R3dUfC9y/ssr0uOSw1LGiB4nbiEojCnz84/Va?=
 =?us-ascii?Q?dylSrtZOtPiMZXGTFtH6J1tXnV51si5EqY7V2kft47f2Xes9cIkmzwpFkXR4?=
 =?us-ascii?Q?v2gT1F8/ppEtbjMvUdTa5V+A25hhyWMnPvSCjfVo6DNtwr5Rn8EGoK1IK+BX?=
 =?us-ascii?Q?lZgswiVpqZpJ6ko9eX5Nfqrh+bplrFT7/QEYd4r3L29iE+vpyz7nOP8UHRGt?=
 =?us-ascii?Q?laY92YURP661AKgQG0Rp5knzuYd69YiZ62/zgjZ2lK2wqkhHp5hF0NaFwJM9?=
 =?us-ascii?Q?4531ICkHdyBKafuLQKf0dm4m1iwSXWI/jBp7T8oee0gtSMHVov8WynSXZlZ4?=
 =?us-ascii?Q?Dm2ytviXpvurnixoKiHDcCbQ21SKtLamuuqbJvt+gDcB6Vx5iYMCNUqYVf8C?=
 =?us-ascii?Q?Bjy0bsVD5P8JGWKVz+PLiudkoWh6Yu0UcgqJllOQUwA+1bC4g0GWVi0XzSG+?=
 =?us-ascii?Q?q0MR6NYTBWsgrQcJfuqQw746djjf9TRiz2umzAqVpZNZHkLK8IlzItgoKDEv?=
 =?us-ascii?Q?loqTzl7TA5y8SMc+wh9CJxR99tuDrBOllFbIstv1Tl2UctKwwRxhDw1OijVy?=
 =?us-ascii?Q?KyJ4RYDPB3tevj3aDRFRRS0ps0g60U6Hhz9hAzor4hpegebLHb51MIU4jTAS?=
 =?us-ascii?Q?VOzbtk37ru16hVAjrY+pPHQq25QI8q59hemySop3ReqCO3A8UQOwPt1BEYqw?=
 =?us-ascii?Q?2b57XvKE0IjTgUw7Pq+6VD/ChxjiwM7AyEUoNAMkmy7C69DVmMYbXUR4EkhR?=
 =?us-ascii?Q?j9PjY5OkM/5m7CFmOMgF7fLFFhXqh2VtjJqBIg3LT7Z8YRDyhpS3KhzEBElT?=
 =?us-ascii?Q?JQ=3D=3D?=
X-OriginatorOrg: seco.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 254896b5-d12f-4247-4b43-08da5953a5f0
X-MS-Exchange-CrossTenant-AuthSource: DB7PR03MB4972.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Jun 2022 22:15:02.8430
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: bebe97c3-6438-442e-ade3-ff17aa50e733
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 9GXsOim+pjyqBEkNOSlJLx4ph4k18swP6V6qE3B75+Wy0ifZHH0Cz/SBOwdF0QbqMzXLnq2zqhUvpIhz4jVDKw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR03MB3013
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Instead of setting the queue depth once during probe, adjust it on the
fly whenever we configure the link. This is a bit unusal, since usually
the DPAA driver calls into the FMAN driver, but here we do the opposite.
We need to add a netdev to struct mac_device for this, but it will soon
live in the phylink config.

I haven't tested this extensively, but it doesn't seem to break
anything. We could possibly optimize this a bit by keeping track of the
last rate, but for now we just update every time. 10GEC probably doesn't
need to call into this at all, but I've added it for consistency.

Signed-off-by: Sean Anderson <sean.anderson@seco.com>
---

Changes in v2:
- New

 .../net/ethernet/freescale/dpaa/dpaa_eth.c    | 38 +++++++++++++++++--
 .../net/ethernet/freescale/fman/fman_dtsec.c  |  1 +
 .../net/ethernet/freescale/fman/fman_memac.c  |  1 +
 .../net/ethernet/freescale/fman/fman_tgec.c   |  7 +++-
 drivers/net/ethernet/freescale/fman/mac.h     |  3 ++
 5 files changed, 44 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c b/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c
index 2b1fce99c004..fd81ebc7be44 100644
--- a/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c
+++ b/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c
@@ -197,6 +197,8 @@ static int dpaa_rx_extra_headroom;
 #define dpaa_get_max_mtu()	\
 	(dpaa_max_frm - (VLAN_ETH_HLEN + ETH_FCS_LEN))
 
+static void dpaa_eth_cgr_set_speed(struct mac_device *mac_dev, int speed);
+
 static int dpaa_netdev_init(struct net_device *net_dev,
 			    const struct net_device_ops *dpaa_ops,
 			    u16 tx_timeout)
@@ -262,6 +264,9 @@ static int dpaa_netdev_init(struct net_device *net_dev,
 	net_dev->needed_headroom = priv->tx_headroom;
 	net_dev->watchdog_timeo = msecs_to_jiffies(tx_timeout);
 
+	mac_dev->net_dev = net_dev;
+	mac_dev->update_speed = dpaa_eth_cgr_set_speed;
+
 	/* start without the RUNNING flag, phylib controls it later */
 	netif_carrier_off(net_dev);
 
@@ -826,10 +831,10 @@ static int dpaa_eth_cgr_init(struct dpaa_priv *priv)
 	initcgr.we_mask = cpu_to_be16(QM_CGR_WE_CSCN_EN | QM_CGR_WE_CS_THRES);
 	initcgr.cgr.cscn_en = QM_CGR_EN;
 
-	/* Set different thresholds based on the MAC speed.
-	 * This may turn suboptimal if the MAC is reconfigured at a speed
-	 * lower than its max, e.g. if a dTSEC later negotiates a 100Mbps link.
-	 * In such cases, we ought to reconfigure the threshold, too.
+	/* Set different thresholds based on the configured MAC speed.
+	 * This may turn suboptimal if the MAC is reconfigured at another
+	 * speed, so MACs must call dpaa_eth_cgr_set_speed in their adjust_link
+	 * callback.
 	 */
 	if (priv->mac_dev->if_support & SUPPORTED_10000baseT_Full)
 		cs_th = DPAA_CS_THRESHOLD_10G;
@@ -858,6 +863,31 @@ static int dpaa_eth_cgr_init(struct dpaa_priv *priv)
 	return err;
 }
 
+static void dpaa_eth_cgr_set_speed(struct mac_device *mac_dev, int speed)
+{
+	struct net_device *net_dev = mac_dev->net_dev;
+	struct dpaa_priv *priv = netdev_priv(net_dev);
+	struct qm_mcc_initcgr opts = { };
+	u32 cs_th;
+	int err;
+
+	opts.we_mask = cpu_to_be16(QM_CGR_WE_CS_THRES);
+	switch (speed) {
+	case SPEED_10000:
+		cs_th = DPAA_CS_THRESHOLD_10G;
+		break;
+	case SPEED_1000:
+	default:
+		cs_th = DPAA_CS_THRESHOLD_1G;
+		break;
+	}
+	qm_cgr_cs_thres_set64(&opts.cgr.cs_thres, cs_th, 1);
+
+	err = qman_update_cgr_safe(&priv->cgr_data.cgr, &opts);
+	if (err)
+		netdev_err(net_dev, "could not update speed: %d\n", err);
+}
+
 static inline void dpaa_setup_ingress(const struct dpaa_priv *priv,
 				      struct dpaa_fq *fq,
 				      const struct qman_fq *template)
diff --git a/drivers/net/ethernet/freescale/fman/fman_dtsec.c b/drivers/net/ethernet/freescale/fman/fman_dtsec.c
index f2dd07b714ea..6617932fd3fd 100644
--- a/drivers/net/ethernet/freescale/fman/fman_dtsec.c
+++ b/drivers/net/ethernet/freescale/fman/fman_dtsec.c
@@ -1244,6 +1244,7 @@ static void adjust_link_dtsec(struct mac_device *mac_dev)
 	}
 
 	dtsec_adjust_link(fman_mac, phy_dev->speed);
+	mac_dev->update_speed(mac_dev, phy_dev->speed);
 	fman_get_pause_cfg(mac_dev, &rx_pause, &tx_pause);
 	err = fman_set_mac_active_pause(mac_dev, rx_pause, tx_pause);
 	if (err < 0)
diff --git a/drivers/net/ethernet/freescale/fman/fman_memac.c b/drivers/net/ethernet/freescale/fman/fman_memac.c
index 8ad93a4c0c21..02b3a0a2d5d1 100644
--- a/drivers/net/ethernet/freescale/fman/fman_memac.c
+++ b/drivers/net/ethernet/freescale/fman/fman_memac.c
@@ -782,6 +782,7 @@ static void adjust_link_memac(struct mac_device *mac_dev)
 
 	fman_mac = mac_dev->fman_mac;
 	memac_adjust_link(fman_mac, phy_dev->speed);
+	mac_dev->update_speed(mac_dev, phy_dev->speed);
 
 	fman_get_pause_cfg(mac_dev, &rx_pause, &tx_pause);
 	err = fman_set_mac_active_pause(mac_dev, rx_pause, tx_pause);
diff --git a/drivers/net/ethernet/freescale/fman/fman_tgec.c b/drivers/net/ethernet/freescale/fman/fman_tgec.c
index f4cdf0cf7c32..33f3b1cc2cfe 100644
--- a/drivers/net/ethernet/freescale/fman/fman_tgec.c
+++ b/drivers/net/ethernet/freescale/fman/fman_tgec.c
@@ -601,8 +601,11 @@ static int tgec_del_hash_mac_address(struct fman_mac *tgec,
 	return 0;
 }
 
-static void adjust_link_void(struct mac_device *mac_dev)
+static void tgec_adjust_link(struct mac_device *mac_dev)
 {
+	struct phy_device *phy_dev = mac_dev->phy_dev;
+
+	mac_dev->update_speed(mac_dev, phy_dev->speed);
 }
 
 static int tgec_set_exception(struct fman_mac *tgec,
@@ -794,7 +797,7 @@ int tgec_initialization(struct mac_device *mac_dev,
 	mac_dev->set_allmulti		= tgec_set_allmulti;
 	mac_dev->set_tstamp		= tgec_set_tstamp;
 	mac_dev->set_multi		= fman_set_multi;
-	mac_dev->adjust_link            = adjust_link_void;
+	mac_dev->adjust_link            = tgec_adjust_link;
 	mac_dev->enable			= tgec_enable;
 	mac_dev->disable		= tgec_disable;
 
diff --git a/drivers/net/ethernet/freescale/fman/mac.h b/drivers/net/ethernet/freescale/fman/mac.h
index a55efcb7998c..b95d384271bd 100644
--- a/drivers/net/ethernet/freescale/fman/mac.h
+++ b/drivers/net/ethernet/freescale/fman/mac.h
@@ -28,6 +28,7 @@ struct mac_device {
 	struct phy_device	*phy_dev;
 	phy_interface_t		phy_if;
 	struct device_node	*phy_node;
+	struct net_device	*net_dev;
 
 	bool autoneg_pause;
 	bool rx_pause_req;
@@ -56,6 +57,8 @@ struct mac_device {
 	int (*remove_hash_mac_addr)(struct fman_mac *mac_dev,
 				    enet_addr_t *eth_addr);
 
+	void (*update_speed)(struct mac_device *mac_dev, int speed);
+
 	struct fman_mac		*fman_mac;
 	struct mac_priv_s	*priv;
 };
-- 
2.35.1.1320.gc452695387.dirty