2022-11-22 14:08:13

by Vladimir Oltean

[permalink] [raw]
Subject: [PATCH net] net: enetc: preserve TX ring priority across reconfiguration

In the blamed commit, a rudimentary reallocation procedure for RX buffer
descriptors was implemented, for the situation when their format changes
between normal (no PTP) and extended (PTP).

enetc_hwtstamp_set() calls enetc_close() and enetc_open() in a sequence,
and this sequence loses information which was previously configured in
the TX BDR Mode Register, specifically via the enetc_set_bdr_prio() call.
The TX ring priority is configured by tc-mqprio and tc-taprio, and
affects important things for TSN such as the TX time of packets. The
issue manifests itself most visibly by the fact that isochron --txtime
reports premature packet transmissions when PTP is first enabled on an
enetc interface.

Save the TX ring priority in a new field in struct enetc_bdr (occupies a
2 byte hole on arm64) in order to make this survive a ring reconfiguration.

Fixes: 434cebabd3a2 ("enetc: Add dynamic allocation of extended Rx BD rings")
Signed-off-by: Vladimir Oltean <[email protected]>
---
drivers/net/ethernet/freescale/enetc/enetc.c | 8 ++++---
drivers/net/ethernet/freescale/enetc/enetc.h | 1 +
.../net/ethernet/freescale/enetc/enetc_qos.c | 21 ++++++++++++-------
3 files changed, 19 insertions(+), 11 deletions(-)

diff --git a/drivers/net/ethernet/freescale/enetc/enetc.c b/drivers/net/ethernet/freescale/enetc/enetc.c
index b4988fe6bb93..9c0ab983b6e9 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc.c
+++ b/drivers/net/ethernet/freescale/enetc/enetc.c
@@ -2061,7 +2061,7 @@ static void enetc_setup_txbdr(struct enetc_hw *hw, struct enetc_bdr *tx_ring)
/* enable Tx ints by setting pkt thr to 1 */
enetc_txbdr_wr(hw, idx, ENETC_TBICR0, ENETC_TBICR0_ICEN | 0x1);

- tbmr = ENETC_TBMR_EN;
+ tbmr = ENETC_TBMR_EN | ENETC_TBMR_SET_PRIO(tx_ring->prio);
if (tx_ring->ndev->features & NETIF_F_HW_VLAN_CTAG_TX)
tbmr |= ENETC_TBMR_VIH;

@@ -2464,7 +2464,8 @@ int enetc_setup_tc_mqprio(struct net_device *ndev, void *type_data)
/* Reset all ring priorities to 0 */
for (i = 0; i < priv->num_tx_rings; i++) {
tx_ring = priv->tx_ring[i];
- enetc_set_bdr_prio(hw, tx_ring->index, 0);
+ tx_ring->prio = 0;
+ enetc_set_bdr_prio(hw, tx_ring->index, tx_ring->prio);
}

return 0;
@@ -2483,7 +2484,8 @@ int enetc_setup_tc_mqprio(struct net_device *ndev, void *type_data)
*/
for (i = 0; i < num_tc; i++) {
tx_ring = priv->tx_ring[i];
- enetc_set_bdr_prio(hw, tx_ring->index, i);
+ tx_ring->prio = i;
+ enetc_set_bdr_prio(hw, tx_ring->index, tx_ring->prio);
}

/* Reset the number of netdev queues based on the TC count */
diff --git a/drivers/net/ethernet/freescale/enetc/enetc.h b/drivers/net/ethernet/freescale/enetc/enetc.h
index ea3ceed5bfa9..40bcd68c0c49 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc.h
+++ b/drivers/net/ethernet/freescale/enetc/enetc.h
@@ -95,6 +95,7 @@ struct enetc_bdr {
void __iomem *rcir;
};
u16 index;
+ u16 prio;
int bd_count; /* # of BDs */
int next_to_use;
int next_to_clean;
diff --git a/drivers/net/ethernet/freescale/enetc/enetc_qos.c b/drivers/net/ethernet/freescale/enetc/enetc_qos.c
index a842e1999122..fcebb54224c0 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc_qos.c
+++ b/drivers/net/ethernet/freescale/enetc/enetc_qos.c
@@ -137,6 +137,7 @@ int enetc_setup_tc_taprio(struct net_device *ndev, void *type_data)
struct tc_taprio_qopt_offload *taprio = type_data;
struct enetc_ndev_priv *priv = netdev_priv(ndev);
struct enetc_hw *hw = &priv->si->hw;
+ struct enetc_bdr *tx_ring;
int err;
int i;

@@ -145,16 +146,20 @@ int enetc_setup_tc_taprio(struct net_device *ndev, void *type_data)
if (priv->tx_ring[i]->tsd_enable)
return -EBUSY;

- for (i = 0; i < priv->num_tx_rings; i++)
- enetc_set_bdr_prio(hw, priv->tx_ring[i]->index,
- taprio->enable ? i : 0);
+ for (i = 0; i < priv->num_tx_rings; i++) {
+ tx_ring = priv->tx_ring[i];
+ tx_ring->prio = taprio->enable ? i : 0;
+ enetc_set_bdr_prio(hw, tx_ring->index, tx_ring->prio);
+ }

err = enetc_setup_taprio(ndev, taprio);
-
- if (err)
- for (i = 0; i < priv->num_tx_rings; i++)
- enetc_set_bdr_prio(hw, priv->tx_ring[i]->index,
- taprio->enable ? 0 : i);
+ if (err) {
+ for (i = 0; i < priv->num_tx_rings; i++) {
+ tx_ring = priv->tx_ring[i];
+ tx_ring->prio = taprio->enable ? 0 : i;
+ enetc_set_bdr_prio(hw, tx_ring->index, tx_ring->prio);
+ }
+ }

return err;
}
--
2.34.1


2022-11-23 16:05:11

by Alexander Lobakin

[permalink] [raw]
Subject: Re: [PATCH net] net: enetc: preserve TX ring priority across reconfiguration

From: Vladimir Oltean <[email protected]>
Date: Tue, 22 Nov 2022 15:09:36 +0200

> In the blamed commit, a rudimentary reallocation procedure for RX buffer
> descriptors was implemented, for the situation when their format changes
> between normal (no PTP) and extended (PTP).
>
> enetc_hwtstamp_set() calls enetc_close() and enetc_open() in a sequence,
> and this sequence loses information which was previously configured in
> the TX BDR Mode Register, specifically via the enetc_set_bdr_prio() call.
> The TX ring priority is configured by tc-mqprio and tc-taprio, and
> affects important things for TSN such as the TX time of packets. The
> issue manifests itself most visibly by the fact that isochron --txtime
> reports premature packet transmissions when PTP is first enabled on an
> enetc interface.
>
> Save the TX ring priority in a new field in struct enetc_bdr (occupies a
> 2 byte hole on arm64) in order to make this survive a ring reconfiguration.
>
> Fixes: 434cebabd3a2 ("enetc: Add dynamic allocation of extended Rx BD rings")
> Signed-off-by: Vladimir Oltean <[email protected]>
> ---
> drivers/net/ethernet/freescale/enetc/enetc.c | 8 ++++---
> drivers/net/ethernet/freescale/enetc/enetc.h | 1 +
> .../net/ethernet/freescale/enetc/enetc_qos.c | 21 ++++++++++++-------
> 3 files changed, 19 insertions(+), 11 deletions(-)

[...]

> err = enetc_setup_taprio(ndev, taprio);
> -
> - if (err)
> - for (i = 0; i < priv->num_tx_rings; i++)
> - enetc_set_bdr_prio(hw, priv->tx_ring[i]->index,
> - taprio->enable ? 0 : i);
> + if (err) {
> + for (i = 0; i < priv->num_tx_rings; i++) {
> + tx_ring = priv->tx_ring[i];
> + tx_ring->prio = taprio->enable ? 0 : i;

Side note: is that `taprio ? 0 : i` correct? It's an error path
IIUC, why not just unconditional 0?
I guess it is, so

Reviewed-by: Alexander Lobakin <[email protected]>

> + enetc_set_bdr_prio(hw, tx_ring->index, tx_ring->prio);
> + }
> + }
>
> return err;
> }
> --
> 2.34.1

Thanks,
Olek

2022-11-23 16:32:18

by Vladimir Oltean

[permalink] [raw]
Subject: Re: [PATCH net] net: enetc: preserve TX ring priority across reconfiguration

On Wed, Nov 23, 2022 at 04:51:16PM +0100, Alexander Lobakin wrote:
> > + if (err) {
> > + for (i = 0; i < priv->num_tx_rings; i++) {
> > + tx_ring = priv->tx_ring[i];
> > + tx_ring->prio = taprio->enable ? 0 : i;
>
> Side note: is that `taprio ? 0 : i` correct? It's an error path
> IIUC, why not just unconditional 0?

Yes, it is intended. On error path the priorities are restored.
If taprio->enable was false, but disabling failed, the ring priorities
need to be set as if taprio->enable was true.

2022-11-24 04:43:37

by patchwork-bot+netdevbpf

[permalink] [raw]
Subject: Re: [PATCH net] net: enetc: preserve TX ring priority across reconfiguration

Hello:

This patch was applied to netdev/net.git (master)
by Jakub Kicinski <[email protected]>:

On Tue, 22 Nov 2022 15:09:36 +0200 you wrote:
> In the blamed commit, a rudimentary reallocation procedure for RX buffer
> descriptors was implemented, for the situation when their format changes
> between normal (no PTP) and extended (PTP).
>
> enetc_hwtstamp_set() calls enetc_close() and enetc_open() in a sequence,
> and this sequence loses information which was previously configured in
> the TX BDR Mode Register, specifically via the enetc_set_bdr_prio() call.
> The TX ring priority is configured by tc-mqprio and tc-taprio, and
> affects important things for TSN such as the TX time of packets. The
> issue manifests itself most visibly by the fact that isochron --txtime
> reports premature packet transmissions when PTP is first enabled on an
> enetc interface.
>
> [...]

Here is the summary with links:
- [net] net: enetc: preserve TX ring priority across reconfiguration
https://git.kernel.org/netdev/net/c/290b5fe096e7

You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html