From: Vitor Soares <[email protected]>
When the mcp251xfd_start_xmit() function fails, the driver stops
processing messages, and the interrupt routine does not return,
running indefinitely even after killing the running application.
Error messages:
[ 441.298819] mcp251xfd spi2.0 can0: ERROR in mcp251xfd_start_xmit: -16
[ 441.306498] mcp251xfd spi2.0 can0: Transmit Event FIFO buffer not empty. (seq=0x000017c7, tef_tail=0x000017cf, tef_head=0x000017d0, tx_head=0x000017d3).
.. and repeat forever.
The issue can be triggered when multiple devices share the same
SPI interface. And there is concurrent access to the bus.
The problem occurs because tx_ring->head increments even if
mcp251xfd_start_xmit() fails. Consequently, the driver skips one
TX package while still expecting a response in
mcp251xfd_handle_tefif_one().
This patch resolves the issue by decreasing tx_ring->head if
mcp251xfd_start_xmit() fails. With the fix, if we attempt to trigger
the issue again, the driver prints an error and discard the message.
Fixes: 55e5b97f003e ("can: mcp25xxfd: add driver for Microchip MCP25xxFD SPI CAN")
Cc: [email protected]
Signed-off-by: Vitor Soares <[email protected]>
---
drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c | 27 ++++++++++----------
1 file changed, 14 insertions(+), 13 deletions(-)
diff --git a/drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c b/drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c
index 160528d3cc26..a8eb941c1b95 100644
--- a/drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c
+++ b/drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c
@@ -181,25 +181,26 @@ netdev_tx_t mcp251xfd_start_xmit(struct sk_buff *skb,
tx_obj = mcp251xfd_get_tx_obj_next(tx_ring);
mcp251xfd_tx_obj_from_skb(priv, tx_obj, skb, tx_ring->head);
- /* Stop queue if we occupy the complete TX FIFO */
tx_head = mcp251xfd_get_tx_head(tx_ring);
- tx_ring->head++;
- if (mcp251xfd_get_tx_free(tx_ring) == 0)
- netif_stop_queue(ndev);
-
frame_len = can_skb_get_frame_len(skb);
- err = can_put_echo_skb(skb, ndev, tx_head, frame_len);
- if (!err)
- netdev_sent_queue(priv->ndev, frame_len);
+ can_put_echo_skb(skb, ndev, tx_head, frame_len);
+
+ tx_ring->head++;
err = mcp251xfd_tx_obj_write(priv, tx_obj);
- if (err)
- goto out_err;
+ if (err) {
+ can_free_echo_skb(ndev, tx_head, NULL);
- return NETDEV_TX_OK;
+ tx_ring->head--;
+
+ netdev_err(priv->ndev, "ERROR in %s: %d\n", __func__, err);
+ } else {
+ /* Stop queue if we occupy the complete TX FIFO */
+ if (mcp251xfd_get_tx_free(tx_ring) == 0)
+ netif_stop_queue(ndev);
- out_err:
- netdev_err(priv->ndev, "ERROR in %s: %d\n", __func__, err);
+ netdev_sent_queue(priv->ndev, frame_len);
+ }
return NETDEV_TX_OK;
}
--
2.34.1
On 07.03.2024 12:04:42, Vitor Soares wrote:
> From: Vitor Soares <[email protected]>
>
> When the mcp251xfd_start_xmit() function fails, the driver stops
> processing messages, and the interrupt routine does not return,
> running indefinitely even after killing the running application.
>
> Error messages:
> [ 441.298819] mcp251xfd spi2.0 can0: ERROR in mcp251xfd_start_xmit: -16
> [ 441.306498] mcp251xfd spi2.0 can0: Transmit Event FIFO buffer not empty. (seq=0x000017c7, tef_tail=0x000017cf, tef_head=0x000017d0, tx_head=0x000017d3).
> ... and repeat forever.
>
> The issue can be triggered when multiple devices share the same
> SPI interface. And there is concurrent access to the bus.
>
> The problem occurs because tx_ring->head increments even if
> mcp251xfd_start_xmit() fails. Consequently, the driver skips one
> TX package while still expecting a response in
> mcp251xfd_handle_tefif_one().
>
> This patch resolves the issue by decreasing tx_ring->head if
> mcp251xfd_start_xmit() fails. With the fix, if we attempt to trigger
> the issue again, the driver prints an error and discard the message.
What about returning NETDEV_TX_BUSY, then the networking stack will
retry.
> Fixes: 55e5b97f003e ("can: mcp25xxfd: add driver for Microchip MCP25xxFD SPI CAN")
> Cc: [email protected]
> Signed-off-by: Vitor Soares <[email protected]>
> ---
> drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c | 27 ++++++++++----------
> 1 file changed, 14 insertions(+), 13 deletions(-)
>
> diff --git a/drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c b/drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c
> index 160528d3cc26..a8eb941c1b95 100644
> --- a/drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c
> +++ b/drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c
> @@ -181,25 +181,26 @@ netdev_tx_t mcp251xfd_start_xmit(struct sk_buff *skb,
> tx_obj = mcp251xfd_get_tx_obj_next(tx_ring);
> mcp251xfd_tx_obj_from_skb(priv, tx_obj, skb, tx_ring->head);
>
> - /* Stop queue if we occupy the complete TX FIFO */
> tx_head = mcp251xfd_get_tx_head(tx_ring);
> - tx_ring->head++;
> - if (mcp251xfd_get_tx_free(tx_ring) == 0)
> - netif_stop_queue(ndev);
> -
> frame_len = can_skb_get_frame_len(skb);
> - err = can_put_echo_skb(skb, ndev, tx_head, frame_len);
> - if (!err)
> - netdev_sent_queue(priv->ndev, frame_len);
> + can_put_echo_skb(skb, ndev, tx_head, frame_len);
> +
> + tx_ring->head++;
>
> err = mcp251xfd_tx_obj_write(priv, tx_obj);
> - if (err)
> - goto out_err;
> + if (err) {
> + can_free_echo_skb(ndev, tx_head, NULL);
>
> - return NETDEV_TX_OK;
> + tx_ring->head--;
> +
I'm not sure, if we want an error message for -EBUSY. We could add
proper ethtool statistics.
> + netdev_err(priv->ndev, "ERROR in %s: %d\n", __func__, err);
> + } else {
> + /* Stop queue if we occupy the complete TX FIFO */
> + if (mcp251xfd_get_tx_free(tx_ring) == 0)
> + netif_stop_queue(ndev);
>
> - out_err:
> - netdev_err(priv->ndev, "ERROR in %s: %d\n", __func__, err);
> + netdev_sent_queue(priv->ndev, frame_len);
> + }
>
> return NETDEV_TX_OK;
> }
regards,
Marc
--
Pengutronix e.K. | Marc Kleine-Budde |
Embedded Linux | https://www.pengutronix.de |
Vertretung Nürnberg | Phone: +49-5121-206917-129 |
Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-9 |
Sorry, resend. I fsck up the linux-can ML's address.
On 07.03.2024 12:04:42, Vitor Soares wrote:
> From: Vitor Soares <[email protected]>
>
> When the mcp251xfd_start_xmit() function fails, the driver stops
> processing messages, and the interrupt routine does not return,
> running indefinitely even after killing the running application.
>
> Error messages:
> [ 441.298819] mcp251xfd spi2.0 can0: ERROR in mcp251xfd_start_xmit: -16
> [ 441.306498] mcp251xfd spi2.0 can0: Transmit Event FIFO buffer not empty. (seq=0x000017c7, tef_tail=0x000017cf, tef_head=0x000017d0, tx_head=0x000017d3).
> ... and repeat forever.
>
> The issue can be triggered when multiple devices share the same
> SPI interface. And there is concurrent access to the bus.
>
> The problem occurs because tx_ring->head increments even if
> mcp251xfd_start_xmit() fails. Consequently, the driver skips one
> TX package while still expecting a response in
> mcp251xfd_handle_tefif_one().
>
> This patch resolves the issue by decreasing tx_ring->head if
> mcp251xfd_start_xmit() fails. With the fix, if we attempt to trigger
> the issue again, the driver prints an error and discard the message.
What about returning NETDEV_TX_BUSY, then the networking stack will
retry.
> Fixes: 55e5b97f003e ("can: mcp25xxfd: add driver for Microchip MCP25xxFD SPI CAN")
> Cc: [email protected]
> Signed-off-by: Vitor Soares <[email protected]>
> ---
> drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c | 27 ++++++++++----------
> 1 file changed, 14 insertions(+), 13 deletions(-)
>
> diff --git a/drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c b/drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c
> index 160528d3cc26..a8eb941c1b95 100644
> --- a/drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c
> +++ b/drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c
> @@ -181,25 +181,26 @@ netdev_tx_t mcp251xfd_start_xmit(struct sk_buff *skb,
> tx_obj = mcp251xfd_get_tx_obj_next(tx_ring);
> mcp251xfd_tx_obj_from_skb(priv, tx_obj, skb, tx_ring->head);
>
> - /* Stop queue if we occupy the complete TX FIFO */
> tx_head = mcp251xfd_get_tx_head(tx_ring);
> - tx_ring->head++;
> - if (mcp251xfd_get_tx_free(tx_ring) == 0)
> - netif_stop_queue(ndev);
> -
> frame_len = can_skb_get_frame_len(skb);
> - err = can_put_echo_skb(skb, ndev, tx_head, frame_len);
> - if (!err)
> - netdev_sent_queue(priv->ndev, frame_len);
> + can_put_echo_skb(skb, ndev, tx_head, frame_len);
> +
> + tx_ring->head++;
>
> err = mcp251xfd_tx_obj_write(priv, tx_obj);
> - if (err)
> - goto out_err;
> + if (err) {
> + can_free_echo_skb(ndev, tx_head, NULL);
>
> - return NETDEV_TX_OK;
> + tx_ring->head--;
> +
I'm not sure, if we want an error message for -EBUSY. We could add
proper ethtool statistics.
> + netdev_err(priv->ndev, "ERROR in %s: %d\n", __func__, err);
> + } else {
> + /* Stop queue if we occupy the complete TX FIFO */
> + if (mcp251xfd_get_tx_free(tx_ring) == 0)
> + netif_stop_queue(ndev);
>
> - out_err:
> - netdev_err(priv->ndev, "ERROR in %s: %d\n", __func__, err);
> + netdev_sent_queue(priv->ndev, frame_len);
> + }
>
> return NETDEV_TX_OK;
> }
regards,
Marc
--
Pengutronix e.K. | Marc Kleine-Budde |
Embedded Linux | https://www.pengutronix.de |
Vertretung Nürnberg | Phone: +49-5121-206917-129 |
Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-9 |
Hi Marc,
On Thu, 2024-03-07 at 13:23 +0100, Marc Kleine-Budde wrote:
> Sorry, resend. I fsck up the linux-can ML's address.
Thanks for that.
>
> On 07.03.2024 12:04:42, Vitor Soares wrote:
> > From: Vitor Soares <[email protected]>
> >
> > When the mcp251xfd_start_xmit() function fails, the driver stops
> > processing messages, and the interrupt routine does not return,
> > running indefinitely even after killing the running application.
> >
> > Error messages:
> > [ 441.298819] mcp251xfd spi2.0 can0: ERROR in
> > mcp251xfd_start_xmit: -16
> > [ 441.306498] mcp251xfd spi2.0 can0: Transmit Event FIFO buffer
> > not empty. (seq=0x000017c7, tef_tail=0x000017cf,
> > tef_head=0x000017d0, tx_head=0x000017d3).
> > ... and repeat forever.
> >
> > The issue can be triggered when multiple devices share the same
> > SPI interface. And there is concurrent access to the bus.
> >
> > The problem occurs because tx_ring->head increments even if
> > mcp251xfd_start_xmit() fails. Consequently, the driver skips one
> > TX package while still expecting a response in
> > mcp251xfd_handle_tefif_one().
> >
> > This patch resolves the issue by decreasing tx_ring->head if
> > mcp251xfd_start_xmit() fails. With the fix, if we attempt to
> > trigger
> > the issue again, the driver prints an error and discard the
> > message.
>
> What about returning NETDEV_TX_BUSY, then the networking stack will
> retry.
Do you mean when err == -EBUSY?
>
> > Fixes: 55e5b97f003e ("can: mcp25xxfd: add driver for Microchip
> > MCP25xxFD SPI CAN")
> > Cc: [email protected]
> > Signed-off-by: Vitor Soares <[email protected]>
> > ---
> > drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c | 27 ++++++++++------
> > ----
> > 1 file changed, 14 insertions(+), 13 deletions(-)
> >
> > diff --git a/drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c
> > b/drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c
> > index 160528d3cc26..a8eb941c1b95 100644
> > --- a/drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c
> > +++ b/drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c
> > @@ -181,25 +181,26 @@ netdev_tx_t mcp251xfd_start_xmit(struct
> > sk_buff *skb,
> > tx_obj = mcp251xfd_get_tx_obj_next(tx_ring);
> > mcp251xfd_tx_obj_from_skb(priv, tx_obj, skb, tx_ring-
> > >head);
> >
> > - /* Stop queue if we occupy the complete TX FIFO */
> > tx_head = mcp251xfd_get_tx_head(tx_ring);
> > - tx_ring->head++;
> > - if (mcp251xfd_get_tx_free(tx_ring) == 0)
> > - netif_stop_queue(ndev);
> > -
> > frame_len = can_skb_get_frame_len(skb);
> > - err = can_put_echo_skb(skb, ndev, tx_head, frame_len);
> > - if (!err)
> > - netdev_sent_queue(priv->ndev, frame_len);
> > + can_put_echo_skb(skb, ndev, tx_head, frame_len);
> > +
> > + tx_ring->head++;
> >
> > err = mcp251xfd_tx_obj_write(priv, tx_obj);
> > - if (err)
> > - goto out_err;
> > + if (err) {
> > + can_free_echo_skb(ndev, tx_head, NULL);
> >
> > - return NETDEV_TX_OK;
> > + tx_ring->head--;
> > +
I'm testing returning here when err = -EBUSY, but can_put_echo_skb()
should be invoked after mcp251xfd_tx_obj_write(). Otherwise, I get a
Kernel NULL pointer dereference error.
>
> I'm not sure, if we want an error message for -EBUSY. We could add
> proper ethtool statistics.
>
> > + netdev_err(priv->ndev, "ERROR in %s: %d\n",
> > __func__, err);
> > + } else {
> > + /* Stop queue if we occupy the complete TX FIFO */
> > + if (mcp251xfd_get_tx_free(tx_ring) == 0)
> > + netif_stop_queue(ndev);
> >
> > - out_err:
> > - netdev_err(priv->ndev, "ERROR in %s: %d\n", __func__, err);
> > + netdev_sent_queue(priv->ndev, frame_len);
> > + }
> >
> > return NETDEV_TX_OK;
> > }
>
> regards,
> Marc
>
Best regards,
Vitor Soares
On 07.03.2024 16:54:13, vitor wrote:
> > On 07.03.2024 12:04:42, Vitor Soares wrote:
> > > From: Vitor Soares <[email protected]>
> > >
> > > When the mcp251xfd_start_xmit() function fails, the driver stops
> > > processing messages, and the interrupt routine does not return,
> > > running indefinitely even after killing the running application.
> > >
> > > Error messages:
> > > [ 441.298819] mcp251xfd spi2.0 can0: ERROR in
> > > mcp251xfd_start_xmit: -16
> > > [ 441.306498] mcp251xfd spi2.0 can0: Transmit Event FIFO buffer
> > > not empty. (seq=0x000017c7, tef_tail=0x000017cf,
> > > tef_head=0x000017d0, tx_head=0x000017d3).
> > > ... and repeat forever.
> > >
> > > The issue can be triggered when multiple devices share the same
> > > SPI interface. And there is concurrent access to the bus.
> > >
> > > The problem occurs because tx_ring->head increments even if
> > > mcp251xfd_start_xmit() fails. Consequently, the driver skips one
> > > TX package while still expecting a response in
> > > mcp251xfd_handle_tefif_one().
> > >
> > > This patch resolves the issue by decreasing tx_ring->head if
> > > mcp251xfd_start_xmit() fails. With the fix, if we attempt to
> > > trigger
> > > the issue again, the driver prints an error and discard the
> > > message.
> >
> > What about returning NETDEV_TX_BUSY, then the networking stack will
> > retry.
>
> Do you mean when err == -EBUSY?
ACK
> >
> > > Fixes: 55e5b97f003e ("can: mcp25xxfd: add driver for Microchip
> > > MCP25xxFD SPI CAN")
> > > Cc: [email protected]
> > > Signed-off-by: Vitor Soares <[email protected]>
> > > ---
> > > drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c | 27 ++++++++++------
> > > ----
> > > 1 file changed, 14 insertions(+), 13 deletions(-)
> > >
> > > diff --git a/drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c
> > > b/drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c
> > > index 160528d3cc26..a8eb941c1b95 100644
> > > --- a/drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c
> > > +++ b/drivers/net/can/spi/mcp251xfd/mcp251xfd-tx.c
> > > @@ -181,25 +181,26 @@ netdev_tx_t mcp251xfd_start_xmit(struct
> > > sk_buff *skb,
> > > tx_obj = mcp251xfd_get_tx_obj_next(tx_ring);
> > > mcp251xfd_tx_obj_from_skb(priv, tx_obj, skb, tx_ring-
> > > >head);
> > >
> > > - /* Stop queue if we occupy the complete TX FIFO */
> > > tx_head = mcp251xfd_get_tx_head(tx_ring);
> > > - tx_ring->head++;
> > > - if (mcp251xfd_get_tx_free(tx_ring) == 0)
> > > - netif_stop_queue(ndev);
> > > -
> > > frame_len = can_skb_get_frame_len(skb);
> > > - err = can_put_echo_skb(skb, ndev, tx_head, frame_len);
> > > - if (!err)
> > > - netdev_sent_queue(priv->ndev, frame_len);
> > > + can_put_echo_skb(skb, ndev, tx_head, frame_len);
> > > +
> > > + tx_ring->head++;
> > >
> > > err = mcp251xfd_tx_obj_write(priv, tx_obj);
> > > - if (err)
> > > - goto out_err;
> > > + if (err) {
> > > + can_free_echo_skb(ndev, tx_head, NULL);
> > >
> > > - return NETDEV_TX_OK;
> > > + tx_ring->head--;
> > > +
>
> I'm testing returning here when err = -EBUSY, but can_put_echo_skb()
> should be invoked after mcp251xfd_tx_obj_write(). Otherwise, I get a
> Kernel NULL pointer dereference error.
I mean if mcp251xfd_tx_obj_write() returns -EBUSY don't print an error
message, on any other error it's ok to print the error.
> >
> > I'm not sure, if we want an error message for -EBUSY. We could add
> > proper ethtool statistics.
> >
> > > + netdev_err(priv->ndev, "ERROR in %s: %d\n",
> > > __func__, err);
> > > + } else {
> > > + /* Stop queue if we occupy the complete TX FIFO */
> > > + if (mcp251xfd_get_tx_free(tx_ring) == 0)
> > > + netif_stop_queue(ndev);
> > >
> > > - out_err:
> > > - netdev_err(priv->ndev, "ERROR in %s: %d\n", __func__, err);
> > > + netdev_sent_queue(priv->ndev, frame_len);
> > > + }
> > >
> > > return NETDEV_TX_OK;
> > > }
Marc
--
Pengutronix e.K. | Marc Kleine-Budde |
Embedded Linux | https://www.pengutronix.de |
Vertretung Nürnberg | Phone: +49-5121-206917-129 |
Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-9 |