2023-03-06 09:00:24

by Sverdlin, Alexander

[permalink] [raw]
Subject: [PATCH] tty: serial: fsl_lpuart: fix race on RX DMA shutdown

From: Alexander Sverdlin <[email protected]>

From time to time DMA completion can come in the middle of DMA shutdown:

<process ctx>: <IRQ>:
lpuart32_shutdown()
lpuart_dma_shutdown()
del_timer_sync()
lpuart_dma_rx_complete()
lpuart_copy_rx_to_tty()
mod_timer()
lpuart_dma_rx_free()

When the timer fires a bit later, sport->dma_rx_desc is NULL:

Unable to handle kernel NULL pointer dereference at virtual address 0000000000000004
pc : lpuart_copy_rx_to_tty+0xcc/0x5bc
lr : lpuart_timer_func+0x1c/0x2c
Call trace:
lpuart_copy_rx_to_tty
lpuart_timer_func
call_timer_fn
__run_timers.part.0
run_timer_softirq
__do_softirq
__irq_exit_rcu
irq_exit
handle_domain_irq
gic_handle_irq
call_on_irq_stack
do_interrupt_handler
...

To fix this fold del_timer_sync() into lpuart_dma_rx_free() after
dmaengine_terminate_sync() to make sure timer will not be re-started in
lpuart_copy_rx_to_tty() <= lpuart_dma_rx_complete().

Signed-off-by: Alexander Sverdlin <[email protected]>
---
drivers/tty/serial/fsl_lpuart.c | 11 +++--------
1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/drivers/tty/serial/fsl_lpuart.c b/drivers/tty/serial/fsl_lpuart.c
index e945f41b93d43..47c267ee22e04 100644
--- a/drivers/tty/serial/fsl_lpuart.c
+++ b/drivers/tty/serial/fsl_lpuart.c
@@ -1354,6 +1354,7 @@ static void lpuart_dma_rx_free(struct uart_port *port)
struct dma_chan *chan = sport->dma_rx_chan;

dmaengine_terminate_sync(chan);
+ del_timer_sync(&sport->lpuart_timer);
dma_unmap_sg(chan->device->dev, &sport->rx_sgl, 1, DMA_FROM_DEVICE);
kfree(sport->rx_ring.buf);
sport->rx_ring.tail = 0;
@@ -1813,7 +1814,6 @@ static int lpuart32_startup(struct uart_port *port)
static void lpuart_dma_shutdown(struct lpuart_port *sport)
{
if (sport->lpuart_dma_rx_use) {
- del_timer_sync(&sport->lpuart_timer);
lpuart_dma_rx_free(&sport->port);
sport->lpuart_dma_rx_use = false;
}
@@ -1973,10 +1973,8 @@ lpuart_set_termios(struct uart_port *port, struct ktermios *termios,
* Since timer function acqures sport->port.lock, need to stop before
* acquring same lock because otherwise del_timer_sync() can deadlock.
*/
- if (old && sport->lpuart_dma_rx_use) {
- del_timer_sync(&sport->lpuart_timer);
+ if (old && sport->lpuart_dma_rx_use)
lpuart_dma_rx_free(&sport->port);
- }

spin_lock_irqsave(&sport->port.lock, flags);

@@ -2210,10 +2208,8 @@ lpuart32_set_termios(struct uart_port *port, struct ktermios *termios,
* Since timer function acqures sport->port.lock, need to stop before
* acquring same lock because otherwise del_timer_sync() can deadlock.
*/
- if (old && sport->lpuart_dma_rx_use) {
- del_timer_sync(&sport->lpuart_timer);
+ if (old && sport->lpuart_dma_rx_use)
lpuart_dma_rx_free(&sport->port);
- }

spin_lock_irqsave(&sport->port.lock, flags);

@@ -3014,7 +3010,6 @@ static int lpuart_suspend(struct device *dev)
* cannot resume as expected, hence gracefully release the
* Rx DMA path before suspend and start Rx DMA path on resume.
*/
- del_timer_sync(&sport->lpuart_timer);
lpuart_dma_rx_free(&sport->port);

/* Disable Rx DMA to use UART port as wakeup source */
--
2.34.1



2023-03-06 09:43:37

by Jiri Slaby

[permalink] [raw]
Subject: Re: [PATCH] tty: serial: fsl_lpuart: fix race on RX DMA shutdown

On 06. 03. 23, 10:00, A. Sverdlin wrote:
> From: Alexander Sverdlin <[email protected]>
>
> From time to time DMA completion can come in the middle of DMA shutdown:
>
> <process ctx>: <IRQ>:
> lpuart32_shutdown()
> lpuart_dma_shutdown()
> del_timer_sync()
> lpuart_dma_rx_complete()
> lpuart_copy_rx_to_tty()
> mod_timer()
> lpuart_dma_rx_free()
>
> When the timer fires a bit later, sport->dma_rx_desc is NULL:
>
> Unable to handle kernel NULL pointer dereference at virtual address 0000000000000004
> pc : lpuart_copy_rx_to_tty+0xcc/0x5bc
> lr : lpuart_timer_func+0x1c/0x2c
> Call trace:
> lpuart_copy_rx_to_tty
> lpuart_timer_func
> call_timer_fn
> __run_timers.part.0
> run_timer_softirq
> __do_softirq
> __irq_exit_rcu
> irq_exit
> handle_domain_irq
> gic_handle_irq
> call_on_irq_stack
> do_interrupt_handler
> ...
>
> To fix this fold del_timer_sync() into lpuart_dma_rx_free() after
> dmaengine_terminate_sync() to make sure timer will not be re-started in
> lpuart_copy_rx_to_tty() <= lpuart_dma_rx_complete().
>
> Signed-off-by: Alexander Sverdlin <[email protected]>

This should have some Fixes: tag, I believe.

> ---
> drivers/tty/serial/fsl_lpuart.c | 11 +++--------
> 1 file changed, 3 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/tty/serial/fsl_lpuart.c b/drivers/tty/serial/fsl_lpuart.c
> index e945f41b93d43..47c267ee22e04 100644
> --- a/drivers/tty/serial/fsl_lpuart.c
> +++ b/drivers/tty/serial/fsl_lpuart.c
> @@ -1354,6 +1354,7 @@ static void lpuart_dma_rx_free(struct uart_port *port)
> struct dma_chan *chan = sport->dma_rx_chan;
>
> dmaengine_terminate_sync(chan);

Maybe I'm missing something (I haven't looked into the code), but what
happens if the timer ticks here? Won't the dmaengine be restarted there?

> + del_timer_sync(&sport->lpuart_timer);
> dma_unmap_sg(chan->device->dev, &sport->rx_sgl, 1, DMA_FROM_DEVICE);
> kfree(sport->rx_ring.buf);
> sport->rx_ring.tail = 0;


--
js
suse labs


2023-03-06 10:25:20

by Sverdlin, Alexander

[permalink] [raw]
Subject: Re: [PATCH] tty: serial: fsl_lpuart: fix race on RX DMA shutdown

Hello Jiri,

On Mon, 2023-03-06 at 10:43 +0100, Jiri Slaby wrote:
> On 06. 03. 23, 10:00, A. Sverdlin wrote:
> > From: Alexander Sverdlin <[email protected]>
> >
> >  From time to time DMA completion can come in the middle of DMA
> > shutdown:
> >
> > <process ctx>:                          <IRQ>:
> > lpuart32_shutdown()
> >    lpuart_dma_shutdown()
> >      del_timer_sync()
> >                                         lpuart_dma_rx_complete()
> >                                           lpuart_copy_rx_to_tty()
> >                                             mod_timer()
> >      lpuart_dma_rx_free()
> >
> > When the timer fires a bit later, sport->dma_rx_desc is NULL:
> >
> > Unable to handle kernel NULL pointer dereference at virtual address
> > 0000000000000004
> > pc : lpuart_copy_rx_to_tty+0xcc/0x5bc
> > lr : lpuart_timer_func+0x1c/0x2c
> > Call trace:
> >   lpuart_copy_rx_to_tty
> >   lpuart_timer_func
> >   call_timer_fn
> >   __run_timers.part.0
> >   run_timer_softirq
> >   __do_softirq
> >   __irq_exit_rcu
> >   irq_exit
> >   handle_domain_irq
> >   gic_handle_irq
> >   call_on_irq_stack
> >   do_interrupt_handler
> >   ...
> >
> > To fix this fold del_timer_sync() into lpuart_dma_rx_free() after
> > dmaengine_terminate_sync() to make sure timer will not be re-
> > started in
> > lpuart_copy_rx_to_tty() <= lpuart_dma_rx_complete().
> >
> > Signed-off-by: Alexander Sverdlin <[email protected]>
>
> This should have some Fixes: tag, I believe.
>

sure, you are right, I'll use
Fixes: 4a8588a1cf86 ("serial: fsl_lpuart: delete timer on shutdown")

on respin.

> > ---
> >   drivers/tty/serial/fsl_lpuart.c | 11 +++--------
> >   1 file changed, 3 insertions(+), 8 deletions(-)
> >
> > diff --git a/drivers/tty/serial/fsl_lpuart.c
> > b/drivers/tty/serial/fsl_lpuart.c
> > index e945f41b93d43..47c267ee22e04 100644
> > --- a/drivers/tty/serial/fsl_lpuart.c
> > +++ b/drivers/tty/serial/fsl_lpuart.c
> > @@ -1354,6 +1354,7 @@ static void lpuart_dma_rx_free(struct
> > uart_port *port)
> >         struct dma_chan *chan = sport->dma_rx_chan;
> >  
> >         dmaengine_terminate_sync(chan);
>
> Maybe I'm missing something (I haven't looked into the code), but
> what
> happens if the timer ticks here? Won't the dmaengine be restarted
> there?
>

It's cyclic (used dmaengine_prep_dma_cyclic()), I see no restart in
lpuart_copy_rx_to_tty(), but maybe someone could double-check.

> > +       del_timer_sync(&sport->lpuart_timer);
> >         dma_unmap_sg(chan->device->dev, &sport->rx_sgl, 1,
> > DMA_FROM_DEVICE);
> >         kfree(sport->rx_ring.buf);
> >         sport->rx_ring.tail = 0;
>
>

--
Alexander Sverdlin
Siemens AG
http://www.siemens.com