According to DMA-330 errata notice[1] 71930, DMAKILL
cannot clear internal signal, named pipeline_req_active.
it makes that pl330 would wait forever in WFP state
although dma already send dma request if pl330 gets
dma request before entering WFP state.
The errata suggests that polling until entering WFP state
as workaround and then peripherals allows to issue dma request.
[1]: https://developer.arm.com/documentation/genc008428/latest
Signed-off-by: Bumyong Lee <[email protected]>
---
drivers/dma/pl330.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/dma/pl330.c b/drivers/dma/pl330.c
index 3cf0b38387ae..c29744bfdf2c 100644
--- a/drivers/dma/pl330.c
+++ b/drivers/dma/pl330.c
@@ -1053,6 +1053,9 @@ static bool _trigger(struct pl330_thread *thrd)
thrd->req_running = idx;
+ if (desc->rqtype == DMA_MEM_TO_DEV || desc->rqtype == DMA_DEV_TO_MEM)
+ UNTIL(thrd, PL330_STATE_WFP);
+
return true;
}
--
2.43.0
On Tue, 19 Dec 2023 14:50:26 +0900, Bumyong Lee wrote:
> According to DMA-330 errata notice[1] 71930, DMAKILL
> cannot clear internal signal, named pipeline_req_active.
> it makes that pl330 would wait forever in WFP state
> although dma already send dma request if pl330 gets
> dma request before entering WFP state.
>
> The errata suggests that polling until entering WFP state
> as workaround and then peripherals allows to issue dma request.
>
> [...]
Applied, thanks!
[1/1] dmaengine: pl330: issue_pending waits until WFP state
commit: d114d3a096194fb2a9c3bedd7be6587b97610625
Best regards,
--
~Vinod
Hi,
On Thu, Dec 21, 2023 at 10:00:26PM +0530, Vinod Koul wrote:
>
> On Tue, 19 Dec 2023 14:50:26 +0900, Bumyong Lee wrote:
> > According to DMA-330 errata notice[1] 71930, DMAKILL
> > cannot clear internal signal, named pipeline_req_active.
> > it makes that pl330 would wait forever in WFP state
> > although dma already send dma request if pl330 gets
> > dma request before entering WFP state.
> >
> > The errata suggests that polling until entering WFP state
> > as workaround and then peripherals allows to issue dma request.
> >
> > [...]
>
> Applied, thanks!
>
> [1/1] dmaengine: pl330: issue_pending waits until WFP state
> commit: d114d3a096194fb2a9c3bedd7be6587b97610625
This seems to cause a stall on my Quartz 64 model B (RK3566) once
Bluetooth over UART is initialized, when combined with a patch of mine
that enables DMA on UARTs [1]. Reverting this patch gets everything
running again.
The following are RCU stalls detected, followed by stack traces
produced with pseudo-NMI. Without pseudo-NMIs no stack traces are
produced.
rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
rcu: 0-...0: (0 ticks this GP) idle=80fc/1/0x4000000000000000 softirq=693/693 fqs=31498
rcu: 3-...0: (3 ticks this GP) idle=2b44/1/0x4000000000000000 softirq=553/556 fqs=31498
rcu: (detected by 1, t=162830 jiffies, g=-307, q=32 ncpus=4)
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 1200 Comm: (udev-worker) Not tainted 6.7.0-rc6-next-20231222-10300-g8b07e3811bc7 #17
Hardware name: Pine64 RK3566 Quartz64-B Board (DT)
pstate: 00400009 (nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : queued_spin_lock_slowpath+0x50/0x330
lr : pl330_irq_handler+0x2f8/0x5a0
sp : ffffffc080003ec0
pmr_save: 00000060
x29: ffffffc080003ec0 x28: ffffff80017c7000 x27: ffffff8001a58d80
x26: 0000000000000060 x25: ffffff80017d0338 x24: ffffff800161ae38
x23: ffffff8001597c00 x22: ffffffc081960000 x21: 0000000000000000
x20: ffffff800161ac80 x19: ffffff80010c5180 x18: 0000000000000000
x17: ffffffc06e724000 x16: ffffffc080000000 x15: 0000000000000000
x14: 0000000000000000 x13: ffffff80042f102f x12: ffffffc083ad3cc4
x11: 0000000000000040 x10: ffffff800022a0a8 x9 : ffffff800022a0a0
x8 : ffffff8000400270 x7 : 0000000000000000 x6 : 0000000000000000
x5 : ffffff8000400248 x4 : ffffffc06e724000 x3 : ffffffc080003fa0
x2 : 0000000000000000 x1 : 0000000000000001 x0 : ffffff800161ae38
Call trace:
queued_spin_lock_slowpath+0x50/0x330
__handle_irq_event_percpu+0x38/0x16c
handle_irq_event+0x44/0xf8
handle_fasteoi_irq+0xb0/0x28c
generic_handle_domain_irq+0x2c/0x44
gic_handle_irq+0x10c/0x240
call_on_irq_stack+0x24/0x4c
do_interrupt_handler+0x80/0x8c
el1_interrupt+0x44/0x98
el1h_64_irq_handler+0x18/0x24
el1h_64_irq+0x78/0x7c
__d_rehash+0x0/0x94
d_add+0x40/0x80
simple_lookup+0x4c/0x78
path_openat+0x5ec/0xed0
do_filp_open+0x80/0x12c
do_sys_openat2+0xb4/0xe8
__arm64_sys_openat+0x64/0xa4
invoke_syscall+0x48/0x114
el0_svc_common.constprop.0+0x40/0xe0
do_el0_svc+0x1c/0x28
el0_svc+0x34/0xd4
el0t_64_sync_handler+0x100/0x12c
el0t_64_sync+0x1a4/0x1a8
Sending NMI from CPU 1 to CPUs 3:
NMI backtrace for cpu 3
CPU: 3 PID: 31 Comm: kworker/3:0 Not tainted 6.7.0-rc6-next-20231222-10300-g8b07e3811bc7 #17
Hardware name: Pine64 RK3566 Quartz64-B Board (DT)
Workqueue: events hci_uart_write_work [hci_uart]
pstate: 80400009 (Nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : _state+0x2c/0x138
lr : pl330_start_thread.isra.0+0x2e0/0x32c
sp : ffffffc08157bb20
pmr_save: 00000060
x29: ffffffc08157bb20 x28: 0000000000000000 x27: 0000000000000060
x26: ffffffc080c4e658 x25: 0000000000000060 x24: 0000000001a20000
x23: ffffff8001555000 x22: ffffffc081960020 x21: ffffff800161b068
x20: 0000000000000000 x19: ffffff800161b050 x18: ffffffffffffffff
x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000001
x14: 0000000000000004 x13: 0000000000000009 x12: 0000000000000005
x11: 0000000000000027 x10: 000000000000002b x9 : 0000000000000032
x8 : ffffffc08154521d x7 : 0000000000000005 x6 : 0000000000000010
x5 : 0000000000000001 x4 : ffffffc081960d04 x3 : ffffff800161b280
x2 : ffffff800161b050 x1 : 0000000000204000 x0 : 0000000000000108
Call trace:
_state+0x2c/0x138
pl330_tasklet+0x1f8/0x818
pl330_issue_pending+0x150/0x178
serial8250_tx_dma+0x150/0x21c
serial8250_start_tx+0x9c/0x1c0
__uart_start+0x74/0xfc
uart_write+0xfc/0x2f0
ttyport_write_buf+0x4c/0x90
serdev_device_write_buf+0x24/0x38
hci_uart_write_work+0x54/0x164 [hci_uart]
process_one_work+0x13c/0x2bc
worker_thread+0x2a0/0x52c
kthread+0xe0/0xe4
ret_from_fork+0x10/0x20
[1] https://lore.kernel.org/linux-arm-kernel/[email protected]/
Hello.
> On Thu, Dec 21, 2023 at 10:00:26PM +0530, Vinod Koul wrote:
>
> This seems to cause a stall on my Quartz 64 model B (RK3566) once
> Bluetooth over UART is initialized, when combined with a patch of mine
> that enables DMA on UARTs [1]. Reverting this patch gets everything
> running again.
>
> The following are RCU stalls detected, followed by stack traces produced
> with pseudo-NMI. Without pseudo-NMIs no stack traces are produced.
>
> rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
> rcu: 0-...0: (0 ticks this GP) idle=80fc/1/0x4000000000000000
> softirq=693/693 fqs=31498
> rcu: 3-...0: (3 ticks this GP) idle=2b44/1/0x4000000000000000
> softirq=553/556 fqs=31498
> rcu: (detected by 1, t=162830 jiffies, g=-307, q=32 ncpus=4)
> Sending NMI from CPU 1 to CPUs 0:
> NMI backtrace for cpu 0
> CPU: 0 PID: 1200 Comm: (udev-worker) Not tainted 6.7.0-rc6-next-
> 20231222-10300-g8b07e3811bc7 #17
> Hardware name: Pine64 RK3566 Quartz64-B Board (DT)
> pstate: 00400009 (nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> pc : queued_spin_lock_slowpath+0x50/0x330
> lr : pl330_irq_handler+0x2f8/0x5a0
> sp : ffffffc080003ec0
> pmr_save: 00000060
> x29: ffffffc080003ec0 x28: ffffff80017c7000 x27: ffffff8001a58d80
> x26: 0000000000000060 x25: ffffff80017d0338 x24: ffffff800161ae38
> x23: ffffff8001597c00 x22: ffffffc081960000 x21: 0000000000000000
> x20: ffffff800161ac80 x19: ffffff80010c5180 x18: 0000000000000000
> x17: ffffffc06e724000 x16: ffffffc080000000 x15: 0000000000000000
> x14: 0000000000000000 x13: ffffff80042f102f x12: ffffffc083ad3cc4
> x11: 0000000000000040 x10: ffffff800022a0a8 x9 : ffffff800022a0a0
> x8 : ffffff8000400270 x7 : 0000000000000000 x6 : 0000000000000000
> x5 : ffffff8000400248 x4 : ffffffc06e724000 x3 : ffffffc080003fa0
> x2 : 0000000000000000 x1 : 0000000000000001 x0 : ffffff800161ae38
> Call trace:
> queued_spin_lock_slowpath+0x50/0x330
> __handle_irq_event_percpu+0x38/0x16c
> handle_irq_event+0x44/0xf8
> handle_fasteoi_irq+0xb0/0x28c
> generic_handle_domain_irq+0x2c/0x44
> gic_handle_irq+0x10c/0x240
> call_on_irq_stack+0x24/0x4c
> do_interrupt_handler+0x80/0x8c
> el1_interrupt+0x44/0x98
> el1h_64_irq_handler+0x18/0x24
> el1h_64_irq+0x78/0x7c
> __d_rehash+0x0/0x94
> d_add+0x40/0x80
> simple_lookup+0x4c/0x78
> path_openat+0x5ec/0xed0
> do_filp_open+0x80/0x12c
> do_sys_openat2+0xb4/0xe8
> __arm64_sys_openat+0x64/0xa4
> invoke_syscall+0x48/0x114
> el0_svc_common.constprop.0+0x40/0xe0
> do_el0_svc+0x1c/0x28
> el0_svc+0x34/0xd4
> el0t_64_sync_handler+0x100/0x12c
> el0t_64_sync+0x1a4/0x1a8
> Sending NMI from CPU 1 to CPUs 3:
> NMI backtrace for cpu 3
> CPU: 3 PID: 31 Comm: kworker/3:0 Not tainted 6.7.0-rc6-next-20231222-
> 10300-g8b07e3811bc7 #17
> Hardware name: Pine64 RK3566 Quartz64-B Board (DT)
> Workqueue: events hci_uart_write_work [hci_uart]
> pstate: 80400009 (Nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> pc : _state+0x2c/0x138
> lr : pl330_start_thread.isra.0+0x2e0/0x32c
> sp : ffffffc08157bb20
> pmr_save: 00000060
> x29: ffffffc08157bb20 x28: 0000000000000000 x27: 0000000000000060
> x26: ffffffc080c4e658 x25: 0000000000000060 x24: 0000000001a20000
> x23: ffffff8001555000 x22: ffffffc081960020 x21: ffffff800161b068
> x20: 0000000000000000 x19: ffffff800161b050 x18: ffffffffffffffff
> x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000001
> x14: 0000000000000004 x13: 0000000000000009 x12: 0000000000000005
> x11: 0000000000000027 x10: 000000000000002b x9 : 0000000000000032
> x8 : ffffffc08154521d x7 : 0000000000000005 x6 : 0000000000000010
> x5 : 0000000000000001 x4 : ffffffc081960d04 x3 : ffffff800161b280
> x2 : ffffff800161b050 x1 : 0000000000204000 x0 : 0000000000000108
> Call trace:
> _state+0x2c/0x138
> pl330_tasklet+0x1f8/0x818
> pl330_issue_pending+0x150/0x178
> serial8250_tx_dma+0x150/0x21c
> serial8250_start_tx+0x9c/0x1c0
> __uart_start+0x74/0xfc
> uart_write+0xfc/0x2f0
> ttyport_write_buf+0x4c/0x90
> serdev_device_write_buf+0x24/0x38
> hci_uart_write_work+0x54/0x164 [hci_uart]
> process_one_work+0x13c/0x2bc
> worker_thread+0x2a0/0x52c
> kthread+0xe0/0xe4
> ret_from_fork+0x10/0x20
I think the dma channel already passed WFP state.
The errata[1] says that
4. The driver polls the status of channel 0 until it observes that
it has entered the "Waiting for peripheral" state
5. The driver enables the peripheral to allow it to issue DMA requests
When I review 8250_dma.c, I think serial8250_do_prepare_tx_dma() of
serial8250_tx_dma() would enable peripheral to allow issue DMA requests.
serial8250_do_prepare_tx_dma(p) performs before dma_async_issue_pending()
It means that serial8250_tx_dma() has reversed sequence step 4 and 5 for
pl330.
But I'm not sure if the errata suggests normal slave driver dma usage
sequence or not.
If it is abnormal sequence, I think it should be handled by quirk
[1]: https://developer.arm.com/documentation/genc008428/latest
On Wed, Dec 27, 2023 at 10:10 AM bumyong.lee <[email protected]> wrote:
>
> Hello.
>
> > On Thu, Dec 21, 2023 at 10:00:26PM +0530, Vinod Koul wrote:
> >
> > This seems to cause a stall on my Quartz 64 model B (RK3566) once
> > Bluetooth over UART is initialized, when combined with a patch of mine
> > that enables DMA on UARTs [1]. Reverting this patch gets everything
> > running again.
> >
> > The following are RCU stalls detected, followed by stack traces produced
> > with pseudo-NMI. Without pseudo-NMIs no stack traces are produced.
> >
> > rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
> > rcu: 0-...0: (0 ticks this GP) idle=80fc/1/0x4000000000000000
> > softirq=693/693 fqs=31498
> > rcu: 3-...0: (3 ticks this GP) idle=2b44/1/0x4000000000000000
> > softirq=553/556 fqs=31498
> > rcu: (detected by 1, t=162830 jiffies, g=-307, q=32 ncpus=4)
> > Sending NMI from CPU 1 to CPUs 0:
> > NMI backtrace for cpu 0
> > CPU: 0 PID: 1200 Comm: (udev-worker) Not tainted 6.7.0-rc6-next-
> > 20231222-10300-g8b07e3811bc7 #17
> > Hardware name: Pine64 RK3566 Quartz64-B Board (DT)
> > pstate: 00400009 (nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> > pc : queued_spin_lock_slowpath+0x50/0x330
> > lr : pl330_irq_handler+0x2f8/0x5a0
> > sp : ffffffc080003ec0
> > pmr_save: 00000060
> > x29: ffffffc080003ec0 x28: ffffff80017c7000 x27: ffffff8001a58d80
> > x26: 0000000000000060 x25: ffffff80017d0338 x24: ffffff800161ae38
> > x23: ffffff8001597c00 x22: ffffffc081960000 x21: 0000000000000000
> > x20: ffffff800161ac80 x19: ffffff80010c5180 x18: 0000000000000000
> > x17: ffffffc06e724000 x16: ffffffc080000000 x15: 0000000000000000
> > x14: 0000000000000000 x13: ffffff80042f102f x12: ffffffc083ad3cc4
> > x11: 0000000000000040 x10: ffffff800022a0a8 x9 : ffffff800022a0a0
> > x8 : ffffff8000400270 x7 : 0000000000000000 x6 : 0000000000000000
> > x5 : ffffff8000400248 x4 : ffffffc06e724000 x3 : ffffffc080003fa0
> > x2 : 0000000000000000 x1 : 0000000000000001 x0 : ffffff800161ae38
> > Call trace:
> > queued_spin_lock_slowpath+0x50/0x330
> > __handle_irq_event_percpu+0x38/0x16c
> > handle_irq_event+0x44/0xf8
> > handle_fasteoi_irq+0xb0/0x28c
> > generic_handle_domain_irq+0x2c/0x44
> > gic_handle_irq+0x10c/0x240
> > call_on_irq_stack+0x24/0x4c
> > do_interrupt_handler+0x80/0x8c
> > el1_interrupt+0x44/0x98
> > el1h_64_irq_handler+0x18/0x24
> > el1h_64_irq+0x78/0x7c
> > __d_rehash+0x0/0x94
> > d_add+0x40/0x80
> > simple_lookup+0x4c/0x78
> > path_openat+0x5ec/0xed0
> > do_filp_open+0x80/0x12c
> > do_sys_openat2+0xb4/0xe8
> > __arm64_sys_openat+0x64/0xa4
> > invoke_syscall+0x48/0x114
> > el0_svc_common.constprop.0+0x40/0xe0
> > do_el0_svc+0x1c/0x28
> > el0_svc+0x34/0xd4
> > el0t_64_sync_handler+0x100/0x12c
> > el0t_64_sync+0x1a4/0x1a8
> > Sending NMI from CPU 1 to CPUs 3:
> > NMI backtrace for cpu 3
> > CPU: 3 PID: 31 Comm: kworker/3:0 Not tainted 6.7.0-rc6-next-20231222-
> > 10300-g8b07e3811bc7 #17
> > Hardware name: Pine64 RK3566 Quartz64-B Board (DT)
> > Workqueue: events hci_uart_write_work [hci_uart]
> > pstate: 80400009 (Nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> > pc : _state+0x2c/0x138
> > lr : pl330_start_thread.isra.0+0x2e0/0x32c
> > sp : ffffffc08157bb20
> > pmr_save: 00000060
> > x29: ffffffc08157bb20 x28: 0000000000000000 x27: 0000000000000060
> > x26: ffffffc080c4e658 x25: 0000000000000060 x24: 0000000001a20000
> > x23: ffffff8001555000 x22: ffffffc081960020 x21: ffffff800161b068
> > x20: 0000000000000000 x19: ffffff800161b050 x18: ffffffffffffffff
> > x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000001
> > x14: 0000000000000004 x13: 0000000000000009 x12: 0000000000000005
> > x11: 0000000000000027 x10: 000000000000002b x9 : 0000000000000032
> > x8 : ffffffc08154521d x7 : 0000000000000005 x6 : 0000000000000010
> > x5 : 0000000000000001 x4 : ffffffc081960d04 x3 : ffffff800161b280
> > x2 : ffffff800161b050 x1 : 0000000000204000 x0 : 0000000000000108
> > Call trace:
> > _state+0x2c/0x138
> > pl330_tasklet+0x1f8/0x818
> > pl330_issue_pending+0x150/0x178
> > serial8250_tx_dma+0x150/0x21c
> > serial8250_start_tx+0x9c/0x1c0
> > __uart_start+0x74/0xfc
> > uart_write+0xfc/0x2f0
> > ttyport_write_buf+0x4c/0x90
> > serdev_device_write_buf+0x24/0x38
> > hci_uart_write_work+0x54/0x164 [hci_uart]
> > process_one_work+0x13c/0x2bc
> > worker_thread+0x2a0/0x52c
> > kthread+0xe0/0xe4
> > ret_from_fork+0x10/0x20
> I think the dma channel already passed WFP state.
>
> The errata[1] says that
> 4. The driver polls the status of channel 0 until it observes that
> it has entered the "Waiting for peripheral" state
> 5. The driver enables the peripheral to allow it to issue DMA requests
>
> When I review 8250_dma.c, I think serial8250_do_prepare_tx_dma() of
> serial8250_tx_dma() would enable peripheral to allow issue DMA requests.
> serial8250_do_prepare_tx_dma(p) performs before dma_async_issue_pending()
> It means that serial8250_tx_dma() has reversed sequence step 4 and 5 for
> pl330.
serial8250_prepare_tx_dma() on Rockchip doesn't do anything. The callback
called in serial8250_prepare_tx_dma() simply isn't set.
For this hardware the DMA request will be asserted when the TX FIFO is
empty, or when the RX FIFO has data, depending on the direction.
From the DesignWare UART release notes:
In mode 0 (single shot DMA), the dma_tx_req_n signal goes active low
under the following conditions:
- When the Transmitter Holding Register is empty in non-FIFO mode
- When the transmitter FIFO is empty in FIFO mode with Programmable THRE
interrupt mode disabled
- When the transmitter FIFO is at or below the programmed threshold with
Programmable THRE interrupt mode enabled
Mode 1 (continuous DMA) is the same, minus the first condition.
> But I'm not sure if the errata suggests normal slave driver dma usage
> sequence or not.
I'm not sure what you mean by this.
> If it is abnormal sequence, I think it should be handled by quirk
A quirk on which device? The DMA controller?
ChenYu
> [1]: https://developer.arm.com/documentation/genc008428/latest
>
Hello.
> > > This seems to cause a stall on my Quartz 64 model B (RK3566) once
> > > Bluetooth over UART is initialized, when combined with a patch of
> > > mine that enables DMA on UARTs [1]. Reverting this patch gets
> > > everything running again.
> > >
> > > The following are RCU stalls detected, followed by stack traces
> > > produced with pseudo-NMI. Without pseudo-NMIs no stack traces are
> produced.
> > >
> > > rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
> > > rcu: 0-...0: (0 ticks this GP) idle=80fc/1/0x4000000000000000
> > > softirq=693/693 fqs=31498
> > > rcu: 3-...0: (3 ticks this GP) idle=2b44/1/0x4000000000000000
> > > softirq=553/556 fqs=31498
> > > rcu: (detected by 1, t=162830 jiffies, g=-307, q=32 ncpus=4)
> > > Sending NMI from CPU 1 to CPUs 0:
> > > NMI backtrace for cpu 0
> > > CPU: 0 PID: 1200 Comm: (udev-worker) Not tainted 6.7.0-rc6-next-
> > > 20231222-10300-g8b07e3811bc7 #17
> > > Hardware name: Pine64 RK3566 Quartz64-B Board (DT)
> > > pstate: 00400009 (nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> > > pc : queued_spin_lock_slowpath+0x50/0x330
> > > lr : pl330_irq_handler+0x2f8/0x5a0
> > > sp : ffffffc080003ec0
> > > pmr_save: 00000060
> > > x29: ffffffc080003ec0 x28: ffffff80017c7000 x27: ffffff8001a58d80
> > > x26: 0000000000000060 x25: ffffff80017d0338 x24: ffffff800161ae38
> > > x23: ffffff8001597c00 x22: ffffffc081960000 x21: 0000000000000000
> > > x20: ffffff800161ac80 x19: ffffff80010c5180 x18: 0000000000000000
> > > x17: ffffffc06e724000 x16: ffffffc080000000 x15: 0000000000000000
> > > x14: 0000000000000000 x13: ffffff80042f102f x12: ffffffc083ad3cc4
> > > x11: 0000000000000040 x10: ffffff800022a0a8 x9 : ffffff800022a0a0
> > > x8 : ffffff8000400270 x7 : 0000000000000000 x6 : 0000000000000000
> > > x5 : ffffff8000400248 x4 : ffffffc06e724000 x3 : ffffffc080003fa0
> > > x2 : 0000000000000000 x1 : 0000000000000001 x0 : ffffff800161ae38
> > > Call trace:
> > > queued_spin_lock_slowpath+0x50/0x330
> > > __handle_irq_event_percpu+0x38/0x16c
> > > handle_irq_event+0x44/0xf8
> > > handle_fasteoi_irq+0xb0/0x28c
> > > generic_handle_domain_irq+0x2c/0x44
> > > gic_handle_irq+0x10c/0x240
> > > call_on_irq_stack+0x24/0x4c
> > > do_interrupt_handler+0x80/0x8c
> > > el1_interrupt+0x44/0x98
> > > el1h_64_irq_handler+0x18/0x24
> > > el1h_64_irq+0x78/0x7c
> > > __d_rehash+0x0/0x94
> > > d_add+0x40/0x80
> > > simple_lookup+0x4c/0x78
> > > path_openat+0x5ec/0xed0
> > > do_filp_open+0x80/0x12c
> > > do_sys_openat2+0xb4/0xe8
> > > __arm64_sys_openat+0x64/0xa4
> > > invoke_syscall+0x48/0x114
> > > el0_svc_common.constprop.0+0x40/0xe0
> > > do_el0_svc+0x1c/0x28
> > > el0_svc+0x34/0xd4
> > > el0t_64_sync_handler+0x100/0x12c
> > > el0t_64_sync+0x1a4/0x1a8
> > > Sending NMI from CPU 1 to CPUs 3:
> > > NMI backtrace for cpu 3
> > > CPU: 3 PID: 31 Comm: kworker/3:0 Not tainted
> > > 6.7.0-rc6-next-20231222-
> > > 10300-g8b07e3811bc7 #17
> > > Hardware name: Pine64 RK3566 Quartz64-B Board (DT)
> > > Workqueue: events hci_uart_write_work [hci_uart]
> > > pstate: 80400009 (Nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> > > pc : _state+0x2c/0x138
> > > lr : pl330_start_thread.isra.0+0x2e0/0x32c
> > > sp : ffffffc08157bb20
> > > pmr_save: 00000060
> > > x29: ffffffc08157bb20 x28: 0000000000000000 x27: 0000000000000060
> > > x26: ffffffc080c4e658 x25: 0000000000000060 x24: 0000000001a20000
> > > x23: ffffff8001555000 x22: ffffffc081960020 x21: ffffff800161b068
> > > x20: 0000000000000000 x19: ffffff800161b050 x18: ffffffffffffffff
> > > x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000001
> > > x14: 0000000000000004 x13: 0000000000000009 x12: 0000000000000005
> > > x11: 0000000000000027 x10: 000000000000002b x9 : 0000000000000032
> > > x8 : ffffffc08154521d x7 : 0000000000000005 x6 : 0000000000000010
> > > x5 : 0000000000000001 x4 : ffffffc081960d04 x3 : ffffff800161b280
> > > x2 : ffffff800161b050 x1 : 0000000000204000 x0 : 0000000000000108
> > > Call trace:
> > > _state+0x2c/0x138
> > > pl330_tasklet+0x1f8/0x818
> > > pl330_issue_pending+0x150/0x178
> > > serial8250_tx_dma+0x150/0x21c
> > > serial8250_start_tx+0x9c/0x1c0
> > > __uart_start+0x74/0xfc
> > > uart_write+0xfc/0x2f0
> > > ttyport_write_buf+0x4c/0x90
> > > serdev_device_write_buf+0x24/0x38
> > > hci_uart_write_work+0x54/0x164 [hci_uart]
> > > process_one_work+0x13c/0x2bc
> > > worker_thread+0x2a0/0x52c
> > > kthread+0xe0/0xe4
> > > ret_from_fork+0x10/0x20
> > I think the dma channel already passed WFP state.
> >
> > The errata[1] says that
> > 4. The driver polls the status of channel 0 until it observes that
> > it has entered the "Waiting for peripheral" state 5. The driver
> > enables the peripheral to allow it to issue DMA requests
> >
> > When I review 8250_dma.c, I think serial8250_do_prepare_tx_dma() of
> > serial8250_tx_dma() would enable peripheral to allow issue DMA requests.
> > serial8250_do_prepare_tx_dma(p) performs before
> > dma_async_issue_pending() It means that serial8250_tx_dma() has
> > reversed sequence step 4 and 5 for pl330.
>
> serial8250_prepare_tx_dma() on Rockchip doesn't do anything. The callback
> called in serial8250_prepare_tx_dma() simply isn't set.
>
> For this hardware the DMA request will be asserted when the TX FIFO is
> empty, or when the RX FIFO has data, depending on the direction.
>
> From the DesignWare UART release notes:
>
> In mode 0 (single shot DMA), the dma_tx_req_n signal goes active low under
> the following conditions:
> - When the Transmitter Holding Register is empty in non-FIFO mode
> - When the transmitter FIFO is empty in FIFO mode with Programmable THRE
> interrupt mode disabled
> - When the transmitter FIFO is at or below the programmed threshold with
> Programmable THRE interrupt mode enabled
>
> Mode 1 (continuous DMA) is the same, minus the first condition.
>
> > But I'm not sure if the errata suggests normal slave driver dma usage
> > sequence or not.
>
> I'm not sure what you mean by this.
Errata suggests that Slave device must not send DMA request until PL330
enters WFP state. so I made wait until WFP state in _trigger().
If slave device didn't send DMA request, I think it is okay to wait
until pl330 changes their state to WFP in _trigger().
According to your description,
In order for serial8250 to follow the sequence of the errata, I guess
serial8250 should change mode from fifo mode to DMA mode after doing
issue_pending() not to send DMA request until WFP.
If case of samsung_tty.c, it changes dma mode after doing issue_pending()
s3c64xx_start_rx_dma(ourport); // do issue_pending()
if (ourport->rx_mode == S3C24XX_RX_PIO)
enable_rx_dma(ourport); // change uart mode to DMA mode
// to send DMA request
But I'm not sure if most of slave devices send DMA request
before doing issue_pending().
>
> > If it is abnormal sequence, I think it should be handled by quirk
>
> A quirk on which device? The DMA controller?
If the sequence of requesting DMA before doing issue_pending() is common,
The errata suggests wrong sequence for DMA usage in linux.
So, it should be applied to pl330 not to wait WFP _trigger() when quirk
is not enabled.
Unfortunately, I couldn't find more good way when I check pl330 trm
and errata.
Best regards