2020-10-28 22:31:49

by Qian Cai

[permalink] [raw]
Subject: [PATCH] powerpc/eeh_cache: Fix a possible debugfs deadlock

Lockdep complains that a possible deadlock below in
eeh_addr_cache_show() because it is acquiring a lock with IRQ enabled,
but eeh_addr_cache_insert_dev() needs to acquire the same lock with IRQ
disabled. Let's just make eeh_addr_cache_show() acquire the lock with
IRQ disabled as well.

CPU0 CPU1
---- ----
lock(&pci_io_addr_cache_root.piar_lock);
local_irq_disable();
lock(&tp->lock);
lock(&pci_io_addr_cache_root.piar_lock);
<Interrupt>
lock(&tp->lock);

*** DEADLOCK ***

lock_acquire+0x140/0x5f0
_raw_spin_lock_irqsave+0x64/0xb0
eeh_addr_cache_insert_dev+0x48/0x390
eeh_probe_device+0xb8/0x1a0
pnv_pcibios_bus_add_device+0x3c/0x80
pcibios_bus_add_device+0x118/0x290
pci_bus_add_device+0x28/0xe0
pci_bus_add_devices+0x54/0xb0
pcibios_init+0xc4/0x124
do_one_initcall+0xac/0x528
kernel_init_freeable+0x35c/0x3fc
kernel_init+0x24/0x148
ret_from_kernel_thread+0x5c/0x80

lock_acquire+0x140/0x5f0
_raw_spin_lock+0x4c/0x70
eeh_addr_cache_show+0x38/0x110
seq_read+0x1a0/0x660
vfs_read+0xc8/0x1f0
ksys_read+0x74/0x130
system_call_exception+0xf8/0x1d0
system_call_common+0xe8/0x218

Fixes: 5ca85ae6318d ("powerpc/eeh_cache: Add a way to dump the EEH address cache")
Signed-off-by: Qian Cai <[email protected]>
---
arch/powerpc/kernel/eeh_cache.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/arch/powerpc/kernel/eeh_cache.c b/arch/powerpc/kernel/eeh_cache.c
index 6b50bf15d8c1..bf3270426d82 100644
--- a/arch/powerpc/kernel/eeh_cache.c
+++ b/arch/powerpc/kernel/eeh_cache.c
@@ -264,8 +264,9 @@ static int eeh_addr_cache_show(struct seq_file *s, void *v)
{
struct pci_io_addr_range *piar;
struct rb_node *n;
+ unsigned long flags;

- spin_lock(&pci_io_addr_cache_root.piar_lock);
+ spin_lock_irqsave(&pci_io_addr_cache_root.piar_lock, flags);
for (n = rb_first(&pci_io_addr_cache_root.rb_root); n; n = rb_next(n)) {
piar = rb_entry(n, struct pci_io_addr_range, rb_node);

@@ -273,7 +274,7 @@ static int eeh_addr_cache_show(struct seq_file *s, void *v)
(piar->flags & IORESOURCE_IO) ? "i/o" : "mem",
&piar->addr_lo, &piar->addr_hi, pci_name(piar->pcidev));
}
- spin_unlock(&pci_io_addr_cache_root.piar_lock);
+ spin_unlock_irqrestore(&pci_io_addr_cache_root.piar_lock, flags);

return 0;
}
--
2.28.0


2020-10-29 09:00:38

by Oliver O'Halloran

[permalink] [raw]
Subject: Re: [PATCH] powerpc/eeh_cache: Fix a possible debugfs deadlock

On Thu, Oct 29, 2020 at 2:27 AM Qian Cai <[email protected]> wrote:
>
> Lockdep complains that a possible deadlock below in
> eeh_addr_cache_show() because it is acquiring a lock with IRQ enabled,
> but eeh_addr_cache_insert_dev() needs to acquire the same lock with IRQ
> disabled. Let's just make eeh_addr_cache_show() acquire the lock with
> IRQ disabled as well.
>
> CPU0 CPU1
> ---- ----
> lock(&pci_io_addr_cache_root.piar_lock);
> local_irq_disable();
> lock(&tp->lock);
> lock(&pci_io_addr_cache_root.piar_lock);
> <Interrupt>
> lock(&tp->lock);
>
> *** DEADLOCK ***
>
> lock_acquire+0x140/0x5f0
> _raw_spin_lock_irqsave+0x64/0xb0
> eeh_addr_cache_insert_dev+0x48/0x390
> eeh_probe_device+0xb8/0x1a0
> pnv_pcibios_bus_add_device+0x3c/0x80
> pcibios_bus_add_device+0x118/0x290
> pci_bus_add_device+0x28/0xe0
> pci_bus_add_devices+0x54/0xb0
> pcibios_init+0xc4/0x124
> do_one_initcall+0xac/0x528
> kernel_init_freeable+0x35c/0x3fc
> kernel_init+0x24/0x148
> ret_from_kernel_thread+0x5c/0x80
>
> lock_acquire+0x140/0x5f0
> _raw_spin_lock+0x4c/0x70
> eeh_addr_cache_show+0x38/0x110
> seq_read+0x1a0/0x660
> vfs_read+0xc8/0x1f0
> ksys_read+0x74/0x130
> system_call_exception+0xf8/0x1d0
> system_call_common+0xe8/0x218
>
> Fixes: 5ca85ae6318d ("powerpc/eeh_cache: Add a way to dump the EEH address cache")
> Signed-off-by: Qian Cai <[email protected]>

Good catch,

Reviewed-by: Oliver O'Halloran <[email protected]>

2020-11-04 11:41:02

by Michael Ellerman

[permalink] [raw]
Subject: Re: [PATCH] powerpc/eeh_cache: Fix a possible debugfs deadlock

On Wed, 28 Oct 2020 11:27:17 -0400, Qian Cai wrote:
> Lockdep complains that a possible deadlock below in
> eeh_addr_cache_show() because it is acquiring a lock with IRQ enabled,
> but eeh_addr_cache_insert_dev() needs to acquire the same lock with IRQ
> disabled. Let's just make eeh_addr_cache_show() acquire the lock with
> IRQ disabled as well.
>
> CPU0 CPU1
> ---- ----
> lock(&pci_io_addr_cache_root.piar_lock);
> local_irq_disable();
> lock(&tp->lock);
> lock(&pci_io_addr_cache_root.piar_lock);
> <Interrupt>
> lock(&tp->lock);
>
> [...]

Applied to powerpc/fixes.

[1/1] powerpc/eeh_cache: Fix a possible debugfs deadlock
https://git.kernel.org/powerpc/c/fd552e0542b4532483289cce48fdbd27b692984b

cheers