2002-06-14 10:26:08

by Zwane Mwaikambo

Subject: [PATCH][2.5] 2.5.21 deadlocks on UP (SMP kernel) w/ IOAPIC

Hi Ingo,
The balance_irq/move code in 2.5.21 currently deadlocks on my UP
box due to the following;

balance_irq() {
entry = irq_balance + irq;
[...]
rdtscl(random_number);
random_number &= 1;
[...]
entry->cpu = move(entry->cpu, allowed_mask, now, random_number);
}

move() {
do {
[...]
cpu=0x0 curr_cpu=0x1 smp_num_cpus=0x1 allowed_mask=0x1 direction=0x0
[...]
} while (!IRQ_ALLOWED(cpu,allowed_mask) ||
(search_idle && !IDLE_ENOUGH(cpu,now)));


}

Please apply

Regards,
Zwane Mwaikambo

Diffed against 2.5.21
--- linux-2.5.19/arch/i386/kernel/io_apic.c.orig Fri Jun 14 11:53:15 2002
+++ linux-2.5.19/arch/i386/kernel/io_apic.c Fri Jun 14 11:55:01 2002
@@ -248,13 +248,14 @@
static inline void balance_irq(int irq)
{
#if CONFIG_SMP
- irq_balance_t *entry = irq_balance + irq;
+ irq_balance_t *entry;
unsigned long now = jiffies;

- if (unlikely(entry->timestamp != now)) {
+ if ((entry->timestamp != now) && (smp_num_cpus > 1)) {
unsigned long allowed_mask;
int random_number;

+ entry = irq_balance + irq;
rdtscl(random_number);
random_number &= 1;


--
http://function.linuxpower.ca

2002-06-14 11:13:40

by Ingo Molnar

Subject: Re: [PATCH][2.5] 2.5.21 deadlocks on UP (SMP kernel) w/ IOAPIC

On Fri, 14 Jun 2002, Zwane Mwaikambo wrote:

> Hi Ingo,
> The balance_irq/move code in 2.5.21 currently deadlocks on my UP
> box due to the following;

> Please apply

looks good to me.

Ingo

2002-06-14 15:43:35

by Zwane Mwaikambo

Subject: Re: [PATCH][2.5] 2.5.21 deadlocks on UP (SMP kernel) w/ IOAPIC

Patch updated to fix a thinko spotted out by Dan Aloni

--- linux-2.5.19/arch/i386/kernel/io_apic.c.orig Fri Jun 14 17:43:20 2002
+++ linux-2.5.19/arch/i386/kernel/io_apic.c Fri Jun 14 17:42:23 2002
@@ -251,7 +251,7 @@
irq_balance_t *entry = irq_balance + irq;
unsigned long now = jiffies;

- if (unlikely(entry->timestamp != now)) {
+ if ((entry->timestamp != now) && (smp_num_cpus > 1)) {
unsigned long allowed_mask;
int random_number;


Linus, please apply.

Regards,
Zwane Mwaikambo
--
http://function.linuxpower.ca

2002-06-15 08:15:13

by Zwane Mwaikambo

Subject: [PATCH][2.5] 2.5.21 deadlocks on UP (SMP kernel) w/ IOAPIC (take 2)

With the help of Andrew Morton we tracked down the original breakage to
the irq_balance_t declaration which assumes there is a cpu#1

typedef struct {
unsigned int cpu;
unsigned long timestamp;
} ____cacheline_aligned irq_balance_t;

static irq_balance_t irq_balance[NR_IRQS] __cacheline_aligned
= { [ 0 ... NR_IRQS-1 ] = { 1, 0 } };

against 2.5.21, test booted on UP(SMP) w/ IOAPIC and SMP

--- linux-2.5.19/arch/i386/kernel/io_apic.c.orig Fri Jun 14 17:43:20 2002
+++ linux-2.5.19/arch/i386/kernel/io_apic.c Sat Jun 15 10:07:11 2002
@@ -207,7 +207,7 @@
} ____cacheline_aligned irq_balance_t;

static irq_balance_t irq_balance[NR_IRQS] __cacheline_aligned
- = { [ 0 ... NR_IRQS-1 ] = { 1, 0 } };
+ = { [ 0 ... NR_IRQS-1 ] = { 0, 0 } };

extern unsigned long irq_affinity [NR_IRQS];

Linus, i promise this is the last one ;)

Regards,
Zwane Mwaikambo

--
http://function.linuxpower.ca