The addition of the generic affinity autoselector has introduced
a regression in SN2 Altix.
In the past, request_irq()'s call to select_smp_affinity() was
essentially a noop.
#ifdef CONFIG_AUTO_IRQ_AFFINITY
extern int select_smp_affinity(unsigned int irq);
#else
static inline int select_smp_affinity(unsigned int irq)
{
return 1;
}
Now changes have created do_irq_select_affinity(), which is
executed by a call to request_irq().
request_irq()
__setup_irq()
do_irq_select_affinity()
do_irq_select_affinity() will call the set_affinity() routine,
with a default mask. On SN2 (and possibly all ia64 platforms),
this will retarget all device interrupts to the same CPU, most
likely CPU0.
#ifndef CONFIG_AUTO_IRQ_AFFINITY
/*
* Generic version of the affinity autoselector.
*/
int do_irq_select_affinity(unsigned int irq, struct irq_desc *desc)
{
cpumask_t mask;
if (!irq_can_set_affinity(irq))
return 0;
cpus_and(mask, cpu_online_map, irq_default_affinity);
/*
* Preserve an userspace affinity setup, but make sure that
* one of the targets is online.
*/
if (desc->status & (IRQ_AFFINITY_SET | IRQ_NO_BALANCING)) {
if (cpus_intersects(desc->affinity, cpu_online_map))
mask = desc->affinity;
else
desc->status &= ~IRQ_AFFINITY_SET;
}
desc->affinity = mask;
desc->chip->set_affinity(irq, mask);
return 0;
}
This set_affinity call undoes all the performance related interrupt
targetting that the SN2 PROM did at boot.
I'm looking at a patch that makes use of the new IRQ_AFFINITY_SET
flag in the irq_desc to get around this.
Is this an issue for all ia64 platforms? Or does irq balancing
factor in here? IRQ balancing is disabled on SN2.
John Keller
On Fri, Dec 5, 2008 at 8:40 AM, John Keller <[email protected]> wrote:
> The addition of the generic affinity autoselector has introduced
> a regression in SN2 Altix.
>
> In the past, request_irq()'s call to select_smp_affinity() was
> essentially a noop.
>
> #ifdef CONFIG_AUTO_IRQ_AFFINITY
> extern int select_smp_affinity(unsigned int irq);
> #else
> static inline int select_smp_affinity(unsigned int irq)
> {
> return 1;
> }
>
>
> Now changes have created do_irq_select_affinity(), which is
> executed by a call to request_irq().
>
> request_irq()
> __setup_irq()
> do_irq_select_affinity()
>
> do_irq_select_affinity() will call the set_affinity() routine,
> with a default mask. On SN2 (and possibly all ia64 platforms),
> this will retarget all device interrupts to the same CPU, most
> likely CPU0.
> I'm looking at a patch that makes use of the new IRQ_AFFINITY_SET
> flag in the irq_desc to get around this.
>
that bit only be set in irq_set_affinity(),
arch/mips/kernel/cevt-bcm1480.c: irq_set_affinity(irq,
cpumask_of_cpu(cpu));
arch/mips/kernel/cevt-sb1250.c: irq_set_affinity(irq, cpumask_of_cpu(cpu));
arch/sparc64/kernel/of_device.c: irq_set_affinity(irq,
numa_mask);
arch/sparc64/kernel/pci_msi.c: irq_set_affinity(irq, numa_mask);
arch/x86/kernel/hpet.c: irq_set_affinity(hdev->irq,
cpumask_of_cpu(hdev->cpu));
arch/x86/kernel/hpet.c: irq_set_affinity(dev->irq, cpumask_of_cpu(dev->cpu));
drivers/xen/events.c: irq_set_affinity(irq, cpumask_of_cpu(0));
kernel/irq/proc.c: irq_set_affinity(irq, new_value);
kernel/time/tick-common.c: irq_set_affinity(newdev->irq, *cpumask);
so you may need to call it in your arch code. like
setup_IO_APIC_irq(), create_irq()...
YH
>
> On Fri, Dec 5, 2008 at 8:40 AM, John Keller <[email protected]> wrote:
> > The addition of the generic affinity autoselector has introduced
> > a regression in SN2 Altix.
> >
> > In the past, request_irq()'s call to select_smp_affinity() was
> > essentially a noop.
> >
> > #ifdef CONFIG_AUTO_IRQ_AFFINITY
> > extern int select_smp_affinity(unsigned int irq);
> > #else
> > static inline int select_smp_affinity(unsigned int irq)
> > {
> > return 1;
> > }
> >
> >
> > Now changes have created do_irq_select_affinity(), which is
> > executed by a call to request_irq().
> >
> > request_irq()
> > __setup_irq()
> > do_irq_select_affinity()
> >
> > do_irq_select_affinity() will call the set_affinity() routine,
> > with a default mask. On SN2 (and possibly all ia64 platforms),
> > this will retarget all device interrupts to the same CPU, most
> > likely CPU0.
>
> > I'm looking at a patch that makes use of the new IRQ_AFFINITY_SET
> > flag in the irq_desc to get around this.
> >
>
> that bit only be set in irq_set_affinity(),
>
> arch/mips/kernel/cevt-bcm1480.c: irq_set_affinity(irq,
> cpumask_of_cpu(cpu));
> arch/mips/kernel/cevt-sb1250.c: irq_set_affinity(irq, cpumask_of_cpu(cpu));
> arch/sparc64/kernel/of_device.c: irq_set_affinity(irq,
> numa_mask);
> arch/sparc64/kernel/pci_msi.c: irq_set_affinity(irq, numa_mask);
> arch/x86/kernel/hpet.c: irq_set_affinity(hdev->irq,
> cpumask_of_cpu(hdev->cpu));
> arch/x86/kernel/hpet.c: irq_set_affinity(dev->irq, cpumask_of_cpu(dev->cpu));
> drivers/xen/events.c: irq_set_affinity(irq, cpumask_of_cpu(0));
>
> kernel/irq/proc.c: irq_set_affinity(irq, new_value);
>
> kernel/time/tick-common.c: irq_set_affinity(newdev->irq, *cpumask);
>
> so you may need to call it in your arch code. like
> setup_IO_APIC_irq(), create_irq()...
>
> YH
>
Yes, I am planning to set it in SN platform specific code,
sn_irq_fixup().
John