Hello,
It seems that IRQ handling is somehow different between i386 and x86_64.
In my Dell PowerEdge 1950 is it possible to enable interrupts spreading
over all CPUs. This a single CPU, four CORE system (Quad-Core E5335 Xeon)
so I think that interrupts migration may be useful. Unfortunately, it
works only with 32-bit kernel. Booting it with x86_64 leads to situation,
when all interrupts goes only to the first cpu matching a smp_affinity
mask.
# cat /proc/interrupts (i386)
CPU0 CPU1 CPU2 CPU3
0: 402 1 3 3 IO-APIC-edge timer
1: 1 0 1 0 IO-APIC-edge i8042
8: 13 13 12 13 IO-APIC-edge rtc
9: 0 0 0 0 IO-APIC-fasteoi acpi
12: 1 1 0 2 IO-APIC-edge i8042
14: 5 4 4 4 IO-APIC-edge libata
15: 0 0 0 0 IO-APIC-edge libata
20: 0 0 0 0 IO-APIC-fasteoi libata
21: 1428 1434 1433 1435 IO-APIC-fasteoi ioc0
22: 6 6 6 6 IO-APIC-fasteoi ehci_hcd:usb1, uhci_hcd:usb2, uhci_hcd:usb4
23: 18 24 23 18 IO-APIC-fasteoi uhci_hcd:usb3
212: 1 1 1 0 PCI-MSI-edge eth1
213: 1045 1039 1039 1041 PCI-MSI-edge eth0
214: 0 0 0 0 PCI-MSI-edge ioat
NMI: 0 0 0 0
LOC: 3284200 3283865 3286809 3283934
ERR: 0
MIS: 0
# cat /proc/interrupts (x86_64)
CPU0 CPU1 CPU2 CPU3
0: 330199 0 0 0 IO-APIC-edge timer
1: 2 0 0 0 IO-APIC-edge i8042
8: 16 0 0 0 IO-APIC-edge rtc
9: 0 0 0 0 IO-APIC-fasteoi acpi
12: 4 0 0 0 IO-APIC-edge i8042
14: 16 0 0 0 IO-APIC-edge libata
15: 0 0 0 0 IO-APIC-edge libata
20: 48 0 0 0 IO-APIC-fasteoi uhci_hcd:usb3
21: 24 0 0 0 IO-APIC-fasteoi ehci_hcd:usb1, uhci_hcd:usb2, uhci_hcd:usb4
23: 0 0 0 0 IO-APIC-fasteoi libata
64: 5104 0 0 0 IO-APIC-fasteoi ioc0
372: 3 0 0 0 PCI-MSI-edge eth1
373: 540 0 0 0 PCI-MSI-edge eth0
374: 0 0 0 0 PCI-MSI-edge ioat
NMI: 0 0 0 0
LOC: 330055 330041 329953 329858
ERR: 0
Please notice also that irq numbers are different:
- libata: 20 on i386, 23 on x86_64
- ioc: 21 on i386, 64 on x86_64
- ehci_hcd:usb1: 22 on i386, 21 on x86_64
- uhci_hcd:usb3: 23 on i386, 20 on x86_64
- eth1, eth0, ioat: 212/213/214 on i386, 372/373/374 on x86_64
Two question:
- Is it possible to make irq spreading working on x86_64?
- Why the irq numbers are different?
Attaching dmesg and lspci from both i386 and x86_64.
Best regards,
Krzysztof Ol?dzki
On Sat, 2007-06-30 at 16:55 +0200, Krzysztof Oledzki wrote:
> Hello,
>
> It seems that IRQ handling is somehow different between i386 and x86_64.
>
> In my Dell PowerEdge 1950 is it possible to enable interrupts spreading
> over all CPUs. This a single CPU, four CORE system (Quad-Core E5335 Xeon)
> so I think that interrupts migration may be useful. Unfortunately, it
> works only with 32-bit kernel. Booting it with x86_64 leads to situation,
> when all interrupts goes only to the first cpu matching a smp_affinity
> mask.
arguably that is the most efficient behavior... round robin of
interrupts is the worst possible case in terms of performance
are you using irqbalance ? (http://www.irqbalance.org)
On Sat, 30 Jun 2007, Arjan van de Ven wrote:
> On Sat, 2007-06-30 at 16:55 +0200, Krzysztof Oledzki wrote:
>> Hello,
>>
>> It seems that IRQ handling is somehow different between i386 and x86_64.
>>
>> In my Dell PowerEdge 1950 is it possible to enable interrupts spreading
>> over all CPUs. This a single CPU, four CORE system (Quad-Core E5335 Xeon)
>> so I think that interrupts migration may be useful. Unfortunately, it
>> works only with 32-bit kernel. Booting it with x86_64 leads to situation,
>> when all interrupts goes only to the first cpu matching a smp_affinity
>> mask.
>
> arguably that is the most efficient behavior... round robin of
> interrupts is the worst possible case in terms of performance
Even on dual/quadro core CPUs with shared cache? So why it is possible to
enable such behaviuor in BIOS, which works only on i386 BTW. :(
> are you using irqbalance ? (http://www.irqbalance.org)
Yes, I'm aware about this useful tool, but in some situations (routing
for example) it cannot help much as it keeps three cpus idle. :(
Best regards,
Krzysztof Ol?dzki