Folks,
This group was instrumental in helping me get my first-ever
linux/PCI-bus device driver working last year, and I'm back for
some more help if you are willing.
I have a PCI card that generates an interrupt when it completes
a DMA transfer to the PCs RAM.
This works just fine on a Dell 4400 running 2.6.10-1.766_FC3
When I try to run the driver on a Dell 2300 FC2/2.6.5-1.358smp
or a Sun W2100Z running FC2/2.6.10-1.14_FC2smp I can see the
DMA-done bit set in the device, but my interrupt service routine
never gets called.
On the Sun, I booted with "noapic" option, and it booted OK,
but then when my device generated an interrupt, there was a
kernel message about Disabling IRQ #5 and the system was hung
solidly.
I think this has something to do with the different interrupt
hardware on the more advanced servers compared to my desktop
Dell 4400, and I somehow need to "enable" the IOAPIC system
so that my interrupt gets through to my service routine, but I
don't know how.
I tried grepping through the kernel/drivers source, and I didn't
find anything that jumped out at me.
The Rubini drivers book didn't help in this area either,
although it's a wonderful book in other areas.
I can post source somewhere if it will help.
I can also post the essential bits from /var/log/messages about
all the incredibly complicated IOAPIC configuration stuff.
Thank you for your past help, and thank you in advance for any
tips you can provide.
-Alan
--
- Alan Kilian <kilian(at)bobodyne.com>
On Wed, 23 Feb 2005, Alan Kilian wrote:
call pci_enable_device(dev)
... before you use the IRQ in dev->irq.
The reported IRQ is bogus until you make that
call. It's a reported BUG, probably won't
ever get fixed because it's considered a
feature.
Also, make sure that your .config for the Dell looks
something like:
CONFIG_X86_IO_APIC=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_PCI=y
# CONFIG_PCI_GOBIOS is not set
# CONFIG_PCI_GODIRECT is not set
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
>
>
> Folks,
>
> This group was instrumental in helping me get my first-ever
> linux/PCI-bus device driver working last year, and I'm back for
> some more help if you are willing.
>
> I have a PCI card that generates an interrupt when it completes
> a DMA transfer to the PCs RAM.
>
> This works just fine on a Dell 4400 running 2.6.10-1.766_FC3
>
> When I try to run the driver on a Dell 2300 FC2/2.6.5-1.358smp
> or a Sun W2100Z running FC2/2.6.10-1.14_FC2smp I can see the
> DMA-done bit set in the device, but my interrupt service routine
> never gets called.
>
> On the Sun, I booted with "noapic" option, and it booted OK,
> but then when my device generated an interrupt, there was a
> kernel message about Disabling IRQ #5 and the system was hung
> solidly.
>
> I think this has something to do with the different interrupt
> hardware on the more advanced servers compared to my desktop
> Dell 4400, and I somehow need to "enable" the IOAPIC system
> so that my interrupt gets through to my service routine, but I
> don't know how.
>
> I tried grepping through the kernel/drivers source, and I didn't
> find anything that jumped out at me.
>
> The Rubini drivers book didn't help in this area either,
> although it's a wonderful book in other areas.
>
> I can post source somewhere if it will help.
>
> I can also post the essential bits from /var/log/messages about
> all the incredibly complicated IOAPIC configuration stuff.
>
> Thank you for your past help, and thank you in advance for any
> tips you can provide.
>
> -Alan
>
> --
> - Alan Kilian <kilian(at)bobodyne.com>
>
Cheers,
Dick Johnson
Penguin : Linux version 2.6.10 on an i686 machine (5537.79 BogoMips).
Notice : All mail here is now cached for review by Dictator Bush.
98.36% of all statistics are fiction.
> On Wed, 23 Feb 2005, Dick Johnson wrote:
>
> call pci_enable_device(dev)
> ... before you use the IRQ in dev->irq.
>
> The reported IRQ is bogus until you make that
> call. It's a reported BUG, probably won't
> ever get fixed because it's considered a
> feature.
>
> Also, make sure that your .config for the Dell looks
> something like:
>
> CONFIG_X86_IO_APIC=y
> CONFIG_X86_LOCAL_APIC=y
> CONFIG_PCI=y
> # CONFIG_PCI_GOBIOS is not set
> # CONFIG_PCI_GODIRECT is not set
> CONFIG_PCI_GOANY=y
> CONFIG_PCI_BIOS=y
> CONFIG_PCI_DIRECT=y
Dick,
Thanks for the quick reply.
1) I call pci_enable_device(dev) immediatly after I call
dev = pci_find_device(0x1492, PCI_ANY_ID, dev);
2) I have verified all the CONFIG settings you suggested.
Here is `cat /proc/interrupts` on my working dell:
CPU0
0: 16891629 XT-PIC timer
1: 10 XT-PIC i8042
2: 0 XT-PIC cascade
3: 2 XT-PIC parport0
5: 764 XT-PIC sse
7: 422 XT-PIC ohci_hcd
8: 1 XT-PIC rtc
9: 0 XT-PIC acpi
11: 35198 XT-PIC eth0
12: 66 XT-PIC i8042
14: 42769 XT-PIC ide0
15: 151569 XT-PIC ide1
NMI: 0
ERR: 0
My driver is called "sse" and is interrupting at IRQ #5
Here is `cat /proc/interrupts` on my non-working Sun:
CPU0 CPU1
0: 7302649 7417311 IO-APIC-edge timer
5: 0 0 IO-APIC-edge sse
8: 0 1 IO-APIC-edge rtc
9: 0 0 IO-APIC-level acpi
15: 1 478 IO-APIC-edge ide1
169: 0 30 IO-APIC-level aic79xx
177: 13991 18084 IO-APIC-level aic79xx
185: 0 3 IO-APIC-level ehci_hcd
193: 0 26 IO-APIC-level ohci_hcd
201: 0 21 IO-APIC-level ohci_hcd
209: 167119 87 IO-APIC-level eth0
NMI: 0 0
LOC: 14719159 14719203
ERR: 0
MIS: 0
It appears that the card is also interrupting at IRQ#5
There is an interesting message on the Sun in /var/log/messages:
Feb 23 14:01:26 sunw1200z kernel: sse: no version magic, tainting
kernel.
Feb 23 14:01:26 sunw1200z kernel: SSE: Found a DeCypher card.
Feb 23 14:01:26 sunw1200z kernel: ACPI: PCI interrupt 0000:13:03.0[A] ->
GSI 36 (level, low) -> IRQ 217
Feb 23 14:01:26 sunw1200z kernel: interrupting on line 5
Feb 23 14:01:26 sunw1200z kernel: SSE: bar[0] From 0xd2806000 to
0xd2806fff F=0x200 MEMORY space
Feb 23 14:01:26 sunw1200z kernel: SSE: bar[1] From 0xd2800000 to
0xd2801fff F=0x200 MEMORY space
Feb 23 14:01:26 sunw1200z kernel: SSE: bar[2] From 0xd2000000 to
0xd27fffff F=0x200 MEMORY space
Feb 23 14:01:26 sunw1200z kernel: SSE: bar[0] mybase = 0xf889a000 size =
0x00001000 D'4096
Feb 23 14:01:26 sunw1200z kernel: SSE: bar[1] mybase = 0xf889c000 size =
0x00002000 D'8192
Feb 23 14:01:26 sunw1200z kernel: SSE: bar[2] mybase = 0xf8b00000 size =
0x00800000 D'8388608
Feb 23 14:01:26 sunw1200z kernel: pci_alloc_consistent returned
0xf0ded000
Feb 23 14:01:26 sunw1200z kernel: sse_read_bus_buffer 0x30ded000
Feb 23 14:01:26 sunw1200z kernel: request_irq() returned 0
Feb 23 14:01:26 sunw1200z kernel: SSE device_id 3, Rev 4
Feb 23 14:01:26 sunw1200z kernel: SSE Before: intstatus = 0x00000000
Feb 23 14:01:26 sunw1200z kernel: SSE Before: intstatus = 0x00000000
Feb 23 14:01:26 sunw1200z kernel: SSE: End of card attachment. Number of
cards = 1
Feb 23 14:01:26 sunw1200z kernel: Iterating through the softp
structures...
Feb 23 14:01:26 sunw1200z kernel: Card at softp->mem_reg1 0xf889a000
minor = 0
The interesting bits seem to be these two lines:
kernel: SSE: Found a DeCypher card.
kernel: ACPI: PCI interrupt 0000:13:03.0[A] -> GSI 36 (level, low) ->
IRQ 217
The first message is in my driver after pci_find_device()
The second is from when I do pci_enable_device(dev);
Can you decode the mysterious ACPI message?
-Alan
--
- Alan Kilian <kilian(at)bobodyne.com>
>>>>> "Alan" == Alan Kilian <[email protected]> writes:
Alan> kernel: SSE: Found a DeCypher card. kernel: ACPI: PCI
Alan> interrupt 0000:13:03.0[A] -> GSI 36 (level, low) -> IRQ 217
If ACPI has set this device up to use interrupt 217, why are you
registering it on IRQ 5?
--
Dr Peter Chubb http://www.gelato.unsw.edu.au peterc AT gelato.unsw.edu.au
The technical we do immediately, the political takes *forever*
> Alan> kernel: SSE: Found a DeCypher card. kernel: ACPI: PCI
> Alan> interrupt 0000:13:03.0[A] -> GSI 36 (level, low) -> IRQ 217
>
> If ACPI has set this device up to use interrupt 217, why are you
> registering it on IRQ 5?
>
Peter,
Maybe that's it.
I ask the card which interrupt line it was given at boot-time:
pci_read_config_byte(dev, PCI_INTERRUPT_LINE,
&softp->interrupt_line);
Then I request an IRQ:
request_irq(softp->interrupt_line, sseintr,
SA_INTERRUPT, "sse", softp);
I don't know who exactly is assigning IRQ 217, and I don't know
where I could read that value.
Are you suggesting I should call request_irq(217, sseintr,
SA_INTERRUPT, "sse", softp);
Maybe you are on to the problem.
I found out that if I boot with "pci-noapic" that everyting
works on the Sun machine!!!
-Alan
--
- Alan Kilian <kilian(at)bobodyne.com>
Alan Kilian wrote:
> kernel: SSE: Found a DeCypher card.
> kernel: ACPI: PCI interrupt 0000:13:03.0[A] -> GSI 36 (level, low) ->
> IRQ 217
>
> The first message is in my driver after pci_find_device()
> The second is from when I do pci_enable_device(dev);
>
> Can you decode the mysterious ACPI message?
Looks like you're requesting the wrong interrupt, 217 is the one your
device is actually on. You always have to request the interrupt listed
in the PCI device structure. If you're looking at your PCI device's
configuration registers to get the IRQ to request, that's wrong, since
that is configured by the BIOS assuming PIC IRQ routing, but APIC IRQ
routing is entirely different.
Alan Kilian wrote:
> Maybe that's it.
>
> I ask the card which interrupt line it was given at boot-time:
>
> pci_read_config_byte(dev, PCI_INTERRUPT_LINE,
> &softp->interrupt_line);
>
> Then I request an IRQ:
>
> request_irq(softp->interrupt_line, sseintr,
> SA_INTERRUPT, "sse", softp);
Yeah, that's wrong. Should be request_irq(dev->irq, ... )
PCI_INTERRUPT_LINE is assigned by the BIOS and has nothing to do with
the routing used in APIC mode. That's why it works with noapic mode
since then the routing is the same as the BIOS assumed.
--
Robert Hancock Saskatoon, SK, Canada
To email, remove "nospam" from [email protected]
Home Page: http://www.roberthancock.com/
Alan> I ask the card which interrupt line it was given at
Alan> boot-time:
Alan> pci_read_config_byte(dev, PCI_INTERRUPT_LINE, &softp->interrupt_line);
Alan> Then I request an IRQ:
Alan> request_irq(softp->interrupt_line, sseintr, SA_INTERRUPT, "sse", softp);
Don't do that. The kernel may need you to use a different interrupt
number than you read from the PCI config header. Use dev->irq, as in
request_irq(dev->irq, sseintr, SA_SHIRQ | SA_INTERRUPT, "sse", softp);
Also, make sure that you do pci_enable_device(dev) before you look at
dev->irq, since the field will not be initialized until you do that.
- R.
On Wed, 2005-02-23 at 15:46 -0800, Roland Dreier wrote:
> Alan> pci_read_config_byte(dev, PCI_INTERRUPT_LINE, &softp->interrupt_line);
> Alan> request_irq(softp->interrupt_line, sseintr, SA_INTERRUPT, "sse", softp);
>
> Don't do that. The kernel may need you to use a different interrupt
> number than you read from the PCI config header. Use dev->irq, as in
>
> request_irq(dev->irq, sseintr, SA_SHIRQ | SA_INTERRUPT, "sse", softp);
Roland,
You are the best. Thank you so much for this information.
That was the ticket.
request_irq(dev->irq, sseintr, SA_SHIRQ | SA_INTERRUPT,
"sse", softp);
It works on all machines now without adding "pci=noapci"
I'm running again!!!
-Alan
--
- Alan Kilian <kilian(at)bobodyne.com>
Where are you getting IRQ5 from? You can't "hard-code" interrupts on
PCI.
> kernel: ACPI: PCI interrupt 0000:13:03.0[A] -> GSI 36 (level, low) ->
> IRQ 217
^^^^^^^^^___________ This is your IRQ
It should be in dev->irq AFTER it's enabled.
[SNIPPED...]
Cheers,
Dick Johnson
Penguin : Linux version 2.6.10 on an i686 machine (5537.79 BogoMips).
Notice : All mail here is now cached for review by Dictator Bush.
98.36% of all statistics are fiction.