2005-11-17 05:44:26

by Jeff Garzik

[permalink] [raw]
Subject: PCI MSI: the new interrupt routing headache


I just got SATA working on Marvell. After fixing a bunch of issues in
the driver, the final issue was lack of interrupts. Disabling
CONFIG_PCI_MSI solved that, and suddenly the driver was working quite
nicely.

The general problem is that pci_enable_msi() is not failing, on systems
that do not support MSI. This leads to Infiniband, tg3, and other
drivers working around this problem by including an MSI-interrupts-work
test during probe.

Perhaps its because I like leading edge stuff, and am playing with
drivers for PCI MSI hardware, but it seems like I am running into this
pci_enable_msi()-doesnt-fail problem more and more frequently. First
tg3, then AHCI, now sata_mv.

What needs to be done, to detect working PCI message signalled
interrupts such that pci_enable_msi() fails properly?

Thanks,

Jeff




2005-11-17 19:07:49

by Max Krasnyansky

[permalink] [raw]
Subject: Re: PCI MSI: the new interrupt routing headache

Hi Jeff,
>
> I just got SATA working on Marvell. After fixing a bunch of issues in
> the driver, the final issue was lack of interrupts. Disabling
> CONFIG_PCI_MSI solved that, and suddenly the driver was working quite
> nicely.
>
> The general problem is that pci_enable_msi() is not failing, on systems
> that do not support MSI. This leads to Infiniband, tg3, and other
> drivers working around this problem by including an MSI-interrupts-work
> test during probe.
>
> Perhaps its because I like leading edge stuff, and am playing with
> drivers for PCI MSI hardware, but it seems like I am running into this
> pci_enable_msi()-doesnt-fail problem more and more frequently. First
> tg3, then AHCI, now sata_mv.
>
> What needs to be done, to detect working PCI message signalled
> interrupts such that pci_enable_msi() fails properly?
It fails correctly when MSI quirk is enabled. I had the same problems
on AMD 8131 based machine with 2.6.11 kernel. E1000 was enabling MSI and
not getting any interrupts. Later (.13 I think) MSI quirk was introduced for
8131 chipset and everything started working (without having to hack E1000
that is).
May be you should just add your platform to the PCI quirks list ?

Max

2005-11-18 20:52:32

by Roland Dreier

[permalink] [raw]
Subject: Re: PCI MSI: the new interrupt routing headache

Jeff> What needs to be done, to detect working PCI message
Jeff> signalled interrupts such that pci_enable_msi() fails
Jeff> properly?

There are two things that cause MSIs not to work. First, the PCI host
bridge may not have working MSI support. To handle this, we have the
"msi_quirk" which is set by the PCI quirk code. For example, this is
used on systems with AMD-8131 PCI-X bridges. (As I've noted
elsewhere, this is actually too crude a method -- actual systems exist
with e.g. both AMD-8131 and Nforce4 PCI bridges, so that MSI works for
PCIe devices but not devices below the AMD-8131)

Second, there are PCI devices that have an MSI capability but which
don't have working MSI support. Most revisions of the e1000 fall into
this category. In this case, it is up to the driver to know when it's
safe to try to enable MSI.

However, given that MSI/MSI-X is not in wide use, there is undoubtedly
much more broken hardware (both chipsets and devices) that we don't
know about and need to add quirks or driver workarounds for.

Hence, in the interest of discovering this hardware and also in
getting less cryptic bug reports, it's a good idea to add a test that
interrupts actually work in any driver that tries to enable MSI or
MSI-X. Since it requires knowledge of a device to know how to get the
device to trigger an interrupt, this test has to be done in each
driver and can't be centralized.

- R.