Hi,
I am seeing the following problem with MSI/MSI-X.
Note: I am copying netdev since other network drivers use
this feature and somebody on the list could throw light.
Our 10G network card(Xframe II) supports MSI and MSI-X.
When I load/unload the driver with MSI support followed
by an attempt to load with MSI-X, I get the following
message from pci_enable_msix:
"Can't enable MSI-X. Device already has an MSI vector assigned"
I seem to be doing the correct things when unloading the
MSI driver. Basically, I do free_irq() followed by pci_disable_msi().
Any idea what I am missing ?
Further analysis:
Looking at the code, the following check(when it finds a match) in
msi_lookup_vector(called by pci_enable_msix) seems to throw up this
message:
if (!msi_desc[vector] || msi_desc[vector]->dev != dev ||
msi_desc[vector]->msi_attrib.type != type ||
msi_desc[vector]->msi_attrib.default_vector != dev->irq)
pci_enable_msi, on successful completion will populate the
fields in msi_desc. But neither pci_disable_msi nor free_irq
seems to undo/unpopulate the msi_desc table.
Could this be the cause for the problem ?
Thanks,
Ravi
I noticed the same behaviour, i.e. can not use both MSI and MSIX without
rebooting.
I had sent a message to the maintainer of the MSI/MSIX source a few
months ago and got a response that they were working on fixing it. Not
sure what the progress is on it.
Ayaz
nvpublic
-----Original Message-----
From: [email protected] [mailto:[email protected]]
On Behalf Of Ravinandan Arakali
Sent: Thursday, May 04, 2006 4:16 PM
To: [email protected]
Cc: Ananda. Raju; [email protected]; Leonid Grossman
Subject: pci_enable_msix throws up error
Hi,
I am seeing the following problem with MSI/MSI-X.
Note: I am copying netdev since other network drivers use
this feature and somebody on the list could throw light.
Our 10G network card(Xframe II) supports MSI and MSI-X.
When I load/unload the driver with MSI support followed
by an attempt to load with MSI-X, I get the following
message from pci_enable_msix:
"Can't enable MSI-X. Device already has an MSI vector assigned"
I seem to be doing the correct things when unloading the
MSI driver. Basically, I do free_irq() followed by pci_disable_msi().
Any idea what I am missing ?
Further analysis:
Looking at the code, the following check(when it finds a match) in
msi_lookup_vector(called by pci_enable_msix) seems to throw up this
message:
if (!msi_desc[vector] || msi_desc[vector]->dev != dev ||
msi_desc[vector]->msi_attrib.type != type ||
msi_desc[vector]->msi_attrib.default_vector != dev->irq)
pci_enable_msi, on successful completion will populate the
fields in msi_desc. But neither pci_disable_msi nor free_irq
seems to undo/unpopulate the msi_desc table.
Could this be the cause for the problem ?
Thanks,
Ravi
On Friday 05 May 2006 07:14, Ayaz Abdulla wrote:
> I noticed the same behaviour, i.e. can not use both MSI and MSIX without
> rebooting.
>
> I had sent a message to the maintainer of the MSI/MSIX source a few
> months ago and got a response that they were working on fixing it. Not
> sure what the progress is on it.
The best way to make progress faster would be for someone like you
who needs it to submit a patch to fix it then.
-Andi
Hi,
I was wondering if anybody has got MSI-X going on AMD 8132 platforms.
Our network card and driver support MSI-X and the combination works
fine on IA64 and xeon platforms. But on the 8132, the MSI-X vectors are
assigned(pci_enable_msix succeeds) but no interrupts get generated.
Note that with a different OS, MSI-X does work on 8132.
Any suggestions are appreciated.
Thanks,
Ravi
"Ravinandan Arakali" <[email protected]> writes:
> I was wondering if anybody has got MSI-X going on AMD 8132 platforms.
> Our network card and driver support MSI-X and the combination works
> fine on IA64 and xeon platforms. But on the 8132, the MSI-X vectors are
> assigned(pci_enable_msix succeeds) but no interrupts get generated.
See erratum #78 in the AMD 8132 Specification update.
It doesn't support the MSI capability and there are no plans to fix that.
AFAIK the only way to get MSI on Opteron is on PCI Express.
> Note that with a different OS, MSI-X does work on 8132.
Are you sure?
-Andi
Andi Kleen wrote:
> "Ravinandan Arakali" <[email protected]> writes:
>
>
>>I was wondering if anybody has got MSI-X going on AMD 8132 platforms.
>>Our network card and driver support MSI-X and the combination works
>>fine on IA64 and xeon platforms. But on the 8132, the MSI-X vectors are
>>assigned(pci_enable_msix succeeds) but no interrupts get generated.
>
>
> See erratum #78 in the AMD 8132 Specification update.
> It doesn't support the MSI capability and there are no plans to fix that.
>
> AFAIK the only way to get MSI on Opteron is on PCI Express.
I do not think that erratum #78 is related to this - it is related to tunnel
itself generating MSI - which is not needed in this case.
>>Note that with a different OS, MSI-X does work on 8132.
>
> Are you sure?
Can you provide 'lspci -vvvxxx' output from AMD8132 bridge? (esp. bytes
0xF4-0xFF from config space of 1022:7458 devices) By default dword at 0xF4 is
0xA8000008, disabling MSI/MSI-X mapping -> hypertransport interrupts. Changing
this to 0xA8010008 should enable this translation (iff qword at 0xF8 is
0x0000FEE00000), allowing MSI to work on respective secondary/subordinate
busses. Unfortunately kernel ignores these HT capabilities...
Petr
Hi Petr,
Thanks for the suggestion.
What you pointed out was the source of the problem. After
enabling the MSI-to-HTInterrupts(the value at offset 0xF6),
I now see interrupts and can ping in both MSI and MSI-X modes.
Ravi
-----Original Message-----
From: Petr Vandrovec [mailto:[email protected]]
Sent: Tuesday, May 16, 2006 8:01 AM
To: Andi Kleen
Cc: [email protected]; Peter. Phan; Leonid Grossman;
[email protected]
Subject: Re: MSI-X support on AMD 8132 platforms ?
Andi Kleen wrote:
> "Ravinandan Arakali" <[email protected]> writes:
>
>
>>I was wondering if anybody has got MSI-X going on AMD 8132 platforms.
>>Our network card and driver support MSI-X and the combination works
>>fine on IA64 and xeon platforms. But on the 8132, the MSI-X vectors are
>>assigned(pci_enable_msix succeeds) but no interrupts get generated.
>
>
> See erratum #78 in the AMD 8132 Specification update.
> It doesn't support the MSI capability and there are no plans to fix that.
>
> AFAIK the only way to get MSI on Opteron is on PCI Express.
I do not think that erratum #78 is related to this - it is related to tunnel
itself generating MSI - which is not needed in this case.
>>Note that with a different OS, MSI-X does work on 8132.
>
> Are you sure?
Can you provide 'lspci -vvvxxx' output from AMD8132 bridge? (esp. bytes
0xF4-0xFF from config space of 1022:7458 devices) By default dword at 0xF4
is
0xA8000008, disabling MSI/MSI-X mapping -> hypertransport interrupts.
Changing
this to 0xA8010008 should enable this translation (iff qword at 0xF8 is
0x0000FEE00000), allowing MSI to work on respective secondary/subordinate
busses. Unfortunately kernel ignores these HT capabilities...
Petr