2009-12-25 09:22:27

by Torsten Kaiser

[permalink] [raw]
Subject: MSI broken in libata?

As reported in http://lkml.org/lkml/2009/12/19/82 the new MSI support
for sata_sil24 does not work for me.
This is still the same with 2.6.33-rc2.

Why I think, this might be a problem within libata:
* other drivers can use MSI successful on my system (tg3, radeon, hda-intel)
* happens both in sata_sil24 and sata_nv
* the count in /proc/interrupts increases for the MSIs assigned to
sata_sil24/sata_nv, so interrupt delivery seems to work
* only writing seems to fail

These are the changes in /proc/interrupts:
Normal boot, without additional cmdline options:
- 4: 0 0 96 174 IO-APIC-edge serial

- 19: 18058 2 3 4738 IO-APIC-fasteoi
sata_sil24, bttv0, Bt87x audio
- 21: 0 0 0 0 IO-APIC-fasteoi sata_nv
- 22: 0 0 0 0 IO-APIC-fasteoi sata_nv
- 23: 872 357 4 1514 IO-APIC-fasteoi
sata_nv, ohci_hcd:usb2

- 29: 4 405 0 140 PCI-MSI-edge HDA Intel
- 30: 0 0 0 3 PCI-MSI-edge eth0
- 31: 3354 0 0 38 PCI-MSI-edge eth1

Boot with sata_nv.msi=1 and sata_sil24.msi=1, only booted into a
rescue shell, thats why serial and eth0+1 are missing:
+ 19: 0 0 0 1 IO-APIC-fasteoi
bttv0, Bt87x audio
+ 23: 0 0 5 13 IO-APIC-fasteoi
ohci_hcd:usb2

+ 29: 0 0 0 73 PCI-MSI-edge sata_sil24
+ 30: 0 0 0 2590 PCI-MSI-edge sata_nv
+ 31: 0 0 0 0 PCI-MSI-edge sata_nv
+ 32: 0 0 0 0 PCI-MSI-edge sata_nv
+ 33: 0 0 0 89 PCI-MSI-edge HDA Intel

Attached is the output from lspci -vv from a normal boot and from a
boot with MSI enabled for both libata drivers.
The errors messages from libata and other information about my system
can be found in http://lkml.org/lkml/2009/12/19/82 ,or just ask if you
need to know something else.

Here are the differences from the lspci-outputs:

lspci-vv-rc2.txt: complete, normal boot
lspci-vv-rc2.msi.txt: init=/bin/sash, MSI for sata_sil24+sata_nv,
network cards not initialised
--- lspci-vv-rc2.txt 2009-12-25 00:11:19.992078760 +0100
+++ lspci-vv-rc2.msi.txt 2009-12-25 00:04:26.713756803 +0100
@@ -1,631 +1,586 @@
00:04.0 IDE interface: nVidia Corporation MCP55 IDE (rev a1) (prog-if
8a [Master SecP PriP])
Subsystem: ASUSTeK Computer Inc. Device 81f0
Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0 (750ns min, 250ns max)
Region 0: [virtual] Memory at 000001f0 (32-bit, non-prefetchable) [size=8]
- Region 1: [virtual] Memory at 000003f0 (type 3, non-prefetchable) [size=1]
+ Region 1: [virtual] Memory at 000003f0 (type 3, non-prefetchable)
Region 2: [virtual] Memory at 00000170 (32-bit, non-prefetchable) [size=8]
- Region 3: [virtual] Memory at 00000370 (type 3, non-prefetchable) [size=1]
+ Region 3: [virtual] Memory at 00000370 (type 3, non-prefetchable)
Region 4: I/O ports at ffa0 [size=16]
Capabilities: [44] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

00:05.0 IDE interface: nVidia Corporation MCP55 SATA Controller (rev
a3) (prog-if 85 [Master SecO PriO])
Subsystem: ASUSTeK Computer Inc. Device 81f0
- Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
+ Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0 (750ns min, 250ns max)
- Interrupt: pin A routed to IRQ 23
+ Interrupt: pin A routed to IRQ 30
Region 0: I/O ports at cc00 [size=8]
Region 1: I/O ports at c880 [size=4]
Region 2: I/O ports at c800 [size=8]
Region 3: I/O ports at c480 [size=4]
Region 4: I/O ports at c400 [size=16]
Region 5: Memory at efafb000 (32-bit, non-prefetchable) [size=4K]
Capabilities: [44] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
- Capabilities: [b0] MSI: Enable- Count=1/4 Maskable- 64bit+
- Address: 0000000000000000 Data: 0000
+ Capabilities: [b0] MSI: Enable+ Count=1/4 Maskable- 64bit+
+ Address: 00000000fee0f00c Data: 4189
Capabilities: [cc] HyperTransport: MSI Mapping Enable- Fixed+

00:05.1 IDE interface: nVidia Corporation MCP55 SATA Controller (rev
a3) (prog-if 85 [Master SecO PriO])
Subsystem: ASUSTeK Computer Inc. Device 81f0
- Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
+ Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0 (750ns min, 250ns max)
- Interrupt: pin B routed to IRQ 22
+ Interrupt: pin B routed to IRQ 31
Region 0: I/O ports at c080 [size=8]
Region 1: I/O ports at c000 [size=4]
Region 2: I/O ports at bc00 [size=8]
Region 3: I/O ports at b880 [size=4]
Region 4: I/O ports at b800 [size=16]
Region 5: Memory at efafa000 (32-bit, non-prefetchable) [size=4K]
Capabilities: [44] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
- Capabilities: [b0] MSI: Enable- Count=1/4 Maskable- 64bit+
- Address: 0000000000000000 Data: 0000
+ Capabilities: [b0] MSI: Enable+ Count=1/4 Maskable- 64bit+
+ Address: 00000000fee0f00c Data: 4199
Capabilities: [cc] HyperTransport: MSI Mapping Enable- Fixed+

00:05.2 IDE interface: nVidia Corporation MCP55 SATA Controller (rev
a3) (prog-if 85 [Master SecO PriO])
Subsystem: ASUSTeK Computer Inc. Device 81f0
- Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
+ Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0 (750ns min, 250ns max)
- Interrupt: pin C routed to IRQ 21
+ Interrupt: pin C routed to IRQ 32
Region 0: I/O ports at b480 [size=8]
Region 1: I/O ports at b400 [size=4]
Region 2: I/O ports at b080 [size=8]
Region 3: I/O ports at b000 [size=4]
Region 4: I/O ports at ac00 [size=16]
Region 5: Memory at efaf9000 (32-bit, non-prefetchable) [size=4K]
Capabilities: [44] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
- Capabilities: [b0] MSI: Enable- Count=1/4 Maskable- 64bit+
- Address: 0000000000000000 Data: 0000
+ Capabilities: [b0] MSI: Enable+ Count=1/4 Maskable- 64bit+
+ Address: 00000000fee0f00c Data: 41a9
Capabilities: [cc] HyperTransport: MSI Mapping Enable- Fixed+

00:06.1 Audio device: nVidia Corporation MCP55 High Definition Audio (rev a2)
Subsystem: ASUSTeK Computer Inc. Device 8210
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0 (500ns min, 1250ns max)
- Interrupt: pin B routed to IRQ 29
+ Interrupt: pin B routed to IRQ 33
Region 0: Memory at efafc000 (32-bit, non-prefetchable) [size=16K]
Capabilities: [44] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable+ Count=1/1 Maskable+ 64bit+
- Address: 00000000fee0200c Data: 41b1
+ Address: 00000000fee0f00c Data: 41d1
Masking: 00000000 Pending: 00000000
Capabilities: [6c] HyperTransport: MSI Mapping Enable- Fixed+

00:0f.0 PCI bridge: nVidia Corporation MCP55 PCI Express bridge (rev
a3) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
I/O behind bridge: 0000d000-0000dfff
Memory behind bridge: efb00000-efbfffff
Prefetchable memory behind bridge: 00000000e0000000-00000000e7ffffff
- Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- <SERR- <PERR-
+ Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort+ <SERR- <PERR-
BridgeCtl: Parity- SERR+ NoISA- VGA+ MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Subsystem: nVidia Corporation Device 0000
Capabilities: [48] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable+ Count=1/2 Maskable- 64bit+
Address: 00000000fee0f00c Data: 4161
Capabilities: [60] HyperTransport: MSI Mapping Enable- Fixed-
Mapping Address Base: 00000000fee00000
Capabilities: [80] Express (v1) Root Port (Slot+), MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x16, ASPM L0s L1, Latency L0
<512ns, L1 <4us
ClockPM- Surprise- LLActRep+ BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive+
BWMgmt- ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surpise-
Slot # 0, PowerLimit 0.000000; Interlock- NoCompl-
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
Control: AttnInd Off, PwrInd On, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
Changed: MRL- PresDet+ LinkState+
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-

01:00.0 VGA compatible controller: ATI Technologies Inc RV370 5B60
[Radeon X300 (PCIE)] (prog-if 00 [VGA controller])
Subsystem: ASUSTeK Computer Inc. Device 0083
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 28
Region 0: Memory at e0000000 (32-bit, prefetchable) [size=128M]
Region 1: I/O ports at d000 [size=256]
Region 2: Memory at efbf0000 (32-bit, non-prefetchable) [size=64K]
Expansion ROM at efbc0000 [disabled] [size=128K]
Capabilities: [50] Power Management version 2
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [58] Express (v1) Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <128ns, L1 <2us
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE- FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x16, ASPM L0s L1, Latency L0
<128ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive-
BWMgmt- ABWMgmt-
Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit+
- Address: 00000000fee0100c Data: 4171
+ Address: 00000000fee0f00c Data: 4171

02:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5754
Gigabit Ethernet PCI Express (rev 02)
Subsystem: ASUSTeK Computer Inc. Device 8209
- Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx+
+ Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
- Interrupt: pin A routed to IRQ 31
+ Interrupt: pin A routed to IRQ 17
Region 0: Memory at efcf0000 (64-bit, non-prefetchable) [size=64K]
- Expansion ROM at <ignored> [disabled]
Capabilities: [48] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] Vital Product Data
- Product Name: Broadcom NetXtreme Gigabit Ethernet Controller
- Read-only fields:
- [PN] Part number: BCM95700A6
- [EC] Engineering changes: 106679-15
- [SN] Serial number: 0123456789
- [MN] Manufacture ID: 31 34 65 34
- [RV] Reserved: checksum bad, 26 byte(s) reserved
- Read/write fields:
- [YA] Asset tag: XYZ01234567
- [RW] Read-write area: 107 byte(s) free
- End
+ Not readable
Capabilities: [58] Vendor Specific Information <?>
- Capabilities: [e8] MSI: Enable+ Count=1/1 Maskable- 64bit+
+ Capabilities: [e8] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 00000000fee0100c Data: 41c1
Capabilities: [d0] Express (v1) Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 4096 bytes
- DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
+ DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 <4us, L1 <64us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive-
BWMgmt- ABWMgmt-

03:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5754
Gigabit Ethernet PCI Express (rev 02)
Subsystem: ASUSTeK Computer Inc. Device 8209
- Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx+
+ Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
- Interrupt: pin A routed to IRQ 30
+ Interrupt: pin A routed to IRQ 18
Region 0: Memory at efdf0000 (64-bit, non-prefetchable) [size=64K]
- Expansion ROM at <ignored> [disabled]
Capabilities: [48] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] Vital Product Data
- Product Name: Broadcom NetXtreme Gigabit Ethernet Controller
- Read-only fields:
- [PN] Part number: BCM95700A6
- [EC] Engineering changes: 106679-15
- [SN] Serial number: 0123456789
- [MN] Manufacture ID: 31 34 65 34
- [RV] Reserved: checksum bad, 26 byte(s) reserved
- Read/write fields:
- [YA] Asset tag: XYZ01234567
- [RW] Read-write area: 107 byte(s) free
- End
+ Not readable
Capabilities: [58] Vendor Specific Information <?>
- Capabilities: [e8] MSI: Enable+ Count=1/1 Maskable- 64bit+
+ Capabilities: [e8] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 00000000fee0f00c Data: 41b9
Capabilities: [d0] Express (v1) Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 4096 bytes
- DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
+ DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 <4us, L1 <64us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive-
BWMgmt- ABWMgmt-

04:00.0 Mass storage controller: Silicon Image, Inc. SiI 3132 Serial
ATA Raid II Controller (rev 01)
Subsystem: ASUSTeK Computer Inc. Device 819f
- Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx-
+ Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
- Interrupt: pin A routed to IRQ 19
+ Interrupt: pin A routed to IRQ 29
Region 0: Memory at efeffc00 (64-bit, non-prefetchable) [size=128]
Region 2: Memory at efef8000 (64-bit, non-prefetchable) [size=16K]
Region 4: I/O ports at ec00 [size=128]
Expansion ROM at efe00000 [disabled] [size=512K]
Capabilities: [54] Power Management version 2
Flags: PMEClk- DSI+ D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
- Capabilities: [5c] MSI: Enable- Count=1/1 Maskable- 64bit+
- Address: 0000000000000000 Data: 0000
+ Capabilities: [5c] MSI: Enable+ Count=1/1 Maskable- 64bit+
+ Address: 00000000fee0f00c Data: 4179
Capabilities: [70] Express (v1) Legacy Endpoint, MSI 00
DevCap: MaxPayload 1024 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 4096 bytes
DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq+ AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0
unlimited, L1 unlimited
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive-
BWMgmt- ABWMgmt-


Attachments:
lspci-vv-rc2.txt (36.20 kB)
lspci-vv-rc2.msi.txt (34.61 kB)
Download all attachments

2010-01-09 23:16:33

by Tejun Heo

[permalink] [raw]
Subject: Re: MSI broken in libata?

On 12/25/2009 06:22 PM, Torsten Kaiser wrote:
> As reported in http://lkml.org/lkml/2009/12/19/82 the new MSI support
> for sata_sil24 does not work for me.
> This is still the same with 2.6.33-rc2.
>
> Why I think, this might be a problem within libata:
> * other drivers can use MSI successful on my system (tg3, radeon, hda-intel)
> * happens both in sata_sil24 and sata_nv
> * the count in /proc/interrupts increases for the MSIs assigned to
> sata_sil24/sata_nv, so interrupt delivery seems to work
> * only writing seems to fail

How does it fail? Timeouts? Also, ahci enables MSI by default if
available and works fine on many configurations so I don't think
anything in libata core layer is broken regarding MSI (there just
isn't anything which can break).

Thanks.

--
tejun

2010-01-10 04:34:01

by Torsten Kaiser

[permalink] [raw]
Subject: Re: MSI broken in libata?

On Sat, Jan 9, 2010 at 10:11 AM, Tejun Heo <[email protected]> wrote:
> On 12/25/2009 06:22 PM, Torsten Kaiser wrote:
>> As reported in http://lkml.org/lkml/2009/12/19/82 the new MSI support
>> for sata_sil24 does not work for me.
>> This is still the same with 2.6.33-rc2.
>>
>> Why I think, this might be a problem within libata:
>> ?* other drivers can use MSI successful on my system (tg3, radeon, hda-intel)
>> ?* happens both in sata_sil24 and sata_nv
>> ?* the count in /proc/interrupts increases for the MSIs assigned to
>> sata_sil24/sata_nv, so interrupt delivery seems to work
>> ?* only writing seems to fail
>
> How does it fail? ?Timeouts?

Yes, timeouts.
I posted the error messages in http://lkml.org/lkml/2009/12/19/82 and
http://lkml.org/lkml/2010/1/6/60

> Also, ahci enables MSI by default if
> available and works fine on many configurations so I don't think
> anything in libata core layer is broken regarding MSI (there just
> isn't anything which can break).

The system I'm using does not have a ahci compatible controller, so I
could not compare this.
(And my other system that is using ahci, does not use MSI for that)

I just found it suspicious that 3 other drivers (tg3, hda-intel and
radeon) can use MSI, but both of the libata drivers (sata_sil24 and
sata_nv) fail in a similar way.

I did try the patch from Robert Hancock in
http://lkml.org/lkml/2010/1/6/417 ,but without success.

if you need any more information, or have something for me to try,
please just ask. I did look at the code and the documentation about
enabling MSI, but did not see anything (obvious) wrong, so I don't
know what to try next.

Torsten

2010-01-11 01:13:55

by Tejun Heo

[permalink] [raw]
Subject: Re: MSI broken in libata?

On 01/10/2010 01:33 PM, Torsten Kaiser wrote:
> I did try the patch from Robert Hancock in
> http://lkml.org/lkml/2010/1/6/417 ,but without success.
>
> if you need any more information, or have something for me to try,
> please just ask. I did look at the code and the documentation about
> enabling MSI, but did not see anything (obvious) wrong, so I don't
> know what to try next.

Can you please try the attached patch?

Thanks.

--
tejun


Attachments:
sil24-msi-possible-fix.patch (1.87 kB)

2010-01-11 01:39:43

by Robert Hancock

[permalink] [raw]
Subject: Re: MSI broken in libata?

On 01/10/2010 07:15 PM, Tejun Heo wrote:
> On 01/10/2010 01:33 PM, Torsten Kaiser wrote:
>> I did try the patch from Robert Hancock in
>> http://lkml.org/lkml/2010/1/6/417 ,but without success.
>>
>> if you need any more information, or have something for me to try,
>> please just ask. I did look at the code and the documentation about
>> enabling MSI, but did not see anything (obvious) wrong, so I don't
>> know what to try next.
>
> Can you please try the attached patch?
>
> Thanks.
>

It'd be interesting to see if it makes a difference, but I don't think
the patch is quite right. According to the datasheet, doing the MSI ack
while the interrupt source is still pending will cause a new MSI to be
sent, so if you do it before handling the interrupt you'll generate a
spurious interrupt after every real one.

Though, apparently my patch that did the MSI ack after the handling
didn't help, so either that's wrong or the problem is unrelated. (I tend
to suspect the latter, given that sata_nv is also failing in the same way.)

2010-01-11 02:15:55

by Tejun Heo

[permalink] [raw]
Subject: Re: MSI broken in libata?

Hello, Robert.

On 01/11/2010 10:39 AM, Robert Hancock wrote:
> It'd be interesting to see if it makes a difference, but I don't think
> the patch is quite right. According to the datasheet, doing the MSI ack
> while the interrupt source is still pending will cause a new MSI to be
> sent, so if you do it before handling the interrupt you'll generate a
> spurious interrupt after every real one.

Ah... okay, so the generation of MSI is level triggered? If so, your
original patch is correct. I was thinking it might be edge triggered
and it should be cleared before checking master event mask.

Thanks.

--
tejun

2010-01-11 06:44:55

by Torsten Kaiser

[permalink] [raw]
Subject: Re: MSI broken in libata?

On Mon, Jan 11, 2010 at 2:15 AM, Tejun Heo <[email protected]> wrote:
> On 01/10/2010 01:33 PM, Torsten Kaiser wrote:
>> I did try the patch from Robert Hancock in
>> http://lkml.org/lkml/2010/1/6/417 ,but without success.
>>
>> if you need any more information, or have something for me to try,
>> please just ask. I did look at the code and the documentation about
>> enabling MSI, but did not see anything (obvious) wrong, so I don't
>> know what to try next.
>
> Can you please try the attached patch?

Yes, but as expected it does not change the result.

After applying the patch and doing a test write, I got the following
error messages:
[ 89.950055] ata2.00: exception Emask 0x0 SAct 0x1 SErr 0x0 action 0x6 frozen
[ 89.960910] ata2.00: failed command: WRITE FPDMA QUEUED
[ 89.969853] ata2.00: cmd 61/00:00:fd:27:3e/04:00:01:00:00/40 tag 0
ncq 524288 out
[ 89.969855] res 40/00:00:00:00:00/00:00:00:00:00/00 Emask
0x4 (timeout)
[ 89.992210] ata2.00: status: { DRDY }
[ 89.999569] ata2: hard resetting link
[ 92.190069] ata2: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
[ 97.190041] ata2.00: qc timeout (cmd 0xec)
[ 97.197898] ata2.00: failed to IDENTIFY (I/O error, err_mask=0x5)
[ 97.207622] ata2.00: revalidation failed (errno=-5)
[ 97.216094] ata2: hard resetting link
[ 99.410049] ata2: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
[ 109.410027] ata2.00: qc timeout (cmd 0xec)
[ 109.417694] ata2.00: failed to IDENTIFY (I/O error, err_mask=0x5)
[ 109.427351] ata2.00: revalidation failed (errno=-5)
[ 109.435733] ata2: limiting SATA link speed to 1.5 Gbps
[ 109.444384] ata2: hard resetting link
[ 111.640049] ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 10)
[ 141.640029] ata2.00: qc timeout (cmd 0xec)
[ 141.647650] ata2.00: failed to IDENTIFY (I/O error, err_mask=0x5)
[ 141.657255] ata2.00: revalidation failed (errno=-5)
[ 141.665577] ata2.00: disabled
[ 141.671956] ata2.00: device reported invalid CHS sector 0
[ 141.680792] ata2: hard resetting link
[ 143.880049] ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 10)
[ 143.880065] ata2: EH complete
[ 143.880096] sd 1:0:0:0: [sdb] Unhandled error code
[ 143.880099] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET
driverbyte=DRIVER_OK
[ 143.880106] sd 1:0:0:0: [sdb] CDB: Write(10): 2a 00 01 3e 27 fd 00 04 00 00
[ 143.880121] end_request: I/O error, dev sdb, sector 20850685

The last error gets repeated for different sectors, then the xfs
filesystem shuts itself down.

Torsten

2010-01-16 21:58:34

by Torsten Kaiser

[permalink] [raw]
Subject: Re: MSI broken in libata?

On Mon, Jan 11, 2010 at 2:39 AM, Robert Hancock <[email protected]> wrote:
> On 01/10/2010 07:15 PM, Tejun Heo wrote:
>>
>> On 01/10/2010 01:33 PM, Torsten Kaiser wrote:
>>>
>>> I did try the patch from Robert Hancock in
>>> http://lkml.org/lkml/2010/1/6/417 ,but without success.
>>>
>>> if you need any more information, or have something for me to try,
>>> please just ask. I did look at the code and the documentation about
>>> enabling MSI, but did not see anything (obvious) wrong, so I don't
>>> know what to try next.
>>
>> Can you please try the attached patch?
>>
>> Thanks.
>>
>
> It'd be interesting to see if it makes a difference, but I don't think the
> patch is quite right.

As written in the other mail: No, Tejuns patch also didn't work.

> According to the datasheet, doing the MSI ack while
> the interrupt source is still pending will cause a new MSI to be sent, so if
> you do it before handling the interrupt you'll generate a spurious interrupt
> after every real one.
>
> Though, apparently my patch that did the MSI ack after the handling didn't
> help, so either that's wrong or the problem is unrelated. (I tend to suspect
> the latter, given that sata_nv is also failing in the same way.)

Reading http://www.siliconimage.com/docs/SiI-DS-0138-D.pdf a possible
cause might have been, that this MSI ACK was never needed. Page 63 of
this PDF says about 'Global Control': "If all interrupt conditions are
removed subsequent to an MSI, it is not necessary to assert this
Acknowledge; another MSI will be generated when an interrupt condition
occurs."

But I did not find anything that might explain my problem.

Looking at my lspci output I noted the following:
For the PCIe-bridges:
Capabilities: [80] Express (v1) Root Port (Slot+), MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
For the tg3 onboard network chips:
Capabilities: [d0] Express (v1) Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 4096 bytes
For the SiI chip:
Capabilities: [70] Express (v1) Legacy Endpoint, MSI 00
DevCap: MaxPayload 1024 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 4096 bytes

So the maximum payload for it is bigger then that of the nVidia bridge.
As I don't have knowlegde of the PCI specs, I guess DevCap is what a
device is physically capable and DevCtl is the value that the BIOS /
kernel hat programmed into it for actual use.
If my guess is correct, then the SiI should be correctly limited to
128 bytes payload and that it should work.

BUT: Page 47 of the SiI-PDF says for 'Device Status and Control' the following:
Bit [14:12]: Max Read Request Size (R/W) ? Allowable values are 000B
to 011B (128 to 1024 bytes).
Default is 010B (512 bytes).

So a MaxReadReq value of 4096 as indicated by lspci for my system
would be out of bounds.

Is is important? (Somehow it seems not: In the Not-MSI-case it is also
4096 bytes, but the system works fine...)


Can I do anything else to help debug this?

Torsten

2010-01-17 19:22:50

by Robert Hancock

[permalink] [raw]
Subject: Re: MSI broken in libata?

On 01/16/2010 03:58 PM, Torsten Kaiser wrote:
> On Mon, Jan 11, 2010 at 2:39 AM, Robert Hancock<[email protected]> wrote:
>> On 01/10/2010 07:15 PM, Tejun Heo wrote:
>>>
>>> On 01/10/2010 01:33 PM, Torsten Kaiser wrote:
>>>>
>>>> I did try the patch from Robert Hancock in
>>>> http://lkml.org/lkml/2010/1/6/417 ,but without success.
>>>>
>>>> if you need any more information, or have something for me to try,
>>>> please just ask. I did look at the code and the documentation about
>>>> enabling MSI, but did not see anything (obvious) wrong, so I don't
>>>> know what to try next.
>>>
>>> Can you please try the attached patch?
>>>
>>> Thanks.
>>>
>>
>> It'd be interesting to see if it makes a difference, but I don't think the
>> patch is quite right.
>
> As written in the other mail: No, Tejuns patch also didn't work.
>
>> According to the datasheet, doing the MSI ack while
>> the interrupt source is still pending will cause a new MSI to be sent, so if
>> you do it before handling the interrupt you'll generate a spurious interrupt
>> after every real one.
>>
>> Though, apparently my patch that did the MSI ack after the handling didn't
>> help, so either that's wrong or the problem is unrelated. (I tend to suspect
>> the latter, given that sata_nv is also failing in the same way.)
>
> Reading http://www.siliconimage.com/docs/SiI-DS-0138-D.pdf a possible
> cause might have been, that this MSI ACK was never needed. Page 63 of
> this PDF says about 'Global Control': "If all interrupt conditions are
> removed subsequent to an MSI, it is not necessary to assert this
> Acknowledge; another MSI will be generated when an interrupt condition
> occurs."
>
> But I did not find anything that might explain my problem.
>
> Looking at my lspci output I noted the following:
> For the PCIe-bridges:
> Capabilities: [80] Express (v1) Root Port (Slot+), MSI 00
> DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s<64ns, L1<1us
> ExtTag- RBE+ FLReset-
> DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
> RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
> MaxPayload 128 bytes, MaxReadReq 512 bytes
> For the tg3 onboard network chips:
> Capabilities: [d0] Express (v1) Endpoint, MSI 00
> DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s<4us, L1 unlimited
> ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
> DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
> RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
> MaxPayload 128 bytes, MaxReadReq 4096 bytes
> For the SiI chip:
> Capabilities: [70] Express (v1) Legacy Endpoint, MSI 00
> DevCap: MaxPayload 1024 bytes, PhantFunc 0, Latency L0s<64ns, L1<1us
> ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
> DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
> RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
> MaxPayload 128 bytes, MaxReadReq 4096 bytes
>
> So the maximum payload for it is bigger then that of the nVidia bridge.
> As I don't have knowlegde of the PCI specs, I guess DevCap is what a
> device is physically capable and DevCtl is the value that the BIOS /
> kernel hat programmed into it for actual use.
> If my guess is correct, then the SiI should be correctly limited to
> 128 bytes payload and that it should work.
>
> BUT: Page 47 of the SiI-PDF says for 'Device Status and Control' the following:
> Bit [14:12]: Max Read Request Size (R/W) ? Allowable values are 000B
> to 011B (128 to 1024 bytes).
> Default is 010B (512 bytes).
>
> So a MaxReadReq value of 4096 as indicated by lspci for my system
> would be out of bounds.
>
> Is is important? (Somehow it seems not: In the Not-MSI-case it is also
> 4096 bytes, but the system works fine...)
>
>
> Can I do anything else to help debug this?

I don't think the MaxReadReq difference would be an issue - it's the max
request size that device is allowed to generate, not the max it can
accept. In any case, not sure how it would affect MSI since those
requests would be a write, not a read, and would be tiny. You could
always try changing it (I think setpci should be able to do it, though
you might need to dig through specs to find out which bits those are).

Unfortunately I don't have any great debug suggestions other than
those.. My first suspect would still be some kind of HT-MSI mapping
issue, but it's strange that only writes seem to be having problems..

2010-01-17 21:11:37

by Torsten Kaiser

[permalink] [raw]
Subject: Re: MSI broken in libata?

On Sun, Jan 17, 2010 at 8:22 PM, Robert Hancock <[email protected]> wrote:
> On 01/16/2010 03:58 PM, Torsten Kaiser wrote:
>> Looking at my lspci output I noted the following:
>> For the PCIe-bridges:
>> ? ? ? ?Capabilities: [80] Express (v1) Root Port (Slot+), MSI 00
>> ? ? ? ? ? ? ? ?DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency
>> L0s<64ns, L1<1us
>> ? ? ? ? ? ? ? ? ? ? ? ?ExtTag- RBE+ FLReset-
>> ? ? ? ? ? ? ? ?DevCtl: Report errors: Correctable- Non-Fatal- Fatal-
>> Unsupported-
>> ? ? ? ? ? ? ? ? ? ? ? ?RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
>> ? ? ? ? ? ? ? ? ? ? ? ?MaxPayload 128 bytes, MaxReadReq 512 bytes
>> For the tg3 onboard network chips:
>> ? ? ? ?Capabilities: [d0] Express (v1) Endpoint, MSI 00
>> ? ? ? ? ? ? ? ?DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s<4us,
>> L1 unlimited
>> ? ? ? ? ? ? ? ? ? ? ? ?ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
>> ? ? ? ? ? ? ? ?DevCtl: Report errors: Correctable- Non-Fatal- Fatal-
>> Unsupported-
>> ? ? ? ? ? ? ? ? ? ? ? ?RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
>> ? ? ? ? ? ? ? ? ? ? ? ?MaxPayload 128 bytes, MaxReadReq 4096 bytes
>> For the SiI chip:
>> ? ? ? ?Capabilities: [70] Express (v1) Legacy Endpoint, MSI 00
>> ? ? ? ? ? ? ? ?DevCap: MaxPayload 1024 bytes, PhantFunc 0, Latency
>> L0s<64ns, L1<1us
>> ? ? ? ? ? ? ? ? ? ? ? ?ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
>> ? ? ? ? ? ? ? ?DevCtl: Report errors: Correctable- Non-Fatal- Fatal-
>> Unsupported-
>> ? ? ? ? ? ? ? ? ? ? ? ?RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
>> ? ? ? ? ? ? ? ? ? ? ? ?MaxPayload 128 bytes, MaxReadReq 4096 bytes
>>
>> So the maximum payload for it is bigger then that of the nVidia bridge.
>> As I don't have knowlegde of the PCI specs, I guess DevCap is what a
>> device is physically capable and DevCtl is the value that the BIOS /
>> kernel hat programmed into it for actual use.
>> If my guess is correct, then the SiI should be correctly limited to
>> 128 bytes payload and that it should work.
>>
>> BUT: Page 47 of the SiI-PDF says for 'Device Status and Control' the
>> following:
>> Bit [14:12]: Max Read Request Size (R/W) ? Allowable values are 000B
>> to 011B (128 to 1024 bytes).
>> Default is 010B (512 bytes).
>>
>> So a MaxReadReq value of 4096 as indicated by lspci for my system
>> would be out of bounds.
>>
>> Is is important? (Somehow it seems not: In the Not-MSI-case it is also
>> 4096 bytes, but the system works fine...)
>>
>>
>> Can I do anything else to help debug this?
>
> I don't think the MaxReadReq difference would be an issue - it's the max
> request size that device is allowed to generate, not the max it can accept.
> In any case, not sure how it would affect MSI since those requests would be
> a write, not a read, and would be tiny. You could always try changing it (I
> think setpci should be able to do it, though you might need to dig through
> specs to find out which bits those are).

As the Not-MSI-case is working with the 4096 setting this does not
look very promising as a fix.
The docs from SiI did say, that this field is R/W so setting it via
setpci might work.
But I will defer poking random bytes into PCI configuration space for
later... ;-)

> Unfortunately I don't have any great debug suggestions other than those.. My
> first suspect would still be some kind of HT-MSI mapping issue, but it's
> strange that only writes seem to be having problems..

The easiest way to trigger it, was booting into a static shell via
init=/bin/sash and doing:
dd if=/dev/zero of=/XFS-FS/on/a/disk/connected/to/sii3132 bs=1k count=1M

Reading seemed to work, as even with MSI enabled I was able to mount
the fs and start several programs (like lspci). But when I tried to
write the output to a temporary file, the errors started to appear.

I did a little bit more stress testing with iozone on a ro mounted fs
and that failed too.
So it is not only writing that fails.

test1: iozone -i 1:
[ 143.010051] ata2.00: exception Emask 0x0 SAct 0x3 SErr 0x0 action 0x6 frozen
[ 143.010060] ata2.00: failed command: READ FPDMA QUEUED
[ 143.010071] ata2.00: cmd 60/00:00:ad:71:d9/01:00:01:00:00/40 tag 0
ncq 131072 in
[ 143.010073] res 40/00:00:01:4f:c2/00:00:00:00:00/00 Emask
0x4 (timeout)
[ 143.010079] ata2.00: status: { DRDY }
[ 143.010083] ata2.00: failed command: READ FPDMA QUEUED
[ 143.010092] ata2.00: cmd 60/00:08:ad:70:d9/01:00:01:00:00/40 tag 1
ncq 131072 in
[ 143.010095] res 40/00:00:00:00:00/00:00:00:00:00/00 Emask
0x4 (timeout)
[ 143.010099] ata2.00: status: { DRDY }
[ 143.010108] ata2: hard resetting link

test2: dd if=file-on-sii-attached-diskl of=/dev/null bs=1k
[ 112.950063] ata2.00: exception Emask 0x0 SAct 0x3 SErr 0x0 action 0x6 frozen
[ 112.950076] ata2.00: failed command: READ FPDMA QUEUED
[ 112.950088] ata2.00: cmd 60/00:00:ad:71:d9/01:00:01:00:00/40 tag 0
ncq 131072 in
[ 112.950091] res 40/00:00:01:4f:c2/00:00:00:00:00/00 Emask
0x4 (timeout)
[ 112.950096] ata2.00: status: { DRDY }
[ 112.950101] ata2.00: failed command: READ FPDMA QUEUED
[ 112.950110] ata2.00: cmd 60/00:08:ad:70:d9/01:00:01:00:00/40 tag 1
ncq 131072 in
[ 112.950113] res 40/00:00:00:00:00/00:00:00:00:00/00 Emask
0x4 (timeout)
[ 112.950117] ata2.00: status: { DRDY }
[ 112.950127] ata2: hard resetting link
dd said, that it read 181 Mb before failing

test3: dd if=file-on-sii-attached-diskl of=/dev/null bs=1k iflag=direct
[ 118.040055] ata2.00: exception Emask 0x0 SAct 0x1 SErr 0x0 action 0x6 frozen
[ 118.040065] ata2.00: failed command: READ FPDMA QUEUED
[ 118.040077] ata2.00: cmd 60/02:00:a5:13:d4/00:00:01:00:00/40 tag 0
ncq 1024 in
[ 118.040079] res 40/00:00:01:4f:c2/00:00:00:00:00/00 Emask
0x4 (timeout)
[ 118.040085] ata2.00: status: { DRDY }
[ 118.040095] ata2: hard resetting link
dd said, that it only copied 1.4 Mb this time

test4: setting the queue_depth to 1 and repeating test2
[ 248.950055] ata2.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
[ 248.950066] ata2.00: failed command: READ DMA
[ 248.950078] ata2.00: cmd c8/00:00:ad:79:d9/00:00:00:00:00/e1 tag 0
dma 131072 in
[ 248.950080] res 40/00:00:01:4f:c2/00:00:00:00:00/00 Emask
0x4 (timeout)
[ 248.950085] ata2.00: status: { DRDY }
[ 248.950095] ata2: hard resetting link
dd failed again at 183 Mb

All these tests where run with an unpatched 2.6.33-rc4.

What was different between -rc4 and my earlier tests was, that there
where additional error messages from do_IRQ.
sata_sil in MSI mode:
[ 2.491103] sata_sil24 0000:04:00.0: version 1.1
[ 2.491124] sata_sil24 0000:04:00.0: PCI INT A -> Link[LNEB] -> GSI
19 (level, low) -> IRQ 19
[ 2.499751] alloc irq_desc for 29 on node 0
[ 2.499752] alloc kstat_irqs on node 0
[ 2.499765] sata_sil24 0000:04:00.0: irq 29 for MSI/MSI-X
[ 2.499772] sata_sil24 0000:04:00.0: Using MSI
[ 2.504265] sata_sil24 0000:04:00.0: setting latency timer to 64
[ 2.504495] scsi0 : sata_sil24
[ 2.507735] scsi1 : sata_sil24
[ 2.510918] ata1: SATA max UDMA/100 host m128@0xefeffc00 port
0xefef8000 irq 29
[ 2.518246] ata2: SATA max UDMA/100 host m128@0xefeffc00 port
0xefefa000 irq 29
[snip]
I mount the test fs with -o ro:
[ 83.649367] XFS mounting filesystem sdb2
[ 83.757942] Ending clean XFS mount for filesystem: sdb2
I start the iozone test:
[ 112.382903] do_IRQ: 0.165 No irq handler for vector (irq -1)
[ 112.382923] do_IRQ: 3.165 No irq handler for vector (irq -1)
[ 112.382939] do_IRQ: 1.165 No irq handler for vector (irq -1)
[ 112.382957] do_IRQ: 2.165 No irq handler for vector (irq -1)
[ 143.010051] ata2.00: exception Emask 0x0 SAct 0x3 SErr 0x0 action 0x6 frozen
[ 143.010060] ata2.00: failed command: READ FPDMA QUEUED
[ 143.010071] ata2.00: cmd 60/00:00:ad:71:d9/01:00:01:00:00/40 tag 0
ncq 131072 in
[ 143.010073] res 40/00:00:01:4f:c2/00:00:00:00:00/00 Emask
0x4 (timeout)
[ 143.010079] ata2.00: status: { DRDY }
[ 143.010083] ata2.00: failed command: READ FPDMA QUEUED
[ 143.010092] ata2.00: cmd 60/00:08:ad:70:d9/01:00:01:00:00/40 tag 1
ncq 131072 in
[ 143.010095] res 40/00:00:00:00:00/00:00:00:00:00/00 Emask
0x4 (timeout)
[ 143.010099] ata2.00: status: { DRDY }
[ 143.010108] ata2: hard resetting link
[ 145.210058] ata2: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
[ 145.215060] do_IRQ: 1.165 No irq handler for vector (irq -1)
[ 145.215074] do_IRQ: 0.165 No irq handler for vector (irq -1)
[ 145.215088] do_IRQ: 2.165 No irq handler for vector (irq -1)
[ 145.215097] do_IRQ: 3.165 No irq handler for vector (irq -1)
[ 150.210036] ata2.00: qc timeout (cmd 0xec)
[ 150.210046] ata2.00: failed to IDENTIFY (I/O error, err_mask=0x5)
[ 150.210050] ata2.00: revalidation failed (errno=-5)
[ 150.210058] ata2: hard resetting link
[ 152.410050] ata2: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
[ 162.410031] ata2.00: qc timeout (cmd 0xec)
[ 162.410039] ata2.00: failed to IDENTIFY (I/O error, err_mask=0x5)
[ 162.410043] ata2.00: revalidation failed (errno=-5)
[ 162.410048] ata2: limiting SATA link speed to 1.5 Gbps
[ 162.410053] ata2: hard resetting link
[ 164.610049] ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 10)
[ 164.615197] ata2.00: failed to read native max address (err_mask=0x1)
[ 164.615203] ata2.00: HPA support seems broken, skipping HPA handling
[ 169.610037] ata2.00: qc timeout (cmd 0xef)
[ 169.610045] ata2.00: failed to set xfermode (err_mask=0x4)
[ 169.610050] ata2.00: disabled
[ 169.610071] ata2.00: device reported invalid CHS sector 0
[ 169.610091] ata2: hard resetting link
[ 171.810055] ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 10)
[ 171.810080] ata2: EH complete
[ 171.810107] sd 1:0:0:0: [sdb] Unhandled error code
[ 171.810112] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET
driverbyte=DRIVER_OK
[ 171.810118] sd 1:0:0:0: [sdb] CDB: Read(10): 28 00 01 d9 70 ad 00 01 00 00
[ 171.810132] end_request: I/O error, dev sdb, sector 31027373
[ 171.810166] sd 1:0:0:0: [sdb] Unhandled error code
[ 171.810170] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET
driverbyte=DRIVER_OK
[ 171.810175] sd 1:0:0:0: [sdb] CDB: Read(10): 28 00 01 d9 71 ad 00 01 00 00
[ 171.810188] end_request: I/O error, dev sdb, sector 31027629
[ 171.810204] sd 1:0:0:0: [sdb] Unhandled error code
[ 171.810209] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET
driverbyte=DRIVER_OK
[ 171.810215] sd 1:0:0:0: [sdb] CDB: Read(10): 28 00 01 d9 70 ad 00 00 08 00
[ 171.810229] end_request: I/O error, dev sdb, sector 31027373
[ 171.810239] sd 1:0:0:0: [sdb] READ CAPACITY(16) failed
[ 171.810245] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET
driverbyte=DRIVER_OK
[ 171.810252] sd 1:0:0:0: [sdb] Sense not available.
[ 171.810305] sd 1:0:0:0: [sdb] READ CAPACITY failed
[ 171.810309] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET
driverbyte=DRIVER_OK
[ 171.810314] sd 1:0:0:0: [sdb] Sense not available.
[ 171.810387] sd 1:0:0:0: [sdb] Asking for cache data failed
[ 171.810392] sd 1:0:0:0: [sdb] Assuming drive cache: write through
[ 171.810405] sdb: detected capacity change from 640135028736 to 0
[ 196.165461] sd 1:0:0:0: [sdb] READ CAPACITY(16) failed
[ 196.165468] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET
driverbyte=DRIVER_OK
[ 196.165475] sd 1:0:0:0: [sdb] Sense not available.
[ 196.165531] sd 1:0:0:0: [sdb] READ CAPACITY failed
[ 196.165535] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET
driverbyte=DRIVER_OK
[ 196.165540] sd 1:0:0:0: [sdb] Sense not available.
[ 196.165617] sd 1:0:0:0: [sdb] Asking for cache data failed
[ 196.165622] sd 1:0:0:0: [sdb] Assuming drive cache: write through

I will look, if I can find out, where the do_IRQ error comes from and
why I didn't see it with -rc1 to -rc3.

Torsten

2010-01-18 20:51:39

by Torsten Kaiser

[permalink] [raw]
Subject: Re: MSI broken in libata?

On Sun, Jan 17, 2010 at 10:11 PM, Torsten Kaiser
<[email protected]> wrote:
> What was different between -rc4 and my earlier tests was, that there
> where additional error messages from do_IRQ.

> [ ?112.382903] do_IRQ: 0.165 No irq handler for vector (irq -1)
> [ ?112.382923] do_IRQ: 3.165 No irq handler for vector (irq -1)
> [ ?112.382939] do_IRQ: 1.165 No irq handler for vector (irq -1)
> [ ?112.382957] do_IRQ: 2.165 No irq handler for vector (irq -1)
> [ ?143.010051] ata2.00: exception Emask 0x0 SAct 0x3 SErr 0x0 action 0x6 frozen
> [ ?143.010060] ata2.00: failed command: READ FPDMA QUEUED
> [ ?143.010071] ata2.00: cmd 60/00:00:ad:71:d9/01:00:01:00:00/40 tag 0
> ncq 131072 in
> [ ?143.010073] ? ? ? ? ?res 40/00:00:01:4f:c2/00:00:00:00:00/00 Emask
> 0x4 (timeout)
> [ ?143.010079] ata2.00: status: { DRDY }
> [ ?143.010083] ata2.00: failed command: READ FPDMA QUEUED
> [ ?143.010092] ata2.00: cmd 60/00:08:ad:70:d9/01:00:01:00:00/40 tag 1
> ncq 131072 in
> [ ?143.010095] ? ? ? ? ?res 40/00:00:00:00:00/00:00:00:00:00/00 Emask
> 0x4 (timeout)
> [ ?143.010099] ata2.00: status: { DRDY }
> [ ?143.010108] ata2: hard resetting link

> I will look, if I can find out, where the do_IRQ error comes from and
> why I didn't see it with -rc1 to -rc3.

OK, this has nothing to do with -rc3 vs. -rc4.
If I kill the drive with reading a 1GB file with dd, I get the above
errors, include the one from do_IRQ.
If I kill the drive with writing a 1GB file with dd, I do not get the
do_IRQ error.
As I tested the earlier -rcs only with writes, I didn't see it there.
(Its not even in the old logs that contain the libata errors)

Should I try to put some printks in arch/x86/kernel/apic/io_apic.c
__assign_irq_vector() or is there a better way to look at the irq
vectors?

Torsten

2010-01-19 02:04:14

by Robert Hancock

[permalink] [raw]
Subject: Re: MSI broken in libata?

On 01/18/2010 02:51 PM, Torsten Kaiser wrote:
> On Sun, Jan 17, 2010 at 10:11 PM, Torsten Kaiser
> <[email protected]> wrote:
>> What was different between -rc4 and my earlier tests was, that there
>> where additional error messages from do_IRQ.
>
>> [ 112.382903] do_IRQ: 0.165 No irq handler for vector (irq -1)
>> [ 112.382923] do_IRQ: 3.165 No irq handler for vector (irq -1)
>> [ 112.382939] do_IRQ: 1.165 No irq handler for vector (irq -1)
>> [ 112.382957] do_IRQ: 2.165 No irq handler for vector (irq -1)
>> [ 143.010051] ata2.00: exception Emask 0x0 SAct 0x3 SErr 0x0 action 0x6 frozen
>> [ 143.010060] ata2.00: failed command: READ FPDMA QUEUED
>> [ 143.010071] ata2.00: cmd 60/00:00:ad:71:d9/01:00:01:00:00/40 tag 0
>> ncq 131072 in
>> [ 143.010073] res 40/00:00:01:4f:c2/00:00:00:00:00/00 Emask
>> 0x4 (timeout)
>> [ 143.010079] ata2.00: status: { DRDY }
>> [ 143.010083] ata2.00: failed command: READ FPDMA QUEUED
>> [ 143.010092] ata2.00: cmd 60/00:08:ad:70:d9/01:00:01:00:00/40 tag 1
>> ncq 131072 in
>> [ 143.010095] res 40/00:00:00:00:00/00:00:00:00:00/00 Emask
>> 0x4 (timeout)
>> [ 143.010099] ata2.00: status: { DRDY }
>> [ 143.010108] ata2: hard resetting link
>
>> I will look, if I can find out, where the do_IRQ error comes from and
>> why I didn't see it with -rc1 to -rc3.
>
> OK, this has nothing to do with -rc3 vs. -rc4.
> If I kill the drive with reading a 1GB file with dd, I get the above
> errors, include the one from do_IRQ.
> If I kill the drive with writing a 1GB file with dd, I do not get the
> do_IRQ error.
> As I tested the earlier -rcs only with writes, I didn't see it there.
> (Its not even in the old logs that contain the libata errors)
>
> Should I try to put some printks in arch/x86/kernel/apic/io_apic.c
> __assign_irq_vector() or is there a better way to look at the irq
> vectors?

Might not be a bad idea. Something definitely seems to be banging on
vector 165. Any references to 165 in dmesg prior to that? I wonder where
that is coming from..

2010-01-19 07:00:47

by Torsten Kaiser

[permalink] [raw]
Subject: Re: MSI broken in libata?

On Tue, Jan 19, 2010 at 3:03 AM, Robert Hancock <[email protected]> wrote:
> On 01/18/2010 02:51 PM, Torsten Kaiser wrote:
>> Should I try to put some printks in arch/x86/kernel/apic/io_apic.c
>> __assign_irq_vector() or is there a better way to look at the irq
>> vectors?
>
> Might not be a bad idea.

OK, I will try it later today.

> Something definitely seems to be banging on vector
> 165. Any references to 165 in dmesg prior to that? I wonder where that is
> coming from..

No, grepping for 165 only shows some timestamps and some output from
the serial driver, because that's a 16550A.

I hope adding these debugs will help, maybe something suspicious
reserved 164 or 166...


Torsten

2010-01-19 20:20:56

by Torsten Kaiser

[permalink] [raw]
Subject: Re: MSI broken in libata?

On Tue, Jan 19, 2010 at 3:03 AM, Robert Hancock <[email protected]> wrote:
> On 01/18/2010 02:51 PM, Torsten Kaiser wrote:
>> If I kill the drive with reading a 1GB file with dd, I get the above
>> errors, include the one from do_IRQ.
>> If I kill the drive with writing a 1GB file with dd, I do not get the
>> do_IRQ error.
>> As I tested the earlier -rcs only with writes, I didn't see it there.
>> (Its not even in the old logs that contain the libata errors)
>>
>> Should I try to put some printks in arch/x86/kernel/apic/io_apic.c
>> __assign_irq_vector() or is there a better way to look at the irq
>> vectors?
>
> Might not be a bad idea. Something definitely seems to be banging on vector
> 165. Any references to 165 in dmesg prior to that? I wonder where that is
> coming from..

I still have no clue, where that interrupt comes from, maybe these
outputs will help:
(complete log attached)

First the PCIe bridges allocate some vectors:
[ 1.789044] pcieport 0000:00:0b.0: setting latency timer to 64
[ 1.789058] alloc irq_desc for 24 on node 0
[ 1.789060] alloc kstat_irqs on node 0
[ 1.789063] __assign_irq_vector:assigning irq 24 to vector 73
[ 1.794655] pcieport 0000:00:0b.0: irq 24 for MSI/MSI-X
[ 1.794968] pcieport 0000:00:0c.0: setting latency timer to 64
[ 1.794980] alloc irq_desc for 25 on node 0
[ 1.794981] alloc kstat_irqs on node 0
[ 1.794983] __assign_irq_vector:assigning irq 25 to vector 81
[ 1.800569] pcieport 0000:00:0c.0: irq 25 for MSI/MSI-X
[ 1.800870] pcieport 0000:00:0d.0: setting latency timer to 64
[ 1.800882] alloc irq_desc for 26 on node 0
[ 1.800883] alloc kstat_irqs on node 0
[ 1.800885] __assign_irq_vector:assigning irq 26 to vector 89
[ 1.806455] pcieport 0000:00:0d.0: irq 26 for MSI/MSI-X
[ 1.806766] pcieport 0000:00:0f.0: setting latency timer to 64
[ 1.806778] alloc irq_desc for 27 on node 0
[ 1.806779] alloc kstat_irqs on node 0
[ 1.806781] __assign_irq_vector:assigning irq 27 to vector 97
[ 1.812366] pcieport 0000:00:0f.0: irq 27 for MSI/MSI-X
radeon allocates to vectors, is that right?
[ 1.861528] [drm] radeon kernel modesetting enabled.
[ 1.866835] ACPI: PCI Interrupt Link [LNEB] enabled at IRQ 19
[ 1.872598] alloc irq_desc for 19 on node 0
[ 1.872600] alloc kstat_irqs on node 0
[ 1.872602] __assign_irq_vector:assigning irq 19 to vector 105
[ 1.878261] radeon 0000:01:00.0: PCI INT A -> Link[LNEB] -> GSI 19
(level, low) -> IRQ 19
[ 1.886673] radeon 0000:01:00.0: setting latency timer to 64
[ 1.888791] [drm] radeon: Initializing kernel modesetting.
[ 1.894449] [drm] register mmio base: 0xEFBF0000
[ 1.899068] [drm] register mmio size: 65536
[ 1.903947] [drm] GPU reset succeed (RBBM_STATUS=0x00000140)
[ 1.909620] [drm] Generation 2 PCI interface, using max accessible memory
[ 1.916436] [drm] radeon: VRAM 128M
[ 1.919930] [drm] radeon: VRAM from 0x00000000 to 0x07FFFFFF
[ 1.925610] [drm] radeon: GTT 512M
[ 1.929022] [drm] radeon: GTT from 0x20000000 to 0x3FFFFFFF
[ 1.934635] alloc irq_desc for 28 on node 0
[ 1.934637] alloc kstat_irqs on node 0
[ 1.934640] __assign_irq_vector:assigning irq 28 to vector 113
[ 1.940314] radeon 0000:01:00.0: irq 28 for MSI/MSI-X
[ 1.940522] [drm] radeon: using MSI.
[ 1.944120] [drm] radeon: irq initialized.
Now sata_sil24 gets vector 121:
[ 2.525316] sata_sil24 0000:04:00.0: version 1.1
[ 2.525337] sata_sil24 0000:04:00.0: PCI INT A -> Link[LNEB] -> GSI
19 (level, low)
-> IRQ 19
[ 2.533990] alloc irq_desc for 29 on node 0
[ 2.533991] alloc kstat_irqs on node 0
[ 2.533994] __assign_irq_vector:assigning irq 29 to vector 121
[ 2.539681] sata_sil24 0000:04:00.0: irq 29 for MSI/MSI-X
[ 2.539882] sata_sil24 0000:04:00.0: Using MSI
[ 2.544372] sata_sil24 0000:04:00.0: setting latency timer to 64
sata_nv gets 3 interrupts (the MCP55 has 6 ports)
[ 2.565829] sata_nv 0000:00:05.0: version 3.5
[ 2.566114] ACPI: PCI Interrupt Link [LSA0] enabled at IRQ 23
[ 2.571896] alloc irq_desc for 23 on node 0
[ 2.571898] alloc kstat_irqs on node 0
[ 2.571900] __assign_irq_vector:assigning irq 23 to vector 129
[ 2.577583] sata_nv 0000:00:05.0: PCI INT A -> Link[LSA0] -> GSI 23
(level, low) ->
IRQ 23
[ 2.586099] sata_nv 0000:00:05.0: Using SWNCQ mode
[ 2.590956] sata_nv 0000:00:05.0: setting latency timer to 64
[ 2.591071] scsi2 : sata_nv
[ 2.594013] scsi3 : sata_nv
[ 2.597018] ata3: SATA max UDMA/133 cmd 0xcc00 ctl 0xc880 bmdma 0xc400 irq 23
[ 2.604196] ata4: SATA max UDMA/133 cmd 0xc800 ctl 0xc480 bmdma 0xc408 irq 23
[ 2.611671] ACPI: PCI Interrupt Link [LSA1] enabled at IRQ 22
[ 2.617435] alloc irq_desc for 22 on node 0
[ 2.617436] alloc kstat_irqs on node 0
[ 2.617438] __assign_irq_vector:assigning irq 22 to vector 137
[ 2.623152] sata_nv 0000:00:05.1: PCI INT B -> Link[LSA1] -> GSI 22
(level, low) ->
IRQ 22
[ 2.631657] sata_nv 0000:00:05.1: Using SWNCQ mode
[ 2.636490] sata_nv 0000:00:05.1: setting latency timer to 64
[ 2.636618] scsi4 : sata_nv
[ 2.639558] scsi5 : sata_nv
[ 2.642573] ata5: SATA max UDMA/133 cmd 0xc080 ctl 0xc000 bmdma 0xb800 irq 22
[ 2.649726] ata6: SATA max UDMA/133 cmd 0xbc00 ctl 0xb880 bmdma 0xb808 irq 22
[ 2.657246] ACPI: PCI Interrupt Link [LSA2] enabled at IRQ 21
[ 2.663034] alloc irq_desc for 21 on node 0
[ 2.663035] alloc kstat_irqs on node 0
[ 2.663037] __assign_irq_vector:assigning irq 21 to vector 145
[ 2.668720] sata_nv 0000:00:05.2: PCI INT C -> Link[LSA2] -> GSI 21
(level, low) -> IRQ 21
[ 2.677237] sata_nv 0000:00:05.2: Using SWNCQ mode
[ 2.682085] sata_nv 0000:00:05.2: setting latency timer to 64
[ 2.682204] scsi6 : sata_nv
[ 2.685148] scsi7 : sata_nv
[ 2.688149] ata7: SATA max UDMA/133 cmd 0xb480 ctl 0xb400 bmdma 0xac00 irq 21
[ 2.695325] ata8: SATA max UDMA/133 cmd 0xb080 ctl 0xb000 bmdma 0xac08 irq 21
the network chips follow:
[ 2.729649] tg3.c:v3.105 (December 2, 2009)
[ 2.735339] ACPI: PCI Interrupt Link [LNEA] enabled at IRQ 18
[ 2.742377] alloc irq_desc for 18 on node 0
[ 2.742378] alloc kstat_irqs on node 0
[ 2.742380] __assign_irq_vector:assigning irq 18 to vector 153
[ 2.748065] tg3 0000:03:00.0: PCI INT A -> Link[LNEA] -> GSI 18
(level, low) -> IRQ 18
...
[ 2.838517] ACPI: PCI Interrupt Link [LNED] enabled at IRQ 17
[ 2.845796] alloc irq_desc for 17 on node 0
[ 2.845797] alloc kstat_irqs on node 0
[ 2.845799] __assign_irq_vector:assigning irq 17 to vector 161
[ 2.851501] tg3 0000:02:00.0: PCI INT A -> Link[LNED] -> GSI 17
(level, low) -> IRQ 17
one for the firewrie chip:
[ 2.939495] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 16
[ 2.946971] alloc irq_desc for 16 on node 0
[ 2.946972] alloc kstat_irqs on node 0
[ 2.946974] __assign_irq_vector:assigning irq 16 to vector 169
[ 2.952677] ohci1394 0000:05:08.0: PCI INT A -> Link[LNKA] -> GSI
16 (level, low) -> IRQ 16
USB:
[ 3.058354] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 3.066956] ACPI: PCI Interrupt Link [LUB2] enabled at IRQ 20
[ 3.074517] alloc irq_desc for 20 on node 0
[ 3.074518] alloc kstat_irqs on node 0
[ 3.074520] __assign_irq_vector:assigning irq 20 to vector 177
[ 3.080223] ehci_hcd 0000:00:02.1: PCI INT B -> Link[LUB2] -> GSI
20 (level, low) -> IRQ 20
and the last allocation for HDA Intel:
[ 4.993274] alloc irq_desc for 30 on node 0
[ 4.993276] alloc kstat_irqs on node 0
[ 4.993279] __assign_irq_vector:assigning irq 30 to vector 185
[ 4.998961] HDA Intel 0000:00:06.1: irq 30 for MSI/MSI-X
During booting the tg3's change there interrupts. Huh?
[ 19.135631] alloc irq_desc for 31 on node 0
[ 19.135639] alloc kstat_irqs on node 0
[ 19.135646] __assign_irq_vector:assigning irq 31 to vector 193
[ 19.135663] tg3 0000:03:00.0: irq 31 for MSI/MSI-X
[ 22.326847] tg3: eth0: Link is up at 1000 Mbps, full duplex.
[ 22.326854] tg3: eth0: Flow control is on for TX and on for RX.
[ 23.434378] alloc irq_desc for 32 on node 0
[ 23.434386] alloc kstat_irqs on node 0
[ 23.434393] __assign_irq_vector:assigning irq 32 to vector 201
[ 23.434410] tg3 0000:02:00.0: irq 32 for MSI/MSI-X
[ 26.104711] tg3: eth1: Link is up at 1000 Mbps, full duplex.
[ 26.104718] tg3: eth1: Flow control is on for TX and on for RX.

Then I rerun my readtest and got the same do_IRQ error again:
[ 100.664581] do_IRQ: 0.165 No irq handler for vector (irq -1)
[ 100.664598] do_IRQ: 1.165 No irq handler for vector (irq -1)
[ 100.664619] do_IRQ: 3.165 No irq handler for vector (irq -1)
[ 100.664632] do_IRQ: 2.165 No irq handler for vector (irq -1)
[ 130.980054] ata2.00: exception Emask 0x0 SAct 0x3 SErr 0x0 action 0x6 frozen
[ 130.980065] ata2.00: failed command: READ FPDMA QUEUED
[ 130.980077] ata2.00: cmd 60/00:00:ad:9b:da/01:00:01:00:00/40 tag 0
ncq 131072 in
[ 130.980080] res 40/00:00:01:4f:c2/00:00:00:00:00/00 Emask
0x4 (timeout)
[ 130.980085] ata2.00: status: { DRDY }
[ 130.980090] ata2.00: failed command: READ FPDMA QUEUED
[ 130.980099] ata2.00: cmd 60/00:08:ad:9c:da/01:00:01:00:00/40 tag 1
ncq 131072 in
[ 130.980101] res 40/00:00:00:00:00/00:00:00:00:00/00 Emask
0x4 (timeout)
[ 130.980106] ata2.00: status: { DRDY }

As I did not see any real relation, I disabled all unneeded drivers
and tested again.
This minimal kernel had the following interrupt map:
CPU0 CPU1 CPU2 CPU3
0: 36 0 0 1 IO-APIC-edge timer
1: 0 0 1 152 IO-APIC-edge i8042
4: 0 0 0 212 IO-APIC-edge serial
7: 1 0 0 0 IO-APIC-edge
9: 0 0 0 0 IO-APIC-fasteoi acpi
12: 0 0 0 4 IO-APIC-edge i8042
21: 0 0 0 0 IO-APIC-fasteoi sata_nv
22: 0 0 0 0 IO-APIC-fasteoi sata_nv
23: 0 5 4 4980 IO-APIC-fasteoi sata_nv
28: 0 1 0 168 PCI-MSI-edge sata_sil24
NMI: 0 0 0 0 Non-maskable interrupts
LOC: 2147 2390 2285 2344 Local timer interrupts
SPU: 0 0 0 0 Spurious interrupts
PMI: 0 0 0 0 Performance
monitoring interrupts
PND: 0 0 0 0 Performance pending work
RES: 3702 3458 2094 2746 Rescheduling interrupts
CAL: 660 33 41 26 Function call interrupts
TLB: 484 200 448 581 TLB shootdowns
THR: 0 0 0 0 Threshold APIC interrupts
MCE: 0 0 0 0 Machine check exceptions
MCP: 1 1 1 1 Machine check polls
ERR: 1
MIS: 0

Output of the __assign_irq:
4x PCIe bridge:
[ 1.759016] __assign_irq_vector:assigning irq 24 to vector 73
[ 1.764928] __assign_irq_vector:assigning irq 25 to vector 81
[ 1.770827] __assign_irq_vector:assigning irq 26 to vector 89
[ 1.776720] __assign_irq_vector:assigning irq 27 to vector 97
sata_sil24 first gets 19:
[ 1.856245] __assign_irq_vector:assigning irq 19 to vector 105
then switches to MSI-28:
[ 1.870768] __assign_irq_vector:assigning irq 28 to vector 113
3 normal interrupts for sata_nv:
[ 1.908504] __assign_irq_vector:assigning irq 23 to vector 121
[ 1.953848] __assign_irq_vector:assigning irq 22 to vector 129
[ 1.999155] __assign_irq_vector:assigning irq 21 to vector 137

Writing failed again without a message, but the read test showed:
[ 85.695745] do_IRQ: 0.165 No irq handler for vector (irq -1)
[ 85.695763] do_IRQ: 1.165 No irq handler for vector (irq -1)
[ 85.695785] do_IRQ: 3.165 No irq handler for vector (irq -1)
[ 85.695800] do_IRQ: 2.165 No irq handler for vector (irq -1)
Same vector in this error, but sata_sil24 had MSI irq 28 instead of 29!
And the vector allocations where not even near 165!


Torsten


Attachments:
dmesg-vector-debug.txt (56.74 kB)

2010-01-20 03:00:43

by Robert Hancock

[permalink] [raw]
Subject: Re: MSI broken in libata?

On Tue, Jan 19, 2010 at 2:20 PM, Torsten Kaiser
<[email protected]> wrote:
> On Tue, Jan 19, 2010 at 3:03 AM, Robert Hancock <[email protected]> wrote:
>> On 01/18/2010 02:51 PM, Torsten Kaiser wrote:
>>> If I kill the drive with reading a 1GB file with dd, I get the above
>>> errors, include the one from do_IRQ.
>>> If I kill the drive with writing a 1GB file with dd, I do not get the
>>> do_IRQ error.
>>> As I tested the earlier -rcs only with writes, I didn't see it there.
>>> (Its not even in the old logs that contain the libata errors)
>>>
>>> Should I try to put some printks in arch/x86/kernel/apic/io_apic.c
>>> __assign_irq_vector() or is there a better way to look at the irq
>>> vectors?
>>
>> Might not be a bad idea. Something definitely seems to be banging on vector
>> 165. Any references to 165 in dmesg prior to that? I wonder where that is
>> coming from..
>
> I still have no clue, where that interrupt comes from, maybe these
> outputs will help:
> (complete log attached)
>
> First the PCIe bridges allocate some vectors:
> [ ? ?1.789044] pcieport 0000:00:0b.0: setting latency timer to 64
> [ ? ?1.789058] ? alloc irq_desc for 24 on node 0
> [ ? ?1.789060] ? alloc kstat_irqs on node 0
> [ ? ?1.789063] __assign_irq_vector:assigning irq 24 to vector 73
> [ ? ?1.794655] pcieport 0000:00:0b.0: irq 24 for MSI/MSI-X
> [ ? ?1.794968] pcieport 0000:00:0c.0: setting latency timer to 64
> [ ? ?1.794980] ? alloc irq_desc for 25 on node 0
> [ ? ?1.794981] ? alloc kstat_irqs on node 0
> [ ? ?1.794983] __assign_irq_vector:assigning irq 25 to vector 81
> [ ? ?1.800569] pcieport 0000:00:0c.0: irq 25 for MSI/MSI-X
> [ ? ?1.800870] pcieport 0000:00:0d.0: setting latency timer to 64
> [ ? ?1.800882] ? alloc irq_desc for 26 on node 0
> [ ? ?1.800883] ? alloc kstat_irqs on node 0
> [ ? ?1.800885] __assign_irq_vector:assigning irq 26 to vector 89
> [ ? ?1.806455] pcieport 0000:00:0d.0: irq 26 for MSI/MSI-X
> [ ? ?1.806766] pcieport 0000:00:0f.0: setting latency timer to 64
> [ ? ?1.806778] ? alloc irq_desc for 27 on node 0
> [ ? ?1.806779] ? alloc kstat_irqs on node 0
> [ ? ?1.806781] __assign_irq_vector:assigning irq 27 to vector 97
> [ ? ?1.812366] pcieport 0000:00:0f.0: irq 27 for MSI/MSI-X
> radeon allocates to vectors, is that right?
> [ ? ?1.861528] [drm] radeon kernel modesetting enabled.
> [ ? ?1.866835] ACPI: PCI Interrupt Link [LNEB] enabled at IRQ 19
> [ ? ?1.872598] ? alloc irq_desc for 19 on node 0
> [ ? ?1.872600] ? alloc kstat_irqs on node 0
> [ ? ?1.872602] __assign_irq_vector:assigning irq 19 to vector 105
> [ ? ?1.878261] radeon 0000:01:00.0: PCI INT A -> Link[LNEB] -> GSI 19
> (level, low) -> IRQ 19
> [ ? ?1.886673] radeon 0000:01:00.0: setting latency timer to 64
> [ ? ?1.888791] [drm] radeon: Initializing kernel modesetting.
> [ ? ?1.894449] [drm] register mmio base: 0xEFBF0000
> [ ? ?1.899068] [drm] register mmio size: 65536
> [ ? ?1.903947] [drm] GPU reset succeed (RBBM_STATUS=0x00000140)
> [ ? ?1.909620] [drm] Generation 2 PCI interface, using max accessible memory
> [ ? ?1.916436] [drm] radeon: VRAM 128M
> [ ? ?1.919930] [drm] radeon: VRAM from 0x00000000 to 0x07FFFFFF
> [ ? ?1.925610] [drm] radeon: GTT 512M
> [ ? ?1.929022] [drm] radeon: GTT from 0x20000000 to 0x3FFFFFFF
> [ ? ?1.934635] ? alloc irq_desc for 28 on node 0
> [ ? ?1.934637] ? alloc kstat_irqs on node 0
> [ ? ?1.934640] __assign_irq_vector:assigning irq 28 to vector 113
> [ ? ?1.940314] radeon 0000:01:00.0: irq 28 for MSI/MSI-X
> [ ? ?1.940522] [drm] radeon: using MSI.
> [ ? ?1.944120] [drm] radeon: irq initialized.
> Now sata_sil24 gets vector 121:
> [ ? ?2.525316] sata_sil24 0000:04:00.0: version 1.1
> [ ? ?2.525337] sata_sil24 0000:04:00.0: PCI INT A -> Link[LNEB] -> GSI
> 19 (level, low)
> -> IRQ 19
> [ ? ?2.533990] ? alloc irq_desc for 29 on node 0
> [ ? ?2.533991] ? alloc kstat_irqs on node 0
> [ ? ?2.533994] __assign_irq_vector:assigning irq 29 to vector 121
> [ ? ?2.539681] sata_sil24 0000:04:00.0: irq 29 for MSI/MSI-X
> [ ? ?2.539882] sata_sil24 0000:04:00.0: Using MSI
> [ ? ?2.544372] sata_sil24 0000:04:00.0: setting latency timer to 64
> sata_nv gets 3 interrupts (the MCP55 has 6 ports)
> [ ? ?2.565829] sata_nv 0000:00:05.0: version 3.5
> [ ? ?2.566114] ACPI: PCI Interrupt Link [LSA0] enabled at IRQ 23
> [ ? ?2.571896] ? alloc irq_desc for 23 on node 0
> [ ? ?2.571898] ? alloc kstat_irqs on node 0
> [ ? ?2.571900] __assign_irq_vector:assigning irq 23 to vector 129
> [ ? ?2.577583] sata_nv 0000:00:05.0: PCI INT A -> Link[LSA0] -> GSI 23
> (level, low) ->
> IRQ 23
> [ ? ?2.586099] sata_nv 0000:00:05.0: Using SWNCQ mode
> [ ? ?2.590956] sata_nv 0000:00:05.0: setting latency timer to 64
> [ ? ?2.591071] scsi2 : sata_nv
> [ ? ?2.594013] scsi3 : sata_nv
> [ ? ?2.597018] ata3: SATA max UDMA/133 cmd 0xcc00 ctl 0xc880 bmdma 0xc400 irq 23
> [ ? ?2.604196] ata4: SATA max UDMA/133 cmd 0xc800 ctl 0xc480 bmdma 0xc408 irq 23
> [ ? ?2.611671] ACPI: PCI Interrupt Link [LSA1] enabled at IRQ 22
> [ ? ?2.617435] ? alloc irq_desc for 22 on node 0
> [ ? ?2.617436] ? alloc kstat_irqs on node 0
> [ ? ?2.617438] __assign_irq_vector:assigning irq 22 to vector 137
> [ ? ?2.623152] sata_nv 0000:00:05.1: PCI INT B -> Link[LSA1] -> GSI 22
> (level, low) ->
> IRQ 22
> [ ? ?2.631657] sata_nv 0000:00:05.1: Using SWNCQ mode
> [ ? ?2.636490] sata_nv 0000:00:05.1: setting latency timer to 64
> [ ? ?2.636618] scsi4 : sata_nv
> [ ? ?2.639558] scsi5 : sata_nv
> [ ? ?2.642573] ata5: SATA max UDMA/133 cmd 0xc080 ctl 0xc000 bmdma 0xb800 irq 22
> [ ? ?2.649726] ata6: SATA max UDMA/133 cmd 0xbc00 ctl 0xb880 bmdma 0xb808 irq 22
> [ ? ?2.657246] ACPI: PCI Interrupt Link [LSA2] enabled at IRQ 21
> [ ? ?2.663034] ? alloc irq_desc for 21 on node 0
> [ ? ?2.663035] ? alloc kstat_irqs on node 0
> [ ? ?2.663037] __assign_irq_vector:assigning irq 21 to vector 145
> [ ? ?2.668720] sata_nv 0000:00:05.2: PCI INT C -> Link[LSA2] -> GSI 21
> (level, low) -> IRQ 21
> [ ? ?2.677237] sata_nv 0000:00:05.2: Using SWNCQ mode
> [ ? ?2.682085] sata_nv 0000:00:05.2: setting latency timer to 64
> [ ? ?2.682204] scsi6 : sata_nv
> [ ? ?2.685148] scsi7 : sata_nv
> [ ? ?2.688149] ata7: SATA max UDMA/133 cmd 0xb480 ctl 0xb400 bmdma 0xac00 irq 21
> [ ? ?2.695325] ata8: SATA max UDMA/133 cmd 0xb080 ctl 0xb000 bmdma 0xac08 irq 21
> the network chips follow:
> [ ? ?2.729649] tg3.c:v3.105 (December 2, 2009)
> [ ? ?2.735339] ACPI: PCI Interrupt Link [LNEA] enabled at IRQ 18
> [ ? ?2.742377] ? alloc irq_desc for 18 on node 0
> [ ? ?2.742378] ? alloc kstat_irqs on node 0
> [ ? ?2.742380] __assign_irq_vector:assigning irq 18 to vector 153
> [ ? ?2.748065] tg3 0000:03:00.0: PCI INT A -> Link[LNEA] -> GSI 18
> (level, low) -> IRQ 18
> ...
> [ ? ?2.838517] ACPI: PCI Interrupt Link [LNED] enabled at IRQ 17
> [ ? ?2.845796] ? alloc irq_desc for 17 on node 0
> [ ? ?2.845797] ? alloc kstat_irqs on node 0
> [ ? ?2.845799] __assign_irq_vector:assigning irq 17 to vector 161
> [ ? ?2.851501] tg3 0000:02:00.0: PCI INT A -> Link[LNED] -> GSI 17
> (level, low) -> IRQ 17
> one for the firewrie chip:
> [ ? ?2.939495] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 16
> [ ? ?2.946971] ? alloc irq_desc for 16 on node 0
> [ ? ?2.946972] ? alloc kstat_irqs on node 0
> [ ? ?2.946974] __assign_irq_vector:assigning irq 16 to vector 169
> [ ? ?2.952677] ohci1394 0000:05:08.0: PCI INT A -> Link[LNKA] -> GSI
> 16 (level, low) -> IRQ 16
> USB:
> [ ? ?3.058354] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
> [ ? ?3.066956] ACPI: PCI Interrupt Link [LUB2] enabled at IRQ 20
> [ ? ?3.074517] ? alloc irq_desc for 20 on node 0
> [ ? ?3.074518] ? alloc kstat_irqs on node 0
> [ ? ?3.074520] __assign_irq_vector:assigning irq 20 to vector 177
> [ ? ?3.080223] ehci_hcd 0000:00:02.1: PCI INT B -> Link[LUB2] -> GSI
> 20 (level, low) -> IRQ 20
> and the last allocation for HDA Intel:
> [ ? ?4.993274] ? alloc irq_desc for 30 on node 0
> [ ? ?4.993276] ? alloc kstat_irqs on node 0
> [ ? ?4.993279] __assign_irq_vector:assigning irq 30 to vector 185
> [ ? ?4.998961] HDA Intel 0000:00:06.1: irq 30 for MSI/MSI-X
> During booting the tg3's change there interrupts. Huh?
> [ ? 19.135631] ? alloc irq_desc for 31 on node 0
> [ ? 19.135639] ? alloc kstat_irqs on node 0
> [ ? 19.135646] __assign_irq_vector:assigning irq 31 to vector 193
> [ ? 19.135663] tg3 0000:03:00.0: irq 31 for MSI/MSI-X
> [ ? 22.326847] tg3: eth0: Link is up at 1000 Mbps, full duplex.
> [ ? 22.326854] tg3: eth0: Flow control is on for TX and on for RX.
> [ ? 23.434378] ? alloc irq_desc for 32 on node 0
> [ ? 23.434386] ? alloc kstat_irqs on node 0
> [ ? 23.434393] __assign_irq_vector:assigning irq 32 to vector 201
> [ ? 23.434410] tg3 0000:02:00.0: irq 32 for MSI/MSI-X
> [ ? 26.104711] tg3: eth1: Link is up at 1000 Mbps, full duplex.
> [ ? 26.104718] tg3: eth1: Flow control is on for TX and on for RX.
>
> Then I rerun my readtest and got the same do_IRQ error again:
> [ ?100.664581] do_IRQ: 0.165 No irq handler for vector (irq -1)
> [ ?100.664598] do_IRQ: 1.165 No irq handler for vector (irq -1)
> [ ?100.664619] do_IRQ: 3.165 No irq handler for vector (irq -1)
> [ ?100.664632] do_IRQ: 2.165 No irq handler for vector (irq -1)
> [ ?130.980054] ata2.00: exception Emask 0x0 SAct 0x3 SErr 0x0 action 0x6 frozen
> [ ?130.980065] ata2.00: failed command: READ FPDMA QUEUED
> [ ?130.980077] ata2.00: cmd 60/00:00:ad:9b:da/01:00:01:00:00/40 tag 0
> ncq 131072 in
> [ ?130.980080] ? ? ? ? ?res 40/00:00:01:4f:c2/00:00:00:00:00/00 Emask
> 0x4 (timeout)
> [ ?130.980085] ata2.00: status: { DRDY }
> [ ?130.980090] ata2.00: failed command: READ FPDMA QUEUED
> [ ?130.980099] ata2.00: cmd 60/00:08:ad:9c:da/01:00:01:00:00/40 tag 1
> ncq 131072 in
> [ ?130.980101] ? ? ? ? ?res 40/00:00:00:00:00/00:00:00:00:00/00 Emask
> 0x4 (timeout)
> [ ?130.980106] ata2.00: status: { DRDY }
>
> As I did not see any real relation, I disabled all unneeded drivers
> and tested again.
> This minimal kernel had the following interrupt map:
> ? ? ? ? ? CPU0 ? ? ? CPU1 ? ? ? CPU2 ? ? ? CPU3
> ?0: ? ? ? ? 36 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?1 ? IO-APIC-edge ? ? ?timer
> ?1: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?1 ? ? ? ?152 ? IO-APIC-edge ? ? ?i8042
> ?4: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ?212 ? IO-APIC-edge ? ? ?serial
> ?7: ? ? ? ? ?1 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? IO-APIC-edge
> ?9: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? IO-APIC-fasteoi ? acpi
> ?12: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?4 ? IO-APIC-edge ? ? ?i8042
> ?21: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? IO-APIC-fasteoi ? sata_nv
> ?22: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? IO-APIC-fasteoi ? sata_nv
> ?23: ? ? ? ? ?0 ? ? ? ? ?5 ? ? ? ? ?4 ? ? ? 4980 ? IO-APIC-fasteoi ? sata_nv
> ?28: ? ? ? ? ?0 ? ? ? ? ?1 ? ? ? ? ?0 ? ? ? ?168 ? PCI-MSI-edge ? ? ?sata_sil24
> NMI: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? Non-maskable interrupts
> LOC: ? ? ? 2147 ? ? ? 2390 ? ? ? 2285 ? ? ? 2344 ? Local timer interrupts
> SPU: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? Spurious interrupts
> PMI: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? Performance
> monitoring interrupts
> PND: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? Performance pending work
> RES: ? ? ? 3702 ? ? ? 3458 ? ? ? 2094 ? ? ? 2746 ? Rescheduling interrupts
> CAL: ? ? ? ?660 ? ? ? ? 33 ? ? ? ? 41 ? ? ? ? 26 ? Function call interrupts
> TLB: ? ? ? ?484 ? ? ? ?200 ? ? ? ?448 ? ? ? ?581 ? TLB shootdowns
> THR: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? Threshold APIC interrupts
> MCE: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? Machine check exceptions
> MCP: ? ? ? ? ?1 ? ? ? ? ?1 ? ? ? ? ?1 ? ? ? ? ?1 ? Machine check polls
> ERR: ? ? ? ? ?1
> MIS: ? ? ? ? ?0
>
> Output of the __assign_irq:
> 4x PCIe bridge:
> [ ? ?1.759016] __assign_irq_vector:assigning irq 24 to vector 73
> [ ? ?1.764928] __assign_irq_vector:assigning irq 25 to vector 81
> [ ? ?1.770827] __assign_irq_vector:assigning irq 26 to vector 89
> [ ? ?1.776720] __assign_irq_vector:assigning irq 27 to vector 97
> sata_sil24 first gets 19:
> [ ? ?1.856245] __assign_irq_vector:assigning irq 19 to vector 105
> then switches to MSI-28:
> [ ? ?1.870768] __assign_irq_vector:assigning irq 28 to vector 113
> 3 normal interrupts for sata_nv:
> [ ? ?1.908504] __assign_irq_vector:assigning irq 23 to vector 121
> [ ? ?1.953848] __assign_irq_vector:assigning irq 22 to vector 129
> [ ? ?1.999155] __assign_irq_vector:assigning irq 21 to vector 137
>
> Writing failed again without a message, but the read test showed:
> [ ? 85.695745] do_IRQ: 0.165 No irq handler for vector (irq -1)
> [ ? 85.695763] do_IRQ: 1.165 No irq handler for vector (irq -1)
> [ ? 85.695785] do_IRQ: 3.165 No irq handler for vector (irq -1)
> [ ? 85.695800] do_IRQ: 2.165 No irq handler for vector (irq -1)
> Same vector in this error, but sata_sil24 had MSI irq 28 instead of 29!
> And the vector allocations where not even near 165!

Hmm.. Can you send the output of lspci -vv with MSI in use?

2010-01-20 06:49:16

by Torsten Kaiser

[permalink] [raw]
Subject: Re: MSI broken in libata?

On Wed, Jan 20, 2010 at 4:00 AM, Robert Hancock <[email protected]> wrote:
> On Tue, Jan 19, 2010 at 2:20 PM, Torsten Kaiser
> <[email protected]> wrote:
>> As I did not see any real relation, I disabled all unneeded drivers
>> and tested again.
>> This minimal kernel had the following interrupt map:
>> ? ? ? ? ? CPU0 ? ? ? CPU1 ? ? ? CPU2 ? ? ? CPU3
>> ?0: ? ? ? ? 36 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?1 ? IO-APIC-edge ? ? ?timer
>> ?1: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?1 ? ? ? ?152 ? IO-APIC-edge ? ? ?i8042
>> ?4: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ?212 ? IO-APIC-edge ? ? ?serial
>> ?7: ? ? ? ? ?1 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? IO-APIC-edge
>> ?9: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? IO-APIC-fasteoi ? acpi
>> ?12: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?4 ? IO-APIC-edge ? ? ?i8042
>> ?21: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? IO-APIC-fasteoi ? sata_nv
>> ?22: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? IO-APIC-fasteoi ? sata_nv
>> ?23: ? ? ? ? ?0 ? ? ? ? ?5 ? ? ? ? ?4 ? ? ? 4980 ? IO-APIC-fasteoi ? sata_nv
>> ?28: ? ? ? ? ?0 ? ? ? ? ?1 ? ? ? ? ?0 ? ? ? ?168 ? PCI-MSI-edge ? ? ?sata_sil24
>> NMI: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? Non-maskable interrupts
>> LOC: ? ? ? 2147 ? ? ? 2390 ? ? ? 2285 ? ? ? 2344 ? Local timer interrupts
>> SPU: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? Spurious interrupts
>> PMI: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? Performance
>> monitoring interrupts
>> PND: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? Performance pending work
>> RES: ? ? ? 3702 ? ? ? 3458 ? ? ? 2094 ? ? ? 2746 ? Rescheduling interrupts
>> CAL: ? ? ? ?660 ? ? ? ? 33 ? ? ? ? 41 ? ? ? ? 26 ? Function call interrupts
>> TLB: ? ? ? ?484 ? ? ? ?200 ? ? ? ?448 ? ? ? ?581 ? TLB shootdowns
>> THR: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? Threshold APIC interrupts
>> MCE: ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? ? ? ? ?0 ? Machine check exceptions
>> MCP: ? ? ? ? ?1 ? ? ? ? ?1 ? ? ? ? ?1 ? ? ? ? ?1 ? Machine check polls
>> ERR: ? ? ? ? ?1
>> MIS: ? ? ? ? ?0
>>
>> Output of the __assign_irq:
>> 4x PCIe bridge:
>> [ ? ?1.759016] __assign_irq_vector:assigning irq 24 to vector 73
>> [ ? ?1.764928] __assign_irq_vector:assigning irq 25 to vector 81
>> [ ? ?1.770827] __assign_irq_vector:assigning irq 26 to vector 89
>> [ ? ?1.776720] __assign_irq_vector:assigning irq 27 to vector 97
>> sata_sil24 first gets 19:
>> [ ? ?1.856245] __assign_irq_vector:assigning irq 19 to vector 105
>> then switches to MSI-28:
>> [ ? ?1.870768] __assign_irq_vector:assigning irq 28 to vector 113
>> 3 normal interrupts for sata_nv:
>> [ ? ?1.908504] __assign_irq_vector:assigning irq 23 to vector 121
>> [ ? ?1.953848] __assign_irq_vector:assigning irq 22 to vector 129
>> [ ? ?1.999155] __assign_irq_vector:assigning irq 21 to vector 137
>>
>> Writing failed again without a message, but the read test showed:
>> [ ? 85.695745] do_IRQ: 0.165 No irq handler for vector (irq -1)
>> [ ? 85.695763] do_IRQ: 1.165 No irq handler for vector (irq -1)
>> [ ? 85.695785] do_IRQ: 3.165 No irq handler for vector (irq -1)
>> [ ? 85.695800] do_IRQ: 2.165 No irq handler for vector (irq -1)
>> Same vector in this error, but sata_sil24 had MSI irq 28 instead of 29!
>> And the vector allocations where not even near 165!
>
> Hmm.. Can you send the output of lspci -vv with MSI in use?

I had already done so at the start of this thread, both for MSI and
the normal, working case:
http://lkml.org/lkml/2009/12/25/12
( http://marc.info/?l=linux-kernel&m=126173296807440&w=2 might be
better, als lkml.org seems to eat the attachment borders)

For the minimal kernel with only the sata_nv and sata_sil24 drivers
the output looks like this:

00:00.0 RAM memory: nVidia Corporation MCP55 Memory Controller (rev a2)
Subsystem: ASUSTeK Computer Inc. Device 81f0
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Capabilities: [44] HyperTransport: Slave or Primary Interface
Command: BaseUnitID=0 UnitCnt=15 MastHost- DefDir- DUL-
Link Control 0: CFlE+ CST- CFE- <LkFail- Init+ EOC- TXO- <CRCErr=0
IsocEn- LSEn+ ExtCTL- 64b-
Link Config 0: MLWI=16bit DwFcIn- MLWO=16bit DwFcOut- LWI=16bit
DwFcInEn- LWO=16bit DwFcOutEn-
Link Control 1: CFlE- CST- CFE- <LkFail+ Init- EOC+ TXO+ <CRCErr=0
IsocEn- LSEn- ExtCTL- 64b-
Link Config 1: MLWI=8bit DwFcIn- MLWO=8bit DwFcOut- LWI=8bit
DwFcInEn- LWO=8bit DwFcOutEn-
Revision ID: 1.03
Link Frequency 0: 1.0GHz
Link Error 0: <Prot- <Ovfl- <EOC- CTLTm-
Link Frequency Capability 0: 200MHz+ 300MHz+ 400MHz+ 500MHz+ 600MHz+
800MHz+ 1.0GHz+ 1.2GHz- 1.4GHz- 1.6GHz- Vend-
Feature Capability: IsocFC+ LDTSTOP+ CRCTM- ECTLT- 64bA- UIDRD-
Link Frequency 1: 200MHz
Link Error 1: <Prot- <Ovfl- <EOC- CTLTm-
Link Frequency Capability 1: 200MHz- 300MHz- 400MHz- 500MHz- 600MHz-
800MHz- 1.0GHz- 1.2GHz- 1.4GHz- 1.6GHz- Vend-
Error Handling: PFlE+ OFlE+ PFE- OFE- EOCFE- RFE- CRCFE- SERRFE- CF-
RE- PNFE- ONFE- EOCNFE- RNFE- CRCNFE- SERRNFE-
Prefetchable memory behind bridge Upper: 00-00
Bus Number: 00
Capabilities: [dc] HyperTransport: MSI Mapping Enable+ Fixed-
Mapping Address Base: 00000000fee00000

00:01.0 ISA bridge: nVidia Corporation MCP55 LPC Bridge (rev a3)
Subsystem: ASUSTeK Computer Inc. Device 81f0
Control: I/O+ Mem+ BusMaster+ SpecCycle+ MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Region 0: I/O ports at 2f00 [size=128]

00:01.1 SMBus: nVidia Corporation MCP55 SMBus (rev a3)
Subsystem: ASUSTeK Computer Inc. Device 81f0
Control: I/O+ Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin A routed to IRQ 11
Region 0: I/O ports at a880 [size=64]
Region 4: I/O ports at 2a00 [size=64]
Region 5: I/O ports at 2a80 [size=64]
Capabilities: [44] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Kernel driver in use: nForce2_smbus

00:02.0 USB Controller: nVidia Corporation MCP55 USB Controller (rev
a1) (prog-if 10 [OHCI])
Subsystem: ASUSTeK Computer Inc. Device 81f0
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0 (750ns min, 250ns max)
Interrupt: pin A routed to IRQ 10
Region 0: Memory at efaf8000 (32-bit, non-prefetchable) [size=4K]
Capabilities: [44] Power Management version 2
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

00:02.1 USB Controller: nVidia Corporation MCP55 USB Controller (rev
a2) (prog-if 20 [EHCI])
Subsystem: ASUSTeK Computer Inc. Device 81f0
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0 (750ns min, 250ns max)
Interrupt: pin B routed to IRQ 11
Region 0: Memory at efaebc00 (32-bit, non-prefetchable) [size=256]
Capabilities: [44] Debug port: BAR=1 offset=0098
Capabilities: [80] Power Management version 2
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

00:04.0 IDE interface: nVidia Corporation MCP55 IDE (rev a1) (prog-if
8a [Master SecP PriP])
Subsystem: ASUSTeK Computer Inc. Device 81f0
Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0 (750ns min, 250ns max)
Region 0: [virtual] Memory at 000001f0 (32-bit, non-prefetchable) [size=8]
Region 1: [virtual] Memory at 000003f0 (type 3, non-prefetchable) [size=1]
Region 2: [virtual] Memory at 00000170 (32-bit, non-prefetchable) [size=8]
Region 3: [virtual] Memory at 00000370 (type 3, non-prefetchable) [size=1]
Region 4: I/O ports at ffa0 [size=16]
Capabilities: [44] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

00:05.0 IDE interface: nVidia Corporation MCP55 SATA Controller (rev
a3) (prog-if 85 [Master SecO PriO])
Subsystem: ASUSTeK Computer Inc. Device 81f0
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0 (750ns min, 250ns max)
Interrupt: pin A routed to IRQ 23
Region 0: I/O ports at cc00 [size=8]
Region 1: I/O ports at c880 [size=4]
Region 2: I/O ports at c800 [size=8]
Region 3: I/O ports at c480 [size=4]
Region 4: I/O ports at c400 [size=16]
Region 5: Memory at efafb000 (32-bit, non-prefetchable) [size=4K]
Capabilities: [44] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [b0] MSI: Enable- Count=1/4 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [cc] HyperTransport: MSI Mapping Enable- Fixed+
Kernel driver in use: sata_nv

00:05.1 IDE interface: nVidia Corporation MCP55 SATA Controller (rev
a3) (prog-if 85 [Master SecO PriO])
Subsystem: ASUSTeK Computer Inc. Device 81f0
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0 (750ns min, 250ns max)
Interrupt: pin B routed to IRQ 22
Region 0: I/O ports at c080 [size=8]
Region 1: I/O ports at c000 [size=4]
Region 2: I/O ports at bc00 [size=8]
Region 3: I/O ports at b880 [size=4]
Region 4: I/O ports at b800 [size=16]
Region 5: Memory at efafa000 (32-bit, non-prefetchable) [size=4K]
Capabilities: [44] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [b0] MSI: Enable- Count=1/4 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [cc] HyperTransport: MSI Mapping Enable- Fixed+
Kernel driver in use: sata_nv

00:05.2 IDE interface: nVidia Corporation MCP55 SATA Controller (rev
a3) (prog-if 85 [Master SecO PriO])
Subsystem: ASUSTeK Computer Inc. Device 81f0
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0 (750ns min, 250ns max)
Interrupt: pin C routed to IRQ 21
Region 0: I/O ports at b480 [size=8]
Region 1: I/O ports at b400 [size=4]
Region 2: I/O ports at b080 [size=8]
Region 3: I/O ports at b000 [size=4]
Region 4: I/O ports at ac00 [size=16]
Region 5: Memory at efaf9000 (32-bit, non-prefetchable) [size=4K]
Capabilities: [44] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [b0] MSI: Enable- Count=1/4 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [cc] HyperTransport: MSI Mapping Enable- Fixed+
Kernel driver in use: sata_nv

00:06.0 PCI bridge: nVidia Corporation MCP55 PCI bridge (rev a2)
(prog-if 01 [Subtractive decode])
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Bus: primary=00, secondary=05, subordinate=05, sec-latency=64
Memory behind bridge: eff00000-efffffff
Prefetchable memory behind bridge: eef00000-eeffffff
Secondary status: 66MHz- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr+ DiscTmrStat- DiscTmrSERREn-
Capabilities: [b8] Subsystem: nVidia Corporation Device cb84
Capabilities: [8c] HyperTransport: MSI Mapping Enable- Fixed-
Mapping Address Base: 00000000fee00000

00:06.1 Audio device: nVidia Corporation MCP55 High Definition Audio (rev a2)
Subsystem: ASUSTeK Computer Inc. Device 8210
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0 (500ns min, 1250ns max)
Interrupt: pin B routed to IRQ 11
Region 0: Memory at efafc000 (32-bit, non-prefetchable) [size=16K]
Capabilities: [44] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
Address: 0000000000000000 Data: 0000
Masking: 00000000 Pending: 00000000
Capabilities: [6c] HyperTransport: MSI Mapping Enable- Fixed+

00:0b.0 PCI bridge: nVidia Corporation MCP55 PCI Express bridge (rev
a3) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Bus: primary=00, secondary=04, subordinate=04, sec-latency=0
I/O behind bridge: 0000e000-0000efff
Memory behind bridge: efe00000-efefffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Subsystem: nVidia Corporation Device 0000
Capabilities: [48] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable+ Count=1/2 Maskable- 64bit+
Address: 00000000fee0f00c Data: 4149
Capabilities: [60] HyperTransport: MSI Mapping Enable- Fixed-
Mapping Address Base: 00000000fee00000
Capabilities: [80] Express (v1) Root Port (Slot+), MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #4, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0
<512ns, L1 <4us
ClockPM- Surprise- LLActRep+ BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+
BWMgmt- ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surpise-
Slot # 0, PowerLimit 0.000000; Interlock- NoCompl-
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
Control: AttnInd Off, PwrInd On, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
Changed: MRL- PresDet+ LinkState+
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
Kernel driver in use: pcieport

00:0c.0 PCI bridge: nVidia Corporation MCP55 PCI Express bridge (rev
a3) (prog-if 00 [Normal decode])
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Bus: primary=00, secondary=03, subordinate=03, sec-latency=0
Memory behind bridge: efd00000-efdfffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort+ <SERR- <PERR-
BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Subsystem: nVidia Corporation Device 0000
Capabilities: [48] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable+ Count=1/2 Maskable- 64bit+
Address: 00000000fee0f00c Data: 4151
Capabilities: [60] HyperTransport: MSI Mapping Enable- Fixed-
Mapping Address Base: 00000000fee00000
Capabilities: [80] Express (v1) Root Port (Slot+), MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #3, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0
<512ns, L1 <4us
ClockPM- Surprise- LLActRep+ BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+
BWMgmt- ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surpise-
Slot # 0, PowerLimit 0.000000; Interlock- NoCompl-
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
Control: AttnInd Off, PwrInd On, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
Changed: MRL- PresDet+ LinkState+
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
Kernel driver in use: pcieport

00:0d.0 PCI bridge: nVidia Corporation MCP55 PCI Express bridge (rev
a3) (prog-if 00 [Normal decode])
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
Memory behind bridge: efc00000-efcfffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort+ <SERR- <PERR-
BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Subsystem: nVidia Corporation Device 0000
Capabilities: [48] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable+ Count=1/2 Maskable- 64bit+
Address: 00000000fee0f00c Data: 4159
Capabilities: [60] HyperTransport: MSI Mapping Enable- Fixed-
Mapping Address Base: 00000000fee00000
Capabilities: [80] Express (v1) Root Port (Slot+), MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #2, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0
<512ns, L1 <4us
ClockPM- Surprise- LLActRep+ BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+
BWMgmt- ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surpise-
Slot # 0, PowerLimit 0.000000; Interlock- NoCompl-
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
Control: AttnInd Off, PwrInd On, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
Changed: MRL- PresDet+ LinkState+
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
Kernel driver in use: pcieport

00:0f.0 PCI bridge: nVidia Corporation MCP55 PCI Express bridge (rev
a3) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
I/O behind bridge: 0000d000-0000dfff
Memory behind bridge: efb00000-efbfffff
Prefetchable memory behind bridge: 00000000e0000000-00000000e7ffffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity- SERR+ NoISA- VGA+ MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Subsystem: nVidia Corporation Device 0000
Capabilities: [48] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable+ Count=1/2 Maskable- 64bit+
Address: 00000000fee0f00c Data: 4161
Capabilities: [60] HyperTransport: MSI Mapping Enable- Fixed-
Mapping Address Base: 00000000fee00000
Capabilities: [80] Express (v1) Root Port (Slot+), MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x16, ASPM L0s L1, Latency L0
<512ns, L1 <4us
ClockPM- Surprise- LLActRep+ BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive+
BWMgmt- ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surpise-
Slot # 0, PowerLimit 0.000000; Interlock- NoCompl-
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
Control: AttnInd Off, PwrInd On, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
Changed: MRL- PresDet+ LinkState+
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
Kernel driver in use: pcieport

00:18.0 Host bridge: Advanced Micro Devices [AMD] K8
[Athlon64/Opteron] HyperTransport Technology Configuration
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Capabilities: [80] HyperTransport: Host or Secondary Interface
Command: WarmRst+ DblEnd- DevNum=0 ChainSide- HostHide+ Slave- <EOCErr- DUL-
Link Control: CFlE- CST- CFE- <LkFail- Init+ EOC- TXO- <CRCErr=0
IsocEn- LSEn- ExtCTL- 64b-
Link Config: MLWI=16bit DwFcIn- MLWO=16bit DwFcOut- LWI=16bit
DwFcInEn- LWO=16bit DwFcOutEn-
Revision ID: 1.02
Link Frequency: 1.0GHz
Link Error: <Prot- <Ovfl- <EOC- CTLTm-
Link Frequency Capability: 200MHz+ 300MHz- 400MHz+ 500MHz- 600MHz+
800MHz+ 1.0GHz+ 1.2GHz- 1.4GHz- 1.6GHz- Vend-
Feature Capability: IsocFC- LDTSTOP+ CRCTM- ECTLT- 64bA- UIDRD- ExtRS- UCnfE-
Capabilities: [a0] HyperTransport: Host or Secondary Interface
Command: WarmRst+ DblEnd- DevNum=0 ChainSide- HostHide+ Slave- <EOCErr- DUL-
Link Control: CFlE- CST- CFE- <LkFail- Init+ EOC- TXO- <CRCErr=0
IsocEn- LSEn- ExtCTL- 64b-
Link Config: MLWI=16bit DwFcIn- MLWO=16bit DwFcOut- LWI=16bit
DwFcInEn- LWO=16bit DwFcOutEn-
Revision ID: 1.02
Link Frequency: 1.0GHz
Link Error: <Prot- <Ovfl- <EOC- CTLTm-
Link Frequency Capability: 200MHz+ 300MHz- 400MHz+ 500MHz- 600MHz+
800MHz+ 1.0GHz+ 1.2GHz- 1.4GHz- 1.6GHz- Vend-
Feature Capability: IsocFC- LDTSTOP+ CRCTM- ECTLT- 64bA- UIDRD- ExtRS- UCnfE-
Capabilities: [c0] HyperTransport: Host or Secondary Interface
Command: WarmRst+ DblEnd- DevNum=0 ChainSide- HostHide+ Slave- <EOCErr- DUL-
Link Control: CFlE- CST- CFE- <LkFail+ Init- EOC+ TXO+ <CRCErr=0
IsocEn- LSEn- ExtCTL- 64b-
Link Config: MLWI=16bit DwFcIn- MLWO=16bit DwFcOut- LWI=N/C
DwFcInEn- LWO=N/C DwFcOutEn-
Revision ID: 1.02
Link Frequency: 200MHz
Link Error: <Prot- <Ovfl- <EOC- CTLTm-
Link Frequency Capability: 200MHz+ 300MHz- 400MHz+ 500MHz- 600MHz+
800MHz+ 1.0GHz+ 1.2GHz- 1.4GHz- 1.6GHz- Vend-
Feature Capability: IsocFC- LDTSTOP+ CRCTM- ECTLT- 64bA- UIDRD- ExtRS- UCnfE-

00:18.1 Host bridge: Advanced Micro Devices [AMD] K8
[Athlon64/Opteron] Address Map
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-

00:18.2 Host bridge: Advanced Micro Devices [AMD] K8
[Athlon64/Opteron] DRAM Controller
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Kernel driver in use: amd64_edac

00:18.3 Host bridge: Advanced Micro Devices [AMD] K8
[Athlon64/Opteron] Miscellaneous Control
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Capabilities: [f0] Secure device <?>
Kernel driver in use: k8temp

00:19.0 Host bridge: Advanced Micro Devices [AMD] K8
[Athlon64/Opteron] HyperTransport Technology Configuration
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Capabilities: [80] HyperTransport: Host or Secondary Interface
Command: WarmRst+ DblEnd- DevNum=0 ChainSide- HostHide+ Slave- <EOCErr- DUL-
Link Control: CFlE- CST- CFE- <LkFail+ Init- EOC+ TXO+ <CRCErr=0
IsocEn- LSEn- ExtCTL- 64b-
Link Config: MLWI=16bit DwFcIn- MLWO=16bit DwFcOut- LWI=N/C
DwFcInEn- LWO=N/C DwFcOutEn-
Revision ID: 1.02
Link Frequency: 200MHz
Link Error: <Prot- <Ovfl- <EOC- CTLTm-
Link Frequency Capability: 200MHz+ 300MHz- 400MHz+ 500MHz- 600MHz+
800MHz+ 1.0GHz+ 1.2GHz- 1.4GHz- 1.6GHz- Vend-
Feature Capability: IsocFC- LDTSTOP+ CRCTM- ECTLT- 64bA- UIDRD- ExtRS- UCnfE-
Capabilities: [a0] HyperTransport: Host or Secondary Interface
Command: WarmRst+ DblEnd- DevNum=0 ChainSide- HostHide+ Slave- <EOCErr- DUL-
Link Control: CFlE- CST- CFE- <LkFail- Init+ EOC- TXO- <CRCErr=0
IsocEn- LSEn- ExtCTL- 64b-
Link Config: MLWI=16bit DwFcIn- MLWO=16bit DwFcOut- LWI=16bit
DwFcInEn- LWO=16bit DwFcOutEn-
Revision ID: 1.02
Link Frequency: 1.0GHz
Link Error: <Prot- <Ovfl- <EOC- CTLTm-
Link Frequency Capability: 200MHz+ 300MHz- 400MHz+ 500MHz- 600MHz+
800MHz+ 1.0GHz+ 1.2GHz- 1.4GHz- 1.6GHz- Vend-
Feature Capability: IsocFC- LDTSTOP+ CRCTM- ECTLT- 64bA- UIDRD- ExtRS- UCnfE-
Capabilities: [c0] HyperTransport: Host or Secondary Interface
Command: WarmRst+ DblEnd- DevNum=0 ChainSide- HostHide+ Slave- <EOCErr- DUL-
Link Control: CFlE- CST- CFE- <LkFail+ Init- EOC+ TXO+ <CRCErr=0
IsocEn- LSEn- ExtCTL- 64b-
Link Config: MLWI=16bit DwFcIn- MLWO=16bit DwFcOut- LWI=N/C
DwFcInEn- LWO=N/C DwFcOutEn-
Revision ID: 1.02
Link Frequency: 200MHz
Link Error: <Prot- <Ovfl- <EOC- CTLTm-
Link Frequency Capability: 200MHz+ 300MHz- 400MHz+ 500MHz- 600MHz+
800MHz+ 1.0GHz+ 1.2GHz- 1.4GHz- 1.6GHz- Vend-
Feature Capability: IsocFC- LDTSTOP+ CRCTM- ECTLT- 64bA- UIDRD- ExtRS- UCnfE-

00:19.1 Host bridge: Advanced Micro Devices [AMD] K8
[Athlon64/Opteron] Address Map
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-

00:19.2 Host bridge: Advanced Micro Devices [AMD] K8
[Athlon64/Opteron] DRAM Controller
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Kernel driver in use: amd64_edac

00:19.3 Host bridge: Advanced Micro Devices [AMD] K8
[Athlon64/Opteron] Miscellaneous Control
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Capabilities: [f0] Secure device <?>
Kernel driver in use: k8temp

01:00.0 VGA compatible controller: ATI Technologies Inc RV370 5B60
[Radeon X300 (PCIE)] (prog-if 00 [VGA controller])
Subsystem: ASUSTeK Computer Inc. Device 0083
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 10
Region 0: Memory at e0000000 (32-bit, prefetchable) [size=128M]
Region 1: I/O ports at d000 [size=256]
Region 2: Memory at efbf0000 (32-bit, non-prefetchable) [size=64K]
Expansion ROM at efbc0000 [disabled] [size=128K]
Capabilities: [50] Power Management version 2
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [58] Express (v1) Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <128ns, L1 <2us
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE- FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x16, ASPM L0s L1, Latency L0
<128ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive-
BWMgmt- ABWMgmt-
Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 0000000000000000 Data: 0000

01:00.1 Display controller: ATI Technologies Inc RV370 [Radeon X300SE]
Subsystem: ASUSTeK Computer Inc. Device 0082
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Region 0: Memory at efbe0000 (32-bit, non-prefetchable) [size=64K]
Capabilities: [50] Power Management version 2
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [58] Express (v1) Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <128ns, L1 <2us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x16, ASPM L0s L1, Latency L0
<128ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive-
BWMgmt- ABWMgmt-

02:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5754
Gigabit Ethernet PCI Express (rev 02)
Subsystem: ASUSTeK Computer Inc. Device 8209
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 5
Region 0: Memory at efcf0000 (64-bit, non-prefetchable) [size=64K]
Capabilities: [48] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] Vital Product Data
Product Name: Broadcom NetXtreme Gigabit Ethernet Controller
Read-only fields:
[PN] Part number: BCM95700A6
[EC] Engineering changes: 106679-15
[SN] Serial number: 0123456789
[MN] Manufacture ID: 31 34 65 34
[RV] Reserved: checksum bad, 26 byte(s) reserved
Read/write fields:
[YA] Asset tag: XYZ01234567
[RW] Read-write area: 107 byte(s) free
End
Capabilities: [58] Vendor Specific Information <?>
Capabilities: [e8] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 00000000fee0100c Data: 41c1
Capabilities: [d0] Express (v1) Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 <4us, L1 <64us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive-
BWMgmt- ABWMgmt-

03:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5754
Gigabit Ethernet PCI Express (rev 02)
Subsystem: ASUSTeK Computer Inc. Device 8209
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 10
Region 0: Memory at efdf0000 (64-bit, non-prefetchable) [size=64K]
Capabilities: [48] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] Vital Product Data
Product Name: Broadcom NetXtreme Gigabit Ethernet Controller
Read-only fields:
[PN] Part number: BCM95700A6
[EC] Engineering changes: 106679-15
[SN] Serial number: 0123456789
[MN] Manufacture ID: 31 34 65 34
[RV] Reserved: checksum bad, 26 byte(s) reserved
Read/write fields:
[YA] Asset tag: XYZ01234567
[RW] Read-write area: 107 byte(s) free
End
Capabilities: [58] Vendor Specific Information <?>
Capabilities: [e8] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 00000000fee0f00c Data: 41b9
Capabilities: [d0] Express (v1) Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 <4us, L1 <64us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive-
BWMgmt- ABWMgmt-

04:00.0 Mass storage controller: Silicon Image, Inc. SiI 3132 Serial
ATA Raid II Controller (rev 01)
Subsystem: ASUSTeK Computer Inc. Device 819f
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 28
Region 0: Memory at efeffc00 (64-bit, non-prefetchable) [size=128]
Region 2: Memory at efef8000 (64-bit, non-prefetchable) [size=16K]
Region 4: I/O ports at ec00 [size=128]
Expansion ROM at efe00000 [disabled] [size=512K]
Capabilities: [54] Power Management version 2
Flags: PMEClk- DSI+ D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [5c] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee0f00c Data: 4171
Capabilities: [70] Express (v1) Legacy Endpoint, MSI 00
DevCap: MaxPayload 1024 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 4096 bytes
DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq+ AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0
unlimited, L1 unlimited
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive-
BWMgmt- ABWMgmt-
Kernel driver in use: sata_sil24

05:06.0 Multimedia video controller: Brooktree Corporation Bt878 Video
Capture (rev 11)
Subsystem: Hauppauge computer works Inc. WinTV Series
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 64 (4000ns min, 10000ns max)
Interrupt: pin A routed to IRQ 10
Region 0: Memory at eefff000 (32-bit, prefetchable) [size=4K]
Capabilities: [44] Vital Product Data
Unknown small resource type 00, will not decode more.
Capabilities: [4c] Power Management version 2
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

05:06.1 Multimedia controller: Brooktree Corporation Bt878 Audio
Capture (rev 11)
Subsystem: Hauppauge computer works Inc. WinTV Series
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 64 (1000ns min, 63750ns max)
Interrupt: pin A routed to IRQ 10
Region 0: Memory at eeffe000 (32-bit, prefetchable) [size=4K]
Capabilities: [44] Vital Product Data
Unknown small resource type 00, will not decode more.
Capabilities: [4c] Power Management version 2
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

05:08.0 FireWire (IEEE 1394): Texas Instruments TSB43AB22/A
IEEE-1394a-2000 Controller (PHY/Link) (prog-if 10 [OHCI])
Subsystem: ASUSTeK Computer Inc. P5W DH Deluxe Motherboard
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 64 (500ns min, 1000ns max), Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 11
Region 0: Memory at effff800 (32-bit, non-prefetchable) [size=2K]
Region 1: Memory at efff8000 (32-bit, non-prefetchable) [size=16K]
Capabilities: [44] Power Management version 2
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

2010-01-22 00:53:17

by Robert Hancock

[permalink] [raw]
Subject: Re: MSI broken in libata?

On Wed, Jan 20, 2010 at 12:48 AM, Torsten Kaiser
<[email protected]> wrote:
>>> Output of the __assign_irq:
>>> 4x PCIe bridge:
>>> [ ? ?1.759016] __assign_irq_vector:assigning irq 24 to vector 73
>>> [ ? ?1.764928] __assign_irq_vector:assigning irq 25 to vector 81
>>> [ ? ?1.770827] __assign_irq_vector:assigning irq 26 to vector 89
>>> [ ? ?1.776720] __assign_irq_vector:assigning irq 27 to vector 97
>>> sata_sil24 first gets 19:
>>> [ ? ?1.856245] __assign_irq_vector:assigning irq 19 to vector 105
>>> then switches to MSI-28:
>>> [ ? ?1.870768] __assign_irq_vector:assigning irq 28 to vector 113
>>> 3 normal interrupts for sata_nv:
>>> [ ? ?1.908504] __assign_irq_vector:assigning irq 23 to vector 121
>>> [ ? ?1.953848] __assign_irq_vector:assigning irq 22 to vector 129
>>> [ ? ?1.999155] __assign_irq_vector:assigning irq 21 to vector 137
>>>
>>> Writing failed again without a message, but the read test showed:
>>> [ ? 85.695745] do_IRQ: 0.165 No irq handler for vector (irq -1)
>>> [ ? 85.695763] do_IRQ: 1.165 No irq handler for vector (irq -1)
>>> [ ? 85.695785] do_IRQ: 3.165 No irq handler for vector (irq -1)
>>> [ ? 85.695800] do_IRQ: 2.165 No irq handler for vector (irq -1)
>>> Same vector in this error, but sata_sil24 had MSI irq 28 instead of 29!
>>> And the vector allocations where not even near 165!
>>
>> Hmm.. Can you send the output of lspci -vv with MSI in use?
>
> I had already done so at the start of this thread, both for MSI and
> the normal, working case:
> http://lkml.org/lkml/2009/12/25/12
> ( http://marc.info/?l=linux-kernel&m=126173296807440&w=2 might be
> better, als lkml.org seems to eat the attachment borders)
>
> For the minimal kernel with only the sata_nv and sata_sil24 drivers
> the output looks like this:

Well, that doesn't seem that illuminating. Still mysterious where 165
could be coming from.

Anyone have any ideas how random IRQ vectors could be showing up on
the APIC with MSI?