I've got a Nehalem testbox that developed a new e1000e problem in this
merge window: after a few minutes of uptime the network interface goes
dead - no rx and no tx. If i ifdown/ifup the interface it comes back. If i
wait too long then even ifdown/ifup does not help anymore - only a reboot.
Other e1000e using testboxes i have are working just fine - so the problem
is specific to this hw.
Is this a known problem?
I have this hw:
01:00.0 Ethernet controller: Intel Corporation 82575EB Gigabit Network Connection (rev 02)
01:00.1 Ethernet controller: Intel Corporation 82575EB Gigabit Network Connection (rev 02)
If this is a new problem, what kind of other info do you need from me to
debug and fix this?
I started seeing this very early in the merge window, so candidates would
be one of these early commits:
eb14f01: Merge branch 'master' of master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6
cb7b48f: igb/e1000e: Naming interrupt vectors
5b9ab2e: Merge branch 'master' of master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6
e243455: e1000e: check return code from NVM accesses and fix bank detection
a20e4cf: e1000e: fix incorrect link status when switch module pulled
8452759: e1000e: store EEPROM version number to prevent unnecessary NVM reads
0285c8d: e1000e: cosmetic newline in debug message
5c48ef3: e1000e: sync change flow control variables with ixgbe
8f12fe8: e1000e: link up/down messages must follow a specific format
75eb0fa: e1000e: ESB2 config after link up
438b365: e1000e: check return of pci_save_state
1605927: e1000e: update comments listing supported parts for each MAC family
63dcf3d: e1000e: 82571 check for link fix on 82571 serdes
5aa49c8: e1000e: commit speed/duplex changes for m88 PHY
005cbdf: e1000e: disable correctable errors for quad ports while going to D3
0082982: netdev: add more functions to netdevice ops
651c246: e1000e: convert to net_device_ops
198d6ba: Merge branch 'master' of master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6
6ea7ae1: e1000e: enable ECC correction on 82571 silicon
4cf1653: netdevice: safe convert to netdev_priv() #part-2
babcda7: drivers/net: Kill now superfluous ->last_rx stores.
7c510e4: net: convert more to %pM
If you suspect a specific list of commits i can test their revert. (But
the box is a slow booter and the problem can take up to 15 minutes to
trigger so i'd rather not spend half a day bisecting it, if it can be
avoided.)
Thanks,
Ingo
>-----Original Message-----
>From: Ingo Molnar [mailto:[email protected]]
>Sent: Wednesday, January 21, 2009 2:29 AM
>To: [email protected]; Kirsher, Jeffrey T; Brandeburg, Jesse;
>Allan, Bruce W; Waskiewicz Jr, Peter P; [email protected];
>[email protected]
>Cc: Rafael J. Wysocki
>Subject: e1000e regression (interface hang) with latest -git
>
>
>I've got a Nehalem testbox that developed a new e1000e problem in this
>merge window: after a few minutes of uptime the network interface goes
>dead - no rx and no tx. If i ifdown/ifup the interface it comes back. If i
>wait too long then even ifdown/ifup does not help anymore - only a reboot.
>
>Other e1000e using testboxes i have are working just fine - so the problem
>is specific to this hw.
>
>Is this a known problem?
>
>I have this hw:
>
> 01:00.0 Ethernet controller: Intel Corporation 82575EB Gigabit Network
>Connection (rev 02)
> 01:00.1 Ethernet controller: Intel Corporation 82575EB Gigabit Network
>Connection (rev 02)
>
>If this is a new problem, what kind of other info do you need from me to
>debug and fix this?
>
>I started seeing this very early in the merge window, so candidates would
>be one of these early commits:
>
>eb14f01: Merge branch 'master' of
>master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6
>cb7b48f: igb/e1000e: Naming interrupt vectors
>5b9ab2e: Merge branch 'master' of
>master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6
>e243455: e1000e: check return code from NVM accesses and fix bank
>detection
>a20e4cf: e1000e: fix incorrect link status when switch module pulled
>8452759: e1000e: store EEPROM version number to prevent unnecessary NVM
>reads
>0285c8d: e1000e: cosmetic newline in debug message
>5c48ef3: e1000e: sync change flow control variables with ixgbe
>8f12fe8: e1000e: link up/down messages must follow a specific format
>75eb0fa: e1000e: ESB2 config after link up
>438b365: e1000e: check return of pci_save_state
>1605927: e1000e: update comments listing supported parts for each MAC
>family
>63dcf3d: e1000e: 82571 check for link fix on 82571 serdes
>5aa49c8: e1000e: commit speed/duplex changes for m88 PHY
>005cbdf: e1000e: disable correctable errors for quad ports while going to
>D3
>0082982: netdev: add more functions to netdevice ops
>651c246: e1000e: convert to net_device_ops
>198d6ba: Merge branch 'master' of
>master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6
>6ea7ae1: e1000e: enable ECC correction on 82571 silicon
>4cf1653: netdevice: safe convert to netdev_priv() #part-2
>babcda7: drivers/net: Kill now superfluous ->last_rx stores.
>7c510e4: net: convert more to %pM
>
>If you suspect a specific list of commits i can test their revert. (But
>the box is a slow booter and the problem can take up to 15 minutes to
>trigger so i'd rather not spend half a day bisecting it, if it can be
>avoided.)
>
>Thanks,
>
> Ingo
82575EB is not supported by e1000e, it is supported by igb. Are you sure that is the correct device? Please send the system log and output of:
# for dev in `lspci | grep Ethernet | awk ' { print $1 } '`; do lspci -s $dev -vvv -n; done
* Allan, Bruce W <[email protected]> wrote:
> >-----Original Message-----
> >From: Ingo Molnar [mailto:[email protected]]
> >Sent: Wednesday, January 21, 2009 2:29 AM
> >To: [email protected]; Kirsher, Jeffrey T; Brandeburg, Jesse;
> >Allan, Bruce W; Waskiewicz Jr, Peter P; [email protected];
> >[email protected]
> >Cc: Rafael J. Wysocki
> >Subject: e1000e regression (interface hang) with latest -git
> >
> >
> >I've got a Nehalem testbox that developed a new e1000e problem in this
> >merge window: after a few minutes of uptime the network interface goes
> >dead - no rx and no tx. If i ifdown/ifup the interface it comes back. If i
> >wait too long then even ifdown/ifup does not help anymore - only a reboot.
> >
> >Other e1000e using testboxes i have are working just fine - so the problem
> >is specific to this hw.
> >
> >Is this a known problem?
> >
> >I have this hw:
> >
> > 01:00.0 Ethernet controller: Intel Corporation 82575EB Gigabit Network
> >Connection (rev 02)
> > 01:00.1 Ethernet controller: Intel Corporation 82575EB Gigabit Network
> >Connection (rev 02)
> >
> >If this is a new problem, what kind of other info do you need from me to
> >debug and fix this?
> >
> >I started seeing this very early in the merge window, so candidates would
> >be one of these early commits:
> >
> >eb14f01: Merge branch 'master' of
> >master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6
> >cb7b48f: igb/e1000e: Naming interrupt vectors
> >5b9ab2e: Merge branch 'master' of
> >master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6
> >e243455: e1000e: check return code from NVM accesses and fix bank
> >detection
> >a20e4cf: e1000e: fix incorrect link status when switch module pulled
> >8452759: e1000e: store EEPROM version number to prevent unnecessary NVM
> >reads
> >0285c8d: e1000e: cosmetic newline in debug message
> >5c48ef3: e1000e: sync change flow control variables with ixgbe
> >8f12fe8: e1000e: link up/down messages must follow a specific format
> >75eb0fa: e1000e: ESB2 config after link up
> >438b365: e1000e: check return of pci_save_state
> >1605927: e1000e: update comments listing supported parts for each MAC
> >family
> >63dcf3d: e1000e: 82571 check for link fix on 82571 serdes
> >5aa49c8: e1000e: commit speed/duplex changes for m88 PHY
> >005cbdf: e1000e: disable correctable errors for quad ports while going to
> >D3
> >0082982: netdev: add more functions to netdevice ops
> >651c246: e1000e: convert to net_device_ops
> >198d6ba: Merge branch 'master' of
> >master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6
> >6ea7ae1: e1000e: enable ECC correction on 82571 silicon
> >4cf1653: netdevice: safe convert to netdev_priv() #part-2
> >babcda7: drivers/net: Kill now superfluous ->last_rx stores.
> >7c510e4: net: convert more to %pM
> >
> >If you suspect a specific list of commits i can test their revert. (But
> >the box is a slow booter and the problem can take up to 15 minutes to
> >trigger so i'd rather not spend half a day bisecting it, if it can be
> >avoided.)
> >
> >Thanks,
> >
> > Ingo
>
> 82575EB is not supported by e1000e, it is supported by igb. Are you
> sure that is the correct device? Please send the system log and output
> of:
yeah - sorry, i got confused - it is using IGB:
CONFIG_IGB=y
CONFIG_IGB_LRO=y
The same point remains though: it worked before (and works fine with a
distro kernel, 2.6.28-0.140.rc9.git1.fc11.x86_64) and doesnt work with
latest -git.
I'll capture some debug data and send it to you.
Ingo
* Ingo Molnar <[email protected]> wrote:
> The same point remains though: it worked before (and works fine with a
> distro kernel, 2.6.28-0.140.rc9.git1.fc11.x86_64) and doesnt work with
> latest -git.
>
> I'll capture some debug data and send it to you.
find attached the debug data:
ethtool.dump.eth0.before
ethtool.dump.eth0.after
ethtool.dump.eth0.after2
boot.log
config
the 'before' dump is before the hang, shortly after bootup. It also
includes 'ifconfig eth0' output.
the 'after' dump is after the hang - while the interface is hung.
'after2' dump is after a 'service network restart' - which brought the
interface back.
Note if i wait long enough (half an hour or so), then not even 'service
network restart' will bring back the device.
The kernel i tested is: v2.6.29-rc2-13-gf3b8436
Ingo
> * Ingo Molnar <[email protected]> wrote:
>
> > The same point remains though: it worked before (and works fine with a
> > distro kernel, 2.6.28-0.140.rc9.git1.fc11.x86_64) and doesnt work with
> > latest -git.
> >
> > I'll capture some debug data and send it to you.
>
> find attached the debug data:
>
> ethtool.dump.eth0.before
> ethtool.dump.eth0.after
> ethtool.dump.eth0.after2
> boot.log
> config
>
> the 'before' dump is before the hang, shortly after bootup. It also
> includes 'ifconfig eth0' output.
>
> the 'after' dump is after the hang - while the interface is hung.
>
> 'after2' dump is after a 'service network restart' - which brought the
> interface back.
>
> Note if i wait long enough (half an hour or so), then not even 'service
> network restart' will bring back the device.
>
> The kernel i tested is: v2.6.29-rc2-13-gf3b8436
As Bruce stated, this is an igb issue, not e1000e. From your boot.log:
igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
One other piece of info (in addition to what Bruce requested) that would
be useful is after you've done ifconfig eht0 up, cat /proc/interrupts |
grep eth0 and send that output (I want to see your MSI-X configuration).
Cheers,
-PJ Waskiewicz
* Waskiewicz Jr, Peter P <[email protected]> wrote:
> > * Ingo Molnar <[email protected]> wrote:
> >
> > > The same point remains though: it worked before (and works fine with a
> > > distro kernel, 2.6.28-0.140.rc9.git1.fc11.x86_64) and doesnt work with
> > > latest -git.
> > >
> > > I'll capture some debug data and send it to you.
> >
> > find attached the debug data:
> >
> > ethtool.dump.eth0.before
> > ethtool.dump.eth0.after
> > ethtool.dump.eth0.after2
> > boot.log
> > config
> >
> > the 'before' dump is before the hang, shortly after bootup. It also
> > includes 'ifconfig eth0' output.
> >
> > the 'after' dump is after the hang - while the interface is hung.
> >
> > 'after2' dump is after a 'service network restart' - which brought the
> > interface back.
> >
> > Note if i wait long enough (half an hour or so), then not even 'service
> > network restart' will bring back the device.
> >
> > The kernel i tested is: v2.6.29-rc2-13-gf3b8436
>
> As Bruce stated, this is an igb issue, not e1000e. From your boot.log:
(yes, as i said it too two mails ago - it is an igb.)
> igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
> igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
>
> One other piece of info (in addition to what Bruce requested) that would
> be useful is after you've done ifconfig eht0 up, cat /proc/interrupts |
> grep eth0 and send that output (I want to see your MSI-X configuration).
here it is:
79: 35 0 9836 7451 1197 1360 766 0 2196 995 770 0 2051 863 4714 1879 PCI-MSI-edge eth0-tx-0
80: 2335 0 2718 3651 5024 9850 533 0 1405 1095 0 0 4421 1229 0 2002 PCI-MSI-edge eth0-tx-1
81: 33 0 0 5478 1125 6585 4865 6306 0 0 0 714 3757 0 2662 1020 PCI-MSI-edge eth0-tx-2
82: 21 1428 3136 3783 4249 6833 0 0 4175 0 0 1465 1940 1780 6286 0 PCI-MSI-edge eth0-tx-3
83: 40 0 0 4788 0 1486 0 7185 3998 8637 1409 6785 0 0 0 10060 PCI-MSI-edge eth0-rx-0
84: 70 1441 2606 1342 0 1678 4301 1839 0 6507 6910 16470 0 1484 1004 0 PCI-MSI-edge eth0-rx-1
85: 40 1368 0 3560 5022 2040 5 1719 0 2037 956 40 0 17264 6456 2833 PCI-MSI-edge eth0-rx-2
86: 44 414 5 1164 30 4513 4595 1542 0 10033 1151 0 4961 4203 0 10703 PCI-MSI-edge eth0-rx-3
87: 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge eth0
Ingo
* Ingo Molnar <[email protected]> wrote:
> > One other piece of info (in addition to what Bruce requested) that
> > would be useful is after you've done ifconfig eht0 up, cat
> > /proc/interrupts | grep eth0 and send that output (I want to see your
> > MSI-X configuration).
>
> here it is:
>
> 79: 35 0 9836 7451 1197 1360 766 0 2196 995 770 0 2051 863 4714 1879 PCI-MSI-edge eth0-tx-0
> 80: 2335 0 2718 3651 5024 9850 533 0 1405 1095 0 0 4421 1229 0 2002 PCI-MSI-edge eth0-tx-1
> 81: 33 0 0 5478 1125 6585 4865 6306 0 0 0 714 3757 0 2662 1020 PCI-MSI-edge eth0-tx-2
> 82: 21 1428 3136 3783 4249 6833 0 0 4175 0 0 1465 1940 1780 6286 0 PCI-MSI-edge eth0-tx-3
> 83: 40 0 0 4788 0 1486 0 7185 3998 8637 1409 6785 0 0 0 10060 PCI-MSI-edge eth0-rx-0
> 84: 70 1441 2606 1342 0 1678 4301 1839 0 6507 6910 16470 0 1484 1004 0 PCI-MSI-edge eth0-rx-1
> 85: 40 1368 0 3560 5022 2040 5 1719 0 2037 956 40 0 17264 6456 2833 PCI-MSI-edge eth0-rx-2
> 86: 44 414 5 1164 30 4513 4595 1542 0 10033 1151 0 4961 4203 0 10703 PCI-MSI-edge eth0-rx-3
> 87: 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge eth0
btw., i turned off CONFIG_PCI_MSI and that worked around the interface
hang - the box has 40 minutes uptime now and still no hang. (it would hang
within 5 minutes previously)
can test the revert of any of these commits:
e42e4ba: igb: fix anoying type mismatch warning on rx/tx queue sizing
8d25332: igb: Fix build warning when DCA is disabled.
26bc19e: igb: re-order queues to support cleaner use of ivar on 82576
0e014cb: igb: defeature tx head writeback
678c610: drivers/net/igb: remove dead code (function 'igb_read_pci_cfg')
908a7a1: net: Remove unused netdev arg from some NAPI interfaces.
ea943d4: igb: fixup AER with proper error handling
5e8427e: igb: Correctly determine pci-e function number in virtual environment
b4557be: igb: update handling of RCTL for smaller buffer sizes
cb7b48f: igb/e1000e: Naming interrupt vectors
40a914f: igb: Add support for pci-e Advanced Error Reporting
527d47c: igb: link up/down messages must follow a specific format
5b9ab2e: Merge branch 'master' of master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6
69d728b: igb: loopback bits not correctly cleared from RCTL register
9b07f3d: igb: remove unneeded bit refrence when enabling jumbo frames
f5f4cf0: igb: do not use phy ops in ethtool test cleanup for non-copper parts
0082982: netdev: add more functions to netdevice ops
68fd991: igb: Fix tx/rx_ring_count parameters for igb on suspend/resume/ring resize
b2d5653: igb: simplify swap in clean_rx_irq if using packet split
2e5c692: igb: convert to net_device_ops
198d6ba: Merge branch 'master' of master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6
4cf1653: netdevice: safe convert to netdev_priv() #part-2
babcda7: drivers/net: Kill now superfluous ->last_rx stores.
7c510e4: net: convert more to %pM
i had a quick look at all of them but none seem to have a direct
connection to MSI-X interrupt generation.
There's a couple of ones that could have side effects:
Maybe this one:
26bc19e: igb: re-order queues to support cleaner use of ivar on 82576
due to its sheer commit size and due to its impact on the hw programming.
or:
0e014cb: igb: defeature tx head writeback
Might have some side-effect on tx-completion IRQs and might tickle
firmware bugs?
Or:
5e8427e: igb: Correctly determine pci-e function number in virtual environment
while it should have no impact on a native kernel, it does change the PCI
config space access sequences.
Or maybe:
69d728b: igb: loopback bits not correctly cleared from RCTL register
as this impacts the PCI programming too.
But ... i dont really know this code so i'm guessing around.
Ingo
On Wednesday 21 January 2009, Ingo Molnar wrote:
>
> * Ingo Molnar <[email protected]> wrote:
>
> > > One other piece of info (in addition to what Bruce requested) that
> > > would be useful is after you've done ifconfig eht0 up, cat
> > > /proc/interrupts | grep eth0 and send that output (I want to see your
> > > MSI-X configuration).
> >
> > here it is:
> >
> > 79: 35 0 9836 7451 1197 1360 766 0 2196 995 770 0 2051 863 4714 1879 PCI-MSI-edge eth0-tx-0
> > 80: 2335 0 2718 3651 5024 9850 533 0 1405 1095 0 0 4421 1229 0 2002 PCI-MSI-edge eth0-tx-1
> > 81: 33 0 0 5478 1125 6585 4865 6306 0 0 0 714 3757 0 2662 1020 PCI-MSI-edge eth0-tx-2
> > 82: 21 1428 3136 3783 4249 6833 0 0 4175 0 0 1465 1940 1780 6286 0 PCI-MSI-edge eth0-tx-3
> > 83: 40 0 0 4788 0 1486 0 7185 3998 8637 1409 6785 0 0 0 10060 PCI-MSI-edge eth0-rx-0
> > 84: 70 1441 2606 1342 0 1678 4301 1839 0 6507 6910 16470 0 1484 1004 0 PCI-MSI-edge eth0-rx-1
> > 85: 40 1368 0 3560 5022 2040 5 1719 0 2037 956 40 0 17264 6456 2833 PCI-MSI-edge eth0-rx-2
> > 86: 44 414 5 1164 30 4513 4595 1542 0 10033 1151 0 4961 4203 0 10703 PCI-MSI-edge eth0-rx-3
> > 87: 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge eth0
>
> btw., i turned off CONFIG_PCI_MSI and that worked around the interface
> hang - the box has 40 minutes uptime now and still no hang. (it would hang
> within 5 minutes previously)
>
> can test the revert of any of these commits:
>
> e42e4ba: igb: fix anoying type mismatch warning on rx/tx queue sizing
> 8d25332: igb: Fix build warning when DCA is disabled.
> 26bc19e: igb: re-order queues to support cleaner use of ivar on 82576
> 0e014cb: igb: defeature tx head writeback
> 678c610: drivers/net/igb: remove dead code (function 'igb_read_pci_cfg')
> 908a7a1: net: Remove unused netdev arg from some NAPI interfaces.
> ea943d4: igb: fixup AER with proper error handling
> 5e8427e: igb: Correctly determine pci-e function number in virtual environment
> b4557be: igb: update handling of RCTL for smaller buffer sizes
> cb7b48f: igb/e1000e: Naming interrupt vectors
> 40a914f: igb: Add support for pci-e Advanced Error Reporting
> 527d47c: igb: link up/down messages must follow a specific format
> 5b9ab2e: Merge branch 'master' of master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6
> 69d728b: igb: loopback bits not correctly cleared from RCTL register
> 9b07f3d: igb: remove unneeded bit refrence when enabling jumbo frames
> f5f4cf0: igb: do not use phy ops in ethtool test cleanup for non-copper parts
> 0082982: netdev: add more functions to netdevice ops
> 68fd991: igb: Fix tx/rx_ring_count parameters for igb on suspend/resume/ring resize
> b2d5653: igb: simplify swap in clean_rx_irq if using packet split
> 2e5c692: igb: convert to net_device_ops
> 198d6ba: Merge branch 'master' of master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6
> 4cf1653: netdevice: safe convert to netdev_priv() #part-2
> babcda7: drivers/net: Kill now superfluous ->last_rx stores.
> 7c510e4: net: convert more to %pM
>
> i had a quick look at all of them but none seem to have a direct
> connection to MSI-X interrupt generation.
>
> There's a couple of ones that could have side effects:
>
> Maybe this one:
>
> 26bc19e: igb: re-order queues to support cleaner use of ivar on 82576
>
> due to its sheer commit size and due to its impact on the hw programming.
>
> or:
>
> 0e014cb: igb: defeature tx head writeback
>
> Might have some side-effect on tx-completion IRQs and might tickle
> firmware bugs?
>
> Or:
>
> 5e8427e: igb: Correctly determine pci-e function number in virtual environment
>
> while it should have no impact on a native kernel, it does change the PCI
> config space access sequences.
>
> Or maybe:
>
> 69d728b: igb: loopback bits not correctly cleared from RCTL register
>
> as this impacts the PCI programming too.
>
> But ... i dont really know this code so i'm guessing around.
Well, AER (Advanced Error Reporting) is currently broken w/ MSI-X, so maybe
this one:
40a914f: igb: Add support for pci-e Advanced Error Reporting
Thanks,
Rafael
* Rafael J. Wysocki <[email protected]> wrote:
> On Wednesday 21 January 2009, Ingo Molnar wrote:
> >
> > * Ingo Molnar <[email protected]> wrote:
> >
> > > > One other piece of info (in addition to what Bruce requested) that
> > > > would be useful is after you've done ifconfig eht0 up, cat
> > > > /proc/interrupts | grep eth0 and send that output (I want to see your
> > > > MSI-X configuration).
> > >
> > > here it is:
> > >
> > > 79: 35 0 9836 7451 1197 1360 766 0 2196 995 770 0 2051 863 4714 1879 PCI-MSI-edge eth0-tx-0
> > > 80: 2335 0 2718 3651 5024 9850 533 0 1405 1095 0 0 4421 1229 0 2002 PCI-MSI-edge eth0-tx-1
> > > 81: 33 0 0 5478 1125 6585 4865 6306 0 0 0 714 3757 0 2662 1020 PCI-MSI-edge eth0-tx-2
> > > 82: 21 1428 3136 3783 4249 6833 0 0 4175 0 0 1465 1940 1780 6286 0 PCI-MSI-edge eth0-tx-3
> > > 83: 40 0 0 4788 0 1486 0 7185 3998 8637 1409 6785 0 0 0 10060 PCI-MSI-edge eth0-rx-0
> > > 84: 70 1441 2606 1342 0 1678 4301 1839 0 6507 6910 16470 0 1484 1004 0 PCI-MSI-edge eth0-rx-1
> > > 85: 40 1368 0 3560 5022 2040 5 1719 0 2037 956 40 0 17264 6456 2833 PCI-MSI-edge eth0-rx-2
> > > 86: 44 414 5 1164 30 4513 4595 1542 0 10033 1151 0 4961 4203 0 10703 PCI-MSI-edge eth0-rx-3
> > > 87: 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge eth0
> >
> > btw., i turned off CONFIG_PCI_MSI and that worked around the interface
> > hang - the box has 40 minutes uptime now and still no hang. (it would hang
> > within 5 minutes previously)
> >
> > can test the revert of any of these commits:
> >
> > e42e4ba: igb: fix anoying type mismatch warning on rx/tx queue sizing
> > 8d25332: igb: Fix build warning when DCA is disabled.
> > 26bc19e: igb: re-order queues to support cleaner use of ivar on 82576
> > 0e014cb: igb: defeature tx head writeback
> > 678c610: drivers/net/igb: remove dead code (function 'igb_read_pci_cfg')
> > 908a7a1: net: Remove unused netdev arg from some NAPI interfaces.
> > ea943d4: igb: fixup AER with proper error handling
> > 5e8427e: igb: Correctly determine pci-e function number in virtual environment
> > b4557be: igb: update handling of RCTL for smaller buffer sizes
> > cb7b48f: igb/e1000e: Naming interrupt vectors
> > 40a914f: igb: Add support for pci-e Advanced Error Reporting
> > 527d47c: igb: link up/down messages must follow a specific format
> > 5b9ab2e: Merge branch 'master' of master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6
> > 69d728b: igb: loopback bits not correctly cleared from RCTL register
> > 9b07f3d: igb: remove unneeded bit refrence when enabling jumbo frames
> > f5f4cf0: igb: do not use phy ops in ethtool test cleanup for non-copper parts
> > 0082982: netdev: add more functions to netdevice ops
> > 68fd991: igb: Fix tx/rx_ring_count parameters for igb on suspend/resume/ring resize
> > b2d5653: igb: simplify swap in clean_rx_irq if using packet split
> > 2e5c692: igb: convert to net_device_ops
> > 198d6ba: Merge branch 'master' of master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6
> > 4cf1653: netdevice: safe convert to netdev_priv() #part-2
> > babcda7: drivers/net: Kill now superfluous ->last_rx stores.
> > 7c510e4: net: convert more to %pM
> >
> > i had a quick look at all of them but none seem to have a direct
> > connection to MSI-X interrupt generation.
> >
> > There's a couple of ones that could have side effects:
> >
> > Maybe this one:
> >
> > 26bc19e: igb: re-order queues to support cleaner use of ivar on 82576
> >
> > due to its sheer commit size and due to its impact on the hw programming.
> >
> > or:
> >
> > 0e014cb: igb: defeature tx head writeback
> >
> > Might have some side-effect on tx-completion IRQs and might tickle
> > firmware bugs?
> >
> > Or:
> >
> > 5e8427e: igb: Correctly determine pci-e function number in virtual environment
> >
> > while it should have no impact on a native kernel, it does change the PCI
> > config space access sequences.
> >
> > Or maybe:
> >
> > 69d728b: igb: loopback bits not correctly cleared from RCTL register
> >
> > as this impacts the PCI programming too.
> >
> > But ... i dont really know this code so i'm guessing around.
>
> Well, AER (Advanced Error Reporting) is currently broken w/ MSI-X, so maybe
> this one:
>
> 40a914f: igb: Add support for pci-e Advanced Error Reporting
ah, yeah, that makes sense. I discarded that commit because i was unsure
whether this box has AER and the boot log said:
calling aer_service_init+0x0/0x20 @ 1
initcall aer_service_init+0x0/0x20 returned 0 after 61 usecs
shouldnt it have printed something if AER was available?
Ingo
On Wednesday 21 January 2009, Ingo Molnar wrote:
>
> * Rafael J. Wysocki <[email protected]> wrote:
>
> > On Wednesday 21 January 2009, Ingo Molnar wrote:
> > >
> > > * Ingo Molnar <[email protected]> wrote:
> > >
> > > > > One other piece of info (in addition to what Bruce requested) that
> > > > > would be useful is after you've done ifconfig eht0 up, cat
> > > > > /proc/interrupts | grep eth0 and send that output (I want to see your
> > > > > MSI-X configuration).
> > > >
> > > > here it is:
> > > >
> > > > 79: 35 0 9836 7451 1197 1360 766 0 2196 995 770 0 2051 863 4714 1879 PCI-MSI-edge eth0-tx-0
> > > > 80: 2335 0 2718 3651 5024 9850 533 0 1405 1095 0 0 4421 1229 0 2002 PCI-MSI-edge eth0-tx-1
> > > > 81: 33 0 0 5478 1125 6585 4865 6306 0 0 0 714 3757 0 2662 1020 PCI-MSI-edge eth0-tx-2
> > > > 82: 21 1428 3136 3783 4249 6833 0 0 4175 0 0 1465 1940 1780 6286 0 PCI-MSI-edge eth0-tx-3
> > > > 83: 40 0 0 4788 0 1486 0 7185 3998 8637 1409 6785 0 0 0 10060 PCI-MSI-edge eth0-rx-0
> > > > 84: 70 1441 2606 1342 0 1678 4301 1839 0 6507 6910 16470 0 1484 1004 0 PCI-MSI-edge eth0-rx-1
> > > > 85: 40 1368 0 3560 5022 2040 5 1719 0 2037 956 40 0 17264 6456 2833 PCI-MSI-edge eth0-rx-2
> > > > 86: 44 414 5 1164 30 4513 4595 1542 0 10033 1151 0 4961 4203 0 10703 PCI-MSI-edge eth0-rx-3
> > > > 87: 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge eth0
> > >
> > > btw., i turned off CONFIG_PCI_MSI and that worked around the interface
> > > hang - the box has 40 minutes uptime now and still no hang. (it would hang
> > > within 5 minutes previously)
> > >
> > > can test the revert of any of these commits:
> > >
> > > e42e4ba: igb: fix anoying type mismatch warning on rx/tx queue sizing
> > > 8d25332: igb: Fix build warning when DCA is disabled.
> > > 26bc19e: igb: re-order queues to support cleaner use of ivar on 82576
> > > 0e014cb: igb: defeature tx head writeback
> > > 678c610: drivers/net/igb: remove dead code (function 'igb_read_pci_cfg')
> > > 908a7a1: net: Remove unused netdev arg from some NAPI interfaces.
> > > ea943d4: igb: fixup AER with proper error handling
> > > 5e8427e: igb: Correctly determine pci-e function number in virtual environment
> > > b4557be: igb: update handling of RCTL for smaller buffer sizes
> > > cb7b48f: igb/e1000e: Naming interrupt vectors
> > > 40a914f: igb: Add support for pci-e Advanced Error Reporting
> > > 527d47c: igb: link up/down messages must follow a specific format
> > > 5b9ab2e: Merge branch 'master' of master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6
> > > 69d728b: igb: loopback bits not correctly cleared from RCTL register
> > > 9b07f3d: igb: remove unneeded bit refrence when enabling jumbo frames
> > > f5f4cf0: igb: do not use phy ops in ethtool test cleanup for non-copper parts
> > > 0082982: netdev: add more functions to netdevice ops
> > > 68fd991: igb: Fix tx/rx_ring_count parameters for igb on suspend/resume/ring resize
> > > b2d5653: igb: simplify swap in clean_rx_irq if using packet split
> > > 2e5c692: igb: convert to net_device_ops
> > > 198d6ba: Merge branch 'master' of master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6
> > > 4cf1653: netdevice: safe convert to netdev_priv() #part-2
> > > babcda7: drivers/net: Kill now superfluous ->last_rx stores.
> > > 7c510e4: net: convert more to %pM
> > >
> > > i had a quick look at all of them but none seem to have a direct
> > > connection to MSI-X interrupt generation.
> > >
> > > There's a couple of ones that could have side effects:
> > >
> > > Maybe this one:
> > >
> > > 26bc19e: igb: re-order queues to support cleaner use of ivar on 82576
> > >
> > > due to its sheer commit size and due to its impact on the hw programming.
> > >
> > > or:
> > >
> > > 0e014cb: igb: defeature tx head writeback
> > >
> > > Might have some side-effect on tx-completion IRQs and might tickle
> > > firmware bugs?
> > >
> > > Or:
> > >
> > > 5e8427e: igb: Correctly determine pci-e function number in virtual environment
> > >
> > > while it should have no impact on a native kernel, it does change the PCI
> > > config space access sequences.
> > >
> > > Or maybe:
> > >
> > > 69d728b: igb: loopback bits not correctly cleared from RCTL register
> > >
> > > as this impacts the PCI programming too.
> > >
> > > But ... i dont really know this code so i'm guessing around.
> >
> > Well, AER (Advanced Error Reporting) is currently broken w/ MSI-X, so maybe
> > this one:
> >
> > 40a914f: igb: Add support for pci-e Advanced Error Reporting
>
> ah, yeah, that makes sense. I discarded that commit because i was unsure
> whether this box has AER and the boot log said:
>
> calling aer_service_init+0x0/0x20 @ 1
> initcall aer_service_init+0x0/0x20 returned 0 after 61 usecs
>
> shouldnt it have printed something if AER was available?
The driver doesn't seem to print anything on success as far as I can see.
It only reports faliures, apparently.
Thanks,
Rafael
Rafael J. Wysocki wrote:
> On Wednesday 21 January 2009, Ingo Molnar wrote:
>>
>> * Rafael J. Wysocki <[email protected]> wrote:
>>
>>> On Wednesday 21 January 2009, Ingo Molnar wrote:
>>>>
>>>> * Ingo Molnar <[email protected]> wrote:
>>>>
>>>>>> One other piece of info (in addition to what Bruce requested)
>>>>>> that would be useful is after you've done ifconfig eht0 up, cat
>>>>>> /proc/interrupts | grep eth0 and send that output (I want to see
>>>>>> your MSI-X configuration).
>>>>>
>>>>> here it is:
>>>>>
>>>>> 79: 35 0 9836 7451 1197
>>>>> 1360 766 0 2196 995 770
>>>>> 0 2051 863 4714 1879 PCI-MSI-edge
>>>>> eth0-tx-0 80: 2335 0 2718 3651
>>>>> 5024 9850 533 0 1405 1095
>>>>> 0 0 4421 1229 0 2002
>>>>> PCI-MSI-edge eth0-tx-1 81: 33 0 0
>>>>> 5478 1125 6585 4865 6306 0
>>>>> 0 0 714 3757 0 2662
>>>>> 1020 PCI-MSI-edge eth0-tx-2 82: 21 1428
>>>>> 3136 3783 4249 6833 0 0
>>>>> 4175 0 0 1465 1940 1780
>>>>> 6286 0 PCI-MSI-edge eth0-tx-3 83: 40
>>>>> 0 0 4788 0 1486 0
>>>>> 7185 3998 8637 1409 6785 0
>>>>> 0 0 10060 PCI-MSI-edge eth0-rx-0 84:
>>>>> 70 1441 2606 1342 0 1678
>>>>> 4301 1839 0 6507 6910 16470
>>>>> 0 1484 1004 0 PCI-MSI-edge eth0-rx-1
>>>>> 85: 40 1368 0 3560 5022
>>>>> 2040 5 1719 0 2037 956
>>>>> 40 0 17264 6456 2833 PCI-MSI-edge
>>>>> eth0-rx-2 86: 44 414 5 1164
>>>>> 30 4513 4595 1542 0 10033
>>>>> 1151 0 4961 4203 0 10703
>>>>> PCI-MSI-edge eth0-rx-3 87: 1 0 0
>>>>> 0 0 0 0 0 0
>>>>> 0 0 0 0 0 0
>>>>> 0 PCI-MSI-edge eth0
>>>>
>>>> btw., i turned off CONFIG_PCI_MSI and that worked around the
>>>> interface
>>>> hang - the box has 40 minutes uptime now and still no hang. (it
>>>> would hang
>>>> within 5 minutes previously)
>>>>
>>>> can test the revert of any of these commits:
>>>>
>>>> e42e4ba: igb: fix anoying type mismatch warning on rx/tx queue
>>>> sizing 8d25332: igb: Fix build warning when DCA is disabled.
>>>> 26bc19e: igb: re-order queues to support cleaner use of ivar on
>>>> 82576 0e014cb: igb: defeature tx head writeback
>>>> 678c610: drivers/net/igb: remove dead code (function
>>>> 'igb_read_pci_cfg') 908a7a1: net: Remove unused netdev arg from
>>>> some NAPI interfaces.
>>>> ea943d4: igb: fixup AER with proper error handling
>>>> 5e8427e: igb: Correctly determine pci-e function number in virtual
>>>> environment
>>>> b4557be: igb: update handling of RCTL for smaller buffer sizes
>>>> cb7b48f: igb/e1000e: Naming interrupt vectors
>>>> 40a914f: igb: Add support for pci-e Advanced Error Reporting
>>>> 527d47c: igb: link up/down messages must follow a specific format
>>>> 5b9ab2e: Merge branch 'master' of
>>>> master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6 69d728b:
>>>> igb: loopback bits not correctly cleared from RCTL register
>>>> 9b07f3d: igb: remove unneeded bit refrence when enabling jumbo
>>>> frames
>>>> f5f4cf0: igb: do not use phy ops in ethtool test cleanup for
>>>> non-copper parts 0082982: netdev: add more functions to netdevice
>>>> ops 68fd991: igb: Fix tx/rx_ring_count parameters for igb on
>>>> suspend/resume/ring resize b2d5653: igb: simplify swap in
>>>> clean_rx_irq if using packet split 2e5c692: igb: convert to
>>>> net_device_ops 198d6ba: Merge branch 'master' of
>>>> master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6 4cf1653:
>>>> netdevice: safe convert to netdev_priv() #part-2
>>>> babcda7: drivers/net: Kill now superfluous ->last_rx stores.
>>>> 7c510e4: net: convert more to %pM
>>>>
>>>> i had a quick look at all of them but none seem to have a direct
>>>> connection to MSI-X interrupt generation.
>>>>
>>>> There's a couple of ones that could have side effects:
>>>>
>>>> Maybe this one:
>>>>
>>>> 26bc19e: igb: re-order queues to support cleaner use of ivar on
>>>> 82576
>>>>
>>>> due to its sheer commit size and due to its impact on the hw
>>>> programming.
>>>>
>>>> or:
>>>>
>>>> 0e014cb: igb: defeature tx head writeback
>>>>
>>>> Might have some side-effect on tx-completion IRQs and might tickle
>>>> firmware bugs?
>>>>
>>>> Or:
>>>>
>>>> 5e8427e: igb: Correctly determine pci-e function number in
>>>> virtual environment
>>>>
>>>> while it should have no impact on a native kernel, it does change
>>>> the PCI
>>>> config space access sequences.
>>>>
>>>> Or maybe:
>>>>
>>>> 69d728b: igb: loopback bits not correctly cleared from RCTL
>>>> register
>>>>
>>>> as this impacts the PCI programming too.
>>>>
>>>> But ... i dont really know this code so i'm guessing around.
>>>
>>> Well, AER (Advanced Error Reporting) is currently broken w/ MSI-X,
>>> so maybe this one:
>>>
>>> 40a914f: igb: Add support for pci-e Advanced Error Reporting
>>
>> ah, yeah, that makes sense. I discarded that commit because i was
>> unsure whether this box has AER and the boot log said:
>>
>> calling aer_service_init+0x0/0x20 @ 1
>> initcall aer_service_init+0x0/0x20 returned 0 after 61 usecs
>>
>> shouldnt it have printed something if AER was available?
>
> The driver doesn't seem to print anything on success as far as I can
> see. It only reports faliures, apparently.
>
> Thanks,
> Rafael
I have noticed that too after going through the logs on one of the Nahalem systems I have here. It might just be best to disable AER in the kernel if it isn't functional on the system and see if that helps to clear up the issue.
I am leaning torward some external influence affecting the igb driver. Of the changes I made the only one I could really think of that would have much affect on MSI-X interrupts would be queue reordering, but that should have little to no effect on 82575 parts as it was intended to only reorder queues on 82576, in addition I would expect an immediate issue since the configuration is pretty much static after the queues are allocated. The only other patch that could even have a remote possiblity would be the TX head writeback defeature, but I would only expect that one to affect tx.
Thanks,
Alex
On Wed, 21 Jan 2009 14:51:07 -0800
"Duyck, Alexander H" <[email protected]> wrote:
> Rafael J. Wysocki wrote:
> > On Wednesday 21 January 2009, Ingo Molnar wrote:
> >>
> >> * Rafael J. Wysocki <[email protected]> wrote:
> >>
> >>> On Wednesday 21 January 2009, Ingo Molnar wrote:
> >>>>
> >>>> * Ingo Molnar <[email protected]> wrote:
> >>>>
> >>>>>> One other piece of info (in addition to what Bruce requested)
> >>>>>> that would be useful is after you've done ifconfig eht0 up, cat
> >>>>>> /proc/interrupts | grep eth0 and send that output (I want to see
> >>>>>> your MSI-X configuration).
> >>>>>
> >>>>> here it is:
> >>>>>
> >>>>> 79: 35 0 9836 7451 1197
> >>>>> 1360 766 0 2196 995 770
> >>>>> 0 2051 863 4714 1879 PCI-MSI-edge
> >>>>> eth0-tx-0 80: 2335 0 2718 3651
> >>>>> 5024 9850 533 0 1405 1095
> >>>>> 0 0 4421 1229 0 2002
> >>>>> PCI-MSI-edge eth0-tx-1 81: 33 0 0
> >>>>> 5478 1125 6585 4865 6306 0
> >>>>> 0 0 714 3757 0 2662
> >>>>> 1020 PCI-MSI-edge eth0-tx-2 82: 21 1428
> >>>>> 3136 3783 4249 6833 0 0
> >>>>> 4175 0 0 1465 1940 1780
> >>>>> 6286 0 PCI-MSI-edge eth0-tx-3 83: 40
> >>>>> 0 0 4788 0 1486 0
> >>>>> 7185 3998 8637 1409 6785 0
> >>>>> 0 0 10060 PCI-MSI-edge eth0-rx-0 84:
> >>>>> 70 1441 2606 1342 0 1678
> >>>>> 4301 1839 0 6507 6910 16470
> >>>>> 0 1484 1004 0 PCI-MSI-edge eth0-rx-1
> >>>>> 85: 40 1368 0 3560 5022
> >>>>> 2040 5 1719 0 2037 956
> >>>>> 40 0 17264 6456 2833 PCI-MSI-edge
> >>>>> eth0-rx-2 86: 44 414 5 1164
> >>>>> 30 4513 4595 1542 0 10033
> >>>>> 1151 0 4961 4203 0 10703
> >>>>> PCI-MSI-edge eth0-rx-3 87: 1 0 0
> >>>>> 0 0 0 0 0 0
> >>>>> 0 0 0 0 0 0
> >>>>> 0 PCI-MSI-edge eth0
> >>>>
> >>>> btw., i turned off CONFIG_PCI_MSI and that worked around the
> >>>> interface
> >>>> hang - the box has 40 minutes uptime now and still no hang. (it
> >>>> would hang
> >>>> within 5 minutes previously)
> >>>>
> >>>> can test the revert of any of these commits:
> >>>>
> >>>> e42e4ba: igb: fix anoying type mismatch warning on rx/tx queue
> >>>> sizing 8d25332: igb: Fix build warning when DCA is disabled.
> >>>> 26bc19e: igb: re-order queues to support cleaner use of ivar on
> >>>> 82576 0e014cb: igb: defeature tx head writeback
> >>>> 678c610: drivers/net/igb: remove dead code (function
> >>>> 'igb_read_pci_cfg') 908a7a1: net: Remove unused netdev arg from
> >>>> some NAPI interfaces.
> >>>> ea943d4: igb: fixup AER with proper error handling
> >>>> 5e8427e: igb: Correctly determine pci-e function number in virtual
> >>>> environment
> >>>> b4557be: igb: update handling of RCTL for smaller buffer sizes
> >>>> cb7b48f: igb/e1000e: Naming interrupt vectors
> >>>> 40a914f: igb: Add support for pci-e Advanced Error Reporting
> >>>> 527d47c: igb: link up/down messages must follow a specific format
> >>>> 5b9ab2e: Merge branch 'master' of
> >>>> master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6 69d728b:
> >>>> igb: loopback bits not correctly cleared from RCTL register
> >>>> 9b07f3d: igb: remove unneeded bit refrence when enabling jumbo
> >>>> frames
> >>>> f5f4cf0: igb: do not use phy ops in ethtool test cleanup for
> >>>> non-copper parts 0082982: netdev: add more functions to netdevice
> >>>> ops 68fd991: igb: Fix tx/rx_ring_count parameters for igb on
> >>>> suspend/resume/ring resize b2d5653: igb: simplify swap in
> >>>> clean_rx_irq if using packet split 2e5c692: igb: convert to
> >>>> net_device_ops 198d6ba: Merge branch 'master' of
> >>>> master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6 4cf1653:
> >>>> netdevice: safe convert to netdev_priv() #part-2
> >>>> babcda7: drivers/net: Kill now superfluous ->last_rx stores.
> >>>> 7c510e4: net: convert more to %pM
> >>>>
> >>>> i had a quick look at all of them but none seem to have a direct
> >>>> connection to MSI-X interrupt generation.
> >>>>
> >>>> There's a couple of ones that could have side effects:
> >>>>
> >>>> Maybe this one:
> >>>>
> >>>> 26bc19e: igb: re-order queues to support cleaner use of ivar on
> >>>> 82576
> >>>>
> >>>> due to its sheer commit size and due to its impact on the hw
> >>>> programming.
> >>>>
> >>>> or:
> >>>>
> >>>> 0e014cb: igb: defeature tx head writeback
> >>>>
> >>>> Might have some side-effect on tx-completion IRQs and might tickle
> >>>> firmware bugs?
> >>>>
> >>>> Or:
> >>>>
> >>>> 5e8427e: igb: Correctly determine pci-e function number in
> >>>> virtual environment
> >>>>
> >>>> while it should have no impact on a native kernel, it does change
> >>>> the PCI
> >>>> config space access sequences.
> >>>>
> >>>> Or maybe:
> >>>>
> >>>> 69d728b: igb: loopback bits not correctly cleared from RCTL
> >>>> register
> >>>>
> >>>> as this impacts the PCI programming too.
> >>>>
> >>>> But ... i dont really know this code so i'm guessing around.
> >>>
> >>> Well, AER (Advanced Error Reporting) is currently broken w/ MSI-X,
> >>> so maybe this one:
> >>>
> >>> 40a914f: igb: Add support for pci-e Advanced Error Reporting
> >>
> >> ah, yeah, that makes sense. I discarded that commit because i was
> >> unsure whether this box has AER and the boot log said:
> >>
> >> calling aer_service_init+0x0/0x20 @ 1
> >> initcall aer_service_init+0x0/0x20 returned 0 after 61 usecs
> >>
> >> shouldnt it have printed something if AER was available?
> >
> > The driver doesn't seem to print anything on success as far as I can
> > see. It only reports faliures, apparently.
> >
> > Thanks,
> > Rafael
>
> I have noticed that too after going through the logs on one of the Nahalem systems I have here. It might just be best to disable AER in the kernel if it isn't functional on the system and see if that helps to clear up the issue.
>
> I am leaning torward some external influence affecting the igb driver. Of the changes I made the only one I could really think of that would have much affect on MSI-X interrupts would be queue reordering, but that should have little to no effect on 82575 parts as it was intended to only reorder queues on 82576, in addition I would expect an immediate issue since the configuration is pretty much static after the queues are allocated. The only other patch that could even have a remote possiblity would be the TX head writeback defeature, but I would only expect that one to affect tx.
>
> Thanks,
>
> Alex
AER wont work unless PCI supports MMCONFIG.
This is a real problem, that I ended up giving up on standard pci access for sky2.