Hi, [please CC me on replies, as I am not subscribed]
booting 2.6.30-rc2-git7 on a HP/DL380G3 (x86_64, RHEL4.3, 64 bit userspace) is much slower that it used to be. It seems I run into timeouts when [trying to] load intel and tg3 microcodes:
[ 14.478892] platform microcode: firmware: requesting intel-ucode/0f-04-0a
[ 74.476741] platform microcode: firmware: requesting intel-ucode/0f-04-0a
[ 134.476638] platform microcode: firmware: requesting intel-ucode/0f-04-0a
[ 194.476637] platform microcode: firmware: requesting intel-ucode/0f-04-0a
[ 254.476493] Microcode Update Driver: v2.00 <[email protected]>, Peter Oruba
[ 254.718489] Microcode Update Driver: v2.00 removed.
So, we see 60 seconds for eaoch of the CPUs
[ 255.273426] tg3 0000:03:01.0: PME# disabled
[ 257.833769] tg3: eth0: Link is up at 1000 Mbps, full duplex.
[ 257.833775] tg3: eth0: Flow control is off for TX and off for RX.
[ 269.643973] tg3 0000:03:01.1: firmware: requesting tigon/tg3_tso.bin
[ 329.640456] eth1: Failed to load firmware "tigon/tg3_tso.bin"
[ 329.640463] eth1: TSO capability disabled.
[ 329.640487] tg3 0000:03:01.1: PME# disabled
[ 333.081753] tg3: eth1: Link is up at 1000 Mbps, full duplex.
[ 333.081759] tg3: eth1: Flow control is off for TX and off for RX.
We see 60 seconds for eth1, complaining about a failed firmware load.
/lib/firmware/tigon/tg3_tso.bin does exist and is from the current
build.
So, any ideas? Config is attached
Cheers
Martin
------------------------------------------------------
Martin Knoblauch
email: k n o b i AT knobisoft DOT de
www: http://www.knobisoft.de
----- Original Message ----
> From: Martin Knoblauch <[email protected]>
> To: [email protected]
> Sent: Tuesday, April 21, 2009 4:21:41 PM
> Subject: Booting 2.6.30-rc2-git7 very slow
>
>
> Hi, [please CC me on replies, as I am not subscribed]
>
> booting 2.6.30-rc2-git7 on a HP/DL380G3 (x86_64, RHEL4.3, 64 bit userspace) is
> much slower that it used to be. It seems I run into timeouts when [trying to]
> load intel and tg3 microcodes:
>
> [ 14.478892] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> [ 74.476741] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> [ 134.476638] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> [ 194.476637] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> [ 254.476493] Microcode Update Driver: v2.00 , Peter Oruba
> [ 254.718489] Microcode Update Driver: v2.00 removed.
>
> So, we see 60 seconds for eaoch of the CPUs. I do not know where the ucode is searched for.
>
> [ 255.273426] tg3 0000:03:01.0: PME# disabled
> [ 257.833769] tg3: eth0: Link is up at 1000 Mbps, full duplex.
> [ 257.833775] tg3: eth0: Flow control is off for TX and off for RX.
> [ 269.643973] tg3 0000:03:01.1: firmware: requesting tigon/tg3_tso.bin
> [ 329.640456] eth1: Failed to load firmware "tigon/tg3_tso.bin"
> [ 329.640463] eth1: TSO capability disabled.
> [ 329.640487] tg3 0000:03:01.1: PME# disabled
> [ 333.081753] tg3: eth1: Link is up at 1000 Mbps, full duplex.
> [ 333.081759] tg3: eth1: Flow control is off for TX and off for RX.
>
> We see 60 seconds for eth1, complaining about a failed firmware load.
> /lib/firmware/tigon/tg3_tso.bin does exist and is from the current
> build.
>
>
> So, any ideas? Config is attached
>
This time really....
Btw. without the timeouts the boot sequence would likely finish in less than 60 seconds, compared to 250 seconds for earlier kernels.
> Cheers
> Martin
> ------------------------------------------------------
> Martin Knoblauch
> email: k n o b i AT knobisoft DOT de
> www: http://www.knobisoft.de
On Tuesday 21 April 2009, Martin Knoblauch wrote:
>
> Hi, [please CC me on replies, as I am not subscribed]
>
> booting 2.6.30-rc2-git7 on a HP/DL380G3 (x86_64, RHEL4.3, 64 bit userspace) is much slower that it used to be. It seems I run into timeouts when [trying to] load intel and tg3 microcodes:
>
> [ 14.478892] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> [ 74.476741] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> [ 134.476638] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> [ 194.476637] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> [ 254.476493] Microcode Update Driver: v2.00 <[email protected]>, Peter Oruba
> [ 254.718489] Microcode Update Driver: v2.00 removed.
>
> So, we see 60 seconds for eaoch of the CPUs
>
> [ 255.273426] tg3 0000:03:01.0: PME# disabled
> [ 257.833769] tg3: eth0: Link is up at 1000 Mbps, full duplex.
> [ 257.833775] tg3: eth0: Flow control is off for TX and off for RX.
> [ 269.643973] tg3 0000:03:01.1: firmware: requesting tigon/tg3_tso.bin
> [ 329.640456] eth1: Failed to load firmware "tigon/tg3_tso.bin"
> [ 329.640463] eth1: TSO capability disabled.
> [ 329.640487] tg3 0000:03:01.1: PME# disabled
> [ 333.081753] tg3: eth1: Link is up at 1000 Mbps, full duplex.
> [ 333.081759] tg3: eth1: Flow control is off for TX and off for RX.
>
> We see 60 seconds for eth1, complaining about a failed firmware load.
> /lib/firmware/tigon/tg3_tso.bin does exist and is from the current
> build.
Do I assume correctly that 2.6.29 did not have this problem?
Rafael
----- Original Message ----
> From: Rafael J. Wysocki <[email protected]>
> To: Martin Knoblauch <[email protected]>
> Cc: [email protected]
> Sent: Tuesday, April 21, 2009 9:12:20 PM
> Subject: Re: Booting 2.6.30-rc2-git7 very slow
>
> On Tuesday 21 April 2009, Martin Knoblauch wrote:
> >
> > Hi, [please CC me on replies, as I am not subscribed]
> >
> > booting 2.6.30-rc2-git7 on a HP/DL380G3 (x86_64, RHEL4.3, 64 bit userspace)
> is much slower that it used to be. It seems I run into timeouts when [trying to]
> load intel and tg3 microcodes:
> >
> > [ 14.478892] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> > [ 74.476741] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> > [ 134.476638] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> > [ 194.476637] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> > [ 254.476493] Microcode Update Driver: v2.00 ,
> Peter Oruba
> > [ 254.718489] Microcode Update Driver: v2.00 removed.
> >
> > So, we see 60 seconds for eaoch of the CPUs
> >
> > [ 255.273426] tg3 0000:03:01.0: PME# disabled
> > [ 257.833769] tg3: eth0: Link is up at 1000 Mbps, full duplex.
> > [ 257.833775] tg3: eth0: Flow control is off for TX and off for RX.
> > [ 269.643973] tg3 0000:03:01.1: firmware: requesting tigon/tg3_tso.bin
> > [ 329.640456] eth1: Failed to load firmware "tigon/tg3_tso.bin"
> > [ 329.640463] eth1: TSO capability disabled.
> > [ 329.640487] tg3 0000:03:01.1: PME# disabled
> > [ 333.081753] tg3: eth1: Link is up at 1000 Mbps, full duplex.
> > [ 333.081759] tg3: eth1: Flow control is off for TX and off for RX.
> >
> > We see 60 seconds for eth1, complaining about a failed firmware load.
> > /lib/firmware/tigon/tg3_tso.bin does exist and is from the current
> > build.
>
> Do I assume correctly that 2.6.29 did not have this problem?
>
I can tell that 2.6.28.2 did not have that problem. I skipped 2.6.29, so I cannot be sure. This is from the 2.6.28.2 boot sequence:
[ 14.148605] platform microcode: firmware: requesting intel-ucode/0f-04-0a
[ 14.194728] platform microcode: firmware: requesting intel-ucode/0f-04-0a
[ 14.226354] platform microcode: firmware: requesting intel-ucode/0f-04-0a
[ 14.258677] platform microcode: firmware: requesting intel-ucode/0f-04-0a
[ 14.291483] Microcode Update Driver: v2.00 <[email protected]>, Peter Oruba
[ 14.530193] Microcode Update Driver: v2.00 removed.
[ 17.629690] tg3: eth0: Link is up at 1000 Mbps, full duplex.
[ 17.629695] tg3: eth0: Flow control is off for TX and off for RX.
[ 33.328198] tg3: eth1: Link is up at 1000 Mbps, full duplex.
[ 33.328203] tg3: eth1: Flow control is off for TX and off for RX.
[ 44.737601] RPC: Registered udp transport module.
[ 44.737606] RPC: Registered tcp transport module.
[ 47.637782] warning: process `snmpd' used the deprecated sysctl system call with 7.2.1.
[ 57.671627] warning: `ntpd' uses 32-bit capabilities (legacy support in use)
Cheers
Martin
> From: Rafael J. Wysocki <[email protected]>
> To: Martin Knoblauch <[email protected]>
> Cc: [email protected]
> Sent: Tuesday, April 21, 2009 9:12:20 PM
> Subject: Re: Booting 2.6.30-rc2-git7 very slow
>
> On Tuesday 21 April 2009, Martin Knoblauch wrote:
> >
> > Hi, [please CC me on replies, as I am not subscribed]
> >
> > booting 2.6.30-rc2-git7 on a HP/DL380G3 (x86_64, RHEL4.3, 64 bit userspace)
> is much slower that it used to be. It seems I run into timeouts when [trying to]
> load intel and tg3 microcodes:
> >
> > [ 14.478892] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> > [ 74.476741] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> > [ 134.476638] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> > [ 194.476637] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> > [ 254.476493] Microcode Update Driver: v2.00 ,
> Peter Oruba
> > [ 254.718489] Microcode Update Driver: v2.00 removed.
> >
> > So, we see 60 seconds for eaoch of the CPUs
> >
> > [ 255.273426] tg3 0000:03:01.0: PME# disabled
> > [ 257.833769] tg3: eth0: Link is up at 1000 Mbps, full duplex.
> > [ 257.833775] tg3: eth0: Flow control is off for TX and off for RX.
> > [ 269.643973] tg3 0000:03:01.1: firmware: requesting tigon/tg3_tso.bin
> > [ 329.640456] eth1: Failed to load firmware "tigon/tg3_tso.bin"
> > [ 329.640463] eth1: TSO capability disabled.
> > [ 329.640487] tg3 0000:03:01.1: PME# disabled
> > [ 333.081753] tg3: eth1: Link is up at 1000 Mbps, full duplex.
> > [ 333.081759] tg3: eth1: Flow control is off for TX and off for RX.
> >
> > We see 60 seconds for eth1, complaining about a failed firmware load.
> > /lib/firmware/tigon/tg3_tso.bin does exist and is from the current
> > build.
>
> Do I assume correctly that 2.6.29 did not have this problem?
>
Just checked. 2.6.29 has exactely the same problem. 2.6.28.2 was OK. This is from the 2.6.29 boot:
[ 14.308340] platform microcode: firmware: requesting intel-ucode/0f-04-0a
[ 74.304612] platform microcode: firmware: requesting intel-ucode/0f-04-0a
[ 134.304651] platform microcode: firmware: requesting intel-ucode/0f-04-0a
[ 194.304638] platform microcode: firmware: requesting intel-ucode/0f-04-0a
[ 254.304597] Microcode Update Driver: v2.00 <[email protected]>, Peter Oruba
[ 254.546200] Microcode Update Driver: v2.00 removed.
[ 255.088405] tg3 0000:03:01.0: PME# disabled
[ 257.669617] tg3: eth0: Link is up at 1000 Mbps, full duplex.
[ 257.669622] tg3: eth0: Flow control is off for TX and off for RX.
[ 269.456132] tg3 0000:03:01.1: firmware: requesting tigon/tg3_tso.bin
[ 329.456495] eth1: Failed to load firmware "tigon/tg3_tso.bin"
[ 329.456500] eth1: TSO capability disabled.
[ 329.456524] tg3 0000:03:01.1: PME# disabled
[ 332.921832] tg3: eth1: Link is up at 1000 Mbps, full duplex.
[ 332.921837] tg3: eth1: Flow control is off for TX and off for RX.
Cheers
Martin
----- Original Message ----
> From: Martin Knoblauch <[email protected]>
> To: Rafael J. Wysocki <[email protected]>
> Cc: [email protected]
> Sent: Wednesday, April 22, 2009 11:56:27 AM
> Subject: Re: Booting 2.6.30-rc2-git7 very slow
>
> > From: Rafael J. Wysocki
>
> > To: Martin Knoblauch
> > Cc: [email protected]
> > Sent: Tuesday, April 21, 2009 9:12:20 PM
> > Subject: Re: Booting 2.6.30-rc2-git7 very slow
> >
> > On Tuesday 21 April 2009, Martin Knoblauch wrote:
> > >
> > > Hi, [please CC me on replies, as I am not subscribed]
> > >
> > > booting 2.6.30-rc2-git7 on a HP/DL380G3 (x86_64, RHEL4.3, 64 bit
> userspace)
> > is much slower that it used to be. It seems I run into timeouts when [trying
> to]
> > load intel and tg3 microcodes:
> > >
> > > [ 14.478892] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> > > [ 74.476741] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> > > [ 134.476638] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> > > [ 194.476637] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> > > [ 254.476493] Microcode Update Driver: v2.00 ,
> > Peter Oruba
> > > [ 254.718489] Microcode Update Driver: v2.00 removed.
> > >
> > > So, we see 60 seconds for eaoch of the CPUs
> > >
> > > [ 255.273426] tg3 0000:03:01.0: PME# disabled
> > > [ 257.833769] tg3: eth0: Link is up at 1000 Mbps, full duplex.
> > > [ 257.833775] tg3: eth0: Flow control is off for TX and off for RX.
> > > [ 269.643973] tg3 0000:03:01.1: firmware: requesting tigon/tg3_tso.bin
> > > [ 329.640456] eth1: Failed to load firmware "tigon/tg3_tso.bin"
> > > [ 329.640463] eth1: TSO capability disabled.
> > > [ 329.640487] tg3 0000:03:01.1: PME# disabled
> > > [ 333.081753] tg3: eth1: Link is up at 1000 Mbps, full duplex.
> > > [ 333.081759] tg3: eth1: Flow control is off for TX and off for RX.
> > >
> > > We see 60 seconds for eth1, complaining about a failed firmware load.
> > > /lib/firmware/tigon/tg3_tso.bin does exist and is from the current
> > > build.
> >
> > Do I assume correctly that 2.6.29 did not have this problem?
> >
>
> Just checked. 2.6.29 has exactely the same problem. 2.6.28.2 was OK. This is
> from the 2.6.29 boot:
>
> [ 14.308340] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> [ 74.304612] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> [ 134.304651] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> [ 194.304638] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> [ 254.304597] Microcode Update Driver: v2.00 , Peter Oruba
> [ 254.546200] Microcode Update Driver: v2.00 removed.
In case of the platform microcode, the delays happen when doing:
/sbin/modprobe microcode
from the init script. I have a "microcode.dat" File in both /etc/ and /etc/firmware.
> [ 255.088405] tg3 0000:03:01.0: PME# disabled
> [ 257.669617] tg3: eth0: Link is up at 1000 Mbps, full duplex.
> [ 257.669622] tg3: eth0: Flow control is off for TX and off for RX.
> [ 269.456132] tg3 0000:03:01.1: firmware: requesting tigon/tg3_tso.bin
> [ 329.456495] eth1: Failed to load firmware "tigon/tg3_tso.bin"
> [ 329.456500] eth1: TSO capability disabled.
> [ 329.456524] tg3 0000:03:01.1: PME# disabled
> [ 332.921832] tg3: eth1: Link is up at 1000 Mbps, full duplex.
> [ 332.921837] tg3: eth1: Flow control is off for TX and off for RX.
>
>
Thanks
Martin
----- Original Message ----
> From: Martin Knoblauch <[email protected]>
> To: Rafael J. Wysocki <[email protected]>
> Cc: [email protected]; [email protected]
> Sent: Wednesday, April 22, 2009 1:32:25 PM
> Subject: Re: Booting 2.6.30-rc2-git7 very slow
>
> ----- Original Message ----
>
> > From: Martin Knoblauch
> > To: Rafael J. Wysocki
> > Cc: [email protected]
> > Sent: Wednesday, April 22, 2009 11:56:27 AM
> > Subject: Re: Booting 2.6.30-rc2-git7 very slow
> >
> > > From: Rafael J. Wysocki
> >
> > > To: Martin Knoblauch
> > > Cc: [email protected]
> > > Sent: Tuesday, April 21, 2009 9:12:20 PM
> > > Subject: Re: Booting 2.6.30-rc2-git7 very slow
> > >
> > > On Tuesday 21 April 2009, Martin Knoblauch wrote:
> > > >
> > > > Hi, [please CC me on replies, as I am not subscribed]
> > > >
> > > > booting 2.6.30-rc2-git7 on a HP/DL380G3 (x86_64, RHEL4.3, 64 bit
> > userspace)
> > > is much slower that it used to be. It seems I run into timeouts when [trying
>
> > to]
> > > load intel and tg3 microcodes:
> > > >
> > > > [ 14.478892] platform microcode: firmware: requesting
> intel-ucode/0f-04-0a
> > > > [ 74.476741] platform microcode: firmware: requesting
> intel-ucode/0f-04-0a
> > > > [ 134.476638] platform microcode: firmware: requesting
> intel-ucode/0f-04-0a
> > > > [ 194.476637] platform microcode: firmware: requesting
> intel-ucode/0f-04-0a
> > > > [ 254.476493] Microcode Update Driver: v2.00 ,
> > > Peter Oruba
> > > > [ 254.718489] Microcode Update Driver: v2.00 removed.
> > > >
> > > > So, we see 60 seconds for eaoch of the CPUs
> > > >
> > > > [ 255.273426] tg3 0000:03:01.0: PME# disabled
> > > > [ 257.833769] tg3: eth0: Link is up at 1000 Mbps, full duplex.
> > > > [ 257.833775] tg3: eth0: Flow control is off for TX and off for RX.
> > > > [ 269.643973] tg3 0000:03:01.1: firmware: requesting tigon/tg3_tso.bin
> > > > [ 329.640456] eth1: Failed to load firmware "tigon/tg3_tso.bin"
> > > > [ 329.640463] eth1: TSO capability disabled.
> > > > [ 329.640487] tg3 0000:03:01.1: PME# disabled
> > > > [ 333.081753] tg3: eth1: Link is up at 1000 Mbps, full duplex.
> > > > [ 333.081759] tg3: eth1: Flow control is off for TX and off for RX.
> > > >
> > > > We see 60 seconds for eth1, complaining about a failed firmware load.
> > > > /lib/firmware/tigon/tg3_tso.bin does exist and is from the current
> > > > build.
> > >
> > > Do I assume correctly that 2.6.29 did not have this problem?
> > >
> >
> > Just checked. 2.6.29 has exactely the same problem. 2.6.28.2 was OK. This is
> > from the 2.6.29 boot:
> >
> > [ 14.308340] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> > [ 74.304612] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> > [ 134.304651] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> > [ 194.304638] platform microcode: firmware: requesting intel-ucode/0f-04-0a
> > [ 254.304597] Microcode Update Driver: v2.00 , Peter Oruba
> > [ 254.546200] Microcode Update Driver: v2.00 removed.
>
> In case of the platform microcode, the delays happen when doing:
>
> /sbin/modprobe microcode
>
> from the init script. I have a "microcode.dat" File in both /etc/ and
> /etc/firmware.
>
> > [ 255.088405] tg3 0000:03:01.0: PME# disabled
> > [ 257.669617] tg3: eth0: Link is up at 1000 Mbps, full duplex.
> > [ 257.669622] tg3: eth0: Flow control is off for TX and off for RX.
> > [ 269.456132] tg3 0000:03:01.1: firmware: requesting tigon/tg3_tso.bin
> > [ 329.456495] eth1: Failed to load firmware "tigon/tg3_tso.bin"
> > [ 329.456500] eth1: TSO capability disabled.
> > [ 329.456524] tg3 0000:03:01.1: PME# disabled
> > [ 332.921832] tg3: eth1: Link is up at 1000 Mbps, full duplex.
> > [ 332.921837] tg3: eth1: Flow control is off for TX and off for RX.
> >
> >
>
OK, I was able to solve the tg3 failure by adding "tigon/tg3_tso.bin" to CONFIG_EXTRA_FIRMWARE. I tried the same with the CPU microcode (after copying /etc/firmware/microcode.dat to $BUILDDIR/firmware/), but no success.
Searching the archives also found some mentioning that I might need special udev-rules to make firmware loading work again transparently. But no explicit solution.
I find this new (since 2.6.29) behaviour:
- very unexpected
- not well documented
- the EXTRA_FIRMWARE solution very unscalable
Cheers
Martin