Hello!
I was busying myself with this for about two weeks, and today i am likely to
say
that this is a bug in the kernel.
The symptoms for short:
On 2.4.x (actually 2.4.27) kernel, disk i/o operations perform well (no
noticable cpu overhead). But: on kernel 2.6.9-rc1-bk8 (and all the other
2.6.x that i tested), disk i/o operations perform bad: full cpu usage
(sys) when copying files or doing other i/o operations.
And to make sure that the new scheduling interface is not the "bad guy" i
tried different schedulers with elevator=<something>. The bug was still
there.
That is what i was doing to resolve the problem:
I wrote to Lionel (the sis5513.c author found in MAINTAINERS) on 20040831:
----8<-----
I am using linux-2.6.x kernel with sis5513 support enabled. I discovered a
strange performance problem that was mentioned on bugzilla.kernel.org some time
before [1.]. For a full and details report you may have a look at [2.].
[1.] http://bugzilla.kernel.org/show_bug.cgi?id=2983
[2.] http://rcswww.urz.tu-dresden.de/~s4248297/gubed/report.html
I dont know if it is a problem of the sis5513.c code, but i hope you may be
helpful in finding the right person to whom i should send this bug.
----8<-----
He tried to figure out, if this behaviour may be a problem of his driver,
and it seems that this is not the case. He sugested that i may post this
problem to this list, that is, what i am doing here. You can see the full
conversation with Linonel at the web page at [2.]
IMHO it must have to do with the Uniform Multi-Platform E-IDE driver. The
2.4 kernel uses revision 7.00beta4-2.4 and the 2.6 uses revision
7.00alpha2. If this sugestion is wrong excuse me, i am not a C programer.
Everyone who is keen on solving this problem should have a look at the very
detailed description of the bug on the web page [2.]. I kept it as close as
possible to the rules in the REPORTING-BUGS file (hey i even made some
screenshots).
The question is: why would the same hardware combination on the 2.4 kernel
perform well, and not so well on 2.6.
PS: please CC me, i am not on the list ;-)
Best regards,
Hendrik Fehr.
Alan Cox <[email protected]> wrote:
> Please post the boot "dmesg" log.
>
This is, what dmesg gives me with 2.6.9-rc1-bk8:
(note: i use ide0=ata66 because my machine is a laptop which uses a short 40c
wire that is equal to an long (i thing 18 inches) 80c cable.)
Linux version 2.6.9-rc1-bk8 (hefe@anfortas) (gcc-Version 3.3.4 20040623 (Gentoo
Linux 3.3.4-r1, ssp-3.3.2-2, pie-8.7.6)) #1 Wed Sep 1 14:44:55 MEST 2004
BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009f400 (usable)
BIOS-e820: 000000000009f400 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000d0000 - 00000000000d8000 (reserved)
BIOS-e820: 00000000000e4000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 000000001fdf0000 (usable)
BIOS-e820: 000000001fdf0000 - 000000001fdfb000 (ACPI data)
BIOS-e820: 000000001fdfb000 - 000000001fe00000 (ACPI NVS)
BIOS-e820: 000000001fe00000 - 0000000020000000 (reserved)
BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved)
BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
BIOS-e820: 00000000fff80000 - 0000000100000000 (reserved)
509MB LOWMEM available.
found SMP MP-table at 000f6990
On node 0 totalpages: 130544
DMA zone: 4096 pages, LIFO batch:1
Normal zone: 126448 pages, LIFO batch:16
HighMem zone: 0 pages, LIFO batch:1
DMI present.
ACPI: RSDP (v000 PTLTD ) @ 0x000f69f0
ACPI: RSDT (v001 PTLTD RSDT 0x06040000 LTP 0x00000000) @ 0x1fdf7151
ACPI: FADT (v001 ECS G732 0x06040000 PTL 0x000f4240) @ 0x1fdfaf3c
ACPI: MADT (v001 PTLTD APIC 0x06040000 LTP 0x00000000) @
0x1fdfafb0
ACPI: DSDT (v001 PTLTD ECS 0x06040000 MSFT 0x0100000e) @ 0x00000000
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
Processor #0 15:2 APIC version 20
ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-23
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 16 low level)
ACPI: IRQ0 used by override.
ACPI: IRQ11 used by override.
Enabling APIC mode: Flat. Using 1 I/O APICs
Using ACPI (MADT) for SMP configuration information
Built 1 zonelists
Kernel command line: BOOT_IMAGE=bk8dev26 root=305 psmouse.proto=imps hdc=cdrom
ide0=ata66
ide_setup: hdc=cdrom
ide_setup: ide0=ata66
Initializing CPU#0
PID hash table entries: 2048 (order: 11, 32768 bytes)
Detected 2390.684 MHz processor.
Using tsc for high-res timesource
Console: colour dummy device 80x25
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
Memory: 514196k/522176k available (1828k kernel code, 7456k reserved, 754k data,
148k init, 0k highmem)
Checking if this processor honours the WP bit even in supervisor mode... Ok.
Calibrating delay loop... 4718.59 BogoMIPS (lpj=2359296)
Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
CPU: After generic identify, caps: bfebfbff 00000000 00000000 00000000
CPU: After vendor identify, caps: bfebfbff 00000000 00000000 00000000
CPU: Trace cache: 12K uops, L1 D cache: 8K
CPU: L2 cache: 512K
CPU: After all inits, caps: bfebfbff 00000000 00000000 00000080
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#0.
CPU0: Intel P4/Xeon Extended MCE MSRs (12) available
CPU0: Thermal monitoring enabled
CPU: Intel(R) Pentium(R) 4 CPU 2.40GHz stepping 07
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Checking 'hlt' instruction... OK.
ENABLING IO-APIC IRQs
..TIMER: vector=0x31 pin1=16 pin2=-1
..MP-BIOS bug: 8254 timer not connected to IO-APIC
...trying to set up timer (IRQ0) through the 8259A ... failed.
...trying to set up timer as Virtual Wire IRQ... works.
NET: Registered protocol family 16
EISA bus registered
PCI: PCI BIOS revision 2.10 entry at 0xfd9b8, last bus=1
PCI: Using configuration type 1
mtrr: v2.0 (20020519)
ACPI: Subsystem revision 20040715
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: PCI Root Bridge [PCI0] (00:00)
PCI: Probing PCI hardware (bus 00)
Uncovering SIS962 that hid as a SIS503 (compatible=1)
Enabling SiS 96x SMBus.
PCI: Ignoring BAR0-3 of IDE controller 0000:00:02.5
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: Power Resource [URP1] (off)
ACPI: Power Resource [URP2] (off)
ACPI: Power Resource [FDDP] (off)
ACPI: Power Resource [LPTP] (off)
ACPI: PCI Interrupt Link [LNKA] (IRQs 5 10 11) *4
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 7 9 10 *11)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 *9 10 11)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 7 *9 10 11)
ACPI: PCI Interrupt Link [LNKE] (IRQs 4 5 10 11) *9
ACPI: PCI Interrupt Link [LNKF] (IRQs 4 5 10 *11)
ACPI: PCI Interrupt Link [LNKG] (IRQs 4 5 *10 11)
ACPI: PCI Interrupt Link [LNKH] (IRQs 4 *5 10 11)
ACPI: Embedded Controller [EC] (gpe 26)
Linux Plug and Play Support v0.97 (c) Adam Belay
pnp: the driver 'system' has been registered
PCI: Using ACPI for IRQ routing
ACPI: PCI interrupt 0000:00:02.1[B] -> GSI 17 (level, low) -> IRQ 17
ACPI: PCI interrupt 0000:00:02.3[B] -> GSI 17 (level, low) -> IRQ 17
ACPI: PCI interrupt 0000:00:02.5[A] -> GSI 16 (level, low) -> IRQ 16
ACPI: PCI interrupt 0000:00:02.6[C] -> GSI 18 (level, low) -> IRQ 18
ACPI: PCI interrupt 0000:00:02.7[C] -> GSI 18 (level, low) -> IRQ 18
ACPI: PCI interrupt 0000:00:03.0[A] -> GSI 20 (level, low) -> IRQ 20
ACPI: PCI interrupt 0000:00:03.1[B] -> GSI 21 (level, low) -> IRQ 21
ACPI: PCI interrupt 0000:00:03.2[C] -> GSI 22 (level, low) -> IRQ 22
ACPI: PCI interrupt 0000:00:03.3[D] -> GSI 23 (level, low) -> IRQ 23
ACPI: PCI interrupt 0000:00:04.0[A] -> GSI 19 (level, low) -> IRQ 19
ACPI: PCI interrupt 0000:00:0a.0[A] -> GSI 17 (level, low) -> IRQ 17
ACPI: PCI interrupt 0000:01:00.0[A] -> GSI 16 (level, low) -> IRQ 16
ACPI: PCI interrupt 0000:01:00.0[A] -> GSI 16 (level, low) -> IRQ 16
radeonfb: Retreived PLL infos from BIOS
radeonfb: Reference=27.00 MHz (RefDiv=12) Memory=260.00 Mhz, System=200.00 MHz
Non-DDC laptop panel detected
radeonfb: Monitor 1 type LCD found
radeonfb: Monitor 2 type no found
radeonfb: panel ID string: Samsung LTN150P1-L02
radeonfb: detected LVDS panel size from BIOS: 1400x1050
radeondb: BIOS provided dividers will be used
radeonfb: Power Management enabled for Mobility chipsets
radeonfb: ATI Radeon Lf DDR SGRAM 64 MB
Machine check exception polling timer started.
IA-32 Microcode Update Driver: v1.14 <[email protected]>
devfs: 2004-01-31 Richard Gooch ([email protected])
devfs: boot_options: 0x1
ACPI: AC Adapter [ADP0] (on-line)
ACPI: Battery Slot [BAT0] (battery present)
ACPI: Power Button (FF) [PWRF]
ACPI: Lid Switch [LID]
ACPI: Processor [CPU0] (supports C1)
isapnp: Scanning for PnP cards...
isapnp: No Plug & Play device found
Console: switching to colour frame buffer device 175x65
Real Time Clock Driver v1.12
Non-volatile memory driver v1.2
Linux agpgart interface v0.100 (c) Dave Jones
agpgart: Detected SiS 646 chipset
agpgart: Maximum main memory to use for agp memory: 437M
agpgart: AGP aperture is 128M @ 0xe0000000
[drm] Initialized radeon 1.11.0 20020828 on minor 0: ATI Technologies Inc Radeon
R250 Lf [Radeon Mobility 9000 M9]
Serial: 8250/16550 driver $Revision: 1.90 $ 8 ports, IRQ sharing disabled
ttyS1 at I/O 0x2f8 (irq = 3) is a 8250
PCI: Enabling device 0000:00:02.6 (0000 -> 0001)
ACPI: PCI interrupt 0000:00:02.6[C] -> GSI 18 (level, low) -> IRQ 18
pnp: the driver 'serial' has been registered
sis900.c: v1.08.07 11/02/2003
ACPI: PCI interrupt 0000:00:04.0[A] -> GSI 19 (level, low) -> IRQ 19
eth0: Realtek RTL8201 PHY transceiver found at address 1.
eth0: Using transceiver found at address 1 as default
eth0: SiS 900 PCI Fast Ethernet at 0xf800, IRQ 19, 00:50:eb:1d:af:9a.
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
SIS5513: IDE controller at PCI slot 0000:00:02.5
ACPI: PCI interrupt 0000:00:02.5[A] -> GSI 16 (level, low) -> IRQ 16
SIS5513: chipset revision 0
SIS5513: not 100% native mode: will probe irqs later
SIS5513: SiS 962/963 MuTIOL IDE UDMA133 controller
ide0: BM-DMA at 0x2000-0x2007, BIOS settings: hda:DMA, hdb:pio
ide1: BM-DMA at 0x2008-0x200f, BIOS settings: hdc:DMA, hdd:pio
Probing IDE interface ide0...
hda: TOSHIBA MK4021GAS, ATA DISK drive
Using anticipatory io scheduler
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
Probing IDE interface ide1...
hdc: QSI CD-RW/DVD-ROM SBW-241, ATAPI CD/DVD-ROM drive
ide1 at 0x170-0x177,0x376 on irq 15
hda: max request size: 128KiB
hda: 78140160 sectors (40007 MB), CHS=65535/16/63, UDMA(100)
hda: cache flushes supported
/dev/ide/host0/bus0/target0/lun0: p1 p2 p3 < p5 p6 p7 p8 >
hdc: ATAPI 24X DVD-ROM CD-R/RW drive, 2048kB Cache, UDMA(33)
Uniform CD-ROM driver Revision: 3.20
mice: PS/2 mouse device common for all mice
serio: i8042 AUX port at 0x60,0x64 irq 12
input: PS/2 Synaptics TouchPad on isa0060/serio1
serio: i8042 KBD port at 0x60,0x64 irq 1
input: AT Translated Set 2 keyboard on isa0060/serio0
NET: Registered protocol family 2
IP: routing cache hash table of 4096 buckets, 32Kbytes
TCP: Hash tables configured (established 32768 bind 65536)
NET: Registered protocol family 1
NET: Registered protocol family 17
ACPI: (supports S0 S3 S4 S5)
ACPI wakeup devices:
PCI0 USB0 USB1 USB2 USB3 LAN PCI1 PCI2 CRD0 AC97
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem) readonly.
kjournald starting. Commit interval 5 seconds
Mounted devfs on /dev
Freeing unused kernel memory: 148k freed
Adding 489972k swap on /dev/hda2. Priority:-1 extents:1
EXT3 FS on hda5, internal journal
kjournald starting. Commit interval 5 seconds
EXT3 FS on hda6, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on hda7, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on hda8, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
eth0: Media Link On 100mbps full-duplex
agpgart: Found an AGP 2.0 compliant device at 0000:00:00.0.
agpgart: Putting AGP V2 device at 0000:00:00.0 into 1x mode
agpgart: Putting AGP V2 device at 0000:01:00.0 into 1x mode
[drm] Loading R200 Microcode
agpgart: Found an AGP 2.0 compliant device at 0000:00:00.0.
agpgart: Putting AGP V2 device at 0000:00:00.0 into 1x mode
agpgart: Putting AGP V2 device at 0000:01:00.0 into 1x mode
[drm] Loading R200 Microcode
APIC error on CPU0: 00(40)
APIC error on CPU0: 40(40)
APIC error on CPU0: 40(40)
Hendrik Fehr writes:
> APIC error on CPU0: 00(40)
> APIC error on CPU0: 40(40)
> APIC error on CPU0: 40(40)
These are "received illegal vector" errors. They indicate
a serious problem, either with the local APIC bus itself,
or with how the ACPI/MP tables cause us to program the local
and I/O APICs.
Do the errors persist if you disable ACPI?
On Gwe, 2004-09-03 at 11:22, Hendrik Fehr wrote:
> (note: i use ide0=ata66 because my machine is a laptop which uses a short 40c
> wire that is equal to an long (i thing 18 inches) 80c cable.)
That should be fine - if it was not you would get CRC errors.
> hda: 78140160 sectors (40007 MB), CHS=65535/16/63, UDMA(100)
> hda: cache flushes supported
> /dev/ide/host0/bus0/target0/lun0: p1 p2 p3 < p5 p6 p7 p8 >
This all looks fine
> hdc: ATAPI 24X DVD-ROM CD-R/RW drive, 2048kB Cache, UDMA(33)
Likewise. So it is booting up with DMA enabled and ought to be fast for
both devices. What does hdparm -t /dev/hda say ?
Mikael Pettersson writes:
> [...]
> These are "received illegal vector" errors. They indicate
> a serious problem, either with the local APIC bus itself,
> or with how the ACPI/MP tables cause us to program the local
> and I/O APICs.
>
> Do the errors persist if you disable ACPI?
>
I just tried the following two things:
Boot option "acpi=off" (made the cusour switching faster on and off). And when
it came to ide setup i get lots of "hda lost interrupt". The system is
unbootable with that option.
With boot option "noapic" there are no more APIC error messages. Should i add
that boot option to my defaults in /etc/lilo.conf?
Alan Cox writes:
> [...]
> Likewise. So it is booting up with DMA enabled and ought to be fast for
> both devices. What does hdparm -t /dev/hda say ?
>
bash-2.05b# hdparm -t /dev/hda
/dev/hda:
Timing buffered disk reads: 72 MB in 3.03 seconds = 23.80 MB/sec
Note: i just added "noapic" to my boot options.
bash-2.05b# cat /proc/cmdline
BOOT_IMAGE=bk8dev26 root=305 psmouse.proto=imps hdc=cdrom ide0=ata66 noapic
I dont know if ~24MB/sec is a good speed, but remember: the cpu is on full
usage
and the BUFF memory usage increases (and is later released again). And as i
wrote to Lionel there seem to be a linear dependency between cpu clock rate and
i/o result. Lionels words to this fact: "There seems to be an unusual amount of
cpu time spent in IRQ processing for your disk drive's accesses but I really
can't put a finger on why."
BTW: those who dont know: i made a nice "bug"-report and put it on:
http://rcswww.urz.tu-dresden.de/~s4248297/gubed/report.html
You will find all the interesting things like dmesg and hdparm -I /dev/hda
output in the "files" section.
Hendrik Fehr writes:
> Mikael Pettersson writes:
> > [...]
> > These are "received illegal vector" errors. They indicate
> > a serious problem, either with the local APIC bus itself,
> > or with how the ACPI/MP tables cause us to program the local
> > and I/O APICs.
> >
> > Do the errors persist if you disable ACPI?
> >
> I just tried the following two things:
> Boot option "acpi=off" (made the cusour switching faster on and off). And when
> it came to ide setup i get lots of "hda lost interrupt". The system is
> unbootable with that option.
>
> With boot option "noapic" there are no more APIC error messages. Should i add
> that boot option to my defaults in /etc/lilo.conf?
Try pci=noacpi (or however that option which prevents ACPI PCI
IRQ routing is spelled) first.
noapic is a workaround for breakage in other subsystems.
You'll lose interrupt vectors, but that may be preferable
to instability.
On Gwe, 2004-09-03 at 14:01, Hendrik Fehr wrote:
>
> With boot option "noapic" there are no more APIC error messages. Should i add
> that boot option to my defaults in /etc/lilo.conf?
For uniprocessor machines you should avoid building with APIC support in
general anyway. A lot of systems simply don't work with APIC
uniprocessor because nobody used to use the APIC in such a
configuration.
On 03.09.2004, at 16:07, Alan Cox wrote:
> For uniprocessor machines you should avoid building with APIC support
> in
> general anyway. A lot of systems simply don't work with APIC
> uniprocessor because nobody used to use the APIC in such a
> configuration.
This statement I don't understand. Wouldn't it be pretty stupid
not to use the APIC of modern systems if available to get all
the benefits, like additional interrupts? At least my Asus A7V600
refuses to (net-)boot at all without a somewhat recent kernel *and*
APIC enabled in BIOS and kernel because the interrupt routing is
completely messed up. I'd rather let all users who have APIC
problems report on the list and wait until someone fixes the
issue instead of having them shut up and use less advanced
techniques instead unless you want to get those "but it works in
Windows" discussions going...
Servus,
Daniel
I think that at least with relatively current motherboards it makes sense to
at least try the APIC support. Speaking of "works in Windows", I think APIC
mode has a greater chance of working these days on UP motherboards because
Microsoft is pushing motherboard/system manufacturers to support IOAPIC in
all new designs:
http://www.microsoft.com/whdc/system/sysperf/IO-APIC.mspx
----- Original Message -----
From: "Daniel Egger" <[email protected]>
Newsgroups: fa.linux.kernel
To: "Alan Cox" <[email protected]>
Cc: "Hendrik Fehr" <[email protected]>; "Linux Kernel Mailing
List" <[email protected]>
Sent: Friday, September 03, 2004 11:14 AM
Subject: Re: PROBLEM: Full CPU-usage on sis5513-chipset disc
input/output-operations
On 03.09.2004, at 16:07, Alan Cox wrote:
> For uniprocessor machines you should avoid building with APIC support
> in
> general anyway. A lot of systems simply don't work with APIC
> uniprocessor because nobody used to use the APIC in such a
> configuration.
This statement I don't understand. Wouldn't it be pretty stupid
not to use the APIC of modern systems if available to get all
the benefits, like additional interrupts? At least my Asus A7V600
refuses to (net-)boot at all without a somewhat recent kernel *and*
APIC enabled in BIOS and kernel because the interrupt routing is
completely messed up. I'd rather let all users who have APIC
problems report on the list and wait until someone fixes the
issue instead of having them shut up and use less advanced
techniques instead unless you want to get those "but it works in
Windows" discussions going...
Servus,
Daniel
Hello again!
SUMMARY:
I reported that i discover full cpu usage when performing disc i/o on an
sis5513 ide dma chipset. Disabling acpi and io-apic in the kernel and
passing pci=biosirq did not solve the problem.
NEWS:
I think i found it!
I did not noticed that the /proc/stat fields changed between 2.4 and 2.6.
(my monitor util mystiriously couted/added "wating for io" to the sys cpu
time.)
That was what drived me crazy.
Yes, yes i did not check CHANGES file, sorry for the disturbing.
question: is "waiting for io" real cpu time, or can i consider this
time as idle? I was unable to figure this out by myself, sorry.
Best regards,
Hendrik Fehr
PS: german for this is something like: "peinlich peinlich".