2005-01-18 21:40:02

by Fabio Coatti

[permalink] [raw]
Subject: 2.6.11-rc1-mm1 (and others): heavy disk I/O -> poor performance

Under heavy disk I/O, the system becomes very unresponsive (i.e. even a drop
down menu takes several seconds to open).
I've noticed this under 2.6.11-rc1-mm1 and 2.6.10-mm2, but I can try whatever
version is suggested. The way to reproduce this is quite simple: I'm using
gentoo, when emerge --sync rebuilds cache the systems slows like a crawl; the
same behaviour can be seen during a updatedb operation. with top, bdflush is
often stuck in "D" state, as well the I/O bound process (say, emerge or
updatedb).
vmstat under load is the following, and config.gz attached. Of course I can
provide any other needed detail; many thanks for any hint.


cova@kefk ~ $ vmstat 1
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
1 0 628 5252 499696 217712 0 0 19 14 80 60 3 1 95 1
0 1 628 25764 498764 205384 0 0 444 1252 2121 943 7 6 48 39
0 1 628 24412 498812 206628 0 0 596 948 2032 1634 11 5 58 27
0 1 628 23584 498816 207372 0 0 380 2604 2045 1408 6 5 70 18
0 1 628 23360 498816 207576 0 0 56 1528 1982 559 3 2 50 45
0 1 628 22292 498820 208592 0 0 496 980 2092 1120 11 5 51 33
0 1 628 20372 498856 210120 0 0 772 1504 2293 1621 21 9 49 21
0 1 628 18964 498912 211356 0 0 620 1432 2170 1615 13 7 53 28
0 1 628 18340 498920 211892 0 0 292 2924 2137 883 5 4 57 34
0 0 628 17636 498956 212536 0 0 264 712 2018 954 5 3 65 28
0 1 628 17316 498968 212796 0 0 148 1096 1983 607 2 3 51 44
0 1 628 16356 499032 213548 0 0 416 952 2061 1417 7 3 58 32
0 0 628 15708 499060 214132 0 0 256 1912 1993 1409 4 4 53 38
1 0 628 14804 499068 214736 0 0 352 2644 2136 1475 7 4 72 16
0 1 628 14548 499076 215136 0 0 196 1676 2046 526 4 2 49 45
0 1 628 13972 499104 215856 0 0 384 816 2062 1033 9 4 51 37
0 1 628 12916 499172 216808 0 0 504 1056 2135 1311 14 5 51 30
0 1 628 12020 499236 217560 0 0 448 1044 2111 1280 17 5 51 27
0 0 628 11380 499268 218072 0 0 256 2048 2039 838 10 4 62 24
1 0 628 11060 499288 218392 0 0 156 2436 2043 832 7 4 83 5
0 1 628 10612 499328 218692 0 0 124 2180 1899 442 5 2 50 44
1 0 628 10292 499336 218888 0 0 104 368 1883 599 2 2 50 47
0 1 628 8292 499384 220540 0 0 788 1536 2283 1524 18 8 49 27
0 0 628 7652 499388 221080 0 0 276 2044 2039 796 5 4 69 22
0 1 628 6948 499392 221688 0 0 288 2352 2086 783 6 4 52 38
1 0 628 6308 499396 222228 0 0 256 356 2008 797 7 3 50 41
1 0 628 5024 499404 223104 0 0 476 1012 2092 983 13 5 49 32
0 1 628 9848 498300 223936 0 0 420 1096 2075 1243 8 4 53 34
0 1 628 9344 498312 224400 0 0 236 3744 2097 1181 5 4 73 19

To be honest I can't say when this started, I've installed gentoo and seen
emerge --sync load only with 2.6.10-mm2

system: P4 IV 2.8/1Gb ram/i875p MB (abit IC7-g)
ide:
hda: MAXTOR 6L060J3
hdc: TEAC DV-W58G
scsi/Sata:
PLEXTOR CD-ROM PX-40TS 1.01
YAMAHA CRW6416S 1.0c
ATA Maxtor 6Y160M0 YAR5

lspci -v:
kefk ide # lspci -v
0000:00:00.0 Host bridge: Intel Corp. 82875P/E7210 Memory Controller Hub (rev
02)
Subsystem: ABIT Computer Corp.: Unknown device 1014
Flags: bus master, fast devsel, latency 0
Memory at d0000000 (32-bit, prefetchable)
Capabilities: [e4] #09 [2106]
Capabilities: [a0] AGP version 3.0

0000:00:01.0 PCI bridge: Intel Corp. 82875P Processor to AGP Controller (rev
02) (prog-if 00 [Normal decode])
Flags: bus master, 66Mhz, fast devsel, latency 64
Bus: primary=00, secondary=01, subordinate=01, sec-latency=32
Memory behind bridge: f0000000-f1ffffff
Prefetchable memory behind bridge: e8000000-efffffff

0000:00:03.0 PCI bridge: Intel Corp. 82875P/E7210 Processor to PCI to CSA
Bridge (rev 02) (prog-if 00 [Normal decode])
Flags: bus master, 66Mhz, fast devsel, latency 32
Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
I/O behind bridge: 00009000-00009fff
Memory behind bridge: f2000000-f20fffff
Expansion ROM at 00009000 [disabled] [size=4K]

0000:00:1d.0 USB Controller: Intel Corp. 82801EB/ER (ICH5/ICH5R) USB UHCI
Controller #1 (rev 02) (prog-if 00 [UHCI])
Subsystem: ABIT Computer Corp.: Unknown device 1014
Flags: bus master, medium devsel, latency 0, IRQ 193
I/O ports at bc00 [size=32]

0000:00:1d.1 USB Controller: Intel Corp. 82801EB/ER (ICH5/ICH5R) USB UHCI
Controller #2 (rev 02) (prog-if 00 [UHCI])
Subsystem: ABIT Computer Corp.: Unknown device 1014
Flags: bus master, medium devsel, latency 0, IRQ 201
I/O ports at b000 [size=32]

0000:00:1d.2 USB Controller: Intel Corp. 82801EB/ER (ICH5/ICH5R) USB UHCI #3
(rev 02) (prog-if 00 [UHCI])
Subsystem: ABIT Computer Corp.: Unknown device 1014
Flags: bus master, medium devsel, latency 0, IRQ 169
I/O ports at b400 [size=32]

0000:00:1d.3 USB Controller: Intel Corp. 82801EB/ER (ICH5/ICH5R) USB UHCI
Controller #4 (rev 02) (prog-if 00 [UHCI])
Subsystem: ABIT Computer Corp.: Unknown device 1014
Flags: bus master, medium devsel, latency 0, IRQ 193
I/O ports at b800 [size=32]

0000:00:1d.7 USB Controller: Intel Corp. 82801EB/ER (ICH5/ICH5R) USB2 EHCI
Controller (rev 02) (prog-if 20 [EHCI])
Subsystem: ABIT Computer Corp.: Unknown device 1014
Flags: bus master, medium devsel, latency 0, IRQ 185
Memory at f2200000 (32-bit, non-prefetchable)
Capabilities: [50] Power Management version 2
Capabilities: [58] #0a [20a0]

0000:00:1e.0 PCI bridge: Intel Corp. 82801 PCI Bridge (rev c2) (prog-if 00
[Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=00, secondary=03, subordinate=03, sec-latency=32
I/O behind bridge: 0000a000-0000afff
Memory behind bridge: f2100000-f21fffff

0000:00:1f.0 ISA bridge: Intel Corp. 82801EB/ER (ICH5/ICH5R) LPC Interface
Bridge (rev 02)
Flags: bus master, medium devsel, latency 0

0000:00:1f.1 IDE interface: Intel Corp. 82801EB/ER (ICH5/ICH5R) IDE Controller
(rev 02) (prog-if 8a [Master SecP PriP])
Subsystem: ABIT Computer Corp.: Unknown device 1014
Flags: bus master, medium devsel, latency 0, IRQ 169
I/O ports at <unassigned>
I/O ports at <unassigned>
I/O ports at <unassigned>
I/O ports at <unassigned>
I/O ports at f000 [size=16]
Memory at 40000000 (32-bit, non-prefetchable) [size=1K]

0000:00:1f.2 IDE interface: Intel Corp. 82801EB (ICH5) SATA Controller (rev
02) (prog-if 8f [Master SecP SecO PriP PriO])
Subsystem: ABIT Computer Corp.: Unknown device 1014
Flags: bus master, 66Mhz, medium devsel, latency 0, IRQ 169
I/O ports at c000
I/O ports at c400 [size=4]
I/O ports at c800 [size=8]
I/O ports at cc00 [size=4]
I/O ports at d000 [size=16]

0000:00:1f.3 SMBus: Intel Corp. 82801EB/ER (ICH5/ICH5R) SMBus Controller (rev
02)
Subsystem: ABIT Computer Corp.: Unknown device 1014
Flags: medium devsel, IRQ 9
I/O ports at 0500 [size=32]

0000:01:00.0 VGA compatible controller: nVidia Corporation NV34 [GeForce FX
5200] (rev a1) (prog-if 00 [VGA])
Subsystem: Unknown device 1682:1280
Flags: bus master, 66Mhz, medium devsel, latency 248, IRQ 193
Memory at f0000000 (32-bit, non-prefetchable)
Memory at e8000000 (32-bit, prefetchable) [size=128M]
Capabilities: [60] Power Management version 2
Capabilities: [44] AGP version 3.0

0000:02:01.0 Ethernet controller: Intel Corp. 82547EI Gigabit Ethernet
Controller (LOM)
Subsystem: ABIT Computer Corp.: Unknown device 1014
Flags: bus master, 66Mhz, medium devsel, latency 0, IRQ 169
Memory at f2000000 (32-bit, non-prefetchable)
I/O ports at 9000 [size=32]
Capabilities: [dc] Power Management version 2

0000:03:04.0 Multimedia audio controller: Creative Labs SB Audigy (rev 04)
Subsystem: Creative Labs SB Audigy 2 ZS (SB0350)
Flags: bus master, medium devsel, latency 32, IRQ 209
I/O ports at a000
Capabilities: [dc] Power Management version 2

0000:03:04.1 Input device controller: Creative Labs SB Audigy MIDI/Game port
(rev 04)
Subsystem: Creative Labs SB Audigy MIDI/Game Port
Flags: bus master, medium devsel, latency 32
I/O ports at a400
Capabilities: [dc] Power Management version 2

0000:03:04.2 FireWire (IEEE 1394): Creative Labs SB Audigy FireWire Port (rev
04) (prog-if 10 [OHCI])
Subsystem: Creative Labs SB Audigy FireWire Port
Flags: bus master, medium devsel, latency 32, IRQ 5
Memory at f2104000 (32-bit, non-prefetchable)
Memory at f2100000 (32-bit, non-prefetchable) [size=16K]
Capabilities: [44] Power Management version 2

0000:03:06.0 SCSI storage controller: Adaptec AHA-7850 (rev 03)
Subsystem: Adaptec AHA-2904/Integrated AIC-7850
Flags: bus master, medium devsel, latency 32, IRQ 177
I/O ports at a800 [disabled]
Memory at f2105000 (32-bit, non-prefetchable) [size=4K]
Capabilities: [dc] Power Management version 1



--
Fabio Coatti http://members.ferrara.linux.it/cova
Ferrara Linux Users Group http://ferrara.linux.it
GnuPG fp:9765 A5B6 6843 17BC A646 BE8C FA56 373A 5374 C703
Old SysOps never die... they simply forget their password.


Attachments:
(No filename) (9.73 kB)
config.gz (8.26 kB)
Download all attachments

2005-01-19 12:42:27

by bert hubert

[permalink] [raw]
Subject: Re: 2.6.11-rc1-mm1 (and others): heavy disk I/O -> poor performance

On Tue, Jan 18, 2005 at 10:39:35PM +0100, Fabio Coatti wrote:
> vmstat under load is the following, and config.gz attached. Of course I can
> provide any other needed detail; many thanks for any hint.

Looks mightily like DMA is not on, even though you compiled the PIIX driver
in, which lists
> 0000:00:1f.1 IDE interface: Intel Corp. 82801EB/ER (ICH5/ICH5R) IDE Controller

Can you show the output of hdparm /dev/hda ? Can you show dmesg?


--
http://www.PowerDNS.com Open source, database driven DNS Software
http://lartc.org Linux Advanced Routing & Traffic Control HOWTO

2005-01-19 23:14:35

by Fabio Coatti

[permalink] [raw]
Subject: Re: 2.6.11-rc1-mm1 (and others): heavy disk I/O -> poor performance

Alle 13:42, mercoled? 19 gennaio 2005, bert hubert ha scritto:
> On Tue, Jan 18, 2005 at 10:39:35PM +0100, Fabio Coatti wrote:
> > vmstat under load is the following, and config.gz attached. Of course I
> > can provide any other needed detail; many thanks for any hint.
>
> Looks mightily like DMA is not on, even though you compiled the PIIX driver
> in, which lists
>
> > 0000:00:1f.1 IDE interface: Intel Corp. 82801EB/ER (ICH5/ICH5R) IDE
> > Controller
>
> Can you show the output of hdparm /dev/hda ? Can you show dmesg?

Sure, here is it:

/dev/hda:
multcount = 16 (on)
IO_support = 0 (default 16-bit)
unmaskirq = 0 (off)
using_dma = 1 (on)
keepsettings = 0 (off)
readonly = 0 (off)
readahead = 256 (on)
geometry = 65535/16/63, sectors = 60040544256, start = 0

I've cut down the ide relevant part of dmesg, please let me know if more
details are needed
an 19 21:43:53 kefk Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
Jan 19 21:43:53 kefk ide: Assuming 33MHz system bus speed for PIO modes;
override with idebus=xx
Jan 19 21:43:53 kefk ICH5: IDE controller at PCI slot 0000:00:1f.1
Jan 19 21:43:53 kefk ACPI: PCI interrupt 0000:00:1f.1[A] -> GSI 18 (level,
low) -> IRQ 169
Jan 19 21:43:53 kefk ICH5: chipset revision 2
Jan 19 21:43:53 kefk ICH5: not 100% native mode: will probe irqs later
Jan 19 21:43:53 kefk ide0: BM-DMA at 0xf000-0xf007, BIOS settings: hda:DMA,
hdb:pio
Jan 19 21:43:53 kefk ide1: BM-DMA at 0xf008-0xf00f, BIOS settings: hdc:pio,
hdd:pio
Jan 19 21:43:53 kefk Probing IDE interface ide0...
Jan 19 21:43:53 kefk hda: MAXTOR 6L060J3, ATA DISK drive
Jan 19 21:43:53 kefk ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
Jan 19 21:43:53 kefk Probing IDE interface ide1...
Jan 19 21:43:53 kefk hdc: TEAC DV-W58G, ATAPI CD/DVD-ROM drive
Jan 19 21:43:53 kefk ide1 at 0x170-0x177,0x376 on irq 15
Jan 19 21:43:53 kefk Probing IDE interface ide2...
Jan 19 21:43:53 kefk ide2: Wait for ready failed before probe !
Jan 19 21:43:53 kefk Probing IDE interface ide3...
Jan 19 21:43:53 kefk ide3: Wait for ready failed before probe !
Jan 19 21:43:53 kefk Probing IDE interface ide4...
Jan 19 21:43:53 kefk ide4: Wait for ready failed before probe !
Jan 19 21:43:53 kefk Probing IDE interface ide5...
Jan 19 21:43:53 kefk ide5: Wait for ready failed before probe !
Jan 19 21:43:53 kefk hda: max request size: 128KiB
Jan 19 21:43:53 kefk hda: 117266688 sectors (60040 MB) w/1819KiB Cache,
CHS=65535/16/63, UDMA(100)
Jan 19 21:43:53 kefk hda: cache flushes supported
Jan 19 21:43:53 kefk hda: hda1 hda2 < hda5 hda6 hda7 > hda3 hda4
Jan 19 21:43:53 kefk PCI: 0000:03:06.0 has unsupported PM cap regs version (1)
Jan 19 21:43:53 kefk ACPI: PCI interrupt 0000:03:06.0[A] -> GSI 22 (level,
low) -> IRQ 177
Jan 19 21:43:53 kefk PCI: 0000:03:06.0 has unsupported PM cap regs version (1)
Jan 19 21:43:53 kefk ahc_pci:3:6:0: Host Adapter Bios disabled. Using default
SCSI device parameters
Jan 19 21:43:53 kefk scsi0 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev
6.2.36
Jan 19 21:43:53 kefk <Adaptec 2902/04/10/15/20C/30C SCSI adapter>
Jan 19 21:43:53 kefk aic7850: Single Channel A, SCSI Id=7, 3/253 SCBs
Jan 19 21:43:53 kefk
Jan 19 21:43:53 kefk Vendor: Nikon Model: COOLSCANIII Rev: 1.31
Jan 19 21:43:53 kefk Type: Scanner ANSI SCSI
revision: 02
Jan 19 21:43:53 kefk (scsi0:A:3): 10.000MB/s transfers (10.000MHz, offset 15)
Jan 19 21:43:53 kefk Vendor: PLEXTOR Model: CD-ROM PX-40TS Rev: 1.01
Jan 19 21:43:53 kefk Type: CD-ROM ANSI SCSI
revision: 02
Jan 19 21:43:53 kefk (scsi0:A:5): 10.000MB/s transfers (10.000MHz, offset 15)
Jan 19 21:43:53 kefk Vendor: YAMAHA Model: CRW6416S Rev: 1.0c
Jan 19 21:43:53 kefk Type: CD-ROM ANSI SCSI
revision: 02
Jan 19 21:43:53 kefk libata version 1.10 loaded.
Jan 19 21:43:53 kefk ata_piix version 1.03
Jan 19 21:43:53 kefk ACPI: PCI interrupt 0000:00:1f.2[A] -> GSI 18 (level,
low) -> IRQ 169
Jan 19 21:43:53 kefk PCI: Setting latency timer of device 0000:00:1f.2 to 64
Jan 19 21:43:53 kefk ata1: SATA max UDMA/133 cmd 0xC000 ctl 0xC402 bmdma
0xD000 irq 169
Jan 19 21:43:53 kefk ata2: SATA max UDMA/133 cmd 0xC800 ctl 0xCC02 bmdma
0xD008 irq 169
Jan 19 21:43:53 kefk ata1: dev 0 cfg 49:2f00 82:7c6b 83:7f09 84:4003 85:7c69
86:3e01 87:4003 88:207f
Jan 19 21:43:53 kefk ata1: dev 0 ATA, max UDMA/133, 320173056 sectors: lba48
Jan 19 21:43:53 kefk ata1: dev 0 configured for UDMA/133
Jan 19 21:43:53 kefk scsi1 : ata_piix
Jan 19 21:43:53 kefk ata2: SATA port has no device.
Jan 19 21:43:53 kefk scsi2 : ata_piix
Jan 19 21:43:53 kefk Vendor: ATA Model: Maxtor 6Y160M0 Rev: YAR5
Jan 19 21:43:53 kefk Type: Direct-Access ANSI SCSI
revision: 05
Jan 19 21:43:53 kefk SCSI device sda: 320173056 512-byte hdwr sectors (163929
MB)
Jan 19 21:43:53 kefk SCSI device sda: drive cache: write back
Jan 19 21:43:53 kefk SCSI device sda: 320173056 512-byte hdwr sectors (163929
MB)
Jan 19 21:43:53 kefk SCSI device sda: drive cache: write back
Jan 19 21:43:53 kefk sda: sda1 sda2 sda3 sda4 < sda5 sda6 >
Jan 19 21:43:53 kefk Attached scsi disk sda at scsi1, channel 0, id 0, lun 0
Jan 19 21:43:53 kefk sr0: scsi-1 drive
Jan 19 21:43:53 kefk Uniform CD-ROM driver Revision: 3.20
Jan 19 21:43:53 kefk Attached scsi CD-ROM sr0 at scsi0, channel 0, id 3, lun 0
Jan 19 21:43:53 kefk sr1: scsi3-mmc drive: 16x/16x writer cd/rw xa/form2 cdda
tray
Jan 19 21:43:53 kefk Attached scsi CD-ROM sr1 at scsi0, channel 0, id 5, lun 0
Jan 19 21:43:53 kefk Attached scsi generic sg0 at scsi0, channel 0, id 2, lun
0, type 6
Jan 19 21:43:53 kefk Attached scsi generic sg1 at scsi0, channel 0, id 3, lun
0, type 5
Jan 19 21:43:53 kefk Attached scsi generic sg2 at scsi0, channel 0, id 5, lun
0, type 5
Jan 19 21:43:53 kefk Attached scsi generic sg3 at scsi1, channel 0, id 0, lun
0, type 0



--
Fabio Coatti http://members.ferrara.linux.it/cova
Ferrara Linux Users Group http://ferrara.linux.it
GnuPG fp:9765 A5B6 6843 17BC A646 BE8C FA56 373A 5374 C703
Old SysOps never die... they simply forget their password.