Hello All!
Has anyone else experienced a drastic drop in read performance on software
RAID-0 with post 2.4.20 kernels? We have a few Athlon XP's here at our
lab with double IDE disks on different channels set up as RAID-0. Some
bonnie++ benchmark results with various kernels, on the same machine
(Athlon XP 2400+, 2 GHz, 1.5 GB RAM, VIA chipset, 2*Maxtor 120 GB
6Y060L0):
write read
2.4.20-ac1: 88,000 135,000 K/sec
2.4.21-pre7: 93,000 75,000
2.4.22-ac4: 94,000 82,000
So the write speed has gone up a bit, but the read speed performance has
plunged. Any ideas on what happened between 2.4.20 and 2.4.21 and what to
do about it? I'm eagerly awaiting suggestions, good read speed is quite
critical for many of our calculations :-) I will of course provide more
details if necessary.
Have a nice day,
Mikael J.
http://www.helsinki.fi/~mpjohans/
On Thu, 20 Nov 2003, Mikael Johansson wrote:
>
> Hello All!
>
> Has anyone else experienced a drastic drop in read performance on software
> RAID-0 with post 2.4.20 kernels? We have a few Athlon XP's here at our
> lab with double IDE disks on different channels set up as RAID-0. Some
> bonnie++ benchmark results with various kernels, on the same machine
> (Athlon XP 2400+, 2 GHz, 1.5 GB RAM, VIA chipset, 2*Maxtor 120 GB
> 6Y060L0):
> write read
> 2.4.20-ac1: 88,000 135,000 K/sec
> 2.4.21-pre7: 93,000 75,000
> 2.4.22-ac4: 94,000 82,000
>
> So the write speed has gone up a bit, but the read speed performance has
> plunged. Any ideas on what happened between 2.4.20 and 2.4.21 and what to
> do about it? I'm eagerly awaiting suggestions, good read speed is quite
> critical for many of our calculations :-) I will of course provide more
> details if necessary.
There have been no significant changes in the RAID driver in 2.4.21, so I
suspect the cause for the slowdowns might the changes to the IO scheduler
(ll_rw_blk.c) or VM changes.
Isolating the -pre which the slowdown starts helps a lot.
Hello Marcelo and All!
On Fri, 21 Nov 2003, Marcelo Tosatti wrote:
> There have been no significant changes in the RAID driver in 2.4.21, so I
> suspect the cause for the slowdowns might the changes to the IO scheduler
> (ll_rw_blk.c) or VM changes.
>
> Isolating the -pre which the slowdown starts helps a lot.
I tested a few kernels to pin point the where tha changes occured. It
turned out that both 2.4.20 and 2.4.21-pre1 have bad read performance. The
2.4.20-ac's show good read speed. I tested it on two machines (different
from yesterday). I also checked the VIA chipset drivers version; that's
not the reason for the differences.
Athlon XP 2400+, 2.09 GHz, 1.5 GB RAM, 2*160 GB Maxtor Maxtor 6Y080L0
VIA write read
2.4.19 none 10,000 9,000 K/sec (chipset not supported)
2.4.20 3.35 73,000 88,000
2.4.20-ac1 3.35-ac 70,000 135,000
2.4.20-ac2 3.35-ac 71,000 140,000
2.4.21-pre1 3.35-ac 71,000 79,000
2.4.21-pre3 3.35-ac 71,000 79,000
Athlon 1.4 GHz, 1.5 GB RAM, 2*80 GB IC35L040AVER07-0 (IBM, I think)
2.4.13-ac8 ? 49,000 44,000 K/sec
2.4.19 3.34 53,000 41,000
2.4.20-ac2 3.35-ac 50,000 69,000
2.4.21-pre1 3.35-ac 53,000 46,000
So there was apparently something very right with the 2.4.20-ac's. It
would be nice to get it back :-)
Have a nice weekend,
Mikael J.
http://www.helsinki.fi/~mpjohans/
>Hello All!
>
>Has anyone else experienced a drastic drop in read performance on
>software
>RAID-0 with post 2.4.20 kernels? We have a few Athlon XP's here at our
>lab with double IDE disks on different channels set up as RAID-0. Some
>bonnie++ benchmark results with various kernels, on the same machine
>(Athlon XP 2400+, 2 GHz, 1.5 GB RAM, VIA chipset, 2*Maxtor 120 GB
>6Y060L0):
>write read
>2.4.20-ac1: 88,000 135,000 K/sec
>2.4.21-pre7: 93,000 75,000
>2.4.22-ac4: 94,000 82,000
Hi,
I can attest a similar drop in read performance on a IA64 box going
from a 2.4.19ish kernel to 2.4.22. In our setup the RAID0 is LVM, not
MD.The RAID is used a a scratch device for a out-of-core finite element
program (NASTRAN).
The setup is some 20 disks on 4 controllers. "iozone" read/reread
Performance went from about 400MB/sec to 260 MB/sec, while write went
up a notch. Unfortunatelly the read performance is more important for
the application in question.
Due to the fact that I have no controll over the use of the system I
cannot make any experiments to find out what killed performance. Sorry
:-(
Martin
=====
------------------------------------------------------
Martin Knoblauch
email: [email protected] or [email protected]
www: http://www.knobisoft.de
On Mon, 2003-11-24 at 02:05, Martin Knoblauch wrote:
> >Hello All!
> >
> >Has anyone else experienced a drastic drop in read performance on
> >software
> >RAID-0 with post 2.4.20 kernels? We have a few Athlon XP's here at our
> >lab with double IDE disks on different channels set up as RAID-0. Some
> >bonnie++ benchmark results with various kernels, on the same machine
> >(Athlon XP 2400+, 2 GHz, 1.5 GB RAM, VIA chipset, 2*Maxtor 120 GB
> >6Y060L0):
> >write read
> >2.4.20-ac1: 88,000 135,000 K/sec
> >2.4.21-pre7: 93,000 75,000
> >2.4.22-ac4: 94,000 82,000
>
> Hi,
>
> I can attest a similar drop in read performance on a IA64 box going
> from a 2.4.19ish kernel to 2.4.22. In our setup the RAID0 is LVM, not
> MD.The RAID is used a a scratch device for a out-of-core finite element
> program (NASTRAN).
>
> The setup is some 20 disks on 4 controllers. "iozone" read/reread
> Performance went from about 400MB/sec to 260 MB/sec, while write went
> up a notch. Unfortunatelly the read performance is more important for
> the application in question.
>
> Due to the fact that I have no controll over the use of the system I
> cannot make any experiments to find out what killed performance. Sorry
> :-(
>
> Martin
>
> =====
> ------------------------------------------------------
> Martin Knoblauch
> email: [email protected] or [email protected]
> www: http://www.knobisoft.de
>
>
And this isn't the read-ahead size change thing?
>>
>
>And this isn't the read-ahead size change thing?
>
Hmm. Sounds interesting/promising. Do you have any pointers to the
change? Which kind of disks does it affect? All? SCSI? IDE?
Thanks
Martin
=====
------------------------------------------------------
Martin Knoblauch
email: [email protected] or [email protected]
www: http://www.knobisoft.de
Hell Joshua and All!
On Mon, 24 Nov 2003, Joshua Schmidlkofer wrote:
> > >Has anyone else experienced a drastic drop in read performance on
> > >software
> > >RAID-0 with post 2.4.20 kernels? We have a few Athlon XP's here at our
> And this isn't the read-ahead size change thing?
When did this change take place? It would be easy to test then. I did some
more tests on the weekend and found that this isn't RAID related; also
normal single disk access is much faster on the 2.4.20-ac's than on other
recent kernels. Just to summarise, the disk speed on one of the Athlon
XP's (2GHz, 1.5 GB RAM, 2*160 GB Maxtor 6Y080L0) according to bonnie++:
RAID-0 VIA write read
2.4.19 none 10,000 9,000
2.4.20 3.35 73,000 88,000
2.4.20-ac1 3.35-ac 70,000 135,000
2.4.20-ac2 3.35-ac 71,000 140,000
2.4.21-pre1 3.35-ac 71,000 79,000
2.4.21-pre3 3.35-ac 71,000 79,000
2.4.23-rc3 3.37 65,000 82,000
single disk
2.4.20-ac2 3.35-ac 55,000 74,000
2.4.23-rc3 3.37 54,000 48,000
So the single disk read performance of 2.4.20-ac2 is as fast as a
double RAID-0 array of other kernels.
Have a nice day,
Mikael J.
http://www.helsinki.fi/~mpjohans/