I origanaly started this "EXT3 scaling to large numbers of drives" thread on both the LSE and the LKM lists.
I forgot to CC the LKML when I posted this yesterday. The LSE is likely the better place for this type of thing so take this as
just an FYI to close out some of the open quiestions that where made durring that thread.
--mgross
---------- Forwarded Message ----------
Subject: [Lse-tech] Block IO write performance (ext2 data) WAS : Ext3 performance bottleneck as the number of spindles gets large
Date: Tue, 25 Jun 2002 17:32:56 -0400
From: mgross <[email protected]>
To: [email protected]
Cc: "'Andrew Morton'" <[email protected]>, "Griffiths, Richard A" <[email protected]>, "'Jens Axboe'" <[email protected]>
I've just finished getting some EXT2 block throughput data as requested by
this list for the "Ext3 performance bottleneck as the number of spindles
gets large" email thread that I started last week.
Recall the system is a dual PCI/66 box with 4 good SCSI cards with up to 6
15KRPM ST318452LC drives per card. (dual 1.2Ghz Pentium 3 with 2 Gig RAM
running kernels high mem support == OFF )
http://www.intel.com/design/servers/scb2/index.htm?iid=ipp_browse+motherbd_s
cb2&
http://www.adaptec.com/worldwide/product/proddetail.html?sess=no&prodkey=ASC
-39160&cat=Products
The benchmark we are using is bonnie++ version 1.02h
ttp://http://www.coker.com.au/bonnie++
Running on 2.4.18 base, 8KB writes to 300MB files I get the following data
when run on the ext 2 file system.
2 adapters (on separate PCI buses)
Drives per card Total system throughput EXT2 EXT3
1 73357 KB/sec 92095 KB/sec
2 115953 KB/sec 110956 KB/sec
3 132176 KB/sec
4 139578 KB/sec
5 139085 KB/sec
6 140033 KB/sec 106883 KB/sec
4 adapters
Drives per card Total system throughput EXT2 EXT3
1 125282 KB/sec 121125 KB/sec
2 146632 KB/sec 117575 KB/sec
3 146622 KB/sec
4 142472 KB/sec
5 142560 KB/sec
6 138835 KB/sec 116570 KB/sec
This performance still isn't very competitive and we see very little evidence
of scaling.
I think that there we should be getting closer to 300,000KB/sec for max
throughput. I'll get some lock meter data tomorrow, as well as run with some
of those patches I've been sent and perhaps even some other kernels ( any
recomendations?).
>From this data I conclude that the BKL that ext3 is abusing using is only
part of the problem.
Comments? Ideas? Recommendations?
Attached is the script I'm using to do the 4 addapter runs. Its not pritty,
but if you have questions regarding the tests it may help clarify things.
--mgross
-------------------------------------------------------