-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Using the osdl hardware (http://www.osdl.org) with contest
(http://contest.kolivas.net) I've conducted a set of benchmarks with
different filesystems. Note that contest does not claim to be a throughput
benchmark.
All of these use kernel 2.5.59
First a set of contest benchmarks with the io load on a different hard disk
containing each of the four filesystems:
io_other:
Kernel [runs] Time CPU% Loads LCPU% Ratio
2559ext3 3 89 84.3 2 5.5 1.13
2559reiser 3 87 86.2 2 5.7 1.10
2559jfs 3 87 86.2 3 5.7 1.10
2559xfs 3 87 86.2 2 4.5 1.10
I found it interesting that there is virtually no difference in kernel
compilation time with all fs. However jfs consistently wrote more during the
io load than the other fs.
This is a set of benchmarks with the kernel compilation and load all performed
on each of the fs:
ctar_load:
Kernel [runs] Time CPU% Loads LCPU% Ratio
2559ext3 2 96 82.3 2 5.2 1.23
2559jfs 2 103 73.8 0 0.0 1.32
2559reiser 2 100 78.0 0 1.0 1.27
2559xfs 2 97 82.5 2 5.2 1.23
Not sure why the jfs load shows up as 0 cpu% unless it's an accounting issue
within the kernel. Subtle differences between fs times.
xtar_load:
Kernel [runs] Time CPU% Loads LCPU% Ratio
2559ext3 2 97 79.4 2 6.2 1.24
2559jfs 2 136 55.9 0 0.0 1.74
2559reiser 2 104 75.0 0 4.8 1.32
2559xfs 2 105 72.4 1 7.6 1.33
Once again jfs shows up no cpu%. These results show signficant prolongation of
kernel compilation with repeated extracting of tars and jfs, without an
increase in load work done.
io_load:
Kernel [runs] Time CPU% Loads LCPU% Ratio
2559ext3 3 109 68.8 4 10.1 1.40
2559jfs 3 138 54.3 11 13.8 1.77
2559reiser 3 98 76.5 2 9.2 1.24
2559xfs 3 124 60.5 6 8.0 1.57
This shows the largest discrepancy with jfs holding up kernel compilation the
most and doing the most work and reiserfs at the other end. Cpu usage by the
jfs load seems to make sense here.
read_load:
Kernel [runs] Time CPU% Loads LCPU% Ratio
2559ext3 2 98 80.6 7 7.1 1.26
2559jfs 2 97 79.4 5 5.2 1.24
2559reiser 2 101 79.2 6 7.9 1.28
2559xfs 2 98 80.6 6 7.1 1.24
Interestingly reading has the opposite order to writing but with probably
insignificant differences in time. Note the lowish cpu usage by jfs again.
Comments?
Con
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.0 (GNU/Linux)
iD4DBQE+OngeF6dfvkL3i1gRAvKhAJYtmOkN1kLVuBMKI7Ygm317nXrUAJ0Y8UWI
IIdOlvqomgW5eEL4ZQkyGA==
=YDEz
-----END PGP SIGNATURE-----
Be sure to create the tar on the same filesystem that you unpack it onto
--- readdir order affects performance.
A result that we are faster for writes and slower for reads for
workloads without large directories or small files is believable.
compilation is not an effective benchmark anymore, not for Linux
filesystems, they are all just too fast (or is it that the compilers are
too slow?....)
I don't know what ioload does....
--
Hans
On Saturday 01 Feb 2003 12:37 am, Hans Reiser wrote:
> Be sure to create the tar on the same filesystem that you unpack it onto
> --- readdir order affects performance.
The tar creation is on the same filesystem as the unpacking
>
> A result that we are faster for writes and slower for reads for
> workloads without large directories or small files is believable.
>
> compilation is not an effective benchmark anymore, not for Linux
> filesystems, they are all just too fast (or is it that the compilers are
> too slow?....)
>
> I don't know what ioload does....
io load simply repeatedly writes a 256Mb file to the same filesystem as the
compilation is occurring in. io_other writes the 256Mb file to a different
hard disk containing the filesystem in question.
Con
Try running with the -E option for gcc, it might be less CPU intensive,
and thus a better FS benchmark.
What do you think?
--
Hans
On Sat, 1 Feb 2003, Con Kolivas wrote:
>Using the osdl hardware (http://www.osdl.org) with contest
>(http://contest.kolivas.net) I've conducted a set of benchmarks with
>different filesystems. Note that contest does not claim to be a throughput
>benchmark.
>
>All of these use kernel 2.5.59
>
>First a set of contest benchmarks with the io load on a different hard disk
>containing each of the four filesystems:
>
>io_other:
>Kernel [runs] Time CPU% Loads LCPU% Ratio
>2559ext3 3 89 84.3 2 5.5 1.13
>2559reiser 3 87 86.2 2 5.7 1.10
>2559jfs 3 87 86.2 3 5.7 1.10
>2559xfs 3 87 86.2 2 4.5 1.10
>
>I found it interesting that there is virtually no difference in kernel
>compilation time with all fs. However jfs consistently wrote more during the
>io load than the other fs.
>
>
>This is a set of benchmarks with the kernel compilation and load all performed
>on each of the fs:
Compilation is inherently CPU bound, not disk I/O bound, so
compiling the kernel (or anything for that matter) isn't going to
show any difference really because the CPU Mhz and L1/L2 cache
are the bottleneck.
--
Mike A. Harris ftp://people.redhat.com/mharris
OS Systems Engineer - XFree86 maintainer - Red Hat
On Saturday 01 Feb 2003 12:56 am, Hans Reiser wrote:
> Try running with the -E option for gcc, it might be less CPU intensive,
> and thus a better FS benchmark.
>
> What do you think?
To be honest I have no idea. I'm not trying to find the filesystem with the
greatest throughput, but which utilises the least resources while maintaining
throughput. Perhaps we're looking for different things, but I'm open to any
suggestions; but they'll have to wait for my waking hours, and for others'
comments. These benchmarks are run manually for every stage as they are way
out of the scope of the design of contest and therefore time consuming.
On Saturday 01 Feb 2003 1:09 am, Mike A. Harris wrote:
> On Sat, 1 Feb 2003, Con Kolivas wrote:
> >Using the osdl hardware (http://www.osdl.org) with contest
> >(http://contest.kolivas.net) I've conducted a set of benchmarks with
> >different filesystems. Note that contest does not claim to be a throughput
> >benchmark.
> >
> >All of these use kernel 2.5.59
> >
> >First a set of contest benchmarks with the io load on a different hard
> > disk containing each of the four filesystems:
> >
> >io_other:
> >Kernel [runs] Time CPU% Loads LCPU% Ratio
> >2559ext3 3 89 84.3 2 5.5 1.13
> >2559reiser 3 87 86.2 2 5.7 1.10
> >2559jfs 3 87 86.2 3 5.7 1.10
> >2559xfs 3 87 86.2 2 4.5 1.10
> >
> >I found it interesting that there is virtually no difference in kernel
> >compilation time with all fs. However jfs consistently wrote more during
> > the io load than the other fs.
> >
> >
> >This is a set of benchmarks with the kernel compilation and load all
> > performed on each of the fs:
>
> Compilation is inherently CPU bound, not disk I/O bound, so
> compiling the kernel (or anything for that matter) isn't going to
> show any difference really because the CPU Mhz and L1/L2 cache
> are the bottleneck.
When the io load is on another hard disk yes, however the results do show
differences when the load is on the same hard disk - these are two different
tests. There is more to kernel compilation than just cpu usage.
> compiling the kernel (or anything for that matter) isn't going to
> show any difference really because the CPU Mhz and L1/L2 cache
> are the bottleneck.
How about a test comprising of lots of mail being
sent/rejected/bounced/deferred/etc.
Usually SMTPs like postfix store lots of directories and files in there.
And thus create lots of reads/writes. We could measure the efficiency of
that.
I think it is possible to DoS a system by thrashing its i/o by forcing the
smtp to do lots of work. With very poor io efficiency that is.
Propably reiserfs would have better results with such a test, whereas
ext3 could have better results on a differents test. (different
application)
I think the 'best result' fs will vary on the test.
Also i think it is better to testdrive the file systems using real
applications on high load.
Regards,
Maciej Soltysiak
On Fri, Jan 31, 2003 at 04:56:07PM +0300, Hans Reiser wrote:
> Try running with the -E option for gcc, it might be less CPU intensive,
> and thus a better FS benchmark.
>
> What do you think?
It's hardly a realistic real-world benchmark if you start nobbling
bits of it though. Not reading the preprocessed output is sure
to bump the benchmark points on an fs optimised for lots of small
writes.
Dave
--
| Dave Jones. http://www.codemonkey.org.uk
| SuSE Labs
Dave Jones wrote:
>On Fri, Jan 31, 2003 at 04:56:07PM +0300, Hans Reiser wrote:
> > Try running with the -E option for gcc, it might be less CPU intensive,
> > and thus a better FS benchmark.
> >
> > What do you think?
>
>It's hardly a realistic real-world benchmark if you start nobbling
>bits of it though. Not reading the preprocessed output is sure
>to bump the benchmark points on an fs optimised for lots of small
>writes.
>
> Dave
>
>
>
Sigh. The alternative is to strace the compile, write a perl scipt or
something to get just the FS related calls out of it, and then create a
program with just the FS related calls. gcc -E sounds easier to me.;-)
--
Hans
On Fri, Jan 31, 2003 at 07:40:22PM +0300, Hans Reiser wrote:
> >> Try running with the -E option for gcc, it might be less CPU intensive,
> >> and thus a better FS benchmark.
> >> What do you think?
> >It's hardly a realistic real-world benchmark if you start nobbling
> >bits of it though. Not reading the preprocessed output is sure
> >to bump the benchmark points on an fs optimised for lots of small
> >writes.
> Sigh. The alternative is to strace the compile, write a perl scipt or
> something to get just the FS related calls out of it, and then create a
> program with just the FS related calls. gcc -E sounds easier to me.;-)
It still seems like perverting a benchmark to turn it into dbench to me.
Dave
--
| Dave Jones. http://www.codemonkey.org.uk
Dave Jones wrote:
>On Fri, Jan 31, 2003 at 07:40:22PM +0300, Hans Reiser wrote:
>
> > >> Try running with the -E option for gcc, it might be less CPU intensive,
> > >> and thus a better FS benchmark.
> > >> What do you think?
> > >It's hardly a realistic real-world benchmark if you start nobbling
> > >bits of it though. Not reading the preprocessed output is sure
> > >to bump the benchmark points on an fs optimised for lots of small
> > >writes.
> > Sigh. The alternative is to strace the compile, write a perl scipt or
> > something to get just the FS related calls out of it, and then create a
> > program with just the FS related calls. gcc -E sounds easier to me.;-)
>
>It still seems like perverting a benchmark to turn it into dbench to me.
>
> Dave
>
>
>
Easier is not always correct.;-) We need better benchmarks in our
field, sigh.... and correct ones are really a lot of work....
--
Hans
Hans Reiser <[email protected]> wrote:
>
> compilation is not an effective benchmark anymore, not for Linux
> filesystems, they are all just too fast (or is it that the compilers are
> too slow?....)
>
The point of this test is to measure interactions, and fairness.
It answers the question "how much impact does heavy filesystem I/O have upon
other system activity?".
The "other system activity" in this test is a kernel compile. That is a
fairly reasonable metric, because it is sensitive to latencies in servicing
reads and it is sensitive to inappropriate page replacement decisions.
A more appropriate foreground load might be opening a word processor and
composing a short letter to Aunt Nellie, but that's harder to automate. We
expect that reduced kernel compilation time will correlate with lower-latency
letter writing.
Andrew Morton wrote:
>Hans Reiser <[email protected]> wrote:
>
>
>>compilation is not an effective benchmark anymore, not for Linux
>>filesystems, they are all just too fast (or is it that the compilers are
>>too slow?....)
>>
>>
>>
>
>The point of this test is to measure interactions, and fairness.
>
>It answers the question "how much impact does heavy filesystem I/O have upon
>other system activity?".
>
>The "other system activity" in this test is a kernel compile. That is a
>fairly reasonable metric, because it is sensitive to latencies in servicing
>reads and it is sensitive to inappropriate page replacement decisions.
>
>A more appropriate foreground load might be opening a word processor and
>composing a short letter to Aunt Nellie, but that's harder to automate. We
>expect that reduced kernel compilation time will correlate with lower-latency
>letter writing.
>
>
>
>
>
I think the result of the test was that this was not a compelling reason
for users selecting a particular one of the filesystems because they all
did well enough at it. Perhaps because of your code.:)
However, it is rather interesting for all the reasons you mention.
There is indeed a tendency for benchmarks to discount the importance of
latency, and this benchmark does not do that, which is good. It is
annoying to be unable to work while a big tar is running in the
background, but few benchmarks capture that.
We should test reiser4 against this next month, it would be
interesting. (It seems we finally fixed the Reiser4 performance problem
that we hit in August, and now we just need to tweak the CPU usage a bit
and we'll have something performing pretty decently in our next release....)
--
Hans
On Saturday 01 Feb 2003 6:29 am, Hans Reiser wrote:
> Andrew Morton wrote:
> >Hans Reiser <[email protected]> wrote:
> >>compilation is not an effective benchmark anymore, not for Linux
> >>filesystems, they are all just too fast (or is it that the compilers are
> >>too slow?....)
> >
> >The point of this test is to measure interactions, and fairness.
> >
> >It answers the question "how much impact does heavy filesystem I/O have
> > upon other system activity?".
> >
> >The "other system activity" in this test is a kernel compile. That is a
> >fairly reasonable metric, because it is sensitive to latencies in
> > servicing reads and it is sensitive to inappropriate page replacement
> > decisions.
> >
> >A more appropriate foreground load might be opening a word processor and
> >composing a short letter to Aunt Nellie, but that's harder to automate.
> > We expect that reduced kernel compilation time will correlate with
> > lower-latency letter writing.
>
> I think the result of the test was that this was not a compelling reason
> for users selecting a particular one of the filesystems because they all
> did well enough at it. Perhaps because of your code.:)
>
> However, it is rather interesting for all the reasons you mention.
> There is indeed a tendency for benchmarks to discount the importance of
> latency, and this benchmark does not do that, which is good. It is
> annoying to be unable to work while a big tar is running in the
> background, but few benchmarks capture that.
>
> We should test reiser4 against this next month, it would be
> interesting. (It seems we finally fixed the Reiser4 performance problem
> that we hit in August, and now we just need to tweak the CPU usage a bit
> and we'll have something performing pretty decently in our next
> release....)
Actually the most "felt" of these loads is io_load and based on these results:
io_load:
Kernel [runs] Time CPU% Loads LCPU% Ratio
2559ext3 3 109 68.8 4 10.1 1.40
2559jfs 3 138 54.3 11 13.8 1.77
2559reiser 3 98 76.5 2 9.2 1.24
2559xfs 3 124 60.5 6 8.0 1.57
I'd say barring any concern about throughput which this doesnt claim to
measure accurately reiserfs causes the least slowdown of the system ;-)
I do have one more load which may be useful. dbench_load continually runs
dbench in the background. I could throw that at it also.
Ext2 was left out for clarity because it wasn't a journalling fs but it's
results are quite different to the journalled fss.
Con
On Saturday 01 Feb 2003 9:21 am, Con Kolivas wrote:
> Actually the most "felt" of these loads is io_load and based on these
> results: io_load:
> Kernel [runs] Time CPU% Loads LCPU% Ratio
> 2559ext3 3 109 68.8 4 10.1 1.40
> 2559jfs 3 138 54.3 11 13.8 1.77
> 2559reiser 3 98 76.5 2 9.2 1.24
> 2559xfs 3 124 60.5 6 8.0 1.57
>
Here is a set of dbench_load results:
dbench_load:
Kernel [runs] Time CPU% Loads LCPU% Ratio
2559ext3 3 115 65.2 3 24.3 1.47
2559jfs 3 123 61.8 3 19.5 1.58
2559reiser 3 108 69.4 3 13.9 1.37
2559xfs 3 118 63.6 3 22.0 1.49
Note the order correlates with the order of the io_load results.
Once again reiserfs held up kernel compilation the least. Note they all
accomplished the same work in that time though.
Con
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Saturday 01 Feb 2003 12:20 am, Con Kolivas wrote:
> Using the osdl hardware (http://www.osdl.org) with contest
> (http://contest.kolivas.net) I've conducted a set of benchmarks with
> different filesystems. Note that contest does not claim to be a throughput
> benchmark.
Ok to complete the picture here is the set of results including dbench_load
and ext2.
ctar_load:
Kernel [runs] Time CPU% Loads LCPU% Ratio
2559ext2 3 92 85.9 2 4.3 1.18
2559ext3 2 96 82.3 2 5.2 1.23
2559jfs 2 103 73.8 0 0.0 1.32
2559reiser 2 100 78.0 0 1.0 1.27
2559xfs 2 97 82.5 2 5.2 1.23
xtar_load:
Kernel [runs] Time CPU% Loads LCPU% Ratio
2559ext2 3 92 82.6 1 3.3 1.18
2559ext3 2 97 79.4 2 6.2 1.24
2559jfs 2 136 55.9 0 0.0 1.74
2559reiser 2 104 75.0 0 4.8 1.32
2559xfs 2 105 72.4 1 7.6 1.33
io_load:
Kernel [runs] Time CPU% Loads LCPU% Ratio
2559ext2 3 105 71.4 5 6.6 1.35
2559ext3 3 109 68.8 4 10.1 1.40
2559jfs 3 138 54.3 11 13.8 1.77
2559reiser 3 98 76.5 2 9.2 1.24
2559xfs 3 124 60.5 6 8.0 1.57
read_load:
Kernel [runs] Time CPU% Loads LCPU% Ratio
2559ext2 3 96 81.2 6 6.2 1.23
2559ext3 2 98 80.6 7 7.1 1.26
2559jfs 2 97 79.4 5 5.2 1.24
2559reiser 2 101 79.2 6 7.9 1.28
2559xfs 2 98 80.6 6 7.1 1.24
dbench_load:
Kernel [runs] Time CPU% Loads LCPU% Ratio
2559ext2 3 102 72.5 3 17.6 1.31
2559ext3 3 115 65.2 3 24.3 1.47
2559jfs 3 123 61.8 3 19.5 1.58
2559reiser 3 108 69.4 3 13.9 1.37
2559xfs 3 118 63.6 3 22.0 1.49
io_other:
Kernel [runs] Time CPU% Loads LCPU% Ratio
2559ext2 3 90 83.3 3 4.4 1.14
2559ext3 3 89 84.3 2 5.5 1.13
2559reiser 3 87 86.2 2 5.7 1.10
2559jfs 3 87 86.2 3 5.7 1.10
2559xfs 3 87 86.2 2 4.5 1.10
Con
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.0 (GNU/Linux)
iD8DBQE+OxEEF6dfvkL3i1gRAsnrAJ0YZ4FYDLA6iA0R37ccJGdIoqIcJwCeJVbu
vs0QdsTDhfti32WI5c9NJHc=
=qN8n
-----END PGP SIGNATURE-----
I think it would be interesting to add ext2 in as well becouse a LOT of
people are still running ext2 and it would be nice to know how much
performance is being lost to gai the advantages of journaling.
David Lang
On Sat, 1 Feb 2003, Con Kolivas wrote:
> Date: Sat, 1 Feb 2003 09:21:59 +1100
> From: Con Kolivas <[email protected]>
> To: Hans Reiser <[email protected]>, Andrew Morton <[email protected]>
> Cc: [email protected], Alexander Zarochentcev <[email protected]>
> Subject: Re: [BENCHMARK] ext3, reiser, jfs, xfs effect on contest
>
> On Saturday 01 Feb 2003 6:29 am, Hans Reiser wrote:
> > Andrew Morton wrote:
> > >Hans Reiser <[email protected]> wrote:
> > >>compilation is not an effective benchmark anymore, not for Linux
> > >>filesystems, they are all just too fast (or is it that the compilers are
> > >>too slow?....)
> > >
> > >The point of this test is to measure interactions, and fairness.
> > >
> > >It answers the question "how much impact does heavy filesystem I/O have
> > > upon other system activity?".
> > >
> > >The "other system activity" in this test is a kernel compile. That is a
> > >fairly reasonable metric, because it is sensitive to latencies in
> > > servicing reads and it is sensitive to inappropriate page replacement
> > > decisions.
> > >
> > >A more appropriate foreground load might be opening a word processor and
> > >composing a short letter to Aunt Nellie, but that's harder to automate.
> > > We expect that reduced kernel compilation time will correlate with
> > > lower-latency letter writing.
> >
> > I think the result of the test was that this was not a compelling reason
> > for users selecting a particular one of the filesystems because they all
> > did well enough at it. Perhaps because of your code.:)
> >
> > However, it is rather interesting for all the reasons you mention.
> > There is indeed a tendency for benchmarks to discount the importance of
> > latency, and this benchmark does not do that, which is good. It is
> > annoying to be unable to work while a big tar is running in the
> > background, but few benchmarks capture that.
> >
> > We should test reiser4 against this next month, it would be
> > interesting. (It seems we finally fixed the Reiser4 performance problem
> > that we hit in August, and now we just need to tweak the CPU usage a bit
> > and we'll have something performing pretty decently in our next
> > release....)
>
> Actually the most "felt" of these loads is io_load and based on these results:
> io_load:
> Kernel [runs] Time CPU% Loads LCPU% Ratio
> 2559ext3 3 109 68.8 4 10.1 1.40
> 2559jfs 3 138 54.3 11 13.8 1.77
> 2559reiser 3 98 76.5 2 9.2 1.24
> 2559xfs 3 124 60.5 6 8.0 1.57
>
> I'd say barring any concern about throughput which this doesnt claim to
> measure accurately reiserfs causes the least slowdown of the system ;-)
>
> I do have one more load which may be useful. dbench_load continually runs
> dbench in the background. I could throw that at it also.
>
> Ext2 was left out for clarity because it wasn't a journalling fs but it's
> results are quite different to the journalled fss.
>
> Con
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>