Hello,
I'm considering replacing Reiser3 filesystem with some newer one.
I ran compilebench benchmark. Read performance is lower with ext4.
Is it expected or is it possible to fix it?
Ext4:
read tree total runs 11 avg 20.47 MB/s (user 0.53s sys 0.74s)
read compiled tree total runs 4 avg 32.94 MB/s (user 0.61s sys 1.17s)
Reiser3:
read tree total runs 11 avg 26.33 MB/s (user 0.54s sys 0.81s)
read compiled tree total runs 4 avg 41.82 MB/s (user 0.62s sys 1.36s)
RAID details:
md8 : active raid10 sda7[0] sdd7[3] sdc7[2] sdb7[1]
62925824 blocks 256K chunks 2 far-copies [4/4] [UUUU]
Ext4:
mkfs.ext4 -E stride=64,stripe-width=128 /dev/md8
mount -t ext4 -o noatime,auto_da_alloc,commit=600 /dev/md8 /mnt/md8
Reiser3:
mount -t reiserfs /dev/md8 /mnt/md8
mount -t reiserfs -o noatime,notail /dev/md8 /dev/md8
Ext4 results:
intial create total runs 10 avg 172.76 MB/s (user 0.43s sys 0.60s)
create total runs 14 avg 36.49 MB/s (user 0.42s sys 0.59s)
patch total runs 15 avg 15.16 MB/s (user 0.24s sys 0.49s)
compile total runs 14 avg 64.07 MB/s (user 0.10s sys 0.59s)
clean total runs 10 avg 393.43 MB/s (user 0.02s sys 0.06s)
read tree total runs 11 avg 20.47 MB/s (user 0.53s sys 0.74s)
read compiled tree total runs 4 avg 32.94 MB/s (user 0.61s sys 1.17s)
delete tree total runs 10 avg 2.51 seconds (user 0.24s sys 0.42s)
delete compiled tree total runs 4 avg 2.63 seconds (user 0.28s sys 0.50s)
stat tree total runs 11 avg 1.99 seconds (user 0.23s sys 0.18s)
stat compiled tree total runs 7 avg 2.11 seconds (user 0.27s sys 0.21s)
Reiser3 results:
intial create total runs 10 avg 82.74 MB/s (user 0.45s sys 1.13s)
create total runs 14 avg 28.54 MB/s (user 0.45s sys 1.19s)
patch total runs 15 avg 10.91 MB/s (user 0.24s sys 0.86s)
compile total runs 14 avg 47.49 MB/s (user 0.10s sys 1.27s)
clean total runs 10 avg 270.21 MB/s (user 0.02s sys 0.15s)
read tree total runs 11 avg 26.33 MB/s (user 0.54s sys 0.81s)
read compiled tree total runs 4 avg 41.82 MB/s (user 0.62s sys 1.36s)
delete tree total runs 10 avg 3.38 seconds (user 0.24s sys 0.72s)
delete compiled tree total runs 4 avg 4.14 seconds (user 0.27s sys 0.88s)
stat tree total runs 11 avg 2.09 seconds (user 0.22s sys 0.18s)
stat compiled tree total runs 7 avg 2.27 seconds (user 0.25s sys 0.21s)
creating tar of compiled kernel takes much longer with ext4
time tar c linux-2.6.31 | cat >/dev/null
reiser3:
real 0m11.425s
user 0m0.229s
sys 0m1.586s
ext4:
real 0m23.942s
user 0m0.183s
sys 0m1.440s
partition was remounted before test
>
> RAID details:
>
> md8 : active raid10 sda7[0] sdd7[3] sdc7[2] sdb7[1]
> 62925824 blocks 256K chunks 2 far-copies [4/4] [UUUU]
>
> Ext4:
> mkfs.ext4 -E stride=64,stripe-width=128 /dev/md8
> mount -t ext4 -o noatime,auto_da_alloc,commit=600 /dev/md8 /mnt/md8
>
> Reiser3:
> mkfs.reiserfs /dev/md8
> mount -t reiserfs -o noatime,notail /dev/md8 /mnt/md8
>
On Sat, Sep 19, 2009 at 08:12:31PM +0200, [email protected] wrote:
>
> I'm considering replacing Reiser3 filesystem with some newer one.
> I ran compilebench benchmark. Read performance is lower with ext4.
> Is it expected or is it possible to fix it?
You didn't say which version of the kernel you are using, which could
be important when asking these sorts of questions about potential
performance problems.
However, in this case, I suspect the issue is the nature of how
compilebench is structured. Compilebench does the following which
makes it work particularly well for filesystems like reiserfs and
btrfs, and not so much for ext3 and ext4. Quoting from the
compilebench web page:
compilebench starts by putting these lists of file names into an
order native to the filesystem it is working on. The files are
created in sorted order based on the filename, and then readdir is
used to find the order the filesystem uses for storing the
names. After this initial phase, the filesystem native order is
used for creates, patches and compile. Deleting, reading and
stating the trees are done in readdir order.
The key here is that it reads the tree in readdir order. Normally,
when you compile a kernel, the order in which you read and write files
is controlled by the Makefile; you don't get to read and write the
files in the order which just happens to be the most convenient for
the file system's b-tree hash algorithm.
Now, there are some workloads which compilebench might accurately
model --- for example, tar'ing up a directory. However, despite the
name of the benchmark, it doesn't accurately model a kernel compile.
If you only care about compilebench numbers, you can try creating the
file system with the dir_index feature disabled. This is the feature
that speeds up random access to directories; unfortunately, it means
that when you read files in readdir order, it causes extra random
reads to th einode table. However, if your real-life workload is one
where file reads are always magically in readdir order, dir_index adds
overhead without adding any benefit.
The bottom line is that I'm not terribly worried about trying to
improve ext4's performance on compilebench, since I don't believe it's
a benchmark that models realistic real-life workloads.
Regards,
- Ted
On Sun, Sep 20, 2009 at 3:47 AM, Daniel J Blueman
<[email protected]> wrote:
> On Sep 19, 7:20 pm, [email protected] wrote:
>>
>> Ext4:
>> read tree total runs 11 avg 20.47 MB/s (user 0.53s sys 0.74s)
>> read compiled tree total runs 4 avg 32.94 MB/s (user 0.61s sys 1.17s)
>>
>> Reiser3:
>> read tree total runs 11 avg 26.33 MB/s (user 0.54s sys 0.81s)
>> read compiled tree total runs 4 avg 41.82 MB/s (user 0.62s sys 1.36s)
>
> It would be interesting to also compare against BTRFS if on 2.6.30 or
> newer, if you can.
>
BTRFS 2.6.31
mkfs.btrfs -d raid10 -m raid10 /dev/sda7 /dev/sdb7 /dev/sdc7 /dev/sdd7
mount -t btrfs -o noatime /dev/sda7 /mnt/md8
intial create total runs 10 avg 158.85 MB/s (user 0.45s sys 0.93s)
create total runs 14 avg 32.67 MB/s (user 0.44s sys 0.90s)
patch total runs 15 avg 8.91 MB/s (user 0.22s sys 0.84s)
compile total runs 14 avg 61.02 MB/s (user 0.09s sys 0.50s)
clean total runs 10 avg 245.12 MB/s (user 0.02s sys 0.18s)
read tree total runs 11 avg 14.03 MB/s (user 0.48s sys 0.87s)
read compiled tree total runs 4 avg 29.14 MB/s (user 0.54s sys 1.37s)
delete tree total runs 10 avg 9.77 seconds (user 0.28s sys 1.37s)
delete compiled tree total runs 4 avg 11.91 seconds (user 0.31s sys 1.60s)
stat tree total runs 11 avg 4.36 seconds (user 0.25s sys 0.33s)
stat compiled tree total runs 7 avg 5.29 seconds (user 0.29s sys 0.37s)
On Sun, Sep 20, 2009 at 1:18 AM, Theodore Tso <[email protected]> wrote:
> You didn't say which version of the kernel you are using, which could
> be important when asking these sorts of questions about potential
> performance problems.
>
2.6.31
>
> Now, there are some workloads which compilebench might accurately
> model --- for example, tar'ing up a directory. However, despite the
> name of the benchmark, it doesn't accurately model a kernel compile.
>
You are right. I created simple benchmark which does real compilation of kernel
with following results:
reiser3
16925 msec: tar xf /usr/src/linux.tar
119173 msec: make -j4 all
ext4
14296 msec: tar xf /usr/src/linux.tar
116692 msec: make -j4 all
btrfs
13493 msec: tar xf /usr/src/linux.tar
116015 msec: make -j4 all
MD RAID10-f2 with 256 kB chunk size was used in all cases.
Source code:
http://www.sendspace.com/file/fmikv7
It depends on Qt Core library. There are hard-coded paths.
It can delete important data if it is started without customization !!!