Hans Reiser wrote:
>
> Green and Zam and Umka, on Monday please start work on seriously
> analyzing how the block allocation differs between the new and the old
> kernel, now that you can finally reproduce the benchmark on the old kernel.
I just sent the Orlov allocator patch to Linus. It will double or
triple ext2 performance in that test, so please make sure you compare
against the latest. There's a copy at
http://www.zip.com.au/~akpm/linux/patches/2.5/2.5.45/shpte-stuff/broken-out/orlov-allocator.patch
We can expect similar gains for ext3, when that's done.
(The 2x-3x is on an 8meg filesystem. Larger filesystems should
gain more)
Drew Roselli did traces of overwrite patterns, and the typical time to
overwrite was about 6 minutes, so if you want the write cache to be
effective you want it to last for more than 6 minutes. I encourage you
to read the PhD thesis she wrote and argue with it and me on it, I am
far from dogmatically certain that 10 minutes is the right amount of
time. 60 seconds is the most I would want for my Dell laptop (laptops
are crash prone). 10 minutes for a non-mobile computer with a UPS, or
in an area with a competent electric utility company, is quite
reasonable though. 10 minutes is clearly the right amount of time for,
say, a user space programmer, and probably too risky for a kernel
programmer. Probably kernel programmers are outnumbered 10 to 1 by user
space programmers? ( I don't really know.)
There simply is not enough empirical data for what we argue about,
unfortunately. Drew Roselli's thesis is the only one, and there is a
need for 5 such theses before one can consider the topic reasonably
understandable by the discerning. I worry a lot that her samples are
distorted by site specific usage patterns that might not resemble those
of the usual linux user.
I wish I personally had a better understanding of what the usual linux
user does in the way of IO.....
Hans
Andreas Dilger wrote:
>On Nov 04, 2002 23:30 -0800, reiser wrote:
>
>
>>The appropriate setting of
>>transaction max age depends on the user. The setting we chose is
>>appropriate for software developers doing compiles. It is not clear to
>>me yet what the right setting is. Perhaps 3 minutes is more
>>appropriate. I was probably overly influenced by Drew Roselli's
>>statistics on how long the cyle is between rewrites. Her statistics are
>>probably skewed by having lots of CS students using the machines she got
>>her data from. 5 seconds is too short to perform good layout
>>optimization for subsequent reads.
>>
>>
>
>I think the bdflush defaults are (were?) something like 5 seconds for
>metadata, and 30 seconds for file data. reiser4 should (if it doesn't
>already) use the parameters set by sys_bdflush() to tune the writeout
>intervals.
>
>I would think that either:
>a) A file was completely written in under 30 seconds (e.g. untar or gcc
> or whatever else you are doing), so deferring allocation and writing
> to disk does not help you at all.
>b) A file is continuing to be written for more than 30 seconds that
> has a very large amount of outstanding data which can be committed
> to disk with (probably) the same read optimization quality as any
> larger amount of data.
>c) A file is continuing to be written for more than 30 seconds that
> is growing slowly and no matter how long you defer the write you
> will only get an incremental read layout. Presumably you could do
> something to pre-allocate/reserve a bunch of space at the end of this
> file as it continues to grow.
>
>So, except for the very unusual case of files with lifespans between 30
>seconds and 300 seconds, or files that are written to between those
>intervals, I would guess that you are not gaining much extra benefit by
>deferring the writes another 270 seconds.
>
>
>Cheers, Andreas
>--
>Andreas Dilger \ "If a man ate a pound of pasta and a pound of antipasto,
> \ would they cancel out, leaving him still hungry?"
>http://www-mddsp.enel.ucalgary.ca/People/adilger/ -- Dogbert
>
>
>
>