Date: Mon, 28 Jan 2002 01:37:28 +0100
From: Andrea Arcangeli <andrea@suse.de>
To: rwhron@earthlink.net
Cc: Rik van Riel <riel@conectiva.com.br>, linux-kernel@vger.kernel.org
Subject: Re: 2.4.18pre4aa1
Message-ID: <20020128013728.S25170@athlon.random>
In-Reply-To: <20020124191927.A809@earthlink.net> <Pine.LNX.4.33L.0201242226360.32617-100000@imladris.surriel.com> <20020124222357.C901@earthlink.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.3.12i
In-Reply-To: <20020124222357.C901@earthlink.net>; from rwhron@earthlink.net on Thu, Jan 24, 2002 at 10:23:57PM -0500
Sender: linux-kernel-owner@vger.kernel.org

On Thu, Jan 24, 2002 at 10:23:57PM -0500, rwhron@earthlink.net wrote:
> >   [snip results:  -aa twice as fast as -rmap for dbench,
> >                   -rmap twice as fast as -aa for tiobench]
> 
> Look closely at all the numbers:
> 
> dbench 64 128 192 on ext completed in 4500 seconds on 2.4.18pre4aa1
> dbench 64 128 192 on ext completed in 12471 seconds on 2.4.17rmap12a
> 
> 2.4.18pre4aa1 completed the three dbenches 277% faster.
> 
> For tiobench:
> 
> Tiobench is interesting because it has the CPU% column.  I mentioned 
> sequential reads because it's a bench where 2.4.17rmap12a was faster.  
> Someone else might say 2.4.18pre4aa1 was 271% faster at random reads.  
> Let's analyze CPU efficiency where threads = 1:
> 
>                Num     Seq Read     Rand Read      Seq Write   Rand Write
>                Thr    Rate (CPU%)  Rate (CPU%)    Rate (CPU%)  Rate (CPU%)
>                ---  -------------  -----------  -------------  -----------
> 2.4.17rmap12a    1   22.85  32.2%   1.15  2.2%   13.10  83.5%   0.71  1.6%
> 2.4.18pre4aa1    1   11.23  21.3%   3.12  4.8%   11.92  66.1%   0.66  1.3%

Those weird numbers generated by rmap12a on tiobench shows that the page
replacement algorithm in rmap is not able to detect cache pollution,
that lefts pollution in cache rather than discarding the pollution, so
later that is causing reads not to be served from disk, but to be served
from cache.

Being tiobench an I/O benchmark the above is a completly fake result,
seq read I/O is not going to be faster with rmap. If you change tiobench
to remount the fs where the output files are been generated between the
"random write" and the "seq read" tests, you should get out comparable
numbers.

I don't consider goodness the fact rmap12a lefts old pollution in the
caches, that seems to proof it will do the wrong thing when the most
recently used data is part of the working set (like after you do the
first cvs checkout, you want the second checkout not to hit the disk,
this page replacement in rmap12a should hit the disk the second time
too).

In some ways tiobench has the same problems of dbench. A broken page
replacement algorithm can generate stellar numbers in both of the two
benchmarks.

Furthmore running the 'seq read' after the 'random write' (tiobench does
that), adds even more "random" to the output of the 'seq read' because
the 'random read' and 'random write' tests are not comparable in first
place too: the random seed is setup always different, and also to make a
real 'seq read' test, the 'seq read' should be run after the 'seq
write', not after the 'random write' (even assuming the random seed is
always initialized to the same value).

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/