I gave the O_STREAMING in Andrew's 2.5-mm tree the treatment..
Short summary: It works.
The streaming read test in the following benchmarks is simply a read()
in 64KB byte chunks of an 800MB file.
First test. Show the cache effects are indeed as we intend and the
overhead is negligible. Here, mem=2G.
O_STREAMING? Wall time Cache Delta
Yes 21.827s 0
No 21.734s +800MB
Second test. Same deal, but mem=8M. There was slight swapping, so I
suspect the reduced VM pressure is why the O_STREAMING run is faster.
O_STREAMING? Wall time Cache Delta
Yes 22.303s 0
No 28.812s +1MB
Third and final test. Kernel compile (make -j2) with a couple streaming
reads in the background. Again, mem=2G. This shows that actually
saving the pagecache from the horrid waste is useful.
O_STREAMING Wall time to complete Kernel compile
Yes 5m30.494s
No 4m59.661s
So, uh, Andrew's 2.5 code works ;-)
Someone buy me a dual Xeon,
Robert Love
> I gave the O_STREAMING in Andrew's 2.5-mm tree the treatment..
> O_STREAMING Wall time to complete Kernel compile
> Yes 5m30.494s
> No 4m59.661s
>
> So, uh, Andrew's 2.5 code works ;-)
?? O_STREAMING makes it slower or is it a typo ?
Bye.
On Thu, 2002-10-17 at 03:33, Giuliano Pochini wrote:
>
> > I gave the O_STREAMING in Andrew's 2.5-mm tree the treatment..
> > O_STREAMING Wall time to complete Kernel compile
> > Yes 5m30.494s
> > No 4m59.661s
> >
> > So, uh, Andrew's 2.5 code works ;-)
>
> ?? O_STREAMING makes it slower or is it a typo ?
Yes, typo, those should be switched.
Robert Love
> Third and final test. Kernel compile (make -j2) with a couple streaming
> reads in the background. Again, mem=2G. This shows that actually
> saving the pagecache from the horrid waste is useful.
>
> O_STREAMING Wall time to complete Kernel compile
> Yes 4m59.661s
> No 5m30.494s
>
> So, uh, Andrew's 2.5 code works ;-)
Corrected version.
> Someone buy me a dual Xeon,
>
> Robert Love
It's "Hammer time"...;-)
Greetings,
Dieter
--
Dieter N?tzel
Graduate Student, Computer Science
University of Hamburg
Department of Computer Science
@home: Dieter.Nuetzel at hamburg.de (replace at with @)
On Thu, 2002-10-17 at 03:53, Robert Love wrote:
> I gave the O_STREAMING in Andrew's 2.5-mm tree the treatment..
>
> Short summary: It works.
>
> The streaming read test in the following benchmarks is simply a read()
> in 64KB byte chunks of an 800MB file.
All you now need to do is make it work with an API thats usable by the
other 99% of real world apps, is extensible and sensible ways and
therefore can be used.
Alan Cox wrote:
> On Thu, 2002-10-17 at 03:53, Robert Love wrote:
>
>>I gave the O_STREAMING in Andrew's 2.5-mm tree the treatment..
>>
>>Short summary: It works.
>>
>>The streaming read test in the following benchmarks is simply a read()
>>in 64KB byte chunks of an 800MB file.
>
> All you now need to do is make it work with an API thats usable by the
> other 99% of real world apps, is extensible and sensible ways and
> therefore can be used.
I'm confused. Isn't this just an O_STREAM flag on open/fcntl ?
How could it be simpler? I suppose the VM could do better than
it currently does automatically but there is no harm in the app
giving a hint like this thus allowing stuff to be dropped from
the cache more aggresively?
P?draig.
Alan Cox <[email protected]> writes:
> All you now need to do is make it work with an API thats usable by the
> other 99% of real world apps, is extensible and sensible ways and
> therefore can be used.
An streaming hint as chattr would be quite nice.
-Andi
On Mon, 2002-10-21 at 15:32, Padraig Brady wrote:
> I'm confused. Isn't this just an O_STREAM flag on open/fcntl ?
> How could it be simpler? I suppose the VM could do better than
> it currently does automatically but there is no harm in the app
> giving a hint like this thus allowing stuff to be dropped from
> the cache more aggresively?
Which bit are you streaming, what kind of things are you doing with the
data ?
Take a database table are you - accessing it sequentially with caching,
accessing it randomly without readahead or streaming it and discarding
data. All three are valid answers, for the same data, and potentially on
a per table basis in the same file.
See the fadvise proposal. O_STREAMING is basically a subset of what is
needed