2002-01-23 09:32:28

by Duraid Madina

[permalink] [raw]
Subject: VM: Where do we stand?

I'm sure at least some of you will immediately recognize these words:

>
>Swap allocation is terrible. Linux uses a linear array which it scans
>looking for a free swap block. It does a relatively simple swap
>cluster cache, but eats the full linear scan if that fails which can be
>terribly nasty. The swap clustering algorithm is a piece of crap,
>too -- once swap becomes fragmented, the linux swapper falls on its
face.
>
>It does read-ahead based on the swapblk which wouldn't be bad if it
>clustered writes by object or didn't have a fragmentation problem.
>As it stands, their read clustering is useless. Swap deallocation is
>fast since they are using a simple reference count array.
>
>File read-ahead is half-hazard at best.
>
>The paging queues ( determing the age of the page and whether to
>free or clean it) need to be written... the algorithms being used
>are terrible.
>
> * For the nominal page scan, it is using a one-hand clock algorithm.
> All I can say is: Oh my god! Are they nuts? That was abandoned
> a decade ago. The priority mechanism they've implemented is nearly
> useless.
>
> * To locate pages to swap out, it takes a pass through the task list.
> Ostensibly it locates the task with the largest RSS to then try to
> swap pages out from rather then select pages that are not in use.
> From my read of the code, it also botches this badly.
>
>Linux does not appear to do any page coloring whatsoever, but it would
>not be hard to add it in.
>

Where does Linux stand, three years on? An O(1) scheduler is
nice, but I tell you what'd be even nicer...

coming out for some food, (it's dark out)
Duraid



2002-01-23 09:44:41

by Rik van Riel

[permalink] [raw]
Subject: Re: VM: Where do we stand?

On Wed, 23 Jan 2002, Duraid Madina wrote:

> I'm sure at least some of you will immediately recognize these words:
>
> >Swap allocation is terrible. Linux uses a linear array which it scans
> >looking for a free swap block. It does a relatively simple swap
> >cluster cache, but eats the full linear scan if that fails which can be
> >terribly nasty. The swap clustering algorithm is a piece of crap,
> >too -- once swap becomes fragmented, the linux swapper falls on its
> >face.

Agreed, scanning for a swap block can take too much CPU
on large machines. We've seen this happen ...

> >It does read-ahead based on the swapblk which wouldn't be bad if it
> >clustered writes by object or didn't have a fragmentation problem.
> >As it stands, their read clustering is useless. Swap deallocation is
> >fast since they are using a simple reference count array.

Swap readahead improvements very much welcome.

> >File read-ahead is half-hazard at best.

How so? File read-ahead seems to work pretty well.

> >The paging queues ( determing the age of the page and whether to
> >free or clean it) need to be written... the algorithms being used
> >are terrible.

Fixed in -rmap, http://surriel.com/patches/

> >Linux does not appear to do any page coloring whatsoever, but it would
> >not be hard to add it in.

Not sure how page colouring would interact with the buddy
allocator, though ;)

> Where does Linux stand, three years on? An O(1) scheduler is
> nice, but I tell you what'd be even nicer...

I'm working on some of the above issues for the -rmap VM;
if there's something else which really bugs you, I accept
patches ;)

cheers,

Rik
--
"Linux holds advantages over the single-vendor commercial OS"
-- Microsoft's "Competing with Linux" document

http://www.surriel.com/ http://distro.conectiva.com/

2002-01-23 17:07:49

by Daniel Phillips

[permalink] [raw]
Subject: Re: VM: Where do we stand?

On January 23, 2002 10:32 am, Duraid Madina wrote:
> >The paging queues ( determing the age of the page and whether to
> >free or clean it) need to be written... the algorithms being used
> >are terrible.
> >
> > * For the nominal page scan, it is using a one-hand clock algorithm.
> > All I can say is: Oh my god! Are they nuts? That was abandoned
> > a decade ago.

We don't use a one-hand clock now, we use an lru list coupled with a virtual
scan which sucks slightly less, but only slightly.

> > The priority mechanism they've implemented is nearly
> > useless.

There's a new priority mechanism now ;-)

> > * To locate pages to swap out, it takes a pass through the task list.
> > Ostensibly it locates the task with the largest RSS to then try to
> > swap pages out from rather then select pages that are not in use.
> > From my read of the code, it also botches this badly.

It now tries to select pages that are not in use.

--
Daniel

Subject: Re: VM: Where do we stand?

"Duraid Madina" <[email protected]> writes:

>I'm sure at least some of you will immediately recognize these words:

This paper is _two_ years old. What is this? Linux 2.4pre? 2.2?

(<[email protected]> for the googlelites).

Regards
Henning


--
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen -- Geschaeftsfuehrer
INTERMETA - Gesellschaft fuer Mehrwertdienste mbH [email protected]

Am Schwabachgrund 22 Fon.: 09131 / 50654-0 [email protected]
D-91054 Buckenhof Fax.: 09131 / 50654-20

2002-01-24 12:17:25

by Rik van Riel

[permalink] [raw]
Subject: Re: VM: Where do we stand?

On Thu, 24 Jan 2002, Henning P. Schmiedehausen wrote:
> "Duraid Madina" <[email protected]> writes:
>
> >I'm sure at least some of you will immediately recognize these words:
>
> This paper is _two_ years old. What is this? Linux 2.4pre? 2.2?
>
> (<[email protected]> for the googlelites).

Doesn't mean it's no longer valid, though.

regards,

Rik
--
"Linux holds advantages over the single-vendor commercial OS"
-- Microsoft's "Competing with Linux" document

http://www.surriel.com/ http://distro.conectiva.com/