Latest version of swap prefetching also available in incremental form and for
download here:
http://ck.kolivas.org/patches/swap-prefetch/
-One unlocking bug was fixed
-Pages are prefetched inactive now so they're the first thing that is swapped
back out again under pressure
-Numerous tweaks and cleanups including:
-Acted on suggestion by John Corbet (thanks!)
Cheers,
Con
Con Kolivas <[email protected]> wrote:
>
> Once pages have been added to the swapped list, a timer is started, testing
> for conditions suitable to prefetch swap pages every 5 seconds. Suitable
> conditions are defined as lack of swapping out or in any pages, and no
> watermark tests failing. Significant amounts of dirtied ram also prevent
> prefetching. It then checks that we have spare ram looking for at
> least 3* pages_high free per zone and if it succeeds that will prefetch
> pages from swap.
Did you consider poking around in gendisk.disk_stats to determine whether
the swap disk(s) are idleish?
On Fri, 30 Sep 2005 07:54 am, Andrew Morton wrote:
> Con Kolivas <[email protected]> wrote:
> > Once pages have been added to the swapped list, a timer is started,
> > testing for conditions suitable to prefetch swap pages every 5 seconds.
> > Suitable conditions are defined as lack of swapping out or in any pages,
> > and no watermark tests failing. Significant amounts of dirtied ram also
> > prevent prefetching. It then checks that we have spare ram looking for at
> > least 3* pages_high free per zone and if it succeeds that will prefetch
> > pages from swap.
>
> Did you consider poking around in gendisk.disk_stats to determine whether
> the swap disk(s) are idleish?
I didn't know where to look for that info. Thanks! I'm open to *any*
suggestions and I'll look into it as I can't take this code much further
without outside help.
Cheers,
Con
Con Kolivas <[email protected]> wrote:
>
> On Fri, 30 Sep 2005 07:54 am, Andrew Morton wrote:
> > Con Kolivas <[email protected]> wrote:
> > > Once pages have been added to the swapped list, a timer is started,
> > > testing for conditions suitable to prefetch swap pages every 5 seconds.
> > > Suitable conditions are defined as lack of swapping out or in any pages,
> > > and no watermark tests failing. Significant amounts of dirtied ram also
> > > prevent prefetching. It then checks that we have spare ram looking for at
> > > least 3* pages_high free per zone and if it succeeds that will prefetch
> > > pages from swap.
> >
> > Did you consider poking around in gendisk.disk_stats to determine whether
> > the swap disk(s) are idleish?
>
> I didn't know where to look for that info. Thanks! I'm open to *any*
> suggestions and I'll look into it as I can't take this code much further
> without outside help.
>
Might also be able to utilise CFQ's I/O priorities. That should be more
efective than a heuristic based on disk_stats, however you'd probably need
to work out whether the swapdev actually supports IO priorities and I'm not
sure how one would query that (cleanly).
On Fri, 30 Sep 2005 09:24 am, Andrew Morton wrote:
> Con Kolivas <[email protected]> wrote:
> > On Fri, 30 Sep 2005 07:54 am, Andrew Morton wrote:
> > > Con Kolivas <[email protected]> wrote:
> > > > Once pages have been added to the swapped list, a timer is started,
> > > > testing for conditions suitable to prefetch swap pages every 5
> > > > seconds. Suitable conditions are defined as lack of swapping out or
> > > > in any pages, and no watermark tests failing. Significant amounts of
> > > > dirtied ram also prevent prefetching. It then checks that we have
> > > > spare ram looking for at least 3* pages_high free per zone and if it
> > > > succeeds that will prefetch pages from swap.
> > >
> > > Did you consider poking around in gendisk.disk_stats to determine
> > > whether the swap disk(s) are idleish?
> >
> > I didn't know where to look for that info. Thanks! I'm open to *any*
> > suggestions and I'll look into it as I can't take this code much further
> > without outside help.
>
> Might also be able to utilise CFQ's I/O priorities. That should be more
> efective than a heuristic based on disk_stats, however you'd probably need
> to work out whether the swapdev actually supports IO priorities and I'm not
> sure how one would query that (cleanly).
Good idea. It should be easy to simply add the ioprio value to the kprefetchd
thread and it either supports it or it doesn't depending on what iosched is
being used.
Further to your original suggestion, prefetching is already delayed when any
non-prefetch related swap in or out is occurring already so checking
diskstats is probably not going to add to that for swap activity. However I
assume it (diskstats) could also be used for all disk i/o as well? Currently
a lot of dirty data can tell me that lots of writing is happening but I have
no way of checking that lots of reads are occurring.
Cheers,
Con
Con Kolivas <[email protected]> wrote:
>
> On Fri, 30 Sep 2005 09:24 am, Andrew Morton wrote:
> > Con Kolivas <[email protected]> wrote:
> > > On Fri, 30 Sep 2005 07:54 am, Andrew Morton wrote:
> > > > Con Kolivas <[email protected]> wrote:
> > > > > Once pages have been added to the swapped list, a timer is started,
> > > > > testing for conditions suitable to prefetch swap pages every 5
> > > > > seconds. Suitable conditions are defined as lack of swapping out or
> > > > > in any pages, and no watermark tests failing. Significant amounts of
> > > > > dirtied ram also prevent prefetching. It then checks that we have
> > > > > spare ram looking for at least 3* pages_high free per zone and if it
> > > > > succeeds that will prefetch pages from swap.
> > > >
> > > > Did you consider poking around in gendisk.disk_stats to determine
> > > > whether the swap disk(s) are idleish?
> > >
> > > I didn't know where to look for that info. Thanks! I'm open to *any*
> > > suggestions and I'll look into it as I can't take this code much further
> > > without outside help.
> >
> > Might also be able to utilise CFQ's I/O priorities. That should be more
> > efective than a heuristic based on disk_stats, however you'd probably need
> > to work out whether the swapdev actually supports IO priorities and I'm not
> > sure how one would query that (cleanly).
>
> Good idea. It should be easy to simply add the ioprio value to the kprefetchd
> thread and it either supports it or it doesn't depending on what iosched is
> being used.
Right. But if the queue doesn't support io priorities then one might want
to fall back to monitoring recent queue activity.
> Further to your original suggestion, prefetching is already delayed when any
> non-prefetch related swap in or out is occurring already so checking
> diskstats is probably not going to add to that for swap activity. However I
> assume it (diskstats) could also be used for all disk i/o as well?
Precisely. Few people would set an entire disk aside for swap.
> Currently
> a lot of dirty data can tell me that lots of writing is happening but I have
> no way of checking that lots of reads are occurring.
diskstats will tell that. Also one could peek at the current disk queue
levels (request_list.count[]).