Date: Tue, 18 Mar 2008 09:57:15 -0700
From: Andrew Morton <akpm@linux-foundation.org>
To: Andi Kleen <andi@firstfloor.org>
Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org
Subject: Re: [PATCH prototype] [0/8] Predictive bitmaps for ELF executables
Message-Id: <20080318095715.27120788.akpm@linux-foundation.org>
In-Reply-To: <20080318141828.GD11966@one.firstfloor.org>
References: <20080318209.039112899@firstfloor.org>
	<20080318003620.d84efb95.akpm@linux-foundation.org>
	<20080318141828.GD11966@one.firstfloor.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2529
Lines: 60

On Tue, 18 Mar 2008 15:18:28 +0100 Andi Kleen <andi@firstfloor.org> wrote:

> On Tue, Mar 18, 2008 at 12:36:20AM -0700, Andrew Morton wrote:
> > On Tue, 18 Mar 2008 02:09:34 +0100 (CET) Andi Kleen <andi@firstfloor.org> wrote:
> > 
> > > This patchkit is an experimental optimization I played around with 
> > > some time ago.
> > > 
> > > This is more a prototype still, but I wanted to push it out 
> > > so that other people can play with it.
> > > 
> > > The basic idea is that most programs have the same working set
> > > over multiple runs. So instead of demand paging all the text pages
> > > in the order the program runs save the working set to disk and prefetch
> > > it at program start and then save it at program exit.
> > > 
> > > This allows some optimizations: 
> > > - it can avoid unnecessary disk seeks because the blocks will be fetched in 
> > > sorted offset order instead of program execution order. 
> > > - batch kernel entries (each demand page exception has some
> > > overhead just for entering the kernel). This keeps the caches hot too.
> > > - The prefetch could be in theory done in the background while the program 
> > > runs (although that is not implemented currently)
> > 
> > Should be worthwhile for some things.
> > 
> > > Some details on the implementation:
> > 
> > Can't this all be done in userspace?  Hook into exit() with an LD_PRELOAD,
> 
> In theory yes, but it would have much more overhead.

s/much/slightly/.  The main win will be in optimising disk patterns.  We
can fault in cached pages at, what?  A few GB/sec?

> Also I think prefetching
> algorithms like this really belong in the kernel.

hrmph.

> > > - Executable files have to be writable by the user executing it
> > > currently to get bitmap updates. It would be possible to let the 
> > > kernel bypass this, but I haven't thought too much about the security 
> > > implications of it.
> > > However any user can use the bitmap data written by a user with
> > > write rights.
> > 
> > Those all get fixed with the userspace version?
> 
> No in fact the permission problem is much harder to fix in user space.

What's the permission problem?  executable-but-not-readable files?  Could
be handled by passing your request to a suitable-privileged server process,
I guess.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/