Message-ID: <4643CCDF.7060608@mbligh.org>
Date: Thu, 10 May 2007 18:54:39 -0700
From: "Martin J. Bligh" <mbligh@mbligh.org>
User-Agent: Thunderbird 1.5.0.10 (X11/20070306)
MIME-Version: 1.0
To: Bas Westerbaan <bas.westerbaan@gmail.com>
Cc: linux-kernel@vger.kernel.org
Subject: Re: Conveying memory pressure to userspace?
References: <6880bed30705101756nb5bd0c5s97935b97d07f846a@mail.gmail.com>
In-Reply-To: <6880bed30705101756nb5bd0c5s97935b97d07f846a@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2732
Lines: 63

Bas Westerbaan wrote:
> Hello,
> 
> Quite a lot of userspace applications cache.  Firefox caches pages;
> mySQL caches queries;  libc' free (like practically all other
> userspace allocators) won't directly return the first byte of memory
> freed, etc.
> 
> These applications are unaware of memory pressure.  While the kernel
> is trying its best to to free memory by for instance dropping,
> possibly more valuable caches, userspace is left blissfully unaware.
> 
> Obviously this isn't a really big problem, given that we've still got
> swap to swap out those rarely used caches, except for when the caches
> aren't _that_ rarely used and of which the backing store (eg.
> precomputed values) might be faster than the disk to swap back the
> pages from.
> 
> A solution would be to either
> 
>  a) let the application make the kernel aware of pages that, when in
> memory pressure, may be dropped.  This would be tricky to implement
> for the userspace: it's hard to avoid an application to race into a
> dropped page.  However, the kernel can directly free a page from
> userspace, which makes it use full when under real pressure.  This in
> contrast to
> b) letting the application register itself with a cache share
> priority.  The application (and other aware applications) would then
> be able to query how fair they are at the moment proportional to their
> cache share priority.  Freeing would still be completely in their own
> hands.
> 
> 
> The only relevant related matter I could find were madvise and mincore.
> 
> With madvise pages can be marked to be unnecessary and these should be
> swapped out earlier.  With mincore one can determine whether pages are
> resident (not cached).  This would make an existing alternative to
> solution a.  However, this doesn't eliminate the writes to the swap
> and polling everytime before accessing a cache isn't really pretty.
> 
> I did consider guessing the memory pressure by looking at
> /proc/meminfo, but I think it isn't that accurate.

The prev_priority field in the zoneinfo stuff is more useful for
memory pressure. I'm playing with making a blocking callback that
can wake someone up when this gets down to a certain priority level
(prio=12 => everything's rosy, prio=0 => we're in deep shit).

> Before hacking something together (and being uncertain about the
> thoroughness with which I searched for existing work, sorry), I would
> like your thoughts on this.
> 
> Please CC me, I'm not in the list.
> 
>  Bas
> 

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/