Message-ID: <4DFE997C.2060805@jp.fujitsu.com>
Date: Mon, 20 Jun 2011 09:51:08 +0900
From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; ja; rv:1.9.2.17) Gecko/20110414 Lightning/1.0b2 Thunderbird/3.1.10
MIME-Version: 1.0
To: david@fromorbit.com
CC: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-mm@kvack.org, xfs@oss.sgi.com
Subject: Re: [PATCH 03/12] vmscan: reduce wind up shrinker->nr when shrinker
 can't do work
References: <1306998067-27659-1-git-send-email-david@fromorbit.com> <1306998067-27659-4-git-send-email-david@fromorbit.com>
In-Reply-To: <1306998067-27659-4-git-send-email-david@fromorbit.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 4146
Lines: 92

(2011/06/02 16:00), Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
> 
> When a shrinker returns -1 to shrink_slab() to indicate it cannot do
> any work given the current memory reclaim requirements, it adds the
> entire total_scan count to shrinker->nr. The idea ehind this is that
> whenteh shrinker is next called and can do work, it will do the work
> of the previously aborted shrinker call as well.
> 
> However, if a filesystem is doing lots of allocation with GFP_NOFS
> set, then we get many, many more aborts from the shrinkers than we
> do successful calls. The result is that shrinker->nr winds up to
> it's maximum permissible value (twice the current cache size) and
> then when the next shrinker call that can do work is issued, it
> has enough scan count built up to free the entire cache twice over.
> 
> This manifests itself in the cache going from full to empty in a
> matter of seconds, even when only a small part of the cache is
> needed to be emptied to free sufficient memory.
> 
> Under metadata intensive workloads on ext4 and XFS, I'm seeing the
> VFS caches increase memory consumption up to 75% of memory (no page
> cache pressure) over a period of 30-60s, and then the shrinker
> empties them down to zero in the space of 2-3s. This cycle repeats
> over and over again, with the shrinker completely trashing the іnode
> and dentry caches every minute or so the workload continues.
> 
> This behaviour was made obvious by the shrink_slab tracepoints added
> earlier in the series, and made worse by the patch that corrected
> the concurrent accounting of shrinker->nr.
> 
> To avoid this problem, stop repeated small increments of the total
> scan value from winding shrinker->nr up to a value that can cause
> the entire cache to be freed. We still need to allow it to wind up,
> so use the delta as the "large scan" threshold check - if the delta
> is more than a quarter of the entire cache size, then it is a large
> scan and allowed to cause lots of windup because we are clearly
> needing to free lots of memory.
> 
> If it isn't a large scan then limit the total scan to half the size
> of the cache so that windup never increases to consume the whole
> cache. Reducing the total scan limit further does not allow enough
> wind-up to maintain the current levels of performance, whilst a
> higher threshold does not prevent the windup from freeing the entire
> cache under sustained workloads.
> 
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
>  mm/vmscan.c |   14 ++++++++++++++
>  1 files changed, 14 insertions(+), 0 deletions(-)
> 
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index dce2767..3688f47 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -277,6 +277,20 @@ unsigned long shrink_slab(struct shrink_control *shrink,
>  		}
>  
>  		/*
> +		 * Avoid excessive windup on fielsystem shrinkers due to large
> +		 * numbers of GFP_NOFS allocations causing the shrinkers to
> +		 * return -1 all the time. This results in a large nr being
> +		 * built up so when a shrink that can do some work comes along
> +		 * it empties the entire cache due to nr >>> max_pass.  This is
> +		 * bad for sustaining a working set in memory.
> +		 *
> +		 * Hence only allow nr to go large when a large delta is
> +		 * calculated.
> +		 */
> +		if (delta < max_pass / 4)
> +			total_scan = min(total_scan, max_pass / 2);
> +
> +		/*
>  		 * Avoid risking looping forever due to too large nr value:
>  		 * never try to free more than twice the estimate number of
>  		 * freeable entries.

I guess "max_pass/4" and "min(total_scan, max_pass / 2)" are your heuristic value. right?
If so, please write your benchmark name and its result into the description. I mean,
currently some mm folks plan to enhance shrinker. So, sharing benchmark may help to avoid
an accidental regression.

I mean, your code itself looks pretty good to me.

thanks.


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/