Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751828AbZGBHlX (ORCPT ); Thu, 2 Jul 2009 03:41:23 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1751359AbZGBHlP (ORCPT ); Thu, 2 Jul 2009 03:41:15 -0400 Received: from mail-px0-f190.google.com ([209.85.216.190]:55917 "EHLO mail-px0-f190.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751312AbZGBHlO (ORCPT ); Thu, 2 Jul 2009 03:41:14 -0400 DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:in-reply-to:references:x-mailer :mime-version:content-type:content-transfer-encoding; b=tNXHcFfk81VzCDd1oLYH7lzdQxVwDAGRJI7xUkcUhDGmXaD1CI/ecLCtVi7wZMu+dt G6wUNwoNgOvb37gdLaRAQMZYm1h+X9ruGaW58fA+UWukLPz3zHhqlp90DS/jWClux2ho X2mbbSVjYvzr7IQJxQfndX9bjYXwpVbVqdxvw= Date: Thu, 2 Jul 2009 16:41:06 +0900 From: Minchan Kim To: David Howells Cc: Minchan Kim , Mel Gorman , Wu Fengguang , KOSAKI Motohiro , Johannes Weiner , "riel\@redhat.com" , Andrew Morton , LKML , Christoph Lameter , "peterz\@infradead.org" , "tytso\@mit.edu" , "linux-mm\@kvack.org" , "elladan\@eskimo.com" , "npiggin\@suse.de" , "Barnes\, Jesse" Subject: Re: Found the commit that causes the OOMs Message-Id: <20090702164106.76db077b.minchan.kim@barrios-desktop> In-Reply-To: <24767.1246391867@redhat.com> References: <20090630130741.c191d042.minchan.kim@barrios-desktop> <28c262360906280630n557bb182n5079e33d21ea4a83@mail.gmail.com> <28c262360906280636l93130ffk14086314e2a6dcb7@mail.gmail.com> <20090628142239.GA20986@localhost> <2f11576a0906280801w417d1b9fpe10585b7a641d41b@mail.gmail.com> <20090628151026.GB25076@localhost> <20090629091741.ab815ae7.minchan.kim@barrios-desktop> <17678.1246270219@redhat.com> <20090629125549.GA22932@localhost> <29432.1246285300@redhat.com> <28c262360906290800v37f91d7av3642b1ad8b5f0477@mail.gmail.com> <20090629160725.GF5065@csn.ul.ie> <24767.1246391867@redhat.com> X-Mailer: Sylpheed 2.6.0 (GTK+ 2.16.1; i486-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2655 Lines: 83 On Tue, 30 Jun 2009 20:57:47 +0100 David Howells wrote: > Minchan Kim wrote: > > > David. Doesn't it happen OOM if you revert my patch, still? > > It does happen, and indeed happens in v2.6.30, but requires two adjacent runs > of msgctl11 to trigger, rather than usually triggering on the first run. If > you interpolate the rest of LTP between the iterations, it doesn't seem to > happen at all on v2.6.30. My guess is that with the rest of LTP interpolated, > there's either enough time for some cleanup or something triggers a cleanup > (the swapfile tests perhaps?). > > > Befor I go to the trip, I made debugging patch in a hurry. Mel and I > > suspect to put the wrong page in lru list. > > > > This patch's goal is that print page's detail on active anon lru when it > > happen OOM. Maybe you could expand your log buffer size. > > Do you mean to expand the dmesg buffer? That's probably unnecessary: I capture > the kernel log over a serial port into a file on another machine. > > > Could you show me the information with OOM, please ? > > Attached. It's compressed as there was rather a lot. > > David > --- Hi, David. Sorry for late response. I looked over your captured data when I got home but I didn't find any problem in lru page moving scheme. As Wu, Kosaki and Rik discussed, I think this issue is also related to process fork bomb. When I tested msgctl11 in my machine with 2.6.31-rc1, I found that: 2.6.31-rc1 real 0m38.628s user 0m10.589s sys 1m12.613s vmstat allocstall 3196 2.6.31-rc1-revert-mypatch real 1m17.396s user 0m11.193s sys 4m3.803s vmstat allocstall 584 Sometimes I got OOM, sometime not in with 2.6.31-rc1. Anyway, the current kernel's test took a rather short time than my reverted patch. In addition, the current kernel has small allocstall(direct reclaim) As you know, my patch was just to remove calling shrink_active_list in case of no swap. shrink_active_list function is a big cost function. The old shrink_active_list could throttle to fork processes by chance. But by removing that function with my patch, we have a high probability to make process fork bomb. Wu, KOSAKI and Rik, does it make sense? So I think you were just lucky with a unnecessary routine. Anyway, AFAIK, Rik is making throttling page reclaim. I think it can solve your problem. -- Kind regards, Minchan Kim -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/