Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934468Ab2FHUVa (ORCPT ); Fri, 8 Jun 2012 16:21:30 -0400 Received: from mail-pb0-f46.google.com ([209.85.160.46]:56078 "EHLO mail-pb0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933473Ab2FHUV3 (ORCPT ); Fri, 8 Jun 2012 16:21:29 -0400 Date: Fri, 8 Jun 2012 13:21:26 -0700 (PDT) From: David Rientjes X-X-Sender: rientjes@chino.kir.corp.google.com To: Linus Torvalds , Andrew Morton cc: Oleg Nesterov , Dave Jones , Hugh Dickins , Ingo Molnar , Peter Zijlstra , Srikar Dronamraju , KOSAKI Motohiro , Ananth N Mavinakayanahalli , Anton Arapov , Masami Hiramatsu , linux-kernel@vger.kernel.org Subject: [patch for-3.5-rc1] mm, oom: fix badness score underflow In-Reply-To: <20120608150437.GA31091@redhat.com> Message-ID: References: <20120607165942.GA31966@redhat.com> <20120608140328.GA26650@redhat.com> <20120608142625.GC1711@redhat.com> <20120608150437.GA31091@redhat.com> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1599 Lines: 42 If the privileges given to root threads (3% of allowable memory) or a negative value of /proc/pid/oom_score_adj happen to exceed the amount of rss of a thread, its badness score overflows as a result of a7f638f999ff ("mm, oom: normalize oom scores to oom_score_adj scale only for userspace"). Fix this by making the type signed and return 1, meaning the thread is still eligible for kill, if the value is negative. Reported-by: Dave Jones Acked-by: Oleg Nesterov Signed-off-by: David Rientjes --- mm/oom_kill.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/oom_kill.c b/mm/oom_kill.c --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -183,7 +183,7 @@ static bool oom_unkillable_task(struct task_struct *p, unsigned long oom_badness(struct task_struct *p, struct mem_cgroup *memcg, const nodemask_t *nodemask, unsigned long totalpages) { - unsigned long points; + long points; if (oom_unkillable_task(p, memcg, nodemask)) return 0; @@ -223,7 +223,7 @@ unsigned long oom_badness(struct task_struct *p, struct mem_cgroup *memcg, * Never return 0 for an eligible task regardless of the root bonus and * oom_score_adj (oom_score_adj can't be OOM_SCORE_ADJ_MIN here). */ - return points ? points : 1; + return points > 0 ? points : 1; } /* -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/