Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752827AbdFTJXg (ORCPT ); Tue, 20 Jun 2017 05:23:36 -0400 Received: from mail-pf0-f175.google.com ([209.85.192.175]:36282 "EHLO mail-pf0-f175.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752698AbdFTJKr (ORCPT ); Tue, 20 Jun 2017 05:10:47 -0400 Date: Tue, 20 Jun 2017 02:10:44 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Linus Torvalds cc: Dave Jones , Oleg Nesterov , Michal Hocko , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH] mm: fix new crash in unmapped_area_topdown() Message-ID: User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1576 Lines: 39 Trinity gets kernel BUG at mm/mmap.c:1963! in about 3 minutes of mmap testing. That's the VM_BUG_ON(gap_end < gap_start) at the end of unmapped_area_topdown(). Linus points out how MAP_FIXED (which does not have to respect our stack guard gap intentions) could result in gap_end below gap_start there. Fix that, and the similar case in its alternative, unmapped_area(). Cc: stable@vger.kernel.org Fixes: 1be7107fbe18 ("mm: larger stack guard gap, between vmas") Reported-by: Dave Jones Debugged-by: Linus Torvalds Signed-off-by: Hugh Dickins --- mm/mmap.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) --- 4.12-rc6/mm/mmap.c 2017-06-19 09:06:10.035407505 -0700 +++ linux/mm/mmap.c 2017-06-19 21:09:28.616707311 -0700 @@ -1817,7 +1817,8 @@ unsigned long unmapped_area(struct vm_un /* Check if current node has a suitable gap */ if (gap_start > high_limit) return -ENOMEM; - if (gap_end >= low_limit && gap_end - gap_start >= length) + if (gap_end >= low_limit && + gap_end > gap_start && gap_end - gap_start >= length) goto found; /* Visit right subtree if it looks promising */ @@ -1920,7 +1921,8 @@ unsigned long unmapped_area_topdown(stru gap_end = vm_start_gap(vma); if (gap_end < low_limit) return -ENOMEM; - if (gap_start <= high_limit && gap_end - gap_start >= length) + if (gap_start <= high_limit && + gap_end > gap_start && gap_end - gap_start >= length) goto found; /* Visit left subtree if it looks promising */