Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752197AbbHTMbO (ORCPT ); Thu, 20 Aug 2015 08:31:14 -0400 Received: from mail-wi0-f179.google.com ([209.85.212.179]:35161 "EHLO mail-wi0-f179.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751849AbbHTMbL (ORCPT ); Thu, 20 Aug 2015 08:31:11 -0400 Date: Thu, 20 Aug 2015 15:31:07 +0300 From: "Kirill A. Shutemov" To: Andrew Morton , Hugh Dickins Cc: Andrea Arcangeli , Dave Hansen , Vlastimil Babka , Johannes Weiner , Michal Hocko , David Rientjes , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCHv3 0/5] Fix compound_head() race Message-ID: <20150820123107.GA31768@node.dhcp.inet.fi> References: <1439976106-137226-1-git-send-email-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="6TrnltStXW4iwmi0" Content-Disposition: inline In-Reply-To: <1439976106-137226-1-git-send-email-kirill.shutemov@linux.intel.com> User-Agent: Mutt/1.5.23.1 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 27296 Lines: 737 --6TrnltStXW4iwmi0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Wed, Aug 19, 2015 at 12:21:41PM +0300, Kirill A. Shutemov wrote: > Here's my attempt on fixing recently discovered race in compound_head(). > It should make compound_head() reliable in all contexts. > > The patchset is against Linus' tree. Let me know if it need to be rebased > onto different baseline. > > It's expected to have conflicts with my page-flags patchset and probably > should be applied before it. > > v3: > - Fix build without hugetlb; > - Drop page->first_page; > - Update comment for free_compound_page(); > - Use 'unsigned int' for page order; > > v2: Per Hugh's suggestion page->compound_head is moved into third double > word. This way we can avoid memory overhead which v1 had in some > cases. > > This place in struct page is rather overloaded. More testing is > required to make sure we don't collide with anyone. Andrew, can we have the patchset applied, if nobody has objections? It applies cleanly into your patchstack just before my page-flags patchset. As expected, it causes few conflicts with patches: page-flags-introduce-page-flags-policies-wrt-compound-pages.patch mm-sanitize-page-mapping-for-tail-pages.patch include-linux-page-flagsh-rename-macros-to-avoid-collisions.patch Updated patches with solved conflicts are attached. Let me know if I need to do anything else about this. Hugh, does it address your worry wrt page-flags? Before you've mentioned races of whether the head page still agrees with the tail. I don't think it's an issue: you can get this kind of race only in very special environments like pfn scanner where you anyway need to re-validate the page after stabilizing it. Bloat from my page-flags is also reduced substantially. Size of your page_is_locked() example in allnoconfig case reduced from 32 to 17 bytes. With the patchset it look this way: 00003070 : 3070: 8b 50 14 mov 0x14(%eax),%edx 3073: f6 c2 01 test $0x1,%dl 3076: 8d 4a ff lea -0x1(%edx),%ecx 3079: 0f 45 c1 cmovne %ecx,%eax 307c: 8b 00 mov (%eax),%eax 307e: 24 01 and $0x1,%al 3080: c3 ret -- Kirill A. Shutemov --6TrnltStXW4iwmi0 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="include-linux-page-flagsh-rename-macros-to-avoid-collisions.patch" >From 1b88b5b6025e81a9f6b99275d66e129de52bd795 Mon Sep 17 00:00:00 2001 From: Andrew Morton Date: Tue, 18 Aug 2015 09:49:52 +1000 Subject: [PATCH] include/linux/page-flags.h: rename macros to avoid collisions Cc: "Kirill A. Shutemov" Signed-off-by: Andrew Morton --- include/linux/page-flags.h | 106 ++++++++++++++++++++++----------------------- 1 file changed, 53 insertions(+), 53 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 3d9270a9e885..cfff9fd5d858 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -130,15 +130,15 @@ enum pageflags { #ifndef __GENERATING_BOUNDS_H /* Page flags policies wrt compound pages */ -#define ANY(page, enforce) page -#define HEAD(page, enforce) compound_head(page) -#define NO_TAIL(page, enforce) ({ \ +#define PF_ANY(page, enforce) page +#define PF_HEAD(page, enforce) compound_head(page) +#define PF_NO_TAIL(page, enforce) ({ \ if (enforce) \ VM_BUG_ON_PAGE(PageTail(page), page); \ else \ page = compound_head(page); \ page;}) -#define NO_COMPOUND(page, enforce) ({ \ +#define PF_NO_COMPOUND(page, enforce) ({ \ if (enforce) \ VM_BUG_ON_PAGE(PageCompound(page), page); \ page;}) @@ -225,55 +225,55 @@ static inline int PageCompound(struct page *page); static inline int PageTail(struct page *page); static struct page *compound_head(struct page *page); -__PAGEFLAG(Locked, locked, NO_TAIL) -PAGEFLAG(Error, error, NO_COMPOUND) TESTCLEARFLAG(Error, error, NO_COMPOUND) -PAGEFLAG(Referenced, referenced, HEAD) - TESTCLEARFLAG(Referenced, referenced, HEAD) - __SETPAGEFLAG(Referenced, referenced, HEAD) -PAGEFLAG(Dirty, dirty, HEAD) TESTSCFLAG(Dirty, dirty, HEAD) - __CLEARPAGEFLAG(Dirty, dirty, HEAD) -PAGEFLAG(LRU, lru, HEAD) __CLEARPAGEFLAG(LRU, lru, HEAD) -PAGEFLAG(Active, active, HEAD) __CLEARPAGEFLAG(Active, active, HEAD) - TESTCLEARFLAG(Active, active, HEAD) -__PAGEFLAG(Slab, slab, NO_TAIL) -__PAGEFLAG(SlobFree, slob_free, NO_TAIL) -PAGEFLAG(Checked, checked, NO_COMPOUND) /* Used by some filesystems */ +__PAGEFLAG(Locked, locked, PF_NO_TAIL) +PAGEFLAG(Error, error, PF_NO_COMPOUND) TESTCLEARFLAG(Error, error, PF_NO_COMPOUND) +PAGEFLAG(Referenced, referenced, PF_HEAD) + TESTCLEARFLAG(Referenced, referenced, PF_HEAD) + __SETPAGEFLAG(Referenced, referenced, PF_HEAD) +PAGEFLAG(Dirty, dirty, PF_HEAD) TESTSCFLAG(Dirty, dirty, PF_HEAD) + __CLEARPAGEFLAG(Dirty, dirty, PF_HEAD) +PAGEFLAG(LRU, lru, PF_HEAD) __CLEARPAGEFLAG(LRU, lru, PF_HEAD) +PAGEFLAG(Active, active, PF_HEAD) __CLEARPAGEFLAG(Active, active, PF_HEAD) + TESTCLEARFLAG(Active, active, PF_HEAD) +__PAGEFLAG(Slab, slab, PF_NO_TAIL) +__PAGEFLAG(SlobFree, slob_free, PF_NO_TAIL) +PAGEFLAG(Checked, checked, PF_NO_COMPOUND) /* Used by some filesystems */ /* Xen */ -PAGEFLAG(Pinned, pinned, NO_COMPOUND) TESTSCFLAG(Pinned, pinned, NO_COMPOUND) -PAGEFLAG(SavePinned, savepinned, NO_COMPOUND) -PAGEFLAG(Foreign, foreign, NO_COMPOUND) +PAGEFLAG(Pinned, pinned, PF_NO_COMPOUND) TESTSCFLAG(Pinned, pinned, PF_NO_COMPOUND) +PAGEFLAG(SavePinned, savepinned, PF_NO_COMPOUND) +PAGEFLAG(Foreign, foreign, PF_NO_COMPOUND) -PAGEFLAG(Reserved, reserved, NO_COMPOUND) - __CLEARPAGEFLAG(Reserved, reserved, NO_COMPOUND) -PAGEFLAG(SwapBacked, swapbacked, NO_TAIL) - __CLEARPAGEFLAG(SwapBacked, swapbacked, NO_TAIL) - __SETPAGEFLAG(SwapBacked, swapbacked, NO_TAIL) +PAGEFLAG(Reserved, reserved, PF_NO_COMPOUND) + __CLEARPAGEFLAG(Reserved, reserved, PF_NO_COMPOUND) +PAGEFLAG(SwapBacked, swapbacked, PF_NO_TAIL) + __CLEARPAGEFLAG(SwapBacked, swapbacked, PF_NO_TAIL) + __SETPAGEFLAG(SwapBacked, swapbacked, PF_NO_TAIL) /* * Private page markings that may be used by the filesystem that owns the page * for its own purposes. * - PG_private and PG_private_2 cause releasepage() and co to be invoked */ -PAGEFLAG(Private, private, ANY) __SETPAGEFLAG(Private, private, ANY) - __CLEARPAGEFLAG(Private, private, ANY) -PAGEFLAG(Private2, private_2, ANY) TESTSCFLAG(Private2, private_2, ANY) -PAGEFLAG(OwnerPriv1, owner_priv_1, ANY) - TESTCLEARFLAG(OwnerPriv1, owner_priv_1, ANY) +PAGEFLAG(Private, private, PF_ANY) __SETPAGEFLAG(Private, private, PF_ANY) + __CLEARPAGEFLAG(Private, private, PF_ANY) +PAGEFLAG(Private2, private_2, PF_ANY) TESTSCFLAG(Private2, private_2, PF_ANY) +PAGEFLAG(OwnerPriv1, owner_priv_1, PF_ANY) + TESTCLEARFLAG(OwnerPriv1, owner_priv_1, PF_ANY) /* * Only test-and-set exist for PG_writeback. The unconditional operators are * risky: they bypass page accounting. */ -TESTPAGEFLAG(Writeback, writeback, NO_COMPOUND) - TESTSCFLAG(Writeback, writeback, NO_COMPOUND) -PAGEFLAG(MappedToDisk, mappedtodisk, NO_COMPOUND) +TESTPAGEFLAG(Writeback, writeback, PF_NO_COMPOUND) + TESTSCFLAG(Writeback, writeback, PF_NO_COMPOUND) +PAGEFLAG(MappedToDisk, mappedtodisk, PF_NO_COMPOUND) /* PG_readahead is only used for reads; PG_reclaim is only for writes */ -PAGEFLAG(Reclaim, reclaim, NO_COMPOUND) - TESTCLEARFLAG(Reclaim, reclaim, NO_COMPOUND) -PAGEFLAG(Readahead, reclaim, NO_COMPOUND) - TESTCLEARFLAG(Readahead, reclaim, NO_COMPOUND) +PAGEFLAG(Reclaim, reclaim, PF_NO_COMPOUND) + TESTCLEARFLAG(Reclaim, reclaim, PF_NO_COMPOUND) +PAGEFLAG(Readahead, reclaim, PF_NO_COMPOUND) + TESTCLEARFLAG(Readahead, reclaim, PF_NO_COMPOUND) #ifdef CONFIG_HIGHMEM /* @@ -286,33 +286,33 @@ PAGEFLAG_FALSE(HighMem) #endif #ifdef CONFIG_SWAP -PAGEFLAG(SwapCache, swapcache, NO_COMPOUND) +PAGEFLAG(SwapCache, swapcache, PF_NO_COMPOUND) #else PAGEFLAG_FALSE(SwapCache) #endif -PAGEFLAG(Unevictable, unevictable, HEAD) - __CLEARPAGEFLAG(Unevictable, unevictable, HEAD) - TESTCLEARFLAG(Unevictable, unevictable, HEAD) +PAGEFLAG(Unevictable, unevictable, PF_HEAD) + __CLEARPAGEFLAG(Unevictable, unevictable, PF_HEAD) + TESTCLEARFLAG(Unevictable, unevictable, PF_HEAD) #ifdef CONFIG_MMU -PAGEFLAG(Mlocked, mlocked, NO_TAIL) __CLEARPAGEFLAG(Mlocked, mlocked, NO_TAIL) - TESTSCFLAG(Mlocked, mlocked, NO_TAIL) - __TESTCLEARFLAG(Mlocked, mlocked, NO_TAIL) +PAGEFLAG(Mlocked, mlocked, PF_NO_TAIL) __CLEARPAGEFLAG(Mlocked, mlocked, PF_NO_TAIL) + TESTSCFLAG(Mlocked, mlocked, PF_NO_TAIL) + __TESTCLEARFLAG(Mlocked, mlocked, PF_NO_TAIL) #else PAGEFLAG_FALSE(Mlocked) __CLEARPAGEFLAG_NOOP(Mlocked) TESTSCFLAG_FALSE(Mlocked) __TESTCLEARFLAG_FALSE(Mlocked) #endif #ifdef CONFIG_ARCH_USES_PG_UNCACHED -PAGEFLAG(Uncached, uncached, NO_COMPOUND) +PAGEFLAG(Uncached, uncached, PF_NO_COMPOUND) #else PAGEFLAG_FALSE(Uncached) #endif #ifdef CONFIG_MEMORY_FAILURE -PAGEFLAG(HWPoison, hwpoison, ANY) -TESTSCFLAG(HWPoison, hwpoison, ANY) +PAGEFLAG(HWPoison, hwpoison, PF_ANY) +TESTSCFLAG(HWPoison, hwpoison, PF_ANY) #define __PG_HWPOISON (1UL << PG_hwpoison) #else PAGEFLAG_FALSE(HWPoison) @@ -402,7 +402,7 @@ static inline void SetPageUptodate(struct page *page) set_bit(PG_uptodate, &page->flags); } -CLEARPAGEFLAG(Uptodate, uptodate, NO_TAIL) +CLEARPAGEFLAG(Uptodate, uptodate, PF_NO_TAIL) int test_clear_page_writeback(struct page *page); int __test_set_page_writeback(struct page *page, bool keep_write); @@ -422,7 +422,7 @@ static inline void set_page_writeback_keepwrite(struct page *page) test_set_page_writeback_keepwrite(page); } -__PAGEFLAG(Head, head, ANY) CLEARPAGEFLAG(Head, head, ANY) +__PAGEFLAG(Head, head, PF_ANY) CLEARPAGEFLAG(Head, head, PF_ANY) static inline int PageTail(struct page *page) { @@ -643,10 +643,10 @@ static inline int page_has_private(struct page *page) return !!(page->flags & PAGE_FLAGS_PRIVATE); } -#undef ANY -#undef HEAD -#undef NO_TAIL -#undef NO_COMPOUND +#undef PF_ANY +#undef PF_HEAD +#undef PF_NO_TAIL +#undef PF_NO_COMPOUND #endif /* !__GENERATING_BOUNDS_H */ #endif /* PAGE_FLAGS_H */ -- 2.5.0 --6TrnltStXW4iwmi0 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="page-flags-introduce-page-flags-policies-wrt-compound-pages.patch" >From 54d99b201f355af4e4bd401a1b39a8570dcda948 Mon Sep 17 00:00:00 2001 From: "Kirill A. Shutemov" Date: Tue, 18 Aug 2015 09:49:48 +1000 Subject: [PATCH] page-flags: introduce page flags policies wrt compound pages This patch adds a third argument to macros which create function definitions for page flags. This argument defines how page-flags helpers behave on compound functions. For now we define four policies: - PF_ANY: the helper function operates on the page it gets, regardless if it's non-compound, head or tail. - PF_HEAD: the helper function operates on the head page of the compound page if it gets tail page. - PF_NO_TAIL: only head and non-compond pages are acceptable for this helper function. - PF_NO_COMPOUND: only non-compound pages are acceptable for this helper function. For now we use policy PF_ANY for all helpers, which matches current behaviour. We do not enforce the policy for TESTPAGEFLAG, because we have flags checked for random pages all over the kernel. Noticeable exception to this is PageTransHuge() which triggers VM_BUG_ON() for tail page. Signed-off-by: Kirill A. Shutemov Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Dave Hansen Cc: Mel Gorman Cc: Rik van Riel Cc: Vlastimil Babka Cc: Christoph Lameter Cc: Naoya Horiguchi Cc: Steve Capper Cc: "Aneesh Kumar K.V" Cc: Johannes Weiner Cc: Michal Hocko Cc: Jerome Marchand Signed-off-by: Andrew Morton --- include/linux/page-flags.h | 153 +++++++++++++++++++++++++++------------------ 1 file changed, 92 insertions(+), 61 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 490fbd3f8552..85b60119523a 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -129,49 +129,68 @@ enum pageflags { #ifndef __GENERATING_BOUNDS_H +/* Page flags policies wrt compound pages */ +#define ANY(page, enforce) page +#define HEAD(page, enforce) compound_head(page) +#define NO_TAIL(page, enforce) ({ \ + if (enforce) \ + VM_BUG_ON_PAGE(PageTail(page), page); \ + else \ + page = compound_head(page); \ + page;}) +#define NO_COMPOUND(page, enforce) ({ \ + if (enforce) \ + VM_BUG_ON_PAGE(PageCompound(page), page); \ + page;}) + /* * Macros to create function definitions for page flags */ -#define TESTPAGEFLAG(uname, lname) \ -static inline int Page##uname(const struct page *page) \ - { return test_bit(PG_##lname, &page->flags); } +#define TESTPAGEFLAG(uname, lname, policy) \ +static inline int Page##uname(struct page *page) \ + { return test_bit(PG_##lname, &policy(page, 0)->flags); } -#define SETPAGEFLAG(uname, lname) \ +#define SETPAGEFLAG(uname, lname, policy) \ static inline void SetPage##uname(struct page *page) \ - { set_bit(PG_##lname, &page->flags); } + { set_bit(PG_##lname, &policy(page, 1)->flags); } -#define CLEARPAGEFLAG(uname, lname) \ +#define CLEARPAGEFLAG(uname, lname, policy) \ static inline void ClearPage##uname(struct page *page) \ - { clear_bit(PG_##lname, &page->flags); } + { clear_bit(PG_##lname, &policy(page, 1)->flags); } -#define __SETPAGEFLAG(uname, lname) \ +#define __SETPAGEFLAG(uname, lname, policy) \ static inline void __SetPage##uname(struct page *page) \ - { __set_bit(PG_##lname, &page->flags); } + { __set_bit(PG_##lname, &policy(page, 1)->flags); } -#define __CLEARPAGEFLAG(uname, lname) \ +#define __CLEARPAGEFLAG(uname, lname, policy) \ static inline void __ClearPage##uname(struct page *page) \ - { __clear_bit(PG_##lname, &page->flags); } + { __clear_bit(PG_##lname, &policy(page, 1)->flags); } -#define TESTSETFLAG(uname, lname) \ +#define TESTSETFLAG(uname, lname, policy) \ static inline int TestSetPage##uname(struct page *page) \ - { return test_and_set_bit(PG_##lname, &page->flags); } + { return test_and_set_bit(PG_##lname, &policy(page, 1)->flags); } -#define TESTCLEARFLAG(uname, lname) \ +#define TESTCLEARFLAG(uname, lname, policy) \ static inline int TestClearPage##uname(struct page *page) \ - { return test_and_clear_bit(PG_##lname, &page->flags); } + { return test_and_clear_bit(PG_##lname, &policy(page, 1)->flags); } -#define __TESTCLEARFLAG(uname, lname) \ +#define __TESTCLEARFLAG(uname, lname, policy) \ static inline int __TestClearPage##uname(struct page *page) \ - { return __test_and_clear_bit(PG_##lname, &page->flags); } + { return __test_and_clear_bit(PG_##lname, &policy(page, 1)->flags); } -#define PAGEFLAG(uname, lname) TESTPAGEFLAG(uname, lname) \ - SETPAGEFLAG(uname, lname) CLEARPAGEFLAG(uname, lname) +#define PAGEFLAG(uname, lname, policy) \ + TESTPAGEFLAG(uname, lname, policy) \ + SETPAGEFLAG(uname, lname, policy) \ + CLEARPAGEFLAG(uname, lname, policy) -#define __PAGEFLAG(uname, lname) TESTPAGEFLAG(uname, lname) \ - __SETPAGEFLAG(uname, lname) __CLEARPAGEFLAG(uname, lname) +#define __PAGEFLAG(uname, lname, policy) \ + TESTPAGEFLAG(uname, lname, policy) \ + __SETPAGEFLAG(uname, lname, policy) \ + __CLEARPAGEFLAG(uname, lname, policy) -#define TESTSCFLAG(uname, lname) \ - TESTSETFLAG(uname, lname) TESTCLEARFLAG(uname, lname) +#define TESTSCFLAG(uname, lname, policy) \ + TESTSETFLAG(uname, lname, policy) \ + TESTCLEARFLAG(uname, lname, policy) #define TESTPAGEFLAG_FALSE(uname) \ static inline int Page##uname(const struct page *page) { return 0; } @@ -200,47 +219,54 @@ static inline int __TestClearPage##uname(struct page *page) { return 0; } #define TESTSCFLAG_FALSE(uname) \ TESTSETFLAG_FALSE(uname) TESTCLEARFLAG_FALSE(uname) -struct page; /* forward declaration */ - -TESTPAGEFLAG(Locked, locked) -PAGEFLAG(Error, error) TESTCLEARFLAG(Error, error) -PAGEFLAG(Referenced, referenced) TESTCLEARFLAG(Referenced, referenced) - __SETPAGEFLAG(Referenced, referenced) -PAGEFLAG(Dirty, dirty) TESTSCFLAG(Dirty, dirty) __CLEARPAGEFLAG(Dirty, dirty) -PAGEFLAG(LRU, lru) __CLEARPAGEFLAG(LRU, lru) -PAGEFLAG(Active, active) __CLEARPAGEFLAG(Active, active) - TESTCLEARFLAG(Active, active) -__PAGEFLAG(Slab, slab) -PAGEFLAG(Checked, checked) /* Used by some filesystems */ -PAGEFLAG(Pinned, pinned) TESTSCFLAG(Pinned, pinned) /* Xen */ -PAGEFLAG(SavePinned, savepinned); /* Xen */ -PAGEFLAG(Foreign, foreign); /* Xen */ -PAGEFLAG(Reserved, reserved) __CLEARPAGEFLAG(Reserved, reserved) -PAGEFLAG(SwapBacked, swapbacked) __CLEARPAGEFLAG(SwapBacked, swapbacked) - __SETPAGEFLAG(SwapBacked, swapbacked) - -__PAGEFLAG(SlobFree, slob_free) +/* Forward declarations */ +struct page; +static inline int PageCompound(struct page *page); +static inline int PageTail(struct page *page); +static struct page *compound_head(struct page *page); + +TESTPAGEFLAG(Locked, locked, ANY) +PAGEFLAG(Error, error, ANY) TESTCLEARFLAG(Error, error, ANY) +PAGEFLAG(Referenced, referenced, ANY) TESTCLEARFLAG(Referenced, referenced, ANY) + __SETPAGEFLAG(Referenced, referenced, ANY) +PAGEFLAG(Dirty, dirty, ANY) TESTSCFLAG(Dirty, dirty, ANY) + __CLEARPAGEFLAG(Dirty, dirty, ANY) +PAGEFLAG(LRU, lru, ANY) __CLEARPAGEFLAG(LRU, lru, ANY) +PAGEFLAG(Active, active, ANY) __CLEARPAGEFLAG(Active, active, ANY) + TESTCLEARFLAG(Active, active, ANY) +__PAGEFLAG(Slab, slab, ANY) +PAGEFLAG(Checked, checked, ANY) /* Used by some filesystems */ +PAGEFLAG(Pinned, pinned, ANY) TESTSCFLAG(Pinned, pinned, ANY) /* Xen */ +PAGEFLAG(SavePinned, savepinned, ANY); /* Xen */ +PAGEFLAG(Foreign, foreign, ANY); /* Xen */ +PAGEFLAG(Reserved, reserved, ANY) __CLEARPAGEFLAG(Reserved, reserved, ANY) +PAGEFLAG(SwapBacked, swapbacked, ANY) + __CLEARPAGEFLAG(SwapBacked, swapbacked, ANY) + __SETPAGEFLAG(SwapBacked, swapbacked, ANY) + +__PAGEFLAG(SlobFree, slob_free, ANY) /* * Private page markings that may be used by the filesystem that owns the page * for its own purposes. * - PG_private and PG_private_2 cause releasepage() and co to be invoked */ -PAGEFLAG(Private, private) __SETPAGEFLAG(Private, private) - __CLEARPAGEFLAG(Private, private) -PAGEFLAG(Private2, private_2) TESTSCFLAG(Private2, private_2) -PAGEFLAG(OwnerPriv1, owner_priv_1) TESTCLEARFLAG(OwnerPriv1, owner_priv_1) +PAGEFLAG(Private, private, ANY) __SETPAGEFLAG(Private, private, ANY) + __CLEARPAGEFLAG(Private, private, ANY) +PAGEFLAG(Private2, private_2, ANY) TESTSCFLAG(Private2, private_2, ANY) +PAGEFLAG(OwnerPriv1, owner_priv_1, ANY) + TESTCLEARFLAG(OwnerPriv1, owner_priv_1, ANY) /* * Only test-and-set exist for PG_writeback. The unconditional operators are * risky: they bypass page accounting. */ -TESTPAGEFLAG(Writeback, writeback) TESTSCFLAG(Writeback, writeback) -PAGEFLAG(MappedToDisk, mappedtodisk) +TESTPAGEFLAG(Writeback, writeback, ANY) TESTSCFLAG(Writeback, writeback, ANY) +PAGEFLAG(MappedToDisk, mappedtodisk, ANY) /* PG_readahead is only used for reads; PG_reclaim is only for writes */ -PAGEFLAG(Reclaim, reclaim) TESTCLEARFLAG(Reclaim, reclaim) -PAGEFLAG(Readahead, reclaim) TESTCLEARFLAG(Readahead, reclaim) +PAGEFLAG(Reclaim, reclaim, ANY) TESTCLEARFLAG(Reclaim, reclaim, ANY) +PAGEFLAG(Readahead, reclaim, ANY) TESTCLEARFLAG(Readahead, reclaim, ANY) #ifdef CONFIG_HIGHMEM /* @@ -253,31 +279,32 @@ PAGEFLAG_FALSE(HighMem) #endif #ifdef CONFIG_SWAP -PAGEFLAG(SwapCache, swapcache) +PAGEFLAG(SwapCache, swapcache, ANY) #else PAGEFLAG_FALSE(SwapCache) #endif -PAGEFLAG(Unevictable, unevictable) __CLEARPAGEFLAG(Unevictable, unevictable) - TESTCLEARFLAG(Unevictable, unevictable) +PAGEFLAG(Unevictable, unevictable, ANY) + __CLEARPAGEFLAG(Unevictable, unevictable, ANY) + TESTCLEARFLAG(Unevictable, unevictable, ANY) #ifdef CONFIG_MMU -PAGEFLAG(Mlocked, mlocked) __CLEARPAGEFLAG(Mlocked, mlocked) - TESTSCFLAG(Mlocked, mlocked) __TESTCLEARFLAG(Mlocked, mlocked) +PAGEFLAG(Mlocked, mlocked, ANY) __CLEARPAGEFLAG(Mlocked, mlocked, ANY) + TESTSCFLAG(Mlocked, mlocked, ANY) __TESTCLEARFLAG(Mlocked, mlocked, ANY) #else PAGEFLAG_FALSE(Mlocked) __CLEARPAGEFLAG_NOOP(Mlocked) TESTSCFLAG_FALSE(Mlocked) __TESTCLEARFLAG_FALSE(Mlocked) #endif #ifdef CONFIG_ARCH_USES_PG_UNCACHED -PAGEFLAG(Uncached, uncached) +PAGEFLAG(Uncached, uncached, ANY) #else PAGEFLAG_FALSE(Uncached) #endif #ifdef CONFIG_MEMORY_FAILURE -PAGEFLAG(HWPoison, hwpoison) -TESTSCFLAG(HWPoison, hwpoison) +PAGEFLAG(HWPoison, hwpoison, ANY) +TESTSCFLAG(HWPoison, hwpoison, ANY) #define __PG_HWPOISON (1UL << PG_hwpoison) #else PAGEFLAG_FALSE(HWPoison) @@ -362,7 +389,7 @@ static inline void SetPageUptodate(struct page *page) set_bit(PG_uptodate, &(page)->flags); } -CLEARPAGEFLAG(Uptodate, uptodate) +CLEARPAGEFLAG(Uptodate, uptodate, ANY) int test_clear_page_writeback(struct page *page); int __test_set_page_writeback(struct page *page, bool keep_write); @@ -382,7 +409,7 @@ static inline void set_page_writeback_keepwrite(struct page *page) test_set_page_writeback_keepwrite(page); } -__PAGEFLAG(Head, head) CLEARPAGEFLAG(Head, head) +__PAGEFLAG(Head, head, ANY) CLEARPAGEFLAG(Head, head, ANY) static inline int PageTail(struct page *page) { @@ -603,6 +630,10 @@ static inline int page_has_private(struct page *page) return !!(page->flags & PAGE_FLAGS_PRIVATE); } +#undef ANY +#undef HEAD +#undef NO_TAIL +#undef NO_COMPOUND #endif /* !__GENERATING_BOUNDS_H */ #endif /* PAGE_FLAGS_H */ -- 2.5.0 --6TrnltStXW4iwmi0 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="mm-sanitize-page-mapping-for-tail-pages.patch" >From cdb3d7e7f717f3e96ee97c4b0a29a745701bd813 Mon Sep 17 00:00:00 2001 From: "Kirill A. Shutemov" Date: Tue, 18 Aug 2015 09:49:51 +1000 Subject: [PATCH] mm: sanitize page->mapping for tail pages We don't define meaning of page->mapping for tail pages. Currently it's always NULL, which can be inconsistent with head page and potentially lead to problems. Let's poison the pointer to catch all illigal uses. page_rmapping(), page_mapping() and page_anon_vma() are changed to look on head page. The only illegal use I've caught so far is __GPF_COMP pages from sound subsystem, mapped with PTEs. do_shared_fault() is changed to use page_rmapping() instead of direct access to fault_page->mapping. Signed-off-by: Kirill A. Shutemov Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Dave Hansen Cc: Mel Gorman Cc: Rik van Riel Cc: Vlastimil Babka Cc: Christoph Lameter Cc: Naoya Horiguchi Cc: Steve Capper Cc: "Aneesh Kumar K.V" Cc: Johannes Weiner Cc: Michal Hocko Cc: Jerome Marchand Signed-off-by: Andrew Morton --- include/linux/poison.h | 4 ++++ mm/huge_memory.c | 2 +- mm/memory.c | 2 +- mm/page_alloc.c | 6 ++++++ mm/util.c | 10 ++++++---- 5 files changed, 18 insertions(+), 6 deletions(-) diff --git a/include/linux/poison.h b/include/linux/poison.h index 2110a81c5e2a..7b2a7fcde6a3 100644 --- a/include/linux/poison.h +++ b/include/linux/poison.h @@ -32,6 +32,10 @@ /********** mm/debug-pagealloc.c **********/ #define PAGE_POISON 0xaa +/********** mm/page_alloc.c ************/ + +#define TAIL_MAPPING ((void *) 0x01014A11 + POISON_POINTER_DELTA) + /********** mm/slab.c **********/ /* * Magic nums for obj red zoning. diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 7ef15f8f8bf3..1a3accef0756 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1772,7 +1772,7 @@ static void __split_huge_page_refcount(struct page *page, */ page_tail->_mapcount = page->_mapcount; - BUG_ON(page_tail->mapping); + BUG_ON(page_tail->mapping != TAIL_MAPPING); page_tail->mapping = page->mapping; page_tail->index = page->index + i; diff --git a/mm/memory.c b/mm/memory.c index 6cd0b2160401..558ee16167d9 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3087,7 +3087,7 @@ static int do_shared_fault(struct mm_struct *mm, struct vm_area_struct *vma, * pinned by vma->vm_file's reference. We rely on unlock_page()'s * release semantics to prevent the compiler from undoing this copying. */ - mapping = fault_page->mapping; + mapping = page_rmapping(fault_page); unlock_page(fault_page); if ((dirtied || vma->vm_ops->page_mkwrite) && mapping) { /* diff --git a/mm/page_alloc.c b/mm/page_alloc.c index d752298a9e48..adefa3ad8e3e 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -470,6 +470,7 @@ void prep_compound_page(struct page *page, unsigned int order) for (i = 1; i < nr_pages; i++) { struct page *p = page + i; set_page_count(p, 0); + p->mapping = TAIL_MAPPING; set_compound_head(p, page); } } @@ -855,6 +856,10 @@ static int free_tail_pages_check(struct page *head_page, struct page *page) ret = 0; goto out; } + if (page->mapping != TAIL_MAPPING) { + bad_page(page, "corrupted mapping in tail page", 0); + goto out; + } if (unlikely(!PageTail(page))) { bad_page(page, "PageTail not set", 0); goto out; @@ -865,6 +870,7 @@ static int free_tail_pages_check(struct page *head_page, struct page *page) } ret = 0; out: + page->mapping = NULL; clear_compound_head(page); return ret; } diff --git a/mm/util.c b/mm/util.c index 68ff8a5361e7..0c7f65e7ef5e 100644 --- a/mm/util.c +++ b/mm/util.c @@ -355,7 +355,9 @@ struct anon_vma *page_anon_vma(struct page *page) struct address_space *page_mapping(struct page *page) { - unsigned long mapping; + struct address_space *mapping; + + page = compound_head(page); /* This happens if someone calls flush_dcache_page on slab page */ if (unlikely(PageSlab(page))) @@ -368,10 +370,10 @@ struct address_space *page_mapping(struct page *page) return swap_address_space(entry); } - mapping = (unsigned long)page->mapping; - if (mapping & PAGE_MAPPING_FLAGS) + mapping = page->mapping; + if ((unsigned long)mapping & PAGE_MAPPING_FLAGS) return NULL; - return page->mapping; + return mapping; } int overcommit_ratio_handler(struct ctl_table *table, int write, -- 2.5.0 --6TrnltStXW4iwmi0-- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/