Replace superfluous VM_BUG_ON() with comment about correct usage.
Technically reverts commit 1d148e218a0d0566b1c06f2f45f1436d53b049b2
("mm: add VM_BUG_ON_PAGE() to page_mapcount()"), but context have changed.
Function isolate_migratepages_block() runs some checks out of lru_lock
when choose pages for migration. After checking PageLRU() it checks extra
page references by comparing page_count() and page_mapcount(). Between
these two checks page could be removed from lru, freed and taken by slab.
As a result this race triggers VM_BUG_ON(PageSlab()) in page_mapcount().
Race window is tiny. For certain workload this happens around once a year.
page:ffffea0105ca9380 count:1 mapcount:0 mapping:ffff88ff7712c180 index:0x0 compound_mapcount: 0
flags: 0x500000000008100(slab|head)
raw: 0500000000008100 dead000000000100 dead000000000200 ffff88ff7712c180
raw: 0000000000000000 0000000080200020 00000001ffffffff 0000000000000000
page dumped because: VM_BUG_ON_PAGE(PageSlab(page))
------------[ cut here ]------------
kernel BUG at ./include/linux/mm.h:628!
invalid opcode: 0000 [#1] SMP NOPTI
CPU: 77 PID: 504 Comm: kcompactd1 Tainted: G W 4.19.109-27 #1
Hardware name: Yandex T175-N41-Y3N/MY81-EX0-Y3N, BIOS R05 06/20/2019
RIP: 0010:isolate_migratepages_block+0x986/0x9b0
Code in isolate_migratepages_block() was added in commit 119d6d59dcc0
("mm, compaction: avoid isolating pinned pages") before adding VM_BUG_ON
into page_mapcount().
This race has been predicted in 2015 by Vlastimil Babka (see link below).
Signed-off-by: Konstantin Khlebnikov <[email protected]>
Fixes: 1d148e218a0d ("mm: add VM_BUG_ON_PAGE() to page_mapcount()")
Link: https://lore.kernel.org/lkml/[email protected]/
Link: https://lore.kernel.org/linux-mm/158937872515.474360.5066096871639561424.stgit@buzz/T/ (v1)
---
include/linux/mm.h | 14 ++++++++++++--
1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 5a323422d783..95f777f482ac 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -782,6 +782,11 @@ static inline void *kvcalloc(size_t n, size_t size, gfp_t flags)
extern void kvfree(const void *addr);
+/*
+ * Mapcount of compound page as a whole, not includes mapped sub-pages.
+ *
+ * Must be called only for compound pages or any their tail sub-pages.
+ */
static inline int compound_mapcount(struct page *page)
{
VM_BUG_ON_PAGE(!PageCompound(page), page);
@@ -801,10 +806,15 @@ static inline void page_mapcount_reset(struct page *page)
int __page_mapcount(struct page *page);
+/*
+ * Mapcount of 0-order page, for sub-page includes compound_mapcount().
+ *
+ * Result is undefined for pages which cannot be mapped into userspace.
+ * For example SLAB or special types of pages. See function page_has_type().
+ * They use this place in struct page differently.
+ */
static inline int page_mapcount(struct page *page)
{
- VM_BUG_ON_PAGE(PageSlab(page), page);
-
if (unlikely(PageCompound(page)))
return __page_mapcount(page);
return atomic_read(&page->_mapcount) + 1;
On Sun, 24 May 2020, Konstantin Khlebnikov wrote:
> Replace superfluous VM_BUG_ON() with comment about correct usage.
>
> Technically reverts commit 1d148e218a0d0566b1c06f2f45f1436d53b049b2
> ("mm: add VM_BUG_ON_PAGE() to page_mapcount()"), but context have changed.
>
> Function isolate_migratepages_block() runs some checks out of lru_lock
> when choose pages for migration. After checking PageLRU() it checks extra
> page references by comparing page_count() and page_mapcount(). Between
> these two checks page could be removed from lru, freed and taken by slab.
>
> As a result this race triggers VM_BUG_ON(PageSlab()) in page_mapcount().
> Race window is tiny. For certain workload this happens around once a year.
>
>
> page:ffffea0105ca9380 count:1 mapcount:0 mapping:ffff88ff7712c180 index:0x0 compound_mapcount: 0
> flags: 0x500000000008100(slab|head)
> raw: 0500000000008100 dead000000000100 dead000000000200 ffff88ff7712c180
> raw: 0000000000000000 0000000080200020 00000001ffffffff 0000000000000000
> page dumped because: VM_BUG_ON_PAGE(PageSlab(page))
> ------------[ cut here ]------------
> kernel BUG at ./include/linux/mm.h:628!
> invalid opcode: 0000 [#1] SMP NOPTI
> CPU: 77 PID: 504 Comm: kcompactd1 Tainted: G W 4.19.109-27 #1
> Hardware name: Yandex T175-N41-Y3N/MY81-EX0-Y3N, BIOS R05 06/20/2019
> RIP: 0010:isolate_migratepages_block+0x986/0x9b0
>
>
> Code in isolate_migratepages_block() was added in commit 119d6d59dcc0
> ("mm, compaction: avoid isolating pinned pages") before adding VM_BUG_ON
> into page_mapcount().
>
> This race has been predicted in 2015 by Vlastimil Babka (see link below).
Ah, kudos to Vlastimil.
>
> Signed-off-by: Konstantin Khlebnikov <[email protected]>
Thanks,
Acked-by: Hugh Dickins <[email protected]>
For the BUG deletion. I got a bit confused by the comments at first:
slight rewording suggested below.
I had also intended a comment in page_mapcount(), on being used in
that racy exceptional way by isolate_migratepages_block(): but now
that the BUG is gone, I think you're right not to add that - it's
not for the function to document its various callers, nor what's
not in it. That kind of info belongs to the commit message,
as you have done.
> Fixes: 1d148e218a0d ("mm: add VM_BUG_ON_PAGE() to page_mapcount()")
> Link: https://lore.kernel.org/lkml/[email protected]/
> Link: https://lore.kernel.org/linux-mm/158937872515.474360.5066096871639561424.stgit@buzz/T/ (v1)
> ---
> include/linux/mm.h | 14 ++++++++++++--
> 1 file changed, 12 insertions(+), 2 deletions(-)
>
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index 5a323422d783..95f777f482ac 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -782,6 +782,11 @@ static inline void *kvcalloc(size_t n, size_t size, gfp_t flags)
>
> extern void kvfree(const void *addr);
>
> +/*
> + * Mapcount of compound page as a whole, not includes mapped sub-pages.
s/not includes/does not include/
> + *
> + * Must be called only for compound pages or any their tail sub-pages.
> + */
> static inline int compound_mapcount(struct page *page)
> {
> VM_BUG_ON_PAGE(!PageCompound(page), page);
> @@ -801,10 +806,15 @@ static inline void page_mapcount_reset(struct page *page)
>
> int __page_mapcount(struct page *page);
>
> +/*
> + * Mapcount of 0-order page, for sub-page includes compound_mapcount().
s/, for sub-page/; when compound sub-page,/
> + *
> + * Result is undefined for pages which cannot be mapped into userspace.
> + * For example SLAB or special types of pages. See function page_has_type().
> + * They use this place in struct page differently.
> + */
> static inline int page_mapcount(struct page *page)
> {
> - VM_BUG_ON_PAGE(PageSlab(page), page);
> -
> if (unlikely(PageCompound(page)))
> return __page_mapcount(page);
> return atomic_read(&page->_mapcount) + 1;
On Sun, May 24, 2020 at 04:43:18PM +0300, Konstantin Khlebnikov wrote:
> Replace superfluous VM_BUG_ON() with comment about correct usage.
>
> Technically reverts commit 1d148e218a0d0566b1c06f2f45f1436d53b049b2
> ("mm: add VM_BUG_ON_PAGE() to page_mapcount()"), but context have changed.
>
> Function isolate_migratepages_block() runs some checks out of lru_lock
> when choose pages for migration. After checking PageLRU() it checks extra
> page references by comparing page_count() and page_mapcount(). Between
> these two checks page could be removed from lru, freed and taken by slab.
>
> As a result this race triggers VM_BUG_ON(PageSlab()) in page_mapcount().
> Race window is tiny. For certain workload this happens around once a year.
>
>
> page:ffffea0105ca9380 count:1 mapcount:0 mapping:ffff88ff7712c180 index:0x0 compound_mapcount: 0
> flags: 0x500000000008100(slab|head)
> raw: 0500000000008100 dead000000000100 dead000000000200 ffff88ff7712c180
> raw: 0000000000000000 0000000080200020 00000001ffffffff 0000000000000000
> page dumped because: VM_BUG_ON_PAGE(PageSlab(page))
> ------------[ cut here ]------------
> kernel BUG at ./include/linux/mm.h:628!
> invalid opcode: 0000 [#1] SMP NOPTI
> CPU: 77 PID: 504 Comm: kcompactd1 Tainted: G W 4.19.109-27 #1
> Hardware name: Yandex T175-N41-Y3N/MY81-EX0-Y3N, BIOS R05 06/20/2019
> RIP: 0010:isolate_migratepages_block+0x986/0x9b0
>
>
> Code in isolate_migratepages_block() was added in commit 119d6d59dcc0
> ("mm, compaction: avoid isolating pinned pages") before adding VM_BUG_ON
> into page_mapcount().
>
> This race has been predicted in 2015 by Vlastimil Babka (see link below).
>
> Signed-off-by: Konstantin Khlebnikov <[email protected]>
> Fixes: 1d148e218a0d ("mm: add VM_BUG_ON_PAGE() to page_mapcount()")
> Link: https://lore.kernel.org/lkml/[email protected]/
> Link: https://lore.kernel.org/linux-mm/158937872515.474360.5066096871639561424.stgit@buzz/T/ (v1)
Acked-by: Kirill A. Shutemov <[email protected]>
--
Kirill A. Shutemov
On 5/24/20 3:43 PM, Konstantin Khlebnikov wrote:
> Replace superfluous VM_BUG_ON() with comment about correct usage.
>
> Technically reverts commit 1d148e218a0d0566b1c06f2f45f1436d53b049b2
> ("mm: add VM_BUG_ON_PAGE() to page_mapcount()"), but context have changed.
>
> Function isolate_migratepages_block() runs some checks out of lru_lock
> when choose pages for migration. After checking PageLRU() it checks extra
> page references by comparing page_count() and page_mapcount(). Between
> these two checks page could be removed from lru, freed and taken by slab.
>
> As a result this race triggers VM_BUG_ON(PageSlab()) in page_mapcount().
> Race window is tiny. For certain workload this happens around once a year.
>
>
> page:ffffea0105ca9380 count:1 mapcount:0 mapping:ffff88ff7712c180 index:0x0 compound_mapcount: 0
> flags: 0x500000000008100(slab|head)
> raw: 0500000000008100 dead000000000100 dead000000000200 ffff88ff7712c180
> raw: 0000000000000000 0000000080200020 00000001ffffffff 0000000000000000
> page dumped because: VM_BUG_ON_PAGE(PageSlab(page))
> ------------[ cut here ]------------
> kernel BUG at ./include/linux/mm.h:628!
> invalid opcode: 0000 [#1] SMP NOPTI
> CPU: 77 PID: 504 Comm: kcompactd1 Tainted: G W 4.19.109-27 #1
> Hardware name: Yandex T175-N41-Y3N/MY81-EX0-Y3N, BIOS R05 06/20/2019
> RIP: 0010:isolate_migratepages_block+0x986/0x9b0
>
>
> Code in isolate_migratepages_block() was added in commit 119d6d59dcc0
> ("mm, compaction: avoid isolating pinned pages") before adding VM_BUG_ON
> into page_mapcount().
>
> This race has been predicted in 2015 by Vlastimil Babka (see link below).
Huh, looks like I made that prediction only half year after that patch has been
posted. Now if only I remembered why... I hope it was just a code inspection
while chasing something else. I most likely didn't actually see the bug happen,
as we don't compile with DEBUG_VM.
> Signed-off-by: Konstantin Khlebnikov <[email protected]>
> Fixes: 1d148e218a0d ("mm: add VM_BUG_ON_PAGE() to page_mapcount()")
> Link: https://lore.kernel.org/lkml/[email protected]/
> Link: https://lore.kernel.org/linux-mm/158937872515.474360.5066096871639561424.stgit@buzz/T/ (v1)
With Hugh's wording tweaks,
Acked-by: Vlastimil Babka <[email protected]>
Thanks.