Many thanks to Kirill for reminding that PageDoubleMap cannot be relied on
to warn of pte mappings in the Anon THP case; and a scan of subpages does
not seem appropriate here. Note how follow_trans_huge_pmd() does not even
mark an Anon THP as mlocked when compound_mapcount != 1: multiple mlocking
of Anon THP is avoided, so simply return from page_mlock() in this case.
Link: https://lore.kernel.org/lkml/[email protected]/
Fixes: d9770fcc1c0c ("mm/rmap: fix old bug: munlocking THP missed other mlocks")
Reported-by: Kirill A. Shutemov <[email protected]>
Signed-off-by: Hugh Dickins <[email protected]>
Acked-by: Kirill A. Shutemov <[email protected]>
Cc: Andrew Morton <[email protected]>
Cc: Alistair Popple <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Ralph Campbell <[email protected]>
Cc: Christoph Hellwig <[email protected]>
Cc: Yang Shi <[email protected]>
Cc: Shakeel Butt <[email protected]>
---
Linus, thanks a lot for last-minute hoovering up those four mm/rmap
patches, with 3/4 fixing the syzbot and 0day reports on munlocking ...
BUT
... the version of 2/4 in 5.14-rc1 is defective (PageDoubleMap is a
confusing flag which behaves differently on anon and file), Kirill had
spotted that, and what he Acked was the v2 which went into mmotm, rather
than the first version I posted. This patch here converts the v1 in rc1
into the v2 Acked by Kirill.
What will go wrong with v1 in? I don't actually know: nothing terrible,
can only affect people splitting and mlocking anon THPs, maybe nobody and
nobot will notice, maybe some VM_BUG_ONs or "Bad page"s will turn up.
I'll be on the lookout to point reporters to this fix (more lines than
strictly necessary, because it removes a level of indentation).
And sorry for putting 2/4 before the more urgent 3/4, but I couldn't
tell what to do in 3/4, without first fixing the older bug in 2/4.
Hugh
mm/rmap.c | 39 ++++++++++++++++++++++-----------------
1 file changed, 22 insertions(+), 17 deletions(-)
--- 5.14-rc1/mm/rmap.c
+++ linux/mm/rmap.c
@@ -1440,21 +1440,20 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma,
/*
* If the page is mlock()d, we cannot swap it out.
*/
- if (!(flags & TTU_IGNORE_MLOCK)) {
- if (vma->vm_flags & VM_LOCKED) {
- /* PTE-mapped THP are never marked as mlocked */
- if (!PageTransCompound(page) ||
- (PageHead(page) && !PageDoubleMap(page))) {
- /*
- * Holding pte lock, we do *not* need
- * mmap_lock here
- */
- mlock_vma_page(page);
- }
- ret = false;
- page_vma_mapped_walk_done(&pvmw);
- break;
- }
+ if (!(flags & TTU_IGNORE_MLOCK) &&
+ (vma->vm_flags & VM_LOCKED)) {
+ /*
+ * PTE-mapped THP are never marked as mlocked: so do
+ * not set it on a DoubleMap THP, nor on an Anon THP
+ * (which may still be PTE-mapped after DoubleMap was
+ * cleared). But stop unmapping even in those cases.
+ */
+ if (!PageTransCompound(page) || (PageHead(page) &&
+ !PageDoubleMap(page) && !PageAnon(page)))
+ mlock_vma_page(page);
+ page_vma_mapped_walk_done(&pvmw);
+ ret = false;
+ break;
}
/* Unexpected PMD-mapped THP? */
@@ -1986,8 +1985,10 @@ static bool page_mlock_one(struct page *page, struct vm_area_struct *vma,
*/
if (vma->vm_flags & VM_LOCKED) {
/*
- * PTE-mapped THP are never marked as mlocked, but
- * this function is never called when PageDoubleMap().
+ * PTE-mapped THP are never marked as mlocked; but
+ * this function is never called on a DoubleMap THP,
+ * nor on an Anon THP (which may still be PTE-mapped
+ * after DoubleMap was cleared).
*/
mlock_vma_page(page);
/*
@@ -2022,6 +2023,10 @@ void page_mlock(struct page *page)
VM_BUG_ON_PAGE(!PageLocked(page) || PageLRU(page), page);
VM_BUG_ON_PAGE(PageCompound(page) && PageDoubleMap(page), page);
+ /* Anon THP are only marked as mlocked when singly mapped */
+ if (PageTransCompound(page) && PageAnon(page))
+ return;
+
rmap_walk(page, &rwc);
}