Convert more migrate functions to use a folio, it is also a preparation
for large folio migration support when numa balance.
The patch 1~2 remove unexpected specific assert for PageTransHuge page.
The Patch 3~6 convert several more migration functions to use folio.
1) add_page_for_migration() used by move_pages
2) migrate_misplaced_page()/numamigrate_isolate_page()
used by numa balance
The patch 7 remove PageHead() check to make hugetlb to migrate the
entire hugetlb page instead of -EACCES errno return when
passed the address of a tail page.
The patch 8 cleanup to unify and simplify code a bit in
add_page_for_migration()
Thanks for all comments and suggestions from Matthew, Hugh, Zi, Mike, Huang.
v3:
- update changelog of patch1
- use folio_estimated_sharers and comment it in migrate_misplaced_folio()
- collect RB/ACK
- rebased on 6.6-rc1
v2:
- keep page_mapcount() check and remove specific assert for
PageTransHuge page.
- separate patch7 to migrate the entire hugetlb page if a tail page passed,
which unified the behavior between HUGETLB and THP when move_page().
Kefeng Wang (8):
mm: migrate: remove PageTransHuge check in numamigrate_isolate_page()
mm: migrate: remove THP mapcount check in numamigrate_isolate_page()
mm: migrate: convert numamigrate_isolate_page() to
numamigrate_isolate_folio()
mm: migrate: convert migrate_misplaced_page() to
migrate_misplaced_folio()
mm: migrate: use __folio_test_movable()
mm: migrate: use a folio in add_page_for_migration()
mm: migrate: remove PageHead() check for HugeTLB in
add_page_for_migration()
mm: migrate: remove isolated variable in add_page_for_migration()
include/linux/migrate.h | 4 +-
mm/huge_memory.c | 2 +-
mm/memory.c | 2 +-
mm/migrate.c | 126 ++++++++++++++++++----------------------
4 files changed, 62 insertions(+), 72 deletions(-)
--
2.27.0
There is some different between hugeTLB and THP behave when passed the
address of a tail page, for THP, it will migrate the entire THP page,
but for HugeTLB, it will return -EACCES, or -ENOENT before commit
e66f17ff7177 ("mm/hugetlb: take page table lock in follow_huge_pmd()"),
-EACCES The page is mapped by multiple processes and can be moved
only if MPOL_MF_MOVE_ALL is specified.
-ENOENT The page is not present.
But when check manual[1], both of the two errnos are not suitable, it
is better to keep the same behave between hugetlb and THP when passed
the address of a tail page, so let's just remove the PageHead() check
for HugeTLB.
[1] https://man7.org/linux/man-pages/man2/move_pages.2.html
Suggested-by: Mike Kravetz <[email protected]>
Acked-by: Zi Yan <[email protected]>
Signed-off-by: Kefeng Wang <[email protected]>
---
mm/migrate.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/mm/migrate.c b/mm/migrate.c
index cf5c9254fdad..7b07c97f5a6f 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -2093,10 +2093,8 @@ static int add_page_for_migration(struct mm_struct *mm, const void __user *p,
goto out_putfolio;
if (folio_test_hugetlb(folio)) {
- if (PageHead(page)) {
- isolated = isolate_hugetlb(folio, pagelist);
- err = isolated ? 1 : -EBUSY;
- }
+ isolated = isolate_hugetlb(folio, pagelist);
+ err = isolated ? 1 : -EBUSY;
} else {
isolated = folio_isolate_lru(folio);
if (!isolated) {
--
2.27.0
The assert VM_BUG_ON_PAGE(order && !PageTransHuge(page), page) is
not very useful,
1) for a tail/base page, order = 0, for a head page, the order > 0 &&
PageTransHuge() is true
2) there is a PageCompound() check and only base page is handled in
do_numa_page(), and do_huge_pmd_numa_page() only handle PMD-mapped
THP
3) even though the page is a tail page, isolate_lru_page() will post
a warning, and fail to isolate the page
4) if large folio/pte-mapped THP migration supported in the future,
we could migrate the entire folio if numa fault on a tail page
so just remove the check.
Suggested-by: Matthew Wilcox (Oracle) <[email protected]>
Signed-off-by: Kefeng Wang <[email protected]>
---
mm/migrate.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/mm/migrate.c b/mm/migrate.c
index b7fa020003f3..646d8ee7f102 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -2483,8 +2483,6 @@ static int numamigrate_isolate_page(pg_data_t *pgdat, struct page *page)
int nr_pages = thp_nr_pages(page);
int order = compound_order(page);
- VM_BUG_ON_PAGE(order && !PageTransHuge(page), page);
-
/* Do not migrate THP mapped by multiple processes */
if (PageTransHuge(page) && total_mapcount(page) > 1)
return 0;
--
2.27.0
Use __folio_test_movable(), no need to convert from folio to page again.
Reviewed-by: Matthew Wilcox (Oracle) <[email protected]>
Reviewed-by: David Hildenbrand <[email protected]>
Reviewed-by: Zi Yan <[email protected]>
Signed-off-by: Kefeng Wang <[email protected]>
---
mm/migrate.c | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/mm/migrate.c b/mm/migrate.c
index caf60b58b44c..264923aac04e 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -157,8 +157,8 @@ void putback_movable_pages(struct list_head *l)
list_del(&folio->lru);
/*
* We isolated non-lru movable folio so here we can use
- * __PageMovable because LRU folio's mapping cannot have
- * PAGE_MAPPING_MOVABLE.
+ * __folio_test_movable because LRU folio's mapping cannot
+ * have PAGE_MAPPING_MOVABLE.
*/
if (unlikely(__folio_test_movable(folio))) {
VM_BUG_ON_FOLIO(!folio_test_isolated(folio), folio);
@@ -943,7 +943,7 @@ static int move_to_new_folio(struct folio *dst, struct folio *src,
enum migrate_mode mode)
{
int rc = -EAGAIN;
- bool is_lru = !__PageMovable(&src->page);
+ bool is_lru = !__folio_test_movable(src);
VM_BUG_ON_FOLIO(!folio_test_locked(src), src);
VM_BUG_ON_FOLIO(!folio_test_locked(dst), dst);
@@ -990,7 +990,7 @@ static int move_to_new_folio(struct folio *dst, struct folio *src,
* src is freed; but stats require that PageAnon be left as PageAnon.
*/
if (rc == MIGRATEPAGE_SUCCESS) {
- if (__PageMovable(&src->page)) {
+ if (__folio_test_movable(src)) {
VM_BUG_ON_FOLIO(!folio_test_isolated(src), src);
/*
@@ -1082,7 +1082,7 @@ static void migrate_folio_done(struct folio *src,
/*
* Compaction can migrate also non-LRU pages which are
* not accounted to NR_ISOLATED_*. They can be recognized
- * as __PageMovable
+ * as __folio_test_movable
*/
if (likely(!__folio_test_movable(src)))
mod_node_page_state(folio_pgdat(src), NR_ISOLATED_ANON +
@@ -1103,7 +1103,7 @@ static int migrate_folio_unmap(new_folio_t get_new_folio,
int rc = -EAGAIN;
int page_was_mapped = 0;
struct anon_vma *anon_vma = NULL;
- bool is_lru = !__PageMovable(&src->page);
+ bool is_lru = !__folio_test_movable(src);
bool locked = false;
bool dst_locked = false;
@@ -1261,7 +1261,7 @@ static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,
int rc;
int page_was_mapped = 0;
struct anon_vma *anon_vma = NULL;
- bool is_lru = !__PageMovable(&src->page);
+ bool is_lru = !__folio_test_movable(src);
struct list_head *prev;
__migrate_folio_extract(dst, &page_was_mapped, &anon_vma);
--
2.27.0