Hi everyone,
This series contains a few fixup patches to fix inaccurate mf_stats, fix
race window when trying to get hugetlb folio and so on. Also there is
minor cleanup for comments and codestyle. More details can be found in
the respective changelogs.
Thanks!
---
v2
collect Acked-by tag per Naoya
1/8: Change to simply remove the dead code per Naoya
8/8: Add some short comment about the race against demotion per Naoya
Thanks Naoya for review.
---
Miaohe Lin (8):
mm: memory-failure: remove unneeded PageHuge() check
mm: memory-failure: ensure moving HWPoison flag to the raw error pages
mm: memory-failure: Don't account hwpoison_filter() filtered pages
mm: memory-failure: use local variable huge to check hugetlb page
mm: memory-failure: remove unneeded header files
mm: memory-failure: minor cleanup for comments and codestyle
mm: memory-failure: fetch compound head after extra page refcnt is
held
mm: memory-failure: fix race window when trying to get hugetlb folio
mm/memory-failure.c | 50 ++++++++++++++++++++++++++-------------------
1 file changed, 29 insertions(+), 21 deletions(-)
--
2.33.0
Use local variable huge to check whether page is hugetlb page to avoid
calling PageHuge() multiple times to save cpu cycles. PageHuge() will
be stable while extra page refcnt is held.
Signed-off-by: Miaohe Lin <[email protected]>
Acked-by: Naoya Horiguchi <[email protected]>
---
mm/memory-failure.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index c80b7d9505d8..239e0711f832 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -2628,7 +2628,7 @@ static int soft_offline_in_use_page(struct page *page)
}
lock_page(page);
- if (!PageHuge(page))
+ if (!huge)
wait_on_page_writeback(page);
if (PageHWPoison(page)) {
unlock_page(page);
@@ -2637,7 +2637,7 @@ static int soft_offline_in_use_page(struct page *page)
return 0;
}
- if (!PageHuge(page) && PageLRU(page) && !PageSwapCache(page))
+ if (!huge && PageLRU(page) && !PageSwapCache(page))
/*
* Try to invalidate first. This should work for
* non dirty unmapped page cache pages.
--
2.33.0
PageHuge() check in me_huge_page() is just for potential problems. Remove
it as it's actually dead code and won't catch anything.
Signed-off-by: Miaohe Lin <[email protected]>
---
mm/memory-failure.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 64d7d34c177a..913fcf02ad38 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -1187,9 +1187,6 @@ static int me_huge_page(struct page_state *ps, struct page *p)
struct address_space *mapping;
bool extra_pins = false;
- if (!PageHuge(hpage))
- return MF_DELAYED;
-
mapping = page_mapping(hpage);
if (mapping) {
res = truncate_error_page(hpage, page_to_pfn(p), mapping);
--
2.33.0
Fix some wrong function names and grammar error in comments. Also remove
unneeded space after for_each_process. No functional change intended.
Signed-off-by: Miaohe Lin <[email protected]>
Acked-by: Naoya Horiguchi <[email protected]>
---
mm/memory-failure.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 0da6ddbdd718..db4c530944d6 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -608,7 +608,7 @@ static void collect_procs_anon(struct page *page, struct list_head *to_kill,
pgoff = page_to_pgoff(page);
read_lock(&tasklist_lock);
- for_each_process (tsk) {
+ for_each_process(tsk) {
struct anon_vma_chain *vmac;
struct task_struct *t = task_early_kill(tsk, force_early);
@@ -652,7 +652,7 @@ static void collect_procs_file(struct page *page, struct list_head *to_kill,
/*
* Send early kill signal to tasks where a vma covers
* the page but the corrupted page is not necessarily
- * mapped it in its pte.
+ * mapped in its pte.
* Assume applications who requested early kill want
* to be informed of all such data corruptions.
*/
@@ -2127,7 +2127,7 @@ static DEFINE_MUTEX(mf_mutex);
* detected by a background scrubber)
*
* Must run in process context (e.g. a work queue) with interrupts
- * enabled and no spinlocks hold.
+ * enabled and no spinlocks held.
*
* Return: 0 for successfully handled the memory error,
* -EOPNOTSUPP for hwpoison_filter() filtered the error event,
@@ -2232,7 +2232,7 @@ int memory_failure(unsigned long pfn, int flags)
* otherwise it may race with THP split.
* And the flag can't be set in get_hwpoison_page() since
* it is called by soft offline too and it is just called
- * for !MF_COUNT_INCREASE. So here seems to be the best
+ * for !MF_COUNT_INCREASED. So here seems to be the best
* place.
*
* Don't need care about the above error handling paths for
@@ -2589,10 +2589,10 @@ static bool isolate_page(struct page *page, struct list_head *pagelist)
/*
* If we succeed to isolate the page, we grabbed another refcount on
- * the page, so we can safely drop the one we got from get_any_pages().
+ * the page, so we can safely drop the one we got from get_any_page().
* If we failed to isolate the page, it means that we cannot go further
* and we will return an error, so drop the reference we got from
- * get_any_pages() as well.
+ * get_any_page() as well.
*/
put_page(page);
return isolated;
--
2.33.0
Remove some unneeded header files. No functional change intended.
Signed-off-by: Miaohe Lin <[email protected]>
Acked-by: Naoya Horiguchi <[email protected]>
---
mm/memory-failure.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 239e0711f832..0da6ddbdd718 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -39,7 +39,6 @@
#include <linux/kernel.h>
#include <linux/mm.h>
#include <linux/page-flags.h>
-#include <linux/kernel-page-flags.h>
#include <linux/sched/signal.h>
#include <linux/sched/task.h>
#include <linux/dax.h>
@@ -50,7 +49,6 @@
#include <linux/swap.h>
#include <linux/backing-dev.h>
#include <linux/migrate.h>
-#include <linux/suspend.h>
#include <linux/slab.h>
#include <linux/swapops.h>
#include <linux/hugetlb.h>
@@ -59,7 +57,6 @@
#include <linux/memremap.h>
#include <linux/kfifo.h>
#include <linux/ratelimit.h>
-#include <linux/page-isolation.h>
#include <linux/pagewalk.h>
#include <linux/shmem_fs.h>
#include <linux/sysctl.h>
--
2.33.0
page_folio() is fetched before calling get_hwpoison_hugetlb_folio()
without hugetlb_lock being held. So hugetlb page could be demoted
before get_hwpoison_hugetlb_folio() holding hugetlb_lock but after
page_folio() is fetched. So get_hwpoison_hugetlb_folio() will hold
unexpected extra refcnt of hugetlb folio while leaving demoted page
un-refcnted.
Fixes: 25182f05ffed ("mm,hwpoison: fix race with hugetlb page allocation")
Signed-off-by: Miaohe Lin <[email protected]>
Acked-by: Naoya Horiguchi <[email protected]>
---
mm/memory-failure.c | 20 ++++++++++++++++----
1 file changed, 16 insertions(+), 4 deletions(-)
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index af34fd4669d3..9ab97016877e 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -1383,8 +1383,15 @@ static int __get_hwpoison_page(struct page *page, unsigned long flags)
bool hugetlb = false;
ret = get_hwpoison_hugetlb_folio(folio, &hugetlb, false);
- if (hugetlb)
- return ret;
+ if (hugetlb) {
+ /* Make sure hugetlb demotion did not happen from under us. */
+ if (folio == page_folio(page))
+ return ret;
+ if (ret > 0) {
+ folio_put(folio);
+ folio = page_folio(page);
+ }
+ }
/*
* This check prevents from calling folio_try_get() for any
@@ -1473,8 +1480,13 @@ static int __get_unpoison_page(struct page *page)
bool hugetlb = false;
ret = get_hwpoison_hugetlb_folio(folio, &hugetlb, true);
- if (hugetlb)
- return ret;
+ if (hugetlb) {
+ /* Make sure hugetlb demotion did not happen from under us. */
+ if (folio == page_folio(page))
+ return ret;
+ if (ret > 0)
+ folio_put(folio);
+ }
/*
* PageHWPoisonTakenOff pages are not only marked as PG_hwpoison,
--
2.33.0
Page might become thp, huge page or being splited after compound head
is fetched but before page refcnt is bumped. So hpage might be a tail
page leading to VM_BUG_ON_PAGE(PageTail(page)) in PageTransHuge().
Fixes: 415c64c1453a ("mm/memory-failure: split thp earlier in memory error handling")
Signed-off-by: Miaohe Lin <[email protected]>
Acked-by: Naoya Horiguchi <[email protected]>
---
mm/memory-failure.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index db4c530944d6..af34fd4669d3 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -2186,8 +2186,6 @@ int memory_failure(unsigned long pfn, int flags)
goto unlock_mutex;
}
- hpage = compound_head(p);
-
/*
* We need/can do nothing about count=0 pages.
* 1) it's a free page, and therefore in safe hand:
@@ -2226,6 +2224,7 @@ int memory_failure(unsigned long pfn, int flags)
}
}
+ hpage = compound_head(p);
if (PageTransHuge(hpage)) {
/*
* The flag must be set after the refcount is bumped
--
2.33.0
If hugetlb_vmemmap_optimized is enabled, folio_clear_hugetlb_hwpoison()
called from try_memory_failure_hugetlb() won't transfer HWPoison flag to
subpages while folio's HWPoison flag is cleared. So when trying to free
this hugetlb page into buddy, folio_clear_hugetlb_hwpoison() is not called
to move HWPoison flag from head page to the raw error pages even if now
hugetlb_vmemmap_optimized is cleared. This will results in HWPoisoned page
being used again and raw_hwp_page leak.
Fixes: ac5fcde0a96a ("mm, hwpoison: make unpoison aware of raw error info in hwpoisoned hugepage")
Signed-off-by: Miaohe Lin <[email protected]>
Acked-by: Naoya Horiguchi <[email protected]>
---
mm/memory-failure.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 913fcf02ad38..feca3c4068a5 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -1920,6 +1920,8 @@ void folio_clear_hugetlb_hwpoison(struct folio *folio)
{
if (folio_test_hugetlb_raw_hwp_unreliable(folio))
return;
+ if (folio_test_hugetlb_vmemmap_optimized(folio))
+ return;
folio_clear_hwpoison(folio);
folio_free_raw_hwp(folio, true);
}
--
2.33.0
On Tue, Jul 11, 2023 at 01:50:09PM +0800, Miaohe Lin wrote:
> PageHuge() check in me_huge_page() is just for potential problems. Remove
> it as it's actually dead code and won't catch anything.
>
> Signed-off-by: Miaohe Lin <[email protected]>
Thank you for updating.
Acked-by: Naoya Horiguchi <[email protected]>
> ---
> mm/memory-failure.c | 3 ---
> 1 file changed, 3 deletions(-)
>
> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
> index 64d7d34c177a..913fcf02ad38 100644
> --- a/mm/memory-failure.c
> +++ b/mm/memory-failure.c
> @@ -1187,9 +1187,6 @@ static int me_huge_page(struct page_state *ps, struct page *p)
> struct address_space *mapping;
> bool extra_pins = false;
>
> - if (!PageHuge(hpage))
> - return MF_DELAYED;
> -
> mapping = page_mapping(hpage);
> if (mapping) {
> res = truncate_error_page(hpage, page_to_pfn(p), mapping);
> --
> 2.33.0
>
>
>