Hi everyone,
This series contains a few fixup patches to fix potential permanently
locked hpage, fix race window when trying to get hugetlb folio and so
on. Also there is minor cleanup for comments and codestyle. More details
can be found in the respective changelogs.
Thanks!
Miaohe Lin (8):
mm: memory-failure: fix potential permanently locked hpage
mm: memory-failure: ensure moving HWPoison flag to the raw error pages
mm: memory-failure: Don't account hwpoison_filter() filtered pages
mm: memory-failure: use local variable huge to check hugetlb page
mm: memory-failure: remove unneeded header files
mm: memory-failure: minor cleanup for comments and codestyle
mm: memory-failure: fetch compound head after extra page refcnt is
held
mm: memory-failure: fix race window when trying to get hugetlb folio
mm/memory-failure.c | 49 +++++++++++++++++++++++++++------------------
1 file changed, 30 insertions(+), 19 deletions(-)
--
2.33.0
Remove some unneeded header files. No functional change intended.
Signed-off-by: Miaohe Lin <[email protected]>
---
mm/memory-failure.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 88e48a4801ee..601936f8d30b 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -39,7 +39,6 @@
#include <linux/kernel.h>
#include <linux/mm.h>
#include <linux/page-flags.h>
-#include <linux/kernel-page-flags.h>
#include <linux/sched/signal.h>
#include <linux/sched/task.h>
#include <linux/dax.h>
@@ -50,7 +49,6 @@
#include <linux/swap.h>
#include <linux/backing-dev.h>
#include <linux/migrate.h>
-#include <linux/suspend.h>
#include <linux/slab.h>
#include <linux/swapops.h>
#include <linux/hugetlb.h>
@@ -59,7 +57,6 @@
#include <linux/memremap.h>
#include <linux/kfifo.h>
#include <linux/ratelimit.h>
-#include <linux/page-isolation.h>
#include <linux/pagewalk.h>
#include <linux/shmem_fs.h>
#include <linux/sysctl.h>
--
2.33.0
If hugetlb_vmemmap_optimized is enabled, folio_clear_hugetlb_hwpoison()
called from try_memory_failure_hugetlb() won't transfer HWPoison flag to
subpages while folio's HWPoison flag is cleared. So when trying to free
this hugetlb page into buddy, folio_clear_hugetlb_hwpoison() is not called
to move HWPoison flag from head page to the raw error pages even if now
hugetlb_vmemmap_optimized is cleared. This will results in HWPoisoned page
being used again and raw_hwp_page leak.
Fixes: ac5fcde0a96a ("mm, hwpoison: make unpoison aware of raw error info in hwpoisoned hugepage")
Signed-off-by: Miaohe Lin <[email protected]>
---
mm/memory-failure.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index d21ee27ad412..c155122e3c66 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -1913,6 +1913,8 @@ void folio_clear_hugetlb_hwpoison(struct folio *folio)
{
if (folio_test_hugetlb_raw_hwp_unreliable(folio))
return;
+ if (folio_test_hugetlb_vmemmap_optimized(folio))
+ return;
folio_clear_hwpoison(folio);
folio_free_raw_hwp(folio, true);
}
--
2.33.0
page_folio() is fetched before calling get_hwpoison_hugetlb_folio()
without hugetlb_lock being held. So hugetlb page could be demoted
before get_hwpoison_hugetlb_folio() holding hugetlb_lock but after
page_folio() is fetched. So get_hwpoison_hugetlb_folio() will hold
unexpected extra refcnt of hugetlb folio while leaving demoted page
un-refcnted.
Fixes: 25182f05ffed ("mm,hwpoison: fix race with hugetlb page allocation")
Signed-off-by: Miaohe Lin <[email protected]>
---
mm/memory-failure.c | 18 ++++++++++++++----
1 file changed, 14 insertions(+), 4 deletions(-)
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 76d88d27cdbe..066bf57f2d22 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -1388,8 +1388,14 @@ static int __get_hwpoison_page(struct page *page, unsigned long flags)
bool hugetlb = false;
ret = get_hwpoison_hugetlb_folio(folio, &hugetlb, false);
- if (hugetlb)
- return ret;
+ if (hugetlb) {
+ if (folio == page_folio(page))
+ return ret;
+ if (ret > 0) {
+ folio_put(folio);
+ folio = page_folio(page);
+ }
+ }
/*
* This check prevents from calling folio_try_get() for any
@@ -1478,8 +1484,12 @@ static int __get_unpoison_page(struct page *page)
bool hugetlb = false;
ret = get_hwpoison_hugetlb_folio(folio, &hugetlb, true);
- if (hugetlb)
- return ret;
+ if (hugetlb) {
+ if (folio == page_folio(page))
+ return ret;
+ if (ret > 0)
+ folio_put(folio);
+ }
/*
* PageHWPoisonTakenOff pages are not only marked as PG_hwpoison,
--
2.33.0
Fix some wrong function names and grammar error in comments. Also remove
unneeded space after for_each_process. No functional change intended.
Signed-off-by: Miaohe Lin <[email protected]>
---
mm/memory-failure.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 601936f8d30b..0f93175ed862 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -608,7 +608,7 @@ static void collect_procs_anon(struct page *page, struct list_head *to_kill,
pgoff = page_to_pgoff(page);
read_lock(&tasklist_lock);
- for_each_process (tsk) {
+ for_each_process(tsk) {
struct anon_vma_chain *vmac;
struct task_struct *t = task_early_kill(tsk, force_early);
@@ -652,7 +652,7 @@ static void collect_procs_file(struct page *page, struct list_head *to_kill,
/*
* Send early kill signal to tasks where a vma covers
* the page but the corrupted page is not necessarily
- * mapped it in its pte.
+ * mapped in its pte.
* Assume applications who requested early kill want
* to be informed of all such data corruptions.
*/
@@ -2120,7 +2120,7 @@ static DEFINE_MUTEX(mf_mutex);
* detected by a background scrubber)
*
* Must run in process context (e.g. a work queue) with interrupts
- * enabled and no spinlocks hold.
+ * enabled and no spinlocks held.
*
* Return: 0 for successfully handled the memory error,
* -EOPNOTSUPP for hwpoison_filter() filtered the error event,
@@ -2225,7 +2225,7 @@ int memory_failure(unsigned long pfn, int flags)
* otherwise it may race with THP split.
* And the flag can't be set in get_hwpoison_page() since
* it is called by soft offline too and it is just called
- * for !MF_COUNT_INCREASE. So here seems to be the best
+ * for !MF_COUNT_INCREASED. So here seems to be the best
* place.
*
* Don't need care about the above error handling paths for
@@ -2582,10 +2582,10 @@ static bool isolate_page(struct page *page, struct list_head *pagelist)
/*
* If we succeed to isolate the page, we grabbed another refcount on
- * the page, so we can safely drop the one we got from get_any_pages().
+ * the page, so we can safely drop the one we got from get_any_page().
* If we failed to isolate the page, it means that we cannot go further
* and we will return an error, so drop the reference we got from
- * get_any_pages() as well.
+ * get_any_page() as well.
*/
put_page(page);
return isolated;
--
2.33.0
On Sat, Jul 08, 2023 at 04:57:38PM +0800, Miaohe Lin wrote:
> If hugetlb_vmemmap_optimized is enabled, folio_clear_hugetlb_hwpoison()
> called from try_memory_failure_hugetlb() won't transfer HWPoison flag to
> subpages while folio's HWPoison flag is cleared. So when trying to free
> this hugetlb page into buddy, folio_clear_hugetlb_hwpoison() is not called
> to move HWPoison flag from head page to the raw error pages even if now
> hugetlb_vmemmap_optimized is cleared. This will results in HWPoisoned page
> being used again and raw_hwp_page leak.
>
> Fixes: ac5fcde0a96a ("mm, hwpoison: make unpoison aware of raw error info in hwpoisoned hugepage")
> Signed-off-by: Miaohe Lin <[email protected]>
Acked-by: Naoya Horiguchi <[email protected]>
On Sat, Jul 08, 2023 at 04:57:44PM +0800, Miaohe Lin wrote:
> page_folio() is fetched before calling get_hwpoison_hugetlb_folio()
> without hugetlb_lock being held. So hugetlb page could be demoted
> before get_hwpoison_hugetlb_folio() holding hugetlb_lock but after
> page_folio() is fetched. So get_hwpoison_hugetlb_folio() will hold
> unexpected extra refcnt of hugetlb folio while leaving demoted page
> un-refcnted.
Very nice, thank you for finding the issue.
>
> Fixes: 25182f05ffed ("mm,hwpoison: fix race with hugetlb page allocation")
> Signed-off-by: Miaohe Lin <[email protected]>
> ---
> mm/memory-failure.c | 18 ++++++++++++++----
> 1 file changed, 14 insertions(+), 4 deletions(-)
>
> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
> index 76d88d27cdbe..066bf57f2d22 100644
> --- a/mm/memory-failure.c
> +++ b/mm/memory-failure.c
> @@ -1388,8 +1388,14 @@ static int __get_hwpoison_page(struct page *page, unsigned long flags)
> bool hugetlb = false;
>
> ret = get_hwpoison_hugetlb_folio(folio, &hugetlb, false);
> - if (hugetlb)
> - return ret;
> + if (hugetlb) {
> + if (folio == page_folio(page))
> + return ret;
Some short comment about the race against demotion here is helpful.
Anyway, the patch looks good to me.
Acked-by: Naoya Horiguchi <[email protected]>
> + if (ret > 0) {
> + folio_put(folio);
> + folio = page_folio(page);
> + }
> + }
>
> /*
> * This check prevents from calling folio_try_get() for any
> @@ -1478,8 +1484,12 @@ static int __get_unpoison_page(struct page *page)
> bool hugetlb = false;
>
> ret = get_hwpoison_hugetlb_folio(folio, &hugetlb, true);
> - if (hugetlb)
> - return ret;
> + if (hugetlb) {
> + if (folio == page_folio(page))
> + return ret;
> + if (ret > 0)
> + folio_put(folio);
> + }
>
> /*
> * PageHWPoisonTakenOff pages are not only marked as PG_hwpoison,
> --
> 2.33.0
>
>
>
On Sat, Jul 08, 2023 at 04:57:41PM +0800, Miaohe Lin wrote:
> Remove some unneeded header files. No functional change intended.
>
> Signed-off-by: Miaohe Lin <[email protected]>
Acked-by: Naoya Horiguchi <[email protected]>
On Sat, Jul 08, 2023 at 04:57:42PM +0800, Miaohe Lin wrote:
> Fix some wrong function names and grammar error in comments. Also remove
> unneeded space after for_each_process. No functional change intended.
>
> Signed-off-by: Miaohe Lin <[email protected]>
Acked-by: Naoya Horiguchi <[email protected]>
On Mon, Jul 10, 2023 at 04:32:27PM +0800, Miaohe Lin wrote:
> On 2023/7/10 15:58, Naoya Horiguchi wrote:
> > On Sat, Jul 08, 2023 at 04:57:44PM +0800, Miaohe Lin wrote:
> >> page_folio() is fetched before calling get_hwpoison_hugetlb_folio()
> >> without hugetlb_lock being held. So hugetlb page could be demoted
> >> before get_hwpoison_hugetlb_folio() holding hugetlb_lock but after
> >> page_folio() is fetched. So get_hwpoison_hugetlb_folio() will hold
> >> unexpected extra refcnt of hugetlb folio while leaving demoted page
> >> un-refcnted.
> >
> > Very nice, thank you for finding the issue.
> >
> >>
> >> Fixes: 25182f05ffed ("mm,hwpoison: fix race with hugetlb page allocation")
> >> Signed-off-by: Miaohe Lin <[email protected]>
> >> ---
> >> mm/memory-failure.c | 18 ++++++++++++++----
> >> 1 file changed, 14 insertions(+), 4 deletions(-)
> >>
> >> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
> >> index 76d88d27cdbe..066bf57f2d22 100644
> >> --- a/mm/memory-failure.c
> >> +++ b/mm/memory-failure.c
> >> @@ -1388,8 +1388,14 @@ static int __get_hwpoison_page(struct page *page, unsigned long flags)
> >> bool hugetlb = false;
> >>
> >> ret = get_hwpoison_hugetlb_folio(folio, &hugetlb, false);
> >> - if (hugetlb)
> >> - return ret;
> >> + if (hugetlb) {
> >> + if (folio == page_folio(page))
> >> + return ret;
> >
> > Some short comment about the race against demotion here is helpful.
>
> Does the below comment makes sense to you?
>
> "
> Make sure hugetlb demotion did not happen from under us.
> "
Yes, this sounds fine.
Thanks,
Naoya Horiguchi
On 2023/7/10 15:58, Naoya Horiguchi wrote:
> On Sat, Jul 08, 2023 at 04:57:44PM +0800, Miaohe Lin wrote:
>> page_folio() is fetched before calling get_hwpoison_hugetlb_folio()
>> without hugetlb_lock being held. So hugetlb page could be demoted
>> before get_hwpoison_hugetlb_folio() holding hugetlb_lock but after
>> page_folio() is fetched. So get_hwpoison_hugetlb_folio() will hold
>> unexpected extra refcnt of hugetlb folio while leaving demoted page
>> un-refcnted.
>
> Very nice, thank you for finding the issue.
>
>>
>> Fixes: 25182f05ffed ("mm,hwpoison: fix race with hugetlb page allocation")
>> Signed-off-by: Miaohe Lin <[email protected]>
>> ---
>> mm/memory-failure.c | 18 ++++++++++++++----
>> 1 file changed, 14 insertions(+), 4 deletions(-)
>>
>> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
>> index 76d88d27cdbe..066bf57f2d22 100644
>> --- a/mm/memory-failure.c
>> +++ b/mm/memory-failure.c
>> @@ -1388,8 +1388,14 @@ static int __get_hwpoison_page(struct page *page, unsigned long flags)
>> bool hugetlb = false;
>>
>> ret = get_hwpoison_hugetlb_folio(folio, &hugetlb, false);
>> - if (hugetlb)
>> - return ret;
>> + if (hugetlb) {
>> + if (folio == page_folio(page))
>> + return ret;
>
> Some short comment about the race against demotion here is helpful.
Does the below comment makes sense to you?
"
Make sure hugetlb demotion did not happen from under us.
"
>
> Anyway, the patch looks good to me.
>
> Acked-by: Naoya Horiguchi <[email protected]>
Many thanks for your review and comment, Naoya.
On 2023/7/10 16:39, Naoya Horiguchi wrote:
> On Mon, Jul 10, 2023 at 04:32:27PM +0800, Miaohe Lin wrote:
>> On 2023/7/10 15:58, Naoya Horiguchi wrote:
>>> On Sat, Jul 08, 2023 at 04:57:44PM +0800, Miaohe Lin wrote:
>>>> page_folio() is fetched before calling get_hwpoison_hugetlb_folio()
>>>> without hugetlb_lock being held. So hugetlb page could be demoted
>>>> before get_hwpoison_hugetlb_folio() holding hugetlb_lock but after
>>>> page_folio() is fetched. So get_hwpoison_hugetlb_folio() will hold
>>>> unexpected extra refcnt of hugetlb folio while leaving demoted page
>>>> un-refcnted.
>>>
>>> Very nice, thank you for finding the issue.
>>>
>>>>
>>>> Fixes: 25182f05ffed ("mm,hwpoison: fix race with hugetlb page allocation")
>>>> Signed-off-by: Miaohe Lin <[email protected]>
>>>> ---
>>>> mm/memory-failure.c | 18 ++++++++++++++----
>>>> 1 file changed, 14 insertions(+), 4 deletions(-)
>>>>
>>>> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
>>>> index 76d88d27cdbe..066bf57f2d22 100644
>>>> --- a/mm/memory-failure.c
>>>> +++ b/mm/memory-failure.c
>>>> @@ -1388,8 +1388,14 @@ static int __get_hwpoison_page(struct page *page, unsigned long flags)
>>>> bool hugetlb = false;
>>>>
>>>> ret = get_hwpoison_hugetlb_folio(folio, &hugetlb, false);
>>>> - if (hugetlb)
>>>> - return ret;
>>>> + if (hugetlb) {
>>>> + if (folio == page_folio(page))
>>>> + return ret;
>>>
>>> Some short comment about the race against demotion here is helpful.
>>
>> Does the below comment makes sense to you?
>>
>> "
>> Make sure hugetlb demotion did not happen from under us.
>> "
>
> Yes, this sounds fine.
Will do it in v2. Thanks.
>
> Thanks,
> Naoya Horiguchi
>
> .
>