LinuxLists.cc - [PATCH -next] mm/hwpoison: clear MF_COUNT_INCREASED before retrying get_any

2021-12-21 07:13:37

Subject: [PATCH -next] mm/hwpoison: clear MF_COUNT_INCREASED before retrying get_any_page()

Hulk Robot reported a panic in put_page_testzero() when testing madvise()
with MADV_SOFT_OFFLINE. The BUG() is triggered when retrying get_any_page().
This is because we keep MF_COUNT_INCREASED flag in second try but the refcnt
is not increased.

page dumped because: VM_BUG_ON_PAGE(page_ref_count(page) == 0)
------------[ cut here ]------------
kernel BUG at include/linux/mm.h:737!
invalid opcode: 0000 [#1] PREEMPT SMP
CPU: 5 PID: 2135 Comm: sshd Tainted: G B 5.16.0-rc6-dirty #373
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1.1 04/01/2014
RIP: 0010:release_pages+0x53f/0x840
Code: 0c 01 4c 8d 60 ff e9 5b fb ff ff 48 c7 c6 d8 97 0c b3 4c 89 e7 48 83 05 0e 7b 3c 0c 01 e8 89 3d 04 00 48 83 05 11 7b 3c 0c 01 <0f> 0b 48 83 05 0f 7b 3c 0c 01 48 83 05 0f 7b 3c 0c 01 48 83 05f
RSP: 0018:ffffc900015a7bc0 EFLAGS: 00010002
RAX: 000000000000003e RBX: ffffffffbace04c8 RCX: 0000000000000002
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 00000000ffffffff
RBP: ffff88817b9acd50 R08: 0000000000000000 R09: c0000000ffefffff
R10: 0000000000000001 R11: ffffc900015a79b0 R12: ffffea0005e1c900
R13: ffffea0005e1de88 R14: 000000000000001f R15: ffff888100071000
FS: 0000000000000000(0000) GS:ffff88842fb40000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f305e8de3d4 CR3: 000000017bb6f000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<TASK>
free_pages_and_swap_cache+0x64/0x80
tlb_flush_mmu+0x6f/0x220
unmap_page_range+0xe6c/0x12c0
unmap_single_vma+0x90/0x170
unmap_vmas+0xc4/0x180
exit_mmap+0xde/0x3a0
mmput+0xa3/0x250
do_exit+0x564/0x1470
do_group_exit+0x3b/0x100
__do_sys_exit_group+0x13/0x20
__x64_sys_exit_group+0x16/0x20
do_syscall_64+0x34/0x80
entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7f30625401d9
Code: Unable to access opcode bytes at RIP 0x7f30625401af.
RSP: 002b:00007ffe391b0c88 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f30625401d9
RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000000000000001
RBP: 00007f306283d838 R08: 000000000000003c R09: 00000000000000e7
R10: fffffffffffffe30 R11: 0000000000000246 R12: 00007f306283d838
R13: 00007f3062842e80 R14: 0000000000000000 R15: ffffaa4fb7932430
</TASK>
Modules linked in:
---[ end trace e99579b570fe0649 ]---
RIP: 0010:release_pages+0x53f/0x840
Code: 0c 01 4c 8d 60 ff e9 5b fb ff ff 48 c7 c6 d8 97 0c b3 4c 89 e7 48 83 05 0e 7b 3c 0c 01 e8 89 3d 04 00 48 83 05 11 7b 3c 0c 01 <0f> 0b 48 83 05 0f 7b 3c 0c 01 48 83 05 0f 7b 3c 0c 01 48 83 05f
RSP: 0018:ffffc900015a7bc0 EFLAGS: 00010002
RAX: 000000000000003e RBX: ffffffffbace04c8 RCX: 0000000000000002
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 00000000ffffffff
RBP: ffff88817b9acd50 R08: 0000000000000000 R09: c0000000ffefffff
R10: 0000000000000001 R11: ffffc900015a79b0 R12: ffffea0005e1c900
R13: ffffea0005e1de88 R14: 000000000000001f R15: ffff888100071000
FS: 0000000000000000(0000) GS:ffff88842fb40000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f305e8de3d4 CR3: 000000017bb6f000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400

Fixes: b94e02822deb ("mm,hwpoison: try to narrow window race for free pages")
Reported-by: Hulk Robot <[email protected]>
Signed-off-by: Liu Shixin <[email protected]>
---
mm/memory-failure.c | 1 +
1 file changed, 1 insertion(+)

diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 68d9a35f8908..160526f83bcb 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -2288,6 +2288,7 @@ int soft_offline_page(unsigned long pfn, int flags)
} else if (ret == 0) {
if (soft_offline_free_page(page) && try_again) {
try_again = false;
+ flags &= ~MF_COUNT_INCREASED;
goto retry;
}
}
--
2.25.1

2021-12-21 09:37:35

by Oscar Salvador

[permalink] [raw]

Subject: Re: [PATCH -next] mm/hwpoison: clear MF_COUNT_INCREASED before retrying get_any_page()

2021-12-22 00:38:05

by Naoya Horiguchi

[permalink] [raw]

Subject: Re: [PATCH -next] mm/hwpoison: clear MF_COUNT_INCREASED before retrying get_any_page()

On Tue, Dec 21, 2021 at 03:49:08PM +0800, Liu Shixin wrote:
> Hulk Robot reported a panic in put_page_testzero() when testing madvise()
> with MADV_SOFT_OFFLINE. The BUG() is triggered when retrying get_any_page().
> This is because we keep MF_COUNT_INCREASED flag in second try but the refcnt
> is not increased.

Hi Shixin,

Thanks you for the report.

I'm not still sure about the logic behind the bug.
madvise(MADV_SOFT_OFFLINE) should not work on free pages because
madvise_inject_error() pins the target page with get_user_pages_fast()
(MF_COUNT_INCREASED means refcount is incremented by the caller).
The retry logic can be called when soft_offline_free_page() fails, so
the logic assumes that the target page is a free page. So why does
clearing MF_COUNT_INCREASED affect and solve the issue?

It seems to me that if soft_offline_page() is called on a free page
with MF_COUNT_INCREASED set, then it's problematic itself.

Thanks,
Naoya Horiguchi

>
> page dumped because: VM_BUG_ON_PAGE(page_ref_count(page) == 0)
> ------------[ cut here ]------------
> kernel BUG at include/linux/mm.h:737!
> invalid opcode: 0000 [#1] PREEMPT SMP
> CPU: 5 PID: 2135 Comm: sshd Tainted: G B 5.16.0-rc6-dirty #373
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1.1 04/01/2014
> RIP: 0010:release_pages+0x53f/0x840
> Code: 0c 01 4c 8d 60 ff e9 5b fb ff ff 48 c7 c6 d8 97 0c b3 4c 89 e7 48 83 05 0e 7b 3c 0c 01 e8 89 3d 04 00 48 83 05 11 7b 3c 0c 01 <0f> 0b 48 83 05 0f 7b 3c 0c 01 48 83 05 0f 7b 3c 0c 01 48 83 05f
> RSP: 0018:ffffc900015a7bc0 EFLAGS: 00010002
> RAX: 000000000000003e RBX: ffffffffbace04c8 RCX: 0000000000000002
> RDX: 0000000000000000 RSI: 0000000000000001 RDI: 00000000ffffffff
> RBP: ffff88817b9acd50 R08: 0000000000000000 R09: c0000000ffefffff
> R10: 0000000000000001 R11: ffffc900015a79b0 R12: ffffea0005e1c900
> R13: ffffea0005e1de88 R14: 000000000000001f R15: ffff888100071000
> FS: 0000000000000000(0000) GS:ffff88842fb40000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00007f305e8de3d4 CR3: 000000017bb6f000 CR4: 00000000000006e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> Call Trace:
> <TASK>
> free_pages_and_swap_cache+0x64/0x80
> tlb_flush_mmu+0x6f/0x220
> unmap_page_range+0xe6c/0x12c0
> unmap_single_vma+0x90/0x170
> unmap_vmas+0xc4/0x180
> exit_mmap+0xde/0x3a0
> mmput+0xa3/0x250
> do_exit+0x564/0x1470
> do_group_exit+0x3b/0x100
> __do_sys_exit_group+0x13/0x20
> __x64_sys_exit_group+0x16/0x20
> do_syscall_64+0x34/0x80
> entry_SYSCALL_64_after_hwframe+0x44/0xae
> RIP: 0033:0x7f30625401d9
> Code: Unable to access opcode bytes at RIP 0x7f30625401af.
> RSP: 002b:00007ffe391b0c88 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
> RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f30625401d9
> RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000000000000001
> RBP: 00007f306283d838 R08: 000000000000003c R09: 00000000000000e7
> R10: fffffffffffffe30 R11: 0000000000000246 R12: 00007f306283d838
> R13: 00007f3062842e80 R14: 0000000000000000 R15: ffffaa4fb7932430
> </TASK>
> Modules linked in:
> ---[ end trace e99579b570fe0649 ]---
> RIP: 0010:release_pages+0x53f/0x840
> Code: 0c 01 4c 8d 60 ff e9 5b fb ff ff 48 c7 c6 d8 97 0c b3 4c 89 e7 48 83 05 0e 7b 3c 0c 01 e8 89 3d 04 00 48 83 05 11 7b 3c 0c 01 <0f> 0b 48 83 05 0f 7b 3c 0c 01 48 83 05 0f 7b 3c 0c 01 48 83 05f
> RSP: 0018:ffffc900015a7bc0 EFLAGS: 00010002
> RAX: 000000000000003e RBX: ffffffffbace04c8 RCX: 0000000000000002
> RDX: 0000000000000000 RSI: 0000000000000001 RDI: 00000000ffffffff
> RBP: ffff88817b9acd50 R08: 0000000000000000 R09: c0000000ffefffff
> R10: 0000000000000001 R11: ffffc900015a79b0 R12: ffffea0005e1c900
> R13: ffffea0005e1de88 R14: 000000000000001f R15: ffff888100071000
> FS: 0000000000000000(0000) GS:ffff88842fb40000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00007f305e8de3d4 CR3: 000000017bb6f000 CR4: 00000000000006e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>
> Fixes: b94e02822deb ("mm,hwpoison: try to narrow window race for free pages")
> Reported-by: Hulk Robot <[email protected]>
> Signed-off-by: Liu Shixin <[email protected]>
> ---
> mm/memory-failure.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
> index 68d9a35f8908..160526f83bcb 100644
> --- a/mm/memory-failure.c
> +++ b/mm/memory-failure.c
> @@ -2288,6 +2288,7 @@ int soft_offline_page(unsigned long pfn, int flags)
> } else if (ret == 0) {
> if (soft_offline_free_page(page) && try_again) {
> try_again = false;
> + flags &= ~MF_COUNT_INCREASED;
> goto retry;
> }
> }
> --
> 2.25.1
>

2021-12-22 01:49:22

by Liu Shixin

[permalink] [raw]

Subject: Re: [PATCH -next] mm/hwpoison: clear MF_COUNT_INCREASED before retrying get_any_page()

On 2021/12/22 8:37, Naoya Horiguchi wrote:
> On Tue, Dec 21, 2021 at 03:49:08PM +0800, Liu Shixin wrote:
>> Hulk Robot reported a panic in put_page_testzero() when testing madvise()
>> with MADV_SOFT_OFFLINE. The BUG() is triggered when retrying get_any_page().
>> This is because we keep MF_COUNT_INCREASED flag in second try but the refcnt
>> is not increased.
> Hi Shixin,
>
> Thanks you for the report.
>
> I'm not still sure about the logic behind the bug.
> madvise(MADV_SOFT_OFFLINE) should not work on free pages because
> madvise_inject_error() pins the target page with get_user_pages_fast()
> (MF_COUNT_INCREASED means refcount is incremented by the caller).
> The retry logic can be called when soft_offline_free_page() fails, so
> the logic assumes that the target page is a free page. So why does
> clearing MF_COUNT_INCREASED affect and solve the issue?
>
> It seems to me that if soft_offline_page() is called on a free page
> with MF_COUNT_INCREASED set, then it's problematic itself.
>
> Thanks,
> Naoya Horiguchi

The page is not free when soft_offline_page() is called. In get_any_page(),
if MF_COUNT_INCREASED is set and !HWPoisonHandlable(page), put_page()
will be called firstly and then goto try_again. The page is freed in this period.

Thanks,
Liu Shixin
>> page dumped because: VM_BUG_ON_PAGE(page_ref_count(page) == 0)
>> ------------[ cut here ]------------
>> kernel BUG at include/linux/mm.h:737!
>> invalid opcode: 0000 [#1] PREEMPT SMP
>> CPU: 5 PID: 2135 Comm: sshd Tainted: G B 5.16.0-rc6-dirty #373
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1.1 04/01/2014
>> RIP: 0010:release_pages+0x53f/0x840
>> Code: 0c 01 4c 8d 60 ff e9 5b fb ff ff 48 c7 c6 d8 97 0c b3 4c 89 e7 48 83 05 0e 7b 3c 0c 01 e8 89 3d 04 00 48 83 05 11 7b 3c 0c 01 <0f> 0b 48 83 05 0f 7b 3c 0c 01 48 83 05 0f 7b 3c 0c 01 48 83 05f
>> RSP: 0018:ffffc900015a7bc0 EFLAGS: 00010002
>> RAX: 000000000000003e RBX: ffffffffbace04c8 RCX: 0000000000000002
>> RDX: 0000000000000000 RSI: 0000000000000001 RDI: 00000000ffffffff
>> RBP: ffff88817b9acd50 R08: 0000000000000000 R09: c0000000ffefffff
>> R10: 0000000000000001 R11: ffffc900015a79b0 R12: ffffea0005e1c900
>> R13: ffffea0005e1de88 R14: 000000000000001f R15: ffff888100071000
>> FS: 0000000000000000(0000) GS:ffff88842fb40000(0000) knlGS:0000000000000000
>> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> CR2: 00007f305e8de3d4 CR3: 000000017bb6f000 CR4: 00000000000006e0
>> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>> Call Trace:
>> <TASK>
>> free_pages_and_swap_cache+0x64/0x80
>> tlb_flush_mmu+0x6f/0x220
>> unmap_page_range+0xe6c/0x12c0
>> unmap_single_vma+0x90/0x170
>> unmap_vmas+0xc4/0x180
>> exit_mmap+0xde/0x3a0
>> mmput+0xa3/0x250
>> do_exit+0x564/0x1470
>> do_group_exit+0x3b/0x100
>> __do_sys_exit_group+0x13/0x20
>> __x64_sys_exit_group+0x16/0x20
>> do_syscall_64+0x34/0x80
>> entry_SYSCALL_64_after_hwframe+0x44/0xae
>> RIP: 0033:0x7f30625401d9
>> Code: Unable to access opcode bytes at RIP 0x7f30625401af.
>> RSP: 002b:00007ffe391b0c88 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
>> RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f30625401d9
>> RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000000000000001
>> RBP: 00007f306283d838 R08: 000000000000003c R09: 00000000000000e7
>> R10: fffffffffffffe30 R11: 0000000000000246 R12: 00007f306283d838
>> R13: 00007f3062842e80 R14: 0000000000000000 R15: ffffaa4fb7932430
>> </TASK>
>> Modules linked in:
>> ---[ end trace e99579b570fe0649 ]---
>> RIP: 0010:release_pages+0x53f/0x840
>> Code: 0c 01 4c 8d 60 ff e9 5b fb ff ff 48 c7 c6 d8 97 0c b3 4c 89 e7 48 83 05 0e 7b 3c 0c 01 e8 89 3d 04 00 48 83 05 11 7b 3c 0c 01 <0f> 0b 48 83 05 0f 7b 3c 0c 01 48 83 05 0f 7b 3c 0c 01 48 83 05f
>> RSP: 0018:ffffc900015a7bc0 EFLAGS: 00010002
>> RAX: 000000000000003e RBX: ffffffffbace04c8 RCX: 0000000000000002
>> RDX: 0000000000000000 RSI: 0000000000000001 RDI: 00000000ffffffff
>> RBP: ffff88817b9acd50 R08: 0000000000000000 R09: c0000000ffefffff
>> R10: 0000000000000001 R11: ffffc900015a79b0 R12: ffffea0005e1c900
>> R13: ffffea0005e1de88 R14: 000000000000001f R15: ffff888100071000
>> FS: 0000000000000000(0000) GS:ffff88842fb40000(0000) knlGS:0000000000000000
>> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> CR2: 00007f305e8de3d4 CR3: 000000017bb6f000 CR4: 00000000000006e0
>> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>>
>> Fixes: b94e02822deb ("mm,hwpoison: try to narrow window race for free pages")
>> Reported-by: Hulk Robot <[email protected]>
>> Signed-off-by: Liu Shixin <[email protected]>
>> ---
>> mm/memory-failure.c | 1 +
>> 1 file changed, 1 insertion(+)
>>
>> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
>> index 68d9a35f8908..160526f83bcb 100644
>> --- a/mm/memory-failure.c
>> +++ b/mm/memory-failure.c
>> @@ -2288,6 +2288,7 @@ int soft_offline_page(unsigned long pfn, int flags)
>> } else if (ret == 0) {
>> if (soft_offline_free_page(page) && try_again) {
>> try_again = false;
>> + flags &= ~MF_COUNT_INCREASED;
>> goto retry;
>> }
>> }
>> --
>> 2.25.1
>>
> .
>

2021-12-22 02:14:57

by Naoya Horiguchi

[permalink] [raw]

Subject: Re: [PATCH -next] mm/hwpoison: clear MF_COUNT_INCREASED before retrying get_any_page()

On Wed, Dec 22, 2021 at 09:49:18AM +0800, Liu Shixin wrote:
>
>
> On 2021/12/22 8:37, Naoya Horiguchi wrote:
> > On Tue, Dec 21, 2021 at 03:49:08PM +0800, Liu Shixin wrote:
> >> Hulk Robot reported a panic in put_page_testzero() when testing madvise()
> >> with MADV_SOFT_OFFLINE. The BUG() is triggered when retrying get_any_page().
> >> This is because we keep MF_COUNT_INCREASED flag in second try but the refcnt
> >> is not increased.
> > Hi Shixin,
> >
> > Thanks you for the report.
> >
> > I'm not still sure about the logic behind the bug.
> > madvise(MADV_SOFT_OFFLINE) should not work on free pages because
> > madvise_inject_error() pins the target page with get_user_pages_fast()
> > (MF_COUNT_INCREASED means refcount is incremented by the caller).
> > The retry logic can be called when soft_offline_free_page() fails, so
> > the logic assumes that the target page is a free page. So why does
> > clearing MF_COUNT_INCREASED affect and solve the issue?
> >
> > It seems to me that if soft_offline_page() is called on a free page
> > with MF_COUNT_INCREASED set, then it's problematic itself.
> >
> > Thanks,
> > Naoya Horiguchi
>
> The page is not free when soft_offline_page() is called. In get_any_page(),
> if MF_COUNT_INCREASED is set and !HWPoisonHandlable(page), put_page()
> will be called firstly and then goto try_again. The page is freed in this period.

Okay, the logic is clear to me now.

> >>
> >> Fixes: b94e02822deb ("mm,hwpoison: try to narrow window race for free pages")
> >> Reported-by: Hulk Robot <[email protected]>
> >> Signed-off-by: Liu Shixin <[email protected]>

Thank you very much.

Acked-by: Naoya Horiguchi <[email protected]>

> >> ---
> >> mm/memory-failure.c | 1 +
> >> 1 file changed, 1 insertion(+)
> >>
> >> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
> >> index 68d9a35f8908..160526f83bcb 100644
> >> --- a/mm/memory-failure.c
> >> +++ b/mm/memory-failure.c
> >> @@ -2288,6 +2288,7 @@ int soft_offline_page(unsigned long pfn, int flags)
> >> } else if (ret == 0) {
> >> if (soft_offline_free_page(page) && try_again) {
> >> try_again = false;
> >> + flags &= ~MF_COUNT_INCREASED;
> >> goto retry;
> >> }
> >> }
> >> --
> >> 2.25.1
> >>
> > .
> >
>