On 10.06.22 13:46, zhenwei pi wrote:
> Currently unpoison_memory(unsigned long pfn) is designed for soft
> poison(hwpoison-inject) only. Since 17fae1294ad9d, the KPTE gets
> cleared on a x86 platform once hardware memory corrupts.
>
> Unpoisoning a hardware corrupted page puts page back buddy only,
> the kernel has a chance to access the page with *NOT PRESENT* KPTE.
> This leads BUG during accessing on the corrupted KPTE.
>
> Do not allow to unpoison hardware corrupted page in unpoison_memory() to
> avoid BUG like this:
>
> Unpoison: Software-unpoisoned page 0x61234
> BUG: unable to handle page fault for address: ffff888061234000
> #PF: supervisor write access in kernel mode
> #PF: error_code(0x0002) - not-present page
> PGD 2c01067 P4D 2c01067 PUD 107267063 PMD 10382b063 PTE 800fffff9edcb062
> Oops: 0002 [#1] PREEMPT SMP NOPTI
> CPU: 4 PID: 26551 Comm: stress Kdump: loaded Tainted: G M OE 5.18.0.bm.1-amd64 #7
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996) ...
> RIP: 0010:clear_page_erms+0x7/0x10
> Code: ...
> RSP: 0000:ffffc90001107bc8 EFLAGS: 00010246
> RAX: 0000000000000000 RBX: 0000000000000901 RCX: 0000000000001000
> RDX: ffffea0001848d00 RSI: ffffea0001848d40 RDI: ffff888061234000
> RBP: ffffea0001848d00 R08: 0000000000000901 R09: 0000000000001276
> R10: 0000000000000003 R11: 0000000000000000 R12: 0000000000000001
> R13: 0000000000000000 R14: 0000000000140dca R15: 0000000000000001
> FS: 00007fd8b2333740(0000) GS:ffff88813fd00000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: ffff888061234000 CR3: 00000001023d2005 CR4: 0000000000770ee0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> PKRU: 55555554
> Call Trace:
> <TASK>
> prep_new_page+0x151/0x170
> get_page_from_freelist+0xca0/0xe20
> ? sysvec_apic_timer_interrupt+0xab/0xc0
> ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
> __alloc_pages+0x17e/0x340
> __folio_alloc+0x17/0x40
> vma_alloc_folio+0x84/0x280
> __handle_mm_fault+0x8d4/0xeb0
> handle_mm_fault+0xd5/0x2a0
> do_user_addr_fault+0x1d0/0x680
> ? kvm_read_and_reset_apf_flags+0x3b/0x50
> exc_page_fault+0x78/0x170
> asm_exc_page_fault+0x27/0x30
>
> Fixes: 847ce401df392 ("HWPOISON: Add unpoisoning support")
> Fixes: 17fae1294ad9d ("x86/{mce,mm}: Unmap the entire page if the whole page is affected and poisoned")
> Cc: Naoya Horiguchi <[email protected]>
> Cc: David Hildenbrand <[email protected]>
> Signed-off-by: zhenwei pi <[email protected]>
> ---
> mm/memory-failure.c | 59 +++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 59 insertions(+)
>
> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
> index b85661cbdc4a..3124f428302c 100644
> --- a/mm/memory-failure.c
> +++ b/mm/memory-failure.c
> @@ -2068,6 +2068,58 @@ static int __init memory_failure_init(void)
> }
> core_initcall(memory_failure_init);
>
> +/*
> + * Unpoisoning a hardware corrupted page with *NOT PRESENT* KPTE leads panic.
> + * Test a page is valid in the kernel mapping.
> + */
> +static bool kmap_valid(struct page *page)
> +{
> + unsigned long addr = (unsigned long)page_to_virt(page);
> + pgd_t *pgd;
> + p4d_t *p4d;
> + pud_t *pud;
> + pmd_t *pmd;
> + pte_t *pte;
> +
> + pgd = pgd_offset_k(addr);
> + if (pgd_none(*pgd))
> + return false;
> + if (pgd_leaf(*pgd))
> + return true;
> + if (pgd_bad(*pgd))
> + return false;
> +
> + p4d = p4d_offset(pgd, addr);
> + if (p4d_none(*p4d))
> + return false;
> + if (p4d_leaf(*p4d))
> + return true;
> + if (p4d_bad(*p4d))
> + return false;
> +
> + pud = pud_offset(p4d, addr);
> + if (pud_none(*pud))
> + return false;
> + if (pud_leaf(*pud))
> + return true;
> + if (pud_bad(*pud))
> + return false;
> +
> + pmd = pmd_offset(pud, addr);
> + if (pmd_none(*pmd))
> + return false;
> + if (pmd_leaf(*pmd))
> + return true;
> + if (pmd_bad(*pmd))
> + return false;
> +
> + pte = pte_offset_map(pmd, addr);
> + if (pte_none(*pte) || !pte_present(*pte))
> + return false;
> +
> + return true;
> +}
> +
> #define unpoison_pr_info(fmt, pfn, rs) \
> ({ \
> if (__ratelimit(rs)) \
> @@ -2109,6 +2161,13 @@ int unpoison_memory(unsigned long pfn)
> goto unlock_mutex;
> }
>
> + if (!kmap_valid(page)) {
> + unpoison_pr_info("Unpoison: Page was hardware poisoned %#lx\n",
> + pfn, &unpoison_rs);
> + ret = -EOPNOTSUPP;
> + goto unlock_mutex;
> + }
> +
> if (page_count(page) > 1) {
> unpoison_pr_info("Unpoison: Someone grabs the hwpoison page %#lx\n",
> pfn, &unpoison_rs);
I really prefer just disabling the unpoisioning mechanism in case there
is a real hw injected error.
--
Thanks,
David / dhildenb
On 6/10/22 20:12, David Hildenbrand wrote:
> On 10.06.22 13:46, zhenwei pi wrote:
>> Currently unpoison_memory(unsigned long pfn) is designed for soft
>> poison(hwpoison-inject) only. Since 17fae1294ad9d, the KPTE gets
>> cleared on a x86 platform once hardware memory corrupts.
>>
>> Unpoisoning a hardware corrupted page puts page back buddy only,
>> the kernel has a chance to access the page with *NOT PRESENT* KPTE.
>> This leads BUG during accessing on the corrupted KPTE.
>>
>> Do not allow to unpoison hardware corrupted page in unpoison_memory() to
>> avoid BUG like this:
>>
>> Unpoison: Software-unpoisoned page 0x61234
>> BUG: unable to handle page fault for address: ffff888061234000
>> #PF: supervisor write access in kernel mode
>> #PF: error_code(0x0002) - not-present page
>> PGD 2c01067 P4D 2c01067 PUD 107267063 PMD 10382b063 PTE 800fffff9edcb062
>> Oops: 0002 [#1] PREEMPT SMP NOPTI
>> CPU: 4 PID: 26551 Comm: stress Kdump: loaded Tainted: G M OE 5.18.0.bm.1-amd64 #7
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996) ...
>> RIP: 0010:clear_page_erms+0x7/0x10
>> Code: ...
>> RSP: 0000:ffffc90001107bc8 EFLAGS: 00010246
>> RAX: 0000000000000000 RBX: 0000000000000901 RCX: 0000000000001000
>> RDX: ffffea0001848d00 RSI: ffffea0001848d40 RDI: ffff888061234000
>> RBP: ffffea0001848d00 R08: 0000000000000901 R09: 0000000000001276
>> R10: 0000000000000003 R11: 0000000000000000 R12: 0000000000000001
>> R13: 0000000000000000 R14: 0000000000140dca R15: 0000000000000001
>> FS: 00007fd8b2333740(0000) GS:ffff88813fd00000(0000) knlGS:0000000000000000
>> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> CR2: ffff888061234000 CR3: 00000001023d2005 CR4: 0000000000770ee0
>> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>> PKRU: 55555554
>> Call Trace:
>> <TASK>
>> prep_new_page+0x151/0x170
>> get_page_from_freelist+0xca0/0xe20
>> ? sysvec_apic_timer_interrupt+0xab/0xc0
>> ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
>> __alloc_pages+0x17e/0x340
>> __folio_alloc+0x17/0x40
>> vma_alloc_folio+0x84/0x280
>> __handle_mm_fault+0x8d4/0xeb0
>> handle_mm_fault+0xd5/0x2a0
>> do_user_addr_fault+0x1d0/0x680
>> ? kvm_read_and_reset_apf_flags+0x3b/0x50
>> exc_page_fault+0x78/0x170
>> asm_exc_page_fault+0x27/0x30
>>
>> Fixes: 847ce401df392 ("HWPOISON: Add unpoisoning support")
>> Fixes: 17fae1294ad9d ("x86/{mce,mm}: Unmap the entire page if the whole page is affected and poisoned")
>> Cc: Naoya Horiguchi <[email protected]>
>> Cc: David Hildenbrand <[email protected]>
>> Signed-off-by: zhenwei pi <[email protected]>
>> ---
>> mm/memory-failure.c | 59 +++++++++++++++++++++++++++++++++++++++++++++
>> 1 file changed, 59 insertions(+)
>>
>> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
>> index b85661cbdc4a..3124f428302c 100644
>> --- a/mm/memory-failure.c
>> +++ b/mm/memory-failure.c
>> @@ -2068,6 +2068,58 @@ static int __init memory_failure_init(void)
>> }
>> core_initcall(memory_failure_init);
>>
>> +/*
>> + * Unpoisoning a hardware corrupted page with *NOT PRESENT* KPTE leads panic.
>> + * Test a page is valid in the kernel mapping.
>> + */
>> +static bool kmap_valid(struct page *page)
>> +{
>> + unsigned long addr = (unsigned long)page_to_virt(page);
>> + pgd_t *pgd;
>> + p4d_t *p4d;
>> + pud_t *pud;
>> + pmd_t *pmd;
>> + pte_t *pte;
>> +
>> + pgd = pgd_offset_k(addr);
>> + if (pgd_none(*pgd))
>> + return false;
>> + if (pgd_leaf(*pgd))
>> + return true;
>> + if (pgd_bad(*pgd))
>> + return false;
>> +
>> + p4d = p4d_offset(pgd, addr);
>> + if (p4d_none(*p4d))
>> + return false;
>> + if (p4d_leaf(*p4d))
>> + return true;
>> + if (p4d_bad(*p4d))
>> + return false;
>> +
>> + pud = pud_offset(p4d, addr);
>> + if (pud_none(*pud))
>> + return false;
>> + if (pud_leaf(*pud))
>> + return true;
>> + if (pud_bad(*pud))
>> + return false;
>> +
>> + pmd = pmd_offset(pud, addr);
>> + if (pmd_none(*pmd))
>> + return false;
>> + if (pmd_leaf(*pmd))
>> + return true;
>> + if (pmd_bad(*pmd))
>> + return false;
>> +
>> + pte = pte_offset_map(pmd, addr);
>> + if (pte_none(*pte) || !pte_present(*pte))
>> + return false;
>> +
>> + return true;
>> +}
>> +
>> #define unpoison_pr_info(fmt, pfn, rs) \
>> ({ \
>> if (__ratelimit(rs)) \
>> @@ -2109,6 +2161,13 @@ int unpoison_memory(unsigned long pfn)
>> goto unlock_mutex;
>> }
>>
>> + if (!kmap_valid(page)) {
>> + unpoison_pr_info("Unpoison: Page was hardware poisoned %#lx\n",
>> + pfn, &unpoison_rs);
>> + ret = -EOPNOTSUPP;
>> + goto unlock_mutex;
>> + }
>> +
>> if (page_count(page) > 1) {
>> unpoison_pr_info("Unpoison: Someone grabs the hwpoison page %#lx\n",
>> pfn, &unpoison_rs);
>
> I really prefer just disabling the unpoisioning mechanism in case there
> is a real hw injected error.
>
Hi, Naoya & Andrew
What is your opinion?
--
zhenwei pi
On Tue, Jun 14, 2022 at 08:22:10AM +0800, zhenwei pi wrote:
>
>
> On 6/10/22 20:12, David Hildenbrand wrote:
> > On 10.06.22 13:46, zhenwei pi wrote:
> > > Currently unpoison_memory(unsigned long pfn) is designed for soft
> > > poison(hwpoison-inject) only. Since 17fae1294ad9d, the KPTE gets
> > > cleared on a x86 platform once hardware memory corrupts.
> > >
> > > Unpoisoning a hardware corrupted page puts page back buddy only,
> > > the kernel has a chance to access the page with *NOT PRESENT* KPTE.
> > > This leads BUG during accessing on the corrupted KPTE.
> > >
> > > Do not allow to unpoison hardware corrupted page in unpoison_memory() to
> > > avoid BUG like this:
> > >
> > > Unpoison: Software-unpoisoned page 0x61234
> > > BUG: unable to handle page fault for address: ffff888061234000
> > > #PF: supervisor write access in kernel mode
> > > #PF: error_code(0x0002) - not-present page
> > > PGD 2c01067 P4D 2c01067 PUD 107267063 PMD 10382b063 PTE 800fffff9edcb062
> > > Oops: 0002 [#1] PREEMPT SMP NOPTI
> > > CPU: 4 PID: 26551 Comm: stress Kdump: loaded Tainted: G M OE 5.18.0.bm.1-amd64 #7
> > > Hardware name: QEMU Standard PC (i440FX + PIIX, 1996) ...
> > > RIP: 0010:clear_page_erms+0x7/0x10
> > > Code: ...
> > > RSP: 0000:ffffc90001107bc8 EFLAGS: 00010246
> > > RAX: 0000000000000000 RBX: 0000000000000901 RCX: 0000000000001000
> > > RDX: ffffea0001848d00 RSI: ffffea0001848d40 RDI: ffff888061234000
> > > RBP: ffffea0001848d00 R08: 0000000000000901 R09: 0000000000001276
> > > R10: 0000000000000003 R11: 0000000000000000 R12: 0000000000000001
> > > R13: 0000000000000000 R14: 0000000000140dca R15: 0000000000000001
> > > FS: 00007fd8b2333740(0000) GS:ffff88813fd00000(0000) knlGS:0000000000000000
> > > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > > CR2: ffff888061234000 CR3: 00000001023d2005 CR4: 0000000000770ee0
> > > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> > > PKRU: 55555554
> > > Call Trace:
> > > <TASK>
> > > prep_new_page+0x151/0x170
> > > get_page_from_freelist+0xca0/0xe20
> > > ? sysvec_apic_timer_interrupt+0xab/0xc0
> > > ? asm_sysvec_apic_timer_interrupt+0x1b/0x20
> > > __alloc_pages+0x17e/0x340
> > > __folio_alloc+0x17/0x40
> > > vma_alloc_folio+0x84/0x280
> > > __handle_mm_fault+0x8d4/0xeb0
> > > handle_mm_fault+0xd5/0x2a0
> > > do_user_addr_fault+0x1d0/0x680
> > > ? kvm_read_and_reset_apf_flags+0x3b/0x50
> > > exc_page_fault+0x78/0x170
> > > asm_exc_page_fault+0x27/0x30
> > >
> > > Fixes: 847ce401df392 ("HWPOISON: Add unpoisoning support")
> > > Fixes: 17fae1294ad9d ("x86/{mce,mm}: Unmap the entire page if the whole page is affected and poisoned")
> > > Cc: Naoya Horiguchi <[email protected]>
> > > Cc: David Hildenbrand <[email protected]>
> > > Signed-off-by: zhenwei pi <[email protected]>
> > > ---
> > > mm/memory-failure.c | 59 +++++++++++++++++++++++++++++++++++++++++++++
> > > 1 file changed, 59 insertions(+)
> > >
> > > diff --git a/mm/memory-failure.c b/mm/memory-failure.c
> > > index b85661cbdc4a..3124f428302c 100644
> > > --- a/mm/memory-failure.c
> > > +++ b/mm/memory-failure.c
> > > @@ -2068,6 +2068,58 @@ static int __init memory_failure_init(void)
> > > }
> > > core_initcall(memory_failure_init);
> > > +/*
> > > + * Unpoisoning a hardware corrupted page with *NOT PRESENT* KPTE leads panic.
> > > + * Test a page is valid in the kernel mapping.
> > > + */
> > > +static bool kmap_valid(struct page *page)
> > > +{
> > > + unsigned long addr = (unsigned long)page_to_virt(page);
> > > + pgd_t *pgd;
> > > + p4d_t *p4d;
> > > + pud_t *pud;
> > > + pmd_t *pmd;
> > > + pte_t *pte;
> > > +
> > > + pgd = pgd_offset_k(addr);
> > > + if (pgd_none(*pgd))
> > > + return false;
> > > + if (pgd_leaf(*pgd))
> > > + return true;
> > > + if (pgd_bad(*pgd))
> > > + return false;
> > > +
> > > + p4d = p4d_offset(pgd, addr);
> > > + if (p4d_none(*p4d))
> > > + return false;
> > > + if (p4d_leaf(*p4d))
> > > + return true;
> > > + if (p4d_bad(*p4d))
> > > + return false;
> > > +
> > > + pud = pud_offset(p4d, addr);
> > > + if (pud_none(*pud))
> > > + return false;
> > > + if (pud_leaf(*pud))
> > > + return true;
> > > + if (pud_bad(*pud))
> > > + return false;
> > > +
> > > + pmd = pmd_offset(pud, addr);
> > > + if (pmd_none(*pmd))
> > > + return false;
> > > + if (pmd_leaf(*pmd))
> > > + return true;
> > > + if (pmd_bad(*pmd))
> > > + return false;
> > > +
> > > + pte = pte_offset_map(pmd, addr);
> > > + if (pte_none(*pte) || !pte_present(*pte))
> > > + return false;
> > > +
> > > + return true;
> > > +}
> > > +
> > > #define unpoison_pr_info(fmt, pfn, rs) \
> > > ({ \
> > > if (__ratelimit(rs)) \
> > > @@ -2109,6 +2161,13 @@ int unpoison_memory(unsigned long pfn)
> > > goto unlock_mutex;
> > > }
> > > + if (!kmap_valid(page)) {
> > > + unpoison_pr_info("Unpoison: Page was hardware poisoned %#lx\n",
> > > + pfn, &unpoison_rs);
> > > + ret = -EOPNOTSUPP;
> > > + goto unlock_mutex;
> > > + }
> > > +
> > > if (page_count(page) > 1) {
> > > unpoison_pr_info("Unpoison: Someone grabs the hwpoison page %#lx\n",
> > > pfn, &unpoison_rs);
> >
> > I really prefer just disabling the unpoisioning mechanism in case there
> > is a real hw injected error.
> >
>
> Hi, Naoya & Andrew
>
> What is your opinion?
Disabling unpoison when a real HW error happens is fine to me. When you
write the patch, could you add a few sentences about the new behavior to
the unpoison-pfn section in Documentation/vm/hwpoison.rst?
Thanks,
Naoya Horiguchi
On 2022/6/10 20:12, David Hildenbrand wrote:
>
> I really prefer just disabling the unpoisioning mechanism in case there
> is a real hw injected error.
IMHO, this would be a simple solution that also works on the arches that does not unmap
the kernel mapping when a page is hardware corrupted.
Thanks!
>