Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   =?iso-2022-jp?B?SE9SSUdVQ0hJIE5BT1lBKBskQktZOH0hIUQ+TGkbKEIp?= 
        <naoya.horiguchi@nec.com>
To:     "Luck, Tony" <tony.luck@intel.com>
CC:     Aili Yao <yaoaili@kingsoft.com>,
        Oscar Salvador <osalvador@suse.de>,
        "david@redhat.com" <david@redhat.com>,
        "akpm@linux-foundation.org" <akpm@linux-foundation.org>,
        "bp@alien8.de" <bp@alien8.de>,
        "tglx@linutronix.de" <tglx@linutronix.de>,
        "mingo@redhat.com" <mingo@redhat.com>,
        "hpa@zytor.com" <hpa@zytor.com>, "x86@kernel.org" <x86@kernel.org>,
        "linux-edac@vger.kernel.org" <linux-edac@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "yangfeng1@kingsoft.com" <yangfeng1@kingsoft.com>,
        "sunhao2@kingsoft.com" <sunhao2@kingsoft.com>
Subject: Re: [PATCH] mm,hwpoison: return -EBUSY when page already poisoned
Thread-Topic: [PATCH] mm,hwpoison: return -EBUSY when page already poisoned
Thread-Index: AQHXCn0F5fAOAyGRl0mX4JfXYioGdapnG5GAgAEgN4CAAIHeAIAAAyEAgAAQYACAAF5TAIAAhxGAgAALNoCAB+cDgIAABNQAgABOzACAAHYEgIAAsYCAgAAiT4CAACi3AIABIFEAgAAZ9wCAAPG5gIAHOEoAgAHAYACAAIkDgIAA1wiAgACxK4CAAHqggIAFKpeA
Date:   Tue, 16 Mar 2021 06:42:16 +0000
Message-ID: <20210316064215.GA23401@hori.linux.bs1.fc.nec.co.jp>
References: <20210304144524.795872d7@alex-virtual-machine>
 <20210304235720.GA215567@agluck-desk2.amr.corp.intel.com>
 <20210305093016.40c87375@alex-virtual-machine>
 <aee5176eafb54c88b19a5b2671d0a1fc@intel.com>
 <20210310141042.4db9ea29@alex-virtual-machine>
 <20210311085529.GA22268@hori.linux.bs1.fc.nec.co.jp>
 <db80e98d2b264e988596d0d7d7c8a776@intel.com>
 <20210312135531.72e33b35@alex-virtual-machine>
 <3900f518d1324c388be52cf81f5220e4@intel.com>
 <af80221baed940d8bcc643e3e7d40036@intel.com>
In-Reply-To: <af80221baed940d8bcc643e3e7d40036@intel.com>
Accept-Language: ja-JP, en-US
Content-Language: en-US
Content-Type: text/plain; charset="iso-2022-jp"
Content-ID: <472DD7338FE0E44BB9635B0A660F59EC@jpnprd01.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 9f8d3459-fa40-4d7b-e9e9-08d8e846a3cb
X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Mar 2021 06:42:16.5080
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: e67df547-9d0d-4f4d-9161-51c6ed1f7d11
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: eSRk1gxvp3psELy4inQWo2Pnx6bPfOAhew/0EYuw8fD5D9Sb86vdRRd7xxzKOgGyBiNuhLLFmJURmFNBVZbqYw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: TYAPR01MB4654
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, Mar 12, 2021 at 11:48:31PM +0000, Luck, Tony wrote:
> >> will memory_failure() find it and unmap it? if succeed, then the curre=
nt will be
> >> signaled with correct vaddr and shift?
> >
> > That's a very good question.  I didn't see a SIGBUS when I first wrote =
this code,
> > hence all the p->mce_vaddr.  But now I'm
> > a) not sure why there wasn't a signal

We have a recent change around this behavior, which might be an answer for =
you.
See commit 30c9cf492704 ("mm,hwpoison: send SIGBUS to PF_MCE_EARLY processe=
s on action required events").

> > b) if we are to fix the problems noted by AndyL, need to make sure that=
 there isn't a SIGBUS
>=20
> Tests on upstream kernel today show that memory_failure() is both unmappi=
ng the page
> and sending a SIGBUS.

Sorry if I misunderstood the exact problem, but if the point is to allow
user processes to find poisoned virtual address without SIGBUS, one possibl=
e
way is to expose hwpoison entry via /proc/pid/pagemap (attached a draft pat=
ch below).
With this patch, processes easily find hwpoison entries without any new int=
erface.

>=20
> My biggest issue with the KERNEL_COPYIN recovery path is that we don't ha=
ve code
> to mark the page not present while we are still in do_machine_check().=20

It sounds to me that even if we have such code, it just narrows down the ra=
ce window
between multiple MCEs on different CPUs. Or does it completely prevent the =
race?
(Another thread could touch the poisoned page just before the first thread =
marks
the page non-present?)

> That's resulted
> in recovery working for simple cases where there is a single get_user() c=
all followed by
> an error return if that failed. But more complex cases require more machi=
ne checks and
> a touching faith that the kernel will eventually give up trying (spoiler:=
 it sometimes doesn't).
>=20
> Thanks to the decode of the instruction we do have the virtual address. S=
o we just need
> a safe walk of pgd->p4d->pud->pmd->pte (truncated if we hit a huge page) =
with a write
> of a "not-present" value. Maybe a different poison type from the one we g=
et from
> memory_failure() so that the #PF code can recognize this as a special cas=
e and do any
> other work that we avoided because we were in #MC context.

As you answered in another email, p->mce_vaddr is set only on KERNEL_COPYIN=
 case,
then if p->mce_vaddr is available also for generic SRAR MCE (I'm assuming t=
hat we are
talking about issues on race between generic SRAR MCE not only for KERNEL_C=
OPYIN case),
that might be helpful, although it might be hard to implement.
And I'm afraid that walking page table could find the wrong virtual address=
 if a process
have multiple entries to the single page. We could send multiple SIGBUSs fo=
r such case,
but maybe that's not an optimal solution.

Thanks,
Naoya Horiguchi

----
From 147449a97e2ea3420ac3523f13523f5d30a13614 Mon Sep 17 00:00:00 2001
From: Naoya Horiguchi <naoya.horiguchi@nec.com>
Date: Tue, 16 Mar 2021 14:22:21 +0900
Subject: [PATCH] pagemap: expose hwpoison entry

not-signed-off-by-yet: Naoya Horiguchi <naoya.horiguchi@nec.com>
---
 fs/proc/task_mmu.c      |  6 ++++++
 include/linux/swapops.h | 12 ++++++++++++
 tools/vm/page-types.c   |  5 ++++-
 3 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index 602e3a52884d..08cea209bae7 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -1300,6 +1300,7 @@ struct pagemapread {
 #define PM_PFRAME_MASK		GENMASK_ULL(PM_PFRAME_BITS - 1, 0)
 #define PM_SOFT_DIRTY		BIT_ULL(55)
 #define PM_MMAP_EXCLUSIVE	BIT_ULL(56)
+#define PM_HWPOISON	       	BIT_ULL(60)
 #define PM_FILE			BIT_ULL(61)
 #define PM_SWAP			BIT_ULL(62)
 #define PM_PRESENT		BIT_ULL(63)
@@ -1385,6 +1386,11 @@ static pagemap_entry_t pte_to_pagemap_entry(struct p=
agemapread *pm,
 		if (is_migration_entry(entry))
 			page =3D migration_entry_to_page(entry);
=20
+		if (is_hwpoison_entry(entry)) {
+			page =3D hwpoison_entry_to_page(entry);
+			flags |=3D PM_HWPOISON;
+		}
+
 		if (is_device_private_entry(entry))
 			page =3D device_private_entry_to_page(entry);
 	}
diff --git a/include/linux/swapops.h b/include/linux/swapops.h
index d9b7c9132c2f..1b9dedbd06ab 100644
--- a/include/linux/swapops.h
+++ b/include/linux/swapops.h
@@ -323,6 +323,13 @@ static inline int is_hwpoison_entry(swp_entry_t entry)
 	return swp_type(entry) =3D=3D SWP_HWPOISON;
 }
=20
+static inline struct page *hwpoison_entry_to_page(swp_entry_t entry)
+{
+	struct page *p =3D pfn_to_page(swp_offset(entry));
+	WARN_ON(!PageHWPoison(p));
+	return p;
+}
+
 static inline void num_poisoned_pages_inc(void)
 {
 	atomic_long_inc(&num_poisoned_pages);
@@ -345,6 +352,11 @@ static inline int is_hwpoison_entry(swp_entry_t swp)
 	return 0;
 }
=20
+static inline struct page *hwpoison_entry_to_page(swp_entry_t entry)
+{
+	return NULL;
+}
+
 static inline void num_poisoned_pages_inc(void)
 {
 }
diff --git a/tools/vm/page-types.c b/tools/vm/page-types.c
index 0517c744b04e..1160d5a14955 100644
--- a/tools/vm/page-types.c
+++ b/tools/vm/page-types.c
@@ -53,6 +53,7 @@
 #define PM_SWAP_OFFSET(x)	(((x) & PM_PFRAME_MASK) >> MAX_SWAPFILES_SHIFT)
 #define PM_SOFT_DIRTY		(1ULL << 55)
 #define PM_MMAP_EXCLUSIVE	(1ULL << 56)
+#define PM_HWPOISON		(1ULL << 60)
 #define PM_FILE			(1ULL << 61)
 #define PM_SWAP			(1ULL << 62)
 #define PM_PRESENT		(1ULL << 63)
@@ -311,6 +312,8 @@ static unsigned long pagemap_pfn(uint64_t val)
=20
 	if (val & PM_PRESENT)
 		pfn =3D PM_PFRAME(val);
+	else if (val & PM_HWPOISON)
+		pfn =3D PM_SWAP_OFFSET(val);
 	else
 		pfn =3D 0;
=20
@@ -742,7 +745,7 @@ static void walk_vma(unsigned long index, unsigned long=
 count)
 			pfn =3D pagemap_pfn(buf[i]);
 			if (pfn)
 				walk_pfn(index + i, pfn, 1, buf[i]);
-			if (buf[i] & PM_SWAP)
+			else if (buf[i] & PM_SWAP)
 				walk_swap(index + i, buf[i]);
 		}
=20
--=20
2.25.1