Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   =?iso-2022-jp?B?SE9SSUdVQ0hJIE5BT1lBKBskQktZOH0hIUQ+TGkbKEIp?= 
        <naoya.horiguchi@nec.com>
To:     Qian Cai <cai@lca.pw>
CC:     Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>,
        Oscar Salvador <osalvador@suse.de>,
        Michal Hocko <mhocko@kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        David Hildenbrand <david@redhat.com>,
        Mike Kravetz <mike.kravetz@oracle.com>
Subject: Re: memory offline infinite loop after soft offline
Thread-Topic: memory offline infinite loop after soft offline
Thread-Index: AQHWKmMR+OImEsyoBU+Os1O8O47V2qiogoWA
Date:   Fri, 15 May 2020 03:48:10 +0000
Message-ID: <20200515034809.GA27576@hori.linux.bs1.fc.nec.co.jp>
References: <20191018063222.GA15406@hori.linux.bs1.fc.nec.co.jp>
 <64DC81FB-C1D2-44F2-981F-C6F766124B91@lca.pw>
 <20191021031641.GA8007@hori.linux.bs1.fc.nec.co.jp>
 <DE0721DF-9E5C-4719-B382-01A4A74C04AD@lca.pw>
In-Reply-To: <DE0721DF-9E5C-4719-B382-01A4A74C04AD@lca.pw>
Accept-Language: ja-JP, en-US
Content-Language: en-US
Content-Type: text/plain; charset="iso-2022-jp"
Content-ID: <50CBFE582D1C6140995E0C2FFA112B47@jpnprd01.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: adb44e31-0e48-4366-b323-08d7f882c973
X-MS-Exchange-CrossTenant-originalarrivaltime: 15 May 2020 03:48:10.4325
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: e67df547-9d0d-4f4d-9161-51c6ed1f7d11
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: mOpzZDBh2a3CBqNCaSxnWjaYFXGQa3BYKtXTMECjRND0QUldIyDzZUBiHS3FkKNZv4WlW+uqd513F1cqbQRbSg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: TY2PR01MB2171
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, May 14, 2020 at 10:46:33PM -0400, Qian Cai wrote:
>=20
>=20
> > On Oct 20, 2019, at 11:16 PM, Naoya Horiguchi <n-horiguchi@ah.jp.nec.co=
m> wrote:
> >=20
> > On Fri, Oct 18, 2019 at 07:56:09AM -0400, Qian Cai wrote:
> >>=20
> >>=20
> >>    On Oct 18, 2019, at 2:35 AM, Naoya Horiguchi <n-horiguchi@ah.jp.nec=
.com>
> >>    wrote:
> >>=20
> >>=20
> >>    You're right, then I don't see how this happens. If the error hugep=
age was
> >>    isolated without having PG_hwpoison set, it's unexpected and proble=
matic.
> >>    I'm testing myself with v5.4-rc2 (simply ran move_pages12 and did h=
otremove
> >>    /hotadd)
> >>    but don't reproduce the issue yet.  Do we need specific kernel vers=
ion/
> >>    config
> >>    to trigger this?
> >>=20
> >>=20
> >> This is reproducible on linux-next with the config. Not sure if it is
> >> reproducible on x86.
> >>=20
> >> https://raw.githubusercontent.com/cailca/linux-mm/master/powerpc.confi=
g
> >>=20
> >> and kernel cmdline if that matters
> >>=20
> >> page_poison=3Don page_owner=3Don numa_balancing=3Denable \
> >> systemd.unified_cgroup_hierarchy=3D1 debug_guardpage_minorder=3D1 \
> >> page_alloc.shuffle=3D1
> >=20
> > Thanks for the info.
> >=20
> >>=20
> >> BTW, where does the code set PG_hwpoison for the head page?
> >=20
> > Precisely speaking, soft offline only sets PG_hwpoison after the target
> > hugepage is successfully dissolved (then it's not a hugepage any more),
> > so PG_hwpoison is set on the raw page in set_hwpoison_free_buddy_page()=
.
> >=20
> > In move_pages12 case, madvise(MADV_SOFT_OFFLINE) is called for the rang=
e
> > of 2 hugepages, so the expected result is that page offset 0 and 512
> > are marked as PG_hwpoison after injection.
> >=20
> > Looking at your dump_page() output, the end_pfn is page offset 1
> > ("page:c00c000800458040" is likely to point to pfn 0x11601.)
> > The page belongs to high order buddy free page, but doesn't have
> > PageBuddy nor PageHWPoison because it was not the head page or
> > the raw error page.
> >=20
> >> Unfortunately, this does not solve the problem. It looks to me that in=
           =20
> >> soft_offline_huge_page(), set_hwpoison_free_buddy_page() will only set=
           =20
> >> PG_hwpoison for buddy pages, so the even the compound_head() has no PG=
_hwpoison  =20
> >> set.                                                                  =
           =20
> >=20
> > Your analysis is totally correct, and this behavior will be fixed by
> > the change (https://lkml.org/lkml/2019/10/17/551) in Oscar's rework.
> > The raw error page will be taken off from buddy system and the other
> > subpages are properly split into lower orderer pages (we'll properly
> > manage PageBuddy flags). So all possible cases would be covered by
> > branches in __test_page_isolated_in_pageblock.
>=20
> Naoya, Oscar, it looks like this series was stuck.
>=20
> https://lkml.org/lkml/2019/10/17/551
>=20
> I can still reproduce this issue as today. Maybe it is best we could post=
 a single patch (which one?) to fix the loop first?

I'm very sorry to be quiet for long, but I think that I agree with
this patchset and try to see what happend if merged into mmtom,
although we need rebaseing to latest mmotm and some basic testing.=