Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Mon, 20 May 2019 17:00:21 +1000
From:   Nicholas Piggin <npiggin@gmail.com>
Subject: Re: PROBLEM: Power9: kernel oops on memory hotunplug from ppc64le
 guest
To:     bharata@linux.ibm.com
Cc:     aneesh.kumar@linux.ibm.com, bharata@linux.vnet.ibm.com,
        linux-kernel@vger.kernel.org, linux-next@vger.kernel.org,
        linuxppc-dev@lists.ozlabs.org,
        Michael Ellerman <mpe@ellerman.id.au>,
        srikanth <sraithal@linux.vnet.ibm.com>
References: <16a7a635-c592-27e2-75b4-d02071833278@linux.vnet.ibm.com>
        <20190518141434.GA22939@in.ibm.com>
        <878sv1993k.fsf@concordia.ellerman.id.au>
        <20190520042533.GB22939@in.ibm.com>
        <1558327521.633yjtl8ki.astroid@bobo.none>
        <20190520055622.GC22939@in.ibm.com>
In-Reply-To: <20190520055622.GC22939@in.ibm.com>
MIME-Version: 1.0
User-Agent: astroid/0.14.0 (https://github.com/astroidmail/astroid)
Message-Id: <1558335484.9inx69a7ea.astroid@bobo.none>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Bharata B Rao's on May 20, 2019 3:56 pm:
> On Mon, May 20, 2019 at 02:48:35PM +1000, Nicholas Piggin wrote:
>> >> > git bisect points to
>> >> >
>> >> > commit 4231aba000f5a4583dd9f67057aadb68c3eca99d
>> >> > Author: Nicholas Piggin <npiggin@gmail.com>
>> >> > Date:   Fri Jul 27 21:48:17 2018 +1000
>> >> >
>> >> >     powerpc/64s: Fix page table fragment refcount race vs speculati=
ve references
>> >> >
>> >> >     The page table fragment allocator uses the main page refcount r=
acily
>> >> >     with respect to speculative references. A customer observed a B=
UG due
>> >> >     to page table page refcount underflow in the fragment allocator=
. This
>> >> >     can be caused by the fragment allocator set_page_count stomping=
 on a
>> >> >     speculative reference, and then the speculative failure handler
>> >> >     decrements the new reference, and the underflow eventually pops=
 when
>> >> >     the page tables are freed.
>> >> >
>> >> >     Fix this by using a dedicated field in the struct page for the =
page
>> >> >     table fragment allocator.
>> >> >
>> >> >     Fixes: 5c1f6ee9a31c ("powerpc: Reduce PTE table memory wastage"=
)
>> >> >     Cc: stable@vger.kernel.org # v3.10+
>> >>=20
>> >> That's the commit that added the BUG_ON(), so prior to that you won't
>> >> see the crash.
>> >=20
>> > Right, but the commit says it fixes page table page refcount underflow=
 by
>> > introducing a new field &page->pt_frag_refcount. Now we are hitting th=
e underflow
>> > for this pt_frag_refcount.
>>=20
>> The fixed underflow is caused by a bug (race on page count) that got=20
>> fixed by that patch. You are hitting a different underflow here. It's
>> not certain my patch caused it, I'm just trying to reproduce now.
>=20
> Ok.

Can't reproduce I'm afraid, tried adding and removing 8GB memory from a
4GB guest (via host adding / removing memory device), and it just works.

It's likely to be an edge case like an off by one or rounding error
that just happens to trigger in your config. Might be easiest if you
could test with a debug patch.

Thanks,
Nick

=