Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Content-Type: text/plain;
        charset=us-ascii
MIME-Version: 1.0
Subject: Re: [PATCH v2 1/2] hugetlb: Do not clear hugetlb dtor until
 allocating vmemmap
From:   Muchun Song <muchun.song@linux.dev>
In-Reply-To: <20230718004942.113174-2-mike.kravetz@oracle.com>
Date:   Wed, 19 Jul 2023 10:34:36 +0800
Cc:     Linux Memory Management List <linux-mm@kvack.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Jiaqi Yan <jiaqiyan@google.com>,
        Naoya Horiguchi <naoya.horiguchi@linux.dev>,
        Muchun Song <songmuchun@bytedance.com>,
        Miaohe Lin <linmiaohe@huawei.com>,
        Axel Rasmussen <axelrasmussen@google.com>,
        James Houghton <jthoughton@google.com>,
        Michal Hocko <mhocko@suse.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        stable@vger.kernel.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <4015AB29-76AD-4CA7-89C4-5916E2F8670B@linux.dev>
References: <20230718004942.113174-1-mike.kravetz@oracle.com>
 <20230718004942.113174-2-mike.kravetz@oracle.com>
To:     Mike Kravetz <mike.kravetz@oracle.com>
Precedence: bulk


> On Jul 18, 2023, at 08:49, Mike Kravetz <mike.kravetz@oracle.com> =
wrote:
>=20
> Freeing a hugetlb page and releasing base pages back to the underlying
> allocator such as buddy or cma is performed in two steps:
> - remove_hugetlb_folio() is called to remove the folio from hugetlb
>  lists, get a ref on the page and remove hugetlb destructor.  This
>  all must be done under the hugetlb lock.  After this call, the page
>  can be treated as a normal compound page or a collection of base
>  size pages.
> - update_and_free_hugetlb_folio() is called to allocate vmemmap if
>  needed and the free routine of the underlying allocator is called
>  on the resulting page.  We can not hold the hugetlb lock here.
>=20
> One issue with this scheme is that a memory error could occur between
> these two steps.  In this case, the memory error handling code treats
> the old hugetlb page as a normal compound page or collection of base
> pages.  It will then try to SetPageHWPoison(page) on the page with an
> error.  If the page with error is a tail page without vmemmap, a write
> error will occur when trying to set the flag.
>=20
> Address this issue by modifying remove_hugetlb_folio() and
> update_and_free_hugetlb_folio() such that the hugetlb destructor is =
not
> cleared until after allocating vmemmap.  Since clearing the destructor
> requires holding the hugetlb lock, the clearing is done in
> remove_hugetlb_folio() if the vmemmap is present.  This saves a
> lock/unlock cycle.  Otherwise, destructor is cleared in
> update_and_free_hugetlb_folio() after allocating vmemmap.
>=20
> Note that this will leave hugetlb pages in a state where they are =
marked
> free (by hugetlb specific page flag) and have a ref count.  This is =
not
> a normal state.  The only code that would notice is the memory error
> code, and it is set up to retry in such a case.
>=20
> A subsequent patch will create a routine to do bulk processing of
> vmemmap allocation.  This will eliminate a lock/unlock cycle for each
> hugetlb page in the case where we are freeing a large number of pages.
>=20
> Fixes: ad2fa3717b74 ("mm: hugetlb: alloc the vmemmap pages associated =
with each HugeTLB page")
> Cc: <stable@vger.kernel.org>
> Signed-off-by: Mike Kravetz <mike.kravetz@oracle.com>

Reviewed-by: Muchun Song <songmuchun@bytedance.com>

Thanks.