Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 27 Oct 2022 11:07:56 -0400
From:   Johannes Weiner <hannes@cmpxchg.org>
To:     Mike Kravetz <mike.kravetz@oracle.com>
Cc:     Rik van Riel <riel@surriel.com>, Chris Mason <clm@meta.com>,
        David Hildenbrand <david@redhat.com>,
        Axel Rasmussen <axelrasmussen@google.com>,
        Peter Xu <peterx@redhat.com>,
        James Houghton <jthoughton@google.com>, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org, kernel-team@meta.com,
        Andrew Morton <akpm@linux-foundation.org>
Subject: Re: [BUG] hugetlbfs_no_page vs MADV_DONTNEED race leading to SIGBUS
Message-ID: <Y1qezB09SaZy9pRS@cmpxchg.org>
References: <215d225585ff3c5ea90c64e6c9bdff04ab548156.camel@surriel.com>
 <Y1mMw2mt0my0qVqP@monkey>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Y1mMw2mt0my0qVqP@monkey>
Precedence: bulk

On Wed, Oct 26, 2022 at 12:38:43PM -0700, Mike Kravetz wrote:
> On 10/25/22 16:37, Rik van Riel wrote:
> > What are the reasonable alternatives here?
> > 
> > Should we see if anybody can come up with a simple solution
> > to the problem, or would it be better to just disable
> > MADV_DONTNEED on hugetlbfs for now?
> 
> Here is one thought (perhaps crazy).  Do not allow MADV_DONTNEED on
> hugetlb mappings.  This would help with the library code passed hugetlb
> mapping.  For the use cases where MADV_DONTNEED on hugetlb is desirable,
> create ?MADV_DONTNEED_HUGETLB? that only operates on hugetlb mappings.
> In this way the caller must be aware they are operating on a hugetlb
> mapping.

What could also work is implementing mlock() for hugetlb, and having
MADV_DONTNEED respect it.

That would allow libraries/general-purpose allocators to continue
using MADV_DONTNEED without being aware of the underlying memory
situation. Whoever is responsible for setting up the memory pool could
just mlock() - or not, if hugetlb overcommit is enabled - and the GP
allocator would do the right thing in both scenarios.

[ Our setup code is actually already calling mlock() on the hugetlb
  ranges. We never wanted DONTNEED to free hugetlb pages - it just
  happened to work so far because DONTNEED wasn't implemented for
  them. If both DONTNEED and mlock() were implemented, we'd be good. ]

Johannes