Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <Y3VkIdVKRuq+fO0N@x1n> <CADrL8HXixUPyTVmYMiwc11Ot5sDMsA3x7VhgXQjimJ93MSZihA@mail.gmail.com>
 <Y6NdN2ADVCcK70ym@x1n> <CADrL8HXqE3s4ckxh0OU5onkhystj=1jMTS+S7GFeiO+kwBo0QQ@mail.gmail.com>
 <Y6N9G0Y2j98V8Pnz@monkey> <Y6OEQB3dLSa083F6@x1n> <Y6OJUtVkvdptEgW7@monkey>
 <CADrL8HX997CguZWkH3vB4+cYjwLc0mYV4GkroE41bCLRqFiPpg@mail.gmail.com>
 <Y6Om/dvlt1Wl2uZw@monkey> <CADrL8HU_4ymb4XMBpujyFECwONKpWP0fgZgy7odb+M23QUx6pQ@mail.gmail.com>
 <Y6RqAbGOFEKFUeQr@x1n>
In-Reply-To: <Y6RqAbGOFEKFUeQr@x1n>
From:   James Houghton <jthoughton@google.com>
Date:   Tue, 27 Dec 2022 12:02:52 -0500
Message-ID: <CADrL8HWt0s_Ev+QRCVQqdkLbe_jX0X95A=UNYn1Ej8qpr=38ng@mail.gmail.com>
Subject: Re: [RFC PATCH v2 33/47] userfaultfd: add UFFD_FEATURE_MINOR_HUGETLBFS_HGM
To:     Peter Xu <peterx@redhat.com>
Cc:     Mike Kravetz <mike.kravetz@oracle.com>,
        Muchun Song <songmuchun@bytedance.com>,
        David Hildenbrand <david@redhat.com>,
        David Rientjes <rientjes@google.com>,
        Axel Rasmussen <axelrasmussen@google.com>,
        Mina Almasry <almasrymina@google.com>,
        "Zach O'Keefe" <zokeefe@google.com>,
        Manish Mishra <manish.mishra@nutanix.com>,
        Naoya Horiguchi <naoya.horiguchi@nec.com>,
        "Dr . David Alan Gilbert" <dgilbert@redhat.com>,
        "Matthew Wilcox (Oracle)" <willy@infradead.org>,
        Vlastimil Babka <vbabka@suse.cz>,
        Baolin Wang <baolin.wang@linux.alibaba.com>,
        Miaohe Lin <linmiaohe@huawei.com>,
        Yang Shi <shy828301@gmail.com>,
        Andrew Morton <akpm@linux-foundation.org>, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Thu, Dec 22, 2022 at 9:30 AM Peter Xu <peterx@redhat.com> wrote:
>
> On Wed, Dec 21, 2022 at 08:24:45PM -0500, James Houghton wrote:
> > Not sure what the best name for this flag is either. MADV_ENABLE_HGM
> > sounds ok. MADV_HUGETLB_HGM or MADV_HUGETLB_SMALL_PAGES could work
> > too. No need to figure it out now.
>
> One more option to consider is MADV_SPLIT (hopefully to be more generic).
>
> We already decided to reuse thp MADV_COLLAPSE, we can also introduce
> MADV_SPLIT and leave thp for later if it can be anything helpful (I
> remember we used to discuss this for thp split).
>
> For hugetlb one SPLIT should enable hgm advise bit on the vma forever.

MADV_SPLIT sounds okay to me -- we'll see how it turns out when I send
v1. However, there's an interesting API question regarding what
address userfaultfd provides. We previously required
UFFD_FEATURE_EXACT_ADDRESS when you specified
UFFD_FEATURE_MINOR_HUGETLBFS_HGM so that there was no ambiguity. Now,
we can do:

1. When MADV_SPLIT is given, userfaultfd will now round addresses to
PAGE_SIZE instead of huge_page_size(hstate), and
UFFD_FEATURE_EXACT_ADDRESS is not needed.
2. Don't change anything. A user must know to provide
UFFD_FEATURE_EXACT_ADDRESS to get the real address, otherwise they get
an (unusable) hugepage-aligned address.

I think #1 sounds fine; let me know if you disagree.

Thanks!
- James