Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
From:   David Hildenbrand <david@redhat.com>
Mime-Version: 1.0 (1.0)
Subject: Re: [PATCH] mm: huge_memory: a new debugfs interface for splitting THP tests.
Date:   Mon, 8 Mar 2021 22:58:54 +0100
Message-Id: <B7E9E82B-70B1-491E-99B9-B6B30AE0DE68@redhat.com>
References: <CAHbLzkpfR=xbnepbriu1b8n5ii8Kp0EgEa3D7+mU2xcS9iGpGg@mail.gmail.com>
Cc:     David Hildenbrand <david@redhat.com>, Zi Yan <ziy@nvidia.com>,
        Linux MM <linux-mm@kvack.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        linux-kselftest@vger.kernel.org,
        "Kirill A . Shutemov" <kirill.shutemov@linux.intel.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Shuah Khan <shuah@kernel.org>,
        John Hubbard <jhubbard@nvidia.com>,
        Sandipan Das <sandipan@linux.ibm.com>,
        David Rientjes <rientjes@google.com>,
        Alex Shi <alex.shi@linux.alibaba.com>
In-Reply-To: <CAHbLzkpfR=xbnepbriu1b8n5ii8Kp0EgEa3D7+mU2xcS9iGpGg@mail.gmail.com>
To:     Yang Shi <shy828301@gmail.com>
Precedence: bulk


> Am 08.03.2021 um 22:25 schrieb Yang Shi <shy828301@gmail.com>:
>=20
> =EF=BB=BFOn Mon, Mar 8, 2021 at 12:36 PM David Hildenbrand <david@redhat.c=
om> wrote:
>>=20
>>=20
>>>> Am 08.03.2021 um 21:18 schrieb Yang Shi <shy828301@gmail.com>:
>>>=20
>>> =EF=BB=BFOn Mon, Mar 8, 2021 at 11:30 AM David Hildenbrand <david@redhat=
.com> wrote:
>>>>=20
>>>>> On 08.03.21 20:11, Yang Shi wrote:
>>>>> On Mon, Mar 8, 2021 at 11:01 AM Zi Yan <ziy@nvidia.com> wrote:
>>>>>>=20
>>>>>> On 8 Mar 2021, at 13:11, David Hildenbrand wrote:
>>>>>>=20
>>>>>>> On 08.03.21 18:49, Zi Yan wrote:
>>>>>>>> On 8 Mar 2021, at 11:17, David Hildenbrand wrote:
>>>>>>>>=20
>>>>>>>>> On 08.03.21 16:22, Zi Yan wrote:
>>>>>>>>>> From: Zi Yan <ziy@nvidia.com>
>>>>>>>>>>=20
>>>>>>>>>> By writing "<pid>,<vaddr_start>,<vaddr_end>" to
>>>>>>>>>> <debugfs>/split_huge_pages_in_range_pid, THPs in the process with=
 the
>>>>>>>>>> given pid and virtual address range are split. It is used to test=

>>>>>>>>>> split_huge_page function. In addition, a selftest program is adde=
d to
>>>>>>>>>> tools/testing/selftests/vm to utilize the interface by splitting
>>>>>>>>>> PMD THPs and PTE-mapped THPs.
>>>>>>>>>=20
>>>>>>>>> Won't something like
>>>>>>>>>=20
>>>>>>>>> 1. MADV_HUGEPAGE
>>>>>>>>>=20
>>>>>>>>> 2. Access memory
>>>>>>>>>=20
>>>>>>>>> 3. MADV_NOHUGEPAGE
>>>>>>>>>=20
>>>>>>>>> Have a similar effect? What's the benefit of this?
>>>>>>>>=20
>>>>>>>> Thanks for checking the patch.
>>>>>>>>=20
>>>>>>>> No, MADV_NOHUGEPAGE just replaces VM_HUGEPAGE with VM_NOHUGEPAGE,
>>>>>>>> nothing else will be done.
>>>>>>>=20
>>>>>>> Ah, okay - maybe my memory was tricking me. There is some s390x KVM c=
ode that forces MADV_NOHUGEPAGE and force-splits everything.
>>>>>>>=20
>>>>>>> I do wonder, though, if this functionality would be worth a proper u=
ser interface (e.g., madvise), though. There might be actual benefit in havi=
ng this as a !debug interface.
>>>>>>>=20
>>>>>>> I think you aware of the discussion in https://lkml.kernel.org/r/d09=
8c392-273a-36a4-1a29-59731cdf5d3d@google.com
>>>>>>=20
>>>>>> Yes. Thanks for bringing this up.
>>>>>>=20
>>>>>>>=20
>>>>>>> If there will be an interface to collapse a THP -- "this memory area=
 is worth extra performance now by collapsing a THP if possible" -- it might=
 also be helpful to have the opposite functionality -- "this memory area is n=
ot worth a THP, rather use that somehwere else".
>>>>>>>=20
>>>>>>> MADV_HUGE_COLLAPSE vs. MADV_HUGE_SPLIT
>>>>>>=20
>>>>>> I agree that MADV_HUGE_SPLIT would be useful as the opposite of COLLA=
PSE when user might just want PAGESIZE mappings.
>>>>>> Right now, HUGE_SPLIT is implicit from mapping changes like mprotect o=
r MADV_DONTNEED.
>>>>>=20
>>>>> IMHO, it sounds not very useful. MADV_DONTNEED would split PMD for any=

>>>>> partial THP. If the range covers the whole THP, the whole THP is going=

>>>>> to be freed anyway. All other places in kernel which need split THP
>>>>> have been covered. So I didn't realize any usecase from userspace for
>>>>> just splitting PMD to PTEs.
>>>>=20
>>>> THP are a limited resource. So indicating which virtual memory regions
>>>> are not performance sensitive right now (e.g., cold pages in a databse)=

>>>> and not worth a THP might be quite valuable, no?
>>>=20
>>> Such functionality could be achieved by MADV_COLD or MADV_PAGEOUT,
>>> right? Then a subsequent call to MADV_NOHUGEPAGE would prevent from
>>> collapsing or allocating THP for that area.
>>>=20
>>=20
>> I remember these deal with optimizing swapping. Not sure how they interac=
t with THP, especially on systems without swap - I would guess they don=E2=80=
=98t as of now.
>=20
> Yes, MADV_PAGEOUT would just swap the THP or sub pages out. I think I
> just forgot to mention MADV_FREE which would be more suitable for this
> usecase.
>=20
>>=20

Can you elaborate? MADV_FREE is destructive, just like a delayed MADV_DONTNE=
ED. How would that help here?

>>>>=20
>>>> --
>>>> Thanks,
>>>>=20
>>>> David / dhildenb
>>>>=20
>>>=20
>>=20
>=20