Hi everyone,
This patchset addresses 2 issues in /proc/kpageflags.
1. We can't easily tell folio from thp, because currently both pages are
judged as thp, and
2. we see some garbage data in records of compound tail pages because
we use tail pages to store some internal data.
These issues require userspace programs to do additional work to understand
the page status, which makes situation more complicated.
This patchset tries to solve these by defining KPF_FOLIO for issue 1., and
by hiding part of page flag info on tail pages of compound pages for issue 2.
I think that technically some compound pages like thp/hugetlb/slab could be
considered as folio, but in this version KPF_FOLIO is set only on folios
in pagecache (so "folios in narrower meaning"). I'm not confident about
this choice, so if you have any idea about this, please let me know.
How we can see using tools/mm/page-types.c will change like below (only focusing
on compound pages).
Before patchset:
// anonymous thp
voffset offset len flags
...
700000000 156c00 1 ___U_l_____Ma_bH______t_____________f_d_____1
700000001 156c01 1 L__U_______Ma___T_____t_____________f_______1
700000002 156c02 1fe ___________Ma___T_____t_____________f_______1
// file thp
700000000 15d600 1 __RUDl_____M__bH______t_____________f__I____1
700000001 15d601 1 L__U_______M____T_____t_____________f_______1
700000002 15d602 1fe ___________M____T_____t_____________f_______1
// large folio
700000000 154f84 1 __RU_l_____M___H______t________P____f_____F_1
700000001 154f85 1 ________W__M____T_____t_____________f_____F_1
700000002 154f86 2 ___________M____T_____t_____________f_____F_1
700000004 14d0a4 1 __RU_l_____M___H______t________P____f_____F_1
700000005 14d0a5 1 ________W__M____T_____t_____________f_____F_1
700000006 14d0a6 2 ___________M____T_____t_____________f_____F_1
...
// free hugetlb (HVO disabled)
offset len flags
...
106a00 1 _______________H_G___________________________
106a01 1 L__U__A_________TG___________________________
106a02 1fe ________________TG___________________________
// anonymous hugetlb (HVO disabled)
700000000 157200 1 ___U_______Ma__H_G__________________f_d_____1
700000001 157201 1 L__U__A____Ma___TG__________________f_______1
700000002 157202 1fe ___________Ma___TG__________________f_______1
// free hugetlb (HVO enabled)
12a600 1 _______________H_G___________________________
12a601 1 L__U__A_________TG___________________________
12a602 3f ________________TG___________________________
12a641 1 L__U__A_________TG___________________________
12a642 3f ________________TG___________________________
...
// anonymous hugetlb (HVO enabled)
700000000 15e600 1 ___U_______Ma__H_G__________________f_d_____1
700000001 15e601 1 L__U__A____Ma___TG__________________f_______1
700000002 15e602 3e ___________Ma___TG__________________f_______1
700000040 15e640 1 ___U_______Ma___TG__________________f_d_____1
700000041 15e641 1 L__U__A____Ma___TG__________________f_______1
700000042 15e642 3e ___________Ma___TG__________________f_______1
...
// slab
flags page-count MB symbolic-flags long-symbolic-flags
0x0000000000000080 5304 20 _______S_____________________________________ slab
0x0000000000008080 1488 5 _______S_______H_____________________________ slab,compound_head
0x0000000000010081 365 1 L______S________T____________________________ locked,slab,compound_tail
0x0000000000010080 4142 16 _______S________T____________________________ slab,compound_tail
0x0000000000010180 649 2 _______SW_______T____________________________ slab,writeback,compound_tail
0x0000000000010181 474 1 L______SW_______T____________________________ locked,slab,writeback,compound_tail
0x0000000000201080 192 0 _______S____a________x_______________________ slab,anonymous,ksm
0x0000000000001080 427 1 _______S____a________________________________ slab,anonymous
0x0000000000409080 237 0 _______S____a__H______t______________________ slab,anonymous,compound_head,thp
0x0000000000411081 78 0 L______S____a___T_____t______________________ locked,slab,anonymous,compound_tail,thp
0x0000000000609080 77 0 _______S____a__H_____xt______________________ slab,anonymous,compound_head,ksm,thp
0x0000000000611081 32 0 L______S____a___T____xt______________________ locked,slab,anonymous,compound_tail,ksm,thp
0x0000000000411080 698 2 _______S____a___T_____t______________________ slab,anonymous,compound_tail,thp
0x0000000000611080 142 0 _______S____a___T____xt______________________ slab,anonymous,compound_tail,ksm,thp
0x0000000000611180 32 0 _______SW___a___T____xt______________________ slab,writeback,anonymous,compound_tail,ksm,thp
0x0000000000411181 95 0 L______SW___a___T_____t______________________ locked,slab,writeback,anonymous,compound_tail,thp
0x0000000000411180 64 0 _______SW___a___T_____t______________________ slab,writeback,anonymous,compound_tail,thp
0x0000000000611181 13 0 L______SW___a___T____xt______________________ locked,slab,writeback,anonymous,compound_tail,ksm,thp
After patchset:
// anonymous thp
700000000 117000 1 ___U_l_____Ma_bH______t_____________f_d_____1
700000001 117001 1ff ________________T_____t_____________f_______1
// file thp
700000000 118400 1 __RUDl_____M__bH______t_____________f__I____1
700000001 118401 1ff ________________T_____t_____________f_______1
// large folio
700000000 148da4 1 __RU_l_____M___H___________f___P____f_____F_1
700000001 148da5 3 ________________T__________f________f_____F_1
700000004 148da8 1 __RU_l_____M___H___________f___P____f_____F_1
700000005 148da9 3 ________________T__________f________f_____F_1
// free hugetlb (HVO disabled)
116000 1 _______________H_G___________________________
116001 1ff ________________TG___________________________
// anonymous hugetlb (HVO disabled)
700000000 116000 1 ___U_______Ma__H_G__________________f_d_____1
700000001 116001 1ff ________________TG__________________f_______1
// free hugetlb (HVO enabled)
116000 1 _______________H_G___________________________
116001 1ff ________________TG___________________________
// anonymous hugetlb (HVO enabled)
700000000 116000 1 ___U_______Ma__H_G__________________f_d_____1
700000001 116001 1ff ________________TG__________________f_______1
// slab
0x0000000000000080 5659 22 _______S_____________________________________ slab
0x0000000000008080 1644 6 _______S_______H_____________________________ slab,compound_head
0x0000000000010080 6196 24 _______S________T____________________________ slab,compound_tail
Thanks,
Naoya Horiguchi
---
Summary:
Naoya Horiguchi (5):
include/uapi/linux/kernel-page-flags.h: define KPF_FOLIO
mm: kpageflags: distinguish thp and folio
mm, kpageflags: separate code path for hugetlb pages
mm, kpageflags: fix invalid output for PageSlab
tools/mm/page-types.c: hide compound pages in non-raw mode
fs/proc/page.c | 90 +++++++++++++++++++---------------
include/uapi/linux/kernel-page-flags.h | 1 +
tools/mm/page-types.c | 3 +-
3 files changed, 53 insertions(+), 41 deletions(-)
On 10.10.23 16:27, Naoya Horiguchi wrote:
> Hi everyone,
>
> This patchset addresses 2 issues in /proc/kpageflags.
>
> 1. We can't easily tell folio from thp, because currently both pages are
> judged as thp, and
> 2. we see some garbage data in records of compound tail pages because
> we use tail pages to store some internal data.
>
> These issues require userspace programs to do additional work to understand
> the page status, which makes situation more complicated.
>
> This patchset tries to solve these by defining KPF_FOLIO for issue 1., and
> by hiding part of page flag info on tail pages of compound pages for issue 2.
>
> I think that technically some compound pages like thp/hugetlb/slab could be
> considered as folio, but in this version KPF_FOLIO is set only on folios
At least thp+hugetlb are most certainly folios. Regarding slab, I
suspect we no longer call them folios (cannot be mapped to user space).
But Im not sure about the type hierarchy.
> in pagecache (so "folios in narrower meaning"). I'm not confident about
> this choice, so if you have any idea about this, please let me know.
It does sound inconsistent. What exactly do you want to tell user space
with the new flag?
--
Cheers,
David / dhildenb
On Thu, Oct 12, 2023 at 10:33:04AM +0200, David Hildenbrand wrote:
> On 10.10.23 16:27, Naoya Horiguchi wrote:
> > Hi everyone,
> >
> > This patchset addresses 2 issues in /proc/kpageflags.
> >
> > 1. We can't easily tell folio from thp, because currently both pages are
> > judged as thp, and
> > 2. we see some garbage data in records of compound tail pages because
> > we use tail pages to store some internal data.
> >
> > These issues require userspace programs to do additional work to understand
> > the page status, which makes situation more complicated.
> >
> > This patchset tries to solve these by defining KPF_FOLIO for issue 1., and
> > by hiding part of page flag info on tail pages of compound pages for issue 2.
> >
> > I think that technically some compound pages like thp/hugetlb/slab could be
> > considered as folio, but in this version KPF_FOLIO is set only on folios
>
> At least thp+hugetlb are most certainly folios. Regarding slab, I suspect we
> no longer call them folios (cannot be mapped to user space). But Im not sure
> about the type hierarchy.
I'm not sure about the exact definition of "folio", and I think it's better
to make KPF_FOLIO set based on the definition.
"being mapped to userspace" can be one possible criteria for the definition.
But reading source code, folio_slab() and slab_folio() convert between
struct slab and struct folio, so I feel that someone might think a slab is
a kind of folio.
>
> > in pagecache (so "folios in narrower meaning"). I'm not confident about
> > this choice, so if you have any idea about this, please let me know.
>
> It does sound inconsistent. What exactly do you want to tell user space with
> the new flag?
The current most problematic behavior is to report folio as thp (order-2
pagecache page is definitely a folio but not a thp), and this is what the
new flag is intended to tell.
Thanks,
Naoya Horiguchi
On 12.10.23 17:02, Naoya Horiguchi wrote:
> On Thu, Oct 12, 2023 at 10:33:04AM +0200, David Hildenbrand wrote:
>> On 10.10.23 16:27, Naoya Horiguchi wrote:
>>> Hi everyone,
>>>
>>> This patchset addresses 2 issues in /proc/kpageflags.
>>>
>>> 1. We can't easily tell folio from thp, because currently both pages are
>>> judged as thp, and
>>> 2. we see some garbage data in records of compound tail pages because
>>> we use tail pages to store some internal data.
>>>
>>> These issues require userspace programs to do additional work to understand
>>> the page status, which makes situation more complicated.
>>>
>>> This patchset tries to solve these by defining KPF_FOLIO for issue 1., and
>>> by hiding part of page flag info on tail pages of compound pages for issue 2.
>>>
>>> I think that technically some compound pages like thp/hugetlb/slab could be
>>> considered as folio, but in this version KPF_FOLIO is set only on folios
>>
>> At least thp+hugetlb are most certainly folios. Regarding slab, I suspect we
>> no longer call them folios (cannot be mapped to user space). But Im not sure
>> about the type hierarchy.
>
> I'm not sure about the exact definition of "folio", and I think it's better
> to make KPF_FOLIO set based on the definition.
Me neither. But in any case a THP *is* a folio. So you'd have to set
that flag in any case.
And any order-0 page (i.e., anon, pagecache) is also a folio. What you
seem to imply with folio is "large folio". So KPF_FOLIO is really wrong
as far as I can tell.
> "being mapped to userspace" can be one possible criteria for the definition.
> But reading source code, folio_slab() and slab_folio() convert between
> struct slab and struct folio, so I feel that someone might think a slab is
> a kind of folio.
I keep forgetting if "folio" is just the generic term for any order-0 or
compound page, or only for some of them. I usually live in the "anon"
world, so I don't get reminded that often :)
>>> in pagecache (so "folios in narrower meaning"). I'm not confident about
>>> this choice, so if you have any idea about this, please let me know.
>>
>> It does sound inconsistent. What exactly do you want to tell user space with
>> the new flag?
>
> The current most problematic behavior is to report folio as thp (order-2
> pagecache page is definitely a folio but not a thp), and this is what the
> new flag is intended to tell.
We are currently considering calling these sub-PMD sized THPs
"small-sized THP". [1] Arguably, we're starting with the anon part where
we won't get around exposing them to the user in sysfs.
So I wouldn't immediately say that these things are not THPs. They are
not PMD-sized THP. A slab/hugetlb is certainly not a thp but a folio.
Whereby slabs can also be order-0 folios, but hugetlb can't.
Looking at other interfaces, we do expose:
include/uapi/linux/kernel-page-flags.h:#define KPF_COMPOUND_HEAD 15
include/uapi/linux/kernel-page-flags.h:#define KPF_COMPOUND_TAIL 16
So maybe we should just continue talking about compound pages or do we
have to use both terms here in this interface?
[1] https://lkml.kernel.org/r/[email protected]
--
Cheers,
David / dhildenb
On Thu, Oct 12, 2023 at 05:30:34PM +0200, David Hildenbrand wrote:
> On 12.10.23 17:02, Naoya Horiguchi wrote:
> > On Thu, Oct 12, 2023 at 10:33:04AM +0200, David Hildenbrand wrote:
> > > On 10.10.23 16:27, Naoya Horiguchi wrote:
> > > > Hi everyone,
> > > >
> > > > This patchset addresses 2 issues in /proc/kpageflags.
> > > >
> > > > 1. We can't easily tell folio from thp, because currently both pages are
> > > > judged as thp, and
> > > > 2. we see some garbage data in records of compound tail pages because
> > > > we use tail pages to store some internal data.
> > > >
> > > > These issues require userspace programs to do additional work to understand
> > > > the page status, which makes situation more complicated.
> > > >
> > > > This patchset tries to solve these by defining KPF_FOLIO for issue 1., and
> > > > by hiding part of page flag info on tail pages of compound pages for issue 2.
> > > >
> > > > I think that technically some compound pages like thp/hugetlb/slab could be
> > > > considered as folio, but in this version KPF_FOLIO is set only on folios
> > >
> > > At least thp+hugetlb are most certainly folios. Regarding slab, I suspect we
> > > no longer call them folios (cannot be mapped to user space). But Im not sure
> > > about the type hierarchy.
> >
> > I'm not sure about the exact definition of "folio", and I think it's better
> > to make KPF_FOLIO set based on the definition.
>
> Me neither. But in any case a THP *is* a folio. So you'd have to set that
> flag in any case.
OK.
>
> And any order-0 page (i.e., anon, pagecache) is also a folio. What you seem
> to imply with folio is "large folio". So KPF_FOLIO is really wrong as far as
> I can tell.
Ah, I meant "large folio" for the new flag, so it might have been better to
name it KPF_LARGE_FOLIO.
>
> > "being mapped to userspace" can be one possible criteria for the definition.
> > But reading source code, folio_slab() and slab_folio() convert between
> > struct slab and struct folio, so I feel that someone might think a slab is
> > a kind of folio.
>
> I keep forgetting if "folio" is just the generic term for any order-0 or
> compound page, or only for some of them. I usually live in the "anon" world,
> so I don't get reminded that often :)
I didn't notice that an order-0 page is also a folio.
>
>
> > > > in pagecache (so "folios in narrower meaning"). I'm not confident about
> > > > this choice, so if you have any idea about this, please let me know.
> > >
> > > It does sound inconsistent. What exactly do you want to tell user space with
> > > the new flag?
> >
> > The current most problematic behavior is to report folio as thp (order-2
> > pagecache page is definitely a folio but not a thp), and this is what the
> > new flag is intended to tell.
>
> We are currently considering calling these sub-PMD sized THPs "small-sized
> THP". [1] Arguably, we're starting with the anon part where we won't get
> around exposing them to the user in sysfs.
>
> So I wouldn't immediately say that these things are not THPs. They are not
> PMD-sized THP. A slab/hugetlb is certainly not a thp but a folio. Whereby
> slabs can also be order-0 folios, but hugetlb can't.
>
>
> Looking at other interfaces, we do expose:
>
> include/uapi/linux/kernel-page-flags.h:#define KPF_COMPOUND_HEAD 15
> include/uapi/linux/kernel-page-flags.h:#define KPF_COMPOUND_TAIL 16
>
> So maybe we should just continue talking about compound pages or do we have
> to use both terms here in this interface?
Extending the concept of thp to arbitrary size of thp sounds good to me.
If patchset [1] will be merged, then setting KPF_THP on large folios is totally
fine and one of my problem in this patchset will be automatically resolved.
So I'm thinking of not adding new flag and just focusing on garbage data issue.
Thank you very much for sharing ideas.
Naoya Horiguchi
>
> [1] https://lkml.kernel.org/r/[email protected]
>
> --
> Cheers,
>
> David / dhildenb
>
On 13.10.23 02:54, Naoya Horiguchi wrote:
> On Thu, Oct 12, 2023 at 05:30:34PM +0200, David Hildenbrand wrote:
>> On 12.10.23 17:02, Naoya Horiguchi wrote:
>>> On Thu, Oct 12, 2023 at 10:33:04AM +0200, David Hildenbrand wrote:
>>>> On 10.10.23 16:27, Naoya Horiguchi wrote:
>>>>> Hi everyone,
>>>>>
>>>>> This patchset addresses 2 issues in /proc/kpageflags.
>>>>>
>>>>> 1. We can't easily tell folio from thp, because currently both pages are
>>>>> judged as thp, and
>>>>> 2. we see some garbage data in records of compound tail pages because
>>>>> we use tail pages to store some internal data.
>>>>>
>>>>> These issues require userspace programs to do additional work to understand
>>>>> the page status, which makes situation more complicated.
>>>>>
>>>>> This patchset tries to solve these by defining KPF_FOLIO for issue 1., and
>>>>> by hiding part of page flag info on tail pages of compound pages for issue 2.
>>>>>
>>>>> I think that technically some compound pages like thp/hugetlb/slab could be
>>>>> considered as folio, but in this version KPF_FOLIO is set only on folios
>>>>
>>>> At least thp+hugetlb are most certainly folios. Regarding slab, I suspect we
>>>> no longer call them folios (cannot be mapped to user space). But Im not sure
>>>> about the type hierarchy.
>>>
>>> I'm not sure about the exact definition of "folio", and I think it's better
>>> to make KPF_FOLIO set based on the definition.
>>
>> Me neither. But in any case a THP *is* a folio. So you'd have to set that
>> flag in any case.
>
> OK.
>
>>
>> And any order-0 page (i.e., anon, pagecache) is also a folio. What you seem
>> to imply with folio is "large folio". So KPF_FOLIO is really wrong as far as
>> I can tell.
>
> Ah, I meant "large folio" for the new flag, so it might have been better to
> name it KPF_LARGE_FOLIO.
>
>>
>>> "being mapped to userspace" can be one possible criteria for the definition.
>>> But reading source code, folio_slab() and slab_folio() convert between
>>> struct slab and struct folio, so I feel that someone might think a slab is
>>> a kind of folio.
>>
>> I keep forgetting if "folio" is just the generic term for any order-0 or
>> compound page, or only for some of them. I usually live in the "anon" world,
>> so I don't get reminded that often :)
>
> I didn't notice that an order-0 page is also a folio.
>
>>
>>
>>>>> in pagecache (so "folios in narrower meaning"). I'm not confident about
>>>>> this choice, so if you have any idea about this, please let me know.
>>>>
>>>> It does sound inconsistent. What exactly do you want to tell user space with
>>>> the new flag?
>>>
>>> The current most problematic behavior is to report folio as thp (order-2
>>> pagecache page is definitely a folio but not a thp), and this is what the
>>> new flag is intended to tell.
>>
>> We are currently considering calling these sub-PMD sized THPs "small-sized
>> THP". [1] Arguably, we're starting with the anon part where we won't get
>> around exposing them to the user in sysfs.
>>
>> So I wouldn't immediately say that these things are not THPs. They are not
>> PMD-sized THP. A slab/hugetlb is certainly not a thp but a folio. Whereby
>> slabs can also be order-0 folios, but hugetlb can't.
>>
>>
>> Looking at other interfaces, we do expose:
>>
>> include/uapi/linux/kernel-page-flags.h:#define KPF_COMPOUND_HEAD 15
>> include/uapi/linux/kernel-page-flags.h:#define KPF_COMPOUND_TAIL 16
>>
>> So maybe we should just continue talking about compound pages or do we have
>> to use both terms here in this interface?
>
> Extending the concept of thp to arbitrary size of thp sounds good to me.
> If patchset [1] will be merged, then setting KPF_THP on large folios is totally
> fine and one of my problem in this patchset will be automatically resolved.
CCing Ryan.
> So I'm thinking of not adding new flag and just focusing on garbage data issue.
That sounds minimal and reasonable! Flags/values that logically belong
to the head (although are stored in the tail) should probably be exposed
along with the head. Flags that apply to the actual tail pages should
stay with the tail pages.
>
> Thank you very much for sharing ideas.
Thank you!
--
Cheers,
David / dhildenb
On Thu, Oct 12, 2023 at 05:30:34PM +0200, David Hildenbrand wrote:
> On 12.10.23 17:02, Naoya Horiguchi wrote:
> > On Thu, Oct 12, 2023 at 10:33:04AM +0200, David Hildenbrand wrote:
> > > On 10.10.23 16:27, Naoya Horiguchi wrote:
> > > > Hi everyone,
> > > >
> > > > This patchset addresses 2 issues in /proc/kpageflags.
> > > >
> > > > 1. We can't easily tell folio from thp, because currently both pages are
> > > > judged as thp, and
> > > > 2. we see some garbage data in records of compound tail pages because
> > > > we use tail pages to store some internal data.
> > > >
> > > > These issues require userspace programs to do additional work to understand
> > > > the page status, which makes situation more complicated.
> > > >
> > > > This patchset tries to solve these by defining KPF_FOLIO for issue 1., and
> > > > by hiding part of page flag info on tail pages of compound pages for issue 2.
> > > >
> > > > I think that technically some compound pages like thp/hugetlb/slab could be
> > > > considered as folio, but in this version KPF_FOLIO is set only on folios
> > >
> > > At least thp+hugetlb are most certainly folios. Regarding slab, I suspect we
> > > no longer call them folios (cannot be mapped to user space). But Im not sure
> > > about the type hierarchy.
> >
> > I'm not sure about the exact definition of "folio", and I think it's better
> > to make KPF_FOLIO set based on the definition.
>
> Me neither. But in any case a THP *is* a folio. So you'd have to set that
> flag in any case.
>
> And any order-0 page (i.e., anon, pagecache) is also a folio. What you seem
> to imply with folio is "large folio". So KPF_FOLIO is really wrong as far as
> I can tell.
Our type hierarchy is degenerate ... in both the neutral and negative
sense of the word. A folio is simply not-a-tail-page. So, as you said,
all head pages and all order-0 pages are folios.
But we're still struggling against the legacy of our "struct page is
everything" mistake, and trying to fix that too. The general term I've
chosen for this is "memdesc", but we aren't very far down the route of
disentangling the various types from either page or folio. I'd imagined
that we'd convert everything to folio, then get into splitting them out,
but at least for ptdesc and slab we've gone for the direct conversion
approach.
At some point we probably want to disentangle anon folios from file
folios, but that's a fair ways down the list, after turning folios into a
separate allocation from struct page. At least on my list ... if someone
wants to do that as a matter of urgency, I'm sure they can be accomodated.
It's not an easy task, for sure. Our needs are better expressed as
(in Java terms) Interfaces rather than subclasses. Or Traits/Generics
if you've started learning Rust.
We definitely have the concept of "mappable to userspace" which applies
to anon, file, netmem, some device driver allocations, some vmalloc
allocations, but not slab, page tables, or free memory. Those memdescs
need refcount, mapcount, dirty flag, lock flag, maybe mapping?
Then we have "managed by the LRU" which applies to anon & file only.
Those memdescs need refcount, lru, and a pile of flags.
There's definitely scope for reordering and shrinking the various
memdescs. Once they're fully separated from struct page. What we _call_
them is a separate struggle. Try to imagine how shrink_folio_list()
works if filemem & anonmem have different types ...
> > > It does sound inconsistent. What exactly do you want to tell user space with
> > > the new flag?
> >
> > The current most problematic behavior is to report folio as thp (order-2
> > pagecache page is definitely a folio but not a thp), and this is what the
> > new flag is intended to tell.
>
> We are currently considering calling these sub-PMD sized THPs "small-sized
> THP". [1] Arguably, we're starting with the anon part where we won't get
> around exposing them to the user in sysfs.
>
> So I wouldn't immediately say that these things are not THPs. They are not
> PMD-sized THP. A slab/hugetlb is certainly not a thp but a folio. Whereby
> slabs can also be order-0 folios, but hugetlb can't.
I think this is a mistake. Users expect THPs to be PMD sized. We already
have the term "large folio" in use for file-backed memory; why do we
need to invent a new term for anon large folios?
> Looking at other interfaces, we do expose:
>
> include/uapi/linux/kernel-page-flags.h:#define KPF_COMPOUND_HEAD 15
> include/uapi/linux/kernel-page-flags.h:#define KPF_COMPOUND_TAIL 16
>
> So maybe we should just continue talking about compound pages or do we have
> to use both terms here in this interface?
I don;t know how easy it's going to be to distinguish between a head
and tail page in the Glorious Future once pages and folios are separated.
>>>> It does sound inconsistent. What exactly do you want to tell user space with
>>>> the new flag?
>>>
>>> The current most problematic behavior is to report folio as thp (order-2
>>> pagecache page is definitely a folio but not a thp), and this is what the
>>> new flag is intended to tell.
>>
>> We are currently considering calling these sub-PMD sized THPs "small-sized
>> THP". [1] Arguably, we're starting with the anon part where we won't get
>> around exposing them to the user in sysfs.
>>
>> So I wouldn't immediately say that these things are not THPs. They are not
>> PMD-sized THP. A slab/hugetlb is certainly not a thp but a folio. Whereby
>> slabs can also be order-0 folios, but hugetlb can't.
>
> I think this is a mistake. Users expect THPs to be PMD sized. We already
> have the term "large folio" in use for file-backed memory; why do we
> need to invent a new term for anon large folios?
I changed my opinion two times, but I stabilized at "these are just huge
pages of different size" when it comes to user-visible features.
Handling/calling them folios internally -- especially to abstract the
page vs. compound page and how we manage/handle the metadata -- is a
reasonable thing to do, because that's what we decided to pass around.
For future reference, here is a writeup about my findings and the reason
for my opinion:
(1) OS-independent concept
Ignoring how the OS manages metadata (e.g., "struct page", "struct
folio", compound head/tail, memdesc, ...), the common term to describe a
"the smallest fixed-length contiguous block of physical memory into
which memory pages are mapped by the operating system.["[1] is a page
frame -- people usually simplify by dropping the "frame" part, so do I.
Larger pages (which we call "huge pages", FreeBSD "superpages", Windows
"large pages") can come in different sizes and were traditionally based
on architecture support, whereby architectures can support multiple ones
[1]; I think what we see is that the OS might use intermediate sizes to
manage memory more efficiently, abstracting/evolving that concept from
the actual hardware page table mapping granularity.
But the foundation is that we are dealing with "blocks of physical
memory" in a unit that is larger than the smallest page sizes. Larger pages.
[the comment about SGI IRIX on [1] is an interesting read; so are
"scattered superpages"[3]]
Users learned the difference between a "page" and a "huge page". I'm
confident that they can learn the difference between a "traditional huge
page" and a "small-sized huge page", just like they did with hugetlb
(below).
We just have to be careful with memory statistics and to default to the
traditional huge pages for now. Slowly, the term "THP" will become more
generic. Apart from that, I fail to see the big source of confusion.
Note: FreeBSD currently similarly calls these things on arm64
"medium-sized superpages", and did not invent new terms for that so far
[2].
(2) hugetlb
Traditional huge pages started out to be PMD-sized. Before 2008, we only
supported a single huge page size. Ever since, we added support for
sizes larger (gigantic) and smaller than that (cont-pte / cont-pmd).
So (a) users did not panic because we also supported huge pages that
were not PMD-sized; (b) we managed to integrate it into the existing
environment, defaulting to the old PMD-sized huge pages towards the user
but still providing configuration knobs and (c) it is natural today to
have multiple huge page sizes supported in hugetlb.
Nowadays, when somebody says that they are using hugetlb huge pages, the
first question frequently is "which huge page size?". The same will
happen with transparent huge pages I believe.
(3) THP preparation for multiple sizes
With
/sys/kernel/mm/transparent_hugepage/hpage_pmd_size
added in 2016, we already provided a way for users to query the PMD size
for THP, implying that there might be multiple sizes in the future.
Therefore, in commit 49920d28781d, Hugh already envisioned " some
transparent support for pud and pgd pages" and ended up calling it
"_pmd_size". Turns out, we want smaller THPs first, not larger ones.
(4) Metadata management
How the OS manages metadata for its memory -- and how it calls the
involved datastructures -- is IMHO an implementation detail (an
important one regarding performance, robustness and metadata overhead as
we learned, though ;) ).
We were able to introduce folios without user-visible changes. We should
be able to implement memdesc (or memory type hierarchies) without
user-visible changes -- except for some interfaces that provide access
to bare "struct page" information (classifies as debugging interfaces IMHO).
Last but not least, we ended up consistently calling these "larger than
a page" things that we map into user space "(transparent) huge page"
towards the user in toggles, stats and documentation. Fortunately we
didn't use the term "compound page" back then; it would have been a mistake.
Regarding the pagecache, we managed to not expose any toggles towards
the user, because memory waste can be better controlled. So the term
"folio" does not pop up as a toggle in /sys and /proc.
t14s: ~ $ find /sys -name "*folio*" 2> /dev/null
t14s: ~ $ find /proc -name "*folio*" 2> /dev/null
Once we want to remove the (sub)page mapcount, we'll likely have to
remove _nr_pages_mapped. To make some workloads that are sensitive to
memory consumption [4] play along when not accounting only the actually
mapped parts, we might have to introduce other ways to control that,
when "/sys/kernel/debug/fault_around_bytes" no longer does the trick.
I'm hoping we can still find ways to avoid exposing any toggles for
that; we'll see.
[1] https://en.wikipedia.org/wiki/Page_(computer_memory)
[2] https://www.freebsd.org/status/report-2022-04-2022-06/superpages/
[3] https://ieeexplore.ieee.org/document/6657040/similar#similar
[4] https://www.suse.com/support/kb/doc/?id=000019017
>
>> Looking at other interfaces, we do expose:
>>
>> include/uapi/linux/kernel-page-flags.h:#define KPF_COMPOUND_HEAD 15
>> include/uapi/linux/kernel-page-flags.h:#define KPF_COMPOUND_TAIL 16
>>
>> So maybe we should just continue talking about compound pages or do we have
>> to use both terms here in this interface?
>
> I don;t know how easy it's going to be to distinguish between a head
> and tail page in the Glorious Future once pages and folios are separated.
Probably a page-based interface would be the wrong interface for that;
fortunately, this interface has a "debugging" smell to it, so we might
be able to replace it.
--
Cheers,
David / dhildenb
On 16/10/2023 11:13, David Hildenbrand wrote:
>>>>> It does sound inconsistent. What exactly do you want to tell user space with
>>>>> the new flag?
>>>>
>>>> The current most problematic behavior is to report folio as thp (order-2
>>>> pagecache page is definitely a folio but not a thp), and this is what the
>>>> new flag is intended to tell.
>>>
>>> We are currently considering calling these sub-PMD sized THPs "small-sized
>>> THP". [1] Arguably, we're starting with the anon part where we won't get
>>> around exposing them to the user in sysfs.
>>>
>>> So I wouldn't immediately say that these things are not THPs. They are not
>>> PMD-sized THP. A slab/hugetlb is certainly not a thp but a folio. Whereby
>>> slabs can also be order-0 folios, but hugetlb can't.
>>
>> I think this is a mistake. Users expect THPs to be PMD sized. We already
>> have the term "large folio" in use for file-backed memory; why do we
>> need to invent a new term for anon large folios?
>
> I changed my opinion two times, but I stabilized at "these are just huge pages
> of different size" when it comes to user-visible features.
>
> Handling/calling them folios internally -- especially to abstract the page vs.
> compound page and how we manage/handle the metadata -- is a reasonable thing to
> do, because that's what we decided to pass around.
>
>
> For future reference, here is a writeup about my findings and the reason for my
> opinion:
>
>
> (1) OS-independent concept
>
> Ignoring how the OS manages metadata (e.g., "struct page", "struct folio",
> compound head/tail, memdesc, ...), the common term to describe a "the smallest
> fixed-length contiguous block of physical memory into which memory pages are
> mapped by the operating system.["[1] is a page frame -- people usually simplify
> by dropping the "frame" part, so do I.
>
> Larger pages (which we call "huge pages", FreeBSD "superpages", Windows "large
> pages") can come in different sizes and were traditionally based on architecture
> support, whereby architectures can support multiple ones [1]; I think what we
> see is that the OS might use intermediate sizes to manage memory more
> efficiently, abstracting/evolving that concept from the actual hardware page
> table mapping granularity.
>
> But the foundation is that we are dealing with "blocks of physical memory" in a
> unit that is larger than the smallest page sizes. Larger pages.
>
> [the comment about SGI IRIX on [1] is an interesting read; so are "scattered
> superpages"[3]]
>
> Users learned the difference between a "page" and a "huge page". I'm confident
> that they can learn the difference between a "traditional huge page" and a
> "small-sized huge page", just like they did with hugetlb (below).
>
> We just have to be careful with memory statistics and to default to the
> traditional huge pages for now. Slowly, the term "THP" will become more generic.
> Apart from that, I fail to see the big source of confusion.
>
> Note: FreeBSD currently similarly calls these things on arm64 "medium-sized
> superpages", and did not invent new terms for that so far [2].
>
>
> (2) hugetlb
>
> Traditional huge pages started out to be PMD-sized. Before 2008, we only
> supported a single huge page size. Ever since, we added support for sizes larger
> (gigantic) and smaller than that (cont-pte / cont-pmd).
>
> So (a) users did not panic because we also supported huge pages that were not
> PMD-sized; (b) we managed to integrate it into the existing environment,
> defaulting to the old PMD-sized huge pages towards the user but still providing
> configuration knobs and (c) it is natural today to have multiple huge page sizes
> supported in hugetlb.
>
> Nowadays, when somebody says that they are using hugetlb huge pages, the first
> question frequently is "which huge page size?". The same will happen with
> transparent huge pages I believe.
>
>
> (3) THP preparation for multiple sizes
>
> With
> /sys/kernel/mm/transparent_hugepage/hpage_pmd_size
> added in 2016, we already provided a way for users to query the PMD size for
> THP, implying that there might be multiple sizes in the future.
>
> Therefore, in commit 49920d28781d, Hugh already envisioned " some transparent
> support for pud and pgd pages" and ended up calling it "_pmd_size". Turns out,
> we want smaller THPs first, not larger ones.
>
>
> (4) Metadata management
>
> How the OS manages metadata for its memory -- and how it calls the involved
> datastructures -- is IMHO an implementation detail (an important one regarding
> performance, robustness and metadata overhead as we learned, though ;) ).
>
> We were able to introduce folios without user-visible changes. We should be able
> to implement memdesc (or memory type hierarchies) without user-visible changes
> -- except for some interfaces that provide access to bare "struct page"
> information (classifies as debugging interfaces IMHO).
>
>
> Last but not least, we ended up consistently calling these "larger than a page"
> things that we map into user space "(transparent) huge page" towards the user in
> toggles, stats and documentation. Fortunately we didn't use the term "compound
> page" back then; it would have been a mistake.
>
>
> Regarding the pagecache, we managed to not expose any toggles towards the user,
> because memory waste can be better controlled. So the term "folio" does not pop
> up as a toggle in /sys and /proc.
>
> t14s: ~ $ find /sys -name "*folio*" 2> /dev/null
> t14s: ~ $ find /proc -name "*folio*" 2> /dev/null
>
> Once we want to remove the (sub)page mapcount, we'll likely have to remove
> _nr_pages_mapped. To make some workloads that are sensitive to memory
> consumption [4] play along when not accounting only the actually mapped parts,
> we might have to introduce other ways to control that, when
> "/sys/kernel/debug/fault_around_bytes" no longer does the trick. I'm hoping we
> can still find ways to avoid exposing any toggles for that; we'll see.
>
>
> [1] https://en.wikipedia.org/wiki/Page_(computer_memory)
> [2] https://www.freebsd.org/status/report-2022-04-2022-06/superpages/
> [3] https://ieeexplore.ieee.org/document/6657040/similar#similar
> [4] https://www.suse.com/support/kb/doc/?id=000019017
+1 for David's reasoning.
FWIW, the way I see it, everything is a folio; a folio is an implementation
detail that neatly abstracts a physically contiguous, power-of-2 number of pages
(including the single page case). So I'm not sure how useful it is to add the
proposed KPF_FOLIO flag. The only real thing I can imagine user space using it
for would be to tell if some extent of virtual memory is physically contiguous,
and you can already do that from the PFN.
Bigger picture interface-wise, I think it is simpler and more understandable to
the user to extend an existing concept (THP) rather than invent a new one
(folios) that substantially overlaps with the existing (PMD-sized) THP concept.
That said, if you have plans in the folio roadmap that I'm not aware of, then
perhaps those would change my mind. There is a thread here [1] where we are
discussing the best way to expose "small-sized THP" (anon large folios) to user
space - Metthew if you you stong feelings, please do reply!
[1]
https://lore.kernel.org/linux-mm/[email protected]/
Thanks,
Ryan
>
>
>>
>>> Looking at other interfaces, we do expose:
>>>
>>> include/uapi/linux/kernel-page-flags.h:#define KPF_COMPOUND_HEAD 15
>>> include/uapi/linux/kernel-page-flags.h:#define KPF_COMPOUND_TAIL 16
>>>
>>> So maybe we should just continue talking about compound pages or do we have
>>> to use both terms here in this interface?
>>
>> I don;t know how easy it's going to be to distinguish between a head
>> and tail page in the Glorious Future once pages and folios are separated.
>
> Probably a page-based interface would be the wrong interface for that;
> fortunately, this interface has a "debugging" smell to it, so we might be able
> to replace it.
>
On Mon, Oct 16, 2023 at 12:36:22PM +0100, Ryan Roberts wrote:
> On 16/10/2023 11:13, David Hildenbrand wrote:
> >>>>> It does sound inconsistent. What exactly do you want to tell user space with
> >>>>> the new flag?
> >>>>
> >>>> The current most problematic behavior is to report folio as thp (order-2
> >>>> pagecache page is definitely a folio but not a thp), and this is what the
> >>>> new flag is intended to tell.
> >>>
> >>> We are currently considering calling these sub-PMD sized THPs "small-sized
> >>> THP". [1] Arguably, we're starting with the anon part where we won't get
> >>> around exposing them to the user in sysfs.
> >>>
> >>> So I wouldn't immediately say that these things are not THPs. They are not
> >>> PMD-sized THP. A slab/hugetlb is certainly not a thp but a folio. Whereby
> >>> slabs can also be order-0 folios, but hugetlb can't.
> >>
> >> I think this is a mistake. Users expect THPs to be PMD sized. We already
> >> have the term "large folio" in use for file-backed memory; why do we
> >> need to invent a new term for anon large folios?
> >
> > I changed my opinion two times, but I stabilized at "these are just huge pages
> > of different size" when it comes to user-visible features.
> >
> > Handling/calling them folios internally -- especially to abstract the page vs.
> > compound page and how we manage/handle the metadata -- is a reasonable thing to
> > do, because that's what we decided to pass around.
> >
> >
> > For future reference, here is a writeup about my findings and the reason for my
> > opinion:
> >
> >
> > (1) OS-independent concept
> >
> > Ignoring how the OS manages metadata (e.g., "struct page", "struct folio",
> > compound head/tail, memdesc, ...), the common term to describe a "the smallest
> > fixed-length contiguous block of physical memory into which memory pages are
> > mapped by the operating system.["[1] is a page frame -- people usually simplify
> > by dropping the "frame" part, so do I.
> >
> > Larger pages (which we call "huge pages", FreeBSD "superpages", Windows "large
> > pages") can come in different sizes and were traditionally based on architecture
> > support, whereby architectures can support multiple ones [1]; I think what we
> > see is that the OS might use intermediate sizes to manage memory more
> > efficiently, abstracting/evolving that concept from the actual hardware page
> > table mapping granularity.
> >
> > But the foundation is that we are dealing with "blocks of physical memory" in a
> > unit that is larger than the smallest page sizes. Larger pages.
> >
> > [the comment about SGI IRIX on [1] is an interesting read; so are "scattered
> > superpages"[3]]
> >
> > Users learned the difference between a "page" and a "huge page". I'm confident
> > that they can learn the difference between a "traditional huge page" and a
> > "small-sized huge page", just like they did with hugetlb (below).
> >
> > We just have to be careful with memory statistics and to default to the
> > traditional huge pages for now. Slowly, the term "THP" will become more generic.
> > Apart from that, I fail to see the big source of confusion.
> >
> > Note: FreeBSD currently similarly calls these things on arm64 "medium-sized
> > superpages", and did not invent new terms for that so far [2].
> >
> >
> > (2) hugetlb
> >
> > Traditional huge pages started out to be PMD-sized. Before 2008, we only
> > supported a single huge page size. Ever since, we added support for sizes larger
> > (gigantic) and smaller than that (cont-pte / cont-pmd).
> >
> > So (a) users did not panic because we also supported huge pages that were not
> > PMD-sized; (b) we managed to integrate it into the existing environment,
> > defaulting to the old PMD-sized huge pages towards the user but still providing
> > configuration knobs and (c) it is natural today to have multiple huge page sizes
> > supported in hugetlb.
> >
> > Nowadays, when somebody says that they are using hugetlb huge pages, the first
> > question frequently is "which huge page size?". The same will happen with
> > transparent huge pages I believe.
> >
> >
> > (3) THP preparation for multiple sizes
> >
> > With
> > /sys/kernel/mm/transparent_hugepage/hpage_pmd_size
> > added in 2016, we already provided a way for users to query the PMD size for
> > THP, implying that there might be multiple sizes in the future.
> >
> > Therefore, in commit 49920d28781d, Hugh already envisioned " some transparent
> > support for pud and pgd pages" and ended up calling it "_pmd_size". Turns out,
> > we want smaller THPs first, not larger ones.
> >
> >
> > (4) Metadata management
> >
> > How the OS manages metadata for its memory -- and how it calls the involved
> > datastructures -- is IMHO an implementation detail (an important one regarding
> > performance, robustness and metadata overhead as we learned, though ;) ).
> >
> > We were able to introduce folios without user-visible changes. We should be able
> > to implement memdesc (or memory type hierarchies) without user-visible changes
> > -- except for some interfaces that provide access to bare "struct page"
> > information (classifies as debugging interfaces IMHO).
> >
> >
> > Last but not least, we ended up consistently calling these "larger than a page"
> > things that we map into user space "(transparent) huge page" towards the user in
> > toggles, stats and documentation. Fortunately we didn't use the term "compound
> > page" back then; it would have been a mistake.
> >
> >
> > Regarding the pagecache, we managed to not expose any toggles towards the user,
> > because memory waste can be better controlled. So the term "folio" does not pop
> > up as a toggle in /sys and /proc.
> >
> > t14s: ~ $ find /sys -name "*folio*" 2> /dev/null
> > t14s: ~ $ find /proc -name "*folio*" 2> /dev/null
> >
> > Once we want to remove the (sub)page mapcount, we'll likely have to remove
> > _nr_pages_mapped. To make some workloads that are sensitive to memory
> > consumption [4] play along when not accounting only the actually mapped parts,
> > we might have to introduce other ways to control that, when
> > "/sys/kernel/debug/fault_around_bytes" no longer does the trick. I'm hoping we
> > can still find ways to avoid exposing any toggles for that; we'll see.
> >
> >
> > [1] https://en.wikipedia.org/wiki/Page_(computer_memory)
> > [2] https://www.freebsd.org/status/report-2022-04-2022-06/superpages/
> > [3] https://ieeexplore.ieee.org/document/6657040/similar#similar
> > [4] https://www.suse.com/support/kb/doc/?id=000019017
>
> +1 for David's reasoning.
>
> FWIW, the way I see it, everything is a folio; a folio is an implementation
> detail that neatly abstracts a physically contiguous, power-of-2 number of pages
> (including the single page case). So I'm not sure how useful it is to add the
> proposed KPF_FOLIO flag. The only real thing I can imagine user space using it
> for would be to tell if some extent of virtual memory is physically contiguous,
> and you can already do that from the PFN.
>
> Bigger picture interface-wise, I think it is simpler and more understandable to
> the user to extend an existing concept (THP) rather than invent a new one
> (folios) that substantially overlaps with the existing (PMD-sized) THP concept.
>
> That said, if you have plans in the folio roadmap that I'm not aware of, then
> perhaps those would change my mind. There is a thread here [1] where we are
> discussing the best way to expose "small-sized THP" (anon large folios) to user
> space - Metthew if you you stong feelings, please do reply!
>
> [1]
> https://lore.kernel.org/linux-mm/[email protected]/
>
> Thanks,
> Ryan
>
>
> >
> >
> >>
> >>> Looking at other interfaces, we do expose:
> >>>
> >>> include/uapi/linux/kernel-page-flags.h:#define KPF_COMPOUND_HEAD 15
> >>> include/uapi/linux/kernel-page-flags.h:#define KPF_COMPOUND_TAIL 16
> >>>
> >>> So maybe we should just continue talking about compound pages or do we have
> >>> to use both terms here in this interface?
> >>
> >> I don;t know how easy it's going to be to distinguish between a head
> >> and tail page in the Glorious Future once pages and folios are separated.
> >
> > Probably a page-based interface would be the wrong interface for that;
> > fortunately, this interface has a "debugging" smell to it, so we might be able
> > to replace it.
This interface exposes per-pfn (not per-page) data records, specifying pfn by
file offset. It does not care about distinction between head and tail.
So I don't think that we can avoid referring to tail pages even after page-to-folio
conversion is complete.
But I agree that this interface is for debugging or testing. To clarify
this, we might consider relocating this interface to a more suitable location
within debugfs, making it effectively invisible to non-debugging processes.
And maybe this could be the case also for other similar interfaces /proc/kpage*.
So all these files can be handled together to address this problem.
Thanks,
Naoya Horiguchi