Commit 53277bcf126d ("mm: support page_mapcount() on page_has_type()
pages") made it impossible to detect mapcount underflows by treating
any negative raw mapcount value as a mapcount of 0.
We perform such underflow checks in zap_present_folio_ptes() and
zap_huge_pmd(), which would currently no longer trigger.
Let's check against PAGE_MAPCOUNT_RESERVE instead by using
page_type_has_type(), like page_has_type() would, so we can still catch
some underflows.
Fixes: 53277bcf126d ("mm: support page_mapcount() on page_has_type() pages")
Signed-off-by: David Hildenbrand <[email protected]>
---
include/linux/mm.h | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/include/linux/mm.h b/include/linux/mm.h
index ef34cf54c14f..0fb8a40f82dd 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -1229,11 +1229,10 @@ static inline void page_mapcount_reset(struct page *page)
*/
static inline int page_mapcount(struct page *page)
{
- int mapcount = atomic_read(&page->_mapcount) + 1;
+ int mapcount = atomic_read(&page->_mapcount);
/* Handle page_has_type() pages */
- if (mapcount < 0)
- mapcount = 0;
+ mapcount = page_type_has_type(mapcount) ? 0 : mapcount + 1;
if (unlikely(PageCompound(page)))
mapcount += folio_entire_mapcount(page_folio(page));
--
2.44.0
On 9 Apr 2024, at 15:22, David Hildenbrand wrote:
> Commit 53277bcf126d ("mm: support page_mapcount() on page_has_type()
> pages") made it impossible to detect mapcount underflows by treating
> any negative raw mapcount value as a mapcount of 0.
>
> We perform such underflow checks in zap_present_folio_ptes() and
> zap_huge_pmd(), which would currently no longer trigger.
>
> Let's check against PAGE_MAPCOUNT_RESERVE instead by using
> page_type_has_type(), like page_has_type() would, so we can still catch
> some underflows.
>
> Fixes: 53277bcf126d ("mm: support page_mapcount() on page_has_type() pages")
> Signed-off-by: David Hildenbrand <[email protected]>
> ---
> include/linux/mm.h | 5 ++---
> 1 file changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index ef34cf54c14f..0fb8a40f82dd 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -1229,11 +1229,10 @@ static inline void page_mapcount_reset(struct page *page)
> */
> static inline int page_mapcount(struct page *page)
> {
> - int mapcount = atomic_read(&page->_mapcount) + 1;
> + int mapcount = atomic_read(&page->_mapcount);
>
> /* Handle page_has_type() pages */
> - if (mapcount < 0)
> - mapcount = 0;
> + mapcount = page_type_has_type(mapcount) ? 0 : mapcount + 1;
> if (unlikely(PageCompound(page)))
> mapcount += folio_entire_mapcount(page_folio(page));
LGTM. This could be picked up separately. Reviewed-by: Zi Yan <[email protected]>
--
Best Regards,
Yan, Zi
On Tue, Apr 09, 2024 at 09:22:44PM +0200, David Hildenbrand wrote:
> Commit 53277bcf126d ("mm: support page_mapcount() on page_has_type()
> pages") made it impossible to detect mapcount underflows by treating
> any negative raw mapcount value as a mapcount of 0.
Yes, but I don't think this is the right place to check for underflow.
We should be checking for that on modification, not on read. I think
it's more important for page_mapcount() to be fast than a debugging aid.
On 09.04.24 23:42, Matthew Wilcox wrote:
> On Tue, Apr 09, 2024 at 09:22:44PM +0200, David Hildenbrand wrote:
>> Commit 53277bcf126d ("mm: support page_mapcount() on page_has_type()
>> pages") made it impossible to detect mapcount underflows by treating
>> any negative raw mapcount value as a mapcount of 0.
>
> Yes, but I don't think this is the right place to check for underflow.
> We should be checking for that on modification, not on read.
While I don't disagree (and we'd check more instances that way, for example
deferred rmap removal), that requires a bit more churn and figuring out of
if losing some information we would have printed in print_bad_pte() is worth
that change.
> I think
> it's more important for page_mapcount() to be fast than a debugging aid.
I really don't think page_mapcount() is a good use of time for
micro-optimizations, but let's investigate:
A big hunk of code in page_mapcount() seems to be the compound handling.
The code before that (reading mapcount, checking for the condition,
conditionally setting it to 0), would generate right now:
177: 8b 42 30 mov 0x30(%rdx),%eax
17a: b9 00 00 00 00 mov $0x0,%ecx
17f: 83 c0 01 add $0x1,%eax
182: 0f 48 c1 cmovs %ecx,%eax
My variant is longer:
17b: 8b 4a 30 mov 0x30(%rdx),%ecx
17e: 81 f9 7f ff ff ff cmp $0xffffff7f,%ecx
184: 8d 41 01 lea 0x1(%rcx),%eax
187: b9 00 00 00 00 mov $0x0,%ecx
18c: 0f 4e c1 cmovle %ecx,%eax
18f: 48 8b 0a mov (%rdx),%rcx
The compiler does not seem to do the smart thing, which would
be rearranging the code to effectively be:
diff --git a/include/linux/mm.h b/include/linux/mm.h
index ef34cf54c14f..7392596882ae 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -1232,7 +1232,7 @@ static inline int page_mapcount(struct page *page)
int mapcount = atomic_read(&page->_mapcount) + 1;
/* Handle page_has_type() pages */
- if (mapcount < 0)
+ if (mapcount < PAGE_MAPCOUNT_RESERVE + 1)
mapcount = 0;
if (unlikely(PageCompound(page)))
mapcount += folio_entire_mapcount(page_folio(page));
Which would result in:
177: 8b 42 30 mov 0x30(%rdx),%eax
17a: 31 c9 xor %ecx,%ecx
17c: 83 c0 01 add $0x1,%eax
17f: 83 f8 80 cmp $0xffffff80,%eax
182: 0f 4e c1 cmovle %ecx,%eax
Same code length, one more instruction. No jumps.
I can switch to the above (essentially inlining
page_type_has_type()) for now and look into different sanity checks --
and extending the documentation around page_mapcount() behavior for
underflows -- separately.
.. unless you insist that we really have to change that immediately.
Thanks!
--
Cheers,
David / dhildenb
On 09.04.24 21:22, David Hildenbrand wrote:
> Commit 53277bcf126d ("mm: support page_mapcount() on page_has_type()
> pages") made it impossible to detect mapcount underflows by treating
> any negative raw mapcount value as a mapcount of 0.
>
> We perform such underflow checks in zap_present_folio_ptes() and
> zap_huge_pmd(), which would currently no longer trigger.
>
> Let's check against PAGE_MAPCOUNT_RESERVE instead by using
> page_type_has_type(), like page_has_type() would, so we can still catch
> some underflows.
>
> Fixes: 53277bcf126d ("mm: support page_mapcount() on page_has_type() pages")
> Signed-off-by: David Hildenbrand <[email protected]>
> ---
> include/linux/mm.h | 5 ++---
> 1 file changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index ef34cf54c14f..0fb8a40f82dd 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -1229,11 +1229,10 @@ static inline void page_mapcount_reset(struct page *page)
> */
> static inline int page_mapcount(struct page *page)
> {
> - int mapcount = atomic_read(&page->_mapcount) + 1;
> + int mapcount = atomic_read(&page->_mapcount);
>
> /* Handle page_has_type() pages */
> - if (mapcount < 0)
> - mapcount = 0;
> + mapcount = page_type_has_type(mapcount) ? 0 : mapcount + 1;
> if (unlikely(PageCompound(page)))
> mapcount += folio_entire_mapcount(page_folio(page));
>
From b49849001f3d2aad0af93cf2098065d7cbd9a959 Mon Sep 17 00:00:00 2001
From: David Hildenbrand <[email protected]>
Date: Wed, 24 Apr 2024 10:50:09 +0200
Subject: [PATCH] !fixup: mm: allow for detecting underflows with
page_mapcount() again
Let's make page_mapcount() slighly more efficient by inlining the
page_type_has_type() check.
Signed-off-by: David Hildenbrand <[email protected]>
---
include/linux/mm.h | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/include/linux/mm.h b/include/linux/mm.h
index dc33f8269fb52..cf700c5cdd58b 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -1229,10 +1229,11 @@ static inline void page_mapcount_reset(struct page *page)
*/
static inline int page_mapcount(struct page *page)
{
- int mapcount = atomic_read(&page->_mapcount);
+ int mapcount = atomic_read(&page->_mapcount) + 1;
/* Handle page_has_type() pages */
- mapcount = page_type_has_type(mapcount) ? 0 : mapcount + 1;
+ if (mapcount < PAGE_MAPCOUNT_RESERVE + 1)
+ mapcount = 0;
if (unlikely(PageCompound(page)))
mapcount += folio_entire_mapcount(page_folio(page));
--
2.44.0
--
Cheers,
David / dhildenb