commit 1d798ca3f1643 ("mm: make compound_head() robust") added
page->compound_head and the associated "unlikely" check for a tail page
in compound_head():
if (unlikely(head & 1))
return (struct page *) (head - 1);
return page;
That worked nicely in 2015. However, in the 8.5 years since then, things
have changed: folios and huge pages are heavily used, with more uses
coming. See for example the various THP enhancements being proposed. And
hugetlbfs remains alive and well. And large folios are being plumbed
into everything.
With that in mind, remove the "unlikely" attribute when checking for a
tail page in compound_head(), and let normal CPU branch prediction do
what it may.
Cc: David Hildenbrand <[email protected]>
Cc: Matthew Wilcox <[email protected]>
Cc: Mike Rapoport (IBM) <[email protected]>
Cc: Theodore Ts'o <[email protected]>
Cc: Vishal Moola (Oracle) <[email protected]>
Cc: Peter Collingbourne <[email protected]>
Signed-off-by: John Hubbard <[email protected]>
---
Hi,
Is this reasonable? I haven't gone out and gathered test data, because
the original patch to create this just assumed that compound pages were
uncommon, and so now it's time to stop making that assumption. I think
that's sufficient reasoning here to leave out the compiler hint, right?
thanks,
John Hubbard
NVIDIA
include/linux/page-flags.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
index 652d77805e99..ae9509c6736c 100644
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -246,7 +246,7 @@ static inline unsigned long _compound_head(const struct page *page)
{
unsigned long head = READ_ONCE(page->compound_head);
- if (unlikely(head & 1))
+ if (head & 1)
return head - 1;
return (unsigned long)page_fixed_fake_head(page);
}
--
2.44.0