Received-SPF: pass (google.com: domain of linux-kernel+bounces-116528-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates
 216.228.118.233 as permitted sender) receiver=protection.outlook.com;
 client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C
From: John Hubbard <jhubbard@nvidia.com>
To: Andrew Morton <akpm@linux-foundation.org>
CC: LKML <linux-kernel@vger.kernel.org>, <linux-mm@kvack.org>, John Hubbard
	<jhubbard@nvidia.com>, David Hildenbrand <david@redhat.com>, Matthew Wilcox
	<willy@infradead.org>, Mike Rapoport <rppt@kernel.org>, Theodore Ts'o
	<tytso@mit.edu>, Vishal Moola <vishal.moola@gmail.com>, Peter Collingbourne
	<pcc@google.com>
Subject: [RFC] mm: page-flags.h: remove the bias against tail pages
Date: Sun, 24 Mar 2024 21:55:19 -0700
Message-ID: <20240325045519.222458-1-jhubbard@nvidia.com>
Precedence: bulk
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Mar 2024 04:56:40.4771
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: a6191545-2444-4299-793f-08dc4c87f5b2
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.233];Helo=[mail.nvidia.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS2PEPF00003442.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4174

commit 1d798ca3f1643 ("mm: make compound_head() robust") added
page->compound_head and the associated "unlikely" check for a tail page
in compound_head():

	if (unlikely(head & 1))
		return (struct page *) (head - 1);
	return page;

That worked nicely in 2015. However, in the 8.5 years since then, things
have changed: folios and huge pages are heavily used, with more uses
coming. See for example the various THP enhancements being proposed. And
hugetlbfs remains alive and well. And large folios are being plumbed
into everything.

With that in mind, remove the "unlikely" attribute when checking for a
tail page in compound_head(), and let normal CPU branch prediction do
what it may.

Cc: David Hildenbrand <david@redhat.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Mike Rapoport (IBM) <rppt@kernel.org>
Cc: Theodore Ts'o <tytso@mit.edu>
Cc: Vishal Moola (Oracle) <vishal.moola@gmail.com>
Cc: Peter Collingbourne <pcc@google.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---

Hi,

Is this reasonable? I haven't gone out and gathered test data, because
the original patch to create this just assumed that compound pages were
uncommon, and so now it's time to stop making that assumption. I think
that's sufficient reasoning here to leave out the compiler hint, right?

thanks,
John Hubbard
NVIDIA

 include/linux/page-flags.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
index 652d77805e99..ae9509c6736c 100644
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -246,7 +246,7 @@ static inline unsigned long _compound_head(const struct page *page)
 {
 	unsigned long head = READ_ONCE(page->compound_head);
 
-	if (unlikely(head & 1))
+	if (head & 1)
 		return head - 1;
 	return (unsigned long)page_fixed_fake_head(page);
 }
-- 
2.44.0