Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Jinyu Tang <tjytimi@163.com>
To:     yuzhao@google.com, yuanchu@google.com
Cc:     akpm@linux-foundation.org, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org, falcon@tinylab.org,
        ttjjyystupid@163.com, Jinyu Tang <tjytimi@163.com>
Subject: [PATCH v1] mm: multi-gen LRU: reduce repeated scans in rmap
Date:   Sun, 15 Jan 2023 20:57:21 +0800
Message-Id: <20230115125721.3553-1-tjytimi@163.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk

The folio is physically and virtually contiguous. If a folio have
more than one pages, lru_gen_look_around() will run several times in
the while loop in folio_referenced_one(), but most of times is
unnecessary. Because these pages always belong to the same pmd and
vma, lru_gen_look_around() will scan the same range.

So add a variable to record the former pvmw.address to reduce
repeated scans if the pages of this folio don't sride across a pmd.

In most codes in memory management now, a folio just have one page,
the while loop in folio_referenced_one() just run one time, so this
patch actually may not reduce scans. But when other memory management
codes expand contiguous pages to a folio in the future, this patch
can be more effective.

Signed-off-by: Jinyu Tang <tjytimi@163.com>
---
 mm/rmap.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/mm/rmap.c b/mm/rmap.c
index 2ec925e5fa6a..b11fcfe812e8 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -809,6 +809,7 @@ static bool folio_referenced_one(struct folio *folio,
 	struct folio_referenced_arg *pra = arg;
 	DEFINE_FOLIO_VMA_WALK(pvmw, folio, vma, address, 0);
 	int referenced = 0;
+	unsigned long former_address = 0;
 
 	while (page_vma_mapped_walk(&pvmw)) {
 		address = pvmw.address;
@@ -825,7 +826,13 @@ static bool folio_referenced_one(struct folio *folio,
 		if (pvmw.pte) {
 			if (lru_gen_enabled() && pte_young(*pvmw.pte) &&
 			    !(vma->vm_flags & (VM_SEQ_READ | VM_RAND_READ))) {
-				lru_gen_look_around(&pvmw);
+				unsigned long pmd_now = pvmw.address & PMD_MASK;
+				unsigned long pmd_former = former_address & PMD_MASK;
+
+				if ((!former_address) || (pmd_now != pmd_former)) {
+					lru_gen_look_around(&pvmw);
+					former_address = pvmw.address;
+				}
 				referenced++;
 			}
 
-- 
2.30.2