Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3;
From:   Huan Yang <link@vivo.com>
To:     Yu Zhao <yuzhao@google.com>, Steven Rostedt <rostedt@goodmis.org>,
        Masami Hiramatsu <mhiramat@kernel.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        Huan Yang <link@vivo.com>,
        Suren Baghdasaryan <surenb@google.com>,
        Vlastimil Babka <vbabka@suse.cz>, linux-kernel@vger.kernel.org,
        linux-trace-kernel@vger.kernel.org, linux-mm@kvack.org,
        bpf@vger.kernel.org
Cc:     opensource.kernel@vivo.com
Subject: [PATCH 2/2] mm: multi-gen lru: fix stat count
Date:   Wed, 18 Oct 2023 16:21:00 +0800
Message-Id: <20231018082104.3918770-3-link@vivo.com>
In-Reply-To: <20231018082104.3918770-1-link@vivo.com>
References: <20231018082104.3918770-1-link@vivo.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?1mnGyRBonP7nfJzGGd0Um/FvTMR58vdZaxRHqIIFzgsilsKyy5clAdGYSP1d?=
 =?us-ascii?Q?hvgZwe4sCfCVXTq8JZ27b+qjTHILP5JFbK3s5ZV4qggD0iS/AHRNWwQSW+Fx?=
 =?us-ascii?Q?nzmogV3PxCcXzGiI5KjTknsvMDYNP5pMjT/E1waJx1+fTtHV2oNNpGsLOuQk?=
 =?us-ascii?Q?1RGLsldoyweJoLFLTF/ofHzyGxLzZpkfVhT1AGLciZWiybkUkiqXGqVinDB8?=
 =?us-ascii?Q?0iaVQQ/1dwznJFQq8A4IlJgie3sEmAkMMsANTT/9xGAvdYezSZ114BofuBXw?=
 =?us-ascii?Q?aul0yOkSMZfBX9z29b/8tSPjdnqfXjB0Xxtm9gw6wRePyopGHnZNNr9tcxCC?=
 =?us-ascii?Q?ojrBI5aGdEamZbLCCuWKuDO8bEtXNfFlwEG3OqUEARRYX2jGwvr1hoj5ceX9?=
 =?us-ascii?Q?zohH3JIeFQDOD1N2c+YkZZUYrBUgkTvIZqlukvhjEJmrrsc/mTOmvz8m9LLP?=
 =?us-ascii?Q?wQ/VzkC6wHUX0EBN9W26wWtDewB/Mq6RVOHe2I2KtyfhYBNvGVKhpKsyLY+8?=
 =?us-ascii?Q?cJUYMkLaaAIWiCK2GqREghKJEBhn0gMGLVa1RocUoAtsHm2IxN6DTJ9mvZWi?=
 =?us-ascii?Q?/GyGe3NUdAiABohIRWF3zn/Igfgi0yHAL1msVdm3CB5P7kmXlsUj3AFP/oFz?=
 =?us-ascii?Q?zx3vRa7zf03owoIXsrUf8kHBca+T7WJ9L/YR101sb7P/wA6XV1SpufKFKpa5?=
 =?us-ascii?Q?tINUWV804zq1kwQHjbR0M/7TND8Swmi3e22mQYR/oLcnMln8suqut0kP0JkX?=
 =?us-ascii?Q?2BsEs1xNWPsz4BIqi0rkBGMsBl2tAGdTFKnIoybqFCOVJF2ZddBbLcikk6vv?=
 =?us-ascii?Q?V3n6nT68lDQL1gMCi/5ErV/J5KTt9hVihk06Rd6FjxHootOd5Beh2zyGIZA7?=
 =?us-ascii?Q?vVzIUCR/lE3NPjJLZ4ooQ0LTICL5y526mtyQA/MTaQn7CWFF7O9I+5uepj8C?=
 =?us-ascii?Q?dPT7X6wEn7aQ7L/FB1C4Fg2tFVIDpHZR6yA1vzkFywmKONvwRzoSBqflx/PA?=
 =?us-ascii?Q?vSm1MDGsZjtCGOTWWhmwzXxz4fDBJc1nmqRwltsM5mdGbkFTH83EeblFu8Qp?=
 =?us-ascii?Q?i8ESswdFs7f1WHkzYVSmo5YKc8JkyS70WtJ31EOFUuxQzpw3IYW3yMRYXLh2?=
 =?us-ascii?Q?BSfe4fw0v6ChAkJl1LEfEntQI7aCMbeHa+fqfeJuHvKzeShQXqmy9PgrkLh+?=
 =?us-ascii?Q?QZYugvEjS4npEwYqXcvNoGpPsDFvLHnLa6dSxXmkEpB/O7ZxqVCih79zJxqM?=
 =?us-ascii?Q?YTonxev19RPF6VFuiL5nMVPZjsyd+SGWNx5bhaiIqG7yN0gbNJZMeF4P/oCz?=
 =?us-ascii?Q?DaoQY1mB3kb229llzSewyzoHsbTs6JQYDuFY0cPzqEaXJOpgcdVyjeYLaiiN?=
 =?us-ascii?Q?03x8rGHh2BkjnU5RBwJ4n4eK/iPbB1AgdJTDnqrjAJSt7y38Wunn141Gi9JC?=
 =?us-ascii?Q?lhM1Mrn06uhg449iq75GqZLrvWtjLOFXfujGBxsubClRIqYMAJgyzKsRMu46?=
 =?us-ascii?Q?o+8m/z+WyFIV9uIS8MtXpFdZEZr0Q6haNoJi8WwrRA0f7kg4zFeZfdw1Fatt?=
 =?us-ascii?Q?IfA41MX+k1rH7eZA03YoG96EKSYTaSkxP9Fm06/9?=
X-OriginatorOrg: vivo.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9545bd03-848c-4418-0106-08dbcfb351cc
X-MS-Exchange-CrossTenant-AuthSource: PUZPR06MB5676.apcprd06.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Oct 2023 08:22:08.2941
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: humrRbJhtzRz7mHw3ImA04YdvnTUCHIFeVefqEAMtiz6oeqaw/URu+bmPYOWCWlYnOX1btJyeZPTSBYDOvXzJg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PSAPR06MB4391
X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no
	version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Wed, 18 Oct 2023 01:22:31 -0700 (PDT)

For multi-gen lru reclaim in evict_folios, like shrink_inactive_list,
gather folios which isolate to reclaim, and invoke shirnk_folio_list.

But, when complete shrink, it not gather shrink reclaim stat into sc,
we can't get info like nr_dirty\congested in reclaim, and then
control writeback, dirty number and mark as LRUVEC_CONGESTED, or
just bpf trace shrink and get correct sc stat.

This patch fix this by simple copy code from shrink_inactive_list when
end of shrink list.

Signed-off-by: Huan Yang <link@vivo.com>
---
 mm/vmscan.c | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/mm/vmscan.c b/mm/vmscan.c
index 21099b9f21e0..88d1d586aea5 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -4593,6 +4593,41 @@ static int evict_folios(struct lruvec *lruvec, struct scan_control *sc, int swap
 	 */
 	nr_taken = sc->nr_scanned - nr_taken;
 
+	/*
+	 * If dirty folios are scanned that are not queued for IO, it
+	 * implies that flushers are not doing their job. This can
+	 * happen when memory pressure pushes dirty folios to the end of
+	 * the LRU before the dirty limits are breached and the dirty
+	 * data has expired. It can also happen when the proportion of
+	 * dirty folios grows not through writes but through memory
+	 * pressure reclaiming all the clean cache. And in some cases,
+	 * the flushers simply cannot keep up with the allocation
+	 * rate. Nudge the flusher threads in case they are asleep.
+	 */
+	if (unlikely(stat.nr_unqueued_dirty == nr_taken)) {
+		wakeup_flusher_threads(WB_REASON_VMSCAN);
+		/*
+		 * For cgroupv1 dirty throttling is achieved by waking up
+		 * the kernel flusher here and later waiting on folios
+		 * which are in writeback to finish (see shrink_folio_list()).
+		 *
+		 * Flusher may not be able to issue writeback quickly
+		 * enough for cgroupv1 writeback throttling to work
+		 * on a large system.
+		 */
+		if (!writeback_throttling_sane(sc))
+			reclaim_throttle(pgdat, VMSCAN_THROTTLE_WRITEBACK);
+	}
+
+	sc->nr.dirty += stat.nr_dirty;
+	sc->nr.congested += stat.nr_congested;
+	sc->nr.unqueued_dirty += stat.nr_unqueued_dirty;
+	sc->nr.writeback += stat.nr_writeback;
+	sc->nr.immediate += stat.nr_immediate;
+	sc->nr.taken += nr_taken;
+	if (type)
+		sc->nr.file_taken += nr_taken;
+
 	sc->nr_reclaimed += total_reclaimed;
 	trace_mm_vmscan_lru_shrink_inactive(pgdat->node_id, nr_taken,
 					     total_reclaimed, &stat,
-- 
2.34.1