Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35;
From:   Huan Yang <link@vivo.com>
To:     Jonathan Corbet <corbet@lwn.net>,
        Andrew Morton <akpm@linux-foundation.org>,
        Johannes Weiner <hannes@cmpxchg.org>,
        Michal Hocko <mhocko@kernel.org>,
        Roman Gushchin <roman.gushchin@linux.dev>,
        Shakeel Butt <shakeelb@google.com>,
        Muchun Song <muchun.song@linux.dev>,
        Yu Zhao <yuzhao@google.com>, Peter Xu <peterx@redhat.com>,
        David Hildenbrand <david@redhat.com>,
        "T.J. Alumbaugh" <talumbau@google.com>,
        Axel Rasmussen <axelrasmussen@google.com>,
        Ryan Roberts <ryan.roberts@arm.com>,
        Kefeng Wang <wangkefeng.wang@huawei.com>,
        Suren Baghdasaryan <surenb@google.com>,
        linux-doc@vger.kernel.org (open list:DOCUMENTATION),
        linux-kernel@vger.kernel.org (open list),
        linux-mm@kvack.org (open list:MEMORY MANAGEMENT),
        cgroups@vger.kernel.org (open list:CONTROL GROUP - MEMORY RESOURCE
        CONTROLLER (MEMCG))
Cc:     opensource.kernel@vivo.com, Huan Yang <link@vivo.com>
Subject: [PATCH 2/3] mm: memcg: add per memcg "lru_gen" node
Date:   Mon,  9 Oct 2023 10:57:21 +0800
Message-Id: <20231009025726.5982-3-link@vivo.com>
In-Reply-To: <20231009025726.5982-1-link@vivo.com>
References: <20231009025726.5982-1-link@vivo.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?K4e0sOIx0Fy3x6VJaPJFC5t8L5IxkqjPddG2/OXcK/xZxvnfgKcOKwMwdlg7?=
 =?us-ascii?Q?baohOc4OJA1o8GYtHyrMjt1HIGtEkzflOct55tmDS17snV5OonwIUKgZTmY5?=
 =?us-ascii?Q?AupaM5ANUIO4p4h44EDHdjrW5bd3geeDnmJxsNh3aTce77rmwTpsZhk59gKi?=
 =?us-ascii?Q?O4SVeFP8C4uOcP/vLo527M6tMEJTQRAqvtBGzjvM3loD4QLP7ui+gc8tnwCz?=
 =?us-ascii?Q?XR5GK9BXyaWDGOWmZETgRxcibn0WvKKV3HfxG3ieTSY1i99V9GK4i2Njagxu?=
 =?us-ascii?Q?J5+B1rXqD4uuHk1Dz8ry1TlOvPkm+Z28z3bHtbshCIeWBtSsX8rcR7nySeK9?=
 =?us-ascii?Q?2UOlPLKfW8DAajjd0XpLYLlKd/gEatj4FAktmP1SqV9aGY3ERVqgiPP6ODl9?=
 =?us-ascii?Q?x0c53rHPH8fzkiWprxCewW996I0w9yk+9sOX8D3QCKvBN/ZZPK/toPHwaiOO?=
 =?us-ascii?Q?9Az9/tmQq2v/S6aP54LJOBxMOkXtQjeTJeF7YnP08MCYettWG0RYxMdK4G3f?=
 =?us-ascii?Q?IowzpG56lDJBBDxZHJ6qyaRxbvqqUMFJ/ods9Mq/Vp9Qb+9Te3YkEXGmTgvJ?=
 =?us-ascii?Q?orEb9BE6nZ89qN/m4/JoeBn9EbR0me6glwjJ8MO/00gNaHa6nZCmks5PmG3c?=
 =?us-ascii?Q?aXmx2JQFgJdKQEawyVSH2DzHzlnznMUBxF/rHOha/2t3NuNm0iSMTFQ479vO?=
 =?us-ascii?Q?T8Fhop8iyLxLfQFx5HDNFEkxSBUsnk13jxAe/wzwOq50LhPwxpCsDvOLVgBm?=
 =?us-ascii?Q?uR4xc3dAxPoLJxGFYxbv5FyjrN7XNjZk/6tbGDzT9OFg3Chv7g8Z8wy2cTZ/?=
 =?us-ascii?Q?Qk6rTyxCTMJxcAzvvTFYyix2x5/Gx4LgjxMuInhtcy0vjdjhs6FNXZs2Do2Q?=
 =?us-ascii?Q?jYhV/gY6nLZs3UkFnpoI107MLmfX2grMvc9kD8XOSwYEjxoEGZh7RhbLfq6I?=
 =?us-ascii?Q?Ijtd53wQ7WblIIqxBY4kxoqrgIlxQrhk5SSEZqPxst0WnZse+HgCqTwAKpar?=
 =?us-ascii?Q?UbMT/2eMXZICurCLnFqz/lklnZIp8RFSrGzHlV6jgBh0HH1sE69tG/mNF9q5?=
 =?us-ascii?Q?Ikk70//CbJjHfqW5Pwxk2LZSP3eLzE9Fc8OFt5fBzawhRfVgCTVX/DTJu5Uf?=
 =?us-ascii?Q?o3sTkIl27bDIXtYeni6rCusCVGmeZbqeav8viAg6dTwzDqpUHn5NoYdY6dI+?=
 =?us-ascii?Q?wBe0RnolC8K47IyUEt176ryQp7yq7LaENk7TTvtZtrQwQBuQgvzbnw56j/oi?=
 =?us-ascii?Q?Yq0muRIct8bCSVx4vHgkL+VZUn+vWTNMr9gIADOEHDWqP3MeGrp7NB4nvWdx?=
 =?us-ascii?Q?DPdiOeHL6jIEWPicu1LbKKKUvH7l9gj6tOA6KfEQe1QYJq9uhDKBQ1D71mg2?=
 =?us-ascii?Q?IV4TRKoq6LxClzgFaGFM7bRLFaLfmSXsdcRzMtr04U4TCo9x+JS7zwVHXhAQ?=
 =?us-ascii?Q?cLzswkSR01/2QI74NRZGwIpRp4b+VqLhfIONjzoM4DNk5ux2Z9RJVz/vAdFz?=
 =?us-ascii?Q?0Bt0D7pjqnsqndFlQ3DnhHftD11xfz3nCt3l02uXAx12vvnziPwqTZK3Qxml?=
 =?us-ascii?Q?F6VZOFFj0O7NUvdaGrHNtYpwZBcbjc4sFye5Ch3N?=
X-OriginatorOrg: vivo.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 19fa420f-6076-4659-15e1-08dbc873912f
X-MS-Exchange-CrossTenant-AuthSource: PUZPR06MB5676.apcprd06.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Oct 2023 02:58:08.7287
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: lrwlA/SuXWLjbv6cO5Qr0kT1PN5S4U6uG55/pMODy80tzj5k4o/S5NdrqpY5K+obd/c36LxHugHN6m7xveFyCQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: TY0PR06MB5428
X-Spam-Status: No, score=2.7 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no
	version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Sun, 08 Oct 2023 19:58:34 -0700 (PDT)
X-Spam-Level: **

From: HuanYang <link@vivo.com>

This patch add "lru_gen" node in mem cgroup both in v1 and v2.

mem_cgroup_lru_gen_show just like global node, show lru_gen info,
but always show the full info due to per memcg no need to simple output.

Just like global node, per memcg "lru_gen" node can input cmd, but we
no need to type memcgid anymore to select one memcg.

Signed-off-by: HuanYang <link@vivo.com>
---
 include/linux/mmzone.h |   4 +-
 mm/memcontrol.c        | 163 +++++++++++++++++++++++++++++++++++++++++
 mm/vmscan.c            |  37 ++++++++++
 3 files changed, 203 insertions(+), 1 deletion(-)

diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 4106fbc5b4b3..3d399ef177a4 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -553,7 +553,9 @@ void lru_gen_online_memcg(struct mem_cgroup *memcg);
 void lru_gen_offline_memcg(struct mem_cgroup *memcg);
 void lru_gen_release_memcg(struct mem_cgroup *memcg);
 void lru_gen_soft_reclaim(struct mem_cgroup *memcg, int nid);
-
+int mem_cgroup_lru_gen_cmd(char cmd, struct mem_cgroup *memcg, int nid,
+			   unsigned long seq, int swappiness,
+			   unsigned long opt);
 #else /* !CONFIG_MEMCG */
 
 #define MEMCG_NR_GENS	1
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index ecc07b47e813..56385142c5b8 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -5052,6 +5052,155 @@ static int mem_cgroup_slab_show(struct seq_file *m, void *p)
 
 static int memory_stat_show(struct seq_file *m, void *v);
 
+#ifdef CONFIG_LRU_GEN
+static ssize_t mem_cgroup_lru_gen_write(struct kernfs_open_file *of, char *buf,
+					size_t nbytes, loff_t off)
+{
+	struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of));
+	int n;
+	int end;
+	char cmd;
+	unsigned int nid;
+	unsigned long seq;
+	unsigned int swappiness = -1;
+	unsigned long opt = -1;
+	int ret;
+
+	buf = strstrip(buf);
+	n = sscanf(buf, "%c %u %lu %n %u %n %lu %n", &cmd, &nid, &seq, &end,
+		   &swappiness, &end, &opt, &end);
+	if (n < 3 || buf[end])
+		return -EINVAL;
+
+	if (nid < 0 || nid >= MAX_NUMNODES || !node_state(nid, N_MEMORY))
+		return -EINVAL;
+
+	ret = mem_cgroup_lru_gen_cmd(cmd, memcg, nid, seq, swappiness, opt);
+	if (ret)
+		return ret;
+
+	return nbytes;
+}
+
+static void __lru_gen_show_info_full(struct seq_file *m, struct lruvec *lruvec,
+				  unsigned long max_seq, unsigned long *min_seq,
+				  unsigned long seq)
+{
+	int i;
+	int type, tier;
+	int hist = lru_hist_from_seq(seq);
+	struct lru_gen_folio *lrugen = &lruvec->lrugen;
+
+	for (tier = 0; tier < MAX_NR_TIERS; tier++) {
+		seq_printf(m, "            %10d", tier);
+		for (type = 0; type < ANON_AND_FILE; type++) {
+			const char *s = "   ";
+			unsigned long n[3] = {};
+
+			if (seq == max_seq) {
+				s = "RT ";
+				n[0] = READ_ONCE(lrugen->avg_refaulted[type][tier]);
+				n[1] = READ_ONCE(lrugen->avg_total[type][tier]);
+			} else if (seq == min_seq[type] || NR_HIST_GENS > 1) {
+				s = "rep";
+				n[0] = atomic_long_read(&lrugen->refaulted[hist][type][tier]);
+				n[1] = atomic_long_read(&lrugen->evicted[hist][type][tier]);
+				if (tier)
+					n[2] = READ_ONCE(lrugen->protected[hist][type][tier - 1]);
+			}
+
+			for (i = 0; i < 3; i++)
+				seq_printf(m, " %10lu%c", n[i], s[i]);
+		}
+		seq_putc(m, '\n');
+	}
+
+	seq_puts(m, "                      ");
+	for (i = 0; i < NR_MM_STATS; i++) {
+		const char *s = "      ";
+		unsigned long n = 0;
+
+		if (seq == max_seq && NR_HIST_GENS == 1) {
+			s = "LOYNFA";
+			n = READ_ONCE(lruvec->mm_state.stats[hist][i]);
+		} else if (seq != max_seq && NR_HIST_GENS > 1) {
+			s = "loynfa";
+			n = READ_ONCE(lruvec->mm_state.stats[hist][i]);
+		}
+
+		seq_printf(m, " %10lu%c", n, s[i]);
+	}
+	seq_putc(m, '\n');
+}
+
+
+static int __lru_gen_show_info(struct seq_file *m, struct mem_cgroup *memcg, int nid)
+{
+	unsigned long seq;
+	struct lruvec *lruvec = mem_cgroup_lruvec(memcg, NODE_DATA(nid));
+	struct lru_gen_folio *lrugen = &lruvec->lrugen;
+	DEFINE_MAX_SEQ(lruvec);
+	DEFINE_MIN_SEQ(lruvec);
+	char *path = kvmalloc(PATH_MAX, GFP_KERNEL);
+
+	if (unlikely(!path))
+		return -ENOMEM;
+
+	if (nid == first_memory_node) {
+		cgroup_path(memcg->css.cgroup, path, PATH_MAX);
+		seq_printf(m, "memcg %5u %s\n", mem_cgroup_id(memcg), path);
+	}
+
+	seq_printf(m, " node %5d\n", nid);
+
+	if (max_seq >= MAX_NR_GENS)
+		seq = max_seq - MAX_NR_GENS + 1;
+	else
+		seq = 0;
+
+	for (; seq <= max_seq; seq++) {
+		int type, zone;
+		int gen = lru_gen_from_seq(seq);
+		unsigned long birth = READ_ONCE(lrugen->timestamps[gen]);
+
+		seq_printf(m, " %10lu %10u", seq, jiffies_to_msecs(jiffies - birth));
+
+		for (type = 0; type < ANON_AND_FILE; type++) {
+			unsigned long size = 0;
+			char mark = seq < min_seq[type] ? 'x' : ' ';
+
+			for (zone = 0; zone < MAX_NR_ZONES; zone++)
+				size += max(READ_ONCE(lrugen->nr_pages[gen][type][zone]), 0L);
+
+			seq_printf(m, " %10lu%c", size, mark);
+		}
+
+		seq_putc(m, '\n');
+
+
+		__lru_gen_show_info_full(m, lruvec, max_seq, min_seq, seq);
+	}
+
+	kvfree(path);
+
+	return 0;
+}
+
+static int mem_cgroup_lru_gen_show(struct seq_file *m, void *v)
+{
+	int nid, ret;
+	struct mem_cgroup *memcg = mem_cgroup_from_seq(m);
+
+	for_each_node_state(nid, N_MEMORY) {
+		ret = __lru_gen_show_info(m, memcg, nid);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+#endif
+
 static struct cftype mem_cgroup_legacy_files[] = {
 	{
 		.name = "usage_in_bytes",
@@ -5172,6 +5321,13 @@ static struct cftype mem_cgroup_legacy_files[] = {
 		.write = mem_cgroup_reset,
 		.read_u64 = mem_cgroup_read_u64,
 	},
+#ifdef CONFIG_LRU_GEN
+	{
+		.name = "lru_gen",
+		.write = mem_cgroup_lru_gen_write,
+		.seq_show = mem_cgroup_lru_gen_show,
+	},
+#endif
 	{ },	/* terminate */
 };
 
@@ -6831,6 +6987,13 @@ static struct cftype memory_files[] = {
 		.flags = CFTYPE_NS_DELEGATABLE,
 		.write = memory_reclaim,
 	},
+#ifdef CONFIG_LRU_GEN
+	{
+		.name = "lru_gen",
+		.write = mem_cgroup_lru_gen_write,
+		.seq_show = mem_cgroup_lru_gen_show,
+	},
+#endif
 	{ }	/* terminate */
 };
 
diff --git a/mm/vmscan.c b/mm/vmscan.c
index f59977964e81..4da200cda0b9 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -6072,6 +6072,43 @@ static int __process_one_cmd(char cmd, struct lruvec *lruvec, unsigned long seq,
 	return err;
 }
 
+#ifdef CONFIG_MEMCG
+int mem_cgroup_lru_gen_cmd(char cmd, struct mem_cgroup *memcg, int nid,
+			   unsigned long seq, int swappiness, unsigned long opt)
+{
+	int err;
+	struct lruvec *lruvec;
+	unsigned int flags;
+	struct blk_plug plug;
+	struct scan_control sc = {
+		.may_writepage = true,
+		.may_unmap = true,
+		.may_swap = true,
+		.reclaim_idx = MAX_NR_ZONES - 1,
+		.gfp_mask = GFP_KERNEL,
+	};
+
+	set_task_reclaim_state(current, &sc.reclaim_state);
+	flags = memalloc_noreclaim_save();
+	blk_start_plug(&plug);
+	if (!set_mm_walk(NULL, true)) {
+		err = -ENOMEM;
+		goto done;
+	}
+
+	lruvec = get_lruvec(memcg, nid);
+	err = __process_one_cmd(cmd, lruvec, seq, &sc, swappiness, opt);
+
+done:
+	clear_mm_walk();
+	blk_finish_plug(&plug);
+	memalloc_noreclaim_restore(flags);
+	set_task_reclaim_state(current, NULL);
+
+	return err;
+}
+#endif
+
 static int run_cmd(char cmd, int memcg_id, int nid, unsigned long seq,
 		   struct scan_control *sc, int swappiness, unsigned long opt)
 {
-- 
2.34.1