Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   lipeifeng@oppo.com
To:     akpm@linux-foundation.org
Cc:     peifeng55@gmail.com, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org, 21cnbao@gmail.com,
        zhangshiming@oppo.com, lipeifeng <lipeifeng@oppo.com>
Subject: [PATCH] mm/page_alloc: give priority to free cma-pages from pcplist to buddy
Date:   Sun, 24 Apr 2022 11:27:33 +0800
Message-Id: <20220424032734.1542-1-lipeifeng@oppo.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?ecgpZcX7PCkO/mYrmRl0aQ/XGvEwWdNymSNsqLKBYSA3/2BQZLF4CX2i7QGW?=
 =?us-ascii?Q?dW+BlW4Kn2n9Tt82oid/6BCEMS7Ds4J15PqYNKOY1F1pPOQPlrmkoeODC1Ug?=
 =?us-ascii?Q?pRNL2JHise2leDnuTnf/tkisOtpdzE6UmVko8niXUKXhALlUlm4/uRCTJ5Ck?=
 =?us-ascii?Q?GXbBwzm3wkIVcUogT23lh0cpxmxx0ZXPo670VQtNcFQkBUF6ARQihCrd1Qyw?=
 =?us-ascii?Q?JbcLAEF8n9XT6cFPzTmCyUCORFqxw25AGjhMJsTbk70fXNFMPoQZDYW5Xj/m?=
 =?us-ascii?Q?Oa19Qec1UY7ek4SAEAfx4klg2GYSGyIKtSQACPOafgsn2RxamZhdv9U5PztZ?=
 =?us-ascii?Q?A/oHHeWT5VWepoTm6XlTJEsKK2pYgcewuZOY+bNBdLkXeXBCwHE0nh7ufusW?=
 =?us-ascii?Q?qWFcz2sJsqbTab+fvzWDjJ7DVFQvSOnNH3D2cfj2hvsh9goYgSuQGpEjDoxs?=
 =?us-ascii?Q?xQnEkDaqHYlSAb1hAPs30aq1QHfNJZ1aLUWlqXgHiuImTrExquoaOHQRIgL+?=
 =?us-ascii?Q?LJ1pLpZAdaVn4HpqA9b2rWd84n6V2KBE9OnseGDMgillA7mXxLRe6gLXr0Ps?=
 =?us-ascii?Q?mUfNG91sdJmYRpM4SgI2y+xuhrDCggllAnUdvA95Cv2/L0D7FiaJInyG61t3?=
 =?us-ascii?Q?7EHn1VnycEUNCjgfUUbogxRz9lIP1lcdqa1XKvr2HBtnJidVAAIwB+ECkC2b?=
 =?us-ascii?Q?kuoiMEieq35rT2Idh4qW4rtrnf6/GMjk2OvRgbX83AhGoIEm9MoLQ4Jm6bJU?=
 =?us-ascii?Q?OHyzZh4SljtZa2mU6+lNWaH0wn6qWqc+zaIJSg7A+hRjzrTCUkTEcXTbGxhV?=
 =?us-ascii?Q?/57uoTFXSDIE++Bply+5Vf3r16FlnMPsjMWKNcN6jtnivpWyRTYomMzRmPGt?=
 =?us-ascii?Q?7EaX8rCisnX6XE9lWAQHdXojs9tZPF7BzJnR84l1VyhO1mGWBh/WRi9BTJxE?=
 =?us-ascii?Q?OWP1iYCPCO6xXUqfHBVMDEpP0duoYnqzyafmkKGlNRAyDAPcbE5MgHk0QLCY?=
 =?us-ascii?Q?ytFoYKlFSLh5wp8I1/aRg0Mg9ZOcOPHgqcRHvr/ZMhUu060MOeW61EdxjtZc?=
 =?us-ascii?Q?YShzXcIOihZoaNvrh/nYlg25iymaL1p+lSq7q5M0L26VJvD3PhuPvK4HIfhj?=
 =?us-ascii?Q?xsJFvOTaMK2q7GKr4Yf0VyFCS3o4iIW6QTfkc48IKbOCddyncms/8/0JDqPr?=
 =?us-ascii?Q?Njv0s9M+NN3GAWjbHplspzvlhURIM9yIL3cNOd6yBJ63MpvRgwcppbLp+t0U?=
 =?us-ascii?Q?nigLDO/H6jj9P1hDZXgDOYE8MWR/dejJDfGjobEuDw3c4oh5JM3tJxHo0ZSH?=
 =?us-ascii?Q?P+wgYmm44bQZUzyOhdHU3ke6jStvGROoTufbyaoKGTLwuMBIJ4hn8UHN0ECC?=
 =?us-ascii?Q?ZO53KlYJVGKjbMreOvYB4ePAr27AYsgTKVTNW20vXU3549SUgPks4fEIHM/J?=
 =?us-ascii?Q?13rE/LyT2OnV30znbA/j8CGanbG/m2uwK1MFqM26zauC4BPn/dp0yfmbl5Un?=
 =?us-ascii?Q?rJHibvZVsREVPR5GsDm57wGtSoMK8caiax9sBMUj4gGMTWDK81s8I1lJ/yLK?=
 =?us-ascii?Q?4TOEuulcVZvLn1/ah6Gzft+oHm5UAsHsI6CSxa63LMCVu/d3DPgm6Hw5PCKG?=
 =?us-ascii?Q?qkuKoKSZbzIxJKVPB6dY/I0l/WhjC/s9IpvUHMFqWb2mPphCXRFowWPo8dLV?=
 =?us-ascii?Q?xmjSOeN2Z5wGkEXPOMHdMBHaibifGpCvPdZ0HrDyVeOMmpnMgsN8VTDATHb+?=
 =?us-ascii?Q?zl9eIYA2qAJtoC4JlfxUyobpNX6LLzc=3D?=
X-OriginatorOrg: oppo.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 60966777-a3ca-4092-8d8e-08da25a26fc5
X-MS-Exchange-CrossTenant-AuthSource: TY2PR02MB4431.apcprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Apr 2022 03:28:01.9150
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f1905eb1-c353-41c5-9516-62b4a54b5ee6
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: pCfj696N6ejORTUIsb9JPKt+TwZCCyI58810RUHrIssfltlRbNjq6uaZ3G/XF3RHauEise6fyefa5EvKZWvJSA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SEYPR02MB5797
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,
        SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

From: lipeifeng <lipeifeng@oppo.com>

Cma-pages will be fallback to movable pages in many scenarios.when cma
pages are freed to pcplist, we give priority to free it from pcplist
to buddy in order to  avoids cma-pages to be used as movable-pages soon
if there is enough free-movable-pages, which saves more cma-pages in
buddy to decrease pages migration when cma_alloc.

Signed-off-by: lipeifeng <lipeifeng@oppo.com>
---
 mm/page_alloc.c | 28 ++++++++++++++++++++++++----
 1 file changed, 24 insertions(+), 4 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 3589feb..69369ed 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -3372,7 +3372,7 @@ static int nr_pcp_high(struct per_cpu_pages *pcp, struct zone *zone)
 }
 
 static void free_unref_page_commit(struct page *page, unsigned long pfn,
-				   int migratetype, unsigned int order)
+				   int migratetype, unsigned int order, bool fast_free)
 {
 	struct zone *zone = page_zone(page);
 	struct per_cpu_pages *pcp;
@@ -3382,7 +3382,10 @@ static void free_unref_page_commit(struct page *page, unsigned long pfn,
 	__count_vm_event(PGFREE);
 	pcp = this_cpu_ptr(zone->per_cpu_pageset);
 	pindex = order_to_pindex(migratetype, order);
-	list_add(&page->lru, &pcp->lists[pindex]);
+	if (fast_free)
+		list_add_tail(&page->lru, &pcp->lists[pindex]);
+	else
+		list_add(&page->lru, &pcp->lists[pindex]);
 	pcp->count += 1 << order;
 	high = nr_pcp_high(pcp, zone);
 	if (pcp->count >= high) {
@@ -3400,6 +3403,7 @@ void free_unref_page(struct page *page, unsigned int order)
 	unsigned long flags;
 	unsigned long pfn = page_to_pfn(page);
 	int migratetype;
+	bool fast_free = false;
 
 	if (!free_unref_page_prepare(page, pfn, order))
 		return;
@@ -3419,9 +3423,15 @@ void free_unref_page(struct page *page, unsigned int order)
 		}
 		migratetype = MIGRATE_MOVABLE;
 	}
+	/*
+	 * Give priority to free cma-pages to buddy in order to
+	 * decrease pages migration when cma_alloc.
+	 */
+	if (migratetype == MIGRATE_CMA)
+		fast_free = true;
 
 	local_lock_irqsave(&pagesets.lock, flags);
-	free_unref_page_commit(page, pfn, migratetype, order);
+	free_unref_page_commit(page, pfn, migratetype, order, fast_free);
 	local_unlock_irqrestore(&pagesets.lock, flags);
 }
 
@@ -3459,6 +3469,8 @@ void free_unref_page_list(struct list_head *list)
 
 	local_lock_irqsave(&pagesets.lock, flags);
 	list_for_each_entry_safe(page, next, list, lru) {
+		bool fast_free = false;
+
 		pfn = page_private(page);
 		set_page_private(page, 0);
 
@@ -3467,11 +3479,19 @@ void free_unref_page_list(struct list_head *list)
 		 * to the MIGRATE_MOVABLE pcp list.
 		 */
 		migratetype = get_pcppage_migratetype(page);
+
+		/*
+		 * Give priority to free cma-pages to buddy in order to
+		 * decrease pages migration when cma_alloc.
+		 */
+		if (migratetype == MIGRATE_CMA)
+			fast_free = true;
+
 		if (unlikely(migratetype >= MIGRATE_PCPTYPES))
 			migratetype = MIGRATE_MOVABLE;
 
 		trace_mm_page_free_batched(page);
-		free_unref_page_commit(page, pfn, migratetype, 0);
+		free_unref_page_commit(page, pfn, migratetype, 0, fast_free);
 
 		/*
 		 * Guard against excessive IRQ disabled times when we get
-- 
2.7.4