Received: by 2002:a05:6a10:eb17:0:0:0:0 with SMTP id hx23csp1417822pxb; Fri, 10 Sep 2021 05:37:04 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyHx/cKWysTgup5A+pe3n8TRStn7Ok9thYrRR8BE0SXuNbopt/0uB2lwY3ledBZFfDxpCmA X-Received: by 2002:a6b:f114:: with SMTP id e20mr6858544iog.41.1631277423932; Fri, 10 Sep 2021 05:37:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1631277423; cv=none; d=google.com; s=arc-20160816; b=KDR1XWA1mrFUXUJc2iKU+YCcjzA7hUB4OjSk2SgHf0uYe923gFSZEDvXROboucyn2I EAnNTZvICBKIc8QFbZJQC1YReCHUGp8ZSPxWfmDbMxLBQt9QY0Cpt5BSdlU/Yf23CjV2 uKmIsIr3+jkUwgGiW6StRBFthi2zeNbCurEbPKLp+SIZTJSYgN3lcTNgmndamSJC2IW8 BGkp2pyqAlkMpGSif6D4PfFTQTDVmrrOH5KrCFKJ+cMVs1ts0kKIXHrMYIUhdU4vbFBU dpBMzvoPtSur77Zu6S9wloXSSvRzX4oVYmJOGiV1uhnytPXEYfuZVLULmQKb0z1eoagy f/8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=kquJ+e5Dv57dUmZt5yI1mPbJavJyEKUro3ikok1WRGI=; b=a1coD5U6gZkOkjKYGwtQTeRxeHredieG+T3KKkFJOxbbR0sK1PnDeexHRFapNCjIc0 LgVfb0RwGsx2qLDB9BB3jwi2Jor6pTN7yntGa835oVdJgsmWBtIXRBq6l+f/pd8GBRnr yTD5cdNs4H+EC1iCp9q45fYLj56mhqoSuJMba/UfZ4NQSbbHubN0S+P86yOQc/3534wK LX8U4mw/PMfY18Is/RSeCbhGZtBKAEM2/u2Hc5XlYeTOk3xM6LxvFFH/xTJLKbWl+sHs JypRhHDbcTld0I0l4bmffvu6WvC/J7qQIapKCCbNgguTP/35wnL/8F1SVjHnMgIyJN5N quQA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linuxfoundation.org header.s=korg header.b=L1QxDF6s; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id r6si4753551ilb.145.2021.09.10.05.36.52; Fri, 10 Sep 2021 05:37:03 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linuxfoundation.org header.s=korg header.b=L1QxDF6s; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234520AbhIJMgM (ORCPT + 99 others); Fri, 10 Sep 2021 08:36:12 -0400 Received: from mail.kernel.org ([198.145.29.99]:53524 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233992AbhIJMfG (ORCPT ); Fri, 10 Sep 2021 08:35:06 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 6B01661209; Fri, 10 Sep 2021 12:33:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1631277235; bh=gXSUf0D006dXUYwf/5240xS5ZCfBTt2hBogLMBMt4ZQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=L1QxDF6sR4CxQHMMET2slfk24N26DRjXWSU/RCDWpnL/EB1IpqQizRhZ+ee53jUU+ TDIkX0n1JT+HMcRN0XGOym7hArYugaLCu5h/wKPik82TgdqWPWfcAB3Uk9a/yc7uns mP9mXVMSv7AEVdW+78206aiO7m65FwXseD6nXbV4= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Muchun Song , Vlastimil Babka , Oscar Salvador , David Hildenbrand , Andrew Morton , Linus Torvalds Subject: [PATCH 5.10 04/26] mm/page_alloc: speed up the iteration of max_order Date: Fri, 10 Sep 2021 14:30:08 +0200 Message-Id: <20210910122916.395703454@linuxfoundation.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210910122916.253646001@linuxfoundation.org> References: <20210910122916.253646001@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Muchun Song commit 7ad69832f37e3cea8557db6df7c793905f1135e8 upstream. When we free a page whose order is very close to MAX_ORDER and greater than pageblock_order, it wastes some CPU cycles to increase max_order to MAX_ORDER one by one and check the pageblock migratetype of that page repeatedly especially when MAX_ORDER is much larger than pageblock_order. We also should not be checking migratetype of buddy when "order == MAX_ORDER - 1" as the buddy pfn may be invalid, so adjust the condition. With the new check, we don't need the max_order check anymore, so we replace it. Also adjust max_order initialization so that it's lower by one than previously, which makes the code hopefully more clear. Link: https://lkml.kernel.org/r/20201204155109.55451-1-songmuchun@bytedance.com Fixes: d9dddbf55667 ("mm/page_alloc: prevent merging between isolated and other pageblocks") Signed-off-by: Muchun Song Acked-by: Vlastimil Babka Reviewed-by: Oscar Salvador Reviewed-by: David Hildenbrand Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman --- mm/page_alloc.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -996,7 +996,7 @@ static inline void __free_one_page(struc struct page *buddy; bool to_tail; - max_order = min_t(unsigned int, MAX_ORDER, pageblock_order + 1); + max_order = min_t(unsigned int, MAX_ORDER - 1, pageblock_order); VM_BUG_ON(!zone_is_initialized(zone)); VM_BUG_ON_PAGE(page->flags & PAGE_FLAGS_CHECK_AT_PREP, page); @@ -1009,7 +1009,7 @@ static inline void __free_one_page(struc VM_BUG_ON_PAGE(bad_range(zone, page), page); continue_merging: - while (order < max_order - 1) { + while (order < max_order) { if (compaction_capture(capc, page, order, migratetype)) { __mod_zone_freepage_state(zone, -(1 << order), migratetype); @@ -1035,7 +1035,7 @@ continue_merging: pfn = combined_pfn; order++; } - if (max_order < MAX_ORDER) { + if (order < MAX_ORDER - 1) { /* If we are here, it means order is >= pageblock_order. * We want to prevent merge between freepages on isolate * pageblock and normal pageblock. Without this, pageblock @@ -1056,7 +1056,7 @@ continue_merging: is_migrate_isolate(buddy_mt))) goto done_merging; } - max_order++; + max_order = order + 1; goto continue_merging; }