Received: by 2002:a6b:fb09:0:0:0:0:0 with SMTP id h9csp194772iog; Sun, 12 Jun 2022 23:57:40 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwd74QXLgmKcKDlo3p/HGiXX97cwOJeyYSKyHf8pdkpLEmon5fRwdoPn+dUzNP5Ldnf2Pmc X-Received: by 2002:a17:907:1c87:b0:6f0:29ea:cc01 with SMTP id nb7-20020a1709071c8700b006f029eacc01mr52004775ejc.671.1655103459819; Sun, 12 Jun 2022 23:57:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1655103459; cv=none; d=google.com; s=arc-20160816; b=IPhDCagxmVp4WCMU27C/V7RYsFtvS5H9fQ4CdlJ+73WcII5z3PwtRKmtn3wvsqP4q2 3iiuIZ9Vq2gWy/K7KudvbrP1XmNe79pg6p316pNMuY7DT+TK4+Vn+AP4fhb+1Ywyo1+C I80mJyv059ixNidaKlNHht+u6nx8LileNrOE4Ts87qVSjyCeW0H8aXKCvkZeqwaJnsW7 QSmnBGrePVlCi0McRTH/ULGTMTs8H9uI9gzIuKMAnQDMYBffaTbNLKs3NEadmMsnTj/j Xr9ZRumQKEgP1MXWTYE4g9CCmBb+AvKEAh7MacoCBI1DcezBqiIxY8z/RVScRo7+333J 2/ow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=XLBHCSl3x40n/JirLCrBjaqZu+zJJs2ufPMKi7d2Sww=; b=MFV5K8fRMBEHDho1AH3dGzSo68uxZdJQ1UE8OjoBvA69kItF7ErGKAAEx48EH54IVg ASG4r8++eo7kcKhSvgavtIzYx0jpqmo8ORPQ9pZEcd1jldVFVgV5APOIIdRIKLPUGIdm uMWX8svwTyIZFMBnZ3c7oH/xvyt1G7nMCkVINroyFUNN+KX47MHB6ek/Q3Yv8X280URL m3JUDjFWrmi6a6FXp9sBWMtzt/R0vg+dO/opOkvWg+TGNf3+oLLppYwlJ4BFLifTAnI2 ez35MKIJy5lTQS1hDRE1BdXNf3cQPTqM6n1GIAnPFUa6lkM5ifTSiSK/n3psnArAY3fC dnaQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=oyts69b4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l16-20020a056402255000b0043156cbbbf3si8316953edb.612.2022.06.12.23.57.15; Sun, 12 Jun 2022 23:57:39 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=oyts69b4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235364AbiFMGfl (ORCPT + 99 others); Mon, 13 Jun 2022 02:35:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38450 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234910AbiFMGfg (ORCPT ); Mon, 13 Jun 2022 02:35:36 -0400 Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 89AD3B872 for ; Sun, 12 Jun 2022 23:35:35 -0700 (PDT) Received: by mail-pl1-x62e.google.com with SMTP id d5so1693448plo.12 for ; Sun, 12 Jun 2022 23:35:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XLBHCSl3x40n/JirLCrBjaqZu+zJJs2ufPMKi7d2Sww=; b=oyts69b4avBW6IfgVF+4QpgyMTlLyDvTK8H8cgqFmeMxrl7bKHSPH0QcLfPSUXkpFx nvC3JmFuxzPBTZ08riXpF3Hnn0WwSf0Xz5YoNlEhowi0FwIsSdFz3JNWmEg73Mu8zFZL yTHQ74HZI/9sqE+udr02AltX1Wk3vQDAL+J9nK9PLL2UcKGGnsTrwzNPqPTZC0uucGgB SIfPr0Zg0Vv+ShY5B1ecsFgDlSe6swxvd7QQA3/qQk0QLSHfPE2ri/ZSGcRQNsI0mcpp c9v+ELelY0gKRbnsL7BJ+AN8XbTC/qE5w/gmOik/HPgrmqeYOvTpOOBkPbC19WeXC67S 4ytg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XLBHCSl3x40n/JirLCrBjaqZu+zJJs2ufPMKi7d2Sww=; b=YKKzBrnp+ozgzo8UK+O3XUhTNuFYxBEKiuPewxd20eq9t07poWRE+37k+fDKpn7ePY /6fsm5c2E4KskY9CHlRKCiGPKSimqU5nBrgoQz39tIbh/43C1GybJLGHoE81tn6u43Aw pS49GXZEeguDevx0nioJ+6/EFqbkFIuoKwg/8enZuJvjHvBL+r4REbjCQkImHkpBu+1I l2lBAqrVbYYGmLUSlAEGPeL3et0Erxce9pPyNjf3s2zIJR0wZ0fRCjbwUgWJ2DrFN1MN +5Q8kbaF0RAXAoJLoL6OoqcurqaEtB0ifsKFEiV6TUB2pac7TTyTlTkEuG5vUiuOiGxO O0cQ== X-Gm-Message-State: AOAM532PPWPo43z7en4rFdJw0SR/tK/zZ6jthjLhRzGaitD+X+V1Lpd6 o8jZkUGdeFgWek9tagtWEgl4Sg== X-Received: by 2002:a17:902:e742:b0:166:4d34:3be8 with SMTP id p2-20020a170902e74200b001664d343be8mr53370044plf.140.1655102135058; Sun, 12 Jun 2022 23:35:35 -0700 (PDT) Received: from FVFYT0MHHV2J.bytedance.net ([139.177.225.255]) by smtp.gmail.com with ESMTPSA id v3-20020aa799c3000000b0051bc538baadsm4366554pfi.184.2022.06.12.23.35.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 12 Jun 2022 23:35:34 -0700 (PDT) From: Muchun Song To: mike.kravetz@oracle.com, david@redhat.com, akpm@linux-foundation.org, corbet@lwn.net Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Muchun Song Subject: [PATCH 2/6] mm: hugetlb_vmemmap: optimize vmemmap_optimize_mode handling Date: Mon, 13 Jun 2022 14:35:08 +0800 Message-Id: <20220613063512.17540-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.32.1 (Apple Git-133) In-Reply-To: <20220613063512.17540-1-songmuchun@bytedance.com> References: <20220613063512.17540-1-songmuchun@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org We hold an another reference to hugetlb_optimize_vmemmap_key when making vmemmap_optimize_mode on, because we use static_key to tell memory_hotplug that memory_hotplug.memmap_on_memory should be overridden. However, this rule has gone when we have introduced SECTION_CANNOT_OPTIMIZE_VMEMMAP. Therefore, we could simplify vmemmap_optimize_mode handling by not holding an another reference to hugetlb_optimize_vmemmap_key. Signed-off-by: Muchun Song --- include/linux/page-flags.h | 6 ++--- mm/hugetlb_vmemmap.c | 65 +++++----------------------------------------- 2 files changed, 9 insertions(+), 62 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index b8b992cb201c..da7ccc3b16ad 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -200,8 +200,7 @@ enum pageflags { #ifndef __GENERATING_BOUNDS_H #ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP -DECLARE_STATIC_KEY_MAYBE(CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP_DEFAULT_ON, - hugetlb_optimize_vmemmap_key); +DECLARE_STATIC_KEY_FALSE(hugetlb_optimize_vmemmap_key); /* * If the feature of optimizing vmemmap pages associated with each HugeTLB @@ -221,8 +220,7 @@ DECLARE_STATIC_KEY_MAYBE(CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP_DEFAULT_ON, */ static __always_inline const struct page *page_fixed_fake_head(const struct page *page) { - if (!static_branch_maybe(CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP_DEFAULT_ON, - &hugetlb_optimize_vmemmap_key)) + if (!static_branch_unlikely(&hugetlb_optimize_vmemmap_key)) return page; /* diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index e20a7082f2f8..132dc83f0130 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -23,42 +23,15 @@ #define RESERVE_VMEMMAP_NR 1U #define RESERVE_VMEMMAP_SIZE (RESERVE_VMEMMAP_NR << PAGE_SHIFT) -enum vmemmap_optimize_mode { - VMEMMAP_OPTIMIZE_OFF, - VMEMMAP_OPTIMIZE_ON, -}; - -DEFINE_STATIC_KEY_MAYBE(CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP_DEFAULT_ON, - hugetlb_optimize_vmemmap_key); +DEFINE_STATIC_KEY_FALSE(hugetlb_optimize_vmemmap_key); EXPORT_SYMBOL(hugetlb_optimize_vmemmap_key); -static enum vmemmap_optimize_mode vmemmap_optimize_mode = +static bool vmemmap_optimize_enabled = IS_ENABLED(CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP_DEFAULT_ON); -static void vmemmap_optimize_mode_switch(enum vmemmap_optimize_mode to) -{ - if (vmemmap_optimize_mode == to) - return; - - if (to == VMEMMAP_OPTIMIZE_OFF) - static_branch_dec(&hugetlb_optimize_vmemmap_key); - else - static_branch_inc(&hugetlb_optimize_vmemmap_key); - WRITE_ONCE(vmemmap_optimize_mode, to); -} - static int __init hugetlb_vmemmap_early_param(char *buf) { - bool enable; - enum vmemmap_optimize_mode mode; - - if (kstrtobool(buf, &enable)) - return -EINVAL; - - mode = enable ? VMEMMAP_OPTIMIZE_ON : VMEMMAP_OPTIMIZE_OFF; - vmemmap_optimize_mode_switch(mode); - - return 0; + return kstrtobool(buf, &vmemmap_optimize_enabled); } early_param("hugetlb_free_vmemmap", hugetlb_vmemmap_early_param); @@ -103,7 +76,7 @@ static unsigned int optimizable_vmemmap_pages(struct hstate *h, unsigned long pfn = page_to_pfn(head); unsigned long end = pfn + pages_per_huge_page(h); - if (READ_ONCE(vmemmap_optimize_mode) == VMEMMAP_OPTIMIZE_OFF) + if (!READ_ONCE(vmemmap_optimize_enabled)) return 0; for (; pfn < end; pfn += PAGES_PER_SECTION) { @@ -155,7 +128,6 @@ void __init hugetlb_vmemmap_init(struct hstate *h) if (!is_power_of_2(sizeof(struct page))) { pr_warn_once("cannot optimize vmemmap pages because \"struct page\" crosses page boundaries\n"); - static_branch_disable(&hugetlb_optimize_vmemmap_key); return; } @@ -176,36 +148,13 @@ void __init hugetlb_vmemmap_init(struct hstate *h) } #ifdef CONFIG_PROC_SYSCTL -static int hugetlb_optimize_vmemmap_handler(struct ctl_table *table, int write, - void *buffer, size_t *length, - loff_t *ppos) -{ - int ret; - enum vmemmap_optimize_mode mode; - static DEFINE_MUTEX(sysctl_mutex); - - if (write && !capable(CAP_SYS_ADMIN)) - return -EPERM; - - mutex_lock(&sysctl_mutex); - mode = vmemmap_optimize_mode; - table->data = &mode; - ret = proc_dointvec_minmax(table, write, buffer, length, ppos); - if (write && !ret) - vmemmap_optimize_mode_switch(mode); - mutex_unlock(&sysctl_mutex); - - return ret; -} - static struct ctl_table hugetlb_vmemmap_sysctls[] = { { .procname = "hugetlb_optimize_vmemmap", - .maxlen = sizeof(enum vmemmap_optimize_mode), + .data = &vmemmap_optimize_enabled, + .maxlen = sizeof(int), .mode = 0644, - .proc_handler = hugetlb_optimize_vmemmap_handler, - .extra1 = SYSCTL_ZERO, - .extra2 = SYSCTL_ONE, + .proc_handler = proc_dobool, }, { } }; -- 2.11.0