Received: by 2002:a25:e74b:0:0:0:0:0 with SMTP id e72csp2928599ybh; Sat, 25 Jul 2020 06:02:31 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzk/ZknoPfMIJObv3sE4v1BIXBeHpudIkzOVE+0/Tda2Qt7Ws+Lreb8M8ITII+GzWs+NlcM X-Received: by 2002:a50:9f46:: with SMTP id b64mr1325337edf.298.1595682150834; Sat, 25 Jul 2020 06:02:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1595682150; cv=none; d=google.com; s=arc-20160816; b=GEhHo1yUPRWYa7QDhpxMoiaKKoqZf5k9YgwnulI9gTU9dV8OP0ikGooaToqQvuWM1f vkj8ufdkwFJpf3FKwL4vObWMMOUzddHu1aMkPjvld7Qikva2j7U1hIl4qesYemlBUXdE kUStf90rb/L7uqTUNoFEgvVa7hrUrrG472u6s1GwyT7QFKAmkuguaG1N/JF9blCLJ3Qc C1C51Z0Fu+cInxa++QtYGjdR7bFTJ+GoZJyRgd3sceO5a+tcbLY0MlJ3/RDo0QxOyOe0 sMOE4ST/BDkEH7pxF18HN00XJm+TcDazxo3SQiVFbpQBFEZ/NIizt3syTLHHckQbcDMb mCow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=kZEtC7LLJF1ocjfa6SFAYk7n+P59Cr42voFlINIwdEo=; b=opJo6RYqBIztXvA3ffZBt9ezOfiOTcnnlofmgfY6DIq1YJ4QrT4HeYJuwA0/5FDaS0 LF1HYc746HA98BZsTimaDAPolIeuryqMGqZ3YpHew2W+wqEJOSXvXNJXHNVrdm6qnXXY 84vlf30SDp+mVOb6KqEDHplUiUGe5TcLqtBkonVM2r8NQjXdWWtd4K0PRKATlKa72QIu Z3jIfgZgcDAD8HJcuhCpqxM0YiMwnFNjinVgla4WzPnQEVpX0M00kH0IWMIaQjZoGkXl u8h2TBeRD9GqyzDDIKPJquanGiz4wqfB/AKADmIGx3Csn4C/ZkENrxFKj+lsto1JZGuE NWtw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id v12si2211371eds.85.2020.07.25.06.02.08; Sat, 25 Jul 2020 06:02:30 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727942AbgGYNBL (ORCPT + 99 others); Sat, 25 Jul 2020 09:01:11 -0400 Received: from out30-56.freemail.mail.aliyun.com ([115.124.30.56]:35963 "EHLO out30-56.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727112AbgGYNAd (ORCPT ); Sat, 25 Jul 2020 09:00:33 -0400 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R181e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e01358;MF=alex.shi@linux.alibaba.com;NM=1;PH=DS;RN=20;SR=0;TI=SMTPD_---0U3l6fmD_1595682015; Received: from aliy8.localdomain(mailfrom:alex.shi@linux.alibaba.com fp:SMTPD_---0U3l6fmD_1595682015) by smtp.aliyun-inc.com(127.0.0.1); Sat, 25 Jul 2020 21:00:23 +0800 From: Alex Shi To: akpm@linux-foundation.org, mgorman@techsingularity.net, tj@kernel.org, hughd@google.com, khlebnikov@yandex-team.ru, daniel.m.jordan@oracle.com, yang.shi@linux.alibaba.com, willy@infradead.org, hannes@cmpxchg.org, lkp@intel.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, shakeelb@google.com, iamjoonsoo.kim@lge.com, richard.weiyang@gmail.com, kirill@shutemov.name, alexander.duyck@gmail.com, rong.a.chen@intel.com Cc: Andrea Arcangeli Subject: [PATCH v17 08/21] mm/thp: narrow lru locking Date: Sat, 25 Jul 2020 20:59:45 +0800 Message-Id: <1595681998-19193-9-git-send-email-alex.shi@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1595681998-19193-1-git-send-email-alex.shi@linux.alibaba.com> References: <1595681998-19193-1-git-send-email-alex.shi@linux.alibaba.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org lru_lock and page cache xa_lock have no reason with current sequence, put them together isn't necessary. let's narrow the lru locking, but left the local_irq_disable to block interrupt re-entry and statistic update. Hugh Dickins point: split_huge_page_to_list() was already silly,to be using the _irqsave variant: it's just been taking sleeping locks, so would already be broken if entered with interrupts enabled. so we can save passing flags argument down to __split_huge_page(). Signed-off-by: Alex Shi Signed-off-by: Wei Yang Reviewed-by: Kirill A. Shutemov Cc: Hugh Dickins Cc: Kirill A. Shutemov Cc: Andrea Arcangeli Cc: Johannes Weiner Cc: Matthew Wilcox Cc: Andrew Morton Cc: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org --- mm/huge_memory.c | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 1fb4147ff854..d866b6e43434 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2423,7 +2423,7 @@ static void __split_huge_page_tail(struct page *head, int tail, } static void __split_huge_page(struct page *page, struct list_head *list, - pgoff_t end, unsigned long flags) + pgoff_t end) { struct page *head = compound_head(page); pg_data_t *pgdat = page_pgdat(head); @@ -2432,8 +2432,6 @@ static void __split_huge_page(struct page *page, struct list_head *list, unsigned long offset = 0; int i; - lruvec = mem_cgroup_page_lruvec(head, pgdat); - /* complete memcg works before add pages to LRU */ mem_cgroup_split_huge_fixup(head); @@ -2445,6 +2443,11 @@ static void __split_huge_page(struct page *page, struct list_head *list, xa_lock(&swap_cache->i_pages); } + /* prevent PageLRU to go away from under us, and freeze lru stats */ + spin_lock(&pgdat->lru_lock); + + lruvec = mem_cgroup_page_lruvec(head, pgdat); + for (i = HPAGE_PMD_NR - 1; i >= 1; i--) { __split_huge_page_tail(head, i, lruvec, list); /* Some pages can be beyond i_size: drop them from page cache */ @@ -2464,6 +2467,8 @@ static void __split_huge_page(struct page *page, struct list_head *list, } ClearPageCompound(head); + spin_unlock(&pgdat->lru_lock); + /* Caller disabled irqs, so they are still disabled here */ split_page_owner(head, HPAGE_PMD_ORDER); @@ -2481,8 +2486,7 @@ static void __split_huge_page(struct page *page, struct list_head *list, page_ref_add(head, 2); xa_unlock(&head->mapping->i_pages); } - - spin_unlock_irqrestore(&pgdat->lru_lock, flags); + local_irq_enable(); remap_page(head); @@ -2621,12 +2625,10 @@ bool can_split_huge_page(struct page *page, int *pextra_pins) int split_huge_page_to_list(struct page *page, struct list_head *list) { struct page *head = compound_head(page); - struct pglist_data *pgdata = NODE_DATA(page_to_nid(head)); struct deferred_split *ds_queue = get_deferred_split_queue(head); struct anon_vma *anon_vma = NULL; struct address_space *mapping = NULL; int count, mapcount, extra_pins, ret; - unsigned long flags; pgoff_t end; VM_BUG_ON_PAGE(is_huge_zero_page(head), head); @@ -2687,9 +2689,8 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) unmap_page(head); VM_BUG_ON_PAGE(compound_mapcount(head), head); - /* prevent PageLRU to go away from under us, and freeze lru stats */ - spin_lock_irqsave(&pgdata->lru_lock, flags); - + /* block interrupt reentry in xa_lock and spinlock */ + local_irq_disable(); if (mapping) { XA_STATE(xas, &mapping->i_pages, page_index(head)); @@ -2719,7 +2720,7 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) __dec_node_page_state(head, NR_FILE_THPS); } - __split_huge_page(page, list, end, flags); + __split_huge_page(page, list, end); if (PageSwapCache(head)) { swp_entry_t entry = { .val = page_private(head) }; @@ -2738,7 +2739,7 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) spin_unlock(&ds_queue->split_queue_lock); fail: if (mapping) xa_unlock(&mapping->i_pages); - spin_unlock_irqrestore(&pgdata->lru_lock, flags); + local_irq_enable(); remap_page(head); ret = -EBUSY; } -- 1.8.3.1