Received: by 2002:a05:6a10:a852:0:0:0:0 with SMTP id d18csp636135pxy; Fri, 30 Apr 2021 12:55:20 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy1iPmVpHoz+qMH+/VhOf3+NF2Mc5Zj8cHnXH1x3e6Lau9x0/cyzUCvK+gJOKzJ35+DgyHG X-Received: by 2002:a17:903:208b:b029:ed:74ad:4c15 with SMTP id d11-20020a170903208bb02900ed74ad4c15mr6840833plc.6.1619812520225; Fri, 30 Apr 2021 12:55:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619812520; cv=none; d=google.com; s=arc-20160816; b=ZVPnzHXGmGvKScCcCCOFs7A0IpwHzZ8getqblgruuzObAqs+XonhaWfJ6YsIiSpzCP Uke3FyYSa+L2ZvhW+U+J7PAVi6hx8/CyIB9MViJ8hwkRH5mnwyPDUsXbgow5PiYQ1LA2 4IlQchuxnH2SPstEyf1onpJGvAGPT27O6mT9o2tmQ3XmBLuS3Epk36La0E4y7xunm7qx HGPwYfPx/V4g5DV7wFwbbYajfO2U/ugt3z7r5Xxa+c3pTxmbC4tBfuWddcrjKTvvcAy/ /PjGZHV40E6COIORMt7u6A6jb+aYRbGk3XVEHudZc5da9rEjYWXA+hserW4FwTwGlv4+ Gl2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-signature; bh=GfsT7km1e4u9essBB1Wu9kC04UOBGN5ahWAVC6IN8es=; b=P8kq05dY0+Sy/U0fvqmpW8ulXck8JohO5K0QNb4eqRpuzzZ/qixwARz/lUThSa50ez ptUXoqldFEAYmj4tt8V/izp1g5lUdsl2bcFjF5veMGMKLN9R2DVNyvmxfQgj4CgoobbA 1+CLO3HuTAeb9UGswNf5i86I+Ih5yULc8cJ3dOfU/YXWbodeOwEM0q9QvCPGG+D+VCAd VmWWoy68wSD1f3mrDo1TrBKjr4gJnet/sLg/oLUiwmQil5AAinnLBctDFluz9n5Jzx7I PGxJxIy6N/hWgnIpdnji/T714BiPN4jLrugWBIRE/s3uaSo8GPV9Jl7Mr6lLpVN+tYd4 2HXw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (no key) header.i=@lespinasse.org header.b=QJjr+jhI; dkim=pass (test mode) header.i=@lespinasse.org header.s=srv-14-rsa header.b=wKDBDf36; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=lespinasse.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id o4si3140471pjp.7.2021.04.30.12.55.07; Fri, 30 Apr 2021 12:55:20 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=neutral (no key) header.i=@lespinasse.org header.b=QJjr+jhI; dkim=pass (test mode) header.i=@lespinasse.org header.s=srv-14-rsa header.b=wKDBDf36; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=lespinasse.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234071AbhD3TzE (ORCPT + 99 others); Fri, 30 Apr 2021 15:55:04 -0400 Received: from server.lespinasse.org ([63.205.204.226]:47601 "EHLO server.lespinasse.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232256AbhD3Txh (ORCPT ); Fri, 30 Apr 2021 15:53:37 -0400 DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=lespinasse.org; i=@lespinasse.org; q=dns/txt; s=srv-14-ed; t=1619812353; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : from; bh=GfsT7km1e4u9essBB1Wu9kC04UOBGN5ahWAVC6IN8es=; b=QJjr+jhIyojTjZZUIgmmlJdhwxp1SplGNE0HRwgP8Nt0MWqhEvdhVMBV6dKGh9nbTPbnl EwzltSshob+oxI6BQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lespinasse.org; i=@lespinasse.org; q=dns/txt; s=srv-14-rsa; t=1619812353; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : from; bh=GfsT7km1e4u9essBB1Wu9kC04UOBGN5ahWAVC6IN8es=; b=wKDBDf36V1KWAErgdFzL/otW+avow+vT/EaKwdglJLtyRP95dA3S7xlKS+lK3mwNFj0lj MHPXq5SZRut4O8MZby6+wWCIuBmJtC8/Zex9JuElWKPSuaeBrbklTpODIHHKojHY3Qp7Sd2 aw4XbgGFObcnn57pMDO9JTHRFsG3cDTzhVCXEamUpBn5vRExQ/8E4+1vI8x6HIx2ZEDzkwR WGynlLA+SouY0L+mVA+lVh/dbllPZsBTwFI86XwEbVz/jaQWEcVzBLsbhScf1TY/3XqI8cF maV2tfOCc2KsvtVGDc/TEZFftizW8kBY1t6UJYzRq1dQRS+JV+xLHrEtv2EQ== Received: from zeus.lespinasse.org (zeus.lespinasse.org [IPv6:fd00::150:0]) by server.lespinasse.org (Postfix) with ESMTPS id B6CC8160365; Fri, 30 Apr 2021 12:52:33 -0700 (PDT) Received: by zeus.lespinasse.org (Postfix, from userid 1000) id A802819F521; Fri, 30 Apr 2021 12:52:33 -0700 (PDT) From: Michel Lespinasse To: Linux-MM , Linux-Kernel Cc: Laurent Dufour , Peter Zijlstra , Michal Hocko , Matthew Wilcox , Rik van Riel , Paul McKenney , Andrew Morton , Suren Baghdasaryan , Joel Fernandes , Andy Lutomirski , Michel Lespinasse Subject: [PATCH 26/29] mm: disable rcu safe vma freeing for single threaded user space Date: Fri, 30 Apr 2021 12:52:27 -0700 Message-Id: <20210430195232.30491-27-michel@lespinasse.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210430195232.30491-1-michel@lespinasse.org> References: <20210430195232.30491-1-michel@lespinasse.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Performance tuning: as single threaded userspace does not use speculative page faults, it does not require rcu safe vma freeing. Turn this off to avoid the related (small) extra overheads. For multi threaded userspace, we often see a performance benefit from the rcu safe vma freeing - even in tests that do not have any frequent concurrent page faults ! This is because rcu safe vma freeing prevents recently released vmas from being immediately reused in a new thread. Signed-off-by: Michel Lespinasse --- kernel/fork.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/kernel/fork.c b/kernel/fork.c index 7c22bf2b1f9d..18659d802d24 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -381,10 +381,12 @@ static void __vm_area_free(struct rcu_head *head) void vm_area_free(struct vm_area_struct *vma) { #ifdef CONFIG_SPECULATIVE_PAGE_FAULT - call_rcu(&vma->vm_rcu, __vm_area_free); -#else - kmem_cache_free(vm_area_cachep, vma); + if (atomic_read(&vma->vm_mm->mm_users) > 1) { + call_rcu(&vma->vm_rcu, __vm_area_free); + return; + } #endif + kmem_cache_free(vm_area_cachep, vma); } static void account_kernel_stack(struct task_struct *tsk, int account) -- 2.20.1