Received: by 2002:a05:7412:1e0b:b0:fc:a2b0:25d7 with SMTP id kr11csp412925rdb; Thu, 15 Feb 2024 04:18:30 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCU2ZSPyhEqqxBN+KKvmYfrx0P71WaYvi7qhLPTLsxM3wFg29CjjnjSYUe0D4CkWe+DWV/yUDRBJi5ASesKxgR+zalWGKJnl/IWTh8UfFw== X-Google-Smtp-Source: AGHT+IGXdKXF8qmLtaC53VlmSdK05YwC29UsHArvtxtVwf5U9I7OeUtv4UzLZHUnaXbiHPuJGJyR X-Received: by 2002:a17:906:ad8b:b0:a3d:991c:5152 with SMTP id la11-20020a170906ad8b00b00a3d991c5152mr1064829ejb.59.1707999509936; Thu, 15 Feb 2024 04:18:29 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707999509; cv=pass; d=google.com; s=arc-20160816; b=cOOhA/afivOK5skM0zohJQpsbeGA8R2YQFrsECoVkf7yetyAdKduxIlwBV49a74lLd ROmJ8oy0QRhoy5pHG4r9pbuhpwG+pWSMAotLvFHbjxRtqIY7WozMGJ/OG3Ag9jojYsgG Tu5HgLeX7pRwXSnkh1MJnnrW1bND3m8KlIhsk1SOgF2XSI6siNt87iIIYg2HtrIzqVDD 2LQG19dWJjn773ub8VINOKLzlQJ3o2g7Iu9bS6eR0G1cWN/PbiIcD2dFxVOo+b/Srko9 JF/Ai4LaVZjk7/2jN79Q3hYy0wavfITWXmT+yHIiTSXPoX0xWtblmtHKu+B042o22vdE z8wg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:message-id:date:subject:cc:to :from; bh=EgATNENApWy8csVmwonky1cJjqyqybzBOEU21JHol90=; fh=2FwJMiZjh7JNs0IOfHdUsjppsJNoMkV8ot/xk2oRsPU=; b=QQLUPbJrzLgQGEBko57wdN1LAt645X0/U9dSIv0JgbP8+PWWDfASWg/tqCHIQAkEu+ 34z0n1RsnchJSXArelVcZFcTGv+mEODQgF+sLN7GnlRH3m1lPeShkTT9Py4oo39Sz659 5uAwmiOgQEFgJ76s7Wc/3W78ARkXU1hSgoUVwYNKlPXpGxhUFGDfI5UEw4TQabL0uE91 vOTPdYqVQKWm+oZXLdPB0AQdR7e+9ZKZEU1OJW9V6UkV4JzF1CiGaOvmAT07SuMqvqWs 3CBzq6u2h6UOUfKarOwvnQhPQkRx3Od2EbBfB42sJczk1O4w67ByFl7I0gzEmSrydObM 4Khw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-66872-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-66872-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Return-Path: Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id h12-20020a17090634cc00b00a3d53b0bf43si621588ejb.46.2024.02.15.04.18.29 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Feb 2024 04:18:29 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-66872-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-66872-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-66872-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 96C221F230BF for ; Thu, 15 Feb 2024 12:18:29 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9C5B912C804; Thu, 15 Feb 2024 12:18:09 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A612E12BEAF for ; Thu, 15 Feb 2024 12:18:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707999489; cv=none; b=Fk8l0vunljLVs2ehqDhvsqAkmbPaHt9z87k+Im7kx3kpyxg0FpODvEqljQtSKKGOtGig5cO99IsrgFjEDEUNG1VCrXx0WUnm/4y3aoLgTr5H2tMdHRPxiYYgEB6Fq4p4JYQxvh7lWYNHwe0r2CSKmoh3TPskk8g4wcwkucb/tCk= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707999489; c=relaxed/simple; bh=x9FtnrleqpJuZy6f30x2br9cQeME+hU2HTs978RoTLA=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=qdG81AQy9t1duLggBJlhR1MNxAUWDG8LCPXd2GZfVlO8Utjn/YDNJTYOvaSWQhQ3U8n13Zokyw/i4EAjjPkhJm+5LivQtv6qAnnbaPcRI4piGa+Z7DjxeNYThtBi6ckB4ycDXQ+JI4GFyhFQLeDC88KE9i+tO5BzzOqumBXNSrs= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C309F1FB; Thu, 15 Feb 2024 04:18:46 -0800 (PST) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.26]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 256853F766; Thu, 15 Feb 2024 04:18:04 -0800 (PST) From: Ryan Roberts To: David Hildenbrand , Mark Rutland , Catalin Marinas , Will Deacon , Alexander Shishkin , Jiri Olsa , Ian Rogers , Adrian Hunter , Andrew Morton , Muchun Song Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC PATCH v1 0/4] Reduce cost of ptep_get_lockless on arm64 Date: Thu, 15 Feb 2024 12:17:52 +0000 Message-Id: <20240215121756.2734131-1-ryan.roberts@arm.com> X-Mailer: git-send-email 2.25.1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit This is an RFC for a series that aims to reduce the cost and complexity of ptep_get_lockless() for arm64 when supporting transparent contpte mappings [1]. The approach came from discussion with Mark and David [2]. It introduces a new helper, ptep_get_lockless_norecency(), which allows the access and dirty bits in the returned pte to be incorrect. This relaxation permits arm64's implementation to just read the single target pte, and avoids having to iterate over the full contpte block to gather the access and dirty bits, for the contpte case. It turns out that none of the call sites using ptep_get_lockless() require accurate access and dirty bit information, so we can also convert those sites. Although a couple of places need care (see patches 2 and 3). Arguably patch 3 is a bit fragile, given the wide accessibility of vmf->orig_pte. So it might make sense to drop this patch and stick to using ptep_get_lockless() in the page fault path. I'm keen to hear opinions. I've chosen the name "recency" because it's shortish and somewhat descriptive, and is alredy used in a couple of places to mean similar things (see mglru and damon). I'm open to other names if anyone has better ideas. If concensus is that this approach is generally acceptable, I intend to create a series in future to do a similar thing with ptep_get() -> ptep_get_norecency(). --- This series applies on top of [1]. [1] https://lore.kernel.org/linux-mm/20240215103205.2607016-1-ryan.roberts@arm.com/ [2] https://lore.kernel.org/linux-mm/a91cfe1c-289e-4828-8cfc-be34eb69a71b@redhat.com/ Thanks, Ryan Ryan Roberts (4): mm: Introduce ptep_get_lockless_norecency() mm/gup: Use ptep_get_lockless_norecency() mm/memory: Use ptep_get_lockless_norecency() for orig_pte arm64/mm: Override ptep_get_lockless_norecency() arch/arm64/include/asm/pgtable.h | 6 ++++ include/linux/pgtable.h | 55 ++++++++++++++++++++++++++++-- kernel/events/core.c | 2 +- mm/gup.c | 7 ++-- mm/hugetlb.c | 2 +- mm/khugepaged.c | 2 +- mm/memory.c | 57 ++++++++++++++++++++------------ mm/swap_state.c | 2 +- mm/swapfile.c | 2 +- 9 files changed, 102 insertions(+), 33 deletions(-) -- 2.25.1