Received: by 2002:ab2:1689:0:b0:1f7:5705:b850 with SMTP id d9csp1300027lqa; Mon, 29 Apr 2024 04:52:57 -0700 (PDT) X-Forwarded-Encrypted: i=3; AJvYcCXLTi43hLuDfEFv2+hKWe4jE4hHgi0MX/+3aPLsVdAPNLFhdzNe1N9fzMUc4UHTjhAFzPhBAssbWZRzaLgOQ3JF+hnM9ogkjqrz9JaoBA== X-Google-Smtp-Source: AGHT+IFytXy0DGX0R4iShVi27Clz5lI5Y964v96VXPVjORFaRIyuDQLmybYUTlwQ3LMZVxCJK4kK X-Received: by 2002:a17:90a:cc7:b0:2a2:cf1d:895c with SMTP id 7-20020a17090a0cc700b002a2cf1d895cmr9061736pjt.41.1714391577445; Mon, 29 Apr 2024 04:52:57 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1714391577; cv=pass; d=google.com; s=arc-20160816; b=Ql9mGboftbRz3WySe6rsHx0uZxmcbgsQCK2z6p1uqBIBvaUnJY7FIm63uQIGQvNA0z 1EVHux8gSVls2JD5o19yNpjsHmRaYJMXlWTYFKtjBiDMiHHsIZR933Aih98trR0KlZ7y cnxWCcn3rJTPEs947o3+8/VhpphFe3vqVC97m2FBkznF9pbxQ44VDssrtJbepTAbn8KP zerBKv7PXHGftquB10M44Nl5asjVIs9Oi9b1SflJpENzJxO2E5TczCMyyVEe3YL+forw LTZvVV4aIKRVlNqGfopqhM2YUbcqX+qL4GnYg5DZWDend9BN8PEbh3zgRCD3Eqb4h+g5 ZrqA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:message-id:date:subject:cc:to :from; bh=YeO2HXEjWCXZAF7H40VVzZHL4FRORn+OEpQDzfmPTkY=; fh=7LjKlt7j7wkfuH3q8Yvy6/OIqUoxE1f624Xs/9Gv6+U=; b=t3FeCaV+skKbfqol0uW7gWUCnm9b8sQLGcuVeCg8oQ+cEFExsWfNjoGV5FQ9sPMFlB CUBxcElC+cDN34qW92oxYEUYNt8xNJcsXZ0YdUuSUuZzMCpMfYhWcUF1NavVshdFlJ9C l+F9Ocf8IppAP/HyV6cjoAcXTuY82kiS34n5SXSXc9wsEhru0eXRfhnqrhSq8PelOKJa yG0kT/ILGOxy++xMtDzxn0S/ZtxPSxrgo9AHMJAfLf2UTIGXtZXNaMIK/RO60HCdnbFH DS0BPtWX8hN8nkNvjpni6eRAr9Yw5+VMw4Yz8cT5U6l2NIlAKx262eKW5vrJZ/MOqYvM omfQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-162168-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-162168-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Return-Path: Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id x4-20020a17090a8a8400b002b1728cdb53si1236900pjn.1.2024.04.29.04.52.56 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Apr 2024 04:52:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel+bounces-162168-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-162168-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-162168-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 2EB84B252A5 for ; Mon, 29 Apr 2024 11:41:31 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B29C6481C4; Mon, 29 Apr 2024 11:41:17 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id EBDA74596C; Mon, 29 Apr 2024 11:41:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714390877; cv=none; b=UsA6MYd53KSL1Oa7+lPxwZB329E3clG/OphoODtkNFoNgLK3Yck3Kt37RDZ93KziT2/iz1ls3UkzA68Og/sN1KUrpx/azvzBT3Ju/v04w66Dn0nLJXNgpKhVNCmh92amg8LBoydpwvcPWW4b/DfDzudiwgsG5akzMVkvGTJJcYI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714390877; c=relaxed/simple; bh=Mb3aQUpWdhn/dnpvjmFr0rgIm6z7baWnACBiOxUtT4E=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=eFO2I5gFJYPfZX8ta+QUdBTE1kUS/FzoGrmzbKEU1CUzBH1LLfTbKEcJpBMX+bcOnyozYbhvxO4EibPDo3i+4Q7at3kilXnH4MTeOEoVXYymc4/GWY4aYnqNogjPSO8RrXAvGbUOrN+p/QoEOkXqTGXzMckvN3wPkH4uuR+dL9Y= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AA5E82F4; Mon, 29 Apr 2024 04:41:40 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E33883F793; Mon, 29 Apr 2024 04:41:12 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , David Hildenbrand , Muhammad Usama Anjum , Peter Xu Cc: Ryan Roberts , linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v1] fs/proc/task_mmu: Fix uffd-wp confusion in pagemap_scan_pmd_entry() Date: Mon, 29 Apr 2024 12:41:04 +0100 Message-Id: <20240429114104.182890-1-ryan.roberts@arm.com> X-Mailer: git-send-email 2.25.1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit pagemap_scan_pmd_entry() checks if uffd-wp is set on each pte to avoid unnecessary if set. However it was previously checking with `pte_uffd_wp(ptep_get(pte))` without first confirming that the pte was present. It is only valid to call pte_uffd_wp() for present ptes. For swap ptes, pte_swp_uffd_wp() must be called because the uffd-wp bit may be kept in a different position, depending on the arch. This was leading to test failures in the pagemap_ioctl mm selftest, when bringing up uffd-wp support on arm64 due to incorrectly interpretting the uffd-wp status of migration entries. Let's fix this by using the correct check based on pte_present(). While we are at it, let's pass the pte to make_uffd_wp_pte() to avoid the pointless extra ptep_get() which can't be optimized out due to READ_ONCE() on many arches. Closes: https://lore.kernel.org/linux-arm-kernel/ZiuyGXt0XWwRgFh9@x1n/ Fixes: 12f6b01a0bcb ("fs/proc/task_mmu: add fast paths to get/clear PAGE_IS_WRITTEN flag") Signed-off-by: Ryan Roberts --- fs/proc/task_mmu.c | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index af4bc1da0c01..102f48668c35 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1817,10 +1817,8 @@ static unsigned long pagemap_page_category(struct pagemap_scan_private *p, } static void make_uffd_wp_pte(struct vm_area_struct *vma, - unsigned long addr, pte_t *pte) + unsigned long addr, pte_t *pte, pte_t ptent) { - pte_t ptent = ptep_get(pte); - if (pte_present(ptent)) { pte_t old_pte; @@ -2175,9 +2173,12 @@ static int pagemap_scan_pmd_entry(pmd_t *pmd, unsigned long start, if ((p->arg.flags & PM_SCAN_WP_MATCHING) && !p->vec_out) { /* Fast path for performing exclusive WP */ for (addr = start; addr != end; pte++, addr += PAGE_SIZE) { - if (pte_uffd_wp(ptep_get(pte))) + pte_t ptent = ptep_get(pte); + + if ((pte_present(ptent) && pte_uffd_wp(ptent)) || + pte_swp_uffd_wp_any(ptent)) continue; - make_uffd_wp_pte(vma, addr, pte); + make_uffd_wp_pte(vma, addr, pte, ptent); if (!flush_end) start = addr; flush_end = addr + PAGE_SIZE; @@ -2190,8 +2191,10 @@ static int pagemap_scan_pmd_entry(pmd_t *pmd, unsigned long start, p->arg.return_mask == PAGE_IS_WRITTEN) { for (addr = start; addr < end; pte++, addr += PAGE_SIZE) { unsigned long next = addr + PAGE_SIZE; + pte_t ptent = ptep_get(pte); - if (pte_uffd_wp(ptep_get(pte))) + if ((pte_present(ptent) && pte_uffd_wp(ptent)) || + pte_swp_uffd_wp_any(ptent)) continue; ret = pagemap_scan_output(p->cur_vma_category | PAGE_IS_WRITTEN, p, addr, &next); @@ -2199,7 +2202,7 @@ static int pagemap_scan_pmd_entry(pmd_t *pmd, unsigned long start, break; if (~p->arg.flags & PM_SCAN_WP_MATCHING) continue; - make_uffd_wp_pte(vma, addr, pte); + make_uffd_wp_pte(vma, addr, pte, ptent); if (!flush_end) start = addr; flush_end = next; @@ -2208,8 +2211,9 @@ static int pagemap_scan_pmd_entry(pmd_t *pmd, unsigned long start, } for (addr = start; addr != end; pte++, addr += PAGE_SIZE) { + pte_t ptent = ptep_get(pte); unsigned long categories = p->cur_vma_category | - pagemap_page_category(p, vma, addr, ptep_get(pte)); + pagemap_page_category(p, vma, addr, ptent); unsigned long next = addr + PAGE_SIZE; if (!pagemap_scan_is_interesting_page(categories, p)) @@ -2224,7 +2228,7 @@ static int pagemap_scan_pmd_entry(pmd_t *pmd, unsigned long start, if (~categories & PAGE_IS_WRITTEN) continue; - make_uffd_wp_pte(vma, addr, pte); + make_uffd_wp_pte(vma, addr, pte, ptent); if (!flush_end) start = addr; flush_end = next; -- 2.25.1