Received-SPF: pass (google.com: domain of linux-kernel+bounces-158782-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
From: Zi Yan <ziy@nvidia.com>
To: David Hildenbrand <david@redhat.com>
Cc: linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>,
 "Matthew Wilcox (Oracle)" <willy@infradead.org>,
 Yang Shi <shy828301@gmail.com>, Ryan Roberts <ryan.roberts@arm.com>,
 Barry Song <21cnbao@gmail.com>, Lance Yang <ioworker0@gmail.com>,
 linux-kernel@vger.kernel.org
Subject: Re: [PATCH v3] mm/rmap: do not add fully unmapped large folio to
 deferred split list
Date: Thu, 25 Apr 2024 11:16:33 -0400
Message-ID: <2586B1FF-91EF-41C2-AD98-7EF699716CAB@nvidia.com>
In-Reply-To: <e1e3d368-4861-470f-b7ca-b222712adef5@redhat.com>
References: <20240424224614.477221-1-zi.yan@sent.com>
 <a5426fa6-442a-453f-a234-63d89675ea79@redhat.com>
 <8BBD1B75-135D-42AA-8937-53B259803AA7@nvidia.com>
 <e1e3d368-4861-470f-b7ca-b222712adef5@redhat.com>
Content-Type: multipart/signed;
 boundary="=_MailMate_8878B8EE-4BC3-4587-A43A-51F5B64E621C_=";
 micalg=pgp-sha512; protocol="application/pgp-signature"
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?RFO84OVbGqV42XLD57FDZlHYTBI++DXGYlQb8A5KolXMKxP7HDgAOmTAtPJa?=
 =?us-ascii?Q?gyneqI8IIHt+NgvCYw3iQTy3aXhe8CIwdojAYx/Y2pnJCeNJvMISEEF+szcz?=
 =?us-ascii?Q?uRhMwet3kNbWXC73lYsvc+xmt2NKyFuIJ/PAPyeIGySa33ML7cYHNsezDrJe?=
 =?us-ascii?Q?531Q75GMFQ3ma9GSWNwss5vClACHsVMBw9e8jPRuUtYQygaMN60Ct7y1Y0XD?=
 =?us-ascii?Q?2vrgQdytEBc8tB+nqnqsB1P58+2vzFvNtpgPRXzi0nyw5p0stfK0ipg+9dAh?=
 =?us-ascii?Q?s0HTaQptWDw65AqXi8dn63Fc+8/64Is03/rNCuIj8upphUTT7Z/ek7QWamrJ?=
 =?us-ascii?Q?wXbFzQqCTWhehqjRRjjCsL62cVRB9twuD5+nnRVHYhNo4Dxl5ljHzUolgKco?=
 =?us-ascii?Q?W7Ht/whEdRkSaecj5Ta3SDOz90xYYjhhBJPmWTYfq2wFDRjuoD/Qjc5gMmba?=
 =?us-ascii?Q?/k11uvioAZyIYcSp9jAJjqL8Pcqd8k+EfzxrHqlyoswZeHxQkqKCfkUVYrCs?=
 =?us-ascii?Q?dZ1RpiIw6yHTH7c6WgQB3SJVGYhfAL+lTB/f5uK+OWap1atyAPoR/bPjzOsf?=
 =?us-ascii?Q?GZEL9Q3GAvILIpZQesl7Ysy5BGicyVJtgORrHGmVECE0T/C34tDikC42GGK0?=
 =?us-ascii?Q?gUdpjGnA46anyyubKhgq7mXgJYI7C1IRYfXUCYOPnltHGNH9o5PZrk3Y/tfb?=
 =?us-ascii?Q?zgxppGN/kiWvKQsg00et9CR2Q2Ny+Fe/M5iNMPTW/HrR38lcy/GoaSBg4Ht2?=
 =?us-ascii?Q?/tysUY37W/TjDHJxWdhcZiT/9fpFItztCYxiHI6+1BDBCYy/Dpc4HMDRMlG3?=
 =?us-ascii?Q?uNLM8j8rPNKQxg4wKkbodiHq9Zd2VWgHVWCKMxCKTSCxdP7BS4EVvxr+39il?=
 =?us-ascii?Q?v7qRvw9EfRf4afzyfjfOl/zMdCQq2tPZm53DKHftc08ycjWyUjJazxDlICay?=
 =?us-ascii?Q?n08jntpwG1KeHkEiCoB6WB4/vZfTiRKudYnCDKi9ftygjFwD6fnRFZyLdTSe?=
 =?us-ascii?Q?NJuSkHmL3ZznqClBk6q42s0EwMedxXek9+RB228TwqLTbzeQIHfZY/9rouBT?=
 =?us-ascii?Q?PmPOceKdAWI9pRds4uXAZwM/ASdlzac8JyVfygpQBZmWCHpcRNZGKdkQp/Oz?=
 =?us-ascii?Q?amH24pgygEMcFT17qDP/1L5/QT8WRNiSOVyoH1Q/hixRm++7+fVogpbjcyBL?=
 =?us-ascii?Q?HkQsTWCsClBWaA16h2PQ+DSiu/m/zyRkp0rvjF8tX+SSuagU3a7/G6YN+JII?=
 =?us-ascii?Q?+vi2Qj2pWgnIAZORcI6oFl0WlCh3YGz9lKUdyDpCOz4bCbpRMd//jyfqvPeN?=
 =?us-ascii?Q?mBGQXVChvsEBQxy8uySuTd0NrvVNHySvmnnZx4CKdna2semETDBY+0BjJh2S?=
 =?us-ascii?Q?P2Q1ns21Wiq1zd7KlYCQml7dxd7ipqqlPdJ9xnoNIrY5MH6XKxutykwJBAZE?=
 =?us-ascii?Q?dxKKKhV36IOakeyywUh3q6eNfm2GODhwSvUTBgt/YySTWz/toXaQlCVTXmfd?=
 =?us-ascii?Q?BBMlqlp0/3LY3baeNUhUI3eeDUVOoRofoHJsX7IEDgQqpZSiPI6yq/WZpIeK?=
 =?us-ascii?Q?ddeEfUHCgGm+XvOrYe0=3D?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 33b67a1d-abbb-4488-7111-08dc653ab2b8
X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB5744.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2024 15:16:36.1948
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ZC4Wy56atFaNuQdOHPQ8Xa7+DHgzva4J3nQme1moI05KhzDxSk33PegDTBM+bW6q
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB8051

--=_MailMate_8878B8EE-4BC3-4587-A43A-51F5B64E621C_=
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On 25 Apr 2024, at 11:15, David Hildenbrand wrote:

> On 25.04.24 16:53, Zi Yan wrote:
>> On 25 Apr 2024, at 3:19, David Hildenbrand wrote:
>>
>>> On 25.04.24 00:46, Zi Yan wrote:
>>>> From: Zi Yan <ziy@nvidia.com>
>>>>
>>>> In __folio_remove_rmap(), a large folio is added to deferred split l=
ist
>>>> if any page in a folio loses its final mapping. It is possible that
>>>> the folio is unmapped fully, but it is unnecessary to add the folio
>>>> to deferred split list at all. Fix it by checking folio->_nr_pages_m=
apped
>>>> before adding a folio to deferred split list. If the folio is alread=
y
>>>> on the deferred split list, it will be skipped. This issue applies t=
o
>>>> both PTE-mapped THP and mTHP.
>>>>
>>>> Commit 98046944a159 ("mm: huge_memory: add the missing
>>>> folio_test_pmd_mappable() for THP split statistics") tried to exclud=
e
>>>> mTHP deferred split stats from THP_DEFERRED_SPLIT_PAGE, but it does =
not
>>>> fix the above issue. A fully unmapped PTE-mapped order-9 THP was sti=
ll
>>>
>>> Once again: your patch won't fix it either.
>>>
>>>> added to deferred split list and counted as THP_DEFERRED_SPLIT_PAGE,=

>>>> since nr is 512 (non zero), level is RMAP_LEVEL_PTE, and inside
>>>> deferred_split_folio() the order-9 folio is folio_test_pmd_mappable(=
).
>>>> However, this miscount was present even earlier due to implementatio=
n,
>>>> since PTEs are unmapped individually and first PTE unmapping adds th=
e THP
>>>> into the deferred split list.
>>>
>>> It will still be present. Just less frequently.
>>
>> OK. Let me reread the email exchanges between you and Yang and clarify=

>> the details in the commit log.
>
> Likely something like:
>
> --
> In __folio_remove_rmap(), a large folio is added to deferred split list=

> if any page in a folio loses its final mapping. But, it is possible tha=
t
> the folio is now fully unmapped and adding it to the deferred split lis=
t is unnecessary.
>
> For PMD-mapped THPs, that was not really an issue, because removing the=
 last PMD mapping in the absence of PTE mappings would not have added the=
 folio to the deferred split queue.
>
> However, for PTE-mapped THPs, which are now more prominent due to mTHP,=
 we will always end up adding them to the deferred split queue.
>
> One side effect of this is that we will frequently increase the THP_DEF=
ERRED_SPLIT_PAGE stat for PTE-mapped THP, making it look like we frequent=
ly get many partially mapped folios -- although we are simply
> unmapping the whole thing stepwise.
>
> Core-mm will now try batch-unmapping consecutive PTEs of PTE-mapped THP=
s where possible. If we're lucky, we unmap the whole thing in one go and =
can avoid adding the folio to the deferred split queue, reducing the THP_=
DEFERRED_SPLIT_PAGE noise.
>
> But there will still be noise when we cannot batch-unmap a complete PTE=
-mapped folio in one go -- or where this type of batching is not implemen=
ted yet.
> --
>
> Feel free to reuse what you consider reasonable.

Sure. Thank you a lot for drafting it!

--
Best Regards,
Yan, Zi

--=_MailMate_8878B8EE-4BC3-4587-A43A-51F5B64E621C_=
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename=signature.asc
Content-Type: application/pgp-signature; name=signature.asc

-----BEGIN PGP SIGNATURE-----

iQJDBAEBCgAtFiEE6rR4j8RuQ2XmaZol4n+egRQHKFQFAmYqc9IPHHppeUBudmlk
aWEuY29tAAoJEOJ/noEUByhUme8P/jtVbLpaTivbJaO3O7dHNq0Tu/1N28oJzW4J
SJFubeu7wRo2ifTiWBxr1psaqRNBEwJkSRpLN9u7uzVuBKPRTdZ1z2sMNdK85Nx/
ESNHLkMZx2ANF6NXxcT23iYAhvnskG+KBT5ZnrE4u0E3fXsXj3pjoWCyavUUxXtT
/Ys/1pJK955crI/ICd+VP4I6GuGq1lLzWL1pBCFPXwsWtUGy8POLbfPjDLA/5Nk+
rWPKQ3j/24NOsWcxpUCM/avBZuegysSU7ob6HQH3g2+DNZiHiipZ19RZafEM6/sg
0BMvgcYY4JGAb9rbSzGXLVwYiFl3sA1+V48q/Kx3R8ItsbmHKqgTbdUpL8TArqHN
IbuH0Tv6F0iWFQ+6vGtBxw/XImKXQzzvKM3W5ZHAxtaUgONeXQnYURkM9K/+vjHK
8qwJ4c/T57wavugMgEJXVjtftwlNu8GaZ4pEXMPdrkaLJR1dJgVKBrtGdiLv/mGq
vJHQlQentH34/12prNRCWqxA2vR0MNQwWf5U1gzPVsWkiGvmVaeYHPwVjKHKifBv
LkVefIlpcZnICxe7RmWKXhRPZjyUogmDvHMRPviETg7kANNXdF12xCzk4E4ELpTT
JRlbT+gkP/MHDa6qr8L2oVhG3XPDz0s97Ag9ko5CaG15t3niq/PJRASKJvNsUP2c
Qst+FF1R
=1uvL
-----END PGP SIGNATURE-----

--=_MailMate_8878B8EE-4BC3-4587-A43A-51F5B64E621C_=--