Date:   Tue, 7 Mar 2023 13:54:20 -0800
From:   Mike Kravetz <mike.kravetz@oracle.com>
To:     James Houghton <jthoughton@google.com>
Cc:     Hugh Dickins <hughd@google.com>,
        Muchun Song <songmuchun@bytedance.com>,
        Peter Xu <peterx@redhat.com>,
        "Matthew Wilcox (Oracle)" <willy@infradead.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        "Kirill A . Shutemov" <kirill.shutemov@linux.intel.com>,
        David Hildenbrand <david@redhat.com>,
        David Rientjes <rientjes@google.com>,
        Axel Rasmussen <axelrasmussen@google.com>,
        Jiaqi Yan <jiaqiyan@google.com>, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org
Subject: Re: [PATCH 1/2] mm: rmap: make hugetlb pages participate in
 _nr_pages_mapped
Message-ID: <20230307215420.GA59222@monkey>
References: <20230306230004.1387007-1-jthoughton@google.com>
 <20230306230004.1387007-2-jthoughton@google.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20230306230004.1387007-2-jthoughton@google.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?dnOHhhEpBpjrpcn0nV8K0P+T/bDLxIEYAh1AGZG9aUuf/pqAylrzcQHe41h3?=
 =?us-ascii?Q?afNVuiPI9iQsaowz8Mmsk7iMkKJ6dSqHI0KZdnxnk9PrZdYWJhMZVb4FpeWG?=
 =?us-ascii?Q?VYItmCxVsuCiykE2+P8h1gC3xUyMyvVRl48g6dckKqQp/5IWyDgG5XHOOcWy?=
 =?us-ascii?Q?z15Uo3bNEOxnStv4jYktxWoomE8/hfhyg3aLeyjTWHxNCvxigxdwygIfzIGK?=
 =?us-ascii?Q?OsowNHwLuQ63irGkmI+kjyRXeiwnJ+MxrtQeeLx26b5FjQ1znRXewfJiMFe6?=
 =?us-ascii?Q?lspWkAGsxtQkGH8gV7NwPoYz/Xtka7bFpUNC0Dvwmm2ZNZa3XHZPzK2xdGIM?=
 =?us-ascii?Q?uvyglPMjxFOSUi64+Uj1TsVwkkz0gxPxXGEmFK6kZQ0oyJ3HqXdTGE989rVE?=
 =?us-ascii?Q?95u1arU3A60y2IHFuDSC2/deXrFxh3I1SxVz3YPB5Jm9R6z7Yafco3XCnj+I?=
 =?us-ascii?Q?yYGvzS9a1yZpFHpsVZh+HlwfMQN5dLcn2vMHG/OJN2TFAJRjnEHpRI7T3ByP?=
 =?us-ascii?Q?47e8V7ayWVPbJzMMjIQxis8xqUPfavF18h0T1tZtbR+8qiLDhlUK0Qben4hL?=
 =?us-ascii?Q?pTzo5ew1vok6Ycp+k+Kdv9rGMt6P6jUM3Ml/vEh3EyobrkyU1maQIkm0cmdH?=
 =?us-ascii?Q?puDQsTFwgR730ccQPBXnP9KkCfwLLQoqv46nkP8dst7ygUiqDihXhr89h91T?=
 =?us-ascii?Q?g4GrQ2jDkdjyDqS+0ZuKhiOxAmP3LXSgGphKOCBtvGYirKbtxxPMK3KD6fwF?=
 =?us-ascii?Q?Kw46JAuR+vwEEJ7P8aBM0osbjBD59vcJV3KcCZLgAzaWjlASKB1SOJnkQQbY?=
 =?us-ascii?Q?g6WWFbFJsbHLVoxZ+rrcKmj1IxhEj4Hj/ivYUz6rMG9ZhkmxkF8HS/xiePre?=
 =?us-ascii?Q?osRx8l1teif3WVL9LY2kAu7IsaAkgiUecDoc7i4u13TTWPGNPmtAh5aRTpyJ?=
 =?us-ascii?Q?7pE6XDAh1zTXSoSjSbC3OlhraFoQbR38VybarChe9HjQ5cTUadb2nQYvAhHu?=
 =?us-ascii?Q?+lfAVe9O0SAqNfNAwlNA+2N0/4hiO/djppLry2gx+dtiI072TdruBwJ3ljR5?=
 =?us-ascii?Q?TL/hJMUDq29WOa88WOKI2M4LRzDY/YJbESAg94EBnrdxqRy3CwI5pIoxggBj?=
 =?us-ascii?Q?N/0aKkzFG1axB1e+dXyVI3ET/ZWnMSNJ2rW0k3p8wkFREVpEy/wOgywsZXCV?=
 =?us-ascii?Q?QcKX2bEzQFmpA1tzlScG3OFviRFesqmoJdVX5cpbe9Nz3xImCpIQiVUVlgSz?=
 =?us-ascii?Q?4Qn7nOUZNp5Mbcn2p3jIpeMAg22snTgnxiDyHIwkxD495Kltk2VVfE+u7s/p?=
 =?us-ascii?Q?YjsXwxI+Ax2IJYryEKVHcLlZ6QPyEJnjQtngeiwd6kIH4eqIlzAiyUqgya/r?=
 =?us-ascii?Q?iR59Vi7whHMgsnNvjSBxM9okVNar2yYBMaJ4AoO3+rjEkzb0LaF7of34JyVl?=
 =?us-ascii?Q?A1F3o4mr0GQdgXQvoRz1OI4dvJNyV0tOwPNxr2IJ0/XgdBqMOafPjxgJ1jJ4?=
 =?us-ascii?Q?mHoD+RTZMpTGkku/R/Lwz5cO4sQWllAIm02ibykSUJaD8fHuNQnckupAmdLQ?=
 =?us-ascii?Q?eorruv5+1emyRmTnEY0SFnwWipRXwcn5C6qoNlTjXCMu4E0mbunPg0gyRLtJ?=
 =?us-ascii?Q?rQ=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: =?us-ascii?Q?SGdFtf89+b9/5eRS5o3gqbjLDVbq+t6MrLKu6cL8YxuNjWNd+zdilChwLpBK?=
 =?us-ascii?Q?6RYM2BsEALiBQ8fUDs6L4cwy+3yPNnG4kUJIhiJhgd1owKBz0tMQ58lUa4k9?=
 =?us-ascii?Q?nscsQX0GXAIEodPpQm7rI445E1ridJfgcQvpOUPjIq7o5ULVAhk0GMo3Fa8u?=
 =?us-ascii?Q?++2ECV16agjPtcjJHLgSvqCiJIqVJgSp8OV2i2IIGBsWCrVWBHnrDmd8Ll6u?=
 =?us-ascii?Q?nl5AvXB+2Ij2DULHY1SVRuvtB8cXxzy4KSpf+ic+n7ZNRN6VXRnKeA2TSV3V?=
 =?us-ascii?Q?tBP1uehcwdagDBXUlBjc2KbKX+uSSeYkDXx7924fP31iJKl0fkTYDpl+k6FK?=
 =?us-ascii?Q?0LePtNUjyVAyZw3pxWrzMqv8Ttukf+Lpp4C0z4NBjUzScTSKFZ+lzVC6IyHs?=
 =?us-ascii?Q?3nyX7oHg5Azfh0/I5HLUTS5AhA3BYyjZhlyxhjP6QVaZTEVlgjTDcAiaVEE9?=
 =?us-ascii?Q?A++CvFYLHTygc5sVA2I0CyctZiEbLRejDGik4hr7sXRN8xCjReaRiacuzOYe?=
 =?us-ascii?Q?BAgcfXBRZv0kcuGLCv8TyrZNvVPQszjiCunkmN/RTH2B6s3ZRk8x/9cad37Z?=
 =?us-ascii?Q?C6dciBi6XGstwFIpy4nQ3U4EVoTQiPgWim0zVzPt/uJCJB36EkJZPq667eTf?=
 =?us-ascii?Q?VTs3/6s4mtNHk/xwhR/15RCHhfhOtrCsn8Ru8O/mGCGMN17Wo6n3Jn/8pTQc?=
 =?us-ascii?Q?N71u/UXBfqHJaHWUVAGBv4NEagtDyCyc6YChNCM348ivGKc+i7HbZE6wBLzU?=
 =?us-ascii?Q?IDfSRY+tSfmUQ1lr8KbiTToH51+wN6XNvuNSbZ9lTPAxzLmtuwFB3Woq7vUw?=
 =?us-ascii?Q?AaaariYKDTglMfj16/cFaEF7an47L4PDfCPoocBuTR0ThCljmtFeFzUW/kIl?=
 =?us-ascii?Q?wka2EnNbPyo34ceD4CtkvGUTsYHM75f67H4sZWfsPRzvdYUQMwRs6pUg7G9e?=
 =?us-ascii?Q?EL2Bx/CX1BmZIgz183cXmqCVe+VnFbGo9IJqvFwN1uXQA1FM4Ql9vePc8YOr?=
 =?us-ascii?Q?xMOteh9kGQ9U3YEZUwPuqGTZRg=3D=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: cd26c784-08ad-43f2-0a34-08db1f568398
X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4196.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Mar 2023 21:54:23.9734
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: JFzS6t4bvuQfNMSk4729+D8bdU8hEnJ2Q2UhWOT+xGAr1QdJ4uLxJua9z99eADVz8nbIgDTpuOOArKYN0GLqOg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR10MB6189
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22
 definitions=2023-03-07_16,2023-03-07_01,2023-02-09_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999
 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 malwarescore=0
 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2212070000 definitions=main-2303070195
X-Proofpoint-GUID: u15DooNKCdUQr0cDgIowFBF7j2PPBgJa
X-Proofpoint-ORIG-GUID: u15DooNKCdUQr0cDgIowFBF7j2PPBgJa
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 03/06/23 23:00, James Houghton wrote:
> For compound mappings (compound=true), _nr_pages_mapped will now be
> incremented by COMPOUND_MAPPED when the first compound mapping is
> created.

This sentence makes it sound like incrementing by COMPOUND_MAPPED for
compound pages is introduced by this patch.  Rather, it is just for
hugetlb (now always) compound mappings.   Perhaps change that to read:
For hugetlb mappings ...

> For small mappings, _nr_pages_mapped is incremented by 1 when the
> particular small page is mapped for the first time. This is incompatible
> with HPageVmemmapOptimize()ed folios, as most of the tail page structs
> will be mapped read-only.
> 
> Currently HugeTLB always passes compound=true, but in the future,
> HugeTLB pages may be mapped with small mappings.
> 
> To implement this change:
>  1. Replace most of HugeTLB's calls to page_dup_file_rmap() with
>     page_add_file_rmap(). The call in copy_hugetlb_page_range() is kept.
>  2. Update page_add_file_rmap() and page_remove_rmap() to support
>     HugeTLB folios.
>  3. Update hugepage_add_anon_rmap() and hugepage_add_new_anon_rmap() to
>     also increment _nr_pages_mapped properly.
> 
> With these changes, folio_large_is_mapped() no longer needs to check
> _entire_mapcount.
> 
> HugeTLB doesn't use LRU or mlock, so page_add_file_rmap() and
> page_remove_rmap() excludes those pieces. It is also important that
> the folio_test_pmd_mappable() check is removed (or changed), as it's
> possible to have a HugeTLB page whose order is not >= HPAGE_PMD_ORDER,
> like arm64's CONT_PTE_SIZE HugeTLB pages.
> 
> This patch limits HugeTLB pages to 16G in size. That limit can be
> increased if COMPOUND_MAPPED is raised.
> 
> Signed-off-by: James Houghton <jthoughton@google.com>
> 

Thanks!

This is a step in the direction of having hugetlb use the same mapcount
scheme as elsewhere.  As you mention, with this in place future mapcount
changes should mostly 'just work' for hugetlb.

Because of this,
Acked-by: Mike Kravetz <mike.kravetz@oracle.com>

I have a few nits below, and I'm sure others will chime in later.

> diff --git a/mm/rmap.c b/mm/rmap.c
> index ba901c416785..4a975429b91a 100644
> --- a/mm/rmap.c
> +++ b/mm/rmap.c
> @@ -1316,19 +1316,21 @@ void page_add_file_rmap(struct page *page, struct vm_area_struct *vma,
>  	int nr = 0, nr_pmdmapped = 0;
>  	bool first;
>  
> -	VM_BUG_ON_PAGE(compound && !PageTransHuge(page), page);
> +	VM_BUG_ON_PAGE(compound && !PageTransHuge(page)
> +				&& !folio_test_hugetlb(folio), page);
>  
>  	/* Is page being mapped by PTE? Is this its first map to be added? */
>  	if (likely(!compound)) {
> +		if (unlikely(folio_test_hugetlb(folio)))
> +			VM_BUG_ON_PAGE(HPageVmemmapOptimized(&folio->page),
> +				       page);
>  		first = atomic_inc_and_test(&page->_mapcount);
>  		nr = first;
>  		if (first && folio_test_large(folio)) {
>  			nr = atomic_inc_return_relaxed(mapped);
>  			nr = (nr < COMPOUND_MAPPED);
>  		}
> -	} else if (folio_test_pmd_mappable(folio)) {
> -		/* That test is redundant: it's for safety or to optimize out */

I 'think' removing this check is OK.  It would seem that the caller
knows if the folio is mappable.  If we want a similar test, we might be
able to use something like:

	arch_hugetlb_valid_size(folio_size(folio))

> -
> +	} else {
>  		first = atomic_inc_and_test(&folio->_entire_mapcount);
>  		if (first) {
>  			nr = atomic_add_return_relaxed(COMPOUND_MAPPED, mapped);
> @@ -1345,6 +1347,9 @@ void page_add_file_rmap(struct page *page, struct vm_area_struct *vma,
>  		}
>  	}
>  
> +	if (folio_test_hugetlb(folio))
> +		return;

IMO, a comment saying hugetlb is special and does not participate in lru
would be appropriate here.

> +
>  	if (nr_pmdmapped)
>  		__lruvec_stat_mod_folio(folio, folio_test_swapbacked(folio) ?
>  			NR_SHMEM_PMDMAPPED : NR_FILE_PMDMAPPED, nr_pmdmapped);
> @@ -1373,24 +1378,18 @@ void page_remove_rmap(struct page *page, struct vm_area_struct *vma,
>  
>  	VM_BUG_ON_PAGE(compound && !PageHead(page), page);
>  
> -	/* Hugetlb pages are not counted in NR_*MAPPED */
> -	if (unlikely(folio_test_hugetlb(folio))) {
> -		/* hugetlb pages are always mapped with pmds */
> -		atomic_dec(&folio->_entire_mapcount);
> -		return;
> -	}
> -
>  	/* Is page being unmapped by PTE? Is this its last map to be removed? */
>  	if (likely(!compound)) {
> +		if (unlikely(folio_test_hugetlb(folio)))
> +			VM_BUG_ON_PAGE(HPageVmemmapOptimized(&folio->page),
> +				       page);
>  		last = atomic_add_negative(-1, &page->_mapcount);
>  		nr = last;
>  		if (last && folio_test_large(folio)) {
>  			nr = atomic_dec_return_relaxed(mapped);
>  			nr = (nr < COMPOUND_MAPPED);
>  		}
> -	} else if (folio_test_pmd_mappable(folio)) {
> -		/* That test is redundant: it's for safety or to optimize out */
> -
> +	} else {
>  		last = atomic_add_negative(-1, &folio->_entire_mapcount);
>  		if (last) {
>  			nr = atomic_sub_return_relaxed(COMPOUND_MAPPED, mapped);
> @@ -1407,6 +1406,9 @@ void page_remove_rmap(struct page *page, struct vm_area_struct *vma,
>  		}
>  	}
>  
> +	if (folio_test_hugetlb(folio))
> +		return;

Same as above in page_add_file_rmap.

> +
>  	if (nr_pmdmapped) {
>  		if (folio_test_anon(folio))
>  			idx = NR_ANON_THPS;
> @@ -2541,9 +2543,11 @@ void hugepage_add_anon_rmap(struct page *page, struct vm_area_struct *vma,
>  	first = atomic_inc_and_test(&folio->_entire_mapcount);
>  	VM_BUG_ON_PAGE(!first && (flags & RMAP_EXCLUSIVE), page);
>  	VM_BUG_ON_PAGE(!first && PageAnonExclusive(page), page);
> -	if (first)
> +	if (first) {
> +		atomic_add(COMPOUND_MAPPED, &folio->_nr_pages_mapped);
>  		__page_set_anon_rmap(folio, page, vma, address,
>  				     !!(flags & RMAP_EXCLUSIVE));
> +	}
>  }
>  
>  void hugepage_add_new_anon_rmap(struct folio *folio,
> @@ -2552,6 +2556,7 @@ void hugepage_add_new_anon_rmap(struct folio *folio,
>  	BUG_ON(address < vma->vm_start || address >= vma->vm_end);
>  	/* increment count (starts at -1) */
>  	atomic_set(&folio->_entire_mapcount, 0);
> +	atomic_set(&folio->_nr_pages_mapped, COMPOUND_MAPPED);
>  	folio_clear_hugetlb_restore_reserve(folio);
>  	__page_set_anon_rmap(folio, &folio->page, vma, address, 1);
>  }

Should we look at perhaps modifying page_add_anon_rmap and
folio_add_new_anon_rmap as well?
-- 
Mike Kravetz