Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7;
Message-ID: <0f64e33c-6aae-440c-a76c-de9f87f3f93a@intel.com>
Date:   Fri, 8 Dec 2023 09:44:29 +0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH RFC 13/39] mm/rmap: factor out adding folio mappings into
 __folio_add_rmap()
Content-Language: en-US
To:     David Hildenbrand <david@redhat.com>,
        <linux-kernel@vger.kernel.org>
CC:     <linux-mm@kvack.org>, Andrew Morton <akpm@linux-foundation.org>,
        "Matthew Wilcox (Oracle)" <willy@infradead.org>,
        Hugh Dickins <hughd@google.com>,
        "Ryan Roberts" <ryan.roberts@arm.com>,
        Mike Kravetz <mike.kravetz@oracle.com>,
        Muchun Song <muchun.song@linux.dev>,
        Peter Xu <peterx@redhat.com>
References: <20231204142146.91437-1-david@redhat.com>
 <20231204142146.91437-14-david@redhat.com>
From:   "Yin, Fengwei" <fengwei.yin@intel.com>
In-Reply-To: <20231204142146.91437-14-david@redhat.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?NmRqR0NlYS90bGMyQTdORC9OTmRmclRJY3JBN0JyRGVJZlF5R1YrM21wTTdB?=
 =?utf-8?B?N29MTUt5Nzl3NURiUEI1U3NRYUxDb2dUcXo5cVE2VE04aHVNNHRMR0V0blZI?=
 =?utf-8?B?MGZpQURxUVhWc0toSytwM3lpV0pOeWZqMksxaWkvaHNwVmVMei9LT1BRVmdQ?=
 =?utf-8?B?YloyTU5Id01aSWw0Yno5clVKcVBaYkxvTDhZMUFQeXJuQUdkODUyclg3b2U5?=
 =?utf-8?B?V3BGNVlqaStnbkZNaW5kNDdRYU8yV1JwS0EydWZOSHVpTHF3azJSUjBta0FT?=
 =?utf-8?B?SmZiTkhDR0RLTHZIU2JncitJU2FvRWQvMk5RdURKbVM2REZlZjk0M2grbXd2?=
 =?utf-8?B?M0toN2doMUk5eTMrRFZCUFJCNGVhZzhrZ0ZBK1VXOGdHaVBGQzkxRExnVzJh?=
 =?utf-8?B?UlIyRVN1UmFoaTR2bSswT0ozdGI4NFZZSVhyMDA0Wm55YUZJMHM5UEcvakhx?=
 =?utf-8?B?N2ViTUpoektqZ3MxYlR2bGpocG1qWm80czd2SXNsL3ZjN3ZnbjZwYS9LNXdl?=
 =?utf-8?B?aGoxc2crcy8vV3dxNWNkTWcyL1RKQVZPVWlsMUFqVGdGTndKK3NWRlJoS2RM?=
 =?utf-8?B?SzJ0RUFocFFGU3MwSURYeVY0QVNoTTZMWGRHZ2JwcWFMYjlaemorZUp5c1F3?=
 =?utf-8?B?QmlqSkJXYjBJMkQrMmw2RHMwQXhzblIwZFZ4QVdKYllFM1dYVkxOTkZjVlNv?=
 =?utf-8?B?ODlsVkgwYURHeTEwUnc4WkZIZzlLMG5vVVU1SmJYa0RjM3BsQnZOc21iOW9V?=
 =?utf-8?B?VUpLRVQ1UGNEbjZaQ1BVbFdMK0ZkV3kzVVJhaXZ5Q3c5eUlTTG5JaHo5UFFw?=
 =?utf-8?B?dVo4OXBqMm94bFp6QUQyazdRUG1LWTV0UVZRVVRMdVhTSjdQWCtoQmgySW0v?=
 =?utf-8?B?SDM0ZlpFUCtqc1BFTlJtWVBjeStlbEpKVUVXVE5YTG1WRGRJdHNVTjIzcEpY?=
 =?utf-8?B?OW02NkxmZWZkOGxEY1h0TFlSSldJRHZydUtJUUJnbUR0Lyt1RVRiYTZxVzBp?=
 =?utf-8?B?cm1UU3lSYzE0bmJaQkpocFNhbVZ1U1VuUUpGa09mTmdjVlJQVzcwdXdCQVFl?=
 =?utf-8?B?NzZmNllKelZYWFhseW9PWnN6NFFvdFN6SVBTTzEwdHN1c0RVTFB1UzBuWjNG?=
 =?utf-8?B?RlAyazNLVWlnV2xWUXByZUY5LzVXTlI4MmQySG9DNFF4a3A2WWd4SlczRTlU?=
 =?utf-8?B?RXFCTTEvL1VRUG4rUm5jYTVhVmJ2MEhQdGZtUjdyNit6NE9ScDEvbFkxZ3Rv?=
 =?utf-8?B?bm16SDMvQzh6allFN2g0ZUZvMndoS1pnb2c1bXJPVVlseXpNY205WXhTem1w?=
 =?utf-8?B?QzEvY3NMOVpyakhuWmhKU2ExVXd1VDczZjZJdy9jbTZaK0FDSnd1ckVrR2dN?=
 =?utf-8?B?aVdHZnRvWnA4ZE9DRFpSVnJ6VzFSckdwekZlTzlHY2ppdlU1L1crUmpIdVA2?=
 =?utf-8?B?N1RCWGQ0MlQxT1VaT3ZMSUh2VGZYenpKTCtmZjQvc01ncnVtMWJERzVCU2xy?=
 =?utf-8?B?Y1lRaG9ZL3RUR3dMQzk3NEN4MC9pV3hJVWVIRkNxRDVBYmFncFFrMU02YnRD?=
 =?utf-8?B?Q0FGcjJoNG16Z0lIWFd3c2dMN3U4OG1ZU0pUemJ6MjVBLzZsOGVFd0lDRTJC?=
 =?utf-8?B?U0F2STk5aGIrd0E3a1ROVVhNN29IMHNNTkVKV05aeEZIUzM1QmhkM0FIcmpO?=
 =?utf-8?B?SVBDZnlaTmQ4ZFJoczcyV2IvMjdpaVpqTVl0YXExWEZhRWRmY3RsMkZaVjJi?=
 =?utf-8?B?MjY4cFVQTDRRVTdnRHBVOXQrYUw4clJTb3NmRjhLNnRnL3hBL00ydVpQNEJy?=
 =?utf-8?B?TEo3andTU0VEU1VJNVM5Tms5TVRyR3NLM2NNRDE2MFBqV1N1azErandURnVU?=
 =?utf-8?B?ZGkvMzd1aGUzbWNhU25rQlRSZG1jMW1BQzRhVTVkZFR2M2hSSEN2REN5OVRD?=
 =?utf-8?B?a3g2R29zOEUxMnRTcHVtOHVQb3JpdjhrTVRDMHZqcVlNdUw2eGxoRitzUDJr?=
 =?utf-8?B?V1lSKzlKb3pYK1IxMFZHRUxXZ3VUcnUyMmh2YWNXMUJ0QmNUVWxCeVdrTk1r?=
 =?utf-8?B?RmFnTjhEejdQV0NpQzE3YXphTDhnNmIrMGNjTjZ5ZGFlaGI5THBwa29HSmdF?=
 =?utf-8?Q?cf5KZbOmGLJ/EMn2F2Rhs9Eeg?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 61171fd1-9db3-4795-09a9-08dbf78f39cb
X-MS-Exchange-CrossTenant-AuthSource: CO1PR11MB4820.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Dec 2023 01:44:32.7767
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: NslDpMpju+JRRueTEbvXskwbiBmoWF5Gp4rmU59rsY9mQ9b/bae+XTdZMjfSEup9iL5CGrIr+/KpprLWCU1Q8w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB8504
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,
        RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Thu, 07 Dec 2023 17:44:41 -0800 (PST)


On 12/4/2023 10:21 PM, David Hildenbrand wrote:
> Let's factor it out to prepare for reuse as we convert
> page_add_anon_rmap() to folio_add_anon_rmap_[pte|ptes|pmd]().
> 
> Make the compiler always special-case on the granularity by using
> __always_inline.
> 
> Signed-off-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Yin Fengwei <fengwei.yin@intel.com>

> ---
>   mm/rmap.c | 75 +++++++++++++++++++++++++++++++------------------------
>   1 file changed, 42 insertions(+), 33 deletions(-)
> 
> diff --git a/mm/rmap.c b/mm/rmap.c
> index 53e2c653be99a..c09b360402599 100644
> --- a/mm/rmap.c
> +++ b/mm/rmap.c
> @@ -1127,6 +1127,46 @@ int folio_total_mapcount(struct folio *folio)
>   	return mapcount;
>   }
>   
> +static __always_inline unsigned int __folio_add_rmap(struct folio *folio,
> +		struct page *page, unsigned int nr_pages, enum rmap_mode mode,
> +		int *nr_pmdmapped)
> +{
> +	atomic_t *mapped = &folio->_nr_pages_mapped;
> +	int first, nr = 0;
> +
> +	__folio_rmap_sanity_checks(folio, page, nr_pages, mode);
> +
> +	/* Is page being mapped by PTE? Is this its first map to be added? */
> +	if (likely(mode == RMAP_MODE_PTE)) {
> +		do {
> +			first = atomic_inc_and_test(&page->_mapcount);
> +			if (first && folio_test_large(folio)) {
> +				first = atomic_inc_return_relaxed(mapped);
> +				first = (first < COMPOUND_MAPPED);
> +			}
> +
> +			if (first)
> +				nr++;
> +		} while (page++, --nr_pages > 0);
> +	} else if (mode == RMAP_MODE_PMD) {
> +		first = atomic_inc_and_test(&folio->_entire_mapcount);
> +		if (first) {
> +			nr = atomic_add_return_relaxed(COMPOUND_MAPPED, mapped);
> +			if (likely(nr < COMPOUND_MAPPED + COMPOUND_MAPPED)) {
> +				*nr_pmdmapped = folio_nr_pages(folio);
> +				nr = *nr_pmdmapped - (nr & FOLIO_PAGES_MAPPED);
> +				/* Raced ahead of a remove and another add? */
> +				if (unlikely(nr < 0))
> +					nr = 0;
> +			} else {
> +				/* Raced ahead of a remove of COMPOUND_MAPPED */
> +				nr = 0;
> +			}
> +		}
> +	}
> +	return nr;
> +}
> +
>   /**
>    * folio_move_anon_rmap - move a folio to our anon_vma
>    * @folio:	The folio to move to our anon_vma
> @@ -1338,42 +1378,11 @@ static __always_inline void __folio_add_file_rmap(struct folio *folio,
>   		struct page *page, unsigned int nr_pages,
>   		struct vm_area_struct *vma, enum rmap_mode mode)
>   {
> -	atomic_t *mapped = &folio->_nr_pages_mapped;
> -	unsigned int nr_pmdmapped = 0, first;
> -	int nr = 0;
> +	unsigned int nr, nr_pmdmapped = 0;
>   
>   	VM_WARN_ON_FOLIO(folio_test_anon(folio), folio);
> -	__folio_rmap_sanity_checks(folio, page, nr_pages, mode);
> -
> -	/* Is page being mapped by PTE? Is this its first map to be added? */
> -	if (likely(mode == RMAP_MODE_PTE)) {
> -		do {
> -			first = atomic_inc_and_test(&page->_mapcount);
> -			if (first && folio_test_large(folio)) {
> -				first = atomic_inc_return_relaxed(mapped);
> -				first = (first < COMPOUND_MAPPED);
> -			}
> -
> -			if (first)
> -				nr++;
> -		} while (page++, --nr_pages > 0);
> -	} else if (mode == RMAP_MODE_PMD) {
> -		first = atomic_inc_and_test(&folio->_entire_mapcount);
> -		if (first) {
> -			nr = atomic_add_return_relaxed(COMPOUND_MAPPED, mapped);
> -			if (likely(nr < COMPOUND_MAPPED + COMPOUND_MAPPED)) {
> -				nr_pmdmapped = folio_nr_pages(folio);
> -				nr = nr_pmdmapped - (nr & FOLIO_PAGES_MAPPED);
> -				/* Raced ahead of a remove and another add? */
> -				if (unlikely(nr < 0))
> -					nr = 0;
> -			} else {
> -				/* Raced ahead of a remove of COMPOUND_MAPPED */
> -				nr = 0;
> -			}
> -		}
> -	}
>   
> +	nr = __folio_add_rmap(folio, page, nr_pages, mode, &nr_pmdmapped);
>   	if (nr_pmdmapped)
>   		__lruvec_stat_mod_folio(folio, folio_test_swapbacked(folio) ?
>   			NR_SHMEM_PMDMAPPED : NR_FILE_PMDMAPPED, nr_pmdmapped);