Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5;
Message-ID: <cbebac71-0be6-ae66-02b3-243d0f8c39e8@oracle.com>
Date:   Sat, 21 Oct 2023 11:20:13 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.14.0
Subject: Re: [PATCH v8 7/8] hugetlb: batch TLB flushes when freeing vmemmap
Content-Language: en-US
To:     Mike Kravetz <mike.kravetz@oracle.com>, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org
References: <20231019023113.345257-1-mike.kravetz@oracle.com>
 <20231019023113.345257-8-mike.kravetz@oracle.com>
From:   Jane Chu <jane.chu@oracle.com>
In-Reply-To: <20231019023113.345257-8-mike.kravetz@oracle.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?cUVZaUdvYm83aVgvRnBlV0ZCTzVPcWl6Q1RBZXFUcDVXNC8zay9GSTdySkY4?=
 =?utf-8?B?WWxYMFM1WWQ0bWs4MXJ6OXk5VkUrdzJKL0ZDTEZXWkNBOXZUVDRodmoxazRt?=
 =?utf-8?B?b2RzRmx6Wmd0S2FlUHNiQXNlcVBaNzljVnVkWWVNU3BuSEhHR240enZUZ3F6?=
 =?utf-8?B?Z2xVdEw1dzRwR2Y4TnM4K0pkOTdzSkxkQWVQaG53WExsbW9DMHdSNzNDUWtQ?=
 =?utf-8?B?dGJaK0RPV1pkcEphakJib0FlMWVBdDZsZjc5WEFoUlprNUs2SGwzMDdmVVR1?=
 =?utf-8?B?MGYxRFRGYnl5YkFFaENRTk41NUs1YmNmbEtzckFDa2R1WHN5VHV1MUJRNWU1?=
 =?utf-8?B?VkxKWkdSRVdNd0IxaXZUbUxTd1JUSXFjdHVxUnBCaHBXNTlzcWxJWnJEdGVx?=
 =?utf-8?B?TkY2YTlKSkVqMDBzSFFDL3hjOVlxZzZpTnh0VlBlNlRFZnZTMyt3OVBZVmhp?=
 =?utf-8?B?ZisrZjlNMzZ5Rm9hd0pzRXhpOFBXT3g4dWlwVVllcnBzN3dkL1VCSDVmWE9l?=
 =?utf-8?B?N0l0YlBib0ZFVmE1bVpiVVExV2NVeDA0U1V4cnBDYXo0L3RKRFBUL1lUNllZ?=
 =?utf-8?B?Rys0NmRsL0xIVWNDWHYzSnhFUi9ucjlpWTBtb0RKWjcvZXFxZlhQTzRsS0U3?=
 =?utf-8?B?SE04bUlpbWhRTlZyNVArMEhYb2EwUTR0eUZUeXJNMGx5cDBzYXY1REc5b1pu?=
 =?utf-8?B?dzFDNm1nVlMvaTRuNUNOczZaSDhZNzhXOVIvU2cxRzc0VmRtMUlXTncxdGtY?=
 =?utf-8?B?MHNJT1VzUDIzWWVrUlBtN2VVQldvMDBIZjJ1VUt0YktmUmF2ZFloNnM3aFhK?=
 =?utf-8?B?SEowcmtKb0ovV3JYQzQ1WnFPU001TVAxWkxaUkpoN1FTNjJuSmR4d2hwODcw?=
 =?utf-8?B?eEFjYTZYRFUrWTRucU9NZ0xyVjN5TXlpT2hiamlqZXJDVzBUNlBuUzVkVVNV?=
 =?utf-8?B?VjBDbi9zT2ZzanBaVHEwaXRqOHFBQmF0SXp4Y0lsUDJXb25YWHNSV1FXTFpQ?=
 =?utf-8?B?ejg3VTZqOWdHeVYrL1RWSjBobytyTnN2clJjTG9DVWJnRHlxdHdwYWJZNklG?=
 =?utf-8?B?ZHBkOHBOZ21tZEFtamNHVVJTZ3hGZnJUS2k2NXVOcUhpMzNJUHdyZWg4clJT?=
 =?utf-8?B?NjNkVFd0VHp5Tys3bkxST2M2cEROL0tybHE4NXV0SW8vZW9lRTUxZkdTelM1?=
 =?utf-8?B?bG1qWXhxZTQwa1FuZlZsbWFtUUZPbjhHTHpITkloSzZzaDRtUHdYYTZ5TEE5?=
 =?utf-8?B?VHdqaHBvL3AySTJDY25XQy9qbkRrK3dRS0JOSnc1N2NTSy9xaHRua25vYnAy?=
 =?utf-8?B?b2hjWmxWTXhyK1hLT3lZeXA0NE54L21PNTRXaUpnaHJUT2Zjb3JwSnh6S2Ro?=
 =?utf-8?B?MzcvNVY4VStwUTVBaGJONTgrd2pocDdVL0k2VzVHdTRwcUd3K2dJTjBnWHJV?=
 =?utf-8?B?dU5WTHJNMkhpelpsOW9la3dTMTlQRHpkNU5ld21tT0gvcWQrdlBla3pHYlFs?=
 =?utf-8?B?bVJrNE5EMFJRWUdvWGtERkZYMlpoeU5sTUwrSU93WFdKL2thb0F6RmMwTis1?=
 =?utf-8?B?OTEvSG5lR1dUZzhSQmhFN1NNRldTZmVEVEVhcFJPd3Z0VythR3pUcm9JQWhi?=
 =?utf-8?B?eEFCQllETmJtUS9LeFA3dWg5bm9YNGdqVDRDN2UzenFTM0NBaTJTVmQybGtC?=
 =?utf-8?B?SlQzV2lnd0tPYXcybVZESXptd052WDJOZjJiMWthNER6bE10YTd0ZnlNcjBV?=
 =?utf-8?B?WXdwZEpoQ0hDR1NRcnZBRGRPTUF5Q1paUGFYcGU3bUtXVFU2eEI0S0xta0xi?=
 =?utf-8?B?ZDdjd1JUK1FIc1R1K2R2ZDBVNWxlSDFWRVU2b0NDSnZ2cHIyMmFxbVJ0bkhw?=
 =?utf-8?B?Q1l6TU80bzNDeGh1MkxoakFwYk9naWhkaWV6dmRtc08zcUlYMzlkVlA2R2Rt?=
 =?utf-8?B?SnN3cHByeUxRbk5oVS83My9vZFhVU0NwTHc5cE9NWEVmZUJ6T2plOFFYTXRB?=
 =?utf-8?B?WWJTbkgySHh6YXY3S0tuTlFhRDdJTFhUSSsvbjNNN2ExdDd2bUNVZkV1SlRH?=
 =?utf-8?B?YkdmZlI1N1hUbHVCOUFvOUlmNTdrYjJsRHJMcUt6dHkxTUw2Wmg4bzR3Y0RR?=
 =?utf-8?Q?6ozqMttB3Y4XfM+FD8txPJqSu?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: FFdT+7tKVmIYhqTTKt79FR4G0klmmEB5IQ4fi4f/gZqyflSPTfkGTQH/CGGl8AC6qGrMvBXFD8zsT0M3AWycaQcsnGlRYWxQ1SaRWNjdf3pxglbrs1+QrZ14TsbwXeDu/hyREjV7HQulBrh/bOFaxrqkqqAh8uKXUwExlSNEK8dioY4kZtvsYUi6qCW7+IROd7fUGSF8f4mY33CU6FmDZYhacGCmCUAa9CELH15CuMPBfNxEKEh6YtdlpbimTxtuPKaIOMTzkzIPyAbDl8Gau8hRNDHr6N3TOfJdFtd3La3VQjK9IxtElB9/3tyHmFKdQdYIjHI5LHEM0GCwEmrX6yDoxGUR6OH8bABMbX9vnprBg80luCQh8gEWF+M4nOpLpTmFQ5WnNKQx1MSPHp28CAjebmEot/6AOsR3vRTskGgyd3ViC90Ti5fOs428ueGU+pl70BVQYT545eoD/S4yKXP1C7PxLwEEzjVTARz/Fawb0K9vkcFxlh3qG1bNK6BZAgBdYrMHf8t3V0ltkb2mInvBpZJG2uxmkdzIe89elZALM6DHgN/1rBGkcmqVxdtU0NVhYC9Qjf8qV1uVD0JfhVch9hLvRZOI1mDgaOaL5Aa7UbF/kkSVHfsq9EPbAMYdsMsx8D4rWl69J8qI5bMjOIr9OsF3GUFDBXhW91F8gZ1BSrmZyN0sU4xmSP5matUi/oQ8V550r19UwsJMwAp57c7jqHGxEyayHnpRar9FYLmm/cfRUItTJN2BsZR78bl1RgB1JKlrTLPdshu5GsJYIeKioxW23zY+x2f7z9kye8I=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c92ca7e3-9bda-466c-4f28-08dbd2626035
X-MS-Exchange-CrossTenant-AuthSource: SJ0PR10MB4429.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Oct 2023 18:20:16.7592
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ouzhfzPAduHYNtU4nIqWaUA3DXt+Hv1OLamo7jXFOfA6U/vwk0kiKGb//7G6+cPTMjGsFE5hoyd0yRuDlG3imQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR10MB6336
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.980,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2023-10-21_11,2023-10-19_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 adultscore=0 suspectscore=0
 phishscore=0 malwarescore=0 mlxscore=0 mlxlogscore=999 spamscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2310170001
 definitions=main-2310210169
X-Proofpoint-GUID: 2RODblrgPABLdZkFt_xgTsXZwVmpcjrV
X-Proofpoint-ORIG-GUID: 2RODblrgPABLdZkFt_xgTsXZwVmpcjrV
X-Spam-Status: No, score=-4.2 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Sat, 21 Oct 2023 11:28:43 -0700 (PDT)

Hi, Mike,

On 10/18/2023 7:31 PM, Mike Kravetz wrote:
> From: Joao Martins <joao.m.martins@oracle.com>
> 
> Now that a list of pages is deduplicated at once, the TLB
> flush can be batched for all vmemmap pages that got remapped.
> 
[..]

>   
> @@ -719,19 +737,28 @@ void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *folio_l
>   
>   	list_for_each_entry(folio, folio_list, lru) {
>   		int ret = __hugetlb_vmemmap_optimize(h, &folio->page,
> -								&vmemmap_pages);
> +						&vmemmap_pages,
> +						VMEMMAP_REMAP_NO_TLB_FLUSH);
>   
>   		/*
>   		 * Pages to be freed may have been accumulated.  If we
>   		 * encounter an ENOMEM,  free what we have and try again.
> +		 * This can occur in the case that both spliting fails
> +		 * halfway and head page allocation also failed. In this
> +		 * case __hugetlb_vmemmap_optimize() would free memory
> +		 * allowing more vmemmap remaps to occur.
>   		 */
>   		if (ret == -ENOMEM && !list_empty(&vmemmap_pages)) {
> +			flush_tlb_all();
>   			free_vmemmap_page_list(&vmemmap_pages);
>   			INIT_LIST_HEAD(&vmemmap_pages);
> -			__hugetlb_vmemmap_optimize(h, &folio->page, &vmemmap_pages);
> +			__hugetlb_vmemmap_optimize(h, &folio->page,
> +						&vmemmap_pages,
> +						VMEMMAP_REMAP_NO_TLB_FLUSH);
>   		}
>   	}
>   
> +	flush_tlb_all();

It seems that if folio_list is empty, we could spend a tlb flush here.
perhaps it's worth to check against empty list up front and return ?

thanks,
-jane

>   	free_vmemmap_page_list(&vmemmap_pages);
>   }
>