Received-SPF: pass (google.com: domain of linux-kernel+bounces-95071-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Message-ID: <a32030f0-b6f6-4bd6-8284-bb762ccf9205@vivo.com>
Date: Thu, 7 Mar 2024 15:40:29 +0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH -next v6 0/2] Make memory reclamation measurable
To: Michal Hocko <mhocko@suse.com>
Cc: Steven Rostedt <rostedt@goodmis.org>, akpm@linux-foundation.org,
 mhiramat@kernel.org, mathieu.desnoyers@efficios.com,
 linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org,
 linux-mm@kvack.org, opensource.kernel@vivo.com
References: <20240105013607.2868-1-cuibixuan@vivo.com>
 <fac8d079-100e-4b8d-9a35-db8219b28b51@vivo.com>
 <20240220212202.59ddc123@gandalf.local.home>
 <3d4f44ee-f533-446f-a9e6-7f58afc78d65@vivo.com> <ZdWpyN_8Z6dvDQ48@tiehlicka>
From: Bixuan Cui <cuibixuan@vivo.com>
In-Reply-To: <ZdWpyN_8Z6dvDQ48@tiehlicka>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?dWxYMU1taTc5VGdEOTV5dmRqTGMyMFZ0bUUwM3NIRUF1NjcwOGhRc2I1S2FM?=
 =?utf-8?B?NldXMXU5dnJHcTQ5ckpZUzhIUWdyeEY3L0VRL1A1Y3VnRlkyaXcvYjVCazVi?=
 =?utf-8?B?TzN1bzhvSVp0OVc3VnhGdXowNFhKd0xSTFZQU1ZVdUhhZS9lSmh5ZTdIbi94?=
 =?utf-8?B?cHMyYllZV2U4RkxmeHVOOHNaamFuOXkxclpkVDVDdm1mdVJqVVZJNWZVTWN2?=
 =?utf-8?B?YjNjOWFrNW9qdjhUa3ZBL0MzQ1FiYXdYMjhGNmZ6WEpSWm8zMWVyQmpKVS9B?=
 =?utf-8?B?NlpleEsvVVVpc3pJTDUxOVFBb050ZkhoWGdlSElYTnpEdkZlYUxNaCtkMVll?=
 =?utf-8?B?QVFLSStyampPK2oxNlZPSTBXS1c3eUM2OWpEdXFqZWNTN28vVzZ4QSs0WDNL?=
 =?utf-8?B?QS9OTDdVTTl3bk8yOW5SdTVXL2MwQzBXN3Z0V0gxQ1VSYUJOL2kxQmxZNmtG?=
 =?utf-8?B?dTRjdEpRLytQYnJXVVBPM3B1TXBqTENvdUxyUFNQR2xUMUhPTlhuV1NsYWlJ?=
 =?utf-8?B?V1krYmhKR1NURHhLRnlTUTFlU3VLamh4Qk9FYlZBNGJPYVZZV0QrbnkxaG5N?=
 =?utf-8?B?eUpnSDdpcVppTjF1bTljYjZLYlJXZXVFTVJxSzdLVXNHN3A0K2Z6ZzIyak0v?=
 =?utf-8?B?cWNmd21Tdk8wMGphSjFEajczSGkzcDNrTnNaQit3aTJLUE5TNlQvLzFtWWh4?=
 =?utf-8?B?ZnNQMkZQblpER0U4WW04Y2RlS1dnSFRLUHpkTVZwdTBIN0Y1TXpYdkZJc05o?=
 =?utf-8?B?eDI3NXEvcWFsd1prdnBGSjVoaGx5a0VmcytzbGJBbm8rejNxRnRkeVVLMWl0?=
 =?utf-8?B?TW5jNFZ5dlErOEk5WS9jV3J1VXgrcnpqV1Y5ZHJqYUJtanBHM3VXanFSQnVC?=
 =?utf-8?B?ekR0ZjhNZFpCbXcxR2wrbkRHY3hwTjA5a1hlUDJXRVRzN0hOcml5MmF6SUlJ?=
 =?utf-8?B?bmNMTTN1UmNZbkhnY2lES0Npb1NOZjZ3WHJFMEcvWStxNDBmVUQrVndPZEJG?=
 =?utf-8?B?VjFsOVJZem1TUzdOelVWUy9NWjliUDdtdklHZTFySTRwY0FMejV1dDc3UVQy?=
 =?utf-8?B?dXNiUG5kck8xNXlRM212c3hWYUptcVhidjgvS1ZkeUQ3eWdZM3ROKzdsWUtw?=
 =?utf-8?B?MlcyeUVqVk1NQStvWEo4V09LV2Zaa1FURHBXUUt0WW44SitINTFPa1NOR0hp?=
 =?utf-8?B?M2J5emRUcWpvRGI4WjhpemlrQ1VtNTRoL0lKdWpSK29UUXByTUdOZHdFUnBp?=
 =?utf-8?B?d1JvZGludnIycll6V09EMmVQQjh0MkUyZ3R2aVd0MUR2aWZpOXdMNTZTNktz?=
 =?utf-8?B?clFPZkxhUnI3bWdLOStEdWlNcEw5WURaNkMwdEpaeENFR0I3eXUwMlJwZnBv?=
 =?utf-8?B?WW5NSlN0VlYxUk1wc2p6V1VzSWJ0QU8zSXZ4dUJqUEpSNVpxWjVJMzhFRHdt?=
 =?utf-8?B?TEdFSmlhM2hyNVRENzhFQVJZVWY5ZEZET215UGttREN4M1Q3bEJlQkV5MlRk?=
 =?utf-8?B?WkhQc2VFSmEvVngvZlhmT0dzMjIzbStmR3JJbmZ5NVQvYk9hNlJRcTYzbm1W?=
 =?utf-8?B?SW1EVm9DaWJEUG1DSUZzcEJSYy9ZczJPcm95V0VBNVVwdE5RbFZpK3N3K0tF?=
 =?utf-8?B?dEJoSzNYZUdVTS9OdlJuZmRGa3B6bUZISW5YS1VpaWtibk93Q2NFOEZtOVFK?=
 =?utf-8?B?dGpwUmh3RkduMmNqN3VYZWR0MmVsejgyUVNrZ25zNXNmNlJYdDdnbHhQbmRF?=
 =?utf-8?B?Nks0WkhCZnRLbkQ5eEYyUTc3eGhEUnRqL01MMThnZFNvQ3Y4OEhuRHBTNzA0?=
 =?utf-8?B?M3ZOMTVRS3hRajJveHBLaTBGY256elJzdUg1ZEtlTk5RVkx6WFVMV05YTU54?=
 =?utf-8?B?UlpYUWpyWnd1KzBGTjhZMUsxenpHd3MvR2xmcFhTZjJQS3djdUNrcG5XUTFZ?=
 =?utf-8?B?WTZQcVF6VkxuZ1VkelJDUkE4RjlZYUk2R2JxcVo3eEV5bkJjbVhlS2FVOEhF?=
 =?utf-8?B?Z3lJczFRenVUa0pCTkhlTWZUVmN6dEpLSHVJdEQrbkZTeUI0Y3NJYzMycGdP?=
 =?utf-8?B?Umw3c1N6b1RQbW84eitDcENleFdhNDFvbU54NkhZQUl6SlB3UTRwZ0VCUzg0?=
 =?utf-8?Q?jx/fxTz5+z8mRZYNA5WvnIX7j?=
X-OriginatorOrg: vivo.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7d1451c9-5162-4d39-5ae0-08dc3e79ed74
X-MS-Exchange-CrossTenant-AuthSource: TYZPR06MB4045.apcprd06.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Mar 2024 07:40:57.7240
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: oYGC6LZ6KH4ZWTYJ3qtKZBC0wI5ex7h3fEUPqexFbGQXXRmZZyBipwunnBXkQjTwFcsTxjNnKT887LUMGmV/VQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SEYPR06MB5962


在 2024/2/21 15:44, Michal Hocko 写道:
> It would be really helpful to have more details on why we need those 
> trace points. It is my understanding that you would like to have a more 
> fine grained numbers for the time duration of different parts of the 
> reclaim process. I can imagine this could be useful in some cases but is 
> it useful enough and for a wider variety of workloads? Is that worth a 
> dedicated static tracepoints? Why an add-hoc dynamic tracepoints or BPF 
> for a very special situation is not sufficient? In other words, tell us 
> more about the usecases and why is this generally useful.
Thank you for your reply, I'm sorry that I forgot to describe the 
detailed reason.

Memory reclamation usually occurs when there is high memory pressure (or 
low memory) and is performed by Kswapd. In embedded systems, CPU 
resources are limited, and it is common for kswapd and critical 
processes (which typically require a large amount of memory and trigger 
memory reclamation) to compete for CPU resources. which in turn affects 
the execution of this key process, causing the execution time to 
increase and causing lags,such as dropped frames or slower startup times 
in mobile games.
Currently, with the help of kernel trace events or tools like Perfetto, 
we can only see that kswapd is competing for CPU and the frequency of 
memory reclamation triggers, but we do not have detailed information or 
metrics about memory reclamation, such as the duration and amount of 
each reclamation, or who is releasing memory (super_cache, f2fs, ext4), 
etc. This makes it impossible to locate the above problems.

Currently this patch helps us solve 2 actual performance problems 
(kswapd preempts the CPU causing game delay)
1. The increased memory allocation in the game (across different 
versions) has led to the degradation of kswapd.
     This is found by calculating the total amount of Reclaim(page) 
during the game startup phase.

2. The adoption of a different file system in the new system version has 
resulted in a slower reclamation rate.
     This is discovered through the OBJ_NAME change. For example, 
OBJ_NAME changes from super_cache_scan to ext4_es_scan.

Subsequently, it is also possible to calculate the memory reclamation 
rate to evaluate the memory performance of different versions.


The main reasons for adding static tracepoints are:
1. To subdivide the time spent in the shrinker->count_objects() and 
shrinker->scan_objects() functions within the do_shrink_slab function. 
Using BPF kprobe, we can only track the time spent in the do_shrink_slab 
function.
2. When tracing frequently called functions, static tracepoints (BPF 
tp/tracepoint) have lower performance impact compared to dynamic 
tracepoints (BPF kprobe).

Thanks
Bixuan Cui