Received-SPF: pass (google.com: domain of linux-kernel+bounces-23994-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Message-ID: <b2096bcb-bda6-4831-85a2-67759e783e4d@oracle.com>
Date: Thu, 11 Jan 2024 10:54:45 -0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] mm, oom: Add lru_add_drain() in __oom_reap_task_mm()
Content-Language: en-US
To: Michal Hocko <mhocko@suse.com>
Cc: akpm@linux-foundation.org, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org
References: <20240109091511.8299-1-jianfeng.w.wang@oracle.com>
 <ZZ5Zb3FYqY8FZgB3@tiehlicka>
 <1d866f1b-94b3-43ec-8f4c-2de31b82d3d1@oracle.com>
 <ZZ-q0PZ-XCDwA4oG@tiehlicka>
From: Jianfeng Wang <jianfeng.w.wang@oracle.com>
In-Reply-To: <ZZ-q0PZ-XCDwA4oG@tiehlicka>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?bzd4T01yRkNhNkZDcXhnenp4d2lObDdjN29IZFh0WFcxZkwwUDZiTmdiUC9x?=
 =?utf-8?B?QnVubDhDT3NNdU9RQk9ybTVyMC80WEg5cTB4TmdpVG51QjhiemRRZ1pFbGQw?=
 =?utf-8?B?cXJGK3dpNlZCb0xic3R6M2xOeHd5eFliR3BhZGFpQkZpWnlXV3ZLM3pwR2V3?=
 =?utf-8?B?R3hmbGRrRCtyQXg5NzN2cDRZVXV2SDF4anJjZy9qVzZCUUMyNVRkcTR1clBp?=
 =?utf-8?B?d0RmMk5oUXloU3VsYkk2MVkwaUVFSXpYRXMwSExLTWdoMzZycWpKYU1Cc0Qy?=
 =?utf-8?B?THcxdU9EZXlTL1AxRXdCT2J2L3JYNkJhbWZMNGRlUnBLQU1aSVJPcW1rc1JQ?=
 =?utf-8?B?eWpZQ1lvWUhhbjhtckI5SkR6MmFuR0w4d251a211TStHaitrRlJpNncrTzE5?=
 =?utf-8?B?M29ITjBJbEI1ZjFoTmZJbmJzbHduY3d2WmorNk96bklneElyd1FJNTdscE1P?=
 =?utf-8?B?TDh0bitoRmg2aXRJa0lyUndiMG5VTFhITGVrSWhLRHdYOUVxWFNabTdOR1JN?=
 =?utf-8?B?MkdSNnlZekFVWW9CaHlPNUFkYWZFNXNHa2draUlKdHBoZTcyNXJwZm1wa3pl?=
 =?utf-8?B?ZVZZc21GWHFMNG4zWFVXSmV3SVlmNU5XY0VEMkdEUnFNbkJtN1lOTjNNYVNL?=
 =?utf-8?B?c2tjcFUxbVFrWmdtNHhuUGRqOFBReDVVdHQ4aGNKbVpyZ290ZVhyZmhKSVFM?=
 =?utf-8?B?R2lYZS9IOFpHbnlFN0xFZVZuRHNwYnpKUWRZYkhzeE5iM2krWWwrUFlaUnp4?=
 =?utf-8?B?dmN2U2FQRDA4N29DR3hRK01udVVRbzNmVTUyelhEb2xCOE1VZFMxc0JxM3BC?=
 =?utf-8?B?bDJtbkwxQ29VS2MxOGEyV0I2eGNhK1libVd1bURERUg3VUZydVdEVklmUkMw?=
 =?utf-8?B?TGQzVlpuTUtQTk5CdmxXZVZobDZDYjIvSmovMElDaTduSlpDdjMzd2lsclF1?=
 =?utf-8?B?TUY2d2JLa2tGSmVtNzlTUWREUUpVWitmU2cvSXZIQjNyZzNUYVFMM3gvV3pa?=
 =?utf-8?B?SXlsN3k3aE5DLzVvcWl1TG9iaEtlcHh0Z1R1QUZoQ1JlUWdvRk5MMzUxbXRU?=
 =?utf-8?B?K2NzWDQvVFZqSTU3YmduSzQyaW5iU1pGZVRWOHZHalMwTTdYckZ2ZXQzYUYz?=
 =?utf-8?B?dDZWcm9KbDN4OFVKZWQxcXIvWUxTaTJ6VFZBUXMrNVRvNWRvYjBQeEViTGlj?=
 =?utf-8?B?VzRlWEowVmJFWGdaUDA0cU1EQmN1RTRBMDRORGFWZ0hEVUZEMmFVWDRDTEln?=
 =?utf-8?B?eHhCWldBYTZOcFlhaldtbGR5NmhzY2JtMGkzUklQTjVTT21ET2I1bWhMbnZG?=
 =?utf-8?B?OU95Q2VaNVVqNXdUMXNCUHpmSjl6Wmk3aEtabmZkV2hqa3NMc2xnSXVEOWNK?=
 =?utf-8?B?S2tJMFpjellCSDh2bkdSdmMva0M3RytTeU9wU25SN0V1bUhTOW9HaGtMWUFT?=
 =?utf-8?B?cElVZ3Zid1o5NnRHbnc1WFM2MWJTOHFadlAwM3RKbmtnR3B0NXRFRU9yNy9Q?=
 =?utf-8?B?R3F0V2lJVmY4UStrTzBmUFYrNHpaTHViK0FTNzJ6QmkvbkJNUUpqV092VUhl?=
 =?utf-8?B?QUFpMVE0SitSVk4wNk5BenN1TVJrdjNRT0hhQVRHK28rV3JaN3FCMlhUNEtk?=
 =?utf-8?B?L1kzTERQOGJxMy9BNkNjbUUrVUVCSGxzR3p1YnprNnAxUkMrRGxkalZDb3lT?=
 =?utf-8?B?c29WVFlLakdPeDJiWjFJWDFvejArblNUQUFYdzZ4TVIxUzZIZlJ3VlRpbDN6?=
 =?utf-8?B?V2ovcCtrYTF0TDJwNG4xa2ttZkVzWktyelhFalQrOUR3YjBYRklmZ0FvUzR1?=
 =?utf-8?B?VTZnbzF6RVpkcDhENHpXVGlBVERmUGxuL1Y4QmlndTk2a0ZUbkNpSjlYOUZG?=
 =?utf-8?B?WEhQak5QL3ZTbXoxK1BRdmo4ckl3ckIyeGJ2SDRjbjl3ekFCMHJqMXdjdjR1?=
 =?utf-8?B?RVNMNEt4cWUwV09sM0NtbzNuRlQzdXIydDErRG9nelRNYjFuZ0huelVFeUhZ?=
 =?utf-8?B?RFFqak80Mk9oOGtPUFRFcnpDRDd1blZKdUdmVXVrZHliT05kUVl6YW9OalZN?=
 =?utf-8?B?MGd3K1N2bEp5cnF6UDh3bXhDUFNhSklpd0hQdlpEL01Nd0Q1dGhEL3ZtRDY4?=
 =?utf-8?B?MDltb3BHWStTenU4NWJuVlU5NW1kdDZsdkxJaXdsaGNJa2NLSGRLWTRheGNj?=
 =?utf-8?B?VkE9PQ==?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	5+LYF6rO/7avDb7lzx81C8+4TAcGZ18NXT6e436MSItCF1Oj1gOXCtJ9ImJx3AW5mXvGMay00Hsxkss+L9Qw/eUDD3Aw35rRnsLAXUb8Ih8cjKHMSNYq97n3J5UlUZAlaWt3KwXfyzL1iKQuBm8QNvEs3n9CrlJewoMTENSR6vDRvmwaRHF9fZ6sqYyy7H47bfk6/jG82SwKaFzP01pKB5MATpRS3NdFAai6+GrWeLCzJQSYF3NwUT9ii3qoQzkLqyoM4gjlGxrdL+lInVwe/qnphdHtKvbK5z82YQVuI01sfTTcwOcKMCzl5usqlCs14O+Vs6iAFod0p1HVr8s0daCqass/1/el5vsCTLHp3lV9MIKzm5nRkI+oWPR9VWQLIUMP8JGl9t3XfoXy8Eqj30DIq8uq39D/sahSNZbZAHyFxPadQMDBEEQtngZ0/F4Rzs7PH9XrFKmU5siu2l43rvABJyCOn8Fyg4sqyjt3NOnvMitNADuflh5UWSbsCUoWGXl5yecLc/HK8pa3ZUnD6Wkinwq1hpfcaSeGT4/ySgFejreI++0NvXvnt7gP3g0/apD4LDriNKURyb21aCbxAIg5vXKIXxAUyvVbXy+mLDg=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 839b779a-0701-430d-fe03-08dc12d6c893
X-MS-Exchange-CrossTenant-AuthSource: MN2SPR01MB0026.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Jan 2024 18:54:47.8364
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: rMiK3yRNC2Xu3zqk+d9Podw2CFiG9VbMbDv6NoXrGLUwqBrYJbF322LyVveDRVaHTZSDkuvMURaZLFf54zjfNZPF2pqWKql6LqNQ///bT0A=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR10MB7437
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-01-11_10,2024-01-11_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 mlxscore=0 bulkscore=0
 mlxlogscore=999 adultscore=0 spamscore=0 phishscore=0 suspectscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000
 definitions=main-2401110148
X-Proofpoint-GUID: MFrnyKBCorDDXkv9WWkxWbZVo2QaVHL4
X-Proofpoint-ORIG-GUID: MFrnyKBCorDDXkv9WWkxWbZVo2QaVHL4


On 1/11/24 12:46 AM, Michal Hocko wrote:
> On Wed 10-01-24 11:02:03, Jianfeng Wang wrote:
>> On 1/10/24 12:46 AM, Michal Hocko wrote:
>>> On Tue 09-01-24 01:15:11, Jianfeng Wang wrote:
>>>> The oom_reaper tries to reclaim additional memory owned by the oom
>>>> victim. In __oom_reap_task_mm(), it uses mmu_gather for batched page
>>>> free. After oom_reaper was added, mmu_gather feature introduced
>>>> CONFIG_MMU_GATHER_NO_GATHER (in 'commit 952a31c9e6fa ("asm-generic/tlb:
>>>> Introduce CONFIG_HAVE_MMU_GATHER_NO_GATHER=y")', an option to skip batched
>>>> page free. If set, tlb_batch_pages_flush(), which is responsible for
>>>> calling lru_add_drain(), is skipped during tlb_finish_mmu(). Without it,
>>>> pages could still be held by per-cpu fbatches rather than be freed.
>>>>
>>>> This fix adds lru_add_drain() prior to mmu_gather. This makes the code
>>>> consistent with other cases where mmu_gather is used for freeing pages.
>>>
>>> Does this fix any actual problem or is this pure code consistency thing?
>>> I am asking because it doesn't make much sense to me TBH, LRU cache
>>> draining is usually important when we want to ensure that cached pages
>>> are put to LRU to be dealt with because otherwise the MM code wouldn't
>>> be able to deal with them. OOM reaper doesn't necessarily run on the
>>> same CPU as the oom victim so draining on a local CPU doesn't
>>> necessarily do anything for the victim's pages.
>>>
>>> While this patch is not harmful I really do not see much point in adding
>>> the local draining here. Could you clarify please?
>>>
>> It targets the case described in the patch's commit message: oom_killer
>> thinks that it 'reclaims' pages while pages are still held by per-cpu
>> fbatches with a ref count.
>>
>> I admit that pages may sit on a different core(s). Given that
>> doing remote calls to all CPUs with lru_add_drain_all() is expensive,
>> this line of code can be helpful if it happens to give back a few pages
>> to the system right away without the overhead, especially when oom is
>> involved. Plus, it also makes the code consistent with other places
>> using mmu_gather feature to free pages in batch.
> 
> I would argue that consistency the biggest problem of this patch. It
> tries to follow a pattern that is just not really correct. First it
> operates on a random CPU from the oom victim perspective and second it
> doesn't really block any unmapping operation and that is the main
> purpose of the reaper. Sure it frees a lot of unmapped memory but if
> there are couple of pages that cannot be freed imeediately because they
> are sitting on a per-cpu LRU caches then this is not a deal breaker. As
> you have noted those pages might be sitting on any per-cpu cache.
> 
> So I do not really see that as a good justification. People will follow
> that pattern even more and spread lru_add_drain to other random places.
> 
> Unless you can show any actual runtime effect of this patch then I think
> it shouldn't be merged.
> 

Thanks for raising your concern.
I'd call it a trade-off rather than "not really correct". Look at
unmap_region() / free_pages_and_swap_cache() written by Linus. These are in
favor of this pattern, which indicates that the trade-off (i.e. draining
local CPU or draining all CPUs or no draining at all) had been made in the
same way in the past. I don't have a specific runtime effect to provide,
except that it will free 10s kB pages immediately during OOM.