Received: by 2002:a05:7412:3290:b0:fa:6e18:a558 with SMTP id ev16csp522021rdb; Fri, 26 Jan 2024 02:57:35 -0800 (PST) X-Google-Smtp-Source: AGHT+IFHJQx9/SyIdfUaSAOY29k6t5gUrQj28w4I5mG5MS4V2LCO047o1x2o+yFGRwMzbBgC6qVd X-Received: by 2002:a05:622a:144d:b0:42a:6c05:bb64 with SMTP id v13-20020a05622a144d00b0042a6c05bb64mr1620252qtx.9.1706266655280; Fri, 26 Jan 2024 02:57:35 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706266655; cv=pass; d=google.com; s=arc-20160816; b=frNNh8xELELE+kqtafhAoFjwiVs5xawzgWY3taBk0uAgYIOuTO5KPIKQJuZBFDuKhT tvfRJmAIPr1ZmJSvJtwGjoDl2tNlIQaLHO1jDBP1BeSsE7mDTTv+0Ind1PiwHbLwXHem 6p0kkEBXfpG3R5CYvuHMbRySxJU5W51dQu+hSPRJxXNWxRjbJYRhHKSW8SzcZCjea3iN A95jxbh1dxkNfOtVkQvX29GirKbrmos1bvmKNamVFc7ODVKzBWMgjlf/+mBw6hujT5JB BaQVRwEQQlXQtUep1YnMl1acqc40JWD0JNe2dgWt8jCbpKZTw4sl2Y0LzBQYrgHLBAZg qrwg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=in-reply-to:content-disposition:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:message-id:subject:cc :to:from:date:dkim-signature:dkim-signature; bh=QJkySAAGmHdAQNOm7UkH9kJ80ptWT6pGsZVtoPtZ6kw=; fh=px2436vcxvTDYcCBVP9YULZ/hheD2LKJVtCEdHESsMM=; b=phECpQpETbdUjgrJAa4xsjoF2VynvZAdNMJ1nYezYuyBJmz1qDgWJ4SHcqIbh6SLWD uQW8VBwEdhOkrX24DCYvwnW+oDVT7IoPLD9KtrwEo7teIov0hqoGMT+7RtPDAbSRAGot PPzUkgP5weuWwkDE2HXSndVGsxzkauySXur26ll8Zr3xnXIccT9vqlb8ZD37hK7+sXoS 3mnTBWeGsX68OjH7NcgD1mKZUh50rTXmak4BtEMnDotWO6vB7Ej+m43c2wt0wKvt1beL NrTvV69KQkgXJok0ENsUNrzdRCTLWTFyzB4hN0krvgbCHOjDVltjD7uRG5XXavqKGcmn 0iIQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=G8tAny3F; dkim=pass header.i=@suse.com header.s=susede1 header.b=G8tAny3F; arc=pass (i=1 spf=pass spfdomain=suse.com dkim=pass dkdomain=suse.com dkim=pass dkdomain=suse.com dmarc=pass fromdomain=suse.com); spf=pass (google.com: domain of linux-kernel+bounces-40009-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-40009-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Return-Path: Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id b10-20020ac85bca000000b0042a64e676ffsi949216qtb.466.2024.01.26.02.57.35 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Jan 2024 02:57:35 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-40009-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=G8tAny3F; dkim=pass header.i=@suse.com header.s=susede1 header.b=G8tAny3F; arc=pass (i=1 spf=pass spfdomain=suse.com dkim=pass dkdomain=suse.com dkim=pass dkdomain=suse.com dmarc=pass fromdomain=suse.com); spf=pass (google.com: domain of linux-kernel+bounces-40009-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-40009-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 0DC341C264E8 for ; Fri, 26 Jan 2024 10:57:35 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0604F13AEE; Fri, 26 Jan 2024 10:57:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="G8tAny3F"; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="G8tAny3F" Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 48E6412B84 for ; Fri, 26 Jan 2024 10:57:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706266639; cv=none; b=cQm4ZZgFnkRpxGLvP3tCjTxkvSNkM4u5y3mHFB6kEOz6VMmAyrWptf+O0DQm1syw0vT6ChJHRjagUyZu4lNdPhSj/SATgQAJkKodHYsQ9yn7aUVqc4cKP7hEIBOdD1Kjj3xgoiZDQFZoGrU1hhBwpHNagzCGc1ozOOtJhIKFWSA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706266639; c=relaxed/simple; bh=NsW7tmFOoZQtL0c0IKwhjYT+xrlehqti7MvzXSJf1wk=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=FwI/ukygJSdC/6pHo5Cj2Rew/xoQz9x5hD1JQsiobSMqtVnCMEhK0z/CmWLkj4e+UkKVxJR3SAKPw/JtarAGnzY6Oz92CdbpJaxvSuop/pqVdyTJ5Cn4+lEurMDliU0VjYBx8GmjmyaS8a1Q3niTVo1+kSrv225j+16ejc3sJkM= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com; spf=pass smtp.mailfrom=suse.com; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b=G8tAny3F; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b=G8tAny3F; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.com Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 7AF0C1FD9E; Fri, 26 Jan 2024 10:57:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1706266635; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=QJkySAAGmHdAQNOm7UkH9kJ80ptWT6pGsZVtoPtZ6kw=; b=G8tAny3FgdJv1kzbj03ZHA1YNTXB9ES1Ve2yiSEpPnzpjtMMPvvxuB06VYIL4bwlcEyk3G 7InyOukpfF/LAHbjY5MPbAT09kbk+5YzofzgeJ8EGFWqZUb3E52N/U6G4Xx0zoCX0b7Z7d +8VpW19RrIewxJEZg2WaAyZqu64Qm0w= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1706266635; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=QJkySAAGmHdAQNOm7UkH9kJ80ptWT6pGsZVtoPtZ6kw=; b=G8tAny3FgdJv1kzbj03ZHA1YNTXB9ES1Ve2yiSEpPnzpjtMMPvvxuB06VYIL4bwlcEyk3G 7InyOukpfF/LAHbjY5MPbAT09kbk+5YzofzgeJ8EGFWqZUb3E52N/U6G4Xx0zoCX0b7Z7d +8VpW19RrIewxJEZg2WaAyZqu64Qm0w= Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 72C9813A22; Fri, 26 Jan 2024 10:57:15 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id GpwEHAuQs2XVKgAAD6G6ig (envelope-from ); Fri, 26 Jan 2024 10:57:15 +0000 Date: Fri, 26 Jan 2024 11:57:15 +0100 From: Michal Hocko To: Zach O'Keefe Cc: Charan Teja Kalla , akpm@linux-foundation.org, mgorman@techsingularity.net, david@redhat.com, vbabka@suse.cz, hannes@cmpxchg.org, quic_pkondeti@quicinc.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Axel Rasmussen , Yosry Ahmed , David Rientjes Subject: Re: [PATCH V3 3/3] mm: page_alloc: drain pcp lists before oom kill Message-ID: References: <5c7f25f9-f86b-8e15-8603-e212b9911cac@quicinc.com> <342a8854-eef5-f68a-15e5-275de70e3f01@quicinc.com> <5adb12eb-8403-5860-28eb-5f6ab12f3c04@quicinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5adb12eb-8403-5860-28eb-5f6ab12f3c04@quicinc.com> Authentication-Results: smtp-out2.suse.de; none X-Spamd-Result: default: False [0.17 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_THREE(0.00)[3]; DKIM_SIGNED(0.00)[suse.com:s=susede1]; RCPT_COUNT_TWELVE(0.00)[13]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-0.23)[72.63%] X-Spam-Level: X-Spam-Flag: NO X-Spam-Score: 0.17 On Fri 26-01-24 16:17:04, Charan Teja Kalla wrote: > Hi Michal/Zach, > > On 1/25/2024 10:06 PM, Zach O'Keefe wrote: > > Thanks for the patch, Charan, and thanks to Yosry for pointing me towards it. > > > > I took a look at data from our fleet, and there are many cases on > > high-cpu-count machines where we find multi-GiB worth of data sitting > > on pcpu free lists at the time of system oom-kill, when free memory > > for the relevant zones are below min watermarks. I.e. clear cases > > where this patch could have prevented OOM. > > > > This kind of issue scales with the number of cpus, so presumably this > > patch will only become increasingly valuable to both datacenters and > > desktops alike going forward. Can we revamp it as a standalone patch? Do you have any example OOM reports? There were recent changes to scale the pcp pages and it would be good to know whether they work reasonably well even under memory pressure. I am not objecting to the patch discussed here but it would be really good to understand the underlying problem and the scale of it. Thanks! -- Michal Hocko SUSE Labs