Received: by 2002:a05:6a10:22f:0:0:0:0 with SMTP id 15csp837058pxk; Thu, 17 Sep 2020 18:30:03 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxYL4bDCNwLY9DDVK6ihrpN3/eRIYH/5A6o4vtpbXtPoF7FjOJZIPud+iAUjq66t4O24FeL X-Received: by 2002:a17:906:aec1:: with SMTP id me1mr16807560ejb.225.1600392602867; Thu, 17 Sep 2020 18:30:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1600392602; cv=none; d=google.com; s=arc-20160816; b=eZYNQZDFQSsYUJa0Y3eV0s/AJOXqZvrDdHzNzP0C2Rh0UIK3+rI9TGC+XgSw18mbQ4 EQ9EKwFf2FJ9HrBLnMaCva3nZC1NwerhIZehRlP4TIDFBH2frs5O10+6LbK2BhiLp8nq Jatp9ihOv9DFR27GaLTG9PSOqD9aWUNx78sT3Mo8tKbuvIvP3SQ1wbrsNTVWdidoa0ZP yA/BdI16Ps3xu/6WwAt5ZcImwCk5KbhB/HSaVqyV7IhTTpU/IzUltXyZ6eEICVAKe2H5 OsJXKdMMnkmXFEaVTOnY9gh/CLITNPYbtx37purvHuoCdr4MuZkRvcPy3nui9VenJYqb JZcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=Nj1ZVs7TBaHuog/pOF/A+DSXwgQYexSiGYspF1L8voc=; b=mBArcYZWhGKWH4PpeI7Hd+oVwqvUmSKaMs5MhK5a+VbcLn++lIcAkLTHgdkyzKgE3I qOoKrtBTkGOtGUoCiPeTVHeYDokVXDgh+OZUJjDpj6WK+ABM/Krd+6hvhazU6bkjcT1l RTejGxQKMf9ma5/Xi/czaSL0JsuthkTWd3UdSYY39WNBB/QtmHLcBJLtRXGsh6P1WEv/ PttkQwxNppUJr3WQ4DyT/+BIj/MI3l1kEdy4qmE3ajc5T6UNN7OP621QZvN+MqRM+Sph zNiY9vFUz1HT+nJgP0I8usqcG/NtSJGhtlnb0cf+TE48YXfPOg5Ju9+czMPFO2sWkyrE ElLQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@cisco.com header.s=iport header.b="O1Cmtn/e"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=cisco.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id h1si945601edt.106.2020.09.17.18.29.38; Thu, 17 Sep 2020 18:30:02 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@cisco.com header.s=iport header.b="O1Cmtn/e"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=cisco.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726139AbgIRB0f (ORCPT + 99 others); Thu, 17 Sep 2020 21:26:35 -0400 Received: from rcdn-iport-6.cisco.com ([173.37.86.77]:15803 "EHLO rcdn-iport-6.cisco.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725987AbgIRB0f (ORCPT ); Thu, 17 Sep 2020 21:26:35 -0400 X-Greylist: delayed 426 seconds by postgrey-1.27 at vger.kernel.org; Thu, 17 Sep 2020 21:26:34 EDT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2613; q=dns/txt; s=iport; t=1600392394; x=1601601994; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=Lvffglabqyr6MEzUPhmZ2cUEKdNq8MacTKeEEyavhZU=; b=O1Cmtn/eePjyhiD//EUYEoeQop16DTwk862eiGv12vHydViQJeW6cnt4 ae/5gMeN4mysngkSleL9rV/iYZdbUrvU3NWMisg2ZdWS7z8VY8s0ZwNQj Qsf25J6LLt+ao/Am1uHX0z0/zvSIo6nHWJ/OWUfg+AHnOodvlDhg4aDIY o=; X-IronPort-AV: E=Sophos;i="5.77,273,1596499200"; d="scan'208";a="828776718" Received: from alln-core-2.cisco.com ([173.36.13.135]) by rcdn-iport-6.cisco.com with ESMTP/TLS/DHE-RSA-SEED-SHA; 18 Sep 2020 01:19:27 +0000 Received: from sjc-ads-9087.cisco.com (sjc-ads-9087.cisco.com [10.30.208.97]) by alln-core-2.cisco.com (8.15.2/8.15.2) with ESMTP id 08I1JRJl021371; Fri, 18 Sep 2020 01:19:27 GMT Received: by sjc-ads-9087.cisco.com (Postfix, from userid 396877) id 99B7FC99; Thu, 17 Sep 2020 18:19:27 -0700 (PDT) From: Julius Hemanth Pitti To: akpm@linux-foundation.org, xlpang@linux.alibaba.com, mhocko@suse.com, vdavydov.dev@gmail.com, ktkhai@virtuozzo.com, hannes@cmpxchg.org Cc: stable@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, xe-linux-external@cisco.com, Linus Torvalds , Julius Hemanth Pitti Subject: [PATCH stable v5.8] mm: memcg: fix memcg reclaim soft lockup Date: Thu, 17 Sep 2020 18:19:13 -0700 Message-Id: <20200918011913.57159-1-jpitti@cisco.com> X-Mailer: git-send-email 2.19.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Auto-Response-Suppress: DR, OOF, AutoReply X-Outbound-SMTP-Client: 10.30.208.97, sjc-ads-9087.cisco.com X-Outbound-Node: alln-core-2.cisco.com Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Xunlei Pang commit e3336cab2579012b1e72b5265adf98e2d6e244ad upstream. We've met softlockup with "CONFIG_PREEMPT_NONE=y", when the target memcg doesn't have any reclaimable memory. It can be easily reproduced as below: watchdog: BUG: soft lockup - CPU#0 stuck for 111s![memcg_test:2204] CPU: 0 PID: 2204 Comm: memcg_test Not tainted 5.9.0-rc2+ #12 Call Trace: shrink_lruvec+0x49f/0x640 shrink_node+0x2a6/0x6f0 do_try_to_free_pages+0xe9/0x3e0 try_to_free_mem_cgroup_pages+0xef/0x1f0 try_charge+0x2c1/0x750 mem_cgroup_charge+0xd7/0x240 __add_to_page_cache_locked+0x2fd/0x370 add_to_page_cache_lru+0x4a/0xc0 pagecache_get_page+0x10b/0x2f0 filemap_fault+0x661/0xad0 ext4_filemap_fault+0x2c/0x40 __do_fault+0x4d/0xf9 handle_mm_fault+0x1080/0x1790 It only happens on our 1-vcpu instances, because there's no chance for oom reaper to run to reclaim the to-be-killed process. Add a cond_resched() at the upper shrink_node_memcgs() to solve this issue, this will mean that we will get a scheduling point for each memcg in the reclaimed hierarchy without any dependency on the reclaimable memory in that memcg thus making it more predictable. Suggested-by: Michal Hocko Signed-off-by: Xunlei Pang Signed-off-by: Andrew Morton Acked-by: Chris Down Acked-by: Michal Hocko Acked-by: Johannes Weiner Link: http://lkml.kernel.org/r/1598495549-67324-1-git-send-email-xlpang@linux.alibaba.com Signed-off-by: Linus Torvalds Fixes: b0dedc49a2da ("mm/vmscan.c: iterate only over charged shrinkers during memcg shrink_slab()") Cc: stable@vger.kernel.org Signed-off-by: Julius Hemanth Pitti --- mm/vmscan.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/mm/vmscan.c b/mm/vmscan.c index 749d239c62b2..8b97bc615d8c 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -2619,6 +2619,14 @@ static void shrink_node_memcgs(pg_data_t *pgdat, struct scan_control *sc) unsigned long reclaimed; unsigned long scanned; + /* + * This loop can become CPU-bound when target memcgs + * aren't eligible for reclaim - either because they + * don't have any reclaimable pages, or because their + * memory is explicitly protected. Avoid soft lockups. + */ + cond_resched(); + switch (mem_cgroup_protected(target_memcg, memcg)) { case MEMCG_PROT_MIN: /* -- 2.17.1