Received: by 2002:ad5:474a:0:0:0:0:0 with SMTP id i10csp655488imu; Fri, 25 Jan 2019 08:38:15 -0800 (PST) X-Google-Smtp-Source: ALg8bN4bwWPN9cXgIxs8oR6jNRQXlX9pP0VLodItqXeni9GnqJpbhlqfZJAx3YzFhdsKiS1tggHo X-Received: by 2002:a63:7e1a:: with SMTP id z26mr10331789pgc.216.1548434295661; Fri, 25 Jan 2019 08:38:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548434295; cv=none; d=google.com; s=arc-20160816; b=S+IRyd0R6vLqiFIs8tlvKgjw0XzBKeD+kq+GRxaL2URA0mLJQ8ZtPwIpWAUddcIlTo FlY+kw6+XJwr0GtxMn8dgjRh7vATQ4vc97LabfF4YXTKkIsIRrSxdsiXBIdOXiSHhjlx Z7IrPV033jAsjqYG96bIqzO3hf78RJSliw5k1h+vWHDasCUAeUvHLOWlUWa0DFEvKQXA bf5OTeyQs1PIRZDqFTq5OyWbnF5xb5obM18kKB0CC7/8MfepvSHSrdejThT6aB7RofwI /3xAcM4fILSvSe39y4FbZc/vKj6/GktYtNu19zyX1DPb/fw0zUvyyEOa+pKY2m2s+T6T BGTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:user-agent:in-reply-to :content-transfer-encoding:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:dkim-signature; bh=RYL7pbha6AtxQjHD3ZK0n6qiStyqrgkMrnAlVCP0NNE=; b=U+gH/lz69EOfmuPCm/oqt2O4aG9LFSLni/53GvTuOosCjg+xLe7ZdvIFayRPtJ7te3 tYSX/NhLjNu00ln5hAE1yhHTiMCvRgL6j68cHtb7HiQABnZTTOHSjOYM2XYC+g0MyZ6o yk3pW7aYvSw3sX4OygbH05H8NoIcg29caolbEubmR3fFdwu50d5oiID3or9OnSjJqzrI YPE7Ivme5jtD8GErsihb26h+0Gb1KVf0UoO3Ha6Es/DCX/z2km+asW+RU5cTghOaVjrY G0nRvd5zkMvhi8C5yYu6H9TCWD1hXGwJhehzJSdN499B/sdbBr3EFc6qnSwFUB49g7YE Ai2w== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@gmail.com header.s=20161025 header.b=hZvlbsqy; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id q8si3730367pli.284.2019.01.25.08.38.00; Fri, 25 Jan 2019 08:38:15 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=fail header.i=@gmail.com header.s=20161025 header.b=hZvlbsqy; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728871AbfAYQhf (ORCPT + 99 others); Fri, 25 Jan 2019 11:37:35 -0500 Received: from mail-yw1-f65.google.com ([209.85.161.65]:36976 "EHLO mail-yw1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726108AbfAYQhf (ORCPT ); Fri, 25 Jan 2019 11:37:35 -0500 Received: by mail-yw1-f65.google.com with SMTP id h193so4139037ywc.4; Fri, 25 Jan 2019 08:37:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=RYL7pbha6AtxQjHD3ZK0n6qiStyqrgkMrnAlVCP0NNE=; b=hZvlbsqyaKWNVGRewUWJIxOTzlUYhWS2zbX7SH2tizlgJpQpx60Q9lFUr1KAvxcC/o 1hPJlxzXj2+3gfQcXN2m7hvMawtI4t9aj1dEem45TLpkEql/eZmnG/q1Rj1mzVyhtxwr HV7sN0Yf6DRbhWtZYwjdL3tBn7xcNcxE0m4fXAVJgEjZPCK1VAIW74UMrRR1vLSuVj0h HyosUDdb9lBIVd02NWUS0S1pWyvOPCIyUMbdo++pIUAlLLlAg3hl9wVbLnyV7iSiBQWQ Cmfs2mFJwxet2D8u3OAyjG/l6Soc19pDLR4LD+aJZAPHD4x6jylVAxQ/YGkiG692WER7 s73w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=RYL7pbha6AtxQjHD3ZK0n6qiStyqrgkMrnAlVCP0NNE=; b=grp19tW/nbo3bDcd8fRB8zEyFb5W9KxS0YOnVn/s1ox0ZLWlxXTHQcw8jmq8bwLSvA Ob54BhFhYKIaXGJSN7tzP0tiuaWt6/M+40IdoOwZmJa3JdlMHyxPfwM/19a8o8DLs1ZE dS/QUueHudkFcoRMGmJCtod0Z4p58lfps5asELBoNS2ycc/O9Vp49PIWHN6+pN1uhqIA 8tll35zCR+um61zbYM+9Xl5Xz3FsWa7A3jAGqNj5T9bu3D8VD8BoEj/zNoDqPXYD/Nit PVvPVPKwHGVFsosidfege9i1VTsnPIJ2rxWhHF4yEv7AYH5iFctskFOEFbzHQuEnJ8AS ZLYA== X-Gm-Message-State: AJcUukfo81jlbaPyyQ29tUJD48XxAyJWWcSsSd/RJRPlUmPjMYRk/KpZ prxn/9zFmmv258PupRuu9k4= X-Received: by 2002:a81:d845:: with SMTP id n5mr11854141ywl.486.1548434253786; Fri, 25 Jan 2019 08:37:33 -0800 (PST) Received: from localhost ([2620:10d:c091:200::7:a62a]) by smtp.gmail.com with ESMTPSA id b129sm10282016ywc.106.2019.01.25.08.37.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 25 Jan 2019 08:37:32 -0800 (PST) Date: Fri, 25 Jan 2019 08:37:31 -0800 From: Tejun Heo To: Arkadiusz =?utf-8?Q?Mi=C5=9Bkiewicz?= Cc: cgroups@vger.kernel.org, Aleksa Sarai , Jay Kamat , Roman Gushchin , Michal Hocko , Johannes Weiner , linux-kernel@vger.kernel.org Subject: Re: pids.current with invalid value for hours [5.0.0 rc3 git] Message-ID: <20190125163731.GJ50184@devbig004.ftw2.facebook.com> References: <20190117122535.njcbqhlmzozdkncw@mikami> <1d36b181-cbaf-6694-1a31-2f7f55d15675@gmail.com> <96ef6615-a5df-30af-b4dc-417a18ca63f1@gmail.com> <1cdbef13-564d-61a6-95f4-579d2cad243d@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <1cdbef13-564d-61a6-95f4-579d2cad243d@gmail.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Jan 25, 2019 at 08:52:11AM +0100, Arkadiusz Miśkiewicz wrote: > On 24/01/2019 12:21, Arkadiusz Miśkiewicz wrote: > > On 17/01/2019 14:17, Arkadiusz Miśkiewicz wrote: > >> On 17/01/2019 13:25, Aleksa Sarai wrote: > >>> On 2019-01-17, Arkadiusz Miśkiewicz wrote: > >>>> Using kernel 4.19.13. > >>>> > >>>> For one cgroup I noticed weird behaviour: > >>>> > >>>> # cat pids.current > >>>> 60 > >>>> # cat cgroup.procs > >>>> # > >>> > >>> Are there any zombies in the cgroup? pids.current is linked up directly > >>> to __put_task_struct (so exit(2) won't decrease it, only the task_struct > >>> actually being freed will decrease it). > >>> > >> > >> There are no zombie processes. > >> > >> In mean time the problem shows on multiple servers and so far saw it > >> only in cgroups that were OOMed. > >> > >> What has changed on these servers (yesterday) is turning on > >> memory.oom.group=1 for all cgroups and changing memory.high from 1G to > >> "max" (leaving memory.max=2G limit only). > >> > >> Previously there was no such problem. > >> > > > > I'm attaching reproducer. This time tried on different distribution > > kernel (arch linux). > > > > After 60s pids.current still shows 37 processes even if there are no > > processes running (according to ps aux). > > > The same test on 5.0.0-rc3-00104-gc04e2a780caf and it's easy to > reproduce bug. No processes in cgroup but pids.current reports 91. Can you please see whether the problem can be reproduced on the current linux-next? git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git Thanks. -- tejun