Received: by 2002:ac0:a5a7:0:0:0:0:0 with SMTP id m36-v6csp6666124imm; Tue, 24 Jul 2018 00:29:52 -0700 (PDT) X-Google-Smtp-Source: AAOMgpdYSmkTYL2vcrhJctEqVNfcKVbO2S/wRfYgvXs2wJIEORTuO/YoAFQPZVBqGCZ+WtYi3Jkb X-Received: by 2002:a63:1f20:: with SMTP id f32-v6mr14662813pgf.84.1532417392262; Tue, 24 Jul 2018 00:29:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1532417392; cv=none; d=google.com; s=arc-20160816; b=EdPFjXKKzu7b59Tr2x7chsaC9vUga91d6miVWmyR0ZqNRBDXjBH7z9doqEE88Q3fhX 8GFAWtGBoc14TQSJFs5T55Y8g5/BSqlOYZO5PEDqD+g1V9+z5A4pgUPhPuAPxDw6d6gS +9W8CnpiCELQw1qinzV0UnrvZy+5hbAJbsMqpfLJ8dtV83NzAy15ufVuDZIkHAAtG0g2 1swoxFhqYcR1wUIrCK2t3koL76HMpQFpRruXsrUGb4ZWE2alt5ukj1cUCtzv9qy4zo18 JgtCejQ+XCWAooQ0qulHPUbPpFXAvDyvMvbOjXiXXCTIpMahsPrBslyFHPGjp8kldpkg RRQg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:user-agent:in-reply-to :content-disposition:mime-version:references:message-id:subject:cc :to:from:date:arc-authentication-results; bh=YXIturRHLzLSQPWnSulr6GFUbvx/XbnML/MCzViG59Q=; b=C+RZMHFHdQcMsZol4jW+Y0tUuoRAb5qW/2G+NBz2axRrZwW5pJpLqGTGTaq5PhNJ4S G6oSOzsDwfgO9GGtKOe+RKvWMUI5h2uIXlG2mo/sVbOQHI9IDkdPkkCuCvCX9b3pmlU3 FqoSe+bsMs+pIAaprZnubLeXTAjiFDOLJZ9TCEghY+4nR7mi9CsYja9QF5a8igjW5dEX le2Zlt5ZoKpUMwPNojr3d1JvJM1ib6kN3qCiwhYtHlelSyU5q7GLaWTtDHkmF7vUQ5Y3 +eLzHuKJMLxhHX2aLfNjACKVOu3ra/xJ8VLtZgD953TQkEKzY4nkvNFhmplvIaMFJ/WX AGKw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id y9-v6si9317127pgv.290.2018.07.24.00.29.37; Tue, 24 Jul 2018 00:29:52 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388504AbeGXIdl (ORCPT + 99 others); Tue, 24 Jul 2018 04:33:41 -0400 Received: from mx2.suse.de ([195.135.220.15]:55162 "EHLO mx1.suse.de" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2388390AbeGXIdk (ORCPT ); Tue, 24 Jul 2018 04:33:40 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 64D23AE1E; Tue, 24 Jul 2018 07:28:34 +0000 (UTC) Date: Tue, 24 Jul 2018 09:28:33 +0200 From: Michal Hocko To: Shakeel Butt Cc: jing.xia.mail@gmail.com, Johannes Weiner , Vladimir Davydov , chunyan.zhang@unisoc.com, Cgroups , Linux MM , LKML , Andrew Morton Subject: Re: [PATCH] mm: memcg: fix use after free in mem_cgroup_iter() Message-ID: <20180724072833.GC28386@dhcp22.suse.cz> References: <1531994807-25639-1-git-send-email-jing.xia@unisoc.com> <20180719104345.GV7193@dhcp22.suse.cz> <20180723064441.GA17905@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.0 (2018-05-17) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon 23-07-18 09:17:28, Shakeel Butt wrote: > On Sun, Jul 22, 2018 at 11:44 PM Michal Hocko wrote: > > > > On Thu 19-07-18 09:23:10, Shakeel Butt wrote: > > > On Thu, Jul 19, 2018 at 3:43 AM Michal Hocko wrote: > > > > > > > > [CC Andrew] > > > > > > > > On Thu 19-07-18 18:06:47, Jing Xia wrote: > > > > > It was reported that a kernel crash happened in mem_cgroup_iter(), > > > > > which can be triggered if the legacy cgroup-v1 non-hierarchical > > > > > mode is used. > > > > > > > > > > Unable to handle kernel paging request at virtual address 6b6b6b6b6b6b8f > > > > > ...... > > > > > Call trace: > > > > > mem_cgroup_iter+0x2e0/0x6d4 > > > > > shrink_zone+0x8c/0x324 > > > > > balance_pgdat+0x450/0x640 > > > > > kswapd+0x130/0x4b8 > > > > > kthread+0xe8/0xfc > > > > > ret_from_fork+0x10/0x20 > > > > > > > > > > mem_cgroup_iter(): > > > > > ...... > > > > > if (css_tryget(css)) <-- crash here > > > > > break; > > > > > ...... > > > > > > > > > > The crashing reason is that mem_cgroup_iter() uses the memcg object > > > > > whose pointer is stored in iter->position, which has been freed before > > > > > and filled with POISON_FREE(0x6b). > > > > > > > > > > And the root cause of the use-after-free issue is that > > > > > invalidate_reclaim_iterators() fails to reset the value of > > > > > iter->position to NULL when the css of the memcg is released in non- > > > > > hierarchical mode. > > > > > > > > Well, spotted! > > > > > > > > I suspect > > > > Fixes: 6df38689e0e9 ("mm: memcontrol: fix possible memcg leak due to interrupted reclaim") > > > > > > > > but maybe it goes further into past. I also suggest > > > > Cc: stable > > > > > > > > even though the non-hierarchical mode is strongly discouraged. > > > > > > Why not set root_mem_cgroup's use_hierarchy to true by default on > > > init? If someone wants non-hierarchical mode, they can explicitly set > > > it to false. > > > > We do not change defaults under users feet usually. > > Then how non-hierarchical mode is being discouraged currently? I don't > see any comments in the docs. css_create warns about non-hierarchical hierarchies. We've been running with a similar warning in (even older) SLES kernels for years now and quite some tools have been updated because they simply didn't know they are doing something wrong. -- Michal Hocko SUSE Labs