Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [RESEND][PATCH] cgroup: fix memory leak caused by missing
 cgroup_bpf_offline
To:     Daniel Borkmann <daniel@iogearbox.net>, Tejun Heo <tj@kernel.org>,
        Zefan Li <lizefan.x@bytedance.com>,
        Johannes Weiner <hannes@cmpxchg.org>,
        Alexei Starovoitov <ast@kernel.org>,
        Andrii Nakryiko <andrii@kernel.org>,
        Ming Lei <ming.lei@redhat.com>, Jens Axboe <axboe@kernel.dk>
Cc:     cgroups@vger.kernel.org, linux-kernel@vger.kernel.org,
        netdev@vger.kernel.org, bpf@vger.kernel.org,
        Roman Gushchin <guro@fb.com>
References: <20211008004600.1717681-1-quanyang.wang@windriver.com>
 <c4d12954-aa78-625e-3be7-06d4fc906bf7@iogearbox.net>
From:   Quanyang Wang <quanyang.wang@windriver.com>
Message-ID: <6c15c2d5-39c2-5c86-b9ab-d64089e8d654@windriver.com>
Date:   Tue, 12 Oct 2021 13:44:03 +0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <c4d12954-aa78-625e-3be7-06d4fc906bf7@iogearbox.net>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?aW56SzlSTk5lZGZFeUpYeUxTMGZmUDdyeUNKdG1rRjNlS09OTGRkQ0pjN3Q3?=
 =?utf-8?B?d3VQRE9FdTMxbDVmOXZ2a29IZEp6Y0I2SnQrVUFkZEhxY05VbFhIK3RpbkJP?=
 =?utf-8?B?Sndjc1dtNFJNeTN0ZVdMd3FDTXdyY1RqcnBBM1gvTDFTcDEzajk4eFI4QTBK?=
 =?utf-8?B?QkY2R0xVSm9MNWh3VitvWld5RXJSWUpZbHZMQU5BOFV4V0lrVCs0TkxYWEZU?=
 =?utf-8?B?Q3BhazJvcEZlb3c4WndKUzBUQ3c0dHNwQ3ZEN2tNTm1mZXluSEZ3M2x1VGtG?=
 =?utf-8?B?aVdqUGZXbTJ2QlI3VWRJc1poblBaYmczSXBkSVNsbGpDWHo1UHlXRmFEZGlR?=
 =?utf-8?B?emZ6SWdEMUUyaDlONXloMjkzVVFoZ1MrN1FyV2hoVk1NQzdrTmo4eDBPV3Vz?=
 =?utf-8?B?MmFuVUt3cnB0dHF3SHR1QnBtOXNxZGJsOVBmSUFKNXAxUmhjVmpUSlRZekZD?=
 =?utf-8?B?MGlPRy90Q1hQdlJrSGJGYTVOdlQ5Tkw1TzJCQ1ppWHFHSkN2NTR5OVRDRUtK?=
 =?utf-8?B?Q0dtRTQ2akZrY3lHSDMxdnBQUmZ0K0lYeWVaQUZ6Y0NkdFZTc0VaVjAyeHdG?=
 =?utf-8?B?UlZjVHRGSTI5ckNQODNmanNrMnExMU5iR3lWTU9YTGN5dk5zanBBWWhWaG8r?=
 =?utf-8?B?d21lbmRIaWQwQlF0SzhxNlhIY00xdVVmbGJ3KzdaMGMvVU9GK0NZejAzbkk4?=
 =?utf-8?B?V1gydS95SW9Ob2tUQU13dnZNRDlYN0ZKRDFKSGtHMnhFSjllbFFjNDlqU3do?=
 =?utf-8?B?OXEzUGlpSitaa0duOExxR0tPdDV5dS9KcWt1SVZoN3NNN1hmNllEd0lMc1Ey?=
 =?utf-8?B?OWZqNHV2blN2czVUUVJNRnN3aVZiZmtYMGNwL0QrTi9vM0E4Tm9sRlBDVTFY?=
 =?utf-8?B?bzR3WCtRM1ExaGM5WDZ6QTFncmREQWFhVlBLSWZRbXVwelh3L0poOGFIZjZ1?=
 =?utf-8?B?ZVRVbVVORFl3WmpJbFEvN29IUDVwVEJXaVZQZmNIaG04MmpJR3dpOVpTRHZO?=
 =?utf-8?B?d2lWa0JmWSthcDRiRVVpc2lMTitwTVphakFvRUFyVC84NWRyQmR2bzQ1QWZP?=
 =?utf-8?B?dWh6d01SYnc2ZUFTNTR4ODFVRy9rMTU0K0V4Q0M2VEtPenlWalJ5VzZCQnhG?=
 =?utf-8?B?OFBWMTd4bWw3QjUvcUVsNzltdVlwNnRDOUZVZGVuMFVUMFVHakxibnBqbkYy?=
 =?utf-8?B?NlBFTi9iajlhY0plQnVHV25jKy9yTlVoK1RPQU1TRDB4TDhIVEMyemo0a3NS?=
 =?utf-8?B?TVFtOTQyZ3hPZS9BckM3YmVOcmxpYzg4WENHL1I3MXZjR1ExK1dlck1zUVI4?=
 =?utf-8?B?MmcxZlhybGV3L2E3TnVaKzV0YXRQVm5mZmtqdEVLQk5vd3RiKzE2Vjc4Nk1x?=
 =?utf-8?B?V3o2a1ZlT2xJdklMTVhWMUFXQWNwanpxVkxnSE9iWitWOUNsV3VBalQ3UEZy?=
 =?utf-8?B?VTFNcWszQlh4SGluVGkzOXQzcThYaDVFZ1NGdzJndTJmVnRJYVJNUlQ0ZEpI?=
 =?utf-8?B?SXBuWGVIY1IrOXJRbm5nakhTbmg3RFFmV3hPQnpBZmMvZnBzWGhhZkZZSkV3?=
 =?utf-8?B?c3prMnp6NWNLNElSZUZxemYrZFE5YUp3NVdzZGhnc0NoeXc0TERxb0N2ZndX?=
 =?utf-8?B?cWRGM2lYa3JQdHFQZElpY24zRjJyMmlQVkFxZnFqUUowai9Dd0FJSzIzNy9s?=
 =?utf-8?B?OWdVU2JTV0pvVlVqMTdoeFFpNEpCS3N4YkFLb3RKT2tzcWk0NFV4MjhaYnpk?=
 =?utf-8?Q?qCRlO45Di+upv2qKAk2eU8I1DH9kApZ7K2jHAZG?=
X-OriginatorOrg: windriver.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b2f9af19-586f-4300-a300-08d98d43573b
X-MS-Exchange-CrossTenant-AuthSource: PH7PR11MB5819.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Oct 2021 05:44:21.7279
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 8ddb2873-a1ad-4a18-ae4e-4644631433be
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 1osrOyrj+au2q6W1lReTWXgAFSid6/7XAVmUPXyWTEFKbPgQC/NP6YNywL9YgYFE5xIUpLkK/nWuyelmI5onBQ7DAZBe5uHjYLnkckkhyMY=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB5128
X-Proofpoint-ORIG-GUID: 70GiwvFX0lmEfPHxC3rsoClhdGoOZLbt
X-Proofpoint-GUID: 70GiwvFX0lmEfPHxC3rsoClhdGoOZLbt
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475
 definitions=2021-10-12_01,2021-10-11_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 spamscore=0
 clxscore=1011 priorityscore=1501 suspectscore=0 lowpriorityscore=0
 bulkscore=0 malwarescore=0 adultscore=0 phishscore=0 impostorscore=0
 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2110120031
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Daniel,
Thank you for your review.

On 10/11/21 9:44 PM, Daniel Borkmann wrote:
> [ +Roman ]
> 
> On 10/8/21 2:46 AM, quanyang.wang@windriver.com wrote:
>> From: Quanyang Wang <quanyang.wang@windriver.com>
>>
>> When enabling CONFIG_CGROUP_BPF, kmemleak can be observed by running
>> the command as below:
>>
>>      $mount -t cgroup -o none,name=foo cgroup cgroup/
>>      $umount cgroup/
>>
>> unreferenced object 0xc3585c40 (size 64):
>>    comm "mount", pid 425, jiffies 4294959825 (age 31.990s)
>>    hex dump (first 32 bytes):
>>      01 00 00 80 84 8c 28 c0 00 00 00 00 00 00 00 00  ......(.........
>>      00 00 00 00 00 00 00 00 6c 43 a0 c3 00 00 00 00  ........lC......
>>    backtrace:
>>      [<e95a2f9e>] cgroup_bpf_inherit+0x44/0x24c
>>      [<1f03679c>] cgroup_setup_root+0x174/0x37c
>>      [<ed4b0ac5>] cgroup1_get_tree+0x2c0/0x4a0
>>      [<f85b12fd>] vfs_get_tree+0x24/0x108
>>      [<f55aec5c>] path_mount+0x384/0x988
>>      [<e2d5e9cd>] do_mount+0x64/0x9c
>>      [<208c9cfe>] sys_mount+0xfc/0x1f4
>>      [<06dd06e0>] ret_fast_syscall+0x0/0x48
>>      [<a8308cb3>] 0xbeb4daa8
>>
>> This is because that root_cgrp->bpf.refcnt.data is allocated by the
>> function percpu_ref_init in cgroup_bpf_inherit which is called by
>> cgroup_setup_root when mounting, but not freed along with root_cgrp
>> when umounting. Adding cgroup_bpf_offline which calls percpu_ref_kill
>> to cgroup_kill_sb can free root_cgrp->bpf.refcnt.data in umount path.
>>
>> Fixes: 2b0d3d3e4fcfb ("percpu_ref: reduce memory footprint of 
>> percpu_ref in fast path")
>> Signed-off-by: Quanyang Wang <quanyang.wang@windriver.com>
>> ---
>> One of the recipients' email is wrong, so I resend this patch.
>> Sorry for any confusion caused.
>> ---
>>   kernel/cgroup/cgroup.c | 4 +++-
>>   1 file changed, 3 insertions(+), 1 deletion(-)
>>
>> diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
>> index 570b0c97392a9..183736ad72f2b 100644
>> --- a/kernel/cgroup/cgroup.c
>> +++ b/kernel/cgroup/cgroup.c
>> @@ -2187,8 +2187,10 @@ static void cgroup_kill_sb(struct super_block *sb)
>>        * And don't kill the default root.
>>        */
>>       if (list_empty(&root->cgrp.self.children) && root != 
>> &cgrp_dfl_root &&
>> -        !percpu_ref_is_dying(&root->cgrp.self.refcnt))
>> +            !percpu_ref_is_dying(&root->cgrp.self.refcnt)) {
>> +        cgroup_bpf_offline(&root->cgrp);
>>           percpu_ref_kill(&root->cgrp.self.refcnt);
>> +    }
>>       cgroup_put(&root->cgrp);
> 
> Doesn't cgroup_bpf_offline() internally bump the root cgroup's refcount 
> via cgroup_get()?
> How does this relate to the single cgroup_put() in the above line?
There is cgroup_get/put pair inside cgroup_bpf_offline.
cgroup_get() is at the beginning of cgroup_bpf_offline.
cgroup_put is called at the routine:
cgroup_bpf_offline
->percpu_ref_kill
-->cgroup_bpf_release_fn
--->cgroup_bpf_release (there is a cgroup_put() at the end of it).
So cgroup_bpf_offline can keep the balance of cgroup's refcount.
  Would
> have been nice to
> see the commit msg elaborate more on this.
 >
OK, I will add description about this in the commit msg.

Thanks,
Quanyang
> 
>>       kernfs_kill_sb(sb);
>>   }
>>
>