When umounting a cgroup controller, in case the controller has no children,
the initial ref will be dropped in cgroup_kill_sb. In cgroup_rmdir path,
the controller is deleted from the parent's children list in
css_release_work_fn, which is run on a kernel worker.
With this simple script
#!/bin/sh
mount -t cgroup -o none,name=test test ./tmp
mkdir -p ./tmp/abc
rmdir ./tmp/abc
umount ./tmp
sleep 5
cat /proc/self/cgroup
The rmdir will remove the last child and umount is expected to kill the
parent controller. However, when running the above script, we may get
1:name=test:/
This shows that the parent controller has not been killed. The reason is
after rmdir is completed, it is not guaranteed that the parent's children
list is empty as css_release_work_fn is deferred to run on a worker. In
case cgroup_kill_sb is run before that work, it does not drop the initial
ref. Later in the worker, it just removes the child from the list without
checking the list is empty to kill the parent controller. As a result, the
parent controller still has the initial ref but without any logical refs
(children ref, mount ref).
This commit adds a free parent controller path into the worker function to
free up the parent controller when the last child is killed.
Signed-off-by: Bui Quang Minh <[email protected]>
---
kernel/cgroup/cgroup.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index a557eea7166f..220eb1742961 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -5157,12 +5157,25 @@ static void css_release_work_fn(struct work_struct *work)
container_of(work, struct cgroup_subsys_state, destroy_work);
struct cgroup_subsys *ss = css->ss;
struct cgroup *cgrp = css->cgroup;
+ struct cgroup *parent = cgroup_parent(cgrp);
mutex_lock(&cgroup_mutex);
css->flags |= CSS_RELEASED;
list_del_rcu(&css->sibling);
+ /*
+ * If parent doesn't have any children, start killing it.
+ * And don't kill the default root.
+ */
+ if (parent && list_empty(&parent->self.children) &&
+ parent != &cgrp_dfl_root.cgrp &&
+ !percpu_ref_is_dying(&parent->self.refcnt)) {
+ if (!percpu_ref_is_dying(&cgrp->bpf.refcnt))
+ cgroup_bpf_offline(parent);
+ percpu_ref_kill(&parent->self.refcnt);
+ }
+
if (ss) {
/* css release path */
if (!list_empty(&css->rstat_css_node)) {
--
2.25.1
Greeting,
FYI, we noticed the following commit (built with gcc-11):
commit: 2bc22feae8a913c7f371bc79ef9967122d8d326c ("[PATCH] cgroup: Kill the parent controller when its last child is killed")
url: https://github.com/intel-lab-lkp/linux/commits/Bui-Quang-Minh/cgroup-Kill-the-parent-controller-when-its-last-child-is-killed/20220403-215911
base: https://git.kernel.org/cgit/linux/kernel/git/tj/cgroup.git for-next
patch link: https://lore.kernel.org/lkml/[email protected]
in testcase: boot
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 16G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <[email protected]>
[ 6.033070][ T1] WARNING: CPU: 0 PID: 1 at lib/percpu-refcount.c:388 percpu_ref_kill_and_confirm (??:?)
[ 6.040408][ T1] Modules linked in:
[ 6.041099][ T1] CPU: 0 PID: 1 Comm: systemd Not tainted 5.16.0-06542-g2bc22feae8a9 #1
[ 6.042403][ T1] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 6.043846][ T1] RIP: 0010:percpu_ref_kill_and_confirm (??:?)
[ 6.044927][ T1] Code: 3d 69 00 70 01 00 75 24 48 8b 45 08 48 c7 c6 a0 c7 09 82 48 c7 c7 26 a5 32 82 c6 05 4e 00 70 01 01 48 8b 50 08 e8 24 18 61 00 <0f> 0b 48 83 4d 00 02 4c 89 ee 48 89 ef e8 f0 fc ff ff 48 89 ef e8
All code
========
0: 3d 69 00 70 01 cmp $0x1700069,%eax
5: 00 75 24 add %dh,0x24(%rbp)
8: 48 8b 45 08 mov 0x8(%rbp),%rax
c: 48 c7 c6 a0 c7 09 82 mov $0xffffffff8209c7a0,%rsi
13: 48 c7 c7 26 a5 32 82 mov $0xffffffff8232a526,%rdi
1a: c6 05 4e 00 70 01 01 movb $0x1,0x170004e(%rip) # 0x170006f
21: 48 8b 50 08 mov 0x8(%rax),%rdx
25: e8 24 18 61 00 callq 0x61184e
2a:* 0f 0b ud2 <-- trapping instruction
2c: 48 83 4d 00 02 orq $0x2,0x0(%rbp)
31: 4c 89 ee mov %r13,%rsi
34: 48 89 ef mov %rbp,%rdi
37: e8 f0 fc ff ff callq 0xfffffffffffffd2c
3c: 48 89 ef mov %rbp,%rdi
3f: e8 .byte 0xe8
Code starting with the faulting instruction
===========================================
0: 0f 0b ud2
2: 48 83 4d 00 02 orq $0x2,0x0(%rbp)
7: 4c 89 ee mov %r13,%rsi
a: 48 89 ef mov %rbp,%rdi
d: e8 f0 fc ff ff callq 0xfffffffffffffd02
12: 48 89 ef mov %rbp,%rdi
15: e8 .byte 0xe8
[ 6.051482][ T1] RSP: 0000:ffffc90000013e28 EFLAGS: 00010082
[ 6.052450][ T1] RAX: 0000000000000000 RBX: ffff88812b68b000 RCX: 0000000000000027
[ 6.053757][ T1] RDX: 0000000000000003 RSI: 0000000000000001 RDI: ffff88842fc1b780
[ 6.055070][ T1] RBP: ffff88812b68b010 R08: 0000000000000000 R09: 0000000000000000
[ 6.056322][ T1] R10: 000000000000000b R11: ffffffff82204f78 R12: 0000000000000246
[ 6.058751][ T1] R13: 0000000000000000 R14: 0000000000000000 R15: ffff88812b1097a0
[ 6.063819][ T1] FS: 0000000000000000(0000) GS:ffff88842fc00000(0063) knlGS:00000000f77e31c0
[ 6.065244][ T1] CS: 0010 DS: 002b ES: 002b CR0: 0000000080050033
[ 6.066251][ T1] CR2: 00000000566f5618 CR3: 000000012b4a4000 CR4: 00000000000406f0
[ 6.067524][ T1] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 6.068975][ T1] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 6.073751][ T1] Call Trace:
[ 6.074375][ T1] <TASK>
[ 6.074915][ T1] cgroup_destroy_locked (cgroup.c:?)
[ 6.075771][ T1] cgroup_rmdir (??:?)
[ 6.076502][ T1] kernfs_iop_rmdir (dir.c:?)
[ 6.077415][ T1] vfs_rmdir (??:?)
[ 6.078170][ T1] do_rmdir (??:?)
[ 6.079048][ T1] __do_sys_rmdir (namei.c:?)
[ 6.079829][ T1] do_int80_syscall_32 (??:?)
[ 6.080698][ T1] entry_INT80_compat (??:?)
[ 6.083561][ T1] RIP: 0023:0xf7f4ba02
[ 6.084304][ T1] Code: 95 01 00 05 25 36 02 00 83 ec 14 8d 80 e8 99 ff ff 50 6a 02 e8 1f ff 00 00 c7 04 24 7f 00 00 00 e8 7e 87 01 00 66 90 90 cd 80 <c3> 8d b6 00 00 00 00 8d bc 27 00 00 00 00 8b 1c 24 c3 8d b6 00 00
All code
========
0: 95 xchg %eax,%ebp
1: 01 00 add %eax,(%rax)
3: 05 25 36 02 00 add $0x23625,%eax
8: 83 ec 14 sub $0x14,%esp
b: 8d 80 e8 99 ff ff lea -0x6618(%rax),%eax
11: 50 push %rax
12: 6a 02 pushq $0x2
14: e8 1f ff 00 00 callq 0xff38
19: c7 04 24 7f 00 00 00 movl $0x7f,(%rsp)
20: e8 7e 87 01 00 callq 0x187a3
25: 66 90 xchg %ax,%ax
27: 90 nop
28: cd 80 int $0x80
2a:* c3 retq <-- trapping instruction
2b: 8d b6 00 00 00 00 lea 0x0(%rsi),%esi
31: 8d bc 27 00 00 00 00 lea 0x0(%rdi,%riz,1),%edi
38: 8b 1c 24 mov (%rsp),%ebx
3b: c3 retq
3c: 8d .byte 0x8d
3d: b6 00 mov $0x0,%dh
...
Code starting with the faulting instruction
===========================================
0: c3 retq
1: 8d b6 00 00 00 00 lea 0x0(%rsi),%esi
7: 8d bc 27 00 00 00 00 lea 0x0(%rdi,%riz,1),%edi
e: 8b 1c 24 mov (%rsp),%ebx
11: c3 retq
12: 8d .byte 0x8d
13: b6 00 mov $0x0,%dh
...
[ 6.087389][ T1] RSP: 002b:00000000ffc47418 EFLAGS: 00000296 ORIG_RAX: 0000000000000028
[ 6.088890][ T1] RAX: ffffffffffffffda RBX: 00000000567287d0 RCX: 00000000f7b0928c
[ 6.090266][ T1] RDX: 00000000f7f46440 RSI: 0000000000000000 RDI: 0000000000000001
[ 6.091612][ T1] RBP: 00000000f77e307c R08: 0000000000000000 R09: 0000000000000000
[ 6.092923][ T1] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[ 6.094354][ T1] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 6.095618][ T1] </TASK>
[ 6.096174][ T1] ---[ end trace bc2fc7fcbaaa4bea ]---
[ 6.157221][ T2070] scsi 1:0:0:0: Attached scsi generic sg0 type 5
[ 6.181679][ T2070] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray
[ 6.182949][ T2070] cdrom: Uniform CD-ROM driver Revision: 3.20
[ OK ] Started Raise network interfaces.
[ OK ] Started udev Coldplug all Devices.
[ 6.215433][ T2070] sr 1:0:0:0: Attached scsi CD-ROM sr0
[ OK ] Reached target System Initialization.
[ OK ] Listening on D-Bus System Message Bus Socket.
[ OK ] Reached target Sockets.
[ OK ] Reached target Basic System.
Starting Login Service...
[ OK ] Started D-Bus System Message Bus.
Starting LSB: Execute the kexec -e command to reboot system...
[ OK ] Started Regular background program processing daemon.
[ OK ] Started Daily apt download activities.
[ OK ] Started Daily apt upgrade and clean activities.
Starting System Logging Service...
[ OK ] Started Daily Cleanup of Temporary Directories.
[ OK ] Reached target Timers.
[ OK ] Reached target Network.
Starting OpenBSD Secure Shell server...
Starting LKP bootstrap...
Starting Permit User Sessions...
[ OK ] Reached target Network is Online.
Starting /etc/rc.local Compatibility...
Starting LSB: Start and stop bmc-watchdog...
[ OK ] Started Login Service.
[ OK ] Started Permit User Sessions.
[ OK ] Started LSB: Execute the kexec -e command to reboot system.
[ 6.234372] rc.local[2437]: PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/lkp/lkp/src/bin
[ OK ] Started /etc/rc.local Compatibility.
[ OK ] Started Getty on tty1.
[ OK ] Reached target Login Prompts.
Starting LSB: Load kernel image with kexec...
[ OK ] Started LSB: Start and stop bmc-watchdog.
[ 6.271554] rc.local[2437]: /lkp/lkp/src/bin/lkp-setup-rootfs: 78: /lkp/lkp/src/bin/lkp-setup-rootfs: cannot create /sys/devices/system/cpu/microcode/reload: Directory nonexistent
[ OK ] Started OpenBSD Secure Shell server.
[ OK ] Started LSB: Load kernel image with kexec.
LKP: ttyS0: 2469: Kernel tests: Boot OK!
LKP: ttyS0: 2469: HOSTNAME vm-snb-49, MAC d6:90:49:e5:94:ed, kernel 5.16.0-06542-g2bc22feae8a9 1
LKP: ttyS0: 2469: /lkp/lkp/src/bin/run-lkp /lkp/jobs/scheduled/vm-snb-49/boot-1-debian-i386-20191205.cgz-2bc22feae8a913c7f371bc79ef9967122d8d326c-20220404-23352-ifucqo-4.yaml
[ 16.268522][ T2492] LKP: stdout: 2469: Kernel tests: Boot OK!
[ 16.268534][ T2492]
[ 20.649263][ T2492] LKP: stdout: 2469: HOSTNAME vm-snb-49, MAC d6:90:49:e5:94:ed, kernel 5.16.0-06542-g2bc22feae8a9 1
[ 20.649275][ T2492]
LKP: ttyS0: 2469: LKP: rebooting forcely
[ 21.111051][ T2492] install debs round one: dpkg -i --force-confdef --force-depends /opt/deb/gawk_1%3a4.1.4+dfsg-1_i386.deb
[ 21.111062][ T2492]
[ 21.115095][ T2492] Selecting previously unselected package gawk.
[ 21.115102][ T2492]
[ 21.119713][ T2492] (Reading database ... 16210 files and directories currently installed.)
[ 21.119721][ T2492]
[ 21.124086][ T2492] Preparing to unpack .../gawk_1%3a4.1.4+dfsg-1_i386.deb ...
[ 21.124094][ T2492]
To reproduce:
# build kernel
cd linux
cp config-5.16.0-06542-g2bc22feae8a9 .config
make HOSTCC=gcc-11 CC=gcc-11 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage modules
make HOSTCC=gcc-11 CC=gcc-11 ARCH=x86_64 INSTALL_MOD_PATH=<mod-install-dir> modules_install
cd <mod-install-dir>
find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email
# if come across any failure that blocks the test,
# please remove ~/.lkp and /lkp dir to run from a clean state.
--
0-DAY CI Kernel Test Service
https://01.org/lkp