Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753998AbaF3Dqg (ORCPT ); Sun, 29 Jun 2014 23:46:36 -0400 Received: from szxga03-in.huawei.com ([119.145.14.66]:61858 "EHLO szxga03-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751283AbaF3Dqe (ORCPT ); Sun, 29 Jun 2014 23:46:34 -0400 Message-ID: <53B0DD8D.5060805@huawei.com> Date: Mon, 30 Jun 2014 11:46:21 +0800 From: Li Zefan User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: Tejun Heo CC: LKML , Cgroups Subject: [BUG] cpuset: lockdep warning Content-Type: text/plain; charset="GB2312" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.177.18.230] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020202.53B0DD94.0037,ss=1,re=0.000,fgs=0, ip=0.0.0.0, so=2013-05-26 15:14:31, dmn=2011-05-27 18:58:46 X-Mirapoint-Loop-Id: 9ed1d278c8ee02c320f75ad9948a4dbf Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Tejun, In this lockdep warning kernfs and workqueue are involved, so I'm not sure what's happening here. This was triggered when tasks were being moved to parent cpuset due to hotplug. The kernel is 3.16-rc1, with no modification. localhost:/ # mount -t cgroup -o cpuset xxx /cpuset localhost:/ # mkdir /cpuset/tmp localhost:/ # echo 1 > /cpuset/tmp/cpuset.cpus localhost:/ # echo 0 > cpuset/tmp/cpuset.mems localhost:/ # echo $$ > /cpuset/tmp/tasks localhost:/ # echo 0 > /sys/devices/system/cpu/cpu1/online [ 1810.292243] ====================================================== [ 1810.292251] [ INFO: possible circular locking dependency detected ] [ 1810.292259] 3.16.0-rc1-0.1-default+ #7 Not tainted [ 1810.292266] ------------------------------------------------------- [ 1810.292273] kworker/1:0/32649 is trying to acquire lock: [ 1810.292280] (cgroup_mutex){+.+.+.}, at: [] cgroup_transfer_tasks+0x37/0x150 [ 1810.292300] [ 1810.292300] but task is already holding lock: [ 1810.292309] (cpuset_hotplug_work){+.+...}, at: [] process_one_work+0x192/0x520 [ 1810.292327] [ 1810.292327] which lock already depends on the new lock. [ 1810.292327] [ 1810.292339] [ 1810.292339] the existing dependency chain (in reverse order) is: [ 1810.292348] [ 1810.292348] -> #2 (cpuset_hotplug_work){+.+...}: [ 1810.292360] [] validate_chain+0x656/0x7c0 [ 1810.292371] [] __lock_acquire+0x382/0x660 [ 1810.292380] [] lock_acquire+0xf9/0x170 [ 1810.292389] [] flush_work+0x39/0x90 [ 1810.292398] [] cpuset_write_resmask+0x51/0x120 [ 1810.292409] [] cgroup_file_write+0x49/0x1f0 [ 1810.292419] [] kernfs_fop_write+0xfd/0x190 [ 1810.292431] [] vfs_write+0xe5/0x190 [ 1810.292443] [] SyS_write+0x5c/0xc0 [ 1810.292452] [] system_call_fastpath+0x16/0x1b [ 1810.292464] [ 1810.292464] -> #1 (s_active#175){++++.+}: [ 1810.292476] [] validate_chain+0x656/0x7c0 [ 1810.292486] [] __lock_acquire+0x382/0x660 [ 1810.292495] [] lock_acquire+0xf9/0x170 [ 1810.292504] [] kernfs_drain+0x13b/0x1c0 [ 1810.292513] [] __kernfs_remove+0xc8/0x220 [ 1810.292523] [] kernfs_remove_by_name_ns+0x50/0xb0 [ 1810.292533] [] cgroup_addrm_files+0x16e/0x290 [ 1810.292543] [] cgroup_clear_dir+0x6d/0xa0 [ 1810.292552] [] rebind_subsystems+0x10f/0x350 [ 1810.292562] [] cgroup_setup_root+0x1bf/0x290 [ 1810.292571] [] cgroup_mount+0x123/0x3d0 [ 1810.292581] [] mount_fs+0x4d/0x1a0 [ 1810.292591] [] vfs_kern_mount+0x79/0x160 [ 1810.292602] [] do_new_mount+0xd9/0x200 [ 1810.292611] [] do_mount+0x1dc/0x220 [ 1810.292621] [] SyS_mount+0xbc/0xe0 [ 1810.292630] [] system_call_fastpath+0x16/0x1b [ 1810.292640] [ 1810.292640] -> #0 (cgroup_mutex){+.+.+.}: [ 1810.292651] [] check_prev_add+0x43e/0x4b0 [ 1810.292660] [] validate_chain+0x656/0x7c0 [ 1810.292669] [] __lock_acquire+0x382/0x660 [ 1810.292678] [] lock_acquire+0xf9/0x170 [ 1810.292687] [] mutex_lock_nested+0x6f/0x380 [ 1810.292697] [] cgroup_transfer_tasks+0x37/0x150 [ 1810.292707] [] hotplug_update_tasks_insane+0x110/0x1d0 [ 1810.292718] [] cpuset_hotplug_update_tasks+0x13d/0x180 [ 1810.292729] [] cpuset_hotplug_workfn+0x18c/0x630 [ 1810.292739] [] process_one_work+0x254/0x520 [ 1810.292748] [] worker_thread+0x13d/0x3d0 [ 1810.292758] [] kthread+0xf8/0x100 [ 1810.292768] [] ret_from_fork+0x7c/0xb0 [ 1810.292778] [ 1810.292778] other info that might help us debug this: [ 1810.292778] [ 1810.292789] Chain exists of: [ 1810.292789] cgroup_mutex --> s_active#175 --> cpuset_hotplug_work [ 1810.292789] [ 1810.292807] Possible unsafe locking scenario: [ 1810.292807] [ 1810.292816] CPU0 CPU1 [ 1810.292822] ---- ---- [ 1810.292827] lock(cpuset_hotplug_work); [ 1810.292835] lock(s_active#175); [ 1810.292845] lock(cpuset_hotplug_work); [ 1810.292855] lock(cgroup_mutex); [ 1810.292862] [ 1810.292862] *** DEADLOCK *** [ 1810.292862] [ 1810.292872] 2 locks held by kworker/1:0/32649: [ 1810.292878] #0: ("events"){.+.+.+}, at: [] process_one_work+0x192/0x520 [ 1810.292895] #1: (cpuset_hotplug_work){+.+...}, at: [] process_one_work+0x192/0x520 [ 1810.292911] [ 1810.292911] stack backtrace: [ 1810.292920] CPU: 1 PID: 32649 Comm: kworker/1:0 Not tainted 3.16.0-rc1-0.1-default+ #7 [ 1810.292929] Hardware name: Huawei Technologies Co., Ltd. Tecal RH2285 /BC11BTSA , BIOS CTSAV036 04/27/2011 [ 1810.292943] Workqueue: events cpuset_hotplug_workfn [ 1810.292951] ffffffff824b01e0 ffff8800afdd3918 ffffffff815a5f78 ffff8800afdd3958 [ 1810.292964] ffffffff810c263f 000000001d1fa490 ffff8800afdd3978 ffff88061d1fa490 [ 1810.292976] 0000000000000000 ffff88061d1fad08 ffff88061d1fad40 ffff8800afdd39f8 [ 1810.292989] Call Trace: [ 1810.292998] [] dump_stack+0x72/0x8a [ 1810.293010] [] print_circular_bug+0x10f/0x120 [ 1810.293019] [] check_prev_add+0x43e/0x4b0 [ 1810.293029] [] validate_chain+0x656/0x7c0 [ 1810.293038] [] __lock_acquire+0x382/0x660 [ 1810.293047] [] lock_acquire+0xf9/0x170 [ 1810.293056] [] ? cgroup_transfer_tasks+0x37/0x150 [ 1810.293066] [] mutex_lock_nested+0x6f/0x380 [ 1810.293075] [] ? cgroup_transfer_tasks+0x37/0x150 [ 1810.293085] [] ? cgroup_transfer_tasks+0x37/0x150 [ 1810.293096] [] ? local_clock+0x13/0x30 [ 1810.293104] [] ? lock_release_holdtime+0x35/0x230 [ 1810.293114] [] cgroup_transfer_tasks+0x37/0x150 [ 1810.293124] [] ? trace_hardirqs_on_caller+0x14d/0x1f0 [ 1810.293134] [] ? trace_hardirqs_on+0xd/0x10 [ 1810.293144] [] hotplug_update_tasks_insane+0x110/0x1d0 [ 1810.293154] [] cpuset_hotplug_update_tasks+0x13d/0x180 [ 1810.293165] [] cpuset_hotplug_workfn+0x18c/0x630 [ 1810.293175] [] ? cpuset_hotplug_workfn+0x250/0x630 [ 1810.293185] [] process_one_work+0x254/0x520 [ 1810.293194] [] ? process_one_work+0x192/0x520 [ 1810.293204] [] worker_thread+0x13d/0x3d0 [ 1810.293214] [] ? maybe_create_worker+0x190/0x190 [ 1810.293223] [] kthread+0xf8/0x100 [ 1810.293233] [] ? __init_kthread_worker+0x70/0x70 [ 1810.293242] [] ret_from_fork+0x7c/0xb0 [ 1810.293251] [] ? __init_kthread_worker+0x70/0x70 [ 1810.317270] numa_remove_cpu cpu 1 node 0: mask now 0,2-3,8-11 [ 1810.318776] smpboot: CPU 1 is now offline -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/