2018-07-23 17:31:08

by syzbot

[permalink] [raw]
Subject: possible deadlock in mnt_want_write

Hello,

syzbot found the following crash on:

HEAD commit: 45ae4df92207 Merge tag 'armsoc-fixes' of git://git.kernel...
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=10e7eee0400000
kernel config: https://syzkaller.appspot.com/x/.config?x=c0bdc4175608181c
dashboard link: https://syzkaller.appspot.com/bug?extid=ae82084b07d0297e566b
compiler: gcc (GCC) 8.0.1 20180413 (experimental)

Unfortunately, I don't have any reproducer for this crash yet.

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: [email protected]

device bridge_slave_0 left promiscuous mode
bridge0: port 1(bridge_slave_0) entered disabled state
IPVS: set_ctl: invalid protocol: 255 0.0.0.0:20004

======================================================
WARNING: possible circular locking dependency detected
4.18.0-rc5+ #159 Not tainted
------------------------------------------------------
syz-executor7/24660 is trying to acquire lock:
000000007bd46ec8 (sb_writers#15){.+.+}, at: sb_start_write
include/linux/fs.h:1554 [inline]
000000007bd46ec8 (sb_writers#15){.+.+}, at: mnt_want_write+0x3f/0xc0
fs/namespace.c:386

but task is already holding lock:
00000000a4a13f7a (&fi->mutex){+.+.}, at: fuse_lock_inode+0xaf/0xe0
fs/fuse/inode.c:363

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #2 (&fi->mutex){+.+.}:
__mutex_lock_common kernel/locking/mutex.c:757 [inline]
__mutex_lock+0x176/0x1820 kernel/locking/mutex.c:894
mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:909
fuse_lock_inode+0xaf/0xe0 fs/fuse/inode.c:363
fuse_lookup+0x8f/0x4c0 fs/fuse/dir.c:359
__lookup_hash+0x12e/0x190 fs/namei.c:1505
filename_create+0x1e5/0x5b0 fs/namei.c:3646
user_path_create fs/namei.c:3703 [inline]
do_mkdirat+0xda/0x310 fs/namei.c:3842
__do_sys_mkdirat fs/namei.c:3861 [inline]
__se_sys_mkdirat fs/namei.c:3859 [inline]
__x64_sys_mkdirat+0x76/0xb0 fs/namei.c:3859
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe

-> #1 (&type->i_mutex_dir_key#5/1){+.+.}:
down_write_nested+0x93/0x130 kernel/locking/rwsem.c:192
inode_lock_nested include/linux/fs.h:750 [inline]
filename_create+0x1b2/0x5b0 fs/namei.c:3645
user_path_create fs/namei.c:3703 [inline]
do_mkdirat+0xda/0x310 fs/namei.c:3842
__do_sys_mkdirat fs/namei.c:3861 [inline]
__se_sys_mkdirat fs/namei.c:3859 [inline]
__x64_sys_mkdirat+0x76/0xb0 fs/namei.c:3859
nla_parse: 14 callbacks suppressed
netlink: 3 bytes leftover after parsing attributes in process
`syz-executor1'.
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe

-> #0 (sb_writers#15){.+.+}:
lock_acquire+0x1e4/0x540 kernel/locking/lockdep.c:3924
percpu_down_read_preempt_disable include/linux/percpu-rwsem.h:36
[inline]
percpu_down_read include/linux/percpu-rwsem.h:59 [inline]
__sb_start_write+0x1e9/0x300 fs/super.c:1403
sb_start_write include/linux/fs.h:1554 [inline]
mnt_want_write+0x3f/0xc0 fs/namespace.c:386
path_removexattr+0xf0/0x210 fs/xattr.c:703
__do_sys_removexattr fs/xattr.c:719 [inline]
__se_sys_removexattr fs/xattr.c:716 [inline]
__x64_sys_removexattr+0x59/0x80 fs/xattr.c:716
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe

other info that might help us debug this:

Chain exists of:
sb_writers#15 --> &type->i_mutex_dir_key#5/1 --> &fi->mutex

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock(&fi->mutex);
lock(&type->i_mutex_dir_key#5/1);
lock(&fi->mutex);
lock(sb_writers#15);

*** DEADLOCK ***

1 lock held by syz-executor7/24660:
#0: 00000000a4a13f7a (&fi->mutex){+.+.}, at: fuse_lock_inode+0xaf/0xe0
fs/fuse/inode.c:363

stack backtrace:
CPU: 1 PID: 24660 Comm: syz-executor7 Not tainted 4.18.0-rc5+ #159
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x1c9/0x2b4 lib/dump_stack.c:113
print_circular_bug.isra.36.cold.57+0x1bd/0x27d
kernel/locking/lockdep.c:1227
check_prev_add kernel/locking/lockdep.c:1867 [inline]
check_prevs_add kernel/locking/lockdep.c:1980 [inline]
validate_chain kernel/locking/lockdep.c:2421 [inline]
__lock_acquire+0x3449/0x5020 kernel/locking/lockdep.c:3435
lock_acquire+0x1e4/0x540 kernel/locking/lockdep.c:3924
percpu_down_read_preempt_disable include/linux/percpu-rwsem.h:36 [inline]
percpu_down_read include/linux/percpu-rwsem.h:59 [inline]
__sb_start_write+0x1e9/0x300 fs/super.c:1403
sb_start_write include/linux/fs.h:1554 [inline]
mnt_want_write+0x3f/0xc0 fs/namespace.c:386
path_removexattr+0xf0/0x210 fs/xattr.c:703
__do_sys_removexattr fs/xattr.c:719 [inline]
__se_sys_removexattr fs/xattr.c:716 [inline]
__x64_sys_removexattr+0x59/0x80 fs/xattr.c:716
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x455ab9
Code: 1d ba fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7
48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff
ff 0f 83 eb b9 fb ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007fee4a211c68 EFLAGS: 00000246 ORIG_RAX: 00000000000000c5
RAX: ffffffffffffffda RBX: 00007fee4a2126d4 RCX: 0000000000455ab9
RDX: 0000000000000000 RSI: 0000000020000080 RDI: 0000000020000040
RBP: 000000000072bea0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff
R13: 00000000004bbc1c R14: 00000000004d0f48 R15: 0000000000000000
team0 (unregistering): Port device team_slave_1 removed
team0 (unregistering): Port device team_slave_0 removed
bond0 (unregistering): Releasing backup interface bond_slave_1
bond0 (unregistering): Releasing backup interface bond_slave_0
bond0 (unregistering): Released all slaves
netlink: 3 bytes leftover after parsing attributes in process
`syz-executor4'.
netlink: 3 bytes leftover after parsing attributes in process
`syz-executor3'.
netlink: 3 bytes leftover after parsing attributes in process
`syz-executor1'.
netlink: 3 bytes leftover after parsing attributes in process
`syz-executor2'.
netlink: 3 bytes leftover after parsing attributes in process
`syz-executor4'.
netlink: 3 bytes leftover after parsing attributes in process
`syz-executor2'.
netlink: 3 bytes leftover after parsing attributes in process
`syz-executor1'.
netlink: 3 bytes leftover after parsing attributes in process
`syz-executor4'.
netlink: 3 bytes leftover after parsing attributes in process
`syz-executor2'.
nla_parse: 16 callbacks suppressed
netlink: 3 bytes leftover after parsing attributes in process
`syz-executor4'.
netlink: 3 bytes leftover after parsing attributes in process
`syz-executor4'.
netlink: 3 bytes leftover after parsing attributes in process
`syz-executor4'.


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at [email protected].

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#bug-status-tracking for how to communicate with
syzbot.


2018-11-21 16:18:07

by syzbot

[permalink] [raw]
Subject: Re: possible deadlock in mnt_want_write

syzbot has found a reproducer for the following crash on:

HEAD commit: c8ce94b8fe53 Merge tag 'mips_fixes_4.20_3' of git://git.ke..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=16a16ed5400000
kernel config: https://syzkaller.appspot.com/x/.config?x=73e2bc0cb6463446
dashboard link: https://syzkaller.appspot.com/bug?extid=ae82084b07d0297e566b
compiler: gcc (GCC) 8.0.1 20180413 (experimental)
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=16d8ac5d400000

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: [email protected]

IPv6: ADDRCONF(NETDEV_CHANGE): veth1: link becomes ready
IPv6: ADDRCONF(NETDEV_CHANGE): veth0: link becomes ready
8021q: adding VLAN 0 to HW filter on device team0

======================================================
WARNING: possible circular locking dependency detected
4.20.0-rc3+ #122 Not tainted
------------------------------------------------------
syz-executor0/6225 is trying to acquire lock:
000000001881f73a (sb_writers#3){.+.+}, at: sb_start_write
include/linux/fs.h:1597 [inline]
000000001881f73a (sb_writers#3){.+.+}, at: mnt_want_write+0x3f/0xc0
fs/namespace.c:360

but task is already holding lock:
00000000c37872d6 (&iint->mutex){+.+.}, at: process_measurement+0x438/0x1bf0
security/integrity/ima/ima_main.c:224

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (&iint->mutex){+.+.}:
__mutex_lock_common kernel/locking/mutex.c:925 [inline]
__mutex_lock+0x166/0x16f0 kernel/locking/mutex.c:1072
mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:1087
process_measurement+0x438/0x1bf0
security/integrity/ima/ima_main.c:224
ima_file_check+0xe5/0x130 security/integrity/ima/ima_main.c:391
do_last fs/namei.c:3422 [inline]
path_openat+0x134a/0x5150 fs/namei.c:3534
do_filp_open+0x255/0x380 fs/namei.c:3564
do_sys_open+0x568/0x700 fs/open.c:1063
__do_sys_open fs/open.c:1081 [inline]
__se_sys_open fs/open.c:1076 [inline]
__x64_sys_open+0x7e/0xc0 fs/open.c:1076
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe

-> #0 (sb_writers#3){.+.+}:
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3844
percpu_down_read_preempt_disable include/linux/percpu-rwsem.h:36
[inline]
percpu_down_read include/linux/percpu-rwsem.h:59 [inline]
__sb_start_write+0x214/0x370 fs/super.c:1387
sb_start_write include/linux/fs.h:1597 [inline]
mnt_want_write+0x3f/0xc0 fs/namespace.c:360
ovl_want_write+0x76/0xa0 fs/overlayfs/util.c:24
ovl_open_maybe_copy_up+0x12c/0x190 fs/overlayfs/copy_up.c:888
ovl_open+0xb3/0x260 fs/overlayfs/file.c:123
do_dentry_open+0x499/0x1250 fs/open.c:771
vfs_open fs/open.c:880 [inline]
dentry_open+0x143/0x1d0 fs/open.c:896
ima_calc_file_hash+0x324/0x570
security/integrity/ima/ima_crypto.c:427
ima_collect_measurement+0x619/0x730
security/integrity/ima/ima_api.c:232
process_measurement+0x11fd/0x1bf0
security/integrity/ima/ima_main.c:284
ima_file_check+0xe5/0x130 security/integrity/ima/ima_main.c:391
do_last fs/namei.c:3422 [inline]
path_openat+0x134a/0x5150 fs/namei.c:3534
do_filp_open+0x255/0x380 fs/namei.c:3564
do_sys_open+0x568/0x700 fs/open.c:1063
__do_sys_open fs/open.c:1081 [inline]
__se_sys_open fs/open.c:1076 [inline]
__x64_sys_open+0x7e/0xc0 fs/open.c:1076
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe

other info that might help us debug this:

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock(&iint->mutex);
lock(sb_writers#3);
lock(&iint->mutex);
lock(sb_writers#3);

*** DEADLOCK ***

1 lock held by syz-executor0/6225:
#0: 00000000c37872d6 (&iint->mutex){+.+.}, at:
process_measurement+0x438/0x1bf0 security/integrity/ima/ima_main.c:224

stack backtrace:
CPU: 0 PID: 6225 Comm: syz-executor0 Not tainted 4.20.0-rc3+ #122
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x244/0x39d lib/dump_stack.c:113
print_circular_bug.isra.35.cold.54+0x1bd/0x27d
kernel/locking/lockdep.c:1221
check_prev_add kernel/locking/lockdep.c:1863 [inline]
check_prevs_add kernel/locking/lockdep.c:1976 [inline]
validate_chain kernel/locking/lockdep.c:2347 [inline]
__lock_acquire+0x3399/0x4c20 kernel/locking/lockdep.c:3341
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3844
percpu_down_read_preempt_disable include/linux/percpu-rwsem.h:36 [inline]
percpu_down_read include/linux/percpu-rwsem.h:59 [inline]
__sb_start_write+0x214/0x370 fs/super.c:1387
sb_start_write include/linux/fs.h:1597 [inline]
mnt_want_write+0x3f/0xc0 fs/namespace.c:360
ovl_want_write+0x76/0xa0 fs/overlayfs/util.c:24
ovl_open_maybe_copy_up+0x12c/0x190 fs/overlayfs/copy_up.c:888
ovl_open+0xb3/0x260 fs/overlayfs/file.c:123
do_dentry_open+0x499/0x1250 fs/open.c:771
vfs_open fs/open.c:880 [inline]
dentry_open+0x143/0x1d0 fs/open.c:896
ima_calc_file_hash+0x324/0x570 security/integrity/ima/ima_crypto.c:427
ima_collect_measurement+0x619/0x730 security/integrity/ima/ima_api.c:232
process_measurement+0x11fd/0x1bf0 security/integrity/ima/ima_main.c:284
ima_file_check+0xe5/0x130 security/integrity/ima/ima_main.c:391
do_last fs/namei.c:3422 [inline]
path_openat+0x134a/0x5150 fs/namei.c:3534
do_filp_open+0x255/0x380 fs/namei.c:3564
do_sys_open+0x568/0x700 fs/open.c:1063
__do_sys_open fs/open.c:1081 [inline]
__se_sys_open fs/open.c:1076 [inline]
__x64_sys_open+0x7e/0xc0 fs/open.c:1076
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x457569
Code: fd b3 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7
48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff
ff 0f 83 cb b3 fb ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007ffd93abed08 EFLAGS: 00000246 ORIG_RAX: 0000000000000002
RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 0000000000457569
RDX: 0000000000000040 RSI: 0000000000000003 RDI: 0000000020000780
RBP: 000000000072bf00 R08: 0000000000000000 R09: 0000000000000000


2018-11-21 19:07:10

by syzbot

[permalink] [raw]
Subject: Re: possible deadlock in mnt_want_write

syzbot has found a reproducer for the following crash on:

HEAD commit: 442b8cea2477 Add linux-next specific files for 20181109
git tree: linux-next
console output: https://syzkaller.appspot.com/x/log.txt?x=11a1426d400000
kernel config: https://syzkaller.appspot.com/x/.config?x=2f72bdb11df9fbe8
dashboard link: https://syzkaller.appspot.com/bug?extid=ae82084b07d0297e566b
compiler: gcc (GCC) 8.0.1 20180413 (experimental)
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1632326d400000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=17a16ed5400000

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: [email protected]

IPv6: ADDRCONF(NETDEV_CHANGE): veth1: link becomes ready
IPv6: ADDRCONF(NETDEV_CHANGE): veth0: link becomes ready
8021q: adding VLAN 0 to HW filter on device team0

======================================================
WARNING: possible circular locking dependency detected
4.20.0-rc1-next-20181109+ #110 Not tainted
------------------------------------------------------
syz-executor599/5968 is trying to acquire lock:
00000000e42cbf00 (sb_writers#3){.+.+}, at: sb_start_write
include/linux/fs.h:1607 [inline]
00000000e42cbf00 (sb_writers#3){.+.+}, at: mnt_want_write+0x3f/0xc0
fs/namespace.c:359

but task is already holding lock:
00000000166f985a (&iint->mutex){+.+.}, at: process_measurement+0x438/0x1bf0
security/integrity/ima/ima_main.c:224

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (&iint->mutex){+.+.}:
__mutex_lock_common kernel/locking/mutex.c:925 [inline]
__mutex_lock+0x166/0x16f0 kernel/locking/mutex.c:1072
mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:1087
process_measurement+0x438/0x1bf0
security/integrity/ima/ima_main.c:224
ima_file_check+0xe5/0x130 security/integrity/ima/ima_main.c:391
do_last fs/namei.c:3422 [inline]
path_openat+0x134a/0x5150 fs/namei.c:3534
do_filp_open+0x255/0x380 fs/namei.c:3564
do_sys_open+0x568/0x700 fs/open.c:1063
__do_sys_open fs/open.c:1081 [inline]
__se_sys_open fs/open.c:1076 [inline]
__x64_sys_open+0x7e/0xc0 fs/open.c:1076
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe

-> #0 (sb_writers#3){.+.+}:
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3844
percpu_down_read_preempt_disable include/linux/percpu-rwsem.h:36
[inline]
percpu_down_read include/linux/percpu-rwsem.h:59 [inline]
__sb_start_write+0x214/0x370 fs/super.c:1564
sb_start_write include/linux/fs.h:1607 [inline]
mnt_want_write+0x3f/0xc0 fs/namespace.c:359
ovl_want_write+0x76/0xa0 fs/overlayfs/util.c:24
ovl_open_maybe_copy_up+0x12c/0x190 fs/overlayfs/copy_up.c:888
ovl_open+0xb3/0x260 fs/overlayfs/file.c:123
do_dentry_open+0x499/0x1250 fs/open.c:771
vfs_open fs/open.c:880 [inline]
dentry_open+0x143/0x1d0 fs/open.c:896
ima_calc_file_hash+0x324/0x570
security/integrity/ima/ima_crypto.c:427
ima_collect_measurement+0x619/0x730
security/integrity/ima/ima_api.c:232
process_measurement+0x11fd/0x1bf0
security/integrity/ima/ima_main.c:284
ima_file_check+0xe5/0x130 security/integrity/ima/ima_main.c:391
do_last fs/namei.c:3422 [inline]
path_openat+0x134a/0x5150 fs/namei.c:3534
do_filp_open+0x255/0x380 fs/namei.c:3564
do_sys_open+0x568/0x700 fs/open.c:1063
__do_sys_open fs/open.c:1081 [inline]
__se_sys_open fs/open.c:1076 [inline]
__x64_sys_open+0x7e/0xc0 fs/open.c:1076
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe

other info that might help us debug this:

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock(&iint->mutex);
lock(sb_writers#3);
lock(&iint->mutex);
lock(sb_writers#3);

*** DEADLOCK ***

1 lock held by syz-executor599/5968:
#0: 00000000166f985a (&iint->mutex){+.+.}, at:
process_measurement+0x438/0x1bf0 security/integrity/ima/ima_main.c:224

stack backtrace:
CPU: 0 PID: 5968 Comm: syz-executor599 Not tainted
4.20.0-rc1-next-20181109+ #110
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x244/0x39d lib/dump_stack.c:113
print_circular_bug.isra.35.cold.56+0x1bd/0x27d
kernel/locking/lockdep.c:1221
check_prev_add kernel/locking/lockdep.c:1863 [inline]
check_prevs_add kernel/locking/lockdep.c:1976 [inline]
validate_chain kernel/locking/lockdep.c:2347 [inline]
__lock_acquire+0x3399/0x4c20 kernel/locking/lockdep.c:3341
lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3844
percpu_down_read_preempt_disable include/linux/percpu-rwsem.h:36 [inline]
percpu_down_read include/linux/percpu-rwsem.h:59 [inline]
__sb_start_write+0x214/0x370 fs/super.c:1564
sb_start_write include/linux/fs.h:1607 [inline]
mnt_want_write+0x3f/0xc0 fs/namespace.c:359
ovl_want_write+0x76/0xa0 fs/overlayfs/util.c:24
ovl_open_maybe_copy_up+0x12c/0x190 fs/overlayfs/copy_up.c:888
ovl_open+0xb3/0x260 fs/overlayfs/file.c:123
do_dentry_open+0x499/0x1250 fs/open.c:771
vfs_open fs/open.c:880 [inline]
dentry_open+0x143/0x1d0 fs/open.c:896
ima_calc_file_hash+0x324/0x570 security/integrity/ima/ima_crypto.c:427
ima_collect_measurement+0x619/0x730 security/integrity/ima/ima_api.c:232
process_measurement+0x11fd/0x1bf0 security/integrity/ima/ima_main.c:284
ima_file_check+0xe5/0x130 security/integrity/ima/ima_main.c:391
do_last fs/namei.c:3422 [inline]
path_openat+0x134a/0x5150 fs/namei.c:3534
do_filp_open+0x255/0x380 fs/namei.c:3564
do_sys_open+0x568/0x700 fs/open.c:1063
__do_sys_open fs/open.c:1081 [inline]
__se_sys_open fs/open.c:1076 [inline]
__x64_sys_open+0x7e/0xc0 fs/open.c:1076
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x441af9
Code: 23 02 00 85 c0 b8 00 00 00 00 48 0f 44 c3 5b c3 90 48 89 f8 48 89 f7
48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff
ff 0f 83 cb 08 fc ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007ffce8c93d48 EFLAGS: 00000246 ORIG_RAX: 0000000000000002
RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 0000000000441af9
RDX: 0000000000000000 RSI: 0000000000000003 RDI: 0000000020000040
RBP: 0000000000000000 R08: 0000000000402850 R09: 0000000000402850
R10: 0000000000402850 R11: 0000000000000246 R12: 0000000000402850
R13: 00000000004028e0 R14: 0000000000000000 R15: 0000000000000000
kobject: 'rx-0' (000000008bf42b56): kobject_cleanup, parent 00000000f75c8bd3
kobject: 'rx-0' (000000008bf42b56): auto cleanup 'remove' event
kobject: 'rx-0' (000000008bf42b56): kobject_uevent_env
kobject: 'rx-0' (000000008bf42b56): fill_kobj_path: path
= '/devices/virtual/net/syz_tun/queues/rx-0'
kobject: 'rx-0' (000000008bf42b56): auto cleanup kobject_del
kobject: 'rx-0' (000000008bf42b56): calling ktype release
kobject: 'rx-0': free name
kobject: 'tx-0' (00000000bbdcdbed): kobject_cleanup, parent 00000000f75c8bd3
kobject: 'tx-0' (00000000bbdcdbed): auto cleanup 'remove' event
kobject: 'tx-0' (00000000bbdcdbed): kobject_uevent_env
kobject: 'tx-0' (00000000bbdcdbed): fill_kobj_path: path
= '/devices/virtual/net/syz_tun/queues/tx-0'
kobject: 'tx-0' (00000000bbdcdbed): auto cleanup kobject_del
kobject: 'tx-0' (00000000bbdcdbed): calling ktype release
kobject: 'tx-0': free name
kobject: 'queues' (00000000f75c8bd3): kobject_cleanup, parent
(null)
kobject: 'queues' (00000000f75c8bd3): calling ktype release
kobject: 'queues' (00000000f75c8bd3): kset_release
kobject: 'queues': free name
kobject: 'syz_tun' (00000000ad4d7dba): kobject_uevent_env
kobject: 'syz_tun' (00000000ad4d7dba): fill_kobj_path: path
= '/devices/virtual/net/syz_tun'
kobject: 'syz_tun' (00000000ad4d7dba): kobject_cleanup, parent
(null)
kobject: 'syz_tun' (00000000ad4d7dba): calling ktype release
kobject: 'syz_tun': free name
kobject: 'rx-0' (0000000017c19f65): kobject_cleanup, parent 000000003ca64aec
kobject: 'rx-0' (0000000017c19f65): auto cleanup 'remove' event
kobject: 'rx-0' (0000000017c19f65): kobject_uevent_env
kobject: 'rx-0' (0000000017c19f65): kobject_uevent_env: uevent_suppress
caused the event to drop!
kobject: 'rx-0' (0000000017c19f65): auto cleanup kobject_del
kobject: 'rx-0' (0000000017c19f65): calling ktype release
kobject: 'rx-0': free name
kobject: 'tx-0' (000000003e77b572): kobject_cleanup, parent 000000003ca64aec
kobject: 'tx-0' (000000003e77b572): auto cleanup 'remove' event
kobject: 'tx-0' (000000003e77b572): kobject_uevent_env
kobject: 'tx-0' (000000003e77b572): kobject_uevent_env: uevent_suppress
caused the event to drop!
kobject: 'tx-0' (000000003e77b572): auto cleanup kobject_del
kobject: 'tx-0' (000000003e77b572): calling ktype release
kobject: 'tx-0': free name


2018-11-21 21:54:39

by Amir Goldstein

[permalink] [raw]
Subject: Re: possible deadlock in mnt_want_write

On Wed, Nov 21, 2018 at 8:33 PM syzbot
<[email protected]> wrote:
>
> syzbot has found a reproducer for the following crash on:
>
> HEAD commit: 442b8cea2477 Add linux-next specific files for 20181109
> git tree: linux-next
> console output: https://syzkaller.appspot.com/x/log.txt?x=11a1426d400000
> kernel config: https://syzkaller.appspot.com/x/.config?x=2f72bdb11df9fbe8
> dashboard link: https://syzkaller.appspot.com/bug?extid=ae82084b07d0297e566b
> compiler: gcc (GCC) 8.0.1 20180413 (experimental)
> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1632326d400000
> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=17a16ed5400000
>
> IMPORTANT: if you fix the bug, please add the following tag to the commit:
> Reported-by: [email protected]
>
> IPv6: ADDRCONF(NETDEV_CHANGE): veth1: link becomes ready
> IPv6: ADDRCONF(NETDEV_CHANGE): veth0: link becomes ready
> 8021q: adding VLAN 0 to HW filter on device team0
>
> ======================================================
> WARNING: possible circular locking dependency detected
> 4.20.0-rc1-next-20181109+ #110 Not tainted
> ------------------------------------------------------
> syz-executor599/5968 is trying to acquire lock:
> 00000000e42cbf00 (sb_writers#3){.+.+}, at: sb_start_write
> include/linux/fs.h:1607 [inline]
> 00000000e42cbf00 (sb_writers#3){.+.+}, at: mnt_want_write+0x3f/0xc0
> fs/namespace.c:359
>
> but task is already holding lock:
> 00000000166f985a (&iint->mutex){+.+.}, at: process_measurement+0x438/0x1bf0
> security/integrity/ima/ima_main.c:224
>
> which lock already depends on the new lock.
>
>
> the existing dependency chain (in reverse order) is:
>
> -> #1 (&iint->mutex){+.+.}:
> __mutex_lock_common kernel/locking/mutex.c:925 [inline]
> __mutex_lock+0x166/0x16f0 kernel/locking/mutex.c:1072
> mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:1087
> process_measurement+0x438/0x1bf0
> security/integrity/ima/ima_main.c:224
> ima_file_check+0xe5/0x130 security/integrity/ima/ima_main.c:391
> do_last fs/namei.c:3422 [inline]
> path_openat+0x134a/0x5150 fs/namei.c:3534
> do_filp_open+0x255/0x380 fs/namei.c:3564
> do_sys_open+0x568/0x700 fs/open.c:1063
> __do_sys_open fs/open.c:1081 [inline]
> __se_sys_open fs/open.c:1076 [inline]
> __x64_sys_open+0x7e/0xc0 fs/open.c:1076
> do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
> entry_SYSCALL_64_after_hwframe+0x49/0xbe
>
> -> #0 (sb_writers#3){.+.+}:
> lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3844
> percpu_down_read_preempt_disable include/linux/percpu-rwsem.h:36
> [inline]
> percpu_down_read include/linux/percpu-rwsem.h:59 [inline]
> __sb_start_write+0x214/0x370 fs/super.c:1564
> sb_start_write include/linux/fs.h:1607 [inline]
> mnt_want_write+0x3f/0xc0 fs/namespace.c:359
> ovl_want_write+0x76/0xa0 fs/overlayfs/util.c:24
> ovl_open_maybe_copy_up+0x12c/0x190 fs/overlayfs/copy_up.c:888
> ovl_open+0xb3/0x260 fs/overlayfs/file.c:123
> do_dentry_open+0x499/0x1250 fs/open.c:771
> vfs_open fs/open.c:880 [inline]
> dentry_open+0x143/0x1d0 fs/open.c:896
> ima_calc_file_hash+0x324/0x570

I suppose ima_calc_file_hash opens the file with write flags
and cause overlay to try to copy up which takes mnt_want_write().
Why does IMA need to open the file with write flags?

Isn't this commit supposed to prevent that:
a408e4a86b36 ima: open a new file instance if no read permissions

Thanks,
Amir.

2018-11-22 03:38:36

by Goldwyn Rodrigues

[permalink] [raw]
Subject: Re: possible deadlock in mnt_want_write

On 20:57 21/11, Amir Goldstein wrote:
> On Wed, Nov 21, 2018 at 8:33 PM syzbot
> <[email protected]> wrote:
> >
> > syzbot has found a reproducer for the following crash on:
> >
> > HEAD commit: 442b8cea2477 Add linux-next specific files for 20181109
> > git tree: linux-next
> > console output: https://syzkaller.appspot.com/x/log.txt?x=11a1426d400000
> > kernel config: https://syzkaller.appspot.com/x/.config?x=2f72bdb11df9fbe8
> > dashboard link: https://syzkaller.appspot.com/bug?extid=ae82084b07d0297e566b
> > compiler: gcc (GCC) 8.0.1 20180413 (experimental)
> > syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1632326d400000
> > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=17a16ed5400000
> >
> > IMPORTANT: if you fix the bug, please add the following tag to the commit:
> > Reported-by: [email protected]
> >
> > IPv6: ADDRCONF(NETDEV_CHANGE): veth1: link becomes ready
> > IPv6: ADDRCONF(NETDEV_CHANGE): veth0: link becomes ready
> > 8021q: adding VLAN 0 to HW filter on device team0
> >
> > ======================================================
> > WARNING: possible circular locking dependency detected
> > 4.20.0-rc1-next-20181109+ #110 Not tainted
> > ------------------------------------------------------
> > syz-executor599/5968 is trying to acquire lock:
> > 00000000e42cbf00 (sb_writers#3){.+.+}, at: sb_start_write
> > include/linux/fs.h:1607 [inline]
> > 00000000e42cbf00 (sb_writers#3){.+.+}, at: mnt_want_write+0x3f/0xc0
> > fs/namespace.c:359
> >
> > but task is already holding lock:
> > 00000000166f985a (&iint->mutex){+.+.}, at: process_measurement+0x438/0x1bf0
> > security/integrity/ima/ima_main.c:224
> >
> > which lock already depends on the new lock.
> >
> >
> > the existing dependency chain (in reverse order) is:
> >
> > -> #1 (&iint->mutex){+.+.}:
> > __mutex_lock_common kernel/locking/mutex.c:925 [inline]
> > __mutex_lock+0x166/0x16f0 kernel/locking/mutex.c:1072
> > mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:1087
> > process_measurement+0x438/0x1bf0
> > security/integrity/ima/ima_main.c:224
> > ima_file_check+0xe5/0x130 security/integrity/ima/ima_main.c:391
> > do_last fs/namei.c:3422 [inline]
> > path_openat+0x134a/0x5150 fs/namei.c:3534
> > do_filp_open+0x255/0x380 fs/namei.c:3564
> > do_sys_open+0x568/0x700 fs/open.c:1063
> > __do_sys_open fs/open.c:1081 [inline]
> > __se_sys_open fs/open.c:1076 [inline]
> > __x64_sys_open+0x7e/0xc0 fs/open.c:1076
> > do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
> > entry_SYSCALL_64_after_hwframe+0x49/0xbe
> >
> > -> #0 (sb_writers#3){.+.+}:
> > lock_acquire+0x1ed/0x520 kernel/locking/lockdep.c:3844
> > percpu_down_read_preempt_disable include/linux/percpu-rwsem.h:36
> > [inline]
> > percpu_down_read include/linux/percpu-rwsem.h:59 [inline]
> > __sb_start_write+0x214/0x370 fs/super.c:1564
> > sb_start_write include/linux/fs.h:1607 [inline]
> > mnt_want_write+0x3f/0xc0 fs/namespace.c:359
> > ovl_want_write+0x76/0xa0 fs/overlayfs/util.c:24
> > ovl_open_maybe_copy_up+0x12c/0x190 fs/overlayfs/copy_up.c:888
> > ovl_open+0xb3/0x260 fs/overlayfs/file.c:123
> > do_dentry_open+0x499/0x1250 fs/open.c:771
> > vfs_open fs/open.c:880 [inline]
> > dentry_open+0x143/0x1d0 fs/open.c:896
> > ima_calc_file_hash+0x324/0x570
>
> I suppose ima_calc_file_hash opens the file with write flags
> and cause overlay to try to copy up which takes mnt_want_write().
> Why does IMA need to open the file with write flags?
>
> Isn't this commit supposed to prevent that:
> a408e4a86b36 ima: open a new file instance if no read permissions
>

Not write, read flags. This patch re-opens the files in O_RDONLY for
files opened with O_WRONLY and cannot be read, so that the hash can be
calculated. IOW, the user opened the file in overlayfs with write flags.

The copy-up, since it is already in write mode can add the security.ima
xattrs.

--
Goldwyn

2018-11-22 06:32:12

by Amir Goldstein

[permalink] [raw]
Subject: Re: possible deadlock in mnt_want_write

On Wed, Nov 21, 2018 at 10:04 PM Goldwyn Rodrigues <[email protected]> wrote:
>
> On 20:57 21/11, Amir Goldstein wrote:
> > On Wed, Nov 21, 2018 at 8:33 PM syzbot
> > <[email protected]> wrote:
> > >
> > > syzbot has found a reproducer for the following crash on:
> > >
> > > HEAD commit: 442b8cea2477 Add linux-next specific files for 20181109
> > > git tree: linux-next
> > > console output: https://syzkaller.appspot.com/x/log.txt?x=11a1426d400000
> > > kernel config: https://syzkaller.appspot.com/x/.config?x=2f72bdb11df9fbe8
> > > dashboard link: https://syzkaller.appspot.com/bug?extid=ae82084b07d0297e566b
> > > compiler: gcc (GCC) 8.0.1 20180413 (experimental)
> > > syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1632326d400000
> > > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=17a16ed5400000
> > >
> > > IMPORTANT: if you fix the bug, please add the following tag to the commit:
> > > Reported-by: [email protected]
> > >
...
> > > percpu_down_read include/linux/percpu-rwsem.h:59 [inline]
> > > __sb_start_write+0x214/0x370 fs/super.c:1564
> > > sb_start_write include/linux/fs.h:1607 [inline]
> > > mnt_want_write+0x3f/0xc0 fs/namespace.c:359
> > > ovl_want_write+0x76/0xa0 fs/overlayfs/util.c:24
> > > ovl_open_maybe_copy_up+0x12c/0x190 fs/overlayfs/copy_up.c:888
> > > ovl_open+0xb3/0x260 fs/overlayfs/file.c:123
> > > do_dentry_open+0x499/0x1250 fs/open.c:771
> > > vfs_open fs/open.c:880 [inline]
> > > dentry_open+0x143/0x1d0 fs/open.c:896
> > > ima_calc_file_hash+0x324/0x570
> >
> > I suppose ima_calc_file_hash opens the file with write flags
> > and cause overlay to try to copy up which takes mnt_want_write().
> > Why does IMA need to open the file with write flags?
> >
> > Isn't this commit supposed to prevent that:
> > a408e4a86b36 ima: open a new file instance if no read permissions
> >
>
> Not write, read flags. This patch re-opens the files in O_RDONLY for
> files opened with O_WRONLY and cannot be read, so that the hash can be
> calculated. IOW, the user opened the file in overlayfs with write flags.
>

My point is:
ovl_open_need_copy_up() -> ovl_open_flags_need_copy_up()
returns false for O_RDONLY flags and never gets to ovl_want_write(),
so how is the stack trace below possible when ima_calc_file_hash()
removes all "write" flags before opening the file?

ovl_want_write+0x76/0xa0 fs/overlayfs/util.c:24
ovl_open_maybe_copy_up+0x12c/0x190 fs/overlayfs/copy_up.c:888
ovl_open+0xb3/0x260 fs/overlayfs/file.c:123
do_dentry_open+0x499/0x1250 fs/open.c:771
vfs_open fs/open.c:880 [inline]
dentry_open+0x143/0x1d0 fs/open.c:896
ima_calc_file_hash+0x324/0x570 security/integrity/ima/ima_crypto.c:427

The answer is found in the zysbot repro: it opens the file with
open(O_WRONLY|O_RDWR) (0x3)
Not nice, but apparently possible.

How about adding O_RDWR to the masked flags in ima_calc_file_hash()?
Since syzbot has a reproducer, you can send it a patch to verify the fix.

Thanks,
Amir.

2019-11-23 02:28:29

by syzbot

[permalink] [raw]
Subject: Re: possible deadlock in mnt_want_write

syzbot has bisected this bug to:

commit 8e54cadab447dae779f80f79c87cbeaea9594f60
Author: Al Viro <[email protected]>
Date: Sun Nov 27 01:05:42 2016 +0000

fix default_file_splice_read()

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=15147a36e00000
start commit: 6d906f99 Merge tag 'arm64-fixes' of git://git.kernel.org/p..
git tree: upstream
final crash: https://syzkaller.appspot.com/x/report.txt?x=17147a36e00000
console output: https://syzkaller.appspot.com/x/log.txt?x=13147a36e00000
kernel config: https://syzkaller.appspot.com/x/.config?x=856fc6d0fbbeede9
dashboard link: https://syzkaller.appspot.com/bug?extid=ae82084b07d0297e566b
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=111767b7200000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=1611ab2d200000

Reported-by: [email protected]
Fixes: 8e54cadab447 ("fix default_file_splice_read()")

For information about bisection process see: https://goo.gl/tpsmEJ#bisection

2020-11-07 12:14:21

by syzbot

[permalink] [raw]
Subject: Re: possible deadlock in mnt_want_write

syzbot suspects this issue was fixed by commit:

commit 146d62e5a5867fbf84490d82455718bfb10fe824
Author: Amir Goldstein <[email protected]>
Date: Thu Apr 18 14:42:08 2019 +0000

ovl: detect overlapping layers

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=11e40184500000
start commit: 6d906f99 Merge tag 'arm64-fixes' of git://git.kernel.org/p..
git tree: upstream
kernel config: https://syzkaller.appspot.com/x/.config?x=856fc6d0fbbeede9
dashboard link: https://syzkaller.appspot.com/bug?extid=ae82084b07d0297e566b
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=111767b7200000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=1611ab2d200000

If the result looks correct, please mark the issue as fixed by replying with:

#syz fix: ovl: detect overlapping layers

For information about bisection process see: https://goo.gl/tpsmEJ#bisection

2020-11-11 13:54:15

by Dmitry Vyukov

[permalink] [raw]
Subject: Re: possible deadlock in mnt_want_write

On Sat, Nov 7, 2020 at 1:10 PM syzbot
<[email protected]> wrote:
>
> syzbot suspects this issue was fixed by commit:
>
> commit 146d62e5a5867fbf84490d82455718bfb10fe824
> Author: Amir Goldstein <[email protected]>
> Date: Thu Apr 18 14:42:08 2019 +0000
>
> ovl: detect overlapping layers
>
> bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=11e40184500000
> start commit: 6d906f99 Merge tag 'arm64-fixes' of git://git.kernel.org/p..
> git tree: upstream
> kernel config: https://syzkaller.appspot.com/x/.config?x=856fc6d0fbbeede9
> dashboard link: https://syzkaller.appspot.com/bug?extid=ae82084b07d0297e566b
> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=111767b7200000
> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=1611ab2d200000
>
> If the result looks correct, please mark the issue as fixed by replying with:
>
> #syz fix: ovl: detect overlapping layers
>
> For information about bisection process see: https://goo.gl/tpsmEJ#bisection

#syz fix: ovl: detect overlapping layers