2023-07-19 13:39:58

by syzbot

[permalink] [raw]
Subject: [syzbot] [overlayfs?] possible deadlock in seq_read_iter (2)

Hello,

syzbot found the following issue on:

HEAD commit: fdf0eaf11452 Linux 6.5-rc2
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=135aae66a80000
kernel config: https://syzkaller.appspot.com/x/.config?x=a4507c291b5ab5d4
dashboard link: https://syzkaller.appspot.com/bug?extid=da4f9f61f96525c62cc7
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40

Unfortunately, I don't have any reproducer for this issue yet.

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/f95d243908cc/disk-fdf0eaf1.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/f55beab9d6de/vmlinux-fdf0eaf1.xz
kernel image: https://storage.googleapis.com/syzbot-assets/bea743a43c4f/bzImage-fdf0eaf1.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: [email protected]

======================================================
WARNING: possible circular locking dependency detected
6.5.0-rc2-syzkaller #0 Not tainted
------------------------------------------------------
syz-executor.0/14284 is trying to acquire lock:
ffff8880870e9c30 (&p->lock){+.+.}-{3:3}, at: seq_read_iter+0xb2/0xd10 fs/seq_file.c:182

but task is already holding lock:
ffff88814bdc8410 (sb_writers#4){.+.+}-{0:0}, at: do_sendfile+0x600/0x1070 fs/read_write.c:1253

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #3 (sb_writers#4){.+.+}-{0:0}:
percpu_down_read include/linux/percpu-rwsem.h:51 [inline]
__sb_start_write include/linux/fs.h:1494 [inline]
sb_start_write+0x4d/0x1c0 include/linux/fs.h:1569
mnt_want_write+0x3f/0x90 fs/namespace.c:403
ovl_create_object+0xf8/0x300 fs/overlayfs/dir.c:629
lookup_open fs/namei.c:3492 [inline]
open_last_lookups fs/namei.c:3560 [inline]
path_openat+0x13e7/0x3180 fs/namei.c:3790
do_filp_open+0x234/0x490 fs/namei.c:3820
do_sys_openat2+0x13e/0x1d0 fs/open.c:1407
do_sys_open fs/open.c:1422 [inline]
__do_sys_openat fs/open.c:1438 [inline]
__se_sys_openat fs/open.c:1433 [inline]
__x64_sys_openat+0x247/0x290 fs/open.c:1433
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd

-> #2 (&ovl_i_mutex_dir_key[depth]){++++}-{3:3}:
down_read+0x47/0x2f0 kernel/locking/rwsem.c:1520
inode_lock_shared include/linux/fs.h:781 [inline]
lookup_slow+0x45/0x70 fs/namei.c:1706
walk_component+0x2d0/0x400 fs/namei.c:1998
lookup_last fs/namei.c:2455 [inline]
path_lookupat+0x16f/0x450 fs/namei.c:2479
filename_lookup+0x255/0x610 fs/namei.c:2508
kern_path+0x3b/0x180 fs/namei.c:2606
lookup_bdev+0xc5/0x290 block/bdev.c:943
resume_store+0x19c/0x700 kernel/power/hibernate.c:1177
kernfs_fop_write_iter+0x3a6/0x4f0 fs/kernfs/file.c:334
call_write_iter include/linux/fs.h:1871 [inline]
new_sync_write fs/read_write.c:491 [inline]
vfs_write+0x782/0xaf0 fs/read_write.c:584
ksys_write+0x1a0/0x2c0 fs/read_write.c:637
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd

-> #1 (&of->mutex){+.+.}-{3:3}:
__mutex_lock_common+0x1d8/0x2530 kernel/locking/mutex.c:603
__mutex_lock kernel/locking/mutex.c:747 [inline]
mutex_lock_nested+0x1b/0x20 kernel/locking/mutex.c:799
kernfs_seq_start+0x53/0x3a0 fs/kernfs/file.c:154
seq_read_iter+0x3d4/0xd10 fs/seq_file.c:225
call_read_iter include/linux/fs.h:1865 [inline]
new_sync_read fs/read_write.c:389 [inline]
vfs_read+0x795/0xb00 fs/read_write.c:470
ksys_read+0x1a0/0x2c0 fs/read_write.c:613
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd

-> #0 (&p->lock){+.+.}-{3:3}:
check_prev_add kernel/locking/lockdep.c:3142 [inline]
check_prevs_add kernel/locking/lockdep.c:3261 [inline]
validate_chain kernel/locking/lockdep.c:3876 [inline]
__lock_acquire+0x39ff/0x7f70 kernel/locking/lockdep.c:5144
lock_acquire+0x1e3/0x520 kernel/locking/lockdep.c:5761
__mutex_lock_common+0x1d8/0x2530 kernel/locking/mutex.c:603
__mutex_lock kernel/locking/mutex.c:747 [inline]
mutex_lock_nested+0x1b/0x20 kernel/locking/mutex.c:799
seq_read_iter+0xb2/0xd10 fs/seq_file.c:182
proc_reg_read_iter+0x1bc/0x290 fs/proc/inode.c:305
call_read_iter include/linux/fs.h:1865 [inline]
copy_splice_read+0x4c9/0x9c0 fs/splice.c:367
splice_direct_to_actor+0x2c4/0x9e0 fs/splice.c:1070
do_splice_direct+0x2ac/0x3f0 fs/splice.c:1195
do_sendfile+0x623/0x1070 fs/read_write.c:1254
__do_sys_sendfile64 fs/read_write.c:1322 [inline]
__se_sys_sendfile64+0x17c/0x1e0 fs/read_write.c:1308
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd

other info that might help us debug this:

Chain exists of:
&p->lock --> &ovl_i_mutex_dir_key[depth] --> sb_writers#4

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
rlock(sb_writers#4);
lock(&ovl_i_mutex_dir_key[depth]);
lock(sb_writers#4);
lock(&p->lock);

*** DEADLOCK ***

1 lock held by syz-executor.0/14284:
#0: ffff88814bdc8410 (sb_writers#4){.+.+}-{0:0}, at: do_sendfile+0x600/0x1070 fs/read_write.c:1253

stack backtrace:
CPU: 1 PID: 14284 Comm: syz-executor.0 Not tainted 6.5.0-rc2-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/03/2023
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x1e7/0x2d0 lib/dump_stack.c:106
check_noncircular+0x375/0x4a0 kernel/locking/lockdep.c:2195
check_prev_add kernel/locking/lockdep.c:3142 [inline]
check_prevs_add kernel/locking/lockdep.c:3261 [inline]
validate_chain kernel/locking/lockdep.c:3876 [inline]
__lock_acquire+0x39ff/0x7f70 kernel/locking/lockdep.c:5144
lock_acquire+0x1e3/0x520 kernel/locking/lockdep.c:5761
__mutex_lock_common+0x1d8/0x2530 kernel/locking/mutex.c:603
__mutex_lock kernel/locking/mutex.c:747 [inline]
mutex_lock_nested+0x1b/0x20 kernel/locking/mutex.c:799
seq_read_iter+0xb2/0xd10 fs/seq_file.c:182
proc_reg_read_iter+0x1bc/0x290 fs/proc/inode.c:305
call_read_iter include/linux/fs.h:1865 [inline]
copy_splice_read+0x4c9/0x9c0 fs/splice.c:367
splice_direct_to_actor+0x2c4/0x9e0 fs/splice.c:1070
do_splice_direct+0x2ac/0x3f0 fs/splice.c:1195
do_sendfile+0x623/0x1070 fs/read_write.c:1254
__do_sys_sendfile64 fs/read_write.c:1322 [inline]
__se_sys_sendfile64+0x17c/0x1e0 fs/read_write.c:1308
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7fed17a7cb29
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 e1 20 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fed188640c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000028
RAX: ffffffffffffffda RBX: 00007fed17b9c1f0 RCX: 00007fed17a7cb29
RDX: 0000000000000000 RSI: 0000000000000009 RDI: 0000000000000008
RBP: 00007fed17ac847a R08: 0000000000000000 R09: 0000000000000000
R10: 4000000000010046 R11: 0000000000000246 R12: 0000000000000000
R13: 000000000000006e R14: 00007fed17b9c1f0 R15: 00007fff433e8a98
</TASK>


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at [email protected].

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the bug is already fixed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want to change bug's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the bug is a duplicate of another bug, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup


2023-12-19 19:43:41

by syzbot

[permalink] [raw]
Subject: Re: [syzbot] [overlayfs?] possible deadlock in seq_read_iter (2)

syzbot has found a reproducer for the following issue on:

HEAD commit: 2cf4f94d8e86 Merge tag 'scsi-fixes' of git://git.kernel.or..
git tree: upstream
console+strace: https://syzkaller.appspot.com/x/log.txt?x=12181571e80000
kernel config: https://syzkaller.appspot.com/x/.config?x=e5751b3a2226135d
dashboard link: https://syzkaller.appspot.com/bug?extid=da4f9f61f96525c62cc7
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=176a4f49e80000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=154aa8d6e80000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/cc8943b61205/disk-2cf4f94d.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/8b515b02658d/vmlinux-2cf4f94d.xz
kernel image: https://storage.googleapis.com/syzbot-assets/1f8ccc925248/bzImage-2cf4f94d.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: [email protected]

======================================================
WARNING: possible circular locking dependency detected
6.7.0-rc6-syzkaller-00010-g2cf4f94d8e86 #0 Not tainted
------------------------------------------------------
syz-executor424/7758 is trying to acquire lock:
ffff88801f1ef9e0 (&p->lock){+.+.}-{3:3}, at: seq_read_iter+0xb2/0xd10 fs/seq_file.c:182

but task is already holding lock:
ffff88814cd7a418 (sb_writers#4){.+.+}-{0:0}, at: do_sendfile+0x607/0x1000 fs/read_write.c:1253

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #3
(sb_writers#4){.+.+}-{0:0}:
lock_acquire+0x1e3/0x530 kernel/locking/lockdep.c:5754
percpu_down_read include/linux/percpu-rwsem.h:51 [inline]
__sb_start_write include/linux/fs.h:1635 [inline]
sb_start_write+0x4d/0x1c0 include/linux/fs.h:1710
mnt_want_write+0x3f/0x90 fs/namespace.c:404
ovl_create_object+0x13b/0x360 fs/overlayfs/dir.c:629
lookup_open fs/namei.c:3477 [inline]
open_last_lookups fs/namei.c:3546 [inline]
path_openat+0x13fa/0x3290 fs/namei.c:3776
do_filp_open+0x234/0x490 fs/namei.c:3809
do_sys_openat2+0x13e/0x1d0 fs/open.c:1437
do_sys_open fs/open.c:1452 [inline]
__do_sys_open fs/open.c:1460 [inline]
__se_sys_open fs/open.c:1456 [inline]
__x64_sys_open+0x225/0x270 fs/open.c:1456
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0x45/0x110 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x63/0x6b

-> #2 (&ovl_i_mutex_dir_key[depth]){++++}-{3:3}:
lock_acquire+0x1e3/0x530 kernel/locking/lockdep.c:5754
down_read+0xb1/0xa40 kernel/locking/rwsem.c:1526
inode_lock_shared include/linux/fs.h:812 [inline]
lookup_slow+0x45/0x70 fs/namei.c:1710
walk_component+0x2d0/0x400 fs/namei.c:2002
lookup_last fs/namei.c:2459 [inline]
path_lookupat+0x16f/0x450 fs/namei.c:2483
filename_lookup+0x255/0x610 fs/namei.c:2512
kern_path+0x35/0x50 fs/namei.c:2610
lookup_bdev+0xc5/0x290 block/bdev.c:979
resume_store+0x1a0/0x710 kernel/power/hibernate.c:1177
kernfs_fop_write_iter+0x3b3/0x510 fs/kernfs/file.c:334
do_iter_readv_writev+0x330/0x4a0
do_iter_write+0x1f6/0x8d0 fs/read_write.c:860
iter_file_splice_write+0x86d/0x1010 fs/splice.c:736
do_splice_from fs/splice.c:933 [inline]
direct_splice_actor+0xea/0x1c0 fs/splice.c:1142
splice_direct_to_actor+0x376/0x9e0 fs/splice.c:1088
do_splice_direct+0x2ac/0x3f0 fs/splice.c:1194
do_sendfile+0x62c/0x1000 fs/read_write.c:1254
__do_sys_sendfile64 fs/read_write.c:1322 [inline]
__se_sys_sendfile64+0x17c/0x1e0 fs/read_write.c:1308
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0x45/0x110 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x63/0x6b

-> #1 (&of->mutex){+.+.}-{3:3}:
lock_acquire+0x1e3/0x530 kernel/locking/lockdep.c:5754
__mutex_lock_common kernel/locking/mutex.c:603 [inline]
__mutex_lock+0x136/0xd60 kernel/locking/mutex.c:747
kernfs_seq_start+0x53/0x3a0 fs/kernfs/file.c:154
seq_read_iter+0x3d4/0xd10 fs/seq_file.c:225
call_read_iter include/linux/fs.h:2014 [inline]
new_sync_read fs/read_write.c:389 [inline]
vfs_read+0x78b/0xb00 fs/read_write.c:470
ksys_read+0x1a0/0x2c0 fs/read_write.c:613
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0x45/0x110 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x63/0x6b

-> #0 (&p->lock){+.+.}-{3:3}:
check_prev_add kernel/locking/lockdep.c:3134 [inline]
check_prevs_add kernel/locking/lockdep.c:3253 [inline]
validate_chain+0x1909/0x5ab0 kernel/locking/lockdep.c:3869
__lock_acquire+0x1345/0x1fd0 kernel/locking/lockdep.c:5137
lock_acquire+0x1e3/0x530 kernel/locking/lockdep.c:5754
__mutex_lock_common kernel/locking/mutex.c:603 [inline]
__mutex_lock+0x136/0xd60 kernel/locking/mutex.c:747
seq_read_iter+0xb2/0xd10 fs/seq_file.c:182
call_read_iter include/linux/fs.h:2014 [inline]
copy_splice_read+0x4c9/0x9c0 fs/splice.c:364
splice_direct_to_actor+0x2c4/0x9e0 fs/splice.c:1069
do_splice_direct+0x2ac/0x3f0 fs/splice.c:1194
do_sendfile+0x62c/0x1000 fs/read_write.c:1254
__do_sys_sendfile64 fs/read_write.c:1322 [inline]
__se_sys_sendfile64+0x17c/0x1e0 fs/read_write.c:1308
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0x45/0x110 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x63/0x6b

other info that might help us debug this:

Chain exists of:
&p->lock --> &ovl_i_mutex_dir_key[depth] --> sb_writers#4

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
rlock(sb_writers#4);
lock(&ovl_i_mutex_dir_key[depth]);
lock(sb_writers#4);
lock(&p->lock);

*** DEADLOCK ***

1 lock held by syz-executor424/7758:
#0: ffff88814cd7a418 (sb_writers#4){.+.+}-{0:0}, at: do_sendfile+0x607/0x1000 fs/read_write.c:1253

stack backtrace:
CPU: 0 PID: 7758 Comm: syz-executor424 Not tainted 6.7.0-rc6-syzkaller-00010-g2cf4f94d8e86 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x1e7/0x2d0 lib/dump_stack.c:106
check_noncircular+0x366/0x490 kernel/locking/lockdep.c:2187
check_prev_add kernel/locking/lockdep.c:3134 [inline]
check_prevs_add kernel/locking/lockdep.c:3253 [inline]
validate_chain+0x1909/0x5ab0 kernel/locking/lockdep.c:3869
__lock_acquire+0x1345/0x1fd0 kernel/locking/lockdep.c:5137
lock_acquire+0x1e3/0x530 kernel/locking/lockdep.c:5754
__mutex_lock_common kernel/locking/mutex.c:603 [inline]
__mutex_lock+0x136/0xd60 kernel/locking/mutex.c:747
seq_read_iter+0xb2/0xd10 fs/seq_file.c:182
call_read_iter include/linux/fs.h:2014 [inline]
copy_splice_read+0x4c9/0x9c0 fs/splice.c:364
splice_direct_to_actor+0x2c4/0x9e0 fs/splice.c:1069
do_splice_direct+0x2ac/0x3f0 fs/splice.c:1194
do_sendfile+0x62c/0x1000 fs/read_write.c:1254
__do_sys_sendfile64 fs/read_write.c:1322 [inline]
__se_sys_sendfile64+0x17c/0x1e0 fs/read_write.c:1308
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0x45/0x110 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x63/0x6b
RIP: 0033:0x7fef41211d49
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 81 18 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fef411d2218 EFLAGS: 00000246 ORIG_RAX: 0000000000000028
RAX: ffffffffffffffda RBX: 00007fef4129c3e8 RCX: 00007fef41211d49
RDX: 0000000000000000 RSI: 0000000000000004 RDI: 0000000000000003
RBP: 00007fef4129c3e0 R08: 0000000000000000 R09: 0000000000000000
R10: 0001000000201007 R11: 0000000000000246 R12: 00007fef41269060
R13: 0030656c69662f2e R14: 6e6f3d6f6e69782c R15: 0079616c7265766f
</TASK>


---
If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.

2023-12-20 00:32:12

by syzbot

[permalink] [raw]
Subject: Re: [syzbot] [overlayfs?] possible deadlock in seq_read_iter (2)

syzbot has bisected this issue to:

commit 1e8c813b083c4122dfeaa5c3b11028331026e85d
Author: Christoph Hellwig <[email protected]>
Date: Wed May 31 12:55:32 2023 +0000

PM: hibernate: don't use early_lookup_bdev in resume_store

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=14b15592e80000
start commit: 2cf4f94d8e86 Merge tag 'scsi-fixes' of git://git.kernel.or..
git tree: upstream
final oops: https://syzkaller.appspot.com/x/report.txt?x=16b15592e80000
console output: https://syzkaller.appspot.com/x/log.txt?x=12b15592e80000
kernel config: https://syzkaller.appspot.com/x/.config?x=e5751b3a2226135d
dashboard link: https://syzkaller.appspot.com/bug?extid=da4f9f61f96525c62cc7
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=176a4f49e80000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=154aa8d6e80000

Reported-by: [email protected]
Fixes: 1e8c813b083c ("PM: hibernate: don't use early_lookup_bdev in resume_store")

For information about bisection process see: https://goo.gl/tpsmEJ#bisection

2023-12-20 03:54:17

by Amir Goldstein

[permalink] [raw]
Subject: Re: [syzbot] [overlayfs?] possible deadlock in seq_read_iter (2)

On Wed, Dec 20, 2023 at 2:30 AM syzbot
<[email protected]> wrote:
>
> syzbot has bisected this issue to:
>
> commit 1e8c813b083c4122dfeaa5c3b11028331026e85d
> Author: Christoph Hellwig <[email protected]>
> Date: Wed May 31 12:55:32 2023 +0000
>
> PM: hibernate: don't use early_lookup_bdev in resume_store
>
> bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=14b15592e80000
> start commit: 2cf4f94d8e86 Merge tag 'scsi-fixes' of git://git.kernel.or..
> git tree: upstream
> final oops: https://syzkaller.appspot.com/x/report.txt?x=16b15592e80000
> console output: https://syzkaller.appspot.com/x/log.txt?x=12b15592e80000
> kernel config: https://syzkaller.appspot.com/x/.config?x=e5751b3a2226135d
> dashboard link: https://syzkaller.appspot.com/bug?extid=da4f9f61f96525c62cc7
> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=176a4f49e80000
> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=154aa8d6e80000
>
> Reported-by: [email protected]
> Fixes: 1e8c813b083c ("PM: hibernate: don't use early_lookup_bdev in resume_store")
>

I'm not sure this bisection is reliable, so I wouldn't use this Fixes tag.
The reproducer may be a little flakey.

Anyway, this is just one of many problems, real or false positives,
related to the unholy dependency which sendfile() creates between
locks on two different filesystems.

I think those changes that are queued for 6.8 are going so fix this class
of problems:

#syz test: https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git vfs.rw

Thanks,
Amir.

2023-12-20 04:13:16

by syzbot

[permalink] [raw]
Subject: Re: [syzbot] [overlayfs?] possible deadlock in seq_read_iter (2)

Hello,

syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Reported-and-tested-by: [email protected]

Tested on:

commit: d9e5d310 fsnotify: optionally pass access range in fil..
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git vfs.rw
console output: https://syzkaller.appspot.com/x/log.txt?x=127b3016e80000
kernel config: https://syzkaller.appspot.com/x/.config?x=a5300d21645bcd09
dashboard link: https://syzkaller.appspot.com/bug?extid=da4f9f61f96525c62cc7
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40

Note: no patches were applied.
Note: testing is done by a robot and is best-effort only.

2024-01-27 02:29:28

by syzbot

[permalink] [raw]
Subject: Re: [syzbot] [overlayfs?] possible deadlock in seq_read_iter (2)

syzbot suspects this issue was fixed by commit:

commit da40448ce4eb4de18eb7b0db61dddece32677939
Author: Amir Goldstein <[email protected]>
Date: Thu Nov 30 14:16:23 2023 +0000

fs: move file_start_write() into direct_splice_actor()

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=105aa1a0180000
start commit: 2cf4f94d8e86 Merge tag 'scsi-fixes' of git://git.kernel.or..
git tree: upstream
kernel config: https://syzkaller.appspot.com/x/.config?x=e5751b3a2226135d
dashboard link: https://syzkaller.appspot.com/bug?extid=da4f9f61f96525c62cc7
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=176a4f49e80000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=154aa8d6e80000

If the result looks correct, please mark the issue as fixed by replying with:

#syz fix: fs: move file_start_write() into direct_splice_actor()

For information about bisection process see: https://goo.gl/tpsmEJ#bisection

2024-01-27 11:46:44

by Hillf Danton

[permalink] [raw]
Subject: Re: [syzbot] [overlayfs?] possible deadlock in seq_read_iter (2)

On Tue, 19 Dec 2023 11:43:27 -0800
> syzbot has found a reproducer for the following issue on:
>
> HEAD commit: 2cf4f94d8e86 Merge tag 'scsi-fixes' of git://git.kernel.or..
> git tree: upstream
> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=154aa8d6e80000

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 2cf4f94d8e86

--- x/fs/namei.c
+++ y/fs/namei.c
@@ -3533,6 +3533,8 @@ static const char *open_last_lookups(str

if (open_flag & (O_CREAT | O_TRUNC | O_WRONLY | O_RDWR)) {
got_write = !mnt_want_write(nd->path.mnt);
+ if (!got_write && (open_flag & O_CREAT))
+ return ERR_PTR(-EISDIR);
/*
* do _not_ fail yet - we might not need that or fail with
* a different error; let lookup_open() decide; we'll be
--

2024-01-27 11:57:11

by syzbot

[permalink] [raw]
Subject: Re: [syzbot] [overlayfs?] possible deadlock in seq_read_iter (2)

Hello,

syzbot tried to test the proposed patch but the build/boot failed:

failed to checkout kernel repo https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git on commit 2cf4f94d8e86: failed to run ["git" "fetch" "--force" "--tags" "4d52a57a3858a6eee0d0b25cc3a0c9533f747d8f" "2cf4f94d8e86"]: exit status 128
fatal: couldn't find remote ref 2cf4f94d8e86



Tested on:

commit: [unknown
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 2cf4f94d8e86
kernel config: https://syzkaller.appspot.com/x/.config?x=e5751b3a2226135d
dashboard link: https://syzkaller.appspot.com/bug?extid=da4f9f61f96525c62cc7
compiler:
patch: https://syzkaller.appspot.com/x/patch.diff?x=102f1930180000


2024-01-28 07:54:56

by Hillf Danton

[permalink] [raw]
Subject: Re: [syzbot] [overlayfs?] possible deadlock in seq_read_iter (2)

On Tue, 19 Dec 2023 11:43:27 -0800
> syzbot has found a reproducer for the following issue on:
>
> HEAD commit: 2cf4f94d8e86 Merge tag 'scsi-fixes' of git://git.kernel.or..
> git tree: upstream
> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=154aa8d6e80000

#syz test https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master

2024-01-28 08:24:18

by syzbot

[permalink] [raw]
Subject: Re: [syzbot] [overlayfs?] possible deadlock in seq_read_iter (2)

Hello,

syzbot has tested the proposed patch and the reproducer did not trigger any issue:

Reported-and-tested-by: [email protected]

Tested on:

commit: 8a696a29 Merge tag 'platform-drivers-x86-v6.8-2' of gi..
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master
console output: https://syzkaller.appspot.com/x/log.txt?x=1516855fe80000
kernel config: https://syzkaller.appspot.com/x/.config?x=5057c9bffe24fc70
dashboard link: https://syzkaller.appspot.com/bug?extid=da4f9f61f96525c62cc7
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40

Note: no patches were applied.
Note: testing is done by a robot and is best-effort only.

2024-01-28 10:12:25

by Amir Goldstein

[permalink] [raw]
Subject: Re: [syzbot] [overlayfs?] possible deadlock in seq_read_iter (2)

On Sat, Jan 27, 2024 at 4:28 AM syzbot
<[email protected]> wrote:
>
> syzbot suspects this issue was fixed by commit:
>
> commit da40448ce4eb4de18eb7b0db61dddece32677939
> Author: Amir Goldstein <[email protected]>
> Date: Thu Nov 30 14:16:23 2023 +0000
>
> fs: move file_start_write() into direct_splice_actor()
>
> bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=105aa1a0180000
> start commit: 2cf4f94d8e86 Merge tag 'scsi-fixes' of git://git.kernel.or..
> git tree: upstream
> kernel config: https://syzkaller.appspot.com/x/.config?x=e5751b3a2226135d
> dashboard link: https://syzkaller.appspot.com/bug?extid=da4f9f61f96525c62cc7
> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=176a4f49e80000
> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=154aa8d6e80000
>
> If the result looks correct, please mark the issue as fixed by replying with:
>
> #syz fix: fs: move file_start_write() into direct_splice_actor()

Yes. I already wrote that this work is going to fix this repro
as well as other potential deadlocks.

#syz fix: fs: move file_start_write() into direct_splice_actor()

Thanks,
Amir.

2024-01-28 22:00:19

by Al Viro

[permalink] [raw]
Subject: Re: [syzbot] [overlayfs?] possible deadlock in seq_read_iter (2)

On Sat, Jan 27, 2024 at 07:46:10PM +0800, Hillf Danton wrote:
> On Tue, 19 Dec 2023 11:43:27 -0800
> > syzbot has found a reproducer for the following issue on:
> >
> > HEAD commit: 2cf4f94d8e86 Merge tag 'scsi-fixes' of git://git.kernel.or..
> > git tree: upstream
> > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=154aa8d6e80000
>
> #syz test https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 2cf4f94d8e86
>
> --- x/fs/namei.c
> +++ y/fs/namei.c
> @@ -3533,6 +3533,8 @@ static const char *open_last_lookups(str
>
> if (open_flag & (O_CREAT | O_TRUNC | O_WRONLY | O_RDWR)) {
> got_write = !mnt_want_write(nd->path.mnt);
> + if (!got_write && (open_flag & O_CREAT))
> + return ERR_PTR(-EISDIR);

NAK.

Please, RTFComment just below your addition. Besides, EISDIR is
obviously bogus in a lot of cases, starting with attempting to
create a new file on a read-only filesystem. Surely
echo bugger > /mnt/cdrom/no_such_file_there
should *not* fail with "no_such_file_there: Is a directory"?

2024-01-29 05:08:12

by Hillf Danton

[permalink] [raw]
Subject: Re: [syzbot] [overlayfs?] possible deadlock in seq_read_iter (2)

On Sun, 28 Jan 2024 21:43:35 +0000 Al Viro <[email protected]>
> On Sat, Jan 27, 2024 at 07:46:10PM +0800, Hillf Danton wrote:
> > On Tue, 19 Dec 2023 11:43:27 -0800
> > > syzbot has found a reproducer for the following issue on:
> > >
> > > HEAD commit: 2cf4f94d8e86 Merge tag 'scsi-fixes' of git://git.kernel.or..
> > > git tree: upstream
> > > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=154aa8d6e80000
> >
> > #syz test https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 2cf4f94d8e86
> >
> > --- x/fs/namei.c
> > +++ y/fs/namei.c
> > @@ -3533,6 +3533,8 @@ static const char *open_last_lookups(str
> >
> > if (open_flag & (O_CREAT | O_TRUNC | O_WRONLY | O_RDWR)) {
> > got_write = !mnt_want_write(nd->path.mnt);
> > + if (!got_write && (open_flag & O_CREAT))
> > + return ERR_PTR(-EISDIR);
>
> NAK.

Thanks for looking at it, the AV legend.
>
> Please, RTFComment just below your addition.

That is a simple debug patch to test why mnt_want_write() is needed in
ovl_create_object() as per the syzbot report [1], given the locking
order in open_last_lookups() in case of O_CREAT.

mnt_want_write();
inode_lock();

> Besides, EISDIR is
> obviously bogus in a lot of cases, starting with attempting to
> create a new file on a read-only filesystem.

EISDIR should have been replaced with EDEADLOCK.

-> #3
(sb_writers#4){.+.+}-{0:0}:
lock_acquire+0x1e3/0x530 kernel/locking/lockdep.c:5754
percpu_down_read include/linux/percpu-rwsem.h:51 [inline]
__sb_start_write include/linux/fs.h:1635 [inline]
sb_start_write+0x4d/0x1c0 include/linux/fs.h:1710
mnt_want_write+0x3f/0x90 fs/namespace.c:404
ovl_create_object+0x13b/0x360 fs/overlayfs/dir.c:629
lookup_open fs/namei.c:3477 [inline]
open_last_lookups fs/namei.c:3546 [inline]
path_openat+0x13fa/0x3290 fs/namei.c:3776
do_filp_open+0x234/0x490 fs/namei.c:3809
do_sys_openat2+0x13e/0x1d0 fs/open.c:1437
do_sys_open fs/open.c:1452 [inline]
__do_sys_open fs/open.c:1460 [inline]
__se_sys_open fs/open.c:1456 [inline]
__x64_sys_open+0x225/0x270 fs/open.c:1456
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0x45/0x110 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x63/0x6b

[1] https://lore.kernel.org/lkml/[email protected]/