2024-06-03 03:50:30

by syzbot

[permalink] [raw]
Subject: [syzbot] [ext4?] INFO: task hung in vfs_rmdir (2)

Hello,

syzbot found the following issue on:

HEAD commit: 4a4be1ad3a6e Revert "vfs: Delete the associated dentry whe..
git tree: upstream
console+strace: https://syzkaller.appspot.com/x/log.txt?x=1466269a980000
kernel config: https://syzkaller.appspot.com/x/.config?x=47d282ddffae809f
dashboard link: https://syzkaller.appspot.com/bug?extid=42986aeeddfd7ed93c8b
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=12a5b3ec980000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=103820f2980000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/91325320f37c/disk-4a4be1ad.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/49af253b674e/vmlinux-4a4be1ad.xz
kernel image: https://storage.googleapis.com/syzbot-assets/04d26ea378d5/bzImage-4a4be1ad.xz
mounted in repro: https://storage.googleapis.com/syzbot-assets/2643c2ec211c/mount_0.gz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: [email protected]

INFO: task syz-executor150:5089 blocked for more than 143 seconds.
Not tainted 6.10.0-rc1-syzkaller-00027-g4a4be1ad3a6e #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor150 state:D stack:24224 pid:5089 tgid:5087 ppid:5086 flags:0x00004006
Call Trace:
<TASK>
context_switch kernel/sched/core.c:5408 [inline]
__schedule+0x1796/0x49d0 kernel/sched/core.c:6745
__schedule_loop kernel/sched/core.c:6822 [inline]
schedule+0x14b/0x320 kernel/sched/core.c:6837
schedule_preempt_disabled+0x13/0x30 kernel/sched/core.c:6894
rwsem_down_write_slowpath+0xeeb/0x13b0 kernel/locking/rwsem.c:1178
__down_write_common+0x1af/0x200 kernel/locking/rwsem.c:1306
inode_lock include/linux/fs.h:791 [inline]
vfs_rmdir+0x101/0x510 fs/namei.c:4203
do_rmdir+0x3b5/0x580 fs/namei.c:4273
__do_sys_rmdir fs/namei.c:4292 [inline]
__se_sys_rmdir fs/namei.c:4290 [inline]
__x64_sys_rmdir+0x49/0x60 fs/namei.c:4290
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f9b91eaed89
RSP: 002b:00007f9b91e65168 EFLAGS: 00000246 ORIG_RAX: 0000000000000054
RAX: ffffffffffffffda RBX: 00007f9b91f375e8 RCX: 00007f9b91eaed89
RDX: ffffffffffffffb0 RSI: e0f7bef392ce73bd RDI: 0000000020000180
RBP: 00007f9b91f375e0 R08: 00007f9b91e656c0 R09: 0000000000000000
R10: 00007f9b91e656c0 R11: 0000000000000246 R12: 00007f9b91f375ec
R13: 0000000000000006 R14: 00007fff5a763930 R15: 00007fff5a763a18
</TASK>

Showing all locks held in the system:
1 lock held by khungtaskd/30:
#0: ffffffff8e333f60 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:329 [inline]
#0: ffffffff8e333f60 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:781 [inline]
#0: ffffffff8e333f60 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x55/0x2a0 kernel/locking/lockdep.c:6614
2 locks held by getty/4840:
#0: ffff88802afc40a0 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x25/0x70 drivers/tty/tty_ldisc.c:243
#1: ffffc90002f0e2f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0x6b5/0x1e10 drivers/tty/n_tty.c:2201
3 locks held by syz-executor150/5089:
#0: ffff88807cf1a420 (sb_writers#9){.+.+}-{0:0}, at: mnt_want_write+0x3f/0x90 fs/namespace.c:409
#1: ffff88807a2d9650 (&sb->s_type->i_mutex_key#14/1){+.+.}-{3:3}, at: inode_lock_nested include/linux/fs.h:826 [inline]
#1: ffff88807a2d9650 (&sb->s_type->i_mutex_key#14/1){+.+.}-{3:3}, at: do_rmdir+0x263/0x580 fs/namei.c:4261
#2: ffff88807a2d9650 (&sb->s_type->i_mutex_key#15){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:791 [inline]
#2: ffff88807a2d9650 (&sb->s_type->i_mutex_key#15){+.+.}-{3:3}, at: vfs_rmdir+0x101/0x510 fs/namei.c:4203

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 30 Comm: khungtaskd Not tainted 6.10.0-rc1-syzkaller-00027-g4a4be1ad3a6e #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x241/0x360 lib/dump_stack.c:114
nmi_cpu_backtrace+0x49c/0x4d0 lib/nmi_backtrace.c:113
nmi_trigger_cpumask_backtrace+0x198/0x320 lib/nmi_backtrace.c:62
trigger_all_cpu_backtrace include/linux/nmi.h:162 [inline]
check_hung_uninterruptible_tasks kernel/hung_task.c:223 [inline]
watchdog+0xfde/0x1020 kernel/hung_task.c:379
kthread+0x2f0/0x390 kernel/kthread.c:389
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
</TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 1093 Comm: kworker/u8:7 Not tainted 6.10.0-rc1-syzkaller-00027-g4a4be1ad3a6e #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:hlock_class kernel/locking/lockdep.c:228 [inline]
RIP: 0010:check_wait_context kernel/locking/lockdep.c:4798 [inline]
RIP: 0010:__lock_acquire+0x876/0x1fd0 kernel/locking/lockdep.c:5087
Code: 8b 5d 00 81 e3 ff 1f 00 00 48 89 d8 48 c1 e8 06 48 8d 3c c5 80 15 f7 92 be 08 00 00 00 e8 c2 2b 86 00 48 0f a3 1d ca b2 84 11 <73> 1a 48 69 c3 c8 00 00 00 48 8d 98 80 74 c5 92 48 ba 00 00 00 00
RSP: 0018:ffffc900047af3d0 EFLAGS: 00000057
RAX: 0000000000000001 RBX: 0000000000000021 RCX: ffffffff817262ae
RDX: 0000000000000000 RSI: 0000000000000008 RDI: ffffffff92f71580
RBP: 0000000000000003 R08: ffffffff92f71587 R09: 1ffffffff25ee2b0
R10: dffffc0000000000 R11: fffffbfff25ee2b1 R12: 0000000000000005
R13: ffff8880223f8bc8 R14: 0000000000000005 R15: ffff8880223f8bc8
FS: 0000000000000000(0000) GS:ffff8880b9500000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055e370ade680 CR3: 000000000e132000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<NMI>
</NMI>
<TASK>
lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5754
__raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
_raw_spin_lock+0x2e/0x40 kernel/locking/spinlock.c:154
spin_lock include/linux/spinlock.h:351 [inline]
__pte_offset_map_lock+0x1ba/0x300 mm/pgtable-generic.c:375
get_locked_pte include/linux/mm.h:2744 [inline]
__text_poke+0x2c5/0xd30 arch/x86/kernel/alternative.c:1883
text_poke arch/x86/kernel/alternative.c:1968 [inline]
text_poke_bp_batch+0x8cd/0xb30 arch/x86/kernel/alternative.c:2357
text_poke_flush arch/x86/kernel/alternative.c:2470 [inline]
text_poke_finish+0x30/0x50 arch/x86/kernel/alternative.c:2477
arch_jump_label_transform_apply+0x1c/0x30 arch/x86/kernel/jump_label.c:146
static_key_enable_cpuslocked+0x136/0x260 kernel/jump_label.c:205
static_key_enable+0x1a/0x20 kernel/jump_label.c:218
toggle_allocation_gate+0xb5/0x250 mm/kfence/core.c:826
process_one_work kernel/workqueue.c:3231 [inline]
process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3312
worker_thread+0x86d/0xd70 kernel/workqueue.c:3393
kthread+0x2f0/0x390 kernel/kthread.c:389
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
</TASK>
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.739 msecs


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at [email protected].

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.

If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup


2024-06-03 03:57:05

by Al Viro

[permalink] [raw]
Subject: Re: [syzbot] [ext4?] INFO: task hung in vfs_rmdir (2)

On Sun, Jun 02, 2024 at 08:50:18PM -0700, syzbot wrote:

> If you want syzbot to run the reproducer, reply with:

#syz test: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6 v6.9

2024-06-03 04:14:11

by syzbot

[permalink] [raw]
Subject: Re: [syzbot] [ext4?] INFO: task hung in vfs_rmdir (2)

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
INFO: task hung in vfs_rmdir

INFO: task syz-executor:5934 blocked for more than 143 seconds.
Not tainted 6.9.0-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor state:D stack:22736 pid:5934 tgid:5932 ppid:5638 flags:0x00000006
Call Trace:
<TASK>
context_switch kernel/sched/core.c:5409 [inline]
__schedule+0x1796/0x4a00 kernel/sched/core.c:6746
__schedule_loop kernel/sched/core.c:6823 [inline]
schedule+0x14b/0x320 kernel/sched/core.c:6838
schedule_preempt_disabled+0x13/0x30 kernel/sched/core.c:6895
rwsem_down_write_slowpath+0xeeb/0x13b0 kernel/locking/rwsem.c:1178
__down_write_common+0x1af/0x200 kernel/locking/rwsem.c:1306
inode_lock include/linux/fs.h:795 [inline]
vfs_rmdir+0x101/0x4c0 fs/namei.c:4195
do_rmdir+0x3b5/0x580 fs/namei.c:4265
__do_sys_rmdir fs/namei.c:4284 [inline]
__se_sys_rmdir fs/namei.c:4282 [inline]
__x64_sys_rmdir+0x49/0x60 fs/namei.c:4282
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xf5/0x240 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7efe6f27cee9
RSP: 002b:00007efe7003c0c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000054
RAX: ffffffffffffffda RBX: 00007efe6f3b3fa0 RCX: 00007efe6f27cee9
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000020000180
RBP: 00007efe6f2c947f R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 000000000000000b R14: 00007efe6f3b3fa0 R15: 00007ffcf06a10f8
</TASK>

Showing all locks held in the system:
1 lock held by khungtaskd/29:
#0: ffffffff8e334da0 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:329 [inline]
#0: ffffffff8e334da0 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:781 [inline]
#0: ffffffff8e334da0 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x55/0x2a0 kernel/locking/lockdep.c:6614
2 locks held by getty/4827:
#0: ffff88802abe60a0 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x25/0x70 drivers/tty/tty_ldisc.c:243
#1: ffffc90002f162f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0x6b5/0x1e10 drivers/tty/n_tty.c:2201
3 locks held by syz-executor/5934:
#0: ffff888020866420 (sb_writers#12){.+.+}-{0:0}, at: mnt_want_write+0x3f/0x90 fs/namespace.c:409
#1: ffff88807f131650 (&sb->s_type->i_mutex_key#20/1){+.+.}-{3:3}, at: inode_lock_nested include/linux/fs.h:830 [inline]
#1: ffff88807f131650 (&sb->s_type->i_mutex_key#20/1){+.+.}-{3:3}, at: do_rmdir+0x263/0x580 fs/namei.c:4253
#2: ffff88807f131650 (&sb->s_type->i_mutex_key#21){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:795 [inline]
#2: ffff88807f131650 (&sb->s_type->i_mutex_key#21){+.+.}-{3:3}, at: vfs_rmdir+0x101/0x4c0 fs/namei.c:4195
4 locks held by syz-executor/5946:

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 29 Comm: khungtaskd Not tainted 6.9.0-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x241/0x360 lib/dump_stack.c:114
nmi_cpu_backtrace+0x49c/0x4d0 lib/nmi_backtrace.c:113
nmi_trigger_cpumask_backtrace+0x198/0x320 lib/nmi_backtrace.c:62
trigger_all_cpu_backtrace include/linux/nmi.h:160 [inline]
check_hung_uninterruptible_tasks kernel/hung_task.c:223 [inline]
watchdog+0xfde/0x1020 kernel/hung_task.c:380
kthread+0x2f0/0x390 kernel/kthread.c:388
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
</TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 14944 Comm: syz-executor Not tainted 6.9.0-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024
RIP: 0010:ext4_ext_map_blocks+0x7c7/0x77b0 fs/ext4/extents.c:4236
Code: 89 ee e8 1c 4d 46 ff 4d 89 fc 45 29 ef 0f 86 c8 02 00 00 e8 4b 4b 46 ff 45 89 ec 4c 8b 74 24 20 e9 07 21 00 00 48 8b 44 24 70 <42> 0f b6 04 38 84 c0 0f 85 2f 4c 00 00 48 8b 44 24 18 44 8b 30 48
RSP: 0018:ffffc90003ef74c0 EFLAGS: 00000202
RAX: 1ffff920007def39 RBX: 0000000000000001 RCX: ffff88801166da00
RDX: ffff88801166da00 RSI: 0000000000000001 RDI: 0000000000000000
RBP: ffffc90003ef77d0 R08: ffffffff824fbb51 R09: 0000000000000000
R10: ffff888028cbf110 R11: ffffed1005197e24 R12: 1ffff920007deee2
R13: 0000000000000000 R14: 0000000000000000 R15: dffffc0000000000
FS: 00007ff4d35a56c0(0000) GS:ffff8880b9500000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 0000000026118000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<NMI>
</NMI>
<TASK>
ext4_map_blocks+0xa5e/0x1d20 fs/ext4/inode.c:623
ext4_getblk+0x1fa/0x850 fs/ext4/inode.c:833
ext4_bread+0x2e/0x180 fs/ext4/inode.c:889
ext4_append+0x327/0x5c0 fs/ext4/namei.c:83
ext4_init_new_dir+0x33e/0xa30 fs/ext4/namei.c:2977
ext4_mkdir+0x4f7/0xcf0 fs/ext4/namei.c:3023
vfs_mkdir+0x2f9/0x4b0 fs/namei.c:4123
do_mkdirat+0x264/0x3a0 fs/namei.c:4146
__do_sys_mkdirat fs/namei.c:4161 [inline]
__se_sys_mkdirat fs/namei.c:4159 [inline]
__x64_sys_mkdirat+0x89/0xa0 fs/namei.c:4159
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xf5/0x240 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7ff4d287b9e7
Code: 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 b8 02 01 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ff4d35a4ef8 EFLAGS: 00000246 ORIG_RAX: 0000000000000102
RAX: ffffffffffffffda RBX: 00007ff4d35a4f80 RCX: 00007ff4d287b9e7
RDX: 00000000000001ff RSI: 0000000020000180 RDI: 00000000ffffff9c
RBP: 0000000020000100 R08: 00000000200001c0 R09: 0000000000000000
R10: 0000000020000100 R11: 0000000000000246 R12: 0000000020000180
R13: 00007ff4d35a4f40 R14: 0000000000000000 R15: 0000000000000000
</TASK>


Tested on:

commit: a38297e3 Linux 6.9
git tree: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6 v6.9
console output: https://syzkaller.appspot.com/x/log.txt?x=117bca16980000
kernel config: https://syzkaller.appspot.com/x/.config?x=df13071aee1d0001
dashboard link: https://syzkaller.appspot.com/bug?extid=42986aeeddfd7ed93c8b
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40

Note: no patches were applied.

2024-06-03 04:21:30

by Al Viro

[permalink] [raw]
Subject: Re: [syzbot] [ext4?] INFO: task hung in vfs_rmdir (2)

On Mon, Jun 03, 2024 at 04:56:49AM +0100, Al Viro wrote:
> On Sun, Jun 02, 2024 at 08:50:18PM -0700, syzbot wrote:
>
> > If you want syzbot to run the reproducer, reply with:
>

#syz test: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6 v5.0

2024-06-03 04:22:31

by Al Viro

[permalink] [raw]
Subject: Re: [syzbot] [ext4?] INFO: task hung in vfs_rmdir (2)

On Mon, Jun 03, 2024 at 05:21:16AM +0100, Al Viro wrote:
> On Mon, Jun 03, 2024 at 04:56:49AM +0100, Al Viro wrote:
> > On Sun, Jun 02, 2024 at 08:50:18PM -0700, syzbot wrote:
> >
> > > If you want syzbot to run the reproducer, reply with:
> >
>
> #syz test: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6 v5.0

[IOW, let's see how far back does that go]

2024-06-03 11:38:02

by Jan Kara

[permalink] [raw]
Subject: Re: [syzbot] [ext4?] INFO: task hung in vfs_rmdir (2)

On Sun 02-06-24 20:50:18, syzbot wrote:
> syzbot found the following issue on:
>
> HEAD commit: 4a4be1ad3a6e Revert "vfs: Delete the associated dentry whe..
> git tree: upstream
> console+strace: https://syzkaller.appspot.com/x/log.txt?x=1466269a980000
> kernel config: https://syzkaller.appspot.com/x/.config?x=47d282ddffae809f
> dashboard link: https://syzkaller.appspot.com/bug?extid=42986aeeddfd7ed93c8b
> compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=12a5b3ec980000
> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=103820f2980000

Based on strace (and also the reproducer) this looks purely in exfat
driver. So:

#syz set subsystems: exfat

Honza
--
Jan Kara <[email protected]>
SUSE Labs, CR