2020-04-21 10:48:38

by syzbot

[permalink] [raw]
Subject: linux-next test error: WARNING: suspicious RCU usage in ipmr_device_event

Hello,

syzbot found the following crash on:

HEAD commit: 39a314cd Add linux-next specific files for 20200421
git tree: linux-next
console output: https://syzkaller.appspot.com/x/log.txt?x=127ede73e00000
kernel config: https://syzkaller.appspot.com/x/.config?x=8ef80c3f5d43f5bd
dashboard link: https://syzkaller.appspot.com/bug?extid=21f82f61c24a7295edf5
compiler: gcc (GCC) 9.0.0 20181231 (experimental)

Unfortunately, I don't have any reproducer for this crash yet.

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: [email protected]

tipc: TX() has been purged, node left!
=============================
WARNING: suspicious RCU usage
5.7.0-rc2-next-20200421-syzkaller #0 Not tainted
-----------------------------
net/ipv4/ipmr.c:1757 RCU-list traversed in non-reader section!!

other info that might help us debug this:


rcu_scheduler_active = 2, debug_locks = 1
4 locks held by kworker/u4:0/7:
#0: ffff8880a9772138 ((wq_completion)netns){+.+.}-{0:0}, at: __write_once_size include/linux/compiler.h:250 [inline]
#0: ffff8880a9772138 ((wq_completion)netns){+.+.}-{0:0}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
#0: ffff8880a9772138 ((wq_completion)netns){+.+.}-{0:0}, at: atomic64_set include/asm-generic/atomic-instrumented.h:856 [inline]
#0: ffff8880a9772138 ((wq_completion)netns){+.+.}-{0:0}, at: atomic_long_set include/asm-generic/atomic-long.h:41 [inline]
#0: ffff8880a9772138 ((wq_completion)netns){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:615 [inline]
#0: ffff8880a9772138 ((wq_completion)netns){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:642 [inline]
#0: ffff8880a9772138 ((wq_completion)netns){+.+.}-{0:0}, at: process_one_work+0x844/0x16a0 kernel/workqueue.c:2239
#1: ffffc90000cdfdc0 (net_cleanup_work){+.+.}-{0:0}, at: process_one_work+0x878/0x16a0 kernel/workqueue.c:2243
#2: ffffffff8a5a2b70 (pernet_ops_rwsem){++++}-{3:3}, at: cleanup_net+0x9b/0xa50 net/core/net_namespace.c:565
#3: ffffffff8a5aeae8 (rtnl_mutex){+.+.}-{3:3}, at: ip6gre_exit_batch_net+0x88/0x700 net/ipv6/ip6_gre.c:1602

stack backtrace:
CPU: 0 PID: 7 Comm: kworker/u4:0 Not tainted 5.7.0-rc2-next-20200421-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: netns cleanup_net
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x188/0x20d lib/dump_stack.c:118
ipmr_device_event+0x240/0x2b0 net/ipv4/ipmr.c:1757
notifier_call_chain+0xc0/0x230 kernel/notifier.c:83
call_netdevice_notifiers_info net/core/dev.c:1948 [inline]
call_netdevice_notifiers_info+0xb5/0x130 net/core/dev.c:1933
call_netdevice_notifiers_extack net/core/dev.c:1960 [inline]
call_netdevice_notifiers net/core/dev.c:1974 [inline]
rollback_registered_many+0x75c/0xe70 net/core/dev.c:8828
unregister_netdevice_many.part.0+0x16/0x1e0 net/core/dev.c:9993
unregister_netdevice_many+0x36/0x50 net/core/dev.c:9992
ip6gre_exit_batch_net+0x4e8/0x700 net/ipv6/ip6_gre.c:1605
ops_exit_list.isra.0+0x103/0x150 net/core/net_namespace.c:189
cleanup_net+0x511/0xa50 net/core/net_namespace.c:603
process_one_work+0x965/0x16a0 kernel/workqueue.c:2268
worker_thread+0x96/0xe20 kernel/workqueue.c:2414
kthread+0x38b/0x470 kernel/kthread.c:274
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at [email protected].

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.


2020-04-21 10:54:44

by Dmitry Vyukov

[permalink] [raw]
Subject: Re: linux-next test error: WARNING: suspicious RCU usage in ipmr_device_event

On Tue, Apr 21, 2020 at 12:46 PM syzbot
<[email protected]> wrote:
>
> Hello,
>
> syzbot found the following crash on:
>
> HEAD commit: 39a314cd Add linux-next specific files for 20200421
> git tree: linux-next

+linux-next, Stephen for a new linux-next breakage

> console output: https://syzkaller.appspot.com/x/log.txt?x=127ede73e00000
> kernel config: https://syzkaller.appspot.com/x/.config?x=8ef80c3f5d43f5bd
> dashboard link: https://syzkaller.appspot.com/bug?extid=21f82f61c24a7295edf5
> compiler: gcc (GCC) 9.0.0 20181231 (experimental)
>
> Unfortunately, I don't have any reproducer for this crash yet.
>
> IMPORTANT: if you fix the bug, please add the following tag to the commit:
> Reported-by: [email protected]
>
> tipc: TX() has been purged, node left!
> =============================
> WARNING: suspicious RCU usage
> 5.7.0-rc2-next-20200421-syzkaller #0 Not tainted
> -----------------------------
> net/ipv4/ipmr.c:1757 RCU-list traversed in non-reader section!!
>
> other info that might help us debug this:
>
>
> rcu_scheduler_active = 2, debug_locks = 1
> 4 locks held by kworker/u4:0/7:
> #0: ffff8880a9772138 ((wq_completion)netns){+.+.}-{0:0}, at: __write_once_size include/linux/compiler.h:250 [inline]
> #0: ffff8880a9772138 ((wq_completion)netns){+.+.}-{0:0}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
> #0: ffff8880a9772138 ((wq_completion)netns){+.+.}-{0:0}, at: atomic64_set include/asm-generic/atomic-instrumented.h:856 [inline]
> #0: ffff8880a9772138 ((wq_completion)netns){+.+.}-{0:0}, at: atomic_long_set include/asm-generic/atomic-long.h:41 [inline]
> #0: ffff8880a9772138 ((wq_completion)netns){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:615 [inline]
> #0: ffff8880a9772138 ((wq_completion)netns){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:642 [inline]
> #0: ffff8880a9772138 ((wq_completion)netns){+.+.}-{0:0}, at: process_one_work+0x844/0x16a0 kernel/workqueue.c:2239
> #1: ffffc90000cdfdc0 (net_cleanup_work){+.+.}-{0:0}, at: process_one_work+0x878/0x16a0 kernel/workqueue.c:2243
> #2: ffffffff8a5a2b70 (pernet_ops_rwsem){++++}-{3:3}, at: cleanup_net+0x9b/0xa50 net/core/net_namespace.c:565
> #3: ffffffff8a5aeae8 (rtnl_mutex){+.+.}-{3:3}, at: ip6gre_exit_batch_net+0x88/0x700 net/ipv6/ip6_gre.c:1602
>
> stack backtrace:
> CPU: 0 PID: 7 Comm: kworker/u4:0 Not tainted 5.7.0-rc2-next-20200421-syzkaller #0
> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
> Workqueue: netns cleanup_net
> Call Trace:
> __dump_stack lib/dump_stack.c:77 [inline]
> dump_stack+0x188/0x20d lib/dump_stack.c:118
> ipmr_device_event+0x240/0x2b0 net/ipv4/ipmr.c:1757
> notifier_call_chain+0xc0/0x230 kernel/notifier.c:83
> call_netdevice_notifiers_info net/core/dev.c:1948 [inline]
> call_netdevice_notifiers_info+0xb5/0x130 net/core/dev.c:1933
> call_netdevice_notifiers_extack net/core/dev.c:1960 [inline]
> call_netdevice_notifiers net/core/dev.c:1974 [inline]
> rollback_registered_many+0x75c/0xe70 net/core/dev.c:8828
> unregister_netdevice_many.part.0+0x16/0x1e0 net/core/dev.c:9993
> unregister_netdevice_many+0x36/0x50 net/core/dev.c:9992
> ip6gre_exit_batch_net+0x4e8/0x700 net/ipv6/ip6_gre.c:1605
> ops_exit_list.isra.0+0x103/0x150 net/core/net_namespace.c:189
> cleanup_net+0x511/0xa50 net/core/net_namespace.c:603
> process_one_work+0x965/0x16a0 kernel/workqueue.c:2268
> worker_thread+0x96/0xe20 kernel/workqueue.c:2414
> kthread+0x38b/0x470 kernel/kthread.c:274
> ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352
>
>
> ---
> This bug is generated by a bot. It may contain errors.
> See https://goo.gl/tpsmEJ for more information about syzbot.
> syzbot engineers can be reached at [email protected].
>
> syzbot will keep track of this bug report. See:
> https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
>
> --
> You received this message because you are subscribed to the Google Groups "syzkaller-bugs" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
> To view this discussion on the web visit https://groups.google.com/d/msgid/syzkaller-bugs/0000000000004ede4505a3cab90e%40google.com.

2020-04-21 11:24:23

by Qian Cai

[permalink] [raw]
Subject: Re: linux-next test error: WARNING: suspicious RCU usage in ipmr_device_event



> On Apr 21, 2020, at 6:51 AM, Dmitry Vyukov <[email protected]> wrote:
>
> +linux-next, Stephen for a new linux-next breakage

I don’t know why you keep sending the same thing over and over again where I replied you just two days ago for the same thing.

2020-04-23 08:53:50

by Dmitry Vyukov

[permalink] [raw]
Subject: Re: linux-next test error: WARNING: suspicious RCU usage in ipmr_device_event

On Tue, Apr 21, 2020 at 1:21 PM Qian Cai <[email protected]> wrote:
> > On Apr 21, 2020, at 6:51 AM, Dmitry Vyukov <[email protected]> wrote:
> >
> > +linux-next, Stephen for a new linux-next breakage
>
> I don’t know why you keep sending the same thing over and over again where I replied you just two days ago for the same thing.

Well, formally this bug was never mailed. The other one is about
"WARNING: suspicious RCU usage in ovs_ct_exit". It just happened that
this "WARNING: suspicious RCU usage in ipmr_device_event" was appended
to that one (because kernel somehow failed to respect panic_on_warn?).
But warnings except the first one are ignored, frequently they are
induced and it's not possible to track 2 different bugs at the same
time in a reasonable way anyway.

So effectively this is the first occurance of this bug (presumably the
fix for ovs_ct_exit has reached linux-next). First occurrence is
always mailed.

Now let's tell syzbot that this one is fixed as well:

#syz fix: ipmr: Fix RCU list debugging warning