LinuxLists.cc - Possible DEADLOCK in rtnl

2017-06-21 09:22:31

Subject: Possible DEADLOCK in rtnl_lock(v4.1.40)

Hi:
I've got the following error report while fuzzing the kernel with
syzkaller on v4.1.40

Syzkaller hit 'possible deadlock in rtnl_lock' bug on commit .

The guilty file is: /home/river/git_new/linux-stable/net/core/rtnetlink.c.

======================================================
[ INFO: possible circular locking dependency detected ]
4.1.40 #4 Not tainted
-------------------------------------------------------
syz-executor1/4765 is trying to acquire lock:
(rtnl_mutex){+.+.+.}, at: [<ffffffff82734b62>] rtnl_lock+0x12/0x20
/home/river/git_new/linux-stable/net/core/rtnetlink.c:70

but task is already holding lock:
(sk_lock-AF_INET){+.+.+.}, at: [<ffffffff82848a75>] lock_sock
/home/river/git_new/linux-stable/include/net/sock.h:1497 [inline]
(sk_lock-AF_INET){+.+.+.}, at: [<ffffffff82848a75>]
do_ip_getsockopt.part.9+0xf5/0x1210
/home/river/git_new/linux-stable/net/ipv4/ip_sockglue.c:1270

which lock already depends on the new lock.

the existing dependency chain (in reverse order) is:

[<ffffffff811da5dd>] lock_acquire+0x13d/0x4d0
/home/river/git_new/linux-stable/kernel/locking/lockdep.c:3623
[<ffffffff826ab259>] lock_sock_nested+0xb9/0x110
/home/river/git_new/linux-stable/net/core/sock.c:2376
[<ffffffff8284ad8f>] lock_sock
/home/river/git_new/linux-stable/include/net/sock.h:1497 [inline]
[<ffffffff8284ad8f>] do_ip_setsockopt.isra.12+0x15f/0x24f0
/home/river/git_new/linux-stable/net/ipv4/ip_sockglue.c:622
[<ffffffff8284d14f>] ip_setsockopt+0x2f/0xb0
/home/river/git_new/linux-stable/net/ipv4/ip_sockglue.c:1200
[<ffffffff826a95a3>] sock_common_setsockopt+0x73/0xf0
/home/river/git_new/linux-stable/net/core/sock.c:2575
[<ffffffff826a6910>] SYSC_setsockopt
/home/river/git_new/linux-stable/net/socket.c:1761 [inline]
[<ffffffff826a6910>] SyS_setsockopt+0x130/0x200
/home/river/git_new/linux-stable/net/socket.c:1740
[<ffffffff82eb9fd7>] system_call_fastpath+0x12/0x6f

[<ffffffff811d6c91>] check_prev_add
/home/river/git_new/linux-stable/kernel/locking/lockdep.c:1853
[inline]
[<ffffffff811d6c91>] check_prevs_add
/home/river/git_new/linux-stable/kernel/locking/lockdep.c:1958
[inline]
[<ffffffff811d6c91>] validate_chain
/home/river/git_new/linux-stable/kernel/locking/lockdep.c:2144
[inline]
[<ffffffff811d6c91>] __lock_acquire+0x3551/0x51f0
/home/river/git_new/linux-stable/kernel/locking/lockdep.c:3205
[<ffffffff811da5dd>] lock_acquire+0x13d/0x4d0
/home/river/git_new/linux-stable/kernel/locking/lockdep.c:3623
[<ffffffff82eb0e50>] __mutex_lock_common
/home/river/git_new/linux-stable/kernel/locking/mutex.c:521 [inline]
[<ffffffff82eb0e50>] mutex_lock_nested+0xc0/0x9c0
/home/river/git_new/linux-stable/kernel/locking/mutex.c:620
[<ffffffff82734b62>] rtnl_lock+0x12/0x20
/home/river/git_new/linux-stable/net/core/rtnetlink.c:70
[<ffffffff8294598d>] ip_mc_msfget+0xdd/0x5b0
/home/river/git_new/linux-stable/net/ipv4/igmp.c:2208
[<ffffffff82848d18>] do_ip_getsockopt.part.9+0x398/0x1210
/home/river/git_new/linux-stable/net/ipv4/ip_sockglue.c:1394
[<ffffffff82849d8c>] do_ip_getsockopt
/home/river/git_new/linux-stable/net/ipv4/ip_sockglue.c:1262 [inline]
[<ffffffff82849d8c>] ip_getsockopt+0x8c/0x150
/home/river/git_new/linux-stable/net/ipv4/ip_sockglue.c:1490
[<ffffffff828688a8>] tcp_getsockopt+0x68/0xd0
/home/river/git_new/linux-stable/net/ipv4/tcp.c:2848
[<ffffffff826a9183>] sock_common_getsockopt+0x73/0xf0
/home/river/git_new/linux-stable/net/core/sock.c:2534
[<ffffffff826a6b07>] SYSC_getsockopt
/home/river/git_new/linux-stable/net/socket.c:1792 [inline]
[<ffffffff826a6b07>] SyS_getsockopt+0x127/0x200
/home/river/git_new/linux-stable/net/socket.c:1774
[<ffffffff82eb9fd7>] system_call_fastpath+0x12/0x6f

other info that might help us debug this:

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock(sk_lock-AF_INET);
lock(rtnl_mutex);
lock(sk_lock-AF_INET);
lock(rtnl_mutex);

*** DEADLOCK ***

1 lock held by syz-executor1/4765:
#0: (sk_lock-AF_INET){+.+.+.}, at: [<ffffffff82848a75>] lock_sock
/home/river/git_new/linux-stable/include/net/sock.h:1497 [inline]
#0: (sk_lock-AF_INET){+.+.+.}, at: [<ffffffff82848a75>]
do_ip_getsockopt.part.9+0xf5/0x1210
/home/river/git_new/linux-stable/net/ipv4/ip_sockglue.c:1270

stack backtrace:
CPU: 3 PID: 4765 Comm: syz-executor1 Not tainted 4.1.40 #4
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
Ubuntu-1.8.2-1ubuntu1 04/01/2014
ffffffff845cf6d0 ffff88003c7f7518 ffffffff82e9d411 ffffffff84586dd0
ffffffff84586dd0 ffff88003c7f7578 ffffffff811cfed8 0000000000000000
0000000000000000 0000000000000000 000000003c4a1b68 ffff88003c4a1b90
Call Trace:
[<ffffffff82e9d411>] __dump_stack
/home/river/git_new/linux-stable/lib/dump_stack.c:15 [inline]
[<ffffffff82e9d411>] dump_stack+0x68/0x92
/home/river/git_new/linux-stable/lib/dump_stack.c:51
[<ffffffff811cfed8>] print_circular_bug+0x2a8/0x370
/home/river/git_new/linux-stable/kernel/locking/lockdep.c:1226
[<ffffffff811d6c91>] check_prev_add
/home/river/git_new/linux-stable/kernel/locking/lockdep.c:1853
[inline]
[<ffffffff811d6c91>] check_prevs_add
/home/river/git_new/linux-stable/kernel/locking/lockdep.c:1958
[inline]
[<ffffffff811d6c91>] validate_chain
/home/river/git_new/linux-stable/kernel/locking/lockdep.c:2144
[inline]
[<ffffffff811d6c91>] __lock_acquire+0x3551/0x51f0
/home/river/git_new/linux-stable/kernel/locking/lockdep.c:3205
[<ffffffff811da5dd>] lock_acquire+0x13d/0x4d0
/home/river/git_new/linux-stable/kernel/locking/lockdep.c:3623
[<ffffffff82eb0e50>] __mutex_lock_common
/home/river/git_new/linux-stable/kernel/locking/mutex.c:521 [inline]
[<ffffffff82eb0e50>] mutex_lock_nested+0xc0/0x9c0
/home/river/git_new/linux-stable/kernel/locking/mutex.c:620
[<ffffffff82734b62>] rtnl_lock+0x12/0x20
/home/river/git_new/linux-stable/net/core/rtnetlink.c:70
[<ffffffff8294598d>] ip_mc_msfget+0xdd/0x5b0
/home/river/git_new/linux-stable/net/ipv4/igmp.c:2208
[<ffffffff82848d18>] do_ip_getsockopt.part.9+0x398/0x1210
/home/river/git_new/linux-stable/net/ipv4/ip_sockglue.c:1394
[<ffffffff82849d8c>] do_ip_getsockopt
/home/river/git_new/linux-stable/net/ipv4/ip_sockglue.c:1262 [inline]
[<ffffffff82849d8c>] ip_getsockopt+0x8c/0x150
/home/river/git_new/linux-stable/net/ipv4/ip_sockglue.c:1490
[<ffffffff828688a8>] tcp_getsockopt+0x68/0xd0
/home/river/git_new/linux-stable/net/ipv4/tcp.c:2848
[<ffffffff826a9183>] sock_common_getsockopt+0x73/0xf0
/home/river/git_new/linux-stable/net/core/sock.c:2534
[<ffffffff826a6b07>] SYSC_getsockopt
/home/river/git_new/linux-stable/net/socket.c:1792 [inline]
[<ffffffff826a6b07>] SyS_getsockopt+0x127/0x200
/home/river/git_new/linux-stable/net/socket.c:1774
[<ffffffff82eb9fd7>] system_call_fastpath+0x12/0x6f
audit: type=1326 audit(1497551764.596:719): auid=4294967295 uid=65534
gid=65534 ses=4294967295 subj=kernel pid=8788 comm="syz-executor0"
exe="/syz-executor0" sig=9 arch=c000003e syscall=202 compat=0
ip=0x451759 code=0x0
audit: type=1326 audit(1497551764.657:720): auid=4294967295 uid=65534
gid=65534 ses=4294967295 subj=kernel pid=8818 comm="syz-executor0"
exe="/syz-executor0" sig=9 arch=c000003e syscall=202 compat=0
ip=0x451759 code=0x0
audit: type=1326 audit(1497551765.271:721): auid=4294967295 uid=65534
gid=65534 ses=4294967295 subj=kernel pid=9250 comm="syz-executor1"
exe="/syz-executor1" sig=9 arch=c000003e syscall=202 compat=0
ip=0x451759 code=0x0
audit: type=1326 audit(1497551765.300:722): auid=4294967295 uid=65534
gid=65534 ses=4294967295 subj=kernel pid=9281 comm="syz-executor2"
exe="/syz-executor2" sig=9 arch=c000003e syscall=202 compat=0
ip=0x451759 code=0x0
audit: type=1326 audit(1497551765.333:723): auid=4294967295 uid=65534
gid=65534 ses=4294967295 subj=kernel pid=9297 comm="syz-executor1"
exe="/syz-executor1" sig=9 arch=c000003e syscall=202 compat=0
ip=0x451759 code=0x0
audit: type=1326 audit(1497551765.346:724): auid=4294967295 uid=65534
gid=65534 ses=4294967295 subj=kernel pid=9302 comm="syz-executor2"
exe="/syz-executor2" sig=9 arch=c000003e syscall=202 compat=0
ip=0x451759 code=0x0
audit: type=1326 audit(1497551768.077:725): auid=4294967295 uid=65534
gid=65534 ses=4294967295 subj=kernel pid=11336 comm="syz-executor1"
exe="/syz-executor1" sig=9 arch=c000003e syscall=202 compat=0
ip=0x451759 code=0x0
audit: type=1326 audit(1497551768.131:726): auid=4294967295 uid=65534
gid=65534 ses=4294967295 subj=kernel pid=11383 comm="syz-executor1"
exe="/syz-executor1" sig=9 arch=c000003e syscall=202 compat=0
ip=0x451759 code=0x0

Syzkaller reproducer:
# {Threaded:true Collide:true Repeat:true Procs:1 Sandbox:setuid
Fault:false FaultCall:-1 FaultNth:0 EnableTun:true UseTmpDir:true
HandleSegv:true WaitRepeat:true Debug:true Repro:false}
mmap(&(0x7f0000000000/0x6000)=nil, (0x6000), 0x3, 0x32, 0xffffffffffffffff, 0x0)
r0 = accept4$inet6(0xffffffffffffff9c, 0x0, &(0x7f0000002000-0x4)=0x0, 0x80800)
r1 = socket$icmp(0x2, 0x2, 0x1)
ppoll(&(0x7f0000000000)=[{r0, 0x0, 0x0}, {r1, 0x1408, 0x0}], 0x2,
&(0x7f0000001000-0x10)={0x0, 0x989680},
&(0x7f0000002000-0x8)={0x35ea}, 0x8)
fcntl$getownex(r1, 0x10, &(0x7f0000002000-0x3)={0x0, 0x0})
ioctl$SNDRV_TIMER_IOCTL_SELECT(0xffffffffffffffff, 0x40345410,
&(0x7f0000002000)={{0x3, 0x3, 0x1f, 0x1, 0x4}, [0x0, 0x0, 0x0, 0x0,
0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,
0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,
0x0]})
syz_open_dev$vcsn(&(0x7f0000005000-0xa)="2f6465762f7663732300", 0x6, 0x404c01)

Attachments:

config (96.29 kB)

2017-06-21 20:04:07

by Cong Wang

[permalink] [raw]

Subject: Re: Possible DEADLOCK in rtnl_lock(v4.1.40)

Hi,

On Wed, Jun 21, 2017 at 2:22 AM, Dison River <[email protected]> wrote:
> Hi:
> I've got the following error report while fuzzing the kernel with
> syzkaller on v4.1.40
>
>
> Syzkaller hit 'possible deadlock in rtnl_lock' bug on commit .
>
> The guilty file is: /home/river/git_new/linux-stable/net/core/rtnetlink.c.
>
>
> ======================================================
> [ INFO: possible circular locking dependency detected ]
> 4.1.40 #4 Not tainted
> -------------------------------------------------------
> syz-executor1/4765 is trying to acquire lock:
> (rtnl_mutex){+.+.+.}, at: [<ffffffff82734b62>] rtnl_lock+0x12/0x20
> /home/river/git_new/linux-stable/net/core/rtnetlink.c:70
>
> but task is already holding lock:
> (sk_lock-AF_INET){+.+.+.}, at: [<ffffffff82848a75>] lock_sock
> /home/river/git_new/linux-stable/include/net/sock.h:1497 [inline]
> (sk_lock-AF_INET){+.+.+.}, at: [<ffffffff82848a75>]
> do_ip_getsockopt.part.9+0xf5/0x1210
> /home/river/git_new/linux-stable/net/ipv4/ip_sockglue.c:1270
>
> which lock already depends on the new lock.
>
>
> the existing dependency chain (in reverse order) is:
>
> [<ffffffff811da5dd>] lock_acquire+0x13d/0x4d0
> /home/river/git_new/linux-stable/kernel/locking/lockdep.c:3623
> [<ffffffff826ab259>] lock_sock_nested+0xb9/0x110
> /home/river/git_new/linux-stable/net/core/sock.c:2376
> [<ffffffff8284ad8f>] lock_sock
> /home/river/git_new/linux-stable/include/net/sock.h:1497 [inline]
> [<ffffffff8284ad8f>] do_ip_setsockopt.isra.12+0x15f/0x24f0
> /home/river/git_new/linux-stable/net/ipv4/ip_sockglue.c:622
> [<ffffffff8284d14f>] ip_setsockopt+0x2f/0xb0
> /home/river/git_new/linux-stable/net/ipv4/ip_sockglue.c:1200
> [<ffffffff826a95a3>] sock_common_setsockopt+0x73/0xf0
> /home/river/git_new/linux-stable/net/core/sock.c:2575
> [<ffffffff826a6910>] SYSC_setsockopt
> /home/river/git_new/linux-stable/net/socket.c:1761 [inline]
> [<ffffffff826a6910>] SyS_setsockopt+0x130/0x200
> /home/river/git_new/linux-stable/net/socket.c:1740
> [<ffffffff82eb9fd7>] system_call_fastpath+0x12/0x6f
>
> [<ffffffff811d6c91>] check_prev_add
> /home/river/git_new/linux-stable/kernel/locking/lockdep.c:1853
> [inline]
> [<ffffffff811d6c91>] check_prevs_add
> /home/river/git_new/linux-stable/kernel/locking/lockdep.c:1958
> [inline]
> [<ffffffff811d6c91>] validate_chain
> /home/river/git_new/linux-stable/kernel/locking/lockdep.c:2144
> [inline]
> [<ffffffff811d6c91>] __lock_acquire+0x3551/0x51f0
> /home/river/git_new/linux-stable/kernel/locking/lockdep.c:3205
> [<ffffffff811da5dd>] lock_acquire+0x13d/0x4d0
> /home/river/git_new/linux-stable/kernel/locking/lockdep.c:3623
> [<ffffffff82eb0e50>] __mutex_lock_common
> /home/river/git_new/linux-stable/kernel/locking/mutex.c:521 [inline]
> [<ffffffff82eb0e50>] mutex_lock_nested+0xc0/0x9c0
> /home/river/git_new/linux-stable/kernel/locking/mutex.c:620
> [<ffffffff82734b62>] rtnl_lock+0x12/0x20
> /home/river/git_new/linux-stable/net/core/rtnetlink.c:70
> [<ffffffff8294598d>] ip_mc_msfget+0xdd/0x5b0

Do you have this fix in your kernel?

commit 87e9f0315952b0dd8b5e51ba04beda03efc009d9
Author: WANG Cong <[email protected]>
Date: Tue Nov 3 15:41:16 2015 -0800

ipv4: fix a potential deadlock in mcast getsockopt() path