Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Kuniyuki Iwashima <kuniyu@amazon.com>
To:     <harperchen1110@gmail.com>
CC:     <Jason@zx2c4.com>, <chentao.kernel@linux.alibaba.com>,
        <davem@davemloft.net>, <edumazet@google.com>,
        <harshit.m.mogalapalli@oracle.com>, <keescook@chromium.org>,
        <kuba@kernel.org>, <kuniyu@amazon.com>,
        <linux-kernel@vger.kernel.org>, <netdev@vger.kernel.org>,
        <pabeni@redhat.com>
Subject: Re: BUG: unable to handle kernel NULL pointer dereference in sk_diag_fill
Date:   Mon, 21 Nov 2022 15:10:28 -0800
Message-ID: <20221121231028.50834-1-kuniyu@amazon.com>
In-Reply-To: <CAO4mrfdvyjFpokhNsiwZiP-wpdSD0AStcJwfKcKQdAALQ9_2Qw@mail.gmail.com>
References: <CAO4mrfdvyjFpokhNsiwZiP-wpdSD0AStcJwfKcKQdAALQ9_2Qw@mail.gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Precedence: bulk

Hi Wei,

Thanks for reporting the issue.

From:   Wei Chen <harperchen1110@gmail.com>
Date:   Mon, 21 Nov 2022 21:34:50 +0800
> Dear Linux Developer,
> 
> Recently when using our tool to fuzz kernel, the following crash was triggered.
> 
> HEAD commit: 147307c69ba
> git tree: upstream
> compiler: clang 12.0.0
> console output:
> https://drive.google.com/file/d/1WZBgd9dodq3qWgqIXNNBHNEpayHthWr5/view?usp=share_link
> kernel config: https://drive.google.com/file/d/1NAf4S43d9VOKD52xbrqw-PUP1Mbj8z-S/view?usp=share_link
> 
> Unfortunately, I didn't have a reproducer for this crash. My manual
> check identifies that sk->sk_socket in sk_user_ns is null.

This gave me a hint.

I guess syzbot is dumping an AF_UNIX sk while close()ing it concurrently.
Then, there seems to be a minor race like

unix_release                  unix_diag_handler_dump  <-- called after refcnt reaches 0
  unix_release_sock              unix_diag_get_exact
                                   unix_lookup_by_ino
                                     sock_hold
    unix_remove_socket  <-- after this, sk cannot be seen in the hash table
    unix_remove_bsd_socket
    sock_orphan
      sk->sk_socket = NULL  <-- clear the sk->sk_socket
                                   sk_diag_fill
                                      sk_diag_dump_uid
                                        sk_user_ns
                                          sk->sk_socket->file->f_cred->user_ns  <-- NULL deref

It seems we need to call refcnt_inc_not_zero() instead of sock_hold() in
unix_lookup_by_ino(), so the fix would be like this.

After checking other paths (at least netlink_dump_start() path) and doing
some tests, I'll submit a patch.

---8<---
diff --git a/net/unix/diag.c b/net/unix/diag.c
index 105f522a89fe..581500296515 100644
--- a/net/unix/diag.c
+++ b/net/unix/diag.c
@@ -241,8 +241,8 @@ static struct sock *unix_lookup_by_ino(struct net *net, unsigned int ino)
 	for (i = 0; i < UNIX_HASH_SIZE; i++) {
 		spin_lock(&net->unx.table.locks[i]);
 		sk_for_each(sk, &net->unx.table.buckets[i]) {
-			if (ino == sock_i_ino(sk)) {
-				sock_hold(sk);
+			if (ino == sock_i_ino(sk) &&
+			    refcount_inc_not_zero(&sk->sk_refcnt)) {
 				spin_unlock(&net->unx.table.locks[i]);
 				return sk;
 			}
---8<---

Thank you.


> 
> IMPORTANT: if you fix the bug, please add the following tag to the commit:
> Reported-by: Wei Chen <harperchen1110@gmail.com>
> 
> BUG: kernel NULL pointer dereference, address: 0000000000000270
> #PF: supervisor read access in kernel mode
> #PF: error_code(0x0000) - not-present page
> PGD 12bbce067 P4D 12bbce067 PUD 12bc40067 PMD 0
> Oops: 0000 [#1] PREEMPT SMP
> CPU: 0 PID: 27942 Comm: syz-executor.0 Not tainted 6.1.0-rc5-next-20221118 #2
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
> rel-1.13.0-48-gd9c812dda519-prebuilt.qemu.org 04/01/2014
> RIP: 0010:sk_user_ns include/net/sock.h:920 [inline]
> RIP: 0010:sk_diag_dump_uid net/unix/diag.c:119 [inline]
> RIP: 0010:sk_diag_fill+0x77d/0x890 net/unix/diag.c:170
> Code: 89 ef e8 66 d4 2d fd c7 44 24 40 00 00 00 00 49 8d 7c 24 18 e8
> 54 d7 2d fd 49 8b 5c 24 18 48 8d bb 70 02 00 00 e8 43 d7 2d fd <48> 8b
> 9b 70 02 00 00 48 8d 7b 10 e8 33 d7 2d fd 48 8b 5b 10 48 8d
> RSP: 0018:ffffc90000d67968 EFLAGS: 00010246
> RAX: ffff88812badaa48 RBX: 0000000000000000 RCX: ffffffff840d481d
> RDX: 0000000000000465 RSI: 0000000000000000 RDI: 0000000000000270
> RBP: ffffc90000d679a8 R08: 0000000000000277 R09: 0000000000000000
> R10: 0001ffffffffffff R11: 0001c90000d679a8 R12: ffff88812ac03800
> R13: ffff88812c87c400 R14: ffff88812ae42210 R15: ffff888103026940
> FS:  00007f08b4e6f700(0000) GS:ffff88813bc00000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 0000000000000270 CR3: 000000012c58b000 CR4: 00000000003506f0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> Call Trace:
>  <TASK>
>  unix_diag_get_exact net/unix/diag.c:285 [inline]
>  unix_diag_handler_dump+0x3f9/0x500 net/unix/diag.c:317
>  __sock_diag_cmd net/core/sock_diag.c:235 [inline]
>  sock_diag_rcv_msg+0x237/0x250 net/core/sock_diag.c:266
>  netlink_rcv_skb+0x13e/0x250 net/netlink/af_netlink.c:2564
>  sock_diag_rcv+0x24/0x40 net/core/sock_diag.c:277
>  netlink_unicast_kernel net/netlink/af_netlink.c:1330 [inline]
>  netlink_unicast+0x5e9/0x6b0 net/netlink/af_netlink.c:1356
>  netlink_sendmsg+0x739/0x860 net/netlink/af_netlink.c:1932
>  sock_sendmsg_nosec net/socket.c:714 [inline]
>  sock_sendmsg net/socket.c:734 [inline]
>  ____sys_sendmsg+0x38f/0x500 net/socket.c:2476
>  ___sys_sendmsg net/socket.c:2530 [inline]
>  __sys_sendmsg+0x197/0x230 net/socket.c:2559
>  __do_sys_sendmsg net/socket.c:2568 [inline]
>  __se_sys_sendmsg net/socket.c:2566 [inline]
>  __x64_sys_sendmsg+0x42/0x50 net/socket.c:2566
>  do_syscall_x64 arch/x86/entry/common.c:50 [inline]
>  do_syscall_64+0x2b/0x70 arch/x86/entry/common.c:80
>  entry_SYSCALL_64_after_hwframe+0x63/0xcd
> RIP: 0033:0x4697f9
> Code: f7 d8 64 89 02 b8 ff ff ff ff c3 66 0f 1f 44 00 00 48 89 f8 48
> 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d
> 01 f0 ff ff 73 01 c3 48 c7 c1 bc ff ff ff f7 d8 64 89 01 48
> RSP: 002b:00007f08b4e6ec48 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
> RAX: ffffffffffffffda RBX: 000000000077bf80 RCX: 00000000004697f9
> RDX: 0000000000000000 RSI: 00000000200001c0 RDI: 0000000000000003
> RBP: 00000000004d29e9 R08: 0000000000000000 R09: 0000000000000000
> R10: 0000000000000000 R11: 0000000000000246 R12: 000000000077bf80
> R13: 0000000000000000 R14: 000000000077bf80 R15: 00007ffdb36bc6c0
>  </TASK>
> Modules linked in:
> CR2: 0000000000000270
> ---[ end trace 0000000000000000 ]---
> RIP: 0010:sk_user_ns include/net/sock.h:920 [inline]
> RIP: 0010:sk_diag_dump_uid net/unix/diag.c:119 [inline]
> RIP: 0010:sk_diag_fill+0x77d/0x890 net/unix/diag.c:170
> Code: 89 ef e8 66 d4 2d fd c7 44 24 40 00 00 00 00 49 8d 7c 24 18 e8
> 54 d7 2d fd 49 8b 5c 24 18 48 8d bb 70 02 00 00 e8 43 d7 2d fd <48> 8b
> 9b 70 02 00 00 48 8d 7b 10 e8 33 d7 2d fd 48 8b 5b 10 48 8d
> RSP: 0018:ffffc90000d67968 EFLAGS: 00010246
> RAX: ffff88812badaa48 RBX: 0000000000000000 RCX: ffffffff840d481d
> RDX: 0000000000000465 RSI: 0000000000000000 RDI: 0000000000000270
> RBP: ffffc90000d679a8 R08: 0000000000000277 R09: 0000000000000000
> R10: 0001ffffffffffff R11: 0001c90000d679a8 R12: ffff88812ac03800
> R13: ffff88812c87c400 R14: ffff88812ae42210 R15: ffff888103026940
> FS:  00007f08b4e6f700(0000) GS:ffff88813bc00000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 0000000000000270 CR3: 000000012c58b000 CR4: 00000000003506f0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> ----------------
> Code disassembly (best guess):
>    0: 89 ef                mov    %ebp,%edi
>    2: e8 66 d4 2d fd        callq  0xfd2dd46d
>    7: c7 44 24 40 00 00 00 movl   $0x0,0x40(%rsp)
>    e: 00
>    f: 49 8d 7c 24 18        lea    0x18(%r12),%rdi
>   14: e8 54 d7 2d fd        callq  0xfd2dd76d
>   19: 49 8b 5c 24 18        mov    0x18(%r12),%rbx
>   1e: 48 8d bb 70 02 00 00 lea    0x270(%rbx),%rdi
>   25: e8 43 d7 2d fd        callq  0xfd2dd76d
> * 2a: 48 8b 9b 70 02 00 00 mov    0x270(%rbx),%rbx <-- trapping instruction
>   31: 48 8d 7b 10          lea    0x10(%rbx),%rdi
>   35: e8 33 d7 2d fd        callq  0xfd2dd76d
>   3a: 48 8b 5b 10          mov    0x10(%rbx),%rbx
>   3e: 48                    rex.W
>   3f: 8d                    .byte 0x8d
> 
> 
> Best,
> Wei