2023-03-26 12:19:20

by Dae R. Jeong

[permalink] [raw]
Subject: general protection fault in raw_seq_start

Hi,

We observed an issue "general protection fault in raw_seq_start"
during fuzzing.

Unfortunately, we have not found a reproducer for the crash yet. We
will inform you if we have any update on this crash.
Detailed crash information is attached below.

Best regards,
Dae R. Jeong

-----

- Kernel version
6.2

- Last executed input
unshare(0x40060200)
r0 = syz_open_procfs(0x0, &(0x7f0000002080)='net/raw\x00')
socket$inet_icmp_raw(0x2, 0x3, 0x1)
ppoll(0x0, 0x0, 0x0, 0x0, 0x0)
socket$inet_icmp_raw(0x2, 0x3, 0x1)
pread64(r0, &(0x7f0000000000)=""/10, 0xa, 0x10000000007f)

- Crash report
general protection fault, probably for non-canonical address 0xdffffc0000000005: 0000 [#1] PREEMPT SMP KASAN
KASAN: null-ptr-deref in range [0x0000000000000028-0x000000000000002f]
CPU: 2 PID: 20952 Comm: syz-executor.0 Not tainted 6.2.0-g048ec869bafd-dirty #7
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
RIP: 0010:read_pnet include/net/net_namespace.h:383 [inline]
RIP: 0010:sock_net include/net/sock.h:649 [inline]
RIP: 0010:raw_get_next net/ipv4/raw.c:974 [inline]
RIP: 0010:raw_get_idx net/ipv4/raw.c:986 [inline]
RIP: 0010:raw_seq_start+0x431/0x800 net/ipv4/raw.c:995
Code: ef e8 33 3d 94 f7 49 8b 6d 00 4c 89 ef e8 b7 65 5f f7 49 89 ed 49 83 c5 98 0f 84 9a 00 00 00 48 83 c5 c8 48 89 e8 48 c1 e8 03 <42> 80 3c 30 00 74 08 48 89 ef e8 00 3d 94 f7 4c 8b 7d 00 48 89 ef
RSP: 0018:ffffc9001154f9b0 EFLAGS: 00010206
RAX: 0000000000000005 RBX: 1ffff1100302c8fd RCX: 0000000000000000
RDX: 0000000000000028 RSI: ffffc9001154f988 RDI: ffffc9000f77a338
RBP: 0000000000000029 R08: ffffffff8a50ffb4 R09: fffffbfff24b6bd9
R10: fffffbfff24b6bd9 R11: 0000000000000000 R12: ffff88801db73b78
R13: fffffffffffffff9 R14: dffffc0000000000 R15: 0000000000000030
FS: 00007f843ae8e700(0000) GS:ffff888063700000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055bb9614b35f CR3: 000000003c672000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<TASK>
seq_read_iter+0x4c6/0x10f0 fs/seq_file.c:225
seq_read+0x224/0x320 fs/seq_file.c:162
pde_read fs/proc/inode.c:316 [inline]
proc_reg_read+0x23f/0x330 fs/proc/inode.c:328
vfs_read+0x31e/0xd30 fs/read_write.c:468
ksys_pread64 fs/read_write.c:665 [inline]
__do_sys_pread64 fs/read_write.c:675 [inline]
__se_sys_pread64 fs/read_write.c:672 [inline]
__x64_sys_pread64+0x1e9/0x280 fs/read_write.c:672
do_syscall_x64 arch/x86/entry/common.c:51 [inline]
do_syscall_64+0x4e/0xa0 arch/x86/entry/common.c:82
entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x478d29
Code: f7 d8 64 89 02 b8 ff ff ff ff c3 66 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 bc ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f843ae8dbe8 EFLAGS: 00000246 ORIG_RAX: 0000000000000011
RAX: ffffffffffffffda RBX: 0000000000791408 RCX: 0000000000478d29
RDX: 000000000000000a RSI: 0000000020000000 RDI: 0000000000000003
RBP: 00000000f477909a R08: 0000000000000000 R09: 0000000000000000
R10: 000010000000007f R11: 0000000000000246 R12: 0000000000791740
R13: 0000000000791414 R14: 0000000000791408 R15: 00007ffc2eb48a50
</TASK>
Modules linked in:
---[ end trace 0000000000000000 ]---
RIP: 0010:read_pnet include/net/net_namespace.h:383 [inline]
RIP: 0010:sock_net include/net/sock.h:649 [inline]
RIP: 0010:raw_get_next net/ipv4/raw.c:974 [inline]
RIP: 0010:raw_get_idx net/ipv4/raw.c:986 [inline]
RIP: 0010:raw_seq_start+0x431/0x800 net/ipv4/raw.c:995
Code: ef e8 33 3d 94 f7 49 8b 6d 00 4c 89 ef e8 b7 65 5f f7 49 89 ed 49 83 c5 98 0f 84 9a 00 00 00 48 83 c5 c8 48 89 e8 48 c1 e8 03 <42> 80 3c 30 00 74 08 48 89 ef e8 00 3d 94 f7 4c 8b 7d 00 48 89 ef
RSP: 0018:ffffc9001154f9b0 EFLAGS: 00010206
RAX: 0000000000000005 RBX: 1ffff1100302c8fd RCX: 0000000000000000
RDX: 0000000000000028 RSI: ffffc9001154f988 RDI: ffffc9000f77a338
RBP: 0000000000000029 R08: ffffffff8a50ffb4 R09: fffffbfff24b6bd9
R10: fffffbfff24b6bd9 R11: 0000000000000000 R12: ffff88801db73b78
R13: fffffffffffffff9 R14: dffffc0000000000 R15: 0000000000000030
FS: 00007f843ae8e700(0000) GS:ffff888063700000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f92ff166000 CR3: 000000003c672000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400


2023-03-30 21:55:53

by Kuniyuki Iwashima

[permalink] [raw]
Subject: Re: general protection fault in raw_seq_start

From: "Dae R. Jeong" <[email protected]>
Date: Sun, 26 Mar 2023 21:12:08 +0900
> Hi,
>
> We observed an issue "general protection fault in raw_seq_start"
> during fuzzing.
>
> Unfortunately, we have not found a reproducer for the crash yet. We
> will inform you if we have any update on this crash.
> Detailed crash information is attached below.
>
> Best regards,
> Dae R. Jeong
>
> -----
>
> - Kernel version
> 6.2
>
> - Last executed input
> unshare(0x40060200)
> r0 = syz_open_procfs(0x0, &(0x7f0000002080)='net/raw\x00')
> socket$inet_icmp_raw(0x2, 0x3, 0x1)
> ppoll(0x0, 0x0, 0x0, 0x0, 0x0)
> socket$inet_icmp_raw(0x2, 0x3, 0x1)
> pread64(r0, &(0x7f0000000000)=""/10, 0xa, 0x10000000007f)

Thanks for reporting the issue.

It seems we need to use RCU variant in raw_get_first().
I'll post a patch.

---
diff --git a/net/ipv4/raw.c b/net/ipv4/raw.c
index 3cf68695b40d..fe0d1ad20b35 100644
--- a/net/ipv4/raw.c
+++ b/net/ipv4/raw.c
@@ -957,7 +957,7 @@ static struct sock *raw_get_first(struct seq_file *seq, int bucket)
for (state->bucket = bucket; state->bucket < RAW_HTABLE_SIZE;
++state->bucket) {
hlist = &h->ht[state->bucket];
- sk_nulls_for_each(sk, hnode, hlist) {
+ sk_nulls_for_each_rcu(sk, hnode, hlist) {
if (sock_net(sk) == seq_file_net(seq))
return sk;
}
---

Thanks,
Kuniyuki



>
> - Crash report
> general protection fault, probably for non-canonical address 0xdffffc0000000005: 0000 [#1] PREEMPT SMP KASAN
> KASAN: null-ptr-deref in range [0x0000000000000028-0x000000000000002f]
> CPU: 2 PID: 20952 Comm: syz-executor.0 Not tainted 6.2.0-g048ec869bafd-dirty #7
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
> RIP: 0010:read_pnet include/net/net_namespace.h:383 [inline]
> RIP: 0010:sock_net include/net/sock.h:649 [inline]
> RIP: 0010:raw_get_next net/ipv4/raw.c:974 [inline]
> RIP: 0010:raw_get_idx net/ipv4/raw.c:986 [inline]
> RIP: 0010:raw_seq_start+0x431/0x800 net/ipv4/raw.c:995
> Code: ef e8 33 3d 94 f7 49 8b 6d 00 4c 89 ef e8 b7 65 5f f7 49 89 ed 49 83 c5 98 0f 84 9a 00 00 00 48 83 c5 c8 48 89 e8 48 c1 e8 03 <42> 80 3c 30 00 74 08 48 89 ef e8 00 3d 94 f7 4c 8b 7d 00 48 89 ef
> RSP: 0018:ffffc9001154f9b0 EFLAGS: 00010206
> RAX: 0000000000000005 RBX: 1ffff1100302c8fd RCX: 0000000000000000
> RDX: 0000000000000028 RSI: ffffc9001154f988 RDI: ffffc9000f77a338
> RBP: 0000000000000029 R08: ffffffff8a50ffb4 R09: fffffbfff24b6bd9
> R10: fffffbfff24b6bd9 R11: 0000000000000000 R12: ffff88801db73b78
> R13: fffffffffffffff9 R14: dffffc0000000000 R15: 0000000000000030
> FS: 00007f843ae8e700(0000) GS:ffff888063700000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 000055bb9614b35f CR3: 000000003c672000 CR4: 00000000003506e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> Call Trace:
> <TASK>
> seq_read_iter+0x4c6/0x10f0 fs/seq_file.c:225
> seq_read+0x224/0x320 fs/seq_file.c:162
> pde_read fs/proc/inode.c:316 [inline]
> proc_reg_read+0x23f/0x330 fs/proc/inode.c:328
> vfs_read+0x31e/0xd30 fs/read_write.c:468
> ksys_pread64 fs/read_write.c:665 [inline]
> __do_sys_pread64 fs/read_write.c:675 [inline]
> __se_sys_pread64 fs/read_write.c:672 [inline]
> __x64_sys_pread64+0x1e9/0x280 fs/read_write.c:672
> do_syscall_x64 arch/x86/entry/common.c:51 [inline]
> do_syscall_64+0x4e/0xa0 arch/x86/entry/common.c:82
> entry_SYSCALL_64_after_hwframe+0x63/0xcd
> RIP: 0033:0x478d29
> Code: f7 d8 64 89 02 b8 ff ff ff ff c3 66 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 bc ff ff ff f7 d8 64 89 01 48
> RSP: 002b:00007f843ae8dbe8 EFLAGS: 00000246 ORIG_RAX: 0000000000000011
> RAX: ffffffffffffffda RBX: 0000000000791408 RCX: 0000000000478d29
> RDX: 000000000000000a RSI: 0000000020000000 RDI: 0000000000000003
> RBP: 00000000f477909a R08: 0000000000000000 R09: 0000000000000000
> R10: 000010000000007f R11: 0000000000000246 R12: 0000000000791740
> R13: 0000000000791414 R14: 0000000000791408 R15: 00007ffc2eb48a50
> </TASK>
> Modules linked in:
> ---[ end trace 0000000000000000 ]---
> RIP: 0010:read_pnet include/net/net_namespace.h:383 [inline]
> RIP: 0010:sock_net include/net/sock.h:649 [inline]
> RIP: 0010:raw_get_next net/ipv4/raw.c:974 [inline]
> RIP: 0010:raw_get_idx net/ipv4/raw.c:986 [inline]
> RIP: 0010:raw_seq_start+0x431/0x800 net/ipv4/raw.c:995
> Code: ef e8 33 3d 94 f7 49 8b 6d 00 4c 89 ef e8 b7 65 5f f7 49 89 ed 49 83 c5 98 0f 84 9a 00 00 00 48 83 c5 c8 48 89 e8 48 c1 e8 03 <42> 80 3c 30 00 74 08 48 89 ef e8 00 3d 94 f7 4c 8b 7d 00 48 89 ef
> RSP: 0018:ffffc9001154f9b0 EFLAGS: 00010206
> RAX: 0000000000000005 RBX: 1ffff1100302c8fd RCX: 0000000000000000
> RDX: 0000000000000028 RSI: ffffc9001154f988 RDI: ffffc9000f77a338
> RBP: 0000000000000029 R08: ffffffff8a50ffb4 R09: fffffbfff24b6bd9
> R10: fffffbfff24b6bd9 R11: 0000000000000000 R12: ffff88801db73b78
> R13: fffffffffffffff9 R14: dffffc0000000000 R15: 0000000000000030
> FS: 00007f843ae8e700(0000) GS:ffff888063700000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00007f92ff166000 CR3: 000000003c672000 CR4: 00000000003506e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400

2023-03-31 05:36:27

by Jason Xing

[permalink] [raw]
Subject: Re: general protection fault in raw_seq_start

On Fri, Mar 31, 2023 at 6:08 AM Kuniyuki Iwashima <[email protected]> wrote:
>
> From: "Dae R. Jeong" <[email protected]>
> Date: Sun, 26 Mar 2023 21:12:08 +0900
> > Hi,
> >
> > We observed an issue "general protection fault in raw_seq_start"
> > during fuzzing.
> >
> > Unfortunately, we have not found a reproducer for the crash yet. We
> > will inform you if we have any update on this crash.
> > Detailed crash information is attached below.
> >
> > Best regards,
> > Dae R. Jeong
> >
> > -----
> >
> > - Kernel version
> > 6.2
> >
> > - Last executed input
> > unshare(0x40060200)
> > r0 = syz_open_procfs(0x0, &(0x7f0000002080)='net/raw\x00')
> > socket$inet_icmp_raw(0x2, 0x3, 0x1)
> > ppoll(0x0, 0x0, 0x0, 0x0, 0x0)
> > socket$inet_icmp_raw(0x2, 0x3, 0x1)
> > pread64(r0, &(0x7f0000000000)=""/10, 0xa, 0x10000000007f)
>
> Thanks for reporting the issue.
>
> It seems we need to use RCU variant in raw_get_first().
> I'll post a patch.
>
> ---
> diff --git a/net/ipv4/raw.c b/net/ipv4/raw.c
> index 3cf68695b40d..fe0d1ad20b35 100644
> --- a/net/ipv4/raw.c
> +++ b/net/ipv4/raw.c
> @@ -957,7 +957,7 @@ static struct sock *raw_get_first(struct seq_file *seq, int bucket)
> for (state->bucket = bucket; state->bucket < RAW_HTABLE_SIZE;
> ++state->bucket) {
> hlist = &h->ht[state->bucket];
> - sk_nulls_for_each(sk, hnode, hlist) {
> + sk_nulls_for_each_rcu(sk, hnode, hlist) {

Hello Kuniyuki,

I'm wondering if we should also use rcu_read_lock()/_unlock() pair to protect?

CC: Eric

It seems that we have to convert all the sk_nulls_for_each() to
sk_nulls_for_each_rcu() with the rcu_read_lock/_unlock protection in
net/ipv4/raw.c?

Thanks,
Jason

> if (sock_net(sk) == seq_file_net(seq))
> return sk;
> }
> ---
>
> Thanks,
> Kuniyuki
>
>
>
> >
> > - Crash report
> > general protection fault, probably for non-canonical address 0xdffffc0000000005: 0000 [#1] PREEMPT SMP KASAN
> > KASAN: null-ptr-deref in range [0x0000000000000028-0x000000000000002f]
> > CPU: 2 PID: 20952 Comm: syz-executor.0 Not tainted 6.2.0-g048ec869bafd-dirty #7
> > Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
> > RIP: 0010:read_pnet include/net/net_namespace.h:383 [inline]
> > RIP: 0010:sock_net include/net/sock.h:649 [inline]
> > RIP: 0010:raw_get_next net/ipv4/raw.c:974 [inline]
> > RIP: 0010:raw_get_idx net/ipv4/raw.c:986 [inline]
> > RIP: 0010:raw_seq_start+0x431/0x800 net/ipv4/raw.c:995
> > Code: ef e8 33 3d 94 f7 49 8b 6d 00 4c 89 ef e8 b7 65 5f f7 49 89 ed 49 83 c5 98 0f 84 9a 00 00 00 48 83 c5 c8 48 89 e8 48 c1 e8 03 <42> 80 3c 30 00 74 08 48 89 ef e8 00 3d 94 f7 4c 8b 7d 00 48 89 ef
> > RSP: 0018:ffffc9001154f9b0 EFLAGS: 00010206
> > RAX: 0000000000000005 RBX: 1ffff1100302c8fd RCX: 0000000000000000
> > RDX: 0000000000000028 RSI: ffffc9001154f988 RDI: ffffc9000f77a338
> > RBP: 0000000000000029 R08: ffffffff8a50ffb4 R09: fffffbfff24b6bd9
> > R10: fffffbfff24b6bd9 R11: 0000000000000000 R12: ffff88801db73b78
> > R13: fffffffffffffff9 R14: dffffc0000000000 R15: 0000000000000030
> > FS: 00007f843ae8e700(0000) GS:ffff888063700000(0000) knlGS:0000000000000000
> > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > CR2: 000055bb9614b35f CR3: 000000003c672000 CR4: 00000000003506e0
> > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> > Call Trace:
> > <TASK>
> > seq_read_iter+0x4c6/0x10f0 fs/seq_file.c:225
> > seq_read+0x224/0x320 fs/seq_file.c:162
> > pde_read fs/proc/inode.c:316 [inline]
> > proc_reg_read+0x23f/0x330 fs/proc/inode.c:328
> > vfs_read+0x31e/0xd30 fs/read_write.c:468
> > ksys_pread64 fs/read_write.c:665 [inline]
> > __do_sys_pread64 fs/read_write.c:675 [inline]
> > __se_sys_pread64 fs/read_write.c:672 [inline]
> > __x64_sys_pread64+0x1e9/0x280 fs/read_write.c:672
> > do_syscall_x64 arch/x86/entry/common.c:51 [inline]
> > do_syscall_64+0x4e/0xa0 arch/x86/entry/common.c:82
> > entry_SYSCALL_64_after_hwframe+0x63/0xcd
> > RIP: 0033:0x478d29
> > Code: f7 d8 64 89 02 b8 ff ff ff ff c3 66 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 bc ff ff ff f7 d8 64 89 01 48
> > RSP: 002b:00007f843ae8dbe8 EFLAGS: 00000246 ORIG_RAX: 0000000000000011
> > RAX: ffffffffffffffda RBX: 0000000000791408 RCX: 0000000000478d29
> > RDX: 000000000000000a RSI: 0000000020000000 RDI: 0000000000000003
> > RBP: 00000000f477909a R08: 0000000000000000 R09: 0000000000000000
> > R10: 000010000000007f R11: 0000000000000246 R12: 0000000000791740
> > R13: 0000000000791414 R14: 0000000000791408 R15: 00007ffc2eb48a50
> > </TASK>
> > Modules linked in:
> > ---[ end trace 0000000000000000 ]---
> > RIP: 0010:read_pnet include/net/net_namespace.h:383 [inline]
> > RIP: 0010:sock_net include/net/sock.h:649 [inline]
> > RIP: 0010:raw_get_next net/ipv4/raw.c:974 [inline]
> > RIP: 0010:raw_get_idx net/ipv4/raw.c:986 [inline]
> > RIP: 0010:raw_seq_start+0x431/0x800 net/ipv4/raw.c:995
> > Code: ef e8 33 3d 94 f7 49 8b 6d 00 4c 89 ef e8 b7 65 5f f7 49 89 ed 49 83 c5 98 0f 84 9a 00 00 00 48 83 c5 c8 48 89 e8 48 c1 e8 03 <42> 80 3c 30 00 74 08 48 89 ef e8 00 3d 94 f7 4c 8b 7d 00 48 89 ef
> > RSP: 0018:ffffc9001154f9b0 EFLAGS: 00010206
> > RAX: 0000000000000005 RBX: 1ffff1100302c8fd RCX: 0000000000000000
> > RDX: 0000000000000028 RSI: ffffc9001154f988 RDI: ffffc9000f77a338
> > RBP: 0000000000000029 R08: ffffffff8a50ffb4 R09: fffffbfff24b6bd9
> > R10: fffffbfff24b6bd9 R11: 0000000000000000 R12: ffff88801db73b78
> > R13: fffffffffffffff9 R14: dffffc0000000000 R15: 0000000000000030
> > FS: 00007f843ae8e700(0000) GS:ffff888063700000(0000) knlGS:0000000000000000
> > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > CR2: 00007f92ff166000 CR3: 000000003c672000 CR4: 00000000003506e0
> > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400

2023-03-31 05:50:39

by Iwashima, Kuniyuki

[permalink] [raw]
Subject: Re: general protection fault in raw_seq_start

From: Jason Xing <[email protected]>
Date: Fri, 31 Mar 2023 13:31:01 +0800
> On Fri, Mar 31, 2023 at 6:08=E2=80=AFAM Kuniyuki Iwashima <[email protected]> wro
> te:
> >
> > From: "Dae R. Jeong" <[email protected]>
> > Date: Sun, 26 Mar 2023 21:12:08 +0900
> > > Hi,
> > >
> > > We observed an issue "general protection fault in raw_seq_start"
> > > during fuzzing.
> > >
> > > Unfortunately, we have not found a reproducer for the crash yet. We
> > > will inform you if we have any update on this crash.
> > > Detailed crash information is attached below.
> > >
> > > Best regards,
> > > Dae R. Jeong
> > >
> > > -----
> > >
> > > - Kernel version
> > > 6.2
> > >
> > > - Last executed input
> > > unshare(0x40060200)
> > > r0 =3D syz_open_procfs(0x0, &(0x7f0000002080)=3D'net/raw\x00')
> > > socket$inet_icmp_raw(0x2, 0x3, 0x1)
> > > ppoll(0x0, 0x0, 0x0, 0x0, 0x0)
> > > socket$inet_icmp_raw(0x2, 0x3, 0x1)
> > > pread64(r0, &(0x7f0000000000)=3D""/10, 0xa, 0x10000000007f)
> >
> > Thanks for reporting the issue.
> >
> > It seems we need to use RCU variant in raw_get_first().
> > I'll post a patch.
> >
> > ---
> > diff --git a/net/ipv4/raw.c b/net/ipv4/raw.c
> > index 3cf68695b40d..fe0d1ad20b35 100644
> > --- a/net/ipv4/raw.c
> > +++ b/net/ipv4/raw.c
> > @@ -957,7 +957,7 @@ static struct sock *raw_get_first(struct seq_file *se
> q, int bucket)
> > for (state->bucket =3D bucket; state->bucket < RAW_HTABLE_SIZE;
> > ++state->bucket) {
> > hlist =3D &h->ht[state->bucket];
> > - sk_nulls_for_each(sk, hnode, hlist) {
> > + sk_nulls_for_each_rcu(sk, hnode, hlist) {
>
> Hello Kuniyuki,
>
> I'm wondering if we should also use rcu_read_lock()/_unlock() pair to prote
> ct?

It's used. See raw_seq_start() and raw_seq_stop().


>
> CC: Eric
>
> It seems that we have to convert all the sk_nulls_for_each() to
> sk_nulls_for_each_rcu() with the rcu_read_lock/_unlock protection in
> net/ipv4/raw.c?

Yes, I'm preparing the change including raw_diag.c, and I think we
need sk_null_next_rcu() as the issue happend in raw_get_next() this
time.

Also, I have the same change for ping.c

Thanks,
Kuniyuki


>
> Thanks,
> Jason
>
> > if (sock_net(sk) =3D=3D seq_file_net(seq))
> > return sk;
> > }
> > ---
> >
> > Thanks,
> > Kuniyuki
> >
> >
> >
> > >
> > > - Crash report
> > > general protection fault, probably for non-canonical address 0xdffffc00
> 00000005: 0000 [#1] PREEMPT SMP KASAN
> > > KASAN: null-ptr-deref in range [0x0000000000000028-0x000000000000002f]
> > > CPU: 2 PID: 20952 Comm: syz-executor.0 Not tainted 6.2.0-g048ec869bafd-
> dirty #7
> > > Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-
> 0-g155821a1990b-prebuilt.qemu.org 04/01/2014
> > > RIP: 0010:read_pnet include/net/net_namespace.h:383 [inline]
> > > RIP: 0010:sock_net include/net/sock.h:649 [inline]
> > > RIP: 0010:raw_get_next net/ipv4/raw.c:974 [inline]
> > > RIP: 0010:raw_get_idx net/ipv4/raw.c:986 [inline]
> > > RIP: 0010:raw_seq_start+0x431/0x800 net/ipv4/raw.c:995
> > > Code: ef e8 33 3d 94 f7 49 8b 6d 00 4c 89 ef e8 b7 65 5f f7 49 89 ed 49
> 83 c5 98 0f 84 9a 00 00 00 48 83 c5 c8 48 89 e8 48 c1 e8 03 <42> 80 3c 30 0
> 0 74 08 48 89 ef e8 00 3d 94 f7 4c 8b 7d 00 48 89 ef
> > > RSP: 0018:ffffc9001154f9b0 EFLAGS: 00010206
> > > RAX: 0000000000000005 RBX: 1ffff1100302c8fd RCX: 0000000000000000
> > > RDX: 0000000000000028 RSI: ffffc9001154f988 RDI: ffffc9000f77a338
> > > RBP: 0000000000000029 R08: ffffffff8a50ffb4 R09: fffffbfff24b6bd9
> > > R10: fffffbfff24b6bd9 R11: 0000000000000000 R12: ffff88801db73b78
> > > R13: fffffffffffffff9 R14: dffffc0000000000 R15: 0000000000000030
> > > FS: 00007f843ae8e700(0000) GS:ffff888063700000(0000) knlGS:00000000000
> 00000
> > > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > > CR2: 000055bb9614b35f CR3: 000000003c672000 CR4: 00000000003506e0
> > > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> > > Call Trace:
> > > <TASK>
> > > seq_read_iter+0x4c6/0x10f0 fs/seq_file.c:225
> > > seq_read+0x224/0x320 fs/seq_file.c:162
> > > pde_read fs/proc/inode.c:316 [inline]
> > > proc_reg_read+0x23f/0x330 fs/proc/inode.c:328
> > > vfs_read+0x31e/0xd30 fs/read_write.c:468
> > > ksys_pread64 fs/read_write.c:665 [inline]
> > > __do_sys_pread64 fs/read_write.c:675 [inline]
> > > __se_sys_pread64 fs/read_write.c:672 [inline]
> > > __x64_sys_pread64+0x1e9/0x280 fs/read_write.c:672
> > > do_syscall_x64 arch/x86/entry/common.c:51 [inline]
> > > do_syscall_64+0x4e/0xa0 arch/x86/entry/common.c:82
> > > entry_SYSCALL_64_after_hwframe+0x63/0xcd
> > > RIP: 0033:0x478d29
> > > Code: f7 d8 64 89 02 b8 ff ff ff ff c3 66 0f 1f 44 00 00 48 89 f8 48 89
> f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 f
> f ff 73 01 c3 48 c7 c1 bc ff ff ff f7 d8 64 89 01 48
> > > RSP: 002b:00007f843ae8dbe8 EFLAGS: 00000246 ORIG_RAX: 0000000000000011
> > > RAX: ffffffffffffffda RBX: 0000000000791408 RCX: 0000000000478d29
> > > RDX: 000000000000000a RSI: 0000000020000000 RDI: 0000000000000003
> > > RBP: 00000000f477909a R08: 0000000000000000 R09: 0000000000000000
> > > R10: 000010000000007f R11: 0000000000000246 R12: 0000000000791740
> > > R13: 0000000000791414 R14: 0000000000791408 R15: 00007ffc2eb48a50
> > > </TASK>
> > > Modules linked in:
> > > ---[ end trace 0000000000000000 ]---
> > > RIP: 0010:read_pnet include/net/net_namespace.h:383 [inline]
> > > RIP: 0010:sock_net include/net/sock.h:649 [inline]
> > > RIP: 0010:raw_get_next net/ipv4/raw.c:974 [inline]
> > > RIP: 0010:raw_get_idx net/ipv4/raw.c:986 [inline]
> > > RIP: 0010:raw_seq_start+0x431/0x800 net/ipv4/raw.c:995
> > > Code: ef e8 33 3d 94 f7 49 8b 6d 00 4c 89 ef e8 b7 65 5f f7 49 89 ed 49
> 83 c5 98 0f 84 9a 00 00 00 48 83 c5 c8 48 89 e8 48 c1 e8 03 <42> 80 3c 30 0
> 0 74 08 48 89 ef e8 00 3d 94 f7 4c 8b 7d 00 48 89 ef
> > > RSP: 0018:ffffc9001154f9b0 EFLAGS: 00010206
> > > RAX: 0000000000000005 RBX: 1ffff1100302c8fd RCX: 0000000000000000
> > > RDX: 0000000000000028 RSI: ffffc9001154f988 RDI: ffffc9000f77a338
> > > RBP: 0000000000000029 R08: ffffffff8a50ffb4 R09: fffffbfff24b6bd9
> > > R10: fffffbfff24b6bd9 R11: 0000000000000000 R12: ffff88801db73b78
> > > R13: fffffffffffffff9 R14: dffffc0000000000 R15: 0000000000000030
> > > FS: 00007f843ae8e700(0000) GS:ffff888063700000(0000) knlGS:00000000000
> 00000
> > > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > > CR2: 00007f92ff166000 CR3: 000000003c672000 CR4: 00000000003506e0
> > > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400

2023-03-31 06:19:29

by Jason Xing

[permalink] [raw]
Subject: Re: general protection fault in raw_seq_start

On Fri, Mar 31, 2023 at 1:40 PM Iwashima, Kuniyuki <[email protected]> wrote:
>
> From: Jason Xing <[email protected]>
> Date: Fri, 31 Mar 2023 13:31:01 +0800
> > On Fri, Mar 31, 2023 at 6:08=E2=80=AFAM Kuniyuki Iwashima <[email protected]> wro
> > te:
> > >
> > > From: "Dae R. Jeong" <[email protected]>
> > > Date: Sun, 26 Mar 2023 21:12:08 +0900
> > > > Hi,
> > > >
> > > > We observed an issue "general protection fault in raw_seq_start"
> > > > during fuzzing.
> > > >
> > > > Unfortunately, we have not found a reproducer for the crash yet. We
> > > > will inform you if we have any update on this crash.
> > > > Detailed crash information is attached below.
> > > >
> > > > Best regards,
> > > > Dae R. Jeong
> > > >
> > > > -----
> > > >
> > > > - Kernel version
> > > > 6.2
> > > >
> > > > - Last executed input
> > > > unshare(0x40060200)
> > > > r0 =3D syz_open_procfs(0x0, &(0x7f0000002080)=3D'net/raw\x00')
> > > > socket$inet_icmp_raw(0x2, 0x3, 0x1)
> > > > ppoll(0x0, 0x0, 0x0, 0x0, 0x0)
> > > > socket$inet_icmp_raw(0x2, 0x3, 0x1)
> > > > pread64(r0, &(0x7f0000000000)=3D""/10, 0xa, 0x10000000007f)
> > >
> > > Thanks for reporting the issue.
> > >
> > > It seems we need to use RCU variant in raw_get_first().
> > > I'll post a patch.
> > >
> > > ---
> > > diff --git a/net/ipv4/raw.c b/net/ipv4/raw.c
> > > index 3cf68695b40d..fe0d1ad20b35 100644
> > > --- a/net/ipv4/raw.c
> > > +++ b/net/ipv4/raw.c
> > > @@ -957,7 +957,7 @@ static struct sock *raw_get_first(struct seq_file *se
> > q, int bucket)
> > > for (state->bucket =3D bucket; state->bucket < RAW_HTABLE_SIZE;
> > > ++state->bucket) {
> > > hlist =3D &h->ht[state->bucket];
> > > - sk_nulls_for_each(sk, hnode, hlist) {
> > > + sk_nulls_for_each_rcu(sk, hnode, hlist) {
> >
> > Hello Kuniyuki,
> >
> > I'm wondering if we should also use rcu_read_lock()/_unlock() pair to prote
> > ct?
>
> It's used. See raw_seq_start() and raw_seq_stop().

Yes, it is.

>
>
> >
> > CC: Eric
> >
> > It seems that we have to convert all the sk_nulls_for_each() to
> > sk_nulls_for_each_rcu() with the rcu_read_lock/_unlock protection in
> > net/ipv4/raw.c?
>
> Yes, I'm preparing the change including raw_diag.c, and I think we
> need sk_null_next_rcu() as the issue happend in raw_get_next() this
> time.
>
> Also, I have the same change for ping.c

Great !

>
> Thanks,
> Kuniyuki
>
>
> >
> > Thanks,
> > Jason
> >
> > > if (sock_net(sk) =3D=3D seq_file_net(seq))
> > > return sk;
> > > }
> > > ---
> > >
> > > Thanks,
> > > Kuniyuki
> > >
> > >
> > >
> > > >
> > > > - Crash report
> > > > general protection fault, probably for non-canonical address 0xdffffc00
> > 00000005: 0000 [#1] PREEMPT SMP KASAN
> > > > KASAN: null-ptr-deref in range [0x0000000000000028-0x000000000000002f]
> > > > CPU: 2 PID: 20952 Comm: syz-executor.0 Not tainted 6.2.0-g048ec869bafd-
> > dirty #7
> > > > Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-
> > 0-g155821a1990b-prebuilt.qemu.org 04/01/2014
> > > > RIP: 0010:read_pnet include/net/net_namespace.h:383 [inline]
> > > > RIP: 0010:sock_net include/net/sock.h:649 [inline]
> > > > RIP: 0010:raw_get_next net/ipv4/raw.c:974 [inline]
> > > > RIP: 0010:raw_get_idx net/ipv4/raw.c:986 [inline]
> > > > RIP: 0010:raw_seq_start+0x431/0x800 net/ipv4/raw.c:995
> > > > Code: ef e8 33 3d 94 f7 49 8b 6d 00 4c 89 ef e8 b7 65 5f f7 49 89 ed 49
> > 83 c5 98 0f 84 9a 00 00 00 48 83 c5 c8 48 89 e8 48 c1 e8 03 <42> 80 3c 30 0
> > 0 74 08 48 89 ef e8 00 3d 94 f7 4c 8b 7d 00 48 89 ef
> > > > RSP: 0018:ffffc9001154f9b0 EFLAGS: 00010206
> > > > RAX: 0000000000000005 RBX: 1ffff1100302c8fd RCX: 0000000000000000
> > > > RDX: 0000000000000028 RSI: ffffc9001154f988 RDI: ffffc9000f77a338
> > > > RBP: 0000000000000029 R08: ffffffff8a50ffb4 R09: fffffbfff24b6bd9
> > > > R10: fffffbfff24b6bd9 R11: 0000000000000000 R12: ffff88801db73b78
> > > > R13: fffffffffffffff9 R14: dffffc0000000000 R15: 0000000000000030
> > > > FS: 00007f843ae8e700(0000) GS:ffff888063700000(0000) knlGS:00000000000
> > 00000
> > > > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > > > CR2: 000055bb9614b35f CR3: 000000003c672000 CR4: 00000000003506e0
> > > > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > > > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> > > > Call Trace:
> > > > <TASK>
> > > > seq_read_iter+0x4c6/0x10f0 fs/seq_file.c:225
> > > > seq_read+0x224/0x320 fs/seq_file.c:162
> > > > pde_read fs/proc/inode.c:316 [inline]
> > > > proc_reg_read+0x23f/0x330 fs/proc/inode.c:328
> > > > vfs_read+0x31e/0xd30 fs/read_write.c:468
> > > > ksys_pread64 fs/read_write.c:665 [inline]
> > > > __do_sys_pread64 fs/read_write.c:675 [inline]
> > > > __se_sys_pread64 fs/read_write.c:672 [inline]
> > > > __x64_sys_pread64+0x1e9/0x280 fs/read_write.c:672
> > > > do_syscall_x64 arch/x86/entry/common.c:51 [inline]
> > > > do_syscall_64+0x4e/0xa0 arch/x86/entry/common.c:82
> > > > entry_SYSCALL_64_after_hwframe+0x63/0xcd
> > > > RIP: 0033:0x478d29
> > > > Code: f7 d8 64 89 02 b8 ff ff ff ff c3 66 0f 1f 44 00 00 48 89 f8 48 89
> > f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 f
> > f ff 73 01 c3 48 c7 c1 bc ff ff ff f7 d8 64 89 01 48
> > > > RSP: 002b:00007f843ae8dbe8 EFLAGS: 00000246 ORIG_RAX: 0000000000000011
> > > > RAX: ffffffffffffffda RBX: 0000000000791408 RCX: 0000000000478d29
> > > > RDX: 000000000000000a RSI: 0000000020000000 RDI: 0000000000000003
> > > > RBP: 00000000f477909a R08: 0000000000000000 R09: 0000000000000000
> > > > R10: 000010000000007f R11: 0000000000000246 R12: 0000000000791740
> > > > R13: 0000000000791414 R14: 0000000000791408 R15: 00007ffc2eb48a50
> > > > </TASK>
> > > > Modules linked in:
> > > > ---[ end trace 0000000000000000 ]---
> > > > RIP: 0010:read_pnet include/net/net_namespace.h:383 [inline]
> > > > RIP: 0010:sock_net include/net/sock.h:649 [inline]
> > > > RIP: 0010:raw_get_next net/ipv4/raw.c:974 [inline]
> > > > RIP: 0010:raw_get_idx net/ipv4/raw.c:986 [inline]
> > > > RIP: 0010:raw_seq_start+0x431/0x800 net/ipv4/raw.c:995
> > > > Code: ef e8 33 3d 94 f7 49 8b 6d 00 4c 89 ef e8 b7 65 5f f7 49 89 ed 49
> > 83 c5 98 0f 84 9a 00 00 00 48 83 c5 c8 48 89 e8 48 c1 e8 03 <42> 80 3c 30 0
> > 0 74 08 48 89 ef e8 00 3d 94 f7 4c 8b 7d 00 48 89 ef
> > > > RSP: 0018:ffffc9001154f9b0 EFLAGS: 00010206
> > > > RAX: 0000000000000005 RBX: 1ffff1100302c8fd RCX: 0000000000000000
> > > > RDX: 0000000000000028 RSI: ffffc9001154f988 RDI: ffffc9000f77a338
> > > > RBP: 0000000000000029 R08: ffffffff8a50ffb4 R09: fffffbfff24b6bd9
> > > > R10: fffffbfff24b6bd9 R11: 0000000000000000 R12: ffff88801db73b78
> > > > R13: fffffffffffffff9 R14: dffffc0000000000 R15: 0000000000000030
> > > > FS: 00007f843ae8e700(0000) GS:ffff888063700000(0000) knlGS:00000000000
> > 00000
> > > > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > > > CR2: 00007f92ff166000 CR3: 000000003c672000 CR4: 00000000003506e0
> > > > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > > > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>

2023-03-31 07:06:44

by Eric Dumazet

[permalink] [raw]
Subject: Re: general protection fault in raw_seq_start

On Thu, Mar 30, 2023 at 11:55 PM Kuniyuki Iwashima <[email protected]> wrote:

> Thanks for reporting the issue.
>
> It seems we need to use RCU variant in raw_get_first().
> I'll post a patch.
>
> ---
> diff --git a/net/ipv4/raw.c b/net/ipv4/raw.c
> index 3cf68695b40d..fe0d1ad20b35 100644
> --- a/net/ipv4/raw.c
> +++ b/net/ipv4/raw.c
> @@ -957,7 +957,7 @@ static struct sock *raw_get_first(struct seq_file *seq, int bucket)
> for (state->bucket = bucket; state->bucket < RAW_HTABLE_SIZE;
> ++state->bucket) {
> hlist = &h->ht[state->bucket];
> - sk_nulls_for_each(sk, hnode, hlist) {
> + sk_nulls_for_each_rcu(sk, hnode, hlist) {
> if (sock_net(sk) == seq_file_net(seq))
> return sk;
>

No, we do not want this.
You missed that sk_nulls_for_each_rcu() needs a specific protocol
(see Documentation/RCU/rculist_nulls.rst for details)

RCU is needed in the data path, not for this control path.

My patch went too far in the RCU conversion. I did not think about
syzbot harassing /proc files :)

We need raw_seq_start and friends to go back to use the lock.

2023-03-31 07:22:32

by Kuniyuki Iwashima

[permalink] [raw]
Subject: Re: general protection fault in raw_seq_start

From: Eric Dumazet <[email protected]>
Date: Fri, 31 Mar 2023 09:04:47 +0200
> On Thu, Mar 30, 2023 at 11:55 PM Kuniyuki Iwashima <[email protected]> wrote:
>
> > Thanks for reporting the issue.
> >
> > It seems we need to use RCU variant in raw_get_first().
> > I'll post a patch.
> >
> > ---
> > diff --git a/net/ipv4/raw.c b/net/ipv4/raw.c
> > index 3cf68695b40d..fe0d1ad20b35 100644
> > --- a/net/ipv4/raw.c
> > +++ b/net/ipv4/raw.c
> > @@ -957,7 +957,7 @@ static struct sock *raw_get_first(struct seq_file *seq, int bucket)
> > for (state->bucket = bucket; state->bucket < RAW_HTABLE_SIZE;
> > ++state->bucket) {
> > hlist = &h->ht[state->bucket];
> > - sk_nulls_for_each(sk, hnode, hlist) {
> > + sk_nulls_for_each_rcu(sk, hnode, hlist) {
> > if (sock_net(sk) == seq_file_net(seq))
> > return sk;
> >
>
> No, we do not want this.
> You missed that sk_nulls_for_each_rcu() needs a specific protocol
> (see Documentation/RCU/rculist_nulls.rst for details)

Ah, exactly SOCK_RAW does not have SLAB_TYPESAFE_BY_RCU.
Thank you for pointing this out!

And I found this seems wrong.

c25b7a7a565e ("inet: ping: use hlist_nulls rcu iterator during lookup")

>
> RCU is needed in the data path, not for this control path.
>
> My patch went too far in the RCU conversion. I did not think about
> syzbot harassing /proc files :)
>
> We need raw_seq_start and friends to go back to use the lock.

Ok, then I'll change /proc/net/{raw, icmp} to use spinlock :)

2023-03-31 10:39:24

by Florian Westphal

[permalink] [raw]
Subject: Re: general protection fault in raw_seq_start

Kuniyuki Iwashima <[email protected]> wrote:
> And I found this seems wrong.
>
> c25b7a7a565e ("inet: ping: use hlist_nulls rcu iterator during lookup")

How so?