2024-05-07 07:01:28

by Sam Sun

[permalink] [raw]
Subject: [Linux kernel bug] general protection fault in nexthop_is_blackhole

Dear developers and maintainers,

We encountered a general protection fault in function
nexthop_is_blackhole. It was tested against the latest upstream linux
(tag 6.9-rc7). C repro and kernel config are attached to this email.
Kernel crash log is listed below.
```
general protection fault, probably for non-canonical address
0xdffffc0080008015: 0000 [#1] PREEMPT SMP KASAN NOPTI
KASAN: probably user-memory-access in range
[0x00000004000400a8-0x00000004000400af]
CPU: 1 PID: 7959 Comm: kworker/u8:2 Not tainted 6.9.0-rc6 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
1.13.0-1ubuntu1.1 04/01/2014
Workqueue: ipv6_addrconf addrconf_dad_work
RIP: 0010:nexthop_is_blackhole+0x23/0x2a0 include/net/nexthop.h:370
Code: 00 00 00 0f 1f 40 00 55 41 57 41 56 53 48 89 fb 49 bf 00 00 00
00 00 fc ff df e8 58 c1 b6 f7 4c 8d 73 66 4c 89 f0 48 c1 e8 03 <42> 8a
04 38 84 c0 0f 85 17 02 00 00 41 0f b6 2e 31 ff 89 ee e8 44
RSP: 0018:ffffc900001d81f8 EFLAGS: 00010203
RAX: 0000000080008015 RBX: 0000000400040048 RCX: ffff88801cbfa500
RDX: 0000000080000101 RSI: 0000000000000000 RDI: 0000000400040048
RBP: ffffc900001d8398 R08: ffffffff89d8fd23 R09: 0000000000000021
R10: ffffc900001d84c0 R11: fffffbfff2273299 R12: ffff88807857e800
R13: 1ffff1100f0afd0c R14: 00000004000400ae R15: dffffc0000000000
FS: 0000000000000000(0000) GS:ffff8880be400000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa8a2420630 CR3: 00000000264f6000 CR4: 0000000000750ef0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
<IRQ>
__find_rr_leaf+0x521/0x890 net/ipv6/route.c:817
find_rr_leaf net/ipv6/route.c:861 [inline]
rt6_select net/ipv6/route.c:896 [inline]
fib6_table_lookup+0x56f/0xbb0 net/ipv6/route.c:2193
ip6_pol_route+0x272/0x1580 net/ipv6/route.c:2229
pol_lookup_func include/net/ip6_fib.h:614 [inline]
fib6_rule_lookup+0x571/0x780 net/ipv6/fib6_rules.c:116
ip6_route_input_lookup net/ipv6/route.c:2298 [inline]
ip6_route_input+0x839/0xd10 net/ipv6/route.c:2594
ip6_rcv_finish net/ipv6/ip6_input.c:77 [inline]
NF_HOOK include/linux/netfilter.h:314 [inline]
ipv6_rcv+0x1dc/0x200 net/ipv6/ip6_input.c:310
__netif_receive_skb_one_core net/core/dev.c:5544 [inline]
__netif_receive_skb+0x1dc/0x640 net/core/dev.c:5658
process_backlog+0x361/0x790 net/core/dev.c:5987
__napi_poll+0xca/0x480 net/core/dev.c:6638
napi_poll net/core/dev.c:6707 [inline]
net_rx_action+0x7c0/0x10a0 net/core/dev.c:6822
__do_softirq+0x272/0x734 kernel/softirq.c:554
do_softirq+0xfe/0x1b0 kernel/softirq.c:455
</IRQ>
<TASK>
__local_bh_enable_ip+0x18a/0x1c0 kernel/softirq.c:382
local_bh_enable include/linux/bottom_half.h:33 [inline]
rcu_read_unlock_bh include/linux/rcupdate.h:851 [inline]
__dev_queue_xmit+0x1d13/0x3a60 net/core/dev.c:4368
neigh_output include/net/neighbour.h:542 [inline]
ip6_finish_output2+0xfcf/0x1600 net/ipv6/ip6_output.c:137
ip6_finish_output+0x3c8/0x7f0 net/ipv6/ip6_output.c:222
NF_HOOK include/linux/netfilter.h:314 [inline]
ndisc_send_skb+0xa39/0xf40 net/ipv6/ndisc.c:509
addrconf_dad_completed+0x734/0xc60 net/ipv6/addrconf.c:4358
addrconf_dad_work+0xd82/0x16b0
process_one_work kernel/workqueue.c:3254 [inline]
process_scheduled_works+0x9c9/0x14a0 kernel/workqueue.c:3335
worker_thread+0x85c/0xd50 kernel/workqueue.c:3416
kthread+0x2ed/0x390 kernel/kthread.c:388
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:244
</TASK>
Modules linked in:
---[ end trace 0000000000000000 ]---
RIP: 0010:nexthop_is_blackhole+0x23/0x2a0 include/net/nexthop.h:370
Code: 00 00 00 0f 1f 40 00 55 41 57 41 56 53 48 89 fb 49 bf 00 00 00
00 00 fc ff df e8 58 c1 b6 f7 4c 8d 73 66 4c 89 f0 48 c1 e8 03 <42> 8a
04 38 84 c0 0f 85 17 02 00 00 41 0f b6 2e 31 ff 89 ee e8 44
RSP: 0018:ffffc900001d81f8 EFLAGS: 00010203

RAX: 0000000080008015 RBX: 0000000400040048 RCX: ffff88801cbfa500
RDX: 0000000080000101 RSI: 0000000000000000 RDI: 0000000400040048
RBP: ffffc900001d8398 R08: ffffffff89d8fd23 R09: 0000000000000021
R10: ffffc900001d84c0 R11: fffffbfff2273299 R12: ffff88807857e800
R13: 1ffff1100f0afd0c R14: 00000004000400ae R15: dffffc0000000000
FS: 0000000000000000(0000) GS:ffff8880be400000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa8a2420630 CR3: 00000000264f6000 CR4: 0000000000750ef0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
----------------
Code disassembly (best guess), 1 bytes skipped:
0: 00 00 add %al,(%rax)
2: 0f 1f 40 00 nopl 0x0(%rax)
6: 55 push %rbp
7: 41 57 push %r15
9: 41 56 push %r14
b: 53 push %rbx
c: 48 89 fb mov %rdi,%rbx
f: 49 bf 00 00 00 00 00 movabs $0xdffffc0000000000,%r15
16: fc ff df
19: e8 58 c1 b6 f7 callq 0xf7b6c176
1e: 4c 8d 73 66 lea 0x66(%rbx),%r14
22: 4c 89 f0 mov %r14,%rax
25: 48 c1 e8 03 shr $0x3,%rax
* 29: 42 8a 04 38 mov (%rax,%r15,1),%al <-- trapping instruction
2d: 84 c0 test %al,%al
2f: 0f 85 17 02 00 00 jne 0x24c
35: 41 0f b6 2e movzbl (%r14),%ebp
39: 31 ff xor %edi,%edi
3b: 89 ee mov %ebp,%esi
3d: e8 .byte 0xe8
3e: 44 rex.R
```
If you have any questions, please contact us.

Reported by Yue Sun <[email protected]>
Reported by xingwei lee <[email protected]>

Best Regards,
Yue


Attachments:
config (242.11 kB)
nexthop_is_blackhole.c (83.38 kB)
Download all attachments

2024-05-07 07:35:42

by Eric Dumazet

[permalink] [raw]
Subject: Re: [Linux kernel bug] general protection fault in nexthop_is_blackhole

On Tue, May 7, 2024 at 9:00 AM Sam Sun <[email protected]> wrote:
>
> Dear developers and maintainers,
>
> We encountered a general protection fault in function
> nexthop_is_blackhole. It was tested against the latest upstream linux
> (tag 6.9-rc7). C repro and kernel config are attached to this email.
> Kernel crash log is listed below.

This is another reiserfs bug, please let's not be mistaken.

We have dozens of syzbot reports about reiserfs.

Thank you.

> ```
> general protection fault, probably for non-canonical address
> 0xdffffc0080008015: 0000 [#1] PREEMPT SMP KASAN NOPTI
> KASAN: probably user-memory-access in range
> [0x00000004000400a8-0x00000004000400af]
> CPU: 1 PID: 7959 Comm: kworker/u8:2 Not tainted 6.9.0-rc6 #1
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
> 1.13.0-1ubuntu1.1 04/01/2014
> Workqueue: ipv6_addrconf addrconf_dad_work
> RIP: 0010:nexthop_is_blackhole+0x23/0x2a0 include/net/nexthop.h:370
> Code: 00 00 00 0f 1f 40 00 55 41 57 41 56 53 48 89 fb 49 bf 00 00 00
> 00 00 fc ff df e8 58 c1 b6 f7 4c 8d 73 66 4c 89 f0 48 c1 e8 03 <42> 8a
> 04 38 84 c0 0f 85 17 02 00 00 41 0f b6 2e 31 ff 89 ee e8 44
> RSP: 0018:ffffc900001d81f8 EFLAGS: 00010203
> RAX: 0000000080008015 RBX: 0000000400040048 RCX: ffff88801cbfa500
> RDX: 0000000080000101 RSI: 0000000000000000 RDI: 0000000400040048
> RBP: ffffc900001d8398 R08: ffffffff89d8fd23 R09: 0000000000000021
> R10: ffffc900001d84c0 R11: fffffbfff2273299 R12: ffff88807857e800
> R13: 1ffff1100f0afd0c R14: 00000004000400ae R15: dffffc0000000000
> FS: 0000000000000000(0000) GS:ffff8880be400000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00007fa8a2420630 CR3: 00000000264f6000 CR4: 0000000000750ef0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> PKRU: 55555554
> Call Trace:
> <IRQ>
> __find_rr_leaf+0x521/0x890 net/ipv6/route.c:817
> find_rr_leaf net/ipv6/route.c:861 [inline]
> rt6_select net/ipv6/route.c:896 [inline]
> fib6_table_lookup+0x56f/0xbb0 net/ipv6/route.c:2193
> ip6_pol_route+0x272/0x1580 net/ipv6/route.c:2229
> pol_lookup_func include/net/ip6_fib.h:614 [inline]
> fib6_rule_lookup+0x571/0x780 net/ipv6/fib6_rules.c:116
> ip6_route_input_lookup net/ipv6/route.c:2298 [inline]
> ip6_route_input+0x839/0xd10 net/ipv6/route.c:2594
> ip6_rcv_finish net/ipv6/ip6_input.c:77 [inline]
> NF_HOOK include/linux/netfilter.h:314 [inline]
> ipv6_rcv+0x1dc/0x200 net/ipv6/ip6_input.c:310
> __netif_receive_skb_one_core net/core/dev.c:5544 [inline]
> __netif_receive_skb+0x1dc/0x640 net/core/dev.c:5658
> process_backlog+0x361/0x790 net/core/dev.c:5987
> __napi_poll+0xca/0x480 net/core/dev.c:6638
> napi_poll net/core/dev.c:6707 [inline]
> net_rx_action+0x7c0/0x10a0 net/core/dev.c:6822
> __do_softirq+0x272/0x734 kernel/softirq.c:554
> do_softirq+0xfe/0x1b0 kernel/softirq.c:455
> </IRQ>
> <TASK>
> __local_bh_enable_ip+0x18a/0x1c0 kernel/softirq.c:382
> local_bh_enable include/linux/bottom_half.h:33 [inline]
> rcu_read_unlock_bh include/linux/rcupdate.h:851 [inline]
> __dev_queue_xmit+0x1d13/0x3a60 net/core/dev.c:4368
> neigh_output include/net/neighbour.h:542 [inline]
> ip6_finish_output2+0xfcf/0x1600 net/ipv6/ip6_output.c:137
> ip6_finish_output+0x3c8/0x7f0 net/ipv6/ip6_output.c:222
> NF_HOOK include/linux/netfilter.h:314 [inline]
> ndisc_send_skb+0xa39/0xf40 net/ipv6/ndisc.c:509
> addrconf_dad_completed+0x734/0xc60 net/ipv6/addrconf.c:4358
> addrconf_dad_work+0xd82/0x16b0
> process_one_work kernel/workqueue.c:3254 [inline]
> process_scheduled_works+0x9c9/0x14a0 kernel/workqueue.c:3335
> worker_thread+0x85c/0xd50 kernel/workqueue.c:3416
> kthread+0x2ed/0x390 kernel/kthread.c:388
> ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
> ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:244
> </TASK>
> Modules linked in:
> ---[ end trace 0000000000000000 ]---
> RIP: 0010:nexthop_is_blackhole+0x23/0x2a0 include/net/nexthop.h:370
> Code: 00 00 00 0f 1f 40 00 55 41 57 41 56 53 48 89 fb 49 bf 00 00 00
> 00 00 fc ff df e8 58 c1 b6 f7 4c 8d 73 66 4c 89 f0 48 c1 e8 03 <42> 8a
> 04 38 84 c0 0f 85 17 02 00 00 41 0f b6 2e 31 ff 89 ee e8 44
> RSP: 0018:ffffc900001d81f8 EFLAGS: 00010203
>
> RAX: 0000000080008015 RBX: 0000000400040048 RCX: ffff88801cbfa500
> RDX: 0000000080000101 RSI: 0000000000000000 RDI: 0000000400040048
> RBP: ffffc900001d8398 R08: ffffffff89d8fd23 R09: 0000000000000021
> R10: ffffc900001d84c0 R11: fffffbfff2273299 R12: ffff88807857e800
> R13: 1ffff1100f0afd0c R14: 00000004000400ae R15: dffffc0000000000
> FS: 0000000000000000(0000) GS:ffff8880be400000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00007fa8a2420630 CR3: 00000000264f6000 CR4: 0000000000750ef0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> PKRU: 55555554
> ----------------
> Code disassembly (best guess), 1 bytes skipped:
> 0: 00 00 add %al,(%rax)
> 2: 0f 1f 40 00 nopl 0x0(%rax)
> 6: 55 push %rbp
> 7: 41 57 push %r15
> 9: 41 56 push %r14
> b: 53 push %rbx
> c: 48 89 fb mov %rdi,%rbx
> f: 49 bf 00 00 00 00 00 movabs $0xdffffc0000000000,%r15
> 16: fc ff df
> 19: e8 58 c1 b6 f7 callq 0xf7b6c176
> 1e: 4c 8d 73 66 lea 0x66(%rbx),%r14
> 22: 4c 89 f0 mov %r14,%rax
> 25: 48 c1 e8 03 shr $0x3,%rax
> * 29: 42 8a 04 38 mov (%rax,%r15,1),%al <-- trapping instruction
> 2d: 84 c0 test %al,%al
> 2f: 0f 85 17 02 00 00 jne 0x24c
> 35: 41 0f b6 2e movzbl (%r14),%ebp
> 39: 31 ff xor %edi,%edi
> 3b: 89 ee mov %ebp,%esi
> 3d: e8 .byte 0xe8
> 3e: 44 rex.R
> ```
> If you have any questions, please contact us.
>
> Reported by Yue Sun <[email protected]>
> Reported by xingwei lee <[email protected]>
>
> Best Regards,
> Yue

2024-05-07 07:39:43

by Sam Sun

[permalink] [raw]
Subject: Re: [Linux kernel bug] general protection fault in nexthop_is_blackhole

On Tue, May 7, 2024 at 3:31 PM Eric Dumazet <[email protected]> wrote:
>
> On Tue, May 7, 2024 at 9:00 AM Sam Sun <[email protected]> wrote:
> >
> > Dear developers and maintainers,
> >
> > We encountered a general protection fault in function
> > nexthop_is_blackhole. It was tested against the latest upstream linux
> > (tag 6.9-rc7). C repro and kernel config are attached to this email.
> > Kernel crash log is listed below.
>
> This is another reiserfs bug, please let's not be mistaken.
>
> We have dozens of syzbot reports about reiserfs.
>
> Thank you.
>

Sorry for my mistake and thanks for pointing out. I only checked the
call stack without checking the repro. I will ban the reiserfs in
future testing.

Best Regards,
Yue