Hello,
We observed a crash "general protection fault in vmci_host_poll"
during fuzzing.
Unfortunately, we have not found a reproducer for the crash yet. We
will inform you if we have any update on this crash.
Detailed crash information is attached at the end of this email.
Best regards,
Dae R. Jeong
------
- Kernel commit:
92f20ff72066d
- Crash report:
general protection fault, probably for non-canonical address 0xdffffc000000000b: 0000 [#1] PREEMPT SMP KASAN
KASAN: null-ptr-deref in range [0x0000000000000058-0x000000000000005f]
CPU: 2 PID: 10337 Comm: syz-executor.0 Not tainted 5.19.0-rc3-32288-g0f3b08299494 #15
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
RIP: 0010:__lock_acquire+0xf3/0x5e00 kernel/locking/lockdep.c:4926
Code: 89 bc 24 88 00 00 00 0f 85 02 27 00 00 83 3d 1b 16 66 10 00 0f 84 97 43 00 00 83 3d d6 ea f9 0e 00 74 31 48 89 f8 48 c1 e8 03 <42> 80 3c 00 00 74 17 e8 81 63 86 00 48 8b bc 24 88 00 00 00 49 b8
RSP: 0018:ffffc90003727320 EFLAGS: 00010002
RAX: 000000000000000b RBX: 0000000000000001 RCX: 1ffff920006e4e84
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000058
RBP: ffffc900037276b0 R08: dffffc0000000000 R09: 0000000000000000
R10: fffffbfff2394d5e R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 00007fc21b11a700(0000) GS:ffff8880b9300000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055b8cfafd7d0 CR3: 000000001c4aa000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<TASK>
lock_acquire+0x1a4/0x4a0 kernel/locking/lockdep.c:5672
__raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
_raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:154
spin_lock include/linux/spinlock.h:349 [inline]
vmci_host_poll+0x16b/0x2b0 drivers/misc/vmw_vmci/vmci_host.c:177
vfs_poll include/linux/poll.h:88 [inline]
do_pollfd fs/select.c:873 [inline]
do_poll fs/select.c:921 [inline]
do_sys_poll+0xc7c/0x1aa0 fs/select.c:1015
__do_sys_ppoll fs/select.c:1121 [inline]
__se_sys_ppoll+0x2cc/0x330 fs/select.c:1101
do_syscall_x64 arch/x86/entry/common.c:51 [inline]
do_syscall_64+0x4e/0xa0 arch/x86/entry/common.c:82
entry_SYSCALL_64_after_hwframe+0x46/0xb0
RIP: 0033:0x478dc9
Code: f7 d8 64 89 02 b8 ff ff ff ff c3 66 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 bc ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fc21b119be8 EFLAGS: 00000246 ORIG_RAX: 000000000000010f
RAX: ffffffffffffffda RBX: 0000000000781408 RCX: 0000000000478dc9
RDX: 0000000020001b40 RSI: 0000000000000001 RDI: 0000000020001ac0
RBP: 00000000f477909a R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000781500
R13: 0000000000781414 R14: 0000000000781408 R15: 00007ffcf3912790
</TASK>
Modules linked in:
---[ end trace 0000000000000000 ]---
RIP: 0010:__lock_acquire+0xf3/0x5e00 kernel/locking/lockdep.c:4926
Code: 89 bc 24 88 00 00 00 0f 85 02 27 00 00 83 3d 1b 16 66 10 00 0f 84 97 43 00 00 83 3d d6 ea f9 0e 00 74 31 48 89 f8 48 c1 e8 03 <42> 80 3c 00 00 74 17 e8 81 63 86 00 48 8b bc 24 88 00 00 00 49 b8
RSP: 0018:ffffc90003727320 EFLAGS: 00010002
RAX: 000000000000000b RBX: 0000000000000001 RCX: 1ffff920006e4e84
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000058
RBP: ffffc900037276b0 R08: dffffc0000000000 R09: 0000000000000000
R10: fffffbfff2394d5e R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 00007fc21b11a700(0000) GS:ffff8880b9300000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055b8cfafd7d0 CR3: 000000001c4aa000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Thank you for reporting this. We have filed an internal bug and will look into it.
Regards,
Vishnu
> On Jul 11, 2022, at 3:58 AM, Dae R. Jeong <[email protected]> wrote:
>
> ⚠ External Email
>
> Hello,
>
> We observed a crash "general protection fault in vmci_host_poll"
> during fuzzing.
>
> Unfortunately, we have not found a reproducer for the crash yet. We
> will inform you if we have any update on this crash.
>
> Detailed crash information is attached at the end of this email.
>
>
> Best regards,
> Dae R. Jeong
> ------
>
> - Kernel commit:
> 92f20ff72066d
>
> - Crash report:
> general protection fault, probably for non-canonical address 0xdffffc000000000b: 0000 [#1] PREEMPT SMP KASAN
> KASAN: null-ptr-deref in range [0x0000000000000058-0x000000000000005f]
> CPU: 2 PID: 10337 Comm: syz-executor.0 Not tainted 5.19.0-rc3-32288-g0f3b08299494 #15
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
> RIP: 0010:__lock_acquire+0xf3/0x5e00 kernel/locking/lockdep.c:4926
> Code: 89 bc 24 88 00 00 00 0f 85 02 27 00 00 83 3d 1b 16 66 10 00 0f 84 97 43 00 00 83 3d d6 ea f9 0e 00 74 31 48 89 f8 48 c1 e8 03 <42> 80 3c 00 00 74 17 e8 81 63 86 00 48 8b bc 24 88 00 00 00 49 b8
> RSP: 0018:ffffc90003727320 EFLAGS: 00010002
> RAX: 000000000000000b RBX: 0000000000000001 RCX: 1ffff920006e4e84
> RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000058
> RBP: ffffc900037276b0 R08: dffffc0000000000 R09: 0000000000000000
> R10: fffffbfff2394d5e R11: 0000000000000000 R12: 0000000000000000
> R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> FS: 00007fc21b11a700(0000) GS:ffff8880b9300000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 000055b8cfafd7d0 CR3: 000000001c4aa000 CR4: 00000000003506e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> Call Trace:
> <TASK>
> lock_acquire+0x1a4/0x4a0 kernel/locking/lockdep.c:5672
> __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
> _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:154
> spin_lock include/linux/spinlock.h:349 [inline]
> vmci_host_poll+0x16b/0x2b0 drivers/misc/vmw_vmci/vmci_host.c:177
> vfs_poll include/linux/poll.h:88 [inline]
> do_pollfd fs/select.c:873 [inline]
> do_poll fs/select.c:921 [inline]
> do_sys_poll+0xc7c/0x1aa0 fs/select.c:1015
> __do_sys_ppoll fs/select.c:1121 [inline]
> __se_sys_ppoll+0x2cc/0x330 fs/select.c:1101
> do_syscall_x64 arch/x86/entry/common.c:51 [inline]
> do_syscall_64+0x4e/0xa0 arch/x86/entry/common.c:82
> entry_SYSCALL_64_after_hwframe+0x46/0xb0
> RIP: 0033:0x478dc9
> Code: f7 d8 64 89 02 b8 ff ff ff ff c3 66 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 bc ff ff ff f7 d8 64 89 01 48
> RSP: 002b:00007fc21b119be8 EFLAGS: 00000246 ORIG_RAX: 000000000000010f
> RAX: ffffffffffffffda RBX: 0000000000781408 RCX: 0000000000478dc9
> RDX: 0000000020001b40 RSI: 0000000000000001 RDI: 0000000020001ac0
> RBP: 00000000f477909a R08: 0000000000000000 R09: 0000000000000000
> R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000781500
> R13: 0000000000781414 R14: 0000000000781408 R15: 00007ffcf3912790
> </TASK>
> Modules linked in:
> ---[ end trace 0000000000000000 ]---
> RIP: 0010:__lock_acquire+0xf3/0x5e00 kernel/locking/lockdep.c:4926
> Code: 89 bc 24 88 00 00 00 0f 85 02 27 00 00 83 3d 1b 16 66 10 00 0f 84 97 43 00 00 83 3d d6 ea f9 0e 00 74 31 48 89 f8 48 c1 e8 03 <42> 80 3c 00 00 74 17 e8 81 63 86 00 48 8b bc 24 88 00 00 00 49 b8
> RSP: 0018:ffffc90003727320 EFLAGS: 00010002
> RAX: 000000000000000b RBX: 0000000000000001 RCX: 1ffff920006e4e84
> RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000058
> RBP: ffffc900037276b0 R08: dffffc0000000000 R09: 0000000000000000
> R10: fffffbfff2394d5e R11: 0000000000000000 R12: 0000000000000000
> R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> FS: 00007fc21b11a700(0000) GS:ffff8880b9300000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 000055b8cfafd7d0 CR3: 000000001c4aa000 CR4: 00000000003506e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>
> ________________________________
>
> ⚠ External Email: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender.
On Aug 10, 2022, at 11:00 AM, Vishnu Dasa <[email protected]> wrote:
> Thank you for reporting this. We have filed an internal bug and will look into it.
>
> Regards,
> Vishnu
>
>> On Jul 11, 2022, at 3:58 AM, Dae R. Jeong <[email protected]> wrote:
>>
>> ⚠ External Email
>>
>> Hello,
>>
>> We observed a crash "general protection fault in vmci_host_poll"
>> during fuzzing.
>>
>> Unfortunately, we have not found a reproducer for the crash yet. We
>> will inform you if we have any update on this crash.
>>
>> Detailed crash information is attached at the end of this email.
>>
>>
>> Best regards,
>> Dae R. Jeong
>> ------
>>
>> - Kernel commit:
>> 92f20ff72066d
>>
>> - Crash report:
>> general protection fault, probably for non-canonical address 0xdffffc000000000b: 0000 [#1] PREEMPT SMP KASAN
>> KASAN: null-ptr-deref in range [0x0000000000000058-0x000000000000005f]
>> CPU: 2 PID: 10337 Comm: syz-executor.0 Not tainted 5.19.0-rc3-32288-g0f3b08299494 #15
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
>> RIP: 0010:__lock_acquire+0xf3/0x5e00 kernel/locking/lockdep.c:4926
>> Code: 89 bc 24 88 00 00 00 0f 85 02 27 00 00 83 3d 1b 16 66 10 00 0f 84 97 43 00 00 83 3d d6 ea f9 0e 00 74 31 48 89 f8 48 c1 e8 03 <42> 80 3c 00 00 74 17 e8 81 63 86 00 48 8b bc 24 88 00 00 00 49 b8
>> RSP: 0018:ffffc90003727320 EFLAGS: 00010002
>> RAX: 000000000000000b RBX: 0000000000000001 RCX: 1ffff920006e4e84
>> RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000058
>> RBP: ffffc900037276b0 R08: dffffc0000000000 R09: 0000000000000000
>> R10: fffffbfff2394d5e R11: 0000000000000000 R12: 0000000000000000
>> R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
>> FS: 00007fc21b11a700(0000) GS:ffff8880b9300000(0000) knlGS:0000000000000000
>> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> CR2: 000055b8cfafd7d0 CR3: 000000001c4aa000 CR4: 00000000003506e0
>> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>> Call Trace:
>> <TASK>
>> lock_acquire+0x1a4/0x4a0 kernel/locking/lockdep.c:5672
>> __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
>> _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:154
>> spin_lock include/linux/spinlock.h:349 [inline]
>> vmci_host_poll+0x16b/0x2b0 drivers/misc/vmw_vmci/vmci_host.c:177
>> vfs_poll include/linux/poll.h:88 [inline]
>> do_pollfd fs/select.c:873 [inline]
>> do_poll fs/select.c:921 [inline]
>> do_sys_poll+0xc7c/0x1aa0 fs/select.c:1015
>> __do_sys_ppoll fs/select.c:1121 [inline]
>> __se_sys_ppoll+0x2cc/0x330 fs/select.c:1101
>> do_syscall_x64 arch/x86/entry/common.c:51 [inline]
>> do_syscall_64+0x4e/0xa0 arch/x86/entry/common.c:82
>> entry_SYSCALL_64_after_hwframe+0x46/0xb0
>> RIP: 0033:0x478dc9
Not my module, so just sharing my 2 cents:
It seems that this is a bug that is related to interaction between different
debugging features, and it might not be related to VMCI. IIUC, KASAN is
yelling at lock-dependency checker.
The code that the failure points to is the entry to the lock_release(),
which raises the question whether additional debug features were enabled
during the failure, specifically ftrace function tracer or kprobes.
On Wed, Aug 10, 2022 at 06:36:02PM +0000, Nadav Amit wrote:
> >> - Crash report:
> >> general protection fault, probably for non-canonical address 0xdffffc000000000b: 0000 [#1] PREEMPT SMP KASAN
> >> KASAN: null-ptr-deref in range [0x0000000000000058-0x000000000000005f]
> >> Call Trace:
> >> <TASK>
> >> lock_acquire+0x1a4/0x4a0 kernel/locking/lockdep.c:5672
> >> __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
> >> _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:154
> >> spin_lock include/linux/spinlock.h:349 [inline]
> >> vmci_host_poll+0x16b/0x2b0 drivers/misc/vmw_vmci/vmci_host.c:177
> >> vfs_poll include/linux/poll.h:88 [inline]
> >> do_pollfd fs/select.c:873 [inline]
> >> do_poll fs/select.c:921 [inline]
> >> do_sys_poll+0xc7c/0x1aa0 fs/select.c:1015
> >> __do_sys_ppoll fs/select.c:1121 [inline]
> >> __se_sys_ppoll+0x2cc/0x330 fs/select.c:1101
> >> do_syscall_x64 arch/x86/entry/common.c:51 [inline]
> >> do_syscall_64+0x4e/0xa0 arch/x86/entry/common.c:82
>
> Not my module, so just sharing my 2 cents:
>
> It seems that this is a bug that is related to interaction between different
> debugging features, and it might not be related to VMCI. IIUC, KASAN is
> yelling at lock-dependency checker.
>
> The code that the failure points to is the entry to the lock_release(),
> which raises the question whether additional debug features were enabled
> during the failure, specifically ftrace function tracer or kprobes.
>
Hello,
This crash keeps occuring in our fuzzing environment, and we looked
into this. For me it seems that is caused by a race condition as
follows:
CPU1 CPU2
vmci_host_poll vmci_host_do_init_context
----- -----
// Read uninitialized context
context = vmci_host_dev->context;
// Initialize context
vmci_host_dev->context = vmci_ctx_create();
vmci_host_dev->ct_type = VMCIOBJ_CONTEXT;
if (vmci_host_dev->ct_type == VMCIOBJ_CONTEXT) {
// Dereferencing the wrong pointer
poll_wait(..., &context->host_context);
}
I think reading `context` after checking `ct_type` in vmci_host_poll()
should be enough to prevent this. Could you check this?
Best regards,
Dae R. Jeong