We report two crashes in v4.19-rc8 (4.20-rc1 as well, I guess):
(Unfortunately, there is no repro for those.)
The two crashes seem to share the same issue.
In both cases, (uninitialized) memory access violation occurs
when "rdp->cblist" is about to be accessed (kernel/rcu/tree.c:2838,1728).
I guess those are freed before the use, but I still haven't figured
out the reason why.
I'm looking forward to some help.
Crash log 1
=========================================
BUG: KMSAN: uninit-value in __rcu_process_callbacks
kernel/rcu/tree.c:2838 [inline]
BUG: KMSAN: uninit-value in rcu_process_callbacks+0x5ac/0x1cb0
kernel/rcu/tree.c:2864
CPU: 0 PID: 20 Comm: kauditd Not tainted 4.19.0-rc8+ #18
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
<IRQ>
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x305/0x460 lib/dump_stack.c:113
kmsan_report+0x1a2/0x2e0 mm/kmsan/kmsan.c:917
__msan_warning+0x7d/0xe0 mm/kmsan/kmsan_instr.c:500
__rcu_process_callbacks kernel/rcu/tree.c:2838 [inline]
rcu_process_callbacks+0x5ac/0x1cb0 kernel/rcu/tree.c:2864
__do_softirq+0x5ff/0xa55 kernel/softirq.c:292
invoke_softirq kernel/softirq.c:373 [inline]
irq_exit+0x22d/0x270 kernel/softirq.c:414
exiting_irq+0xe/0x10 arch/x86/include/asm/apic.h:536
smp_apic_timer_interrupt+0x64/0x90 arch/x86/kernel/apic/apic.c:1059
apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:869
</IRQ>
RIP: 0010:finish_lock_switch+0x2b/0x40 kernel/sched/core.c:2578
Code: 48 89 e5 53 48 89 fb e8 e3 43 9a 00 8b b8 88 0c 00 00 48 8b 00
48 85 c0 75 12 48 89 df e8 7d 38 9a 00 c6 00 00 c6 03 00 fb 5b <5d> c3
e8 de 42 9a 00 eb e7 66 66 66 2e 0f 1f 84 00 00 00 00 00 55
RSP: 0018:ffff88010622fca0 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff13
RAX: ffff8801105bcc40 RBX: ffff8801061554c0 RCX: ffff8801105bdc40
RDX: ffff8801105bdc40 RSI: aaaaaaaaaaaab000 RDI: ffffea00077ec560
RBP: ffff88010622fca0 R08: ffffffff7fffffff R09: 0000000000000002
R10: 0000000000000000 R11: 0000000000000000 R12: ffff8800751cb880
R13: 0000000000000000 R14: ffff880106155db8 R15: ffff88013fcb9c40
finish_task_switch+0xe3/0x270 kernel/sched/core.c:2679
context_switch kernel/sched/core.c:2832 [inline]
__schedule+0x78f/0x8f0 kernel/sched/core.c:3479
schedule+0x1cc/0x300 kernel/sched/core.c:3523
kauditd_thread+0xc64/0xee0 kernel/audit.c:889
kthread+0x5b1/0x5f0 kernel/kthread.c:247
ret_from_fork+0x35/0x40 arch/x86/entry/entry_64.S:416
Uninit was created at:
kmsan_save_stack_with_flags mm/kmsan/kmsan.c:255 [inline]
kmsan_internal_alloc_meta_for_pages+0x157/0x730 mm/kmsan/kmsan.c:693
kmsan_alloc_page+0x80/0xe0 mm/kmsan/kmsan_hooks.c:320
__alloc_pages_nodemask+0x128c/0x69b0 mm/page_alloc.c:4416
alloc_pages_current+0x51f/0x760 mm/mempolicy.c:2093
alloc_pages include/linux/gfp.h:511 [inline]
alloc_slab_page mm/slub.c:1459 [inline]
allocate_slab mm/slub.c:1604 [inline]
new_slab+0x552/0x1f30 mm/slub.c:1675
new_slab_objects mm/slub.c:2438 [inline]
___slab_alloc+0x1414/0x1dd0 mm/slub.c:2590
__slab_alloc mm/slub.c:2630 [inline]
slab_alloc_node mm/slub.c:2693 [inline]
slab_alloc mm/slub.c:2735 [inline]
kmem_cache_alloc+0xc9b/0xda0 mm/slub.c:2740
kmem_cache_zalloc include/linux/slab.h:697 [inline]
avc_alloc_node+0x109/0xb90 security/selinux/avc.c:572
avc_update_node+0x172/0x1ee0 security/selinux/avc.c:859
avc_denied+0x312/0x360 security/selinux/avc.c:1024
avc_has_perm_noaudit+0x733/0x770 security/selinux/avc.c:1155
avc_has_perm+0x172/0x480 security/selinux/avc.c:1184
sock_has_perm security/selinux/hooks.c:4539 [inline]
selinux_socket_sendmsg+0x297/0x360 security/selinux/hooks.c:4875
security_socket_sendmsg+0x127/0x200 security/security.c:1410
sock_sendmsg net/socket.c:628 [inline]
___sys_sendmsg+0xd5f/0x1290 net/socket.c:2116
__sys_sendmsg net/socket.c:2154 [inline]
__do_sys_sendmsg net/socket.c:2163 [inline]
__se_sys_sendmsg+0x307/0x460 net/socket.c:2161
__x64_sys_sendmsg+0x4a/0x70 net/socket.c:2161
do_syscall_64+0xb8/0x100 arch/x86/entry/common.c:291
entry_SYSCALL_64_after_hwframe+0x63/0xe7
=========================================================
Crash log 2
=========================================================
BUG: KMSAN: uninit-value in rcu_accelerate_cbs+0x821/0x990
kernel/rcu/tree.c:1728
CPU: 0 PID: 10 Comm: rcu_sched Not tainted 4.19.0-rc8+ #18
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x305/0x460 lib/dump_stack.c:113
kmsan_report+0x1a2/0x2e0 mm/kmsan/kmsan.c:917
__msan_warning+0x7d/0xe0 mm/kmsan/kmsan_instr.c:500
rcu_accelerate_cbs+0x821/0x990 kernel/rcu/tree.c:1728
__note_gp_changes+0x2ac/0x9e0 kernel/rcu/tree.c:1807
rcu_gp_cleanup kernel/rcu/tree.c:2109 [inline]
rcu_gp_kthread+0x3019/0x3990 kernel/rcu/tree.c:2236
kthread+0x5b1/0x5f0 kernel/kthread.c:247
ret_from_fork+0x35/0x40 arch/x86/entry/entry_64.S:416
Uninit was created at:
kmsan_save_stack_with_flags mm/kmsan/kmsan.c:255 [inline]
kmsan_internal_alloc_meta_for_pages+0x157/0x730 mm/kmsan/kmsan.c:693
kmsan_alloc_page+0x80/0xe0 mm/kmsan/kmsan_hooks.c:320
__alloc_pages_nodemask+0x128c/0x69b0 mm/page_alloc.c:4416
alloc_pages_current+0x51f/0x760 mm/mempolicy.c:2093
alloc_pages include/linux/gfp.h:511 [inline]
alloc_slab_page mm/slub.c:1459 [inline]
allocate_slab mm/slub.c:1604 [inline]
new_slab+0x552/0x1f30 mm/slub.c:1675
new_slab_objects mm/slub.c:2438 [inline]
___slab_alloc+0x1414/0x1dd0 mm/slub.c:2590
__slab_alloc mm/slub.c:2630 [inline]
slab_alloc_node mm/slub.c:2693 [inline]
slab_alloc mm/slub.c:2735 [inline]
kmem_cache_alloc+0xc9b/0xda0 mm/slub.c:2740
kmem_cache_zalloc include/linux/slab.h:697 [inline]
avc_alloc_node+0x109/0xb90 security/selinux/avc.c:572
avc_insert security/selinux/avc.c:696 [inline]
avc_compute_av+0x31e/0x1050 security/selinux/avc.c:1008
avc_has_perm_noaudit+0x516/0x770 security/selinux/avc.c:1149
avc_has_perm+0x172/0x480 security/selinux/avc.c:1184
selinux_socket_create+0x248/0x3c0 security/selinux/hooks.c:4560
security_socket_create+0x146/0x210 security/security.c:1372
__sock_create+0x26b/0xf30 net/socket.c:1232
sock_create net/socket.c:1317 [inline]
__sys_socket+0x180/0x670 net/socket.c:1347
__do_sys_socket net/socket.c:1356 [inline]
__se_sys_socket+0x8d/0xb0 net/socket.c:1354
__x64_sys_socket+0x4a/0x70 net/socket.c:1354
do_syscall_64+0xb8/0x100 arch/x86/entry/common.c:291
entry_SYSCALL_64_after_hwframe+0x63/0xe7
=========================================================
Thanks,
Kyungtae Kim
On Wed, Nov 14, 2018 at 04:03:33AM -0500, Kyungtae Kim wrote:
> We report two crashes in v4.19-rc8 (4.20-rc1 as well, I guess):
> (Unfortunately, there is no repro for those.)
>
> The two crashes seem to share the same issue.
> In both cases, (uninitialized) memory access violation occurs
> when "rdp->cblist" is about to be accessed (kernel/rcu/tree.c:2838,1728).
> I guess those are freed before the use, but I still haven't figured
> out the reason why.
> I'm looking forward to some help.
You lost me on this one. In both cases, rdp references a per-CPU
variable that is implicitly initialized to all zeroes, due to being
(sort of) a C-language global.
If a callback is queued early, then the following lines in __call_rcu()
will make an honest list of that field because of the :
if (rcu_segcblist_empty(&rdp->cblist))
rcu_segcblist_init(&rdp->cblist);
Otherwise, when rcu_init() is invoked during early boot, we have this
in rcu_init_percpu_data(), which is called from rcutree_prepare_cpu()
which is called from rcu_init(), which is called from start_kernel():
if (rcu_segcblist_empty(&rdp->cblist) && /* No early-boot CBs? */
!init_nocb_callback_list(rdp))
rcu_segcblist_init(&rdp->cblist); /* Re-enable callbacks. */
So either init_nocb_callback_list() initializes the alternative callback
lists for a no-CBs CPU or rcu_segcblist_init() again makes an honest
list of that field.
My guess is that your tool is missing the
rdp = this_cpu_ptr(rsp->rda);
in the __call_rcu() case, and also missing the
struct rcu_data *rdp = per_cpu_ptr(rsp->rda, cpu);
Note that the ->rda field is explicitly compile-time initialized to
the base address of the per-CPU variable, which is rcu_preempt_data,
rcu_bh_data, or rcu_sched_data, depending on which RCU flavor is at hand.
(In v4.20-rc1, these are all merged into a single flavor to rule them all.)
Alternatively, your tool might be missing the implicit initialization
of per-CPU variables.
Or maybe I am missing something. If so, please let me know what it is.
Thanx, Paul
> Crash log 1
> =========================================
> BUG: KMSAN: uninit-value in __rcu_process_callbacks
> kernel/rcu/tree.c:2838 [inline]
> BUG: KMSAN: uninit-value in rcu_process_callbacks+0x5ac/0x1cb0
> kernel/rcu/tree.c:2864
> CPU: 0 PID: 20 Comm: kauditd Not tainted 4.19.0-rc8+ #18
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
> <IRQ>
> __dump_stack lib/dump_stack.c:77 [inline]
> dump_stack+0x305/0x460 lib/dump_stack.c:113
> kmsan_report+0x1a2/0x2e0 mm/kmsan/kmsan.c:917
> __msan_warning+0x7d/0xe0 mm/kmsan/kmsan_instr.c:500
> __rcu_process_callbacks kernel/rcu/tree.c:2838 [inline]
> rcu_process_callbacks+0x5ac/0x1cb0 kernel/rcu/tree.c:2864
> __do_softirq+0x5ff/0xa55 kernel/softirq.c:292
> invoke_softirq kernel/softirq.c:373 [inline]
> irq_exit+0x22d/0x270 kernel/softirq.c:414
> exiting_irq+0xe/0x10 arch/x86/include/asm/apic.h:536
> smp_apic_timer_interrupt+0x64/0x90 arch/x86/kernel/apic/apic.c:1059
> apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:869
> </IRQ>
> RIP: 0010:finish_lock_switch+0x2b/0x40 kernel/sched/core.c:2578
> Code: 48 89 e5 53 48 89 fb e8 e3 43 9a 00 8b b8 88 0c 00 00 48 8b 00
> 48 85 c0 75 12 48 89 df e8 7d 38 9a 00 c6 00 00 c6 03 00 fb 5b <5d> c3
> e8 de 42 9a 00 eb e7 66 66 66 2e 0f 1f 84 00 00 00 00 00 55
> RSP: 0018:ffff88010622fca0 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff13
> RAX: ffff8801105bcc40 RBX: ffff8801061554c0 RCX: ffff8801105bdc40
> RDX: ffff8801105bdc40 RSI: aaaaaaaaaaaab000 RDI: ffffea00077ec560
> RBP: ffff88010622fca0 R08: ffffffff7fffffff R09: 0000000000000002
> R10: 0000000000000000 R11: 0000000000000000 R12: ffff8800751cb880
> R13: 0000000000000000 R14: ffff880106155db8 R15: ffff88013fcb9c40
> finish_task_switch+0xe3/0x270 kernel/sched/core.c:2679
> context_switch kernel/sched/core.c:2832 [inline]
> __schedule+0x78f/0x8f0 kernel/sched/core.c:3479
> schedule+0x1cc/0x300 kernel/sched/core.c:3523
> kauditd_thread+0xc64/0xee0 kernel/audit.c:889
> kthread+0x5b1/0x5f0 kernel/kthread.c:247
> ret_from_fork+0x35/0x40 arch/x86/entry/entry_64.S:416
>
> Uninit was created at:
> kmsan_save_stack_with_flags mm/kmsan/kmsan.c:255 [inline]
> kmsan_internal_alloc_meta_for_pages+0x157/0x730 mm/kmsan/kmsan.c:693
> kmsan_alloc_page+0x80/0xe0 mm/kmsan/kmsan_hooks.c:320
> __alloc_pages_nodemask+0x128c/0x69b0 mm/page_alloc.c:4416
> alloc_pages_current+0x51f/0x760 mm/mempolicy.c:2093
> alloc_pages include/linux/gfp.h:511 [inline]
> alloc_slab_page mm/slub.c:1459 [inline]
> allocate_slab mm/slub.c:1604 [inline]
> new_slab+0x552/0x1f30 mm/slub.c:1675
> new_slab_objects mm/slub.c:2438 [inline]
> ___slab_alloc+0x1414/0x1dd0 mm/slub.c:2590
> __slab_alloc mm/slub.c:2630 [inline]
> slab_alloc_node mm/slub.c:2693 [inline]
> slab_alloc mm/slub.c:2735 [inline]
> kmem_cache_alloc+0xc9b/0xda0 mm/slub.c:2740
> kmem_cache_zalloc include/linux/slab.h:697 [inline]
> avc_alloc_node+0x109/0xb90 security/selinux/avc.c:572
> avc_update_node+0x172/0x1ee0 security/selinux/avc.c:859
> avc_denied+0x312/0x360 security/selinux/avc.c:1024
> avc_has_perm_noaudit+0x733/0x770 security/selinux/avc.c:1155
> avc_has_perm+0x172/0x480 security/selinux/avc.c:1184
> sock_has_perm security/selinux/hooks.c:4539 [inline]
> selinux_socket_sendmsg+0x297/0x360 security/selinux/hooks.c:4875
> security_socket_sendmsg+0x127/0x200 security/security.c:1410
> sock_sendmsg net/socket.c:628 [inline]
> ___sys_sendmsg+0xd5f/0x1290 net/socket.c:2116
> __sys_sendmsg net/socket.c:2154 [inline]
> __do_sys_sendmsg net/socket.c:2163 [inline]
> __se_sys_sendmsg+0x307/0x460 net/socket.c:2161
> __x64_sys_sendmsg+0x4a/0x70 net/socket.c:2161
> do_syscall_64+0xb8/0x100 arch/x86/entry/common.c:291
> entry_SYSCALL_64_after_hwframe+0x63/0xe7
> =========================================================
>
> Crash log 2
> =========================================================
> BUG: KMSAN: uninit-value in rcu_accelerate_cbs+0x821/0x990
> kernel/rcu/tree.c:1728
> CPU: 0 PID: 10 Comm: rcu_sched Not tainted 4.19.0-rc8+ #18
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
> __dump_stack lib/dump_stack.c:77 [inline]
> dump_stack+0x305/0x460 lib/dump_stack.c:113
> kmsan_report+0x1a2/0x2e0 mm/kmsan/kmsan.c:917
> __msan_warning+0x7d/0xe0 mm/kmsan/kmsan_instr.c:500
> rcu_accelerate_cbs+0x821/0x990 kernel/rcu/tree.c:1728
> __note_gp_changes+0x2ac/0x9e0 kernel/rcu/tree.c:1807
> rcu_gp_cleanup kernel/rcu/tree.c:2109 [inline]
> rcu_gp_kthread+0x3019/0x3990 kernel/rcu/tree.c:2236
> kthread+0x5b1/0x5f0 kernel/kthread.c:247
> ret_from_fork+0x35/0x40 arch/x86/entry/entry_64.S:416
>
> Uninit was created at:
> kmsan_save_stack_with_flags mm/kmsan/kmsan.c:255 [inline]
> kmsan_internal_alloc_meta_for_pages+0x157/0x730 mm/kmsan/kmsan.c:693
> kmsan_alloc_page+0x80/0xe0 mm/kmsan/kmsan_hooks.c:320
> __alloc_pages_nodemask+0x128c/0x69b0 mm/page_alloc.c:4416
> alloc_pages_current+0x51f/0x760 mm/mempolicy.c:2093
> alloc_pages include/linux/gfp.h:511 [inline]
> alloc_slab_page mm/slub.c:1459 [inline]
> allocate_slab mm/slub.c:1604 [inline]
> new_slab+0x552/0x1f30 mm/slub.c:1675
> new_slab_objects mm/slub.c:2438 [inline]
> ___slab_alloc+0x1414/0x1dd0 mm/slub.c:2590
> __slab_alloc mm/slub.c:2630 [inline]
> slab_alloc_node mm/slub.c:2693 [inline]
> slab_alloc mm/slub.c:2735 [inline]
> kmem_cache_alloc+0xc9b/0xda0 mm/slub.c:2740
> kmem_cache_zalloc include/linux/slab.h:697 [inline]
> avc_alloc_node+0x109/0xb90 security/selinux/avc.c:572
> avc_insert security/selinux/avc.c:696 [inline]
> avc_compute_av+0x31e/0x1050 security/selinux/avc.c:1008
> avc_has_perm_noaudit+0x516/0x770 security/selinux/avc.c:1149
> avc_has_perm+0x172/0x480 security/selinux/avc.c:1184
> selinux_socket_create+0x248/0x3c0 security/selinux/hooks.c:4560
> security_socket_create+0x146/0x210 security/security.c:1372
> __sock_create+0x26b/0xf30 net/socket.c:1232
> sock_create net/socket.c:1317 [inline]
> __sys_socket+0x180/0x670 net/socket.c:1347
> __do_sys_socket net/socket.c:1356 [inline]
> __se_sys_socket+0x8d/0xb0 net/socket.c:1354
> __x64_sys_socket+0x4a/0x70 net/socket.c:1354
> do_syscall_64+0xb8/0x100 arch/x86/entry/common.c:291
> entry_SYSCALL_64_after_hwframe+0x63/0xe7
> =========================================================
>
> Thanks,
> Kyungtae Kim
>
On Wed, Nov 14, 2018 at 4:09 PM Paul E. McKenney <[email protected]> wrote:
>
> On Wed, Nov 14, 2018 at 04:03:33AM -0500, Kyungtae Kim wrote:
> > We report two crashes in v4.19-rc8 (4.20-rc1 as well, I guess):
> > (Unfortunately, there is no repro for those.)
> >
> > The two crashes seem to share the same issue.
> > In both cases, (uninitialized) memory access violation occurs
> > when "rdp->cblist" is about to be accessed (kernel/rcu/tree.c:2838,1728).
> > I guess those are freed before the use, but I still haven't figured
> > out the reason why.
> > I'm looking forward to some help.
First of all, I'd avoid reporting KMSAN bugs without clear reproducers.
The tool is still in beta and may still give false positives due to
either missed initialization or rare memory corruptions.
> You lost me on this one. In both cases, rdp references a per-CPU
> variable that is implicitly initialized to all zeroes, due to being
> (sort of) a C-language global.
>
> If a callback is queued early, then the following lines in __call_rcu()
> will make an honest list of that field because of the :
>
> if (rcu_segcblist_empty(&rdp->cblist))
> rcu_segcblist_init(&rdp->cblist);
>
> Otherwise, when rcu_init() is invoked during early boot, we have this
> in rcu_init_percpu_data(), which is called from rcutree_prepare_cpu()
> which is called from rcu_init(), which is called from start_kernel():
>
> if (rcu_segcblist_empty(&rdp->cblist) && /* No early-boot CBs? */
> !init_nocb_callback_list(rdp))
> rcu_segcblist_init(&rdp->cblist); /* Re-enable callbacks. */
>
> So either init_nocb_callback_list() initializes the alternative callback
> lists for a no-CBs CPU or rcu_segcblist_init() again makes an honest
> list of that field.
>
> My guess is that your tool is missing the
>
> rdp = this_cpu_ptr(rsp->rda);
>
> in the __call_rcu() case, and also missing the
>
> struct rcu_data *rdp = per_cpu_ptr(rsp->rda, cpu);
>
> Note that the ->rda field is explicitly compile-time initialized to
> the base address of the per-CPU variable, which is rcu_preempt_data,
> rcu_bh_data, or rcu_sched_data, depending on which RCU flavor is at hand.
> (In v4.20-rc1, these are all merged into a single flavor to rule them all.)
>
> Alternatively, your tool might be missing the implicit initialization
> of per-CPU variables.
This used to be fine, but after rebasing to v4.20-rc2 I also started
seeing strange reports on per-CPU variables. Taking a look.
> Or maybe I am missing something. If so, please let me know what it is.
>
> Thanx, Paul
>
> > Crash log 1
> > =========================================
> > BUG: KMSAN: uninit-value in __rcu_process_callbacks
> > kernel/rcu/tree.c:2838 [inline]
> > BUG: KMSAN: uninit-value in rcu_process_callbacks+0x5ac/0x1cb0
> > kernel/rcu/tree.c:2864
> > CPU: 0 PID: 20 Comm: kauditd Not tainted 4.19.0-rc8+ #18
> > Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> > Call Trace:
> > <IRQ>
> > __dump_stack lib/dump_stack.c:77 [inline]
> > dump_stack+0x305/0x460 lib/dump_stack.c:113
> > kmsan_report+0x1a2/0x2e0 mm/kmsan/kmsan.c:917
> > __msan_warning+0x7d/0xe0 mm/kmsan/kmsan_instr.c:500
> > __rcu_process_callbacks kernel/rcu/tree.c:2838 [inline]
> > rcu_process_callbacks+0x5ac/0x1cb0 kernel/rcu/tree.c:2864
> > __do_softirq+0x5ff/0xa55 kernel/softirq.c:292
> > invoke_softirq kernel/softirq.c:373 [inline]
> > irq_exit+0x22d/0x270 kernel/softirq.c:414
> > exiting_irq+0xe/0x10 arch/x86/include/asm/apic.h:536
> > smp_apic_timer_interrupt+0x64/0x90 arch/x86/kernel/apic/apic.c:1059
> > apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:869
> > </IRQ>
> > RIP: 0010:finish_lock_switch+0x2b/0x40 kernel/sched/core.c:2578
> > Code: 48 89 e5 53 48 89 fb e8 e3 43 9a 00 8b b8 88 0c 00 00 48 8b 00
> > 48 85 c0 75 12 48 89 df e8 7d 38 9a 00 c6 00 00 c6 03 00 fb 5b <5d> c3
> > e8 de 42 9a 00 eb e7 66 66 66 2e 0f 1f 84 00 00 00 00 00 55
> > RSP: 0018:ffff88010622fca0 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff13
> > RAX: ffff8801105bcc40 RBX: ffff8801061554c0 RCX: ffff8801105bdc40
> > RDX: ffff8801105bdc40 RSI: aaaaaaaaaaaab000 RDI: ffffea00077ec560
> > RBP: ffff88010622fca0 R08: ffffffff7fffffff R09: 0000000000000002
> > R10: 0000000000000000 R11: 0000000000000000 R12: ffff8800751cb880
> > R13: 0000000000000000 R14: ffff880106155db8 R15: ffff88013fcb9c40
> > finish_task_switch+0xe3/0x270 kernel/sched/core.c:2679
> > context_switch kernel/sched/core.c:2832 [inline]
> > __schedule+0x78f/0x8f0 kernel/sched/core.c:3479
> > schedule+0x1cc/0x300 kernel/sched/core.c:3523
> > kauditd_thread+0xc64/0xee0 kernel/audit.c:889
> > kthread+0x5b1/0x5f0 kernel/kthread.c:247
> > ret_from_fork+0x35/0x40 arch/x86/entry/entry_64.S:416
> >
> > Uninit was created at:
> > kmsan_save_stack_with_flags mm/kmsan/kmsan.c:255 [inline]
> > kmsan_internal_alloc_meta_for_pages+0x157/0x730 mm/kmsan/kmsan.c:693
> > kmsan_alloc_page+0x80/0xe0 mm/kmsan/kmsan_hooks.c:320
> > __alloc_pages_nodemask+0x128c/0x69b0 mm/page_alloc.c:4416
> > alloc_pages_current+0x51f/0x760 mm/mempolicy.c:2093
> > alloc_pages include/linux/gfp.h:511 [inline]
> > alloc_slab_page mm/slub.c:1459 [inline]
> > allocate_slab mm/slub.c:1604 [inline]
> > new_slab+0x552/0x1f30 mm/slub.c:1675
> > new_slab_objects mm/slub.c:2438 [inline]
> > ___slab_alloc+0x1414/0x1dd0 mm/slub.c:2590
> > __slab_alloc mm/slub.c:2630 [inline]
> > slab_alloc_node mm/slub.c:2693 [inline]
> > slab_alloc mm/slub.c:2735 [inline]
> > kmem_cache_alloc+0xc9b/0xda0 mm/slub.c:2740
> > kmem_cache_zalloc include/linux/slab.h:697 [inline]
> > avc_alloc_node+0x109/0xb90 security/selinux/avc.c:572
> > avc_update_node+0x172/0x1ee0 security/selinux/avc.c:859
> > avc_denied+0x312/0x360 security/selinux/avc.c:1024
> > avc_has_perm_noaudit+0x733/0x770 security/selinux/avc.c:1155
> > avc_has_perm+0x172/0x480 security/selinux/avc.c:1184
> > sock_has_perm security/selinux/hooks.c:4539 [inline]
> > selinux_socket_sendmsg+0x297/0x360 security/selinux/hooks.c:4875
> > security_socket_sendmsg+0x127/0x200 security/security.c:1410
> > sock_sendmsg net/socket.c:628 [inline]
> > ___sys_sendmsg+0xd5f/0x1290 net/socket.c:2116
> > __sys_sendmsg net/socket.c:2154 [inline]
> > __do_sys_sendmsg net/socket.c:2163 [inline]
> > __se_sys_sendmsg+0x307/0x460 net/socket.c:2161
> > __x64_sys_sendmsg+0x4a/0x70 net/socket.c:2161
> > do_syscall_64+0xb8/0x100 arch/x86/entry/common.c:291
> > entry_SYSCALL_64_after_hwframe+0x63/0xe7
> > =========================================================
> >
> > Crash log 2
> > =========================================================
> > BUG: KMSAN: uninit-value in rcu_accelerate_cbs+0x821/0x990
> > kernel/rcu/tree.c:1728
> > CPU: 0 PID: 10 Comm: rcu_sched Not tainted 4.19.0-rc8+ #18
> > Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> > Call Trace:
> > __dump_stack lib/dump_stack.c:77 [inline]
> > dump_stack+0x305/0x460 lib/dump_stack.c:113
> > kmsan_report+0x1a2/0x2e0 mm/kmsan/kmsan.c:917
> > __msan_warning+0x7d/0xe0 mm/kmsan/kmsan_instr.c:500
> > rcu_accelerate_cbs+0x821/0x990 kernel/rcu/tree.c:1728
> > __note_gp_changes+0x2ac/0x9e0 kernel/rcu/tree.c:1807
> > rcu_gp_cleanup kernel/rcu/tree.c:2109 [inline]
> > rcu_gp_kthread+0x3019/0x3990 kernel/rcu/tree.c:2236
> > kthread+0x5b1/0x5f0 kernel/kthread.c:247
> > ret_from_fork+0x35/0x40 arch/x86/entry/entry_64.S:416
> >
> > Uninit was created at:
> > kmsan_save_stack_with_flags mm/kmsan/kmsan.c:255 [inline]
> > kmsan_internal_alloc_meta_for_pages+0x157/0x730 mm/kmsan/kmsan.c:693
> > kmsan_alloc_page+0x80/0xe0 mm/kmsan/kmsan_hooks.c:320
> > __alloc_pages_nodemask+0x128c/0x69b0 mm/page_alloc.c:4416
> > alloc_pages_current+0x51f/0x760 mm/mempolicy.c:2093
> > alloc_pages include/linux/gfp.h:511 [inline]
> > alloc_slab_page mm/slub.c:1459 [inline]
> > allocate_slab mm/slub.c:1604 [inline]
> > new_slab+0x552/0x1f30 mm/slub.c:1675
> > new_slab_objects mm/slub.c:2438 [inline]
> > ___slab_alloc+0x1414/0x1dd0 mm/slub.c:2590
> > __slab_alloc mm/slub.c:2630 [inline]
> > slab_alloc_node mm/slub.c:2693 [inline]
> > slab_alloc mm/slub.c:2735 [inline]
> > kmem_cache_alloc+0xc9b/0xda0 mm/slub.c:2740
> > kmem_cache_zalloc include/linux/slab.h:697 [inline]
> > avc_alloc_node+0x109/0xb90 security/selinux/avc.c:572
> > avc_insert security/selinux/avc.c:696 [inline]
> > avc_compute_av+0x31e/0x1050 security/selinux/avc.c:1008
> > avc_has_perm_noaudit+0x516/0x770 security/selinux/avc.c:1149
> > avc_has_perm+0x172/0x480 security/selinux/avc.c:1184
> > selinux_socket_create+0x248/0x3c0 security/selinux/hooks.c:4560
> > security_socket_create+0x146/0x210 security/security.c:1372
> > __sock_create+0x26b/0xf30 net/socket.c:1232
> > sock_create net/socket.c:1317 [inline]
> > __sys_socket+0x180/0x670 net/socket.c:1347
> > __do_sys_socket net/socket.c:1356 [inline]
> > __se_sys_socket+0x8d/0xb0 net/socket.c:1354
> > __x64_sys_socket+0x4a/0x70 net/socket.c:1354
> > do_syscall_64+0xb8/0x100 arch/x86/entry/common.c:291
> > entry_SYSCALL_64_after_hwframe+0x63/0xe7
> > =========================================================
> >
> > Thanks,
> > Kyungtae Kim
> >
>
> --
> You received this message because you are subscribed to the Google Groups "syzkaller" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
> For more options, visit https://groups.google.com/d/optout.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
On Wed, Nov 14, 2018 at 04:31:11PM +0100, Alexander Potapenko wrote:
> On Wed, Nov 14, 2018 at 4:09 PM Paul E. McKenney <[email protected]> wrote:
> >
> > On Wed, Nov 14, 2018 at 04:03:33AM -0500, Kyungtae Kim wrote:
> > > We report two crashes in v4.19-rc8 (4.20-rc1 as well, I guess):
> > > (Unfortunately, there is no repro for those.)
> > >
> > > The two crashes seem to share the same issue.
> > > In both cases, (uninitialized) memory access violation occurs
> > > when "rdp->cblist" is about to be accessed (kernel/rcu/tree.c:2838,1728).
> > > I guess those are freed before the use, but I still haven't figured
> > > out the reason why.
> > > I'm looking forward to some help.
>
> First of all, I'd avoid reporting KMSAN bugs without clear reproducers.
> The tool is still in beta and may still give false positives due to
> either missed initialization or rare memory corruptions.
OK, I will set this aside, then, thank you!
Thanx, Paul
> > You lost me on this one. In both cases, rdp references a per-CPU
> > variable that is implicitly initialized to all zeroes, due to being
> > (sort of) a C-language global.
> >
> > If a callback is queued early, then the following lines in __call_rcu()
> > will make an honest list of that field because of the :
> >
> > if (rcu_segcblist_empty(&rdp->cblist))
> > rcu_segcblist_init(&rdp->cblist);
> >
> > Otherwise, when rcu_init() is invoked during early boot, we have this
> > in rcu_init_percpu_data(), which is called from rcutree_prepare_cpu()
> > which is called from rcu_init(), which is called from start_kernel():
> >
> > if (rcu_segcblist_empty(&rdp->cblist) && /* No early-boot CBs? */
> > !init_nocb_callback_list(rdp))
> > rcu_segcblist_init(&rdp->cblist); /* Re-enable callbacks. */
> >
> > So either init_nocb_callback_list() initializes the alternative callback
> > lists for a no-CBs CPU or rcu_segcblist_init() again makes an honest
> > list of that field.
> >
> > My guess is that your tool is missing the
> >
> > rdp = this_cpu_ptr(rsp->rda);
> >
> > in the __call_rcu() case, and also missing the
> >
> > struct rcu_data *rdp = per_cpu_ptr(rsp->rda, cpu);
> >
> > Note that the ->rda field is explicitly compile-time initialized to
> > the base address of the per-CPU variable, which is rcu_preempt_data,
> > rcu_bh_data, or rcu_sched_data, depending on which RCU flavor is at hand.
> > (In v4.20-rc1, these are all merged into a single flavor to rule them all.)
> >
> > Alternatively, your tool might be missing the implicit initialization
> > of per-CPU variables.
> This used to be fine, but after rebasing to v4.20-rc2 I also started
> seeing strange reports on per-CPU variables. Taking a look.
> > Or maybe I am missing something. If so, please let me know what it is.
> >
> > Thanx, Paul
> >
> > > Crash log 1
> > > =========================================
> > > BUG: KMSAN: uninit-value in __rcu_process_callbacks
> > > kernel/rcu/tree.c:2838 [inline]
> > > BUG: KMSAN: uninit-value in rcu_process_callbacks+0x5ac/0x1cb0
> > > kernel/rcu/tree.c:2864
> > > CPU: 0 PID: 20 Comm: kauditd Not tainted 4.19.0-rc8+ #18
> > > Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> > > Call Trace:
> > > <IRQ>
> > > __dump_stack lib/dump_stack.c:77 [inline]
> > > dump_stack+0x305/0x460 lib/dump_stack.c:113
> > > kmsan_report+0x1a2/0x2e0 mm/kmsan/kmsan.c:917
> > > __msan_warning+0x7d/0xe0 mm/kmsan/kmsan_instr.c:500
> > > __rcu_process_callbacks kernel/rcu/tree.c:2838 [inline]
> > > rcu_process_callbacks+0x5ac/0x1cb0 kernel/rcu/tree.c:2864
> > > __do_softirq+0x5ff/0xa55 kernel/softirq.c:292
> > > invoke_softirq kernel/softirq.c:373 [inline]
> > > irq_exit+0x22d/0x270 kernel/softirq.c:414
> > > exiting_irq+0xe/0x10 arch/x86/include/asm/apic.h:536
> > > smp_apic_timer_interrupt+0x64/0x90 arch/x86/kernel/apic/apic.c:1059
> > > apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:869
> > > </IRQ>
> > > RIP: 0010:finish_lock_switch+0x2b/0x40 kernel/sched/core.c:2578
> > > Code: 48 89 e5 53 48 89 fb e8 e3 43 9a 00 8b b8 88 0c 00 00 48 8b 00
> > > 48 85 c0 75 12 48 89 df e8 7d 38 9a 00 c6 00 00 c6 03 00 fb 5b <5d> c3
> > > e8 de 42 9a 00 eb e7 66 66 66 2e 0f 1f 84 00 00 00 00 00 55
> > > RSP: 0018:ffff88010622fca0 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff13
> > > RAX: ffff8801105bcc40 RBX: ffff8801061554c0 RCX: ffff8801105bdc40
> > > RDX: ffff8801105bdc40 RSI: aaaaaaaaaaaab000 RDI: ffffea00077ec560
> > > RBP: ffff88010622fca0 R08: ffffffff7fffffff R09: 0000000000000002
> > > R10: 0000000000000000 R11: 0000000000000000 R12: ffff8800751cb880
> > > R13: 0000000000000000 R14: ffff880106155db8 R15: ffff88013fcb9c40
> > > finish_task_switch+0xe3/0x270 kernel/sched/core.c:2679
> > > context_switch kernel/sched/core.c:2832 [inline]
> > > __schedule+0x78f/0x8f0 kernel/sched/core.c:3479
> > > schedule+0x1cc/0x300 kernel/sched/core.c:3523
> > > kauditd_thread+0xc64/0xee0 kernel/audit.c:889
> > > kthread+0x5b1/0x5f0 kernel/kthread.c:247
> > > ret_from_fork+0x35/0x40 arch/x86/entry/entry_64.S:416
> > >
> > > Uninit was created at:
> > > kmsan_save_stack_with_flags mm/kmsan/kmsan.c:255 [inline]
> > > kmsan_internal_alloc_meta_for_pages+0x157/0x730 mm/kmsan/kmsan.c:693
> > > kmsan_alloc_page+0x80/0xe0 mm/kmsan/kmsan_hooks.c:320
> > > __alloc_pages_nodemask+0x128c/0x69b0 mm/page_alloc.c:4416
> > > alloc_pages_current+0x51f/0x760 mm/mempolicy.c:2093
> > > alloc_pages include/linux/gfp.h:511 [inline]
> > > alloc_slab_page mm/slub.c:1459 [inline]
> > > allocate_slab mm/slub.c:1604 [inline]
> > > new_slab+0x552/0x1f30 mm/slub.c:1675
> > > new_slab_objects mm/slub.c:2438 [inline]
> > > ___slab_alloc+0x1414/0x1dd0 mm/slub.c:2590
> > > __slab_alloc mm/slub.c:2630 [inline]
> > > slab_alloc_node mm/slub.c:2693 [inline]
> > > slab_alloc mm/slub.c:2735 [inline]
> > > kmem_cache_alloc+0xc9b/0xda0 mm/slub.c:2740
> > > kmem_cache_zalloc include/linux/slab.h:697 [inline]
> > > avc_alloc_node+0x109/0xb90 security/selinux/avc.c:572
> > > avc_update_node+0x172/0x1ee0 security/selinux/avc.c:859
> > > avc_denied+0x312/0x360 security/selinux/avc.c:1024
> > > avc_has_perm_noaudit+0x733/0x770 security/selinux/avc.c:1155
> > > avc_has_perm+0x172/0x480 security/selinux/avc.c:1184
> > > sock_has_perm security/selinux/hooks.c:4539 [inline]
> > > selinux_socket_sendmsg+0x297/0x360 security/selinux/hooks.c:4875
> > > security_socket_sendmsg+0x127/0x200 security/security.c:1410
> > > sock_sendmsg net/socket.c:628 [inline]
> > > ___sys_sendmsg+0xd5f/0x1290 net/socket.c:2116
> > > __sys_sendmsg net/socket.c:2154 [inline]
> > > __do_sys_sendmsg net/socket.c:2163 [inline]
> > > __se_sys_sendmsg+0x307/0x460 net/socket.c:2161
> > > __x64_sys_sendmsg+0x4a/0x70 net/socket.c:2161
> > > do_syscall_64+0xb8/0x100 arch/x86/entry/common.c:291
> > > entry_SYSCALL_64_after_hwframe+0x63/0xe7
> > > =========================================================
> > >
> > > Crash log 2
> > > =========================================================
> > > BUG: KMSAN: uninit-value in rcu_accelerate_cbs+0x821/0x990
> > > kernel/rcu/tree.c:1728
> > > CPU: 0 PID: 10 Comm: rcu_sched Not tainted 4.19.0-rc8+ #18
> > > Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> > > Call Trace:
> > > __dump_stack lib/dump_stack.c:77 [inline]
> > > dump_stack+0x305/0x460 lib/dump_stack.c:113
> > > kmsan_report+0x1a2/0x2e0 mm/kmsan/kmsan.c:917
> > > __msan_warning+0x7d/0xe0 mm/kmsan/kmsan_instr.c:500
> > > rcu_accelerate_cbs+0x821/0x990 kernel/rcu/tree.c:1728
> > > __note_gp_changes+0x2ac/0x9e0 kernel/rcu/tree.c:1807
> > > rcu_gp_cleanup kernel/rcu/tree.c:2109 [inline]
> > > rcu_gp_kthread+0x3019/0x3990 kernel/rcu/tree.c:2236
> > > kthread+0x5b1/0x5f0 kernel/kthread.c:247
> > > ret_from_fork+0x35/0x40 arch/x86/entry/entry_64.S:416
> > >
> > > Uninit was created at:
> > > kmsan_save_stack_with_flags mm/kmsan/kmsan.c:255 [inline]
> > > kmsan_internal_alloc_meta_for_pages+0x157/0x730 mm/kmsan/kmsan.c:693
> > > kmsan_alloc_page+0x80/0xe0 mm/kmsan/kmsan_hooks.c:320
> > > __alloc_pages_nodemask+0x128c/0x69b0 mm/page_alloc.c:4416
> > > alloc_pages_current+0x51f/0x760 mm/mempolicy.c:2093
> > > alloc_pages include/linux/gfp.h:511 [inline]
> > > alloc_slab_page mm/slub.c:1459 [inline]
> > > allocate_slab mm/slub.c:1604 [inline]
> > > new_slab+0x552/0x1f30 mm/slub.c:1675
> > > new_slab_objects mm/slub.c:2438 [inline]
> > > ___slab_alloc+0x1414/0x1dd0 mm/slub.c:2590
> > > __slab_alloc mm/slub.c:2630 [inline]
> > > slab_alloc_node mm/slub.c:2693 [inline]
> > > slab_alloc mm/slub.c:2735 [inline]
> > > kmem_cache_alloc+0xc9b/0xda0 mm/slub.c:2740
> > > kmem_cache_zalloc include/linux/slab.h:697 [inline]
> > > avc_alloc_node+0x109/0xb90 security/selinux/avc.c:572
> > > avc_insert security/selinux/avc.c:696 [inline]
> > > avc_compute_av+0x31e/0x1050 security/selinux/avc.c:1008
> > > avc_has_perm_noaudit+0x516/0x770 security/selinux/avc.c:1149
> > > avc_has_perm+0x172/0x480 security/selinux/avc.c:1184
> > > selinux_socket_create+0x248/0x3c0 security/selinux/hooks.c:4560
> > > security_socket_create+0x146/0x210 security/security.c:1372
> > > __sock_create+0x26b/0xf30 net/socket.c:1232
> > > sock_create net/socket.c:1317 [inline]
> > > __sys_socket+0x180/0x670 net/socket.c:1347
> > > __do_sys_socket net/socket.c:1356 [inline]
> > > __se_sys_socket+0x8d/0xb0 net/socket.c:1354
> > > __x64_sys_socket+0x4a/0x70 net/socket.c:1354
> > > do_syscall_64+0xb8/0x100 arch/x86/entry/common.c:291
> > > entry_SYSCALL_64_after_hwframe+0x63/0xe7
> > > =========================================================
> > >
> > > Thanks,
> > > Kyungtae Kim
> > >
> >
> > --
> > You received this message because you are subscribed to the Google Groups "syzkaller" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
> > For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> Alexander Potapenko
> Software Engineer
>
> Google Germany GmbH
> Erika-Mann-Stra?e, 33
> 80636 M?nchen
>
> Gesch?ftsf?hrer: Paul Manicle, Halimah DeLaine Prado
> Registergericht und -nummer: Hamburg, HRB 86891
> Sitz der Gesellschaft: Hamburg
>
Thank you for all your comments.
Thanks,
Kyungtae Kim
On Wed, Nov 14, 2018 at 11:05 AM Paul E. McKenney <[email protected]> wrote:
>
> On Wed, Nov 14, 2018 at 04:31:11PM +0100, Alexander Potapenko wrote:
> > On Wed, Nov 14, 2018 at 4:09 PM Paul E. McKenney <[email protected]> wrote:
> > >
> > > On Wed, Nov 14, 2018 at 04:03:33AM -0500, Kyungtae Kim wrote:
> > > > We report two crashes in v4.19-rc8 (4.20-rc1 as well, I guess):
> > > > (Unfortunately, there is no repro for those.)
> > > >
> > > > The two crashes seem to share the same issue.
> > > > In both cases, (uninitialized) memory access violation occurs
> > > > when "rdp->cblist" is about to be accessed (kernel/rcu/tree.c:2838,1728).
> > > > I guess those are freed before the use, but I still haven't figured
> > > > out the reason why.
> > > > I'm looking forward to some help.
> >
> > First of all, I'd avoid reporting KMSAN bugs without clear reproducers.
> > The tool is still in beta and may still give false positives due to
> > either missed initialization or rare memory corruptions.
>
> OK, I will set this aside, then, thank you!
>
> Thanx, Paul
>
> > > You lost me on this one. In both cases, rdp references a per-CPU
> > > variable that is implicitly initialized to all zeroes, due to being
> > > (sort of) a C-language global.
> > >
> > > If a callback is queued early, then the following lines in __call_rcu()
> > > will make an honest list of that field because of the :
> > >
> > > if (rcu_segcblist_empty(&rdp->cblist))
> > > rcu_segcblist_init(&rdp->cblist);
> > >
> > > Otherwise, when rcu_init() is invoked during early boot, we have this
> > > in rcu_init_percpu_data(), which is called from rcutree_prepare_cpu()
> > > which is called from rcu_init(), which is called from start_kernel():
> > >
> > > if (rcu_segcblist_empty(&rdp->cblist) && /* No early-boot CBs? */
> > > !init_nocb_callback_list(rdp))
> > > rcu_segcblist_init(&rdp->cblist); /* Re-enable callbacks. */
> > >
> > > So either init_nocb_callback_list() initializes the alternative callback
> > > lists for a no-CBs CPU or rcu_segcblist_init() again makes an honest
> > > list of that field.
> > >
> > > My guess is that your tool is missing the
> > >
> > > rdp = this_cpu_ptr(rsp->rda);
> > >
> > > in the __call_rcu() case, and also missing the
> > >
> > > struct rcu_data *rdp = per_cpu_ptr(rsp->rda, cpu);
> > >
> > > Note that the ->rda field is explicitly compile-time initialized to
> > > the base address of the per-CPU variable, which is rcu_preempt_data,
> > > rcu_bh_data, or rcu_sched_data, depending on which RCU flavor is at hand.
> > > (In v4.20-rc1, these are all merged into a single flavor to rule them all.)
> > >
> > > Alternatively, your tool might be missing the implicit initialization
> > > of per-CPU variables.
> > This used to be fine, but after rebasing to v4.20-rc2 I also started
> > seeing strange reports on per-CPU variables. Taking a look.
> > > Or maybe I am missing something. If so, please let me know what it is.
> > >
> > > Thanx, Paul
> > >
> > > > Crash log 1
> > > > =========================================
> > > > BUG: KMSAN: uninit-value in __rcu_process_callbacks
> > > > kernel/rcu/tree.c:2838 [inline]
> > > > BUG: KMSAN: uninit-value in rcu_process_callbacks+0x5ac/0x1cb0
> > > > kernel/rcu/tree.c:2864
> > > > CPU: 0 PID: 20 Comm: kauditd Not tainted 4.19.0-rc8+ #18
> > > > Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> > > > Call Trace:
> > > > <IRQ>
> > > > __dump_stack lib/dump_stack.c:77 [inline]
> > > > dump_stack+0x305/0x460 lib/dump_stack.c:113
> > > > kmsan_report+0x1a2/0x2e0 mm/kmsan/kmsan.c:917
> > > > __msan_warning+0x7d/0xe0 mm/kmsan/kmsan_instr.c:500
> > > > __rcu_process_callbacks kernel/rcu/tree.c:2838 [inline]
> > > > rcu_process_callbacks+0x5ac/0x1cb0 kernel/rcu/tree.c:2864
> > > > __do_softirq+0x5ff/0xa55 kernel/softirq.c:292
> > > > invoke_softirq kernel/softirq.c:373 [inline]
> > > > irq_exit+0x22d/0x270 kernel/softirq.c:414
> > > > exiting_irq+0xe/0x10 arch/x86/include/asm/apic.h:536
> > > > smp_apic_timer_interrupt+0x64/0x90 arch/x86/kernel/apic/apic.c:1059
> > > > apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:869
> > > > </IRQ>
> > > > RIP: 0010:finish_lock_switch+0x2b/0x40 kernel/sched/core.c:2578
> > > > Code: 48 89 e5 53 48 89 fb e8 e3 43 9a 00 8b b8 88 0c 00 00 48 8b 00
> > > > 48 85 c0 75 12 48 89 df e8 7d 38 9a 00 c6 00 00 c6 03 00 fb 5b <5d> c3
> > > > e8 de 42 9a 00 eb e7 66 66 66 2e 0f 1f 84 00 00 00 00 00 55
> > > > RSP: 0018:ffff88010622fca0 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff13
> > > > RAX: ffff8801105bcc40 RBX: ffff8801061554c0 RCX: ffff8801105bdc40
> > > > RDX: ffff8801105bdc40 RSI: aaaaaaaaaaaab000 RDI: ffffea00077ec560
> > > > RBP: ffff88010622fca0 R08: ffffffff7fffffff R09: 0000000000000002
> > > > R10: 0000000000000000 R11: 0000000000000000 R12: ffff8800751cb880
> > > > R13: 0000000000000000 R14: ffff880106155db8 R15: ffff88013fcb9c40
> > > > finish_task_switch+0xe3/0x270 kernel/sched/core.c:2679
> > > > context_switch kernel/sched/core.c:2832 [inline]
> > > > __schedule+0x78f/0x8f0 kernel/sched/core.c:3479
> > > > schedule+0x1cc/0x300 kernel/sched/core.c:3523
> > > > kauditd_thread+0xc64/0xee0 kernel/audit.c:889
> > > > kthread+0x5b1/0x5f0 kernel/kthread.c:247
> > > > ret_from_fork+0x35/0x40 arch/x86/entry/entry_64.S:416
> > > >
> > > > Uninit was created at:
> > > > kmsan_save_stack_with_flags mm/kmsan/kmsan.c:255 [inline]
> > > > kmsan_internal_alloc_meta_for_pages+0x157/0x730 mm/kmsan/kmsan.c:693
> > > > kmsan_alloc_page+0x80/0xe0 mm/kmsan/kmsan_hooks.c:320
> > > > __alloc_pages_nodemask+0x128c/0x69b0 mm/page_alloc.c:4416
> > > > alloc_pages_current+0x51f/0x760 mm/mempolicy.c:2093
> > > > alloc_pages include/linux/gfp.h:511 [inline]
> > > > alloc_slab_page mm/slub.c:1459 [inline]
> > > > allocate_slab mm/slub.c:1604 [inline]
> > > > new_slab+0x552/0x1f30 mm/slub.c:1675
> > > > new_slab_objects mm/slub.c:2438 [inline]
> > > > ___slab_alloc+0x1414/0x1dd0 mm/slub.c:2590
> > > > __slab_alloc mm/slub.c:2630 [inline]
> > > > slab_alloc_node mm/slub.c:2693 [inline]
> > > > slab_alloc mm/slub.c:2735 [inline]
> > > > kmem_cache_alloc+0xc9b/0xda0 mm/slub.c:2740
> > > > kmem_cache_zalloc include/linux/slab.h:697 [inline]
> > > > avc_alloc_node+0x109/0xb90 security/selinux/avc.c:572
> > > > avc_update_node+0x172/0x1ee0 security/selinux/avc.c:859
> > > > avc_denied+0x312/0x360 security/selinux/avc.c:1024
> > > > avc_has_perm_noaudit+0x733/0x770 security/selinux/avc.c:1155
> > > > avc_has_perm+0x172/0x480 security/selinux/avc.c:1184
> > > > sock_has_perm security/selinux/hooks.c:4539 [inline]
> > > > selinux_socket_sendmsg+0x297/0x360 security/selinux/hooks.c:4875
> > > > security_socket_sendmsg+0x127/0x200 security/security.c:1410
> > > > sock_sendmsg net/socket.c:628 [inline]
> > > > ___sys_sendmsg+0xd5f/0x1290 net/socket.c:2116
> > > > __sys_sendmsg net/socket.c:2154 [inline]
> > > > __do_sys_sendmsg net/socket.c:2163 [inline]
> > > > __se_sys_sendmsg+0x307/0x460 net/socket.c:2161
> > > > __x64_sys_sendmsg+0x4a/0x70 net/socket.c:2161
> > > > do_syscall_64+0xb8/0x100 arch/x86/entry/common.c:291
> > > > entry_SYSCALL_64_after_hwframe+0x63/0xe7
> > > > =========================================================
> > > >
> > > > Crash log 2
> > > > =========================================================
> > > > BUG: KMSAN: uninit-value in rcu_accelerate_cbs+0x821/0x990
> > > > kernel/rcu/tree.c:1728
> > > > CPU: 0 PID: 10 Comm: rcu_sched Not tainted 4.19.0-rc8+ #18
> > > > Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> > > > Call Trace:
> > > > __dump_stack lib/dump_stack.c:77 [inline]
> > > > dump_stack+0x305/0x460 lib/dump_stack.c:113
> > > > kmsan_report+0x1a2/0x2e0 mm/kmsan/kmsan.c:917
> > > > __msan_warning+0x7d/0xe0 mm/kmsan/kmsan_instr.c:500
> > > > rcu_accelerate_cbs+0x821/0x990 kernel/rcu/tree.c:1728
> > > > __note_gp_changes+0x2ac/0x9e0 kernel/rcu/tree.c:1807
> > > > rcu_gp_cleanup kernel/rcu/tree.c:2109 [inline]
> > > > rcu_gp_kthread+0x3019/0x3990 kernel/rcu/tree.c:2236
> > > > kthread+0x5b1/0x5f0 kernel/kthread.c:247
> > > > ret_from_fork+0x35/0x40 arch/x86/entry/entry_64.S:416
> > > >
> > > > Uninit was created at:
> > > > kmsan_save_stack_with_flags mm/kmsan/kmsan.c:255 [inline]
> > > > kmsan_internal_alloc_meta_for_pages+0x157/0x730 mm/kmsan/kmsan.c:693
> > > > kmsan_alloc_page+0x80/0xe0 mm/kmsan/kmsan_hooks.c:320
> > > > __alloc_pages_nodemask+0x128c/0x69b0 mm/page_alloc.c:4416
> > > > alloc_pages_current+0x51f/0x760 mm/mempolicy.c:2093
> > > > alloc_pages include/linux/gfp.h:511 [inline]
> > > > alloc_slab_page mm/slub.c:1459 [inline]
> > > > allocate_slab mm/slub.c:1604 [inline]
> > > > new_slab+0x552/0x1f30 mm/slub.c:1675
> > > > new_slab_objects mm/slub.c:2438 [inline]
> > > > ___slab_alloc+0x1414/0x1dd0 mm/slub.c:2590
> > > > __slab_alloc mm/slub.c:2630 [inline]
> > > > slab_alloc_node mm/slub.c:2693 [inline]
> > > > slab_alloc mm/slub.c:2735 [inline]
> > > > kmem_cache_alloc+0xc9b/0xda0 mm/slub.c:2740
> > > > kmem_cache_zalloc include/linux/slab.h:697 [inline]
> > > > avc_alloc_node+0x109/0xb90 security/selinux/avc.c:572
> > > > avc_insert security/selinux/avc.c:696 [inline]
> > > > avc_compute_av+0x31e/0x1050 security/selinux/avc.c:1008
> > > > avc_has_perm_noaudit+0x516/0x770 security/selinux/avc.c:1149
> > > > avc_has_perm+0x172/0x480 security/selinux/avc.c:1184
> > > > selinux_socket_create+0x248/0x3c0 security/selinux/hooks.c:4560
> > > > security_socket_create+0x146/0x210 security/security.c:1372
> > > > __sock_create+0x26b/0xf30 net/socket.c:1232
> > > > sock_create net/socket.c:1317 [inline]
> > > > __sys_socket+0x180/0x670 net/socket.c:1347
> > > > __do_sys_socket net/socket.c:1356 [inline]
> > > > __se_sys_socket+0x8d/0xb0 net/socket.c:1354
> > > > __x64_sys_socket+0x4a/0x70 net/socket.c:1354
> > > > do_syscall_64+0xb8/0x100 arch/x86/entry/common.c:291
> > > > entry_SYSCALL_64_after_hwframe+0x63/0xe7
> > > > =========================================================
> > > >
> > > > Thanks,
> > > > Kyungtae Kim
> > > >
> > >
> > > --
> > > You received this message because you are subscribed to the Google Groups "syzkaller" group.
> > > To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
> > > For more options, visit https://groups.google.com/d/optout.
> >
> >
> >
> > --
> > Alexander Potapenko
> > Software Engineer
> >
> > Google Germany GmbH
> > Erika-Mann-Straße, 33
> > 80636 München
> >
> > Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
> > Registergericht und -nummer: Hamburg, HRB 86891
> > Sitz der Gesellschaft: Hamburg
> >
>
On Thu, Nov 15, 2018 at 5:05 AM Kyungtae Kim <[email protected]> wrote:
>
> Thank you for all your comments.
>
> Thanks,
> Kyungtae Kim
> On Wed, Nov 14, 2018 at 11:05 AM Paul E. McKenney <[email protected]> wrote:
> >
> > On Wed, Nov 14, 2018 at 04:31:11PM +0100, Alexander Potapenko wrote:
> > > On Wed, Nov 14, 2018 at 4:09 PM Paul E. McKenney <[email protected]> wrote:
> > > >
> > > > On Wed, Nov 14, 2018 at 04:03:33AM -0500, Kyungtae Kim wrote:
> > > > > We report two crashes in v4.19-rc8 (4.20-rc1 as well, I guess):
> > > > > (Unfortunately, there is no repro for those.)
> > > > >
> > > > > The two crashes seem to share the same issue.
> > > > > In both cases, (uninitialized) memory access violation occurs
> > > > > when "rdp->cblist" is about to be accessed (kernel/rcu/tree.c:2838,1728).
> > > > > I guess those are freed before the use, but I still haven't figured
> > > > > out the reason why.
> > > > > I'm looking forward to some help.
> > >
> > > First of all, I'd avoid reporting KMSAN bugs without clear reproducers.
> > > The tool is still in beta and may still give false positives due to
> > > either missed initialization or rare memory corruptions.
> >
> > OK, I will set this aside, then, thank you!
> >
> > Thanx, Paul
> >
> > > > You lost me on this one. In both cases, rdp references a per-CPU
> > > > variable that is implicitly initialized to all zeroes, due to being
> > > > (sort of) a C-language global.
> > > >
> > > > If a callback is queued early, then the following lines in __call_rcu()
> > > > will make an honest list of that field because of the :
> > > >
> > > > if (rcu_segcblist_empty(&rdp->cblist))
> > > > rcu_segcblist_init(&rdp->cblist);
> > > >
> > > > Otherwise, when rcu_init() is invoked during early boot, we have this
> > > > in rcu_init_percpu_data(), which is called from rcutree_prepare_cpu()
> > > > which is called from rcu_init(), which is called from start_kernel():
> > > >
> > > > if (rcu_segcblist_empty(&rdp->cblist) && /* No early-boot CBs? */
> > > > !init_nocb_callback_list(rdp))
> > > > rcu_segcblist_init(&rdp->cblist); /* Re-enable callbacks. */
> > > >
> > > > So either init_nocb_callback_list() initializes the alternative callback
> > > > lists for a no-CBs CPU or rcu_segcblist_init() again makes an honest
> > > > list of that field.
> > > >
> > > > My guess is that your tool is missing the
> > > >
> > > > rdp = this_cpu_ptr(rsp->rda);
> > > >
> > > > in the __call_rcu() case, and also missing the
> > > >
> > > > struct rcu_data *rdp = per_cpu_ptr(rsp->rda, cpu);
> > > >
> > > > Note that the ->rda field is explicitly compile-time initialized to
> > > > the base address of the per-CPU variable, which is rcu_preempt_data,
> > > > rcu_bh_data, or rcu_sched_data, depending on which RCU flavor is at hand.
> > > > (In v4.20-rc1, these are all merged into a single flavor to rule them all.)
> > > >
> > > > Alternatively, your tool might be missing the implicit initialization
> > > > of per-CPU variables.
> > > This used to be fine, but after rebasing to v4.20-rc2 I also started
> > > seeing strange reports on per-CPU variables. Taking a look.
No, this was a bug in kmsan_check_memory(), which isn't involved in
the bug reported by Kyungtae Kim.
Therefore I'm assuming per-CPU variables are innocent, but would still
love to see a reproducer for the bug.
> > > > Or maybe I am missing something. If so, please let me know what it is.
> > > >
> > > > Thanx, Paul
> > > >
> > > > > Crash log 1
> > > > > =========================================
> > > > > BUG: KMSAN: uninit-value in __rcu_process_callbacks
> > > > > kernel/rcu/tree.c:2838 [inline]
> > > > > BUG: KMSAN: uninit-value in rcu_process_callbacks+0x5ac/0x1cb0
> > > > > kernel/rcu/tree.c:2864
> > > > > CPU: 0 PID: 20 Comm: kauditd Not tainted 4.19.0-rc8+ #18
> > > > > Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> > > > > Call Trace:
> > > > > <IRQ>
> > > > > __dump_stack lib/dump_stack.c:77 [inline]
> > > > > dump_stack+0x305/0x460 lib/dump_stack.c:113
> > > > > kmsan_report+0x1a2/0x2e0 mm/kmsan/kmsan.c:917
> > > > > __msan_warning+0x7d/0xe0 mm/kmsan/kmsan_instr.c:500
> > > > > __rcu_process_callbacks kernel/rcu/tree.c:2838 [inline]
> > > > > rcu_process_callbacks+0x5ac/0x1cb0 kernel/rcu/tree.c:2864
> > > > > __do_softirq+0x5ff/0xa55 kernel/softirq.c:292
> > > > > invoke_softirq kernel/softirq.c:373 [inline]
> > > > > irq_exit+0x22d/0x270 kernel/softirq.c:414
> > > > > exiting_irq+0xe/0x10 arch/x86/include/asm/apic.h:536
> > > > > smp_apic_timer_interrupt+0x64/0x90 arch/x86/kernel/apic/apic.c:1059
> > > > > apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:869
> > > > > </IRQ>
> > > > > RIP: 0010:finish_lock_switch+0x2b/0x40 kernel/sched/core.c:2578
> > > > > Code: 48 89 e5 53 48 89 fb e8 e3 43 9a 00 8b b8 88 0c 00 00 48 8b 00
> > > > > 48 85 c0 75 12 48 89 df e8 7d 38 9a 00 c6 00 00 c6 03 00 fb 5b <5d> c3
> > > > > e8 de 42 9a 00 eb e7 66 66 66 2e 0f 1f 84 00 00 00 00 00 55
> > > > > RSP: 0018:ffff88010622fca0 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff13
> > > > > RAX: ffff8801105bcc40 RBX: ffff8801061554c0 RCX: ffff8801105bdc40
> > > > > RDX: ffff8801105bdc40 RSI: aaaaaaaaaaaab000 RDI: ffffea00077ec560
> > > > > RBP: ffff88010622fca0 R08: ffffffff7fffffff R09: 0000000000000002
> > > > > R10: 0000000000000000 R11: 0000000000000000 R12: ffff8800751cb880
> > > > > R13: 0000000000000000 R14: ffff880106155db8 R15: ffff88013fcb9c40
> > > > > finish_task_switch+0xe3/0x270 kernel/sched/core.c:2679
> > > > > context_switch kernel/sched/core.c:2832 [inline]
> > > > > __schedule+0x78f/0x8f0 kernel/sched/core.c:3479
> > > > > schedule+0x1cc/0x300 kernel/sched/core.c:3523
> > > > > kauditd_thread+0xc64/0xee0 kernel/audit.c:889
> > > > > kthread+0x5b1/0x5f0 kernel/kthread.c:247
> > > > > ret_from_fork+0x35/0x40 arch/x86/entry/entry_64.S:416
> > > > >
> > > > > Uninit was created at:
> > > > > kmsan_save_stack_with_flags mm/kmsan/kmsan.c:255 [inline]
> > > > > kmsan_internal_alloc_meta_for_pages+0x157/0x730 mm/kmsan/kmsan.c:693
> > > > > kmsan_alloc_page+0x80/0xe0 mm/kmsan/kmsan_hooks.c:320
> > > > > __alloc_pages_nodemask+0x128c/0x69b0 mm/page_alloc.c:4416
> > > > > alloc_pages_current+0x51f/0x760 mm/mempolicy.c:2093
> > > > > alloc_pages include/linux/gfp.h:511 [inline]
> > > > > alloc_slab_page mm/slub.c:1459 [inline]
> > > > > allocate_slab mm/slub.c:1604 [inline]
> > > > > new_slab+0x552/0x1f30 mm/slub.c:1675
> > > > > new_slab_objects mm/slub.c:2438 [inline]
> > > > > ___slab_alloc+0x1414/0x1dd0 mm/slub.c:2590
> > > > > __slab_alloc mm/slub.c:2630 [inline]
> > > > > slab_alloc_node mm/slub.c:2693 [inline]
> > > > > slab_alloc mm/slub.c:2735 [inline]
> > > > > kmem_cache_alloc+0xc9b/0xda0 mm/slub.c:2740
> > > > > kmem_cache_zalloc include/linux/slab.h:697 [inline]
> > > > > avc_alloc_node+0x109/0xb90 security/selinux/avc.c:572
> > > > > avc_update_node+0x172/0x1ee0 security/selinux/avc.c:859
> > > > > avc_denied+0x312/0x360 security/selinux/avc.c:1024
> > > > > avc_has_perm_noaudit+0x733/0x770 security/selinux/avc.c:1155
> > > > > avc_has_perm+0x172/0x480 security/selinux/avc.c:1184
> > > > > sock_has_perm security/selinux/hooks.c:4539 [inline]
> > > > > selinux_socket_sendmsg+0x297/0x360 security/selinux/hooks.c:4875
> > > > > security_socket_sendmsg+0x127/0x200 security/security.c:1410
> > > > > sock_sendmsg net/socket.c:628 [inline]
> > > > > ___sys_sendmsg+0xd5f/0x1290 net/socket.c:2116
> > > > > __sys_sendmsg net/socket.c:2154 [inline]
> > > > > __do_sys_sendmsg net/socket.c:2163 [inline]
> > > > > __se_sys_sendmsg+0x307/0x460 net/socket.c:2161
> > > > > __x64_sys_sendmsg+0x4a/0x70 net/socket.c:2161
> > > > > do_syscall_64+0xb8/0x100 arch/x86/entry/common.c:291
> > > > > entry_SYSCALL_64_after_hwframe+0x63/0xe7
> > > > > =========================================================
> > > > >
> > > > > Crash log 2
> > > > > =========================================================
> > > > > BUG: KMSAN: uninit-value in rcu_accelerate_cbs+0x821/0x990
> > > > > kernel/rcu/tree.c:1728
> > > > > CPU: 0 PID: 10 Comm: rcu_sched Not tainted 4.19.0-rc8+ #18
> > > > > Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> > > > > Call Trace:
> > > > > __dump_stack lib/dump_stack.c:77 [inline]
> > > > > dump_stack+0x305/0x460 lib/dump_stack.c:113
> > > > > kmsan_report+0x1a2/0x2e0 mm/kmsan/kmsan.c:917
> > > > > __msan_warning+0x7d/0xe0 mm/kmsan/kmsan_instr.c:500
> > > > > rcu_accelerate_cbs+0x821/0x990 kernel/rcu/tree.c:1728
> > > > > __note_gp_changes+0x2ac/0x9e0 kernel/rcu/tree.c:1807
> > > > > rcu_gp_cleanup kernel/rcu/tree.c:2109 [inline]
> > > > > rcu_gp_kthread+0x3019/0x3990 kernel/rcu/tree.c:2236
> > > > > kthread+0x5b1/0x5f0 kernel/kthread.c:247
> > > > > ret_from_fork+0x35/0x40 arch/x86/entry/entry_64.S:416
> > > > >
> > > > > Uninit was created at:
> > > > > kmsan_save_stack_with_flags mm/kmsan/kmsan.c:255 [inline]
> > > > > kmsan_internal_alloc_meta_for_pages+0x157/0x730 mm/kmsan/kmsan.c:693
> > > > > kmsan_alloc_page+0x80/0xe0 mm/kmsan/kmsan_hooks.c:320
> > > > > __alloc_pages_nodemask+0x128c/0x69b0 mm/page_alloc.c:4416
> > > > > alloc_pages_current+0x51f/0x760 mm/mempolicy.c:2093
> > > > > alloc_pages include/linux/gfp.h:511 [inline]
> > > > > alloc_slab_page mm/slub.c:1459 [inline]
> > > > > allocate_slab mm/slub.c:1604 [inline]
> > > > > new_slab+0x552/0x1f30 mm/slub.c:1675
> > > > > new_slab_objects mm/slub.c:2438 [inline]
> > > > > ___slab_alloc+0x1414/0x1dd0 mm/slub.c:2590
> > > > > __slab_alloc mm/slub.c:2630 [inline]
> > > > > slab_alloc_node mm/slub.c:2693 [inline]
> > > > > slab_alloc mm/slub.c:2735 [inline]
> > > > > kmem_cache_alloc+0xc9b/0xda0 mm/slub.c:2740
> > > > > kmem_cache_zalloc include/linux/slab.h:697 [inline]
> > > > > avc_alloc_node+0x109/0xb90 security/selinux/avc.c:572
> > > > > avc_insert security/selinux/avc.c:696 [inline]
> > > > > avc_compute_av+0x31e/0x1050 security/selinux/avc.c:1008
> > > > > avc_has_perm_noaudit+0x516/0x770 security/selinux/avc.c:1149
> > > > > avc_has_perm+0x172/0x480 security/selinux/avc.c:1184
> > > > > selinux_socket_create+0x248/0x3c0 security/selinux/hooks.c:4560
> > > > > security_socket_create+0x146/0x210 security/security.c:1372
> > > > > __sock_create+0x26b/0xf30 net/socket.c:1232
> > > > > sock_create net/socket.c:1317 [inline]
> > > > > __sys_socket+0x180/0x670 net/socket.c:1347
> > > > > __do_sys_socket net/socket.c:1356 [inline]
> > > > > __se_sys_socket+0x8d/0xb0 net/socket.c:1354
> > > > > __x64_sys_socket+0x4a/0x70 net/socket.c:1354
> > > > > do_syscall_64+0xb8/0x100 arch/x86/entry/common.c:291
> > > > > entry_SYSCALL_64_after_hwframe+0x63/0xe7
> > > > > =========================================================
> > > > >
> > > > > Thanks,
> > > > > Kyungtae Kim
> > > > >
> > > >
> > > > --
> > > > You received this message because you are subscribed to the Google Groups "syzkaller" group.
> > > > To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
> > > > For more options, visit https://groups.google.com/d/optout.
> > >
> > >
> > >
> > > --
> > > Alexander Potapenko
> > > Software Engineer
> > >
> > > Google Germany GmbH
> > > Erika-Mann-Straße, 33
> > > 80636 München
> > >
> > > Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
> > > Registergericht und -nummer: Hamburg, HRB 86891
> > > Sitz der Gesellschaft: Hamburg
> > >
> >
>
> --
> You received this message because you are subscribed to the Google Groups "syzkaller" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
> For more options, visit https://groups.google.com/d/optout.
--
Alexander Potapenko
Software Engineer
Google Germany GmbH
Erika-Mann-Straße, 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg