FYI, we noticed the below changes on
https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git hrtimer/monotonic-raw
commit 512d7dce5e97b2242cd50032ae7cf56c63342f2c ("hrtimer: Catch illegal clockids")
+----------------+------------+------------+
| | f144b63298 | 512d7dce5e |
+----------------+------------+------------+
| boot_successes | 44 | 0 |
+----------------+------------+------------+
[ 1.171029] BUG: unable to handle kernel NULL pointer dereference at 0000000000000018
[ 1.171035] IP: [<ffffffff8bafac73>] __hrtimer_get_next_event+0x33/0x70
[ 1.171037] PGD 0
[ 1.171039] Oops: 0000 [#1] SMP
[ 1.171041] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.4.0-rc4-00002-g512d7dc #2
[ 1.171042] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Debian-1.8.2-1 04/01/2014
[ 1.171044] task: ffff8800001a8000 ti: ffff88000017c000 task.ti: ffff88000017c000
[ 1.171046] RIP: 0010:[<ffffffff8bafac73>] [<ffffffff8bafac73>] __hrtimer_get_next_event+0x33/0x70
[ 1.171047] RSP: 0000:ffff880013c03ee0 EFLAGS: 00010002
[ 1.171049] RAX: 7fffffffffffffff RBX: ffff880013dcd900 RCX: ffff880013dcdb00
[ 1.171050] RDX: 0000000000000001 RSI: 000001cd42e4dffb RDI: ffff880013dcd900
[ 1.171050] RBP: ffff880013c03ee0 R08: 0000000000000000 R09: 0000000000000000
[ 1.171051] R10: 0000000000006df8 R11: ffffffff8da362ed R12: ffff880013dcd900
[ 1.171052] R13: 0000000000000000 R14: ffffffff8da3dda8 R15: 0000000000000000
[ 1.171054] FS: 0000000000000000(0000) GS:ffff880013c00000(0000) knlGS:0000000000000000
[ 1.171055] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 1.171056] CR2: 0000000000000018 CR3: 000000000ca09000 CR4: 00000000000006f0
[ 1.171060] Stack:
[ 1.171062] ffff880013c03ef8 ffffffff8bafb0c5 ffff880013dcd900 ffff880013c03f18
[ 1.171064] ffffffff8bafbb3e ffff8800001a8000 0000000000000000 ffff880013c03f38
[ 1.171066] ffffffff8bafab02 ffff880013c0c500 7fffffffffffffff ffff880013c03f50
[ 1.171067] Call Trace:
[ 1.171071] <IRQ>
[ 1.171071] [<ffffffff8bafb0c5>] retrigger_next_event+0x65/0x90
[ 1.171073] [<ffffffff8bafbb3e>] hrtimer_run_queues+0x6e/0xd0
[ 1.171075] [<ffffffff8bafab02>] update_process_times+0x22/0x60
[ 1.171078] [<ffffffff8bb0764a>] tick_periodic+0x2a/0xb0
[ 1.171080] [<ffffffff8bb076f0>] ? tick_handle_periodic+0x20/0x70
[ 1.171082] [<ffffffff8bb076f0>] tick_handle_periodic+0x20/0x70
[ 1.171085] [<ffffffff8ba2d083>] smp_trace_apic_timer_interrupt+0x63/0x90
[ 1.171086] [<ffffffff8ba2d0b9>] smp_apic_timer_interrupt+0x9/0x10
[ 1.171090] [<ffffffff8c2f8c44>] apic_timer_interrupt+0x84/0x90
[ 1.171094] <EOI>
[ 1.171094] [<ffffffff8bd94945>] ? serial8250_console_write+0xe5/0x220
[ 1.171098] [<ffffffff8bae8f50>] ? console_unlock+0x2f0/0x530
[ 1.171101] [<ffffffff8bae948f>] vprintk_emit+0x2ff/0x560
[ 1.171102] [<ffffffff8bae980a>] vprintk_default+0x1a/0x20
[ 1.171105] [<ffffffff8bb33316>] printk+0x43/0x4b
[ 1.171107] [<ffffffff8bb03d67>] __clocksource_select+0xd7/0x110
[ 1.171111] [<ffffffff8ce69c0e>] ? ntp_init+0xb/0xb
[ 1.171112] [<ffffffff8ce69c42>] clocksource_done_booting+0x34/0x44
[ 1.171115] [<ffffffff8ce51f5f>] do_one_initcall+0xe2/0x172
[ 1.171116] [<ffffffff8ce521a8>] kernel_init_freeable+0x1b9/0x23c
[ 1.171119] [<ffffffff8c2f0ff0>] ? rest_init+0xd0/0xd0
[ 1.171121] [<ffffffff8c2f0ff9>] kernel_init+0x9/0xe0
[ 1.171123] [<ffffffff8c2f829f>] ret_from_fork+0x3f/0x70
[ 1.171125] [<ffffffff8c2f0ff0>] ? rest_init+0xd0/0xd0
[ 1.171152] Code: c7 87 a0 00 00 00 00 00 00 00 48 89 e5 85 d2 74 47 48 8d 8f c0 00 00 00 48 b8 ff ff ff ff ff ff ff 7f f6 c2 01 74 1b 4c 8b 41 18 <49> 8b 70 18 48 2b 71 28 48 39 c6 7d 0a 4c 89 87 a0 00 00 00 48
[ 1.171155] RIP [<ffffffff8bafac73>] __hrtimer_get_next_event+0x33/0x70
[ 1.171155] RSP <ffff880013c03ee0>
[ 1.171156] CR2: 0000000000000018
[ 1.171162] ---[ end trace 164f7953b3ea131d ]---
[ 1.171163] Kernel panic - not syncing: Fatal exception in interrupt
Thanks,
Kernel Test Robot
Hi Marc,
FYI, this bisect shows commit f144b632 also has the problem.
https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git hrtimer/monotonic-raw
commit f144b6329838b2996b64a070cb1b99dcf6f0bb30
Author: Marc Zyngier <[email protected]>
AuthorDate: Sun Dec 13 14:42:14 2015 +0000
Commit: Marc Zyngier <[email protected]>
CommitDate: Sun Dec 13 15:04:33 2015 +0000
hrtimer: Add support for CLOCK_MONOTONIC_RAW
The KVM/ARM timer implementation arms a hrtimer when a vcpu is
blocked (usually because it is waiting for an interrupt)
while its timer is going to kick in the future.
It is essential that this timer doesn't get adjusted, or the
guest will end up being woken-up at the wrong time (NTP seems
to confuse the hell of some guests when running on the host).
In order to allow this, let's add CLOCK_MONOTONIC_RAW support
to hrtimer (it is so far only supported for userspace). It also
has the (limited) benefit of fixing de0421d53bfb ("mac80211_hwsim:
shuffle code to prepare for dynamic radios"), which already uses
this functionnality without it being implemented (just being
lucky...).
Signed-off-by: Marc Zyngier <[email protected]>
+------------------------------------------------+----------+------------+------------+
| | v4.4-rc4 | f144b63298 | abb6cc75ee |
+------------------------------------------------+----------+------------+------------+
| boot_successes | 644 | 152 | 0 |
| boot_failures | 16 | 2 | 17 |
| BUG:kernel_boot_hang | 6 | | |
| IP-Config:Auto-configuration_of_network_failed | 10 | | |
| BUG:unable_to_handle_kernel | 0 | 2 | 17 |
| Oops | 0 | 2 | 17 |
| EIP_is_at_hrtimer_get_next_event | 0 | 2 | 17 |
| Kernel_panic-not_syncing:Fatal_exception | 0 | 2 | 17 |
| backtrace:cpu_startup_entry | 0 | 2 | 17 |
+------------------------------------------------+----------+------------+------------+
[ 5.466489] init: Failed to create pty - disabling logging for job
[ 5.473086] init: Failed to create pty - disabling logging for job
Kernel tests: Boot OK!
[ 15.299670] BUG: unable to handle kernel NULL pointer dereference at 00000010
[ 15.301386] IP: [<c110e4c8>] hrtimer_get_next_event+0xd8/0x1f0
[ 15.302799] *pdpt = 0000000011cef001 *pde = 0000000000000000
[ 15.304228] Oops: 0000 [#1]
[ 15.305163] Modules linked in:
[ 15.306124] CPU: 0 PID: 0 Comm: swapper Not tainted 4.4.0-rc4-00001-gf144b63 #1
[ 15.308079] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Debian-1.8.2-1 04/01/2014
[ 15.310269] task: c1c30140 ti: c1c28000 task.ti: c1c28000
[ 15.311556] EIP: 0060:[<c110e4c8>] EFLAGS: 00210002 CPU: 0
[ 15.312855] EIP is at hrtimer_get_next_event+0xd8/0x1f0
[ 15.314105] EAX: 00000001 EBX: 00000003 ECX: 00000000 EDX: c1c70980
[ 15.315570] ESI: 00000034 EDI: 00000000 EBP: c1c29e9c ESP: c1c29e70
[ 15.316991] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068
[ 15.318295] CR0: 8005003b CR2: 00000010 CR3: 123d06c0 CR4: 000006f0
[ 15.319748] Stack:
[ 15.320497] 00200046 00000031 00000000 851f015e 00000002 00000000 01010000 00000003
[ 15.323460] fffba63f 78d31000 00000003 c1c29f64 c110c153 00000004 00000000 00000004
[ 15.326289] 8e7f8ead 00000004 8d0d08ce 00000004 00000000 00000001 fffba63e c1c29ed8
[ 15.328927] Call Trace:
[ 15.329732] [<c110c153>] get_next_timer_interrupt+0x243/0x9d0
[ 15.331050] [<c10718c2>] ? pvclock_clocksource_read+0xe2/0x2c0
[ 15.333965] [<c112b405>] ? tick_program_event+0xd5/0x130
[ 15.335223] [<c106f910>] ? kvm_sched_clock_read+0x30/0x70
[ 15.336502] [<c10144d7>] ? sched_clock+0x17/0x30
[ 15.338044] [<c10da449>] ? sched_clock_cpu+0x49/0x240
[ 15.339727] [<c112c1ca>] __tick_nohz_idle_enter+0x70a/0x840
[ 15.341450] [<c112cc0e>] tick_nohz_idle_enter+0x12e/0x1c0
[ 15.342718] [<c10e3759>] cpu_idle_loop+0xa9/0x780
[ 15.343920] [<c10e3e5e>] cpu_startup_entry+0x2e/0x30
[ 15.345122] [<c1a19c46>] rest_init+0xe6/0x100
[ 15.346500] [<c1df77fb>] start_kernel+0x9d8/0x9ed
[ 15.347661] [<c1df6344>] i386_start_kernel+0x10e/0x120
[ 15.348877] Code: 09 c7 c1 89 4d dc eb 16 8d 76 00 83 45 e4 01 c6 45 ef 01 83 55 e8 00 83 c2 20 d1 e8 74 3e 83 c6 01 83 d7 00 a8 01 74 e3 8b 4a 10 <8b> 59 10 8b 49 0c 2b 4a 18 1b 5a 1c 3b 5d f0 7f db 7c 05 3b 4d
[ 15.356618] EIP: [<c110e4c8>] hrtimer_get_next_event+0xd8/0x1f0 SS:ESP 0068:c1c29e70
[ 15.358711] CR2: 0000000000000010
[ 15.359634] ---[ end trace 06172c6b5101e6ea ]---
[ 15.360765] Kernel panic - not syncing: Fatal exception
git bisect start abb6cc75eeed7e929d106d3bb331fbea62b75421 527e9316f8ec44bd53d90fb9f611fa7ffff52bb9 --
git bisect bad 7704041cff9ed96df680e5c286c97d8b0b1ee164 # 01:38 0- 44 Merge 'linux-review/Paul-Gortmaker/drivers-pci-avoid-module_init-in-non-modular-host-pci/20151213-094811' into devel-spot-201512132342
git bisect bad b8d3a14e5074c9c67216a0c892828927bc2fba73 # 01:38 0- 4 Merge 'drm-exynos/exynos-drm-next' into devel-spot-201512132342
git bisect bad d630073ad2301da0a11521746b5911ee0ecc3980 # 01:38 0- 3 Merge 'linux-review/Yoshihiro-Kaneko/ravb-Remove-clear-unhandled-interrupt/20151213-232031' into devel-spot-201512132342
git bisect bad 12fdbff8a5fdecfd75c4d4974f5a0f9366fdc3cb # 01:46 0- 8 Merge 'arm-platforms/hrtimer/monotonic-raw' into devel-spot-201512132342
git bisect good 2346da3a815459d7f049802688b885a2bd4095e0 # 01:54 148+ 2 0day base guard for 'devel-spot-201512132342'
git bisect good 01440bcdb89558614980818ca10c8562d4fa9d13 # 02:05 147+ 0 Merge 'linux-review/Wang-Zhi-A/mm-mempool-Factor-out-mempool_refill/20151213-233708' into devel-spot-201512132342
git bisect bad 512d7dce5e97b2242cd50032ae7cf56c63342f2c # 02:16 0- 5 hrtimer: Catch illegal clockids
git bisect bad f144b6329838b2996b64a070cb1b99dcf6f0bb30 # 02:31 4- 1 hrtimer: Add support for CLOCK_MONOTONIC_RAW
# first bad commit: [f144b6329838b2996b64a070cb1b99dcf6f0bb30] hrtimer: Add support for CLOCK_MONOTONIC_RAW
git bisect good 527e9316f8ec44bd53d90fb9f611fa7ffff52bb9 # 02:43 421+ 15 Linux 4.4-rc4
# extra tests with DEBUG_INFO
git bisect bad f144b6329838b2996b64a070cb1b99dcf6f0bb30 # 02:54 67- 1 hrtimer: Add support for CLOCK_MONOTONIC_RAW
# extra tests on HEAD of linux-devel/devel-spot-201512132342
git bisect bad abb6cc75eeed7e929d106d3bb331fbea62b75421 # 02:54 0- 17 0day head guard for 'devel-spot-201512132342'
# extra tests on tree/branch arm-platforms/hrtimer/monotonic-raw
git bisect bad 770b0e16b97493b02d358e882630cda9fd2bb2ed # 03:03 0- 19 KVM: arm/arm64: timer: Switch to CLOCK_MONOTONIC_RAW
# extra tests with first bad commit reverted
git bisect bad 4d33ba53a9e70d2a5ea670c44b2e0cfe70993083 # 03:17 0- 58 Revert "hrtimer: Add support for CLOCK_MONOTONIC_RAW"
# extra tests on tree/branch linus/master
git bisect good 097b285d32c7cb22dd4af2286ba61668a6c367ef # 03:28 427+ 2 Merge tag 'fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc
# extra tests on tree/branch linux-next/master
git bisect bad 9379b045413ee30c88965e81d376af989ec32cf3 # 03:30 0- 5 Add linux-next specific files for 20151211
This script may reproduce the error.
----------------------------------------------------------------------------
#!/bin/bash
kernel=$1
initrd=quantal-core-i386.cgz
wget --no-clobber https://github.com/fengguang/reproduce-kernel-bug/raw/master/initrd/$initrd
kvm=(
qemu-system-x86_64
-enable-kvm
-cpu kvm64
-kernel $kernel
-initrd $initrd
-m 300
-smp 2
-device e1000,netdev=net0
-netdev user,id=net0
-boot order=nc
-no-reboot
-watchdog i6300esb
-rtc base=localtime
-serial stdio
-display none
-monitor null
)
append=(
hung_task_panic=1
earlyprintk=ttyS0,115200
systemd.log_level=err
debug
apic=debug
sysrq_always_enabled
rcupdate.rcu_cpu_stall_timeout=100
panic=-1
softlockup_panic=1
nmi_watchdog=panic
oops=panic
load_ramdisk=2
prompt_ramdisk=0
console=ttyS0,115200
console=tty0
vga=normal
root=/dev/ram0
rw
drbd.minor_count=8
)
"${kvm[@]}" --append "${append[*]}"
----------------------------------------------------------------------------
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/lkp Intel Corporation