[ CC Nick ]
[ CC Arnd ]
[ CC JF ]
Hi Matthias,
I was following [1] a bit and fell over your original posting in [2].
I wrote some early documentation (wiki) and tested/booted a
clang-compiled kernel on x86-64 bare metal.
The project was called "lll-project" these days.
The followup - LLVMlinux project - seems to be somehow dead?
After reading your posting, I got really excited and had a quick look
over the Linux v4.9.y-LTS patch-stack with additional stuff:
$ git fetch https://chromium.googlesource.com/chromiumos/third_party/kernel
refs/sandbox/mka/llvm/v4.9_ext
$ git checkout -b llvm_v4.9_ext FETCH_HEAD
I like your commit subjects and messages.
Before starting to compile I have some questions, I hope you can answer them.
[ CLANG-VERSION ]
Here on Debian/testing (will get Version 10 with codename "buster")
AMD64 I have the choice of installing CLANG v4.0.1, v5.0.1 and v6.0.0.
As you point out LLVM/CLANG v5.0 or higher are a good and faster
choice than v4.0.
Using CLANG >= 5.0 makes the *CLANG* patch-series obsolete?
[ REFRESH PATCH-STACK ]
What about the *FROMGIT* and *CUSTOM* patch-series?
Are they in upstream in the meantime?
If yes, will you refresh this v4.9.y-LTS patch-stack?
[ ASM-GOTO ]
Foremore, I have seen you have a "refs/sandbox/mka/llvm/v4.14" Git branch.
Linux v4.14 is also an LTS release.
After Linux v4.9 "asm-goto" support was added which is GCC specific.
Is there a workaround to compile Linux-kernel with any CLANG version?
You happen to know the status in LLVM upstream?
[ X86-EFLAGS/IF ]
AFAICS, I remember an EFLAGS/IF problem on X86 and interrupt handling.
You happen to know if this is fixed in LLVM upstream?
Some LLVM developers were interested in fixing this.
Can you comment on this, JF (we had some email conversation in private in 2016)?
I am looking forward to hearing from you.
Best regards,
- Sedat -
References:
[1] https://marc.info/?l=linux-kernel&m=152425978517449&w=2
[2] https://marc.info/?l=linux-kernel&m=150344365613264&w=2
[3] https://bugs.llvm.org/show_bug.cgi?id=9295
References from JF:
[0] Hans' fix: http://reviews.llvm.org/D15240
[1] my original pushf/popf disable: http://reviews.llvm.org/D6629
[2] More effort, such as: http://reviews.llvm.org/D15157
P.S.: Your Linux v4.9.y-LTS patch-stack
[ UPSTREAM ]
0001-UPSTREAM-kbuild-Add-better-clang-cross-build-support.patch
0002-UPSTREAM-kbuild-clang-add-no-integrated-as-to-KBUILD.patch
0003-UPSTREAM-kbuild-Consolidate-header-generation-from-A.patch
0004-UPSTREAM-kbuild-consolidate-redundant-sed-script-ASM.patch
0005-UPSTREAM-kbuild-fix-asm-offset-generation-to-work-wi.patch
0006-UPSTREAM-kbuild-drop-Wno-unknown-warning-option-from.patch
0007-UPSTREAM-kbuild-LLVMLinux-Add-Werror-to-cc-option-to.patch
0008-UPSTREAM-kbuild-use-Oz-instead-of-Os-when-using-clan.patch
0009-UPSTREAM-kbuild-Add-support-to-generate-LLVM-assembl.patch
0010-UPSTREAM-modules-mark-__inittest-__exittest-as-__may.patch
0011-UPSTREAM-compiler-clang-suppress-warning-for-unused-.patch
0012-UPSTREAM-compiler-clang-properly-override-inline-for.patch
0013-UPSTREAM-x86-kbuild-Use-cc-option-to-enable-falign-j.patch
0014-UPSTREAM-crypto-x86-aesni-fix-token-pasting-for-clan.patch
0015-UPSTREAM-x86-mm-kaslr-Use-the-_ASM_MUL-macro-for-mul.patch
0016-UPSTREAM-kbuild-Add-__cc-option-macro.patch
0017-UPSTREAM-x86-build-Use-__cc-option-for-boot-code-com.patch
0018-UPSTREAM-x86-build-Specify-stack-alignment-for-clang.patch
0019-UPSTREAM-kbuild-clang-Disable-address-of-packed-memb.patch
0020-UPSTREAM-crypto-arm64-sha-avoid-non-standard-inline-.patch
0021-UPSTREAM-llist-clang-introduce-member_address_is_non.patch
0022-UPSTREAM-x86-boot-undef-memcpy-et-al-in-string.c.patch
0023-UPSTREAM-compiler-clang-always-inline-when-CONFIG_OP.patch
0024-UPSTREAM-efi-libstub-arm64-Use-hidden-attribute-for-.patch
0025-UPSTREAM-efi-libstub-arm64-Force-hidden-visibility-f.patch
0026-UPSTREAM-efi-libstub-arm64-Set-fpie-when-building-th.patch
0027-UPSTREAM-x86-build-Fix-stack-alignment-for-CLang.patch
0028-UPSTREAM-x86-build-Use-cc-option-to-validate-stack-a.patch
0029-UPSTREAM-x86-asm-Fix-inline-asm-call-constraints-for.patch
0030-UPSTREAM-x86-asm-Fix-inline-asm-call-constraints-for.patch
0031-UPSTREAM-Kbuild-use-fshort-wchar-globally.patch
0043-UPSTREAM-arm64-ensure-extension-of-smp_store_release.patch
0044-UPSTREAM-arm64-xchg-hazard-against-entire-exchange-v.patch
0045-UPSTREAM-arm64-uaccess-ensure-extension-of-access_ok.patch
0046-UPSTREAM-arm64-armv8_deprecated-ensure-extension-of-.patch
0047-UPSTREAM-arm64-atomic_lse-match-asm-register-sizes.patch
0048-UPSTREAM-arm64-uaccess-suppress-spurious-clang-warni.patch
0049-UPSTREAM-drm-i915-Remove-unused-BSM_MASK-causing-war.patch
0050-UPSTREAM-audit-add-tty-field-to-LOGIN-event.patch
0051-UPSTREAM-audit-move-audit_get_tty-to-reduce-scope-an.patch
0052-UPSTREAM-media-uvcvideo-Fix-a-wrong-macro.patch
0053-UPSTREAM-dmaengine-Fix-array-index-out-of-bounds-war.patch
0054-UPSTREAM-hpet-Make-cmd-parameter-of-hpet_ioctl_commo.patch
0055-UPSTREAM-ALSA-hda-ca0132-Remove-double-parentheses.patch
0056-UPSTREAM-selinux-Remove-unnecessary-check-of-array-b.patch
0057-UPSTREAM-mac80211-Fix-clang-warning-about-constant-o.patch
0058-UPSTREAM-cpumask-Add-helper-cpumask_available.patch
0059-UPSTREAM-genirq-Use-cpumask_available-for-check-of-c.patch
0060-UPSTREAM-cfg80211-Fix-array-bounds-warning-in-fragme.patch
0061-UPSTREAM-mac80211-ibss-Fix-channel-type-enum-in-ieee.patch
0062-UPSTREAM-nl80211-Fix-enum-type-of-variable-in-nl8021.patch
0063-UPSTREAM-ALSA-hda-ca0132-Limit-values-for-chip-addre.patch
0064-UPSTREAM-media-vcodec-mediatek-Remove-double-parenth.patch
0065-UPSTREAM-misc-lkdtm-Add-volatile-to-intentional-NULL.patch
0066-UPSTREAM-jiffies.h-declare-jiffies-and-jiffies_64-wi.patch
0067-UPSTREAM-fs-compat-Remove-warning-from-COMPATIBLE_IO.patch
0068-UPSTREAM-dm-ioctl-remove-double-parentheses.patch
0069-UPSTREAM-ath9k-Add-cast-to-u8-to-FREQ2FBIN-macro.patch
0070-UPSTREAM-tracing-Use-cpumask_available-to-check-if-c.patch
0071-UPSTREAM-netfilter-ctnetlink-Make-some-parameters-in.patch
0072-UPSTREAM-selinux-Remove-redundant-check-for-unknown-.patch
0076-UPSTREAM-x86-boot-Disable-the-address-of-packed-memb.patch
0077-UPSTREAM-zram-rework-copy-of-compressor-name-in-comp.patch
0078-UPSTREAM-arm64-avoid-overflow-in-VA_START-and-PAGE_O.patch
0079-UPSTREAM-bitops-avoid-integer-overflow-in-GENMASK-_U.patch
0080-UPSTREAM-btrfs-Remove-extra-parentheses-from-conditi.patch
0081-UPSTREAM-netpoll-Fix-device-name-check-in-netpoll_se.patch
[ BACKPORT ]
0032-BACKPORT-kbuild-fix-linker-feature-test-macros-when-.patch
0033-BACKPORT-kbuild-Set-KBUILD_CFLAGS-before-incl.-arch-.patch
0034-BACKPORT-crypto-arm64-aes-ce-cipher-move-assembler-c.patch
0073-BACKPORT-clocksource-Use-GENMASK_ULL-in-definition-o.patch
0074-BACKPORT-PCI-Make-PCI_ROM_ADDRESS_MASK-a-32-bit-cons.patch
0075-BACKPORT-iio-light-isl29018-Only-declare-ACPI-table-.patch
0082-BACKPORT-drm-i915-Consistently-use-enum-pipe-for-PCH.patch
0083-BACKPORT-drm-i915-Pass-enum-pipe-to-intel_set_pch_fi.patch
[ FROMGIT ]
0084-FROMGIT-x86-boot-Disable-Clang-warnings-about-GNU-ex.patch
0085-FROMGIT-sched-sysctl-Fix-attributes-of-some-extern-d.patch
[ CUSTOM ]
0035-CUSTOM-kbuild-clang-Disable-the-duplicate-decl-speci.patch
0036-CUSTOM-arm64-Disable-asm-operand-width-warning-for-c.patch
[ CLANG ]
0037-CLANG4-crypto-arm64-crc32-explictly-pass-mcpu-option.patch
0038-CLANG4-kbuild-Add-meabi-gnu-to-the-clang-parameters.patch
0039-CLANG4-arm64-prefetch-Use-__builtin_arm_prefetch-for.patch
0040-CLANG4-Disable-lkdtm-when-ftrace-is-enabled.patch
0041-CLANG4-futex-don-t-optimize-futex_detect_cmpxchg-on-.patch
0042-CLANG3-core-clang-work-around-x86-regparm-intrinsics.patch
0086-CLANG4-x86-mce-Disable-gcc-compat-warning-for-mce.c-.patch
0087-CLANG4-task_work-Disable-gcc-compat-warning-locally.patch
Hi Sedat,
On Mon, Apr 23, 2018 at 03:56:41PM +0200, Sedat Dilek wrote:
> [ CC Nick ]
> [ CC Arnd ]
> [ CC JF ]
> I wrote some early documentation (wiki) and tested/booted a
> clang-compiled kernel on x86-64 bare metal.
> The project was called "lll-project" these days.
> The followup - LLVMlinux project - seems to be somehow dead?
Yes, I think it stalled in 2014 or so. There is still a mailing list
with very occasional traffic.
> After reading your posting, I got really excited and had a quick look
> over the Linux v4.9.y-LTS patch-stack with additional stuff:
>
> $ git fetch https://chromium.googlesource.com/chromiumos/third_party/kernel
> refs/sandbox/mka/llvm/v4.9_ext
> $ git checkout -b llvm_v4.9_ext FETCH_HEAD
>
> I like your commit subjects and messages.
>
> Before starting to compile I have some questions, I hope you can answer them.
>
> [ CLANG-VERSION ]
>
> Here on Debian/testing (will get Version 10 with codename "buster")
> AMD64 I have the choice of installing CLANG v4.0.1, v5.0.1 and v6.0.0.
> As you point out LLVM/CLANG v5.0 or higher are a good and faster
> choice than v4.0.
> Using CLANG >= 5.0 makes the *CLANG* patch-series obsolete?
Correct, preferably use clang v6 or newer.
> [ REFRESH PATCH-STACK ]
>
> What about the *FROMGIT* and *CUSTOM* patch-series?
> Are they in upstream in the meantime?
CUSTOM patches address issues in older kernels that have been fixed
upstream in a different, not easily backportable way. They will never
land in upstream.
FROMGIT patches should eventually land upstream, since they were taken
from a maintainer tree.
The commit message includes the upstream git hash, so you can easily
check if it has landed (though in some cases the hash changes ...)
> If yes, will you refresh this v4.9.y-LTS patch-stack?
I eventually will, the two FROMGIT patches are relatively recent, and
respinning the trees just to update the tags isn't one of my
priorities.
> [ ASM-GOTO ]
>
> Foremore, I have seen you have a "refs/sandbox/mka/llvm/v4.14" Git branch.
> Linux v4.14 is also an LTS release.
Yes, there is also a follow up post:
https://lkml.org/lkml/2017/11/22/943
> After Linux v4.9 "asm-goto" support was added which is GCC specific.
asm-goto was optional until a few weeks ago, when x86 maintainers
decided to make it mandatory, and thus break clang builds for x86:
https://lkml.org/lkml/2018/4/2/486
> Is there a workaround to compile Linux-kernel with any CLANG
> version?
You could revert the patch that makes asm-goto mandatory.
> You happen to know the status in LLVM upstream?
I know people are actively working on this, but don't know an ETA.
> [ X86-EFLAGS/IF ]
>
> AFAICS, I remember an EFLAGS/IF problem on X86 and interrupt handling.
> You happen to know if this is fixed in LLVM upstream?
> Some LLVM developers were interested in fixing this.
> Can you comment on this, JF (we had some email conversation in private in 2016)?
Yes, this has been fixed recently:
https://bugs.llvm.org/show_bug.cgi?id=36028
Cheers
Matthias
Hi Matthias,
a big thank you for giving all the informations!
I used your mka/llvm/v4.14_ext Git tree...
https://chromium.googlesource.com/chromiumos/third_party/kernel/+log/sandbox/mka/llvm/v4.14_ext
...and was able to compile with clang-6.0 from Debian/buster64 with...
...reverting the clang-3/clang-4 patches...
user$ for p in 69e44656ae43 222b88977a00 cdfcf1e45537 26f14c9225a6
0385a18e9995 68dab143c9b4 ; do echo [ $p ] ; LC_ALL=C git revert
--no-edit $p ; done
0001-Revert-CLANG3-core-clang-work-around-x86-regparm-int.patch
0002-Revert-CLANG4-futex-don-t-optimize-futex_detect_cmpx.patch
0003-Revert-CLANG4-Disable-lkdtm-when-ftrace-is-enabled.patch
0004-Revert-CLANG4-arm64-prefetch-Use-__builtin_arm_prefe.patch
0005-Revert-CLANG4-kbuild-Add-meabi-gnu-to-the-clang-para.patch
0006-Revert-CLANG4-crypto-arm64-aes-ce-Explicitly-pass-th.patch
...and needed two additional patches from upstream:
0007-kbuild-clang-remove-crufty-HOSTCFLAGS.patch <--- Label with
BACKPORT (XXX: Backported; Required when using HOSTCC in make-line,
see below)
0008-x86-xen-remove-the-use-of-VLAIS.patch <--- Label with UPSTREAM
(XXX: cherry-picked)
These 3 patches in your Git branch are in Linux v4.14.36...
9af5ddf981ed BACKPORT: kbuild: disable clang's default use of
-fmerge-all-constants
f24088a3842c BACKPORT: kbuild: Set KBUILD_CFLAGS before incl. arch Makefile
d4dfe384346d UPSTREAM: kbuild: fix linker feature test macros when
cross compiling with Clang
...and can be dropped.
I do not know your work-flow/policy: Maybe you want to keep your
patch-stack against vanilla Linux v4.14 - without stables from
linux-stable-4.14.y.
For easy switching "mycompiler" I use a wrapper-script:
root# cat /usr/bin/mycompiler
#!/bin/bash
clang-6.0 "$@"
- EOF -
user$ cp -v /boot/config-4.14.35-1-iniza-amd64 .config
user$ MAKE="make V=1" ; COMPILER="mycompiler" ;
MAKE_OPTS="CC=$COMPILER HOSTCC=$COMPILER"
user$ yes "" | $MAKE $MAKE_OPTS oldconfig && $MAKE $MAKE_OPTS
silentoldconfig < /dev/null
The diffconfig to my current kernel-config-4.14 looks like this...
user$ ./scripts/diffconfig /boot/config-4.14.35-1-iniza-amd64 .config
-ARCH_HAS_REFCOUNT y
-BPF_JIT_ALWAYS_ON n
-EXOFS_DEBUG n
-EXOFS_FS m
-GENERIC_CPU_VULNERABILITIES y
-KASAN n
-ORE m
-PAGE_TABLE_ISOLATION y
-RETPOLINE y
-UNWINDER_FRAME_POINTER n
-UNWINDER_GUESS n
-UNWINDER_ORC y
+FRAME_POINTER y
+FRAME_POINTER_UNWINDER y
+GUESS_UNWINDER n
+HAVE_ARCH_KMEMCHECK y
+HAVE_RELIABLE_STACKTRACE y
+ORC_UNWINDER n
Unfortunately, I cannot boot into the generated kernel on bare metal.
Checking with QEMU (version: 2.12~rc3) and catching earlyprintk, I see this...
user$ echo $KPATH
$HOME/src/linux-kernel/important-files
user$ ls -al
insgesamt 344916
drwxr-xr-x 2 sdi sdi 4096 Apr 24 13:15 .
drwxr-xr-x 20 sdi sdi 4096 Apr 24 13:13 ..
-rw-r--r-- 1 sdi sdi 4528416 Apr 24 12:42 bzImage
lrwxrwxrwx 1 sdi sdi 35 Apr 24 13:15 initrd.img ->
initrd.img-4.14.0-1-iniza-llvmlinux
-rw-r--r-- 1 sdi sdi 25572955 Apr 24 13:08
initrd.img-4.14.0-1-iniza-llvmlinux
-rw-r--r-- 1 sdi sdi 2887195 Apr 24 12:42 System.map
-rwxr-xr-x 1 sdi sdi 326116744 Apr 24 12:42 vmlinux
user$ sudo qemu-system-x86_64 -enable-kvm -M pc -kernel $KPATH/bzImage
-initrd $KPATH/initrd.img -m 512 -net none -serial stdio -append
"root=/dev/ram0 console=ttyS0 hung_task_panic=1
earlyprintk=ttyS0,115200"
Probing EDD (edd=off to disable)... ok
[ 0.000000] Linux version 4.14.0-1-iniza-llvmlinux
([email protected]@iniza) (clang version 6.0.0-1
(tags/RELEASE_600/final)) #1 SMP Tue Apr 24 12:42:21 CEST 2018
[ 0.000000] Command line: root=/dev/ram0 console=ttyS0
hung_task_panic=1 earlyprintk=ttyS0,115200
[ 0.000000] x86/fpu: x87 FPU will use FXSAVE
[ 0.000000] e820: BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000001ffdffff] usable
[ 0.000000] BIOS-e820: [mem 0x000000001ffe0000-0x000000001fffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[ 0.000000] bootconsole [earlyser0] enabled
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] random: fast init done
[ 0.000000] SMBIOS 2.8 present.
[ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
1.11.1-1 04/01/2014
[ 0.000000] Hypervisor detected: KVM
[ 0.000000] tsc: Fast TSC calibration using PIT
[ 0.000000] e820: last_pfn = 0x1ffe0 max_arch_pfn = 0x400000000
[ 0.000000] x86/PAT: PAT not supported by CPU.
[ 0.000000] x86/PAT: Configuration [0-7]: WB WT UC- UC WB WT UC- UC
Memory KASLR using RDTSC...
[ 0.000000] found SMP MP-table at [mem 0x000f5d60-0x000f5d6f]
mapped at [ffffffffff000d60]
[ 0.000000] RAMDISK: [mem 0x1e77c000-0x1ffdffff]
[ 0.000000] ACPI: Early table checksum verification disabled
[ 0.000000] ACPI: RSDP 0x00000000000F5B90 000014 (v00 BOCHS )
[ 0.000000] ACPI: RSDT 0x000000001FFE157C 000030 (v01 BOCHS
BXPCRSDT 00000001 BXPC 00000001)
[ 0.000000] ACPI: FACP 0x000000001FFE1458 000074 (v01 BOCHS
BXPCFACP 00000001 BXPC 00000001)
[ 0.000000] ACPI: DSDT 0x000000001FFE0040 001418 (v01 BOCHS
BXPCDSDT 00000001 BXPC 00000001)
[ 0.000000] ACPI: FACS 0x000000001FFE0000 000040
[ 0.000000] ACPI: APIC 0x000000001FFE14CC 000078 (v01 BOCHS
BXPCAPIC 00000001 BXPC 00000001)
[ 0.000000] ACPI: HPET 0x000000001FFE1544 000038 (v01 BOCHS
BXPCHPET 00000001 BXPC 00000001)
[ 0.000000] No NUMA configuration found
[ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000001ffdffff]
[ 0.000000] NODE_DATA(0) allocated [mem 0x1e777000-0x1e77bfff]
[ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[ 0.000000] kvm-clock: cpu 0, msr 0:1e76f001, primary cpu clock
[ 0.000000] kvm-clock: using sched offset of 528742140 cycles
[ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff
max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.000000] DMA32 [mem 0x0000000001000000-0x000000001ffdffff]
[ 0.000000] Normal empty
[ 0.000000] Device empty
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.000000] node 0: [mem 0x0000000000100000-0x000000001ffdffff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x000000001ffdffff]
[ 0.000000] ACPI: PM-Timer IO Port: 0x608
[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[ 0.000000] Using ACPI (MADT) for SMP configuration information
[ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000
[ 0.000000] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
[ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]
[ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
[ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff]
[ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff]
[ 0.000000] e820: [mem 0x20000000-0xfeffbfff] available for PCI devices
[ 0.000000] Booting paravirtualized kernel on KVM
[ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff
max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[ 0.000000] setup_percpu: NR_CPUS:512 nr_cpumask_bits:512
nr_cpu_ids:1 nr_node_ids:1
[ 0.000000] percpu: Embedded 37 pages/cpu @ffff96dd9e400000 s114392
r8192 d28968 u2097152
[ 0.000000] KVM setup async PF for cpu 0
[ 0.000000] kvm-stealtime: cpu 0, msr 1e40d900
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 128872
[ 0.000000] Policy zone: DMA32
[ 0.000000] Kernel command line: root=/dev/ram0 console=ttyS0
hung_task_panic=1 earlyprintk=ttyS0,115200
[ 0.000000] PID hash table entries: 2048 (order: 2, 16384 bytes)
[ 0.000000] Memory: 474480K/523768K available (7639K kernel code,
1005K rwdata, 2936K rodata, 1636K init, 688K bss, 49288K reserved, 0K
cma-reserved)
[ 0.000000] ftrace: allocating 27586 entries in 108 pages
[ 0.004000] Hierarchical RCU implementation.
[ 0.004000] RCU restricting CPUs from NR_CPUS=512 to nr_cpu_ids=1.
[ 0.004000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[ 0.004000] NR_IRQS: 33024, nr_irqs: 256, preallocated irqs: 16
[ 0.004000] Console: colour VGA+ 80x25
[ 0.004000] console [ttyS0] enabled
[ 0.004000] console [ttyS0] enabled
[ 0.004000] bootconsole [earlyser0] disabled
[ 0.004000] bootconsole [earlyser0] disabled
[ 0.004000] clocksource: hpet: mask: 0xffffffff max_cycles:
0xffffffff, max_idle_ns: 19112604467 ns
[ 0.004000] general protection fault: 0000 [#1] SMP
[ 0.004000] Modules linked in:
[ 0.004000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted
4.14.0-1-iniza-llvmlinux #1
[ 0.004000] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS 1.11.1-1 04/01/2014
[ 0.004000] task: ffffffffaa610480 task.stack: ffffffffaa600000
[ 0.004000] RIP: 0010:irq_work_tick+0x9d/0x110
[ 0.004000] RSP: 0000:ffff96dd9e403e58 EFLAGS: 00010046
[ 0.004000] RAX: 0000000000000082 RBX: ffff96dd9e411d80 RCX: adecc9cc04e2ca00
[ 0.004000] RDX: 000000000001ba00 RSI: fffffffffffffed4 RDI: ffff96dd9e41ba38
[ 0.004000] RBP: ffff96dd9e403e78 R08: 0000000000000000 R09: 0000000000000018
[ 0.004000] R10: 0000000000000000 R11: 0000000000000018 R12: ffffffffaa61bd40
[ 0.004000] R13: 0000000000000000 R14: ffffffffaa610480 R15: 0000000000000000
[ 0.004000] FS: 0000000000000000(0000) GS:ffff96dd9e400000(0000)
knlGS:0000000000000000
[ 0.004000] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 0.004000] CR2: 00000000ffffffff CR3: 0000000008a09000 CR4: 00000000000006b0
[ 0.004000] Call Trace:
[ 0.004000] <IRQ>
[ 0.004000] update_process_times+0x6e/0xa0
[ 0.004000] tick_periodic+0x78/0x90
[ 0.004000] tick_handle_periodic+0x26/0x80
[ 0.004000] timer_interrupt+0x13/0x20
[ 0.004000] __handle_irq_event_percpu+0x106/0x230
[ 0.004000] handle_irq_event+0x5a/0xc0
[ 0.004000] handle_level_irq+0x11a/0x190
[ 0.004000] handle_irq+0x1f/0x30
[ 0.004000] do_IRQ+0x4b/0xd0
[ 0.004000] common_interrupt+0x93/0x93
[ 0.004000] </IRQ>
[ 0.004000] RIP: 0010:native_restore_fl+0x12/0x20
[ 0.004000] RSP: 0000:ffffffffaa603e10 EFLAGS: 00000286 ORIG_RAX:
ffffffffffffffcf
[ 0.004000] RAX: 0000000000000001 RBX: ffff96dd9d0c0608 RCX: 0000000000000000
[ 0.004000] RDX: ffff96dd9d0c0400 RSI: 0000000000000286 RDI: 0000000000000286
[ 0.004000] RBP: ffffffffaa603e18 R08: 0000000000000001 R09: 000000000000003f
[ 0.004000] R10: 0000000000000286 R11: 0000000000000007 R12: ffff96dd9d0c0514
[ 0.004000] R13: ffff96dd9d0c04e0 R14: ffffffffaa61bd40 R15: ffff96dd9d0c0400
[ 0.004000] _raw_spin_unlock_irqrestore+0x1a/0x20
[ 0.004000] __setup_irq+0x5d9/0x780
[ 0.004000] setup_irq+0x5c/0x90
[ 0.004000] hpet_time_init+0x32/0x40
[ 0.004000] x86_late_time_init+0x10/0x20
[ 0.004000] start_kernel+0x45d/0x580
[ 0.004000] x86_64_start_kernel+0x30f/0x320
[ 0.004000] secondary_startup_64+0xa5/0xa5
[ 0.004000] Code: f0 f0 4c 0f b1 7b f8 4c 89 e3 4d 85 e4 75 cf 48
c7 c1 88 5b 01 00 65 48 03 0c 25 d8 a1 00 00 ff 14 25 10 08 62 aa f6
c4 02 75 4d <48> 83 39 00 74 3e 31 db 48 87 19 48 85 db 74 34 0f 1f 00
48 8d
[ 0.004000] RIP: irq_work_tick+0x9d/0x110 RSP: ffff96dd9e403e58
[ 0.004000] ---[ end trace 8c7851007fbc6b6f ]---
[ 0.004000] Kernel panic - not syncing: Fatal exception in interrupt
[ 0.004000] ---[ end Kernel panic - not syncing: Fatal exception in interrupt
Is this saying someting to you?
I am attaching my kernel-config file and the two additional patches.
Do you need further informations?
Thanks.
Regards,
- Sedat -
On Tue, Apr 24, 2018 at 01:54:29PM +0200, Sedat Dilek wrote:
> Hi Matthias,
>
> a big thank you for giving all the informations!
>
> I used your mka/llvm/v4.14_ext Git tree...
>
> https://chromium.googlesource.com/chromiumos/third_party/kernel/+log/sandbox/mka/llvm/v4.14_ext
>
> ...and was able to compile with clang-6.0 from Debian/buster64 with...
>
> ...reverting the clang-3/clang-4 patches...
>
> user$ for p in 69e44656ae43 222b88977a00 cdfcf1e45537 26f14c9225a6
> 0385a18e9995 68dab143c9b4 ; do echo [ $p ] ; LC_ALL=C git revert
> --no-edit $p ; done
>
> 0001-Revert-CLANG3-core-clang-work-around-x86-regparm-int.patch
> 0002-Revert-CLANG4-futex-don-t-optimize-futex_detect_cmpx.patch
> 0003-Revert-CLANG4-Disable-lkdtm-when-ftrace-is-enabled.patch
> 0004-Revert-CLANG4-arm64-prefetch-Use-__builtin_arm_prefe.patch
> 0005-Revert-CLANG4-kbuild-Add-meabi-gnu-to-the-clang-para.patch
> 0006-Revert-CLANG4-crypto-arm64-aes-ce-Explicitly-pass-th.patch
>
> ...and needed two additional patches from upstream:
>
> 0007-kbuild-clang-remove-crufty-HOSTCFLAGS.patch <--- Label with
> BACKPORT (XXX: Backported; Required when using HOSTCC in make-line,
> see below)
> 0008-x86-xen-remove-the-use-of-VLAIS.patch <--- Label with UPSTREAM
> (XXX: cherry-picked)
>
> These 3 patches in your Git branch are in Linux v4.14.36...
>
> 9af5ddf981ed BACKPORT: kbuild: disable clang's default use of
> -fmerge-all-constants
> f24088a3842c BACKPORT: kbuild: Set KBUILD_CFLAGS before incl. arch Makefile
> d4dfe384346d UPSTREAM: kbuild: fix linker feature test macros when
> cross compiling with Clang
>
> ...and can be dropped.
> I do not know your work-flow/policy: Maybe you want to keep your
> patch-stack against vanilla Linux v4.14 - without stables from
> linux-stable-4.14.y.
Yes, these patch stacks are based on vanilla Linux v4.14, one reason
is that it isn't a moving target. Also I think it makes it easier for
folks not merging LTS (though they probably should!) to locate all
patches, and it's fairly trivial to skip the (currently) few patches
not needed with the -stable tree.
> For easy switching "mycompiler" I use a wrapper-script:
>
> root# cat /usr/bin/mycompiler
> #!/bin/bash
>
> clang-6.0 "$@"
> - EOF -
>
> user$ cp -v /boot/config-4.14.35-1-iniza-amd64 .config
> user$ MAKE="make V=1" ; COMPILER="mycompiler" ;
> MAKE_OPTS="CC=$COMPILER HOSTCC=$COMPILER"
> user$ yes "" | $MAKE $MAKE_OPTS oldconfig && $MAKE $MAKE_OPTS
> silentoldconfig < /dev/null
>
> The diffconfig to my current kernel-config-4.14 looks like this...
>
> user$ ./scripts/diffconfig /boot/config-4.14.35-1-iniza-amd64 .config
> -ARCH_HAS_REFCOUNT y
> -BPF_JIT_ALWAYS_ON n
> -EXOFS_DEBUG n
> -EXOFS_FS m
> -GENERIC_CPU_VULNERABILITIES y
> -KASAN n
> -ORE m
> -PAGE_TABLE_ISOLATION y
> -RETPOLINE y
> -UNWINDER_FRAME_POINTER n
> -UNWINDER_GUESS n
> -UNWINDER_ORC y
> +FRAME_POINTER y
> +FRAME_POINTER_UNWINDER y
> +GUESS_UNWINDER n
> +HAVE_ARCH_KMEMCHECK y
> +HAVE_RELIABLE_STACKTRACE y
> +ORC_UNWINDER n
>
> Unfortunately, I cannot boot into the generated kernel on bare metal.
>
> Checking with QEMU (version: 2.12~rc3) and catching earlyprintk, I see this...
>
> user$ echo $KPATH
> $HOME/src/linux-kernel/important-files
>
> user$ ls -al
> insgesamt 344916
> drwxr-xr-x 2 sdi sdi 4096 Apr 24 13:15 .
> drwxr-xr-x 20 sdi sdi 4096 Apr 24 13:13 ..
> -rw-r--r-- 1 sdi sdi 4528416 Apr 24 12:42 bzImage
> lrwxrwxrwx 1 sdi sdi 35 Apr 24 13:15 initrd.img ->
> initrd.img-4.14.0-1-iniza-llvmlinux
> -rw-r--r-- 1 sdi sdi 25572955 Apr 24 13:08
> initrd.img-4.14.0-1-iniza-llvmlinux
> -rw-r--r-- 1 sdi sdi 2887195 Apr 24 12:42 System.map
> -rwxr-xr-x 1 sdi sdi 326116744 Apr 24 12:42 vmlinux
>
> user$ sudo qemu-system-x86_64 -enable-kvm -M pc -kernel $KPATH/bzImage
> -initrd $KPATH/initrd.img -m 512 -net none -serial stdio -append
> "root=/dev/ram0 console=ttyS0 hung_task_panic=1
> earlyprintk=ttyS0,115200"
>
> Probing EDD (edd=off to disable)... ok
> [ 0.000000] Linux version 4.14.0-1-iniza-llvmlinux
> ([email protected]@iniza) (clang version 6.0.0-1
> (tags/RELEASE_600/final)) #1 SMP Tue Apr 24 12:42:21 CEST 2018
> [ 0.000000] Command line: root=/dev/ram0 console=ttyS0
> hung_task_panic=1 earlyprintk=ttyS0,115200
> [ 0.000000] x86/fpu: x87 FPU will use FXSAVE
> [ 0.000000] e820: BIOS-provided physical RAM map:
> [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
> [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
> [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
> [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000001ffdffff] usable
> [ 0.000000] BIOS-e820: [mem 0x000000001ffe0000-0x000000001fffffff] reserved
> [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
> [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
> [ 0.000000] bootconsole [earlyser0] enabled
> [ 0.000000] NX (Execute Disable) protection: active
> [ 0.000000] random: fast init done
> [ 0.000000] SMBIOS 2.8 present.
> [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
> 1.11.1-1 04/01/2014
> [ 0.000000] Hypervisor detected: KVM
> [ 0.000000] tsc: Fast TSC calibration using PIT
> [ 0.000000] e820: last_pfn = 0x1ffe0 max_arch_pfn = 0x400000000
> [ 0.000000] x86/PAT: PAT not supported by CPU.
> [ 0.000000] x86/PAT: Configuration [0-7]: WB WT UC- UC WB WT UC- UC
> Memory KASLR using RDTSC...
> [ 0.000000] found SMP MP-table at [mem 0x000f5d60-0x000f5d6f]
> mapped at [ffffffffff000d60]
> [ 0.000000] RAMDISK: [mem 0x1e77c000-0x1ffdffff]
> [ 0.000000] ACPI: Early table checksum verification disabled
> [ 0.000000] ACPI: RSDP 0x00000000000F5B90 000014 (v00 BOCHS )
> [ 0.000000] ACPI: RSDT 0x000000001FFE157C 000030 (v01 BOCHS
> BXPCRSDT 00000001 BXPC 00000001)
> [ 0.000000] ACPI: FACP 0x000000001FFE1458 000074 (v01 BOCHS
> BXPCFACP 00000001 BXPC 00000001)
> [ 0.000000] ACPI: DSDT 0x000000001FFE0040 001418 (v01 BOCHS
> BXPCDSDT 00000001 BXPC 00000001)
> [ 0.000000] ACPI: FACS 0x000000001FFE0000 000040
> [ 0.000000] ACPI: APIC 0x000000001FFE14CC 000078 (v01 BOCHS
> BXPCAPIC 00000001 BXPC 00000001)
> [ 0.000000] ACPI: HPET 0x000000001FFE1544 000038 (v01 BOCHS
> BXPCHPET 00000001 BXPC 00000001)
> [ 0.000000] No NUMA configuration found
> [ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000001ffdffff]
> [ 0.000000] NODE_DATA(0) allocated [mem 0x1e777000-0x1e77bfff]
> [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
> [ 0.000000] kvm-clock: cpu 0, msr 0:1e76f001, primary cpu clock
> [ 0.000000] kvm-clock: using sched offset of 528742140 cycles
> [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff
> max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
> [ 0.000000] Zone ranges:
> [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
> [ 0.000000] DMA32 [mem 0x0000000001000000-0x000000001ffdffff]
> [ 0.000000] Normal empty
> [ 0.000000] Device empty
> [ 0.000000] Movable zone start for each node
> [ 0.000000] Early memory node ranges
> [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff]
> [ 0.000000] node 0: [mem 0x0000000000100000-0x000000001ffdffff]
> [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x000000001ffdffff]
> [ 0.000000] ACPI: PM-Timer IO Port: 0x608
> [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
> [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
> [ 0.000000] Using ACPI (MADT) for SMP configuration information
> [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000
> [ 0.000000] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
> [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]
> [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
> [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff]
> [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff]
> [ 0.000000] e820: [mem 0x20000000-0xfeffbfff] available for PCI devices
> [ 0.000000] Booting paravirtualized kernel on KVM
> [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff
> max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
> [ 0.000000] setup_percpu: NR_CPUS:512 nr_cpumask_bits:512
> nr_cpu_ids:1 nr_node_ids:1
> [ 0.000000] percpu: Embedded 37 pages/cpu @ffff96dd9e400000 s114392
> r8192 d28968 u2097152
> [ 0.000000] KVM setup async PF for cpu 0
> [ 0.000000] kvm-stealtime: cpu 0, msr 1e40d900
> [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 128872
> [ 0.000000] Policy zone: DMA32
> [ 0.000000] Kernel command line: root=/dev/ram0 console=ttyS0
> hung_task_panic=1 earlyprintk=ttyS0,115200
> [ 0.000000] PID hash table entries: 2048 (order: 2, 16384 bytes)
> [ 0.000000] Memory: 474480K/523768K available (7639K kernel code,
> 1005K rwdata, 2936K rodata, 1636K init, 688K bss, 49288K reserved, 0K
> cma-reserved)
> [ 0.000000] ftrace: allocating 27586 entries in 108 pages
> [ 0.004000] Hierarchical RCU implementation.
> [ 0.004000] RCU restricting CPUs from NR_CPUS=512 to nr_cpu_ids=1.
> [ 0.004000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
> [ 0.004000] NR_IRQS: 33024, nr_irqs: 256, preallocated irqs: 16
> [ 0.004000] Console: colour VGA+ 80x25
> [ 0.004000] console [ttyS0] enabled
> [ 0.004000] console [ttyS0] enabled
> [ 0.004000] bootconsole [earlyser0] disabled
> [ 0.004000] bootconsole [earlyser0] disabled
> [ 0.004000] clocksource: hpet: mask: 0xffffffff max_cycles:
> 0xffffffff, max_idle_ns: 19112604467 ns
> [ 0.004000] general protection fault: 0000 [#1] SMP
> [ 0.004000] Modules linked in:
> [ 0.004000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted
> 4.14.0-1-iniza-llvmlinux #1
> [ 0.004000] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
> BIOS 1.11.1-1 04/01/2014
> [ 0.004000] task: ffffffffaa610480 task.stack: ffffffffaa600000
> [ 0.004000] RIP: 0010:irq_work_tick+0x9d/0x110
> [ 0.004000] RSP: 0000:ffff96dd9e403e58 EFLAGS: 00010046
> [ 0.004000] RAX: 0000000000000082 RBX: ffff96dd9e411d80 RCX: adecc9cc04e2ca00
> [ 0.004000] RDX: 000000000001ba00 RSI: fffffffffffffed4 RDI: ffff96dd9e41ba38
> [ 0.004000] RBP: ffff96dd9e403e78 R08: 0000000000000000 R09: 0000000000000018
> [ 0.004000] R10: 0000000000000000 R11: 0000000000000018 R12: ffffffffaa61bd40
> [ 0.004000] R13: 0000000000000000 R14: ffffffffaa610480 R15: 0000000000000000
> [ 0.004000] FS: 0000000000000000(0000) GS:ffff96dd9e400000(0000)
> knlGS:0000000000000000
> [ 0.004000] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 0.004000] CR2: 00000000ffffffff CR3: 0000000008a09000 CR4: 00000000000006b0
> [ 0.004000] Call Trace:
> [ 0.004000] <IRQ>
> [ 0.004000] update_process_times+0x6e/0xa0
> [ 0.004000] tick_periodic+0x78/0x90
> [ 0.004000] tick_handle_periodic+0x26/0x80
> [ 0.004000] timer_interrupt+0x13/0x20
> [ 0.004000] __handle_irq_event_percpu+0x106/0x230
> [ 0.004000] handle_irq_event+0x5a/0xc0
> [ 0.004000] handle_level_irq+0x11a/0x190
> [ 0.004000] handle_irq+0x1f/0x30
> [ 0.004000] do_IRQ+0x4b/0xd0
> [ 0.004000] common_interrupt+0x93/0x93
> [ 0.004000] </IRQ>
> [ 0.004000] RIP: 0010:native_restore_fl+0x12/0x20
> [ 0.004000] RSP: 0000:ffffffffaa603e10 EFLAGS: 00000286 ORIG_RAX:
> ffffffffffffffcf
> [ 0.004000] RAX: 0000000000000001 RBX: ffff96dd9d0c0608 RCX: 0000000000000000
> [ 0.004000] RDX: ffff96dd9d0c0400 RSI: 0000000000000286 RDI: 0000000000000286
> [ 0.004000] RBP: ffffffffaa603e18 R08: 0000000000000001 R09: 000000000000003f
> [ 0.004000] R10: 0000000000000286 R11: 0000000000000007 R12: ffff96dd9d0c0514
> [ 0.004000] R13: ffff96dd9d0c04e0 R14: ffffffffaa61bd40 R15: ffff96dd9d0c0400
> [ 0.004000] _raw_spin_unlock_irqrestore+0x1a/0x20
> [ 0.004000] __setup_irq+0x5d9/0x780
> [ 0.004000] setup_irq+0x5c/0x90
> [ 0.004000] hpet_time_init+0x32/0x40
> [ 0.004000] x86_late_time_init+0x10/0x20
> [ 0.004000] start_kernel+0x45d/0x580
> [ 0.004000] x86_64_start_kernel+0x30f/0x320
> [ 0.004000] secondary_startup_64+0xa5/0xa5
> [ 0.004000] Code: f0 f0 4c 0f b1 7b f8 4c 89 e3 4d 85 e4 75 cf 48
> c7 c1 88 5b 01 00 65 48 03 0c 25 d8 a1 00 00 ff 14 25 10 08 62 aa f6
> c4 02 75 4d <48> 83 39 00 74 3e 31 db 48 87 19 48 85 db 74 34 0f 1f 00
> 48 8d
> [ 0.004000] RIP: irq_work_tick+0x9d/0x110 RSP: ffff96dd9e403e58
> [ 0.004000] ---[ end trace 8c7851007fbc6b6f ]---
> [ 0.004000] Kernel panic - not syncing: Fatal exception in interrupt
> [ 0.004000] ---[ end Kernel panic - not syncing: Fatal exception in interrupt
>
> Is this saying someting to you?
It doesn't ring a bell, but I can repro it (with different offsets):
[ 0.004000] general protection fault: 0000 [#1] SMP
[ 0.004000] Modules linked in:
[ 0.004000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.14.0+ #10
[ 0.004000] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 0.004000] task: ffffffffbb410480 task.stack: ffffffffbb400000
[ 0.004000] RIP: 0010:irq_work_tick+0xaf/0x120
[ 0.004000] RSP: 0000:ffff96fb5fc03e58 EFLAGS: 00010046
[ 0.004000] RAX: 0000000000000082 RBX: ffff96fb5fc11d80 RCX: 47e992c1bc778d00
[ 0.004000] RDX: 000000000001ba00 RSI: fffffffffffffed4 RDI: ffff96fb5fc1ba38
[ 0.004000] RBP: ffff96fb5fc03e78 R08: 0000000000000000 R09: 0000000000000018
[ 0.004000] R10: 0000000000000000 R11: 0000000000000018 R12: ffffffffbb41bd40
[ 0.004000] R13: 0000000000000000 R14: ffffffffbb410480 R15: 0000000000000000
[ 0.004000] FS: 0000000000000000(0000) GS:ffff96fb5fc00000(0000) knlGS:0000000000000000
[ 0.004000] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 0.004000] CR2: 00000000ffffffff CR3: 000000001ec09000 CR4: 00000000000006b0
[ 0.004000] Call Trace:
[ 0.004000] <IRQ>
[ 0.004000] update_process_times+0x6e/0xa0
[ 0.004000] tick_periodic+0x78/0x90
[ 0.004000] tick_handle_periodic+0x26/0x80
[ 0.004000] timer_interrupt+0x13/0x20
[ 0.004000] __handle_irq_event_percpu+0x106/0x230
[ 0.004000] handle_irq_event+0x5a/0xc0
[ 0.004000] handle_level_irq+0x11a/0x190
[ 0.004000] handle_irq+0x1f/0x30
[ 0.004000] do_IRQ+0x4b/0xd0
[ 0.004000] common_interrupt+0x93/0x93
[ 0.004000] </IRQ>
[ 0.004000] RIP: 0010:native_restore_fl+0xd/0x20
[ 0.004000] RSP: 0000:ffffffffbb403e08 EFLAGS: 00000282 ORIG_RAX: ffffffffffffffcf
[ 0.004000] RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000000
[ 0.004000] RDX: ffff96fb5f01b800 RSI: 0000000000000282 RDI: 0000000000000282
[ 0.004000] RBP: ffffffffbb403e10 R08: 0000000000000001 R09: 000000000000003f
[ 0.004000] R10: ffff96fb5f01b800 R11: 0000000000000007 R12: ffff96fb5f01ba08
[ 0.004000] R13: ffff96fb5f01b8e0 R14: ffffffffbb41bd40 R15: ffff96fb5f01b800
[ 0.004000] _raw_spin_unlock_irqrestore+0x1a/0x20
[ 0.004000] __setup_irq+0x610/0x7b0
[ 0.004000] setup_irq+0x5b/0x90
[ 0.004000] hpet_time_init+0x32/0x40
[ 0.004000] x86_late_time_init+0x10/0x20
[ 0.004000] start_kernel+0x460/0x580
[ 0.004000] x86_64_start_kernel+0x30f/0x320
[ 0.004000] secondary_startup_64+0xa5/0xa5
[ 0.004000] Code: 4c 0f b1 7b f8 4c 89 e3 4d 85 e4 75 cf 48 c7 c1 88 5b 01 00 65 48 03 0c 25 d8 a1 00 00 ff 14 25 10 08 42 bb a9 00 02 00 00 75 4b <48> 83 39 00 74 3c 31 db 48 87 19 48 85 db 74 32 90 48 8d 7b f8
[ 0.004000] RIP: irq_work_tick+0xaf/0x120 RSP: ffff96fb5fc03e58
[ 0.004000] ---[ end trace 08945838e05bf5b2 ]---
[ 0.004000] Kernel panic - not syncing: Fatal exception in interrupt
[ 0.004000] ---[ end Kernel panic - not syncing: Fatal exception in interrupt
The exception occurs at 'irq_work_tick+0xaf'
objdump -d -S --start-address=0x$(grep irq_work_tick System.map | sed -e
"s/ \+.*//") vmlinux | less
...
void irq_work_tick(void)
{
ffffffff81193200: 55 push %rbp
...
static inline struct llist_node *llist_del_all(struct llist_head *head)
{
return xchg(&head->first, NULL);
ffffffff8119324a: 48 87 19 xchg %rbx,(%rcx)
while (llnode != NULL) {
ffffffff8119324d: 48 85 db test %rbx,%rbx
ffffffff81193250: 74 3f je ffffffff81193291 <irq_work_tick+0x91>
ffffffff81193252: 0f 1f 40 00 nopl 0x0(%rax)
ffffffff81193256: 66 2e 0f 1f 84 00 00 nopw %cs:0x0(%rax,%rax,1)
ffffffff8119325d: 00 00 00
work = llist_entry(llnode, struct irq_work, llnode);
ffffffff81193260: 48 8d 7b f8 lea -0x8(%rbx),%rdi
flags = work->flags & ~IRQ_WORK_PENDING;
ffffffff81193264: 4c 8b 7b f8 mov -0x8(%rbx),%r15
return node->next;
ffffffff81193268: 4c 8b 23 mov (%rbx),%r12
ffffffff8119326b: 4d 89 fe mov %r15,%r14
ffffffff8119326e: 49 83 e6 fe and $0xfffffffffffffffe,%r14
xchg(&work->flags, flags);
ffffffff81193272: 4c 89 f0 mov %r14,%rax
ffffffff81193275: 48 87 43 f8 xchg %rax,-0x8(%rbx)
work->func(work);
ffffffff81193279: ff 53 08 callq *0x8(%rbx)
(void)cmpxchg(&work->flags, flags, flags & ~IRQ_WORK_BUSY);
ffffffff8119327c: 49 83 e7 fc and $0xfffffffffffffffc,%r15
ffffffff81193280: 4c 89 f0 mov %r14,%rax
ffffffff81193283: f0 4c 0f b1 7b f8 lock cmpxchg %r15,-0x8(%rbx)
ffffffff81193289: 4c 89 e3 mov %r12,%rbx
while (llnode != NULL) {
ffffffff8119328c: 4d 85 e4 test %r12,%r12
ffffffff8119328f: 75 cf jne ffffffff81193260 <irq_work_tick+0x60>
irq_work_run_list(raised);
irq_work_run_list(this_cpu_ptr(&lazy_list));
ffffffff81193291: 48 c7 c1 88 5b 01 00 mov $0x15b88,%rcx
ffffffff81193298: 65 48 03 0c 25 d8 a1 add %gs:0xa1d8,%rcx
ffffffff8119329f: 00 00
ffffffff811932a1: ff 14 25 10 08 c2 81 callq *0xffffffff81c20810
BUG_ON(!irqs_disabled());
ffffffff811932a8: a9 00 02 00 00 test $0x200,%eax
ffffffff811932ad: 75 4b jne ffffffff811932fa <irq_work_tick+0xfa>
return ACCESS_ONCE(head->first) == NULL;
ffffffff811932af: 48 83 39 00 cmpq $0x0,(%rcx)
I'm no x86/assembly expert, anyway my interpretation is:
'irq_work_tick+0xaf' corresponds to address 0xffffffff811932af, so the
exception occurs during the execution of 'cmpq $0x0,(%rcx)' or
'ACCESS_ONCE(head->first) == NULL'. The instruction checks if the
memory location in RCX contains 0. Supposedly the register should
contain the address of 'head->first', however the value is
0x47e992c1bc778d00, which doesn't look like a valid address.
RCX is set shortly before:
ffffffff81193291: 48 c7 c1 88 5b 01 00 mov $0x15b88,%rcx
ffffffff81193298: 65 48 03 0c 25 d8 a1 add %gs:0xa1d8,%rcx
As stated in https://www.kernel.org/doc/Documentation/this_cpu_ops.txt
the GS segment register is used by the kernel for per-cpu
variables. And effectively System.map tells us the offset 0x15b88
corresponds to the per-cpu variable lazy_list:
0000000000015b88 d lazy_list
The register dump shows GS with a value of 0xffff96fb5fc00000, which
looks reasonable.
But wait, right after setting RCX we do this:
ffffffff811932a1: ff 14 25 10 08 c2 81 callq *0xffffffff81c20810
BUG_ON(!irqs_disabled());
According to my (limited) understanding of x86 calling conventions RCX
is a caller-saved register, thus the caller should save it on the
stack to preserve its value across a function call.
(https://en.wikipedia.org/wiki/X86_calling_conventions)
The call to '*0xffffffff81c20810' appears to be related with
KVM/paravirtualization:
grep ffffffff81c20810 System.map
ffffffff81c20810 D pv_irq_ops
Since there is no offset it must be calling the first function in the
structure, which is 'safe_fl' and points to 'native_save_fl'
(https://elixir.bootlin.com/linux/v4.14.36/source/arch/x86/kernel/paravirt.c#L316)
objdump -d -S --start-address=0x$(grep native_save_fl System.map | sed
-e "s/ \+.*//") vmlinux | less
static inline unsigned long native_save_fl(void)
{
ffffffff81060240: 55 push %rbp
ffffffff81060241: 48 89 e5 mov %rsp,%rbp
ffffffff81060244: 48 83 ec 10 sub $0x10,%rsp
ffffffff81060248: 65 48 8b 04 25 28 00 mov %gs:0x28,%rax
ffffffff8106024f: 00 00
ffffffff81060251: 48 89 45 f8 mov %rax,-0x8(%rbp)
/*
* "=rm" is safe here, because "pop" adjusts the stack before
* it evaluates its effective address -- this is part of the
* documented behavior of the "pop" instruction.
*/
asm volatile("# __raw_save_flags\n\t"
ffffffff81060255: 9c pushfq
ffffffff81060256: 8f 45 f0 popq -0x10(%rbp)
"pushf ; pop %0"
: "=rm" (flags)
: /* no input */
: "memory");
return flags;
ffffffff81060259: 48 8b 45 f0 mov -0x10(%rbp),%rax
ffffffff8106025d: 65 48 8b 0c 25 28 00 mov %gs:0x28,%rcx
ffffffff81060264: 00 00
ffffffff81060266: 48 3b 4d f8 cmp -0x8(%rbp),%rcx
ffffffff8106026a: 75 06 jne ffffffff81060272 <native_save_fl+0x32>
ffffffff8106026c: 48 83 c4 10 add $0x10,%rsp
ffffffff81060270: 5d pop %rbp
ffffffff81060271: c3 retq
ffffffff81060272: e8 09 ec 01 00 callq ffffffff8107ee80 <__stack_chk_fail>
ffffffff81060277: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
ffffffff8106027e: 00 0
At 0xffffffff8106025d this clobbers RCX! I don't know why clang
doesn't save the value on the stack before calling native_save_fl(),
but that seems to be the problem.
Again, I'm not an expert in this area and ventured into territory
unknown to me, so please excuse if I got something totally wrong ...
For the record: a Chrome OS v4.14.35 kernel ('based' on the
sandbox/mka stack) built with clang still boots on an actual x86
device.
Matthias
On Mon, Apr 23, 2018 at 7:42 PM, Matthias Kaehlcke <[email protected]> wrote:
[...]
>> [ ASM-GOTO ]
>>
>> Foremore, I have seen you have a "refs/sandbox/mka/llvm/v4.14" Git branch.
>> Linux v4.14 is also an LTS release.
>
> Yes, there is also a follow up post:
> https://lkml.org/lkml/2017/11/22/943
>
>> After Linux v4.9 "asm-goto" support was added which is GCC specific.
>
> asm-goto was optional until a few weeks ago, when x86 maintainers
> decided to make it mandatory, and thus break clang builds for x86:
>
> https://lkml.org/lkml/2018/4/2/486
>
>> Is there a workaround to compile Linux-kernel with any CLANG
>> version?
>
> You could revert the patch that makes asm-goto mandatory.
>
>> You happen to know the status in LLVM upstream?
>
> I know people are actively working on this, but don't know an ETA.
>
[ CC Dimitry ]
Sorry for the late response.
This weekend I had some time to play with clang-7 and Linux-4.17-rcN.
I tried with Linux v4.17-rc3-261-gc1c07416cdd4 and reverted the
mandatory patch [1].
commit e501ce957a786ecd076ea0cfb10b114e6e4d0f40
"x86: Force asm-goto"
This needed a follow up [2]...
commit d0266046ad54e0c964941364cd82a0d0478ce286
"x86: Remove FAST_FEATURE_TESTS"
...but breaks my build.
That seems to be a known issue [3] and Dimitry bisected this to [4].
Is there a fix for this?
I have attached my kernel-config and the broken build-log.
Hope this helps.
Regards,
- Sedat -
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e501ce957a786ecd076ea0cfb10b114e6e4d0f40
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/arch/x86/include/asm/cpufeature.h?id=d0266046ad54e0c964941364cd82a0d0478ce286
[3] https://www.spinics.net/lists/linux-mm/msg147669.html
[4] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=8bf705d130396e69c04cd8e6e010244ad2ce71f4
On Sun, May 6, 2018 at 8:35 AM, Sedat Dilek <[email protected]> wrote:
> On Mon, Apr 23, 2018 at 7:42 PM, Matthias Kaehlcke <[email protected]> wrote:
> [...]
>>> [ ASM-GOTO ]
>>>
>>> Foremore, I have seen you have a "refs/sandbox/mka/llvm/v4.14" Git branch.
>>> Linux v4.14 is also an LTS release.
>>
>> Yes, there is also a follow up post:
>> https://lkml.org/lkml/2017/11/22/943
>>
>>> After Linux v4.9 "asm-goto" support was added which is GCC specific.
>>
>> asm-goto was optional until a few weeks ago, when x86 maintainers
>> decided to make it mandatory, and thus break clang builds for x86:
>>
>> https://lkml.org/lkml/2018/4/2/486
>>
>>> Is there a workaround to compile Linux-kernel with any CLANG
>>> version?
>>
>> You could revert the patch that makes asm-goto mandatory.
>>
>>> You happen to know the status in LLVM upstream?
>>
>> I know people are actively working on this, but don't know an ETA.
>>
>
> [ CC Dimitry ]
>
> Sorry for the late response.
> This weekend I had some time to play with clang-7 and Linux-4.17-rcN.
>
> I tried with Linux v4.17-rc3-261-gc1c07416cdd4 and reverted the
> mandatory patch [1].
>
> commit e501ce957a786ecd076ea0cfb10b114e6e4d0f40
> "x86: Force asm-goto"
>
> This needed a follow up [2]...
>
> commit d0266046ad54e0c964941364cd82a0d0478ce286
> "x86: Remove FAST_FEATURE_TESTS"
>
> ...but breaks my build.
>
> That seems to be a known issue [3] and Dimitry bisected this to [4].
> Is there a fix for this?
>
> I have attached my kernel-config and the broken build-log.
>
> Hope this helps.
>
> Regards,
> - Sedat -
>
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e501ce957a786ecd076ea0cfb10b114e6e4d0f40
> [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/arch/x86/include/asm/cpufeature.h?id=d0266046ad54e0c964941364cd82a0d0478ce286
> [3] https://www.spinics.net/lists/linux-mm/msg147669.html
> [4] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=8bf705d130396e69c04cd8e6e010244ad2ce71f4
Hi Sedat,
Sorry for the breakage.
This was reported and discussed here:
https://groups.google.com/forum/#!topic/kasan-dev/oMgCP37n1vw
The plan is to make clang behave the same way as gcc wrt asm
constraint checking. But I don't know what's the progress.
On Wed, Apr 25, 2018 at 1:06 AM, Matthias Kaehlcke <[email protected]> wrote:
> On Tue, Apr 24, 2018 at 01:54:29PM +0200, Sedat Dilek wrote:
>> Hi Matthias,
>>
>> a big thank you for giving all the informations!
>>
>> I used your mka/llvm/v4.14_ext Git tree...
>>
>> https://chromium.googlesource.com/chromiumos/third_party/kernel/+log/sandbox/mka/llvm/v4.14_ext
>>
>> ...and was able to compile with clang-6.0 from Debian/buster64 with...
>>
>> ...reverting the clang-3/clang-4 patches...
>>
>> user$ for p in 69e44656ae43 222b88977a00 cdfcf1e45537 26f14c9225a6
>> 0385a18e9995 68dab143c9b4 ; do echo [ $p ] ; LC_ALL=C git revert
>> --no-edit $p ; done
>>
>> 0001-Revert-CLANG3-core-clang-work-around-x86-regparm-int.patch
>> 0002-Revert-CLANG4-futex-don-t-optimize-futex_detect_cmpx.patch
>> 0003-Revert-CLANG4-Disable-lkdtm-when-ftrace-is-enabled.patch
>> 0004-Revert-CLANG4-arm64-prefetch-Use-__builtin_arm_prefe.patch
>> 0005-Revert-CLANG4-kbuild-Add-meabi-gnu-to-the-clang-para.patch
>> 0006-Revert-CLANG4-crypto-arm64-aes-ce-Explicitly-pass-th.patch
>>
>> ...and needed two additional patches from upstream:
>>
>> 0007-kbuild-clang-remove-crufty-HOSTCFLAGS.patch <--- Label with
>> BACKPORT (XXX: Backported; Required when using HOSTCC in make-line,
>> see below)
>> 0008-x86-xen-remove-the-use-of-VLAIS.patch <--- Label with UPSTREAM
>> (XXX: cherry-picked)
>>
>> These 3 patches in your Git branch are in Linux v4.14.36...
>>
>> 9af5ddf981ed BACKPORT: kbuild: disable clang's default use of
>> -fmerge-all-constants
>> f24088a3842c BACKPORT: kbuild: Set KBUILD_CFLAGS before incl. arch Makefile
>> d4dfe384346d UPSTREAM: kbuild: fix linker feature test macros when
>> cross compiling with Clang
>>
>> ...and can be dropped.
>> I do not know your work-flow/policy: Maybe you want to keep your
>> patch-stack against vanilla Linux v4.14 - without stables from
>> linux-stable-4.14.y.
>
> Yes, these patch stacks are based on vanilla Linux v4.14, one reason
> is that it isn't a moving target. Also I think it makes it easier for
> folks not merging LTS (though they probably should!) to locate all
> patches, and it's fairly trivial to skip the (currently) few patches
> not needed with the -stable tree.
>
>> For easy switching "mycompiler" I use a wrapper-script:
>>
>> root# cat /usr/bin/mycompiler
>> #!/bin/bash
>>
>> clang-6.0 "$@"
>> - EOF -
>>
>> user$ cp -v /boot/config-4.14.35-1-iniza-amd64 .config
>> user$ MAKE="make V=1" ; COMPILER="mycompiler" ;
>> MAKE_OPTS="CC=$COMPILER HOSTCC=$COMPILER"
>> user$ yes "" | $MAKE $MAKE_OPTS oldconfig && $MAKE $MAKE_OPTS
>> silentoldconfig < /dev/null
>>
>> The diffconfig to my current kernel-config-4.14 looks like this...
>>
>> user$ ./scripts/diffconfig /boot/config-4.14.35-1-iniza-amd64 .config
>> -ARCH_HAS_REFCOUNT y
>> -BPF_JIT_ALWAYS_ON n
>> -EXOFS_DEBUG n
>> -EXOFS_FS m
>> -GENERIC_CPU_VULNERABILITIES y
>> -KASAN n
>> -ORE m
>> -PAGE_TABLE_ISOLATION y
>> -RETPOLINE y
>> -UNWINDER_FRAME_POINTER n
>> -UNWINDER_GUESS n
>> -UNWINDER_ORC y
>> +FRAME_POINTER y
>> +FRAME_POINTER_UNWINDER y
>> +GUESS_UNWINDER n
>> +HAVE_ARCH_KMEMCHECK y
>> +HAVE_RELIABLE_STACKTRACE y
>> +ORC_UNWINDER n
>>
>> Unfortunately, I cannot boot into the generated kernel on bare metal.
>>
>> Checking with QEMU (version: 2.12~rc3) and catching earlyprintk, I see this...
>>
>> user$ echo $KPATH
>> $HOME/src/linux-kernel/important-files
>>
>> user$ ls -al
>> insgesamt 344916
>> drwxr-xr-x 2 sdi sdi 4096 Apr 24 13:15 .
>> drwxr-xr-x 20 sdi sdi 4096 Apr 24 13:13 ..
>> -rw-r--r-- 1 sdi sdi 4528416 Apr 24 12:42 bzImage
>> lrwxrwxrwx 1 sdi sdi 35 Apr 24 13:15 initrd.img ->
>> initrd.img-4.14.0-1-iniza-llvmlinux
>> -rw-r--r-- 1 sdi sdi 25572955 Apr 24 13:08
>> initrd.img-4.14.0-1-iniza-llvmlinux
>> -rw-r--r-- 1 sdi sdi 2887195 Apr 24 12:42 System.map
>> -rwxr-xr-x 1 sdi sdi 326116744 Apr 24 12:42 vmlinux
>>
>> user$ sudo qemu-system-x86_64 -enable-kvm -M pc -kernel $KPATH/bzImage
>> -initrd $KPATH/initrd.img -m 512 -net none -serial stdio -append
>> "root=/dev/ram0 console=ttyS0 hung_task_panic=1
>> earlyprintk=ttyS0,115200"
>>
>> Probing EDD (edd=off to disable)... ok
>> [ 0.000000] Linux version 4.14.0-1-iniza-llvmlinux
>> ([email protected]@iniza) (clang version 6.0.0-1
>> (tags/RELEASE_600/final)) #1 SMP Tue Apr 24 12:42:21 CEST 2018
>> [ 0.000000] Command line: root=/dev/ram0 console=ttyS0
>> hung_task_panic=1 earlyprintk=ttyS0,115200
>> [ 0.000000] x86/fpu: x87 FPU will use FXSAVE
>> [ 0.000000] e820: BIOS-provided physical RAM map:
>> [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
>> [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
>> [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
>> [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000001ffdffff] usable
>> [ 0.000000] BIOS-e820: [mem 0x000000001ffe0000-0x000000001fffffff] reserved
>> [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
>> [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
>> [ 0.000000] bootconsole [earlyser0] enabled
>> [ 0.000000] NX (Execute Disable) protection: active
>> [ 0.000000] random: fast init done
>> [ 0.000000] SMBIOS 2.8 present.
>> [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
>> 1.11.1-1 04/01/2014
>> [ 0.000000] Hypervisor detected: KVM
>> [ 0.000000] tsc: Fast TSC calibration using PIT
>> [ 0.000000] e820: last_pfn = 0x1ffe0 max_arch_pfn = 0x400000000
>> [ 0.000000] x86/PAT: PAT not supported by CPU.
>> [ 0.000000] x86/PAT: Configuration [0-7]: WB WT UC- UC WB WT UC- UC
>> Memory KASLR using RDTSC...
>> [ 0.000000] found SMP MP-table at [mem 0x000f5d60-0x000f5d6f]
>> mapped at [ffffffffff000d60]
>> [ 0.000000] RAMDISK: [mem 0x1e77c000-0x1ffdffff]
>> [ 0.000000] ACPI: Early table checksum verification disabled
>> [ 0.000000] ACPI: RSDP 0x00000000000F5B90 000014 (v00 BOCHS )
>> [ 0.000000] ACPI: RSDT 0x000000001FFE157C 000030 (v01 BOCHS
>> BXPCRSDT 00000001 BXPC 00000001)
>> [ 0.000000] ACPI: FACP 0x000000001FFE1458 000074 (v01 BOCHS
>> BXPCFACP 00000001 BXPC 00000001)
>> [ 0.000000] ACPI: DSDT 0x000000001FFE0040 001418 (v01 BOCHS
>> BXPCDSDT 00000001 BXPC 00000001)
>> [ 0.000000] ACPI: FACS 0x000000001FFE0000 000040
>> [ 0.000000] ACPI: APIC 0x000000001FFE14CC 000078 (v01 BOCHS
>> BXPCAPIC 00000001 BXPC 00000001)
>> [ 0.000000] ACPI: HPET 0x000000001FFE1544 000038 (v01 BOCHS
>> BXPCHPET 00000001 BXPC 00000001)
>> [ 0.000000] No NUMA configuration found
>> [ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000001ffdffff]
>> [ 0.000000] NODE_DATA(0) allocated [mem 0x1e777000-0x1e77bfff]
>> [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
>> [ 0.000000] kvm-clock: cpu 0, msr 0:1e76f001, primary cpu clock
>> [ 0.000000] kvm-clock: using sched offset of 528742140 cycles
>> [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff
>> max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
>> [ 0.000000] Zone ranges:
>> [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
>> [ 0.000000] DMA32 [mem 0x0000000001000000-0x000000001ffdffff]
>> [ 0.000000] Normal empty
>> [ 0.000000] Device empty
>> [ 0.000000] Movable zone start for each node
>> [ 0.000000] Early memory node ranges
>> [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff]
>> [ 0.000000] node 0: [mem 0x0000000000100000-0x000000001ffdffff]
>> [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x000000001ffdffff]
>> [ 0.000000] ACPI: PM-Timer IO Port: 0x608
>> [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
>> [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
>> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
>> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
>> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
>> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
>> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
>> [ 0.000000] Using ACPI (MADT) for SMP configuration information
>> [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000
>> [ 0.000000] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
>> [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]
>> [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
>> [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff]
>> [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff]
>> [ 0.000000] e820: [mem 0x20000000-0xfeffbfff] available for PCI devices
>> [ 0.000000] Booting paravirtualized kernel on KVM
>> [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff
>> max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
>> [ 0.000000] setup_percpu: NR_CPUS:512 nr_cpumask_bits:512
>> nr_cpu_ids:1 nr_node_ids:1
>> [ 0.000000] percpu: Embedded 37 pages/cpu @ffff96dd9e400000 s114392
>> r8192 d28968 u2097152
>> [ 0.000000] KVM setup async PF for cpu 0
>> [ 0.000000] kvm-stealtime: cpu 0, msr 1e40d900
>> [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 128872
>> [ 0.000000] Policy zone: DMA32
>> [ 0.000000] Kernel command line: root=/dev/ram0 console=ttyS0
>> hung_task_panic=1 earlyprintk=ttyS0,115200
>> [ 0.000000] PID hash table entries: 2048 (order: 2, 16384 bytes)
>> [ 0.000000] Memory: 474480K/523768K available (7639K kernel code,
>> 1005K rwdata, 2936K rodata, 1636K init, 688K bss, 49288K reserved, 0K
>> cma-reserved)
>> [ 0.000000] ftrace: allocating 27586 entries in 108 pages
>> [ 0.004000] Hierarchical RCU implementation.
>> [ 0.004000] RCU restricting CPUs from NR_CPUS=512 to nr_cpu_ids=1.
>> [ 0.004000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
>> [ 0.004000] NR_IRQS: 33024, nr_irqs: 256, preallocated irqs: 16
>> [ 0.004000] Console: colour VGA+ 80x25
>> [ 0.004000] console [ttyS0] enabled
>> [ 0.004000] console [ttyS0] enabled
>> [ 0.004000] bootconsole [earlyser0] disabled
>> [ 0.004000] bootconsole [earlyser0] disabled
>> [ 0.004000] clocksource: hpet: mask: 0xffffffff max_cycles:
>> 0xffffffff, max_idle_ns: 19112604467 ns
>> [ 0.004000] general protection fault: 0000 [#1] SMP
>> [ 0.004000] Modules linked in:
>> [ 0.004000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted
>> 4.14.0-1-iniza-llvmlinux #1
>> [ 0.004000] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
>> BIOS 1.11.1-1 04/01/2014
>> [ 0.004000] task: ffffffffaa610480 task.stack: ffffffffaa600000
>> [ 0.004000] RIP: 0010:irq_work_tick+0x9d/0x110
>> [ 0.004000] RSP: 0000:ffff96dd9e403e58 EFLAGS: 00010046
>> [ 0.004000] RAX: 0000000000000082 RBX: ffff96dd9e411d80 RCX: adecc9cc04e2ca00
>> [ 0.004000] RDX: 000000000001ba00 RSI: fffffffffffffed4 RDI: ffff96dd9e41ba38
>> [ 0.004000] RBP: ffff96dd9e403e78 R08: 0000000000000000 R09: 0000000000000018
>> [ 0.004000] R10: 0000000000000000 R11: 0000000000000018 R12: ffffffffaa61bd40
>> [ 0.004000] R13: 0000000000000000 R14: ffffffffaa610480 R15: 0000000000000000
>> [ 0.004000] FS: 0000000000000000(0000) GS:ffff96dd9e400000(0000)
>> knlGS:0000000000000000
>> [ 0.004000] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> [ 0.004000] CR2: 00000000ffffffff CR3: 0000000008a09000 CR4: 00000000000006b0
>> [ 0.004000] Call Trace:
>> [ 0.004000] <IRQ>
>> [ 0.004000] update_process_times+0x6e/0xa0
>> [ 0.004000] tick_periodic+0x78/0x90
>> [ 0.004000] tick_handle_periodic+0x26/0x80
>> [ 0.004000] timer_interrupt+0x13/0x20
>> [ 0.004000] __handle_irq_event_percpu+0x106/0x230
>> [ 0.004000] handle_irq_event+0x5a/0xc0
>> [ 0.004000] handle_level_irq+0x11a/0x190
>> [ 0.004000] handle_irq+0x1f/0x30
>> [ 0.004000] do_IRQ+0x4b/0xd0
>> [ 0.004000] common_interrupt+0x93/0x93
>> [ 0.004000] </IRQ>
>> [ 0.004000] RIP: 0010:native_restore_fl+0x12/0x20
>> [ 0.004000] RSP: 0000:ffffffffaa603e10 EFLAGS: 00000286 ORIG_RAX:
>> ffffffffffffffcf
>> [ 0.004000] RAX: 0000000000000001 RBX: ffff96dd9d0c0608 RCX: 0000000000000000
>> [ 0.004000] RDX: ffff96dd9d0c0400 RSI: 0000000000000286 RDI: 0000000000000286
>> [ 0.004000] RBP: ffffffffaa603e18 R08: 0000000000000001 R09: 000000000000003f
>> [ 0.004000] R10: 0000000000000286 R11: 0000000000000007 R12: ffff96dd9d0c0514
>> [ 0.004000] R13: ffff96dd9d0c04e0 R14: ffffffffaa61bd40 R15: ffff96dd9d0c0400
>> [ 0.004000] _raw_spin_unlock_irqrestore+0x1a/0x20
>> [ 0.004000] __setup_irq+0x5d9/0x780
>> [ 0.004000] setup_irq+0x5c/0x90
>> [ 0.004000] hpet_time_init+0x32/0x40
>> [ 0.004000] x86_late_time_init+0x10/0x20
>> [ 0.004000] start_kernel+0x45d/0x580
>> [ 0.004000] x86_64_start_kernel+0x30f/0x320
>> [ 0.004000] secondary_startup_64+0xa5/0xa5
>> [ 0.004000] Code: f0 f0 4c 0f b1 7b f8 4c 89 e3 4d 85 e4 75 cf 48
>> c7 c1 88 5b 01 00 65 48 03 0c 25 d8 a1 00 00 ff 14 25 10 08 62 aa f6
>> c4 02 75 4d <48> 83 39 00 74 3e 31 db 48 87 19 48 85 db 74 34 0f 1f 00
>> 48 8d
>> [ 0.004000] RIP: irq_work_tick+0x9d/0x110 RSP: ffff96dd9e403e58
>> [ 0.004000] ---[ end trace 8c7851007fbc6b6f ]---
>> [ 0.004000] Kernel panic - not syncing: Fatal exception in interrupt
>> [ 0.004000] ---[ end Kernel panic - not syncing: Fatal exception in interrupt
>>
>> Is this saying someting to you?
>
> It doesn't ring a bell, but I can repro it (with different offsets):
>
> [ 0.004000] general protection fault: 0000 [#1] SMP
> [ 0.004000] Modules linked in:
> [ 0.004000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.14.0+ #10
> [ 0.004000] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
> [ 0.004000] task: ffffffffbb410480 task.stack: ffffffffbb400000
> [ 0.004000] RIP: 0010:irq_work_tick+0xaf/0x120
> [ 0.004000] RSP: 0000:ffff96fb5fc03e58 EFLAGS: 00010046
> [ 0.004000] RAX: 0000000000000082 RBX: ffff96fb5fc11d80 RCX: 47e992c1bc778d00
> [ 0.004000] RDX: 000000000001ba00 RSI: fffffffffffffed4 RDI: ffff96fb5fc1ba38
> [ 0.004000] RBP: ffff96fb5fc03e78 R08: 0000000000000000 R09: 0000000000000018
> [ 0.004000] R10: 0000000000000000 R11: 0000000000000018 R12: ffffffffbb41bd40
> [ 0.004000] R13: 0000000000000000 R14: ffffffffbb410480 R15: 0000000000000000
> [ 0.004000] FS: 0000000000000000(0000) GS:ffff96fb5fc00000(0000) knlGS:0000000000000000
> [ 0.004000] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 0.004000] CR2: 00000000ffffffff CR3: 000000001ec09000 CR4: 00000000000006b0
> [ 0.004000] Call Trace:
> [ 0.004000] <IRQ>
> [ 0.004000] update_process_times+0x6e/0xa0
> [ 0.004000] tick_periodic+0x78/0x90
> [ 0.004000] tick_handle_periodic+0x26/0x80
> [ 0.004000] timer_interrupt+0x13/0x20
> [ 0.004000] __handle_irq_event_percpu+0x106/0x230
> [ 0.004000] handle_irq_event+0x5a/0xc0
> [ 0.004000] handle_level_irq+0x11a/0x190
> [ 0.004000] handle_irq+0x1f/0x30
> [ 0.004000] do_IRQ+0x4b/0xd0
> [ 0.004000] common_interrupt+0x93/0x93
> [ 0.004000] </IRQ>
> [ 0.004000] RIP: 0010:native_restore_fl+0xd/0x20
> [ 0.004000] RSP: 0000:ffffffffbb403e08 EFLAGS: 00000282 ORIG_RAX: ffffffffffffffcf
> [ 0.004000] RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000000
> [ 0.004000] RDX: ffff96fb5f01b800 RSI: 0000000000000282 RDI: 0000000000000282
> [ 0.004000] RBP: ffffffffbb403e10 R08: 0000000000000001 R09: 000000000000003f
> [ 0.004000] R10: ffff96fb5f01b800 R11: 0000000000000007 R12: ffff96fb5f01ba08
> [ 0.004000] R13: ffff96fb5f01b8e0 R14: ffffffffbb41bd40 R15: ffff96fb5f01b800
> [ 0.004000] _raw_spin_unlock_irqrestore+0x1a/0x20
> [ 0.004000] __setup_irq+0x610/0x7b0
> [ 0.004000] setup_irq+0x5b/0x90
> [ 0.004000] hpet_time_init+0x32/0x40
> [ 0.004000] x86_late_time_init+0x10/0x20
> [ 0.004000] start_kernel+0x460/0x580
> [ 0.004000] x86_64_start_kernel+0x30f/0x320
> [ 0.004000] secondary_startup_64+0xa5/0xa5
> [ 0.004000] Code: 4c 0f b1 7b f8 4c 89 e3 4d 85 e4 75 cf 48 c7 c1 88 5b 01 00 65 48 03 0c 25 d8 a1 00 00 ff 14 25 10 08 42 bb a9 00 02 00 00 75 4b <48> 83 39 00 74 3c 31 db 48 87 19 48 85 db 74 32 90 48 8d 7b f8
> [ 0.004000] RIP: irq_work_tick+0xaf/0x120 RSP: ffff96fb5fc03e58
> [ 0.004000] ---[ end trace 08945838e05bf5b2 ]---
> [ 0.004000] Kernel panic - not syncing: Fatal exception in interrupt
> [ 0.004000] ---[ end Kernel panic - not syncing: Fatal exception in interrupt
>
> The exception occurs at 'irq_work_tick+0xaf'
>
> objdump -d -S --start-address=0x$(grep irq_work_tick System.map | sed -e
> "s/ \+.*//") vmlinux | less
>
> ...
> void irq_work_tick(void)
> {
> ffffffff81193200: 55 push %rbp
> ...
> static inline struct llist_node *llist_del_all(struct llist_head *head)
> {
> return xchg(&head->first, NULL);
> ffffffff8119324a: 48 87 19 xchg %rbx,(%rcx)
> while (llnode != NULL) {
> ffffffff8119324d: 48 85 db test %rbx,%rbx
> ffffffff81193250: 74 3f je ffffffff81193291 <irq_work_tick+0x91>
> ffffffff81193252: 0f 1f 40 00 nopl 0x0(%rax)
> ffffffff81193256: 66 2e 0f 1f 84 00 00 nopw %cs:0x0(%rax,%rax,1)
> ffffffff8119325d: 00 00 00
> work = llist_entry(llnode, struct irq_work, llnode);
> ffffffff81193260: 48 8d 7b f8 lea -0x8(%rbx),%rdi
> flags = work->flags & ~IRQ_WORK_PENDING;
> ffffffff81193264: 4c 8b 7b f8 mov -0x8(%rbx),%r15
> return node->next;
> ffffffff81193268: 4c 8b 23 mov (%rbx),%r12
> ffffffff8119326b: 4d 89 fe mov %r15,%r14
> ffffffff8119326e: 49 83 e6 fe and $0xfffffffffffffffe,%r14
> xchg(&work->flags, flags);
> ffffffff81193272: 4c 89 f0 mov %r14,%rax
> ffffffff81193275: 48 87 43 f8 xchg %rax,-0x8(%rbx)
> work->func(work);
> ffffffff81193279: ff 53 08 callq *0x8(%rbx)
> (void)cmpxchg(&work->flags, flags, flags & ~IRQ_WORK_BUSY);
> ffffffff8119327c: 49 83 e7 fc and $0xfffffffffffffffc,%r15
> ffffffff81193280: 4c 89 f0 mov %r14,%rax
> ffffffff81193283: f0 4c 0f b1 7b f8 lock cmpxchg %r15,-0x8(%rbx)
> ffffffff81193289: 4c 89 e3 mov %r12,%rbx
> while (llnode != NULL) {
> ffffffff8119328c: 4d 85 e4 test %r12,%r12
> ffffffff8119328f: 75 cf jne ffffffff81193260 <irq_work_tick+0x60>
> irq_work_run_list(raised);
> irq_work_run_list(this_cpu_ptr(&lazy_list));
> ffffffff81193291: 48 c7 c1 88 5b 01 00 mov $0x15b88,%rcx
> ffffffff81193298: 65 48 03 0c 25 d8 a1 add %gs:0xa1d8,%rcx
> ffffffff8119329f: 00 00
> ffffffff811932a1: ff 14 25 10 08 c2 81 callq *0xffffffff81c20810
> BUG_ON(!irqs_disabled());
> ffffffff811932a8: a9 00 02 00 00 test $0x200,%eax
> ffffffff811932ad: 75 4b jne ffffffff811932fa <irq_work_tick+0xfa>
> return ACCESS_ONCE(head->first) == NULL;
> ffffffff811932af: 48 83 39 00 cmpq $0x0,(%rcx)
>
>
> I'm no x86/assembly expert, anyway my interpretation is:
>
> 'irq_work_tick+0xaf' corresponds to address 0xffffffff811932af, so the
> exception occurs during the execution of 'cmpq $0x0,(%rcx)' or
> 'ACCESS_ONCE(head->first) == NULL'. The instruction checks if the
> memory location in RCX contains 0. Supposedly the register should
> contain the address of 'head->first', however the value is
> 0x47e992c1bc778d00, which doesn't look like a valid address.
>
> RCX is set shortly before:
>
> ffffffff81193291: 48 c7 c1 88 5b 01 00 mov $0x15b88,%rcx
> ffffffff81193298: 65 48 03 0c 25 d8 a1 add %gs:0xa1d8,%rcx
>
> As stated in https://www.kernel.org/doc/Documentation/this_cpu_ops.txt
> the GS segment register is used by the kernel for per-cpu
> variables. And effectively System.map tells us the offset 0x15b88
> corresponds to the per-cpu variable lazy_list:
>
> 0000000000015b88 d lazy_list
>
> The register dump shows GS with a value of 0xffff96fb5fc00000, which
> looks reasonable.
>
> But wait, right after setting RCX we do this:
>
> ffffffff811932a1: ff 14 25 10 08 c2 81 callq *0xffffffff81c20810
> BUG_ON(!irqs_disabled());
>
> According to my (limited) understanding of x86 calling conventions RCX
> is a caller-saved register, thus the caller should save it on the
> stack to preserve its value across a function call.
> (https://en.wikipedia.org/wiki/X86_calling_conventions)
>
> The call to '*0xffffffff81c20810' appears to be related with
> KVM/paravirtualization:
>
> grep ffffffff81c20810 System.map
> ffffffff81c20810 D pv_irq_ops
>
> Since there is no offset it must be calling the first function in the
> structure, which is 'safe_fl' and points to 'native_save_fl'
> (https://elixir.bootlin.com/linux/v4.14.36/source/arch/x86/kernel/paravirt.c#L316)
>
> objdump -d -S --start-address=0x$(grep native_save_fl System.map | sed
> -e "s/ \+.*//") vmlinux | less
>
> static inline unsigned long native_save_fl(void)
> {
> ffffffff81060240: 55 push %rbp
> ffffffff81060241: 48 89 e5 mov %rsp,%rbp
> ffffffff81060244: 48 83 ec 10 sub $0x10,%rsp
> ffffffff81060248: 65 48 8b 04 25 28 00 mov %gs:0x28,%rax
> ffffffff8106024f: 00 00
> ffffffff81060251: 48 89 45 f8 mov %rax,-0x8(%rbp)
> /*
> * "=rm" is safe here, because "pop" adjusts the stack before
> * it evaluates its effective address -- this is part of the
> * documented behavior of the "pop" instruction.
> */
> asm volatile("# __raw_save_flags\n\t"
> ffffffff81060255: 9c pushfq
> ffffffff81060256: 8f 45 f0 popq -0x10(%rbp)
> "pushf ; pop %0"
> : "=rm" (flags)
> : /* no input */
> : "memory");
>
> return flags;
> ffffffff81060259: 48 8b 45 f0 mov -0x10(%rbp),%rax
> ffffffff8106025d: 65 48 8b 0c 25 28 00 mov %gs:0x28,%rcx
> ffffffff81060264: 00 00
> ffffffff81060266: 48 3b 4d f8 cmp -0x8(%rbp),%rcx
> ffffffff8106026a: 75 06 jne ffffffff81060272 <native_save_fl+0x32>
> ffffffff8106026c: 48 83 c4 10 add $0x10,%rsp
> ffffffff81060270: 5d pop %rbp
> ffffffff81060271: c3 retq
> ffffffff81060272: e8 09 ec 01 00 callq ffffffff8107ee80 <__stack_chk_fail>
> ffffffff81060277: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> ffffffff8106027e: 00 0
>
> At 0xffffffff8106025d this clobbers RCX! I don't know why clang
> doesn't save the value on the stack before calling native_save_fl(),
> but that seems to be the problem.
>
> Again, I'm not an expert in this area and ventured into territory
> unknown to me, so please excuse if I got something totally wrong ...
>
Hi Matthias,
again you helped me a lot.
I recalled that I had success in the early days of llvmlinux with...
CONFIG_PARAVIRT=n
...as suggested by Steve Rostedt.
This needed some more hackish kconfig kungfu these days - patches attached.
In the end I could boot into a 4.14.37-1-iniza-llvmlinux kernel
(kernel-config and dmesg-log attached).
Concerning your analysis...
Is this a clang-bug?
Or can this be solved in the linux-kernel code?
What are your suggestion?
And can you help on getting this fixed with the help of your Chromium
team and/or bring it to the right persons?
IMHO an analysis like you did might help, too.
I am not an expert in this area to say it is good/correct but I could
follow and read it like a thriller.
Where is Lieutenant Columbo when you need him :-)?
Moreover, a test-case can be helpful, e.g. "Is clang clobbering RCX?"
when it's a clang-bug.
I tried to find some suitable test-case myself.
The clang-source has a test-dir, but this is new to me.
I tried to run a single test like test/Sema/asm.c in [1] with
llvm-tools Debian-package which ships "lit.py" script.
Not sure if this is a suitable for x86-64 (amd64) as the first line says...
// RUN: %clang_cc1 %s -Wno-private-extern -triple i386-pc-linux-gnu
-verify -fsyntax-only
root# cd /path/to/clang-source
root# python /usr/lib/llvm-7/build/utils/lit/lit.py -sv test/Sema
lit.py: /usr/lib/llvm-7/build/utils/lit/lit/TestingConfig.py:101:
fatal: unable to parse config file
'/path/to/clang-source/test/lit.cfg.py', traceback: Traceback (most
recent call last):
File "/usr/lib/llvm-7/build/utils/lit/lit/TestingConfig.py", line
88, in load_from_path
exec(compile(data, path, 'exec'), cfg_globals, None)
File "/home/sdi/src/linux-kernel/clang/test/lit.cfg.py", line 25, in <module>
config.test_format = lit.formats.ShTest(not llvm_config.use_lit_shell)
AttributeError: 'NoneType' object has no attribute 'use_lit_shell'
Not sure if you have any experiences in this area, but I think asking
on llvm-dev is a good idea.
If we have a reproducible test-case we could ask on llvm-dev?
In the case of linux-kernel...
Ask Linux/x86 folks for a test-case to see if the compiler clobbers RCX?
Show your analysis to Linux/(x86)/KVM folks?
Some ideas.
Regards,
- Sedat -
[1] https://raw.githubusercontent.com/llvm-mirror/clang/master/test/Sema/asm.c
[2] https://llvm.org/docs/CommandGuide/lit.html
[3] https://llvm.org/docs/TestingGuide.html
[4] https://llvm.org/docs/TestSuiteMakefileGuide.html
> For the record: a Chrome OS v4.14.35 kernel ('based' on the
> sandbox/mka stack) built with clang still boots on an actual x86
> device.
>
I believe you, really.
Can you give me your kernel-config?
What is the setting of CONFIG_PARAVIRT?
Thank you.
Regards,
- Sedat -
On Sun, May 6, 2018 at 9:41 AM, Dmitry Vyukov <[email protected]> wrote:
> On Sun, May 6, 2018 at 8:35 AM, Sedat Dilek <[email protected]> wrote:
>> On Mon, Apr 23, 2018 at 7:42 PM, Matthias Kaehlcke <[email protected]> wrote:
>> [...]
>>>> [ ASM-GOTO ]
>>>>
>>>> Foremore, I have seen you have a "refs/sandbox/mka/llvm/v4.14" Git branch.
>>>> Linux v4.14 is also an LTS release.
>>>
>>> Yes, there is also a follow up post:
>>> https://lkml.org/lkml/2017/11/22/943
>>>
>>>> After Linux v4.9 "asm-goto" support was added which is GCC specific.
>>>
>>> asm-goto was optional until a few weeks ago, when x86 maintainers
>>> decided to make it mandatory, and thus break clang builds for x86:
>>>
>>> https://lkml.org/lkml/2018/4/2/486
>>>
>>>> Is there a workaround to compile Linux-kernel with any CLANG
>>>> version?
>>>
>>> You could revert the patch that makes asm-goto mandatory.
>>>
>>>> You happen to know the status in LLVM upstream?
>>>
>>> I know people are actively working on this, but don't know an ETA.
>>>
>>
>> [ CC Dimitry ]
>>
>> Sorry for the late response.
>> This weekend I had some time to play with clang-7 and Linux-4.17-rcN.
>>
>> I tried with Linux v4.17-rc3-261-gc1c07416cdd4 and reverted the
>> mandatory patch [1].
>>
>> commit e501ce957a786ecd076ea0cfb10b114e6e4d0f40
>> "x86: Force asm-goto"
>>
>> This needed a follow up [2]...
>>
>> commit d0266046ad54e0c964941364cd82a0d0478ce286
>> "x86: Remove FAST_FEATURE_TESTS"
>>
>> ...but breaks my build.
>>
>> That seems to be a known issue [3] and Dimitry bisected this to [4].
>> Is there a fix for this?
>>
>> I have attached my kernel-config and the broken build-log.
>>
>> Hope this helps.
>>
>> Regards,
>> - Sedat -
>>
>> [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e501ce957a786ecd076ea0cfb10b114e6e4d0f40
>> [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/arch/x86/include/asm/cpufeature.h?id=d0266046ad54e0c964941364cd82a0d0478ce286
>> [3] https://www.spinics.net/lists/linux-mm/msg147669.html
>> [4] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=8bf705d130396e69c04cd8e6e010244ad2ce71f4
>
>
> Hi Sedat,
>
> Sorry for the breakage.
>
> This was reported and discussed here:
> https://groups.google.com/forum/#!topic/kasan-dev/oMgCP37n1vw
> The plan is to make clang behave the same way as gcc wrt asm
> constraint checking. But I don't know what's the progress.
Hi Dimitry,
that's the thread I looked into and also looked into the involved
commits and appropriate files in the linux-source without knowing of
[1].
Just just for the records...
I used the kernel-config of Debian's
linux-image-4.17.0-rc3-amd64-unsigned (4.17~rc3-1~exp1) as a base with
this modification:
$ cd /path/to/linux-source
$ LINUX_CONFIG="./.config"
$ scripts/config --file $LINUX_CONFIG --disable X86_X32
...and...
# CONFIG_KASAN is not set
...if this matters.
Kind regards,
- Sedat -
[1] https://bugs.llvm.org/show_bug.cgi?id=33587
[2] https://packages.debian.org/experimental/linux-image-4.17.0-rc3-amd64-unsigned
[...]
> Moreover, a test-case can be helpful, e.g. "Is clang clobbering RCX?"
> when it's a clang-bug.
>
> I tried to find some suitable test-case myself.
>
> The clang-source has a test-dir, but this is new to me.
>
> I tried to run a single test like test/Sema/asm.c in [1] with
> llvm-tools Debian-package which ships "lit.py" script.
>
[...]
> [1] https://raw.githubusercontent.com/llvm-mirror/clang/master/test/Sema/asm.c
> [2] https://llvm.org/docs/CommandGuide/lit.html
> [3] https://llvm.org/docs/TestingGuide.html
> [4] https://llvm.org/docs/TestSuiteMakefileGuide.html
In [5] are some hints on running a single test...
- Sedat -
[5] http://clang.llvm.org/hacking.html
On Sun, May 06, 2018 at 09:42:09AM +0200, Sedat Dilek wrote:
> On Wed, Apr 25, 2018 at 1:06 AM, Matthias Kaehlcke <[email protected]> wrote:
> > On Tue, Apr 24, 2018 at 01:54:29PM +0200, Sedat Dilek wrote:
> >> Hi Matthias,
> >>
> >> a big thank you for giving all the informations!
> >>
> >> I used your mka/llvm/v4.14_ext Git tree...
> >>
> >> https://chromium.googlesource.com/chromiumos/third_party/kernel/+log/sandbox/mka/llvm/v4.14_ext
> >>
> >> ...and was able to compile with clang-6.0 from Debian/buster64 with...
> >>
> >> ...reverting the clang-3/clang-4 patches...
> >>
> >> user$ for p in 69e44656ae43 222b88977a00 cdfcf1e45537 26f14c9225a6
> >> 0385a18e9995 68dab143c9b4 ; do echo [ $p ] ; LC_ALL=C git revert
> >> --no-edit $p ; done
> >>
> >> 0001-Revert-CLANG3-core-clang-work-around-x86-regparm-int.patch
> >> 0002-Revert-CLANG4-futex-don-t-optimize-futex_detect_cmpx.patch
> >> 0003-Revert-CLANG4-Disable-lkdtm-when-ftrace-is-enabled.patch
> >> 0004-Revert-CLANG4-arm64-prefetch-Use-__builtin_arm_prefe.patch
> >> 0005-Revert-CLANG4-kbuild-Add-meabi-gnu-to-the-clang-para.patch
> >> 0006-Revert-CLANG4-crypto-arm64-aes-ce-Explicitly-pass-th.patch
> >>
> >> ...and needed two additional patches from upstream:
> >>
> >> 0007-kbuild-clang-remove-crufty-HOSTCFLAGS.patch <--- Label with
> >> BACKPORT (XXX: Backported; Required when using HOSTCC in make-line,
> >> see below)
> >> 0008-x86-xen-remove-the-use-of-VLAIS.patch <--- Label with UPSTREAM
> >> (XXX: cherry-picked)
> >>
> >> These 3 patches in your Git branch are in Linux v4.14.36...
> >>
> >> 9af5ddf981ed BACKPORT: kbuild: disable clang's default use of
> >> -fmerge-all-constants
> >> f24088a3842c BACKPORT: kbuild: Set KBUILD_CFLAGS before incl. arch Makefile
> >> d4dfe384346d UPSTREAM: kbuild: fix linker feature test macros when
> >> cross compiling with Clang
> >>
> >> ...and can be dropped.
> >> I do not know your work-flow/policy: Maybe you want to keep your
> >> patch-stack against vanilla Linux v4.14 - without stables from
> >> linux-stable-4.14.y.
> >
> > Yes, these patch stacks are based on vanilla Linux v4.14, one reason
> > is that it isn't a moving target. Also I think it makes it easier for
> > folks not merging LTS (though they probably should!) to locate all
> > patches, and it's fairly trivial to skip the (currently) few patches
> > not needed with the -stable tree.
> >
> >> For easy switching "mycompiler" I use a wrapper-script:
> >>
> >> root# cat /usr/bin/mycompiler
> >> #!/bin/bash
> >>
> >> clang-6.0 "$@"
> >> - EOF -
> >>
> >> user$ cp -v /boot/config-4.14.35-1-iniza-amd64 .config
> >> user$ MAKE="make V=1" ; COMPILER="mycompiler" ;
> >> MAKE_OPTS="CC=$COMPILER HOSTCC=$COMPILER"
> >> user$ yes "" | $MAKE $MAKE_OPTS oldconfig && $MAKE $MAKE_OPTS
> >> silentoldconfig < /dev/null
> >>
> >> The diffconfig to my current kernel-config-4.14 looks like this...
> >>
> >> user$ ./scripts/diffconfig /boot/config-4.14.35-1-iniza-amd64 .config
> >> -ARCH_HAS_REFCOUNT y
> >> -BPF_JIT_ALWAYS_ON n
> >> -EXOFS_DEBUG n
> >> -EXOFS_FS m
> >> -GENERIC_CPU_VULNERABILITIES y
> >> -KASAN n
> >> -ORE m
> >> -PAGE_TABLE_ISOLATION y
> >> -RETPOLINE y
> >> -UNWINDER_FRAME_POINTER n
> >> -UNWINDER_GUESS n
> >> -UNWINDER_ORC y
> >> +FRAME_POINTER y
> >> +FRAME_POINTER_UNWINDER y
> >> +GUESS_UNWINDER n
> >> +HAVE_ARCH_KMEMCHECK y
> >> +HAVE_RELIABLE_STACKTRACE y
> >> +ORC_UNWINDER n
> >>
> >> Unfortunately, I cannot boot into the generated kernel on bare metal.
> >>
> >> Checking with QEMU (version: 2.12~rc3) and catching earlyprintk, I see this...
> >>
> >> user$ echo $KPATH
> >> $HOME/src/linux-kernel/important-files
> >>
> >> user$ ls -al
> >> insgesamt 344916
> >> drwxr-xr-x 2 sdi sdi 4096 Apr 24 13:15 .
> >> drwxr-xr-x 20 sdi sdi 4096 Apr 24 13:13 ..
> >> -rw-r--r-- 1 sdi sdi 4528416 Apr 24 12:42 bzImage
> >> lrwxrwxrwx 1 sdi sdi 35 Apr 24 13:15 initrd.img ->
> >> initrd.img-4.14.0-1-iniza-llvmlinux
> >> -rw-r--r-- 1 sdi sdi 25572955 Apr 24 13:08
> >> initrd.img-4.14.0-1-iniza-llvmlinux
> >> -rw-r--r-- 1 sdi sdi 2887195 Apr 24 12:42 System.map
> >> -rwxr-xr-x 1 sdi sdi 326116744 Apr 24 12:42 vmlinux
> >>
> >> user$ sudo qemu-system-x86_64 -enable-kvm -M pc -kernel $KPATH/bzImage
> >> -initrd $KPATH/initrd.img -m 512 -net none -serial stdio -append
> >> "root=/dev/ram0 console=ttyS0 hung_task_panic=1
> >> earlyprintk=ttyS0,115200"
> >>
> >> Probing EDD (edd=off to disable)... ok
> >> [ 0.000000] Linux version 4.14.0-1-iniza-llvmlinux
> >> ([email protected]@iniza) (clang version 6.0.0-1
> >> (tags/RELEASE_600/final)) #1 SMP Tue Apr 24 12:42:21 CEST 2018
> >> [ 0.000000] Command line: root=/dev/ram0 console=ttyS0
> >> hung_task_panic=1 earlyprintk=ttyS0,115200
> >> [ 0.000000] x86/fpu: x87 FPU will use FXSAVE
> >> [ 0.000000] e820: BIOS-provided physical RAM map:
> >> [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
> >> [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
> >> [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
> >> [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000001ffdffff] usable
> >> [ 0.000000] BIOS-e820: [mem 0x000000001ffe0000-0x000000001fffffff] reserved
> >> [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
> >> [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
> >> [ 0.000000] bootconsole [earlyser0] enabled
> >> [ 0.000000] NX (Execute Disable) protection: active
> >> [ 0.000000] random: fast init done
> >> [ 0.000000] SMBIOS 2.8 present.
> >> [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
> >> 1.11.1-1 04/01/2014
> >> [ 0.000000] Hypervisor detected: KVM
> >> [ 0.000000] tsc: Fast TSC calibration using PIT
> >> [ 0.000000] e820: last_pfn = 0x1ffe0 max_arch_pfn = 0x400000000
> >> [ 0.000000] x86/PAT: PAT not supported by CPU.
> >> [ 0.000000] x86/PAT: Configuration [0-7]: WB WT UC- UC WB WT UC- UC
> >> Memory KASLR using RDTSC...
> >> [ 0.000000] found SMP MP-table at [mem 0x000f5d60-0x000f5d6f]
> >> mapped at [ffffffffff000d60]
> >> [ 0.000000] RAMDISK: [mem 0x1e77c000-0x1ffdffff]
> >> [ 0.000000] ACPI: Early table checksum verification disabled
> >> [ 0.000000] ACPI: RSDP 0x00000000000F5B90 000014 (v00 BOCHS )
> >> [ 0.000000] ACPI: RSDT 0x000000001FFE157C 000030 (v01 BOCHS
> >> BXPCRSDT 00000001 BXPC 00000001)
> >> [ 0.000000] ACPI: FACP 0x000000001FFE1458 000074 (v01 BOCHS
> >> BXPCFACP 00000001 BXPC 00000001)
> >> [ 0.000000] ACPI: DSDT 0x000000001FFE0040 001418 (v01 BOCHS
> >> BXPCDSDT 00000001 BXPC 00000001)
> >> [ 0.000000] ACPI: FACS 0x000000001FFE0000 000040
> >> [ 0.000000] ACPI: APIC 0x000000001FFE14CC 000078 (v01 BOCHS
> >> BXPCAPIC 00000001 BXPC 00000001)
> >> [ 0.000000] ACPI: HPET 0x000000001FFE1544 000038 (v01 BOCHS
> >> BXPCHPET 00000001 BXPC 00000001)
> >> [ 0.000000] No NUMA configuration found
> >> [ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000001ffdffff]
> >> [ 0.000000] NODE_DATA(0) allocated [mem 0x1e777000-0x1e77bfff]
> >> [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
> >> [ 0.000000] kvm-clock: cpu 0, msr 0:1e76f001, primary cpu clock
> >> [ 0.000000] kvm-clock: using sched offset of 528742140 cycles
> >> [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff
> >> max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
> >> [ 0.000000] Zone ranges:
> >> [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
> >> [ 0.000000] DMA32 [mem 0x0000000001000000-0x000000001ffdffff]
> >> [ 0.000000] Normal empty
> >> [ 0.000000] Device empty
> >> [ 0.000000] Movable zone start for each node
> >> [ 0.000000] Early memory node ranges
> >> [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff]
> >> [ 0.000000] node 0: [mem 0x0000000000100000-0x000000001ffdffff]
> >> [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x000000001ffdffff]
> >> [ 0.000000] ACPI: PM-Timer IO Port: 0x608
> >> [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
> >> [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
> >> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
> >> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
> >> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
> >> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
> >> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
> >> [ 0.000000] Using ACPI (MADT) for SMP configuration information
> >> [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000
> >> [ 0.000000] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
> >> [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]
> >> [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
> >> [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff]
> >> [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff]
> >> [ 0.000000] e820: [mem 0x20000000-0xfeffbfff] available for PCI devices
> >> [ 0.000000] Booting paravirtualized kernel on KVM
> >> [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff
> >> max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
> >> [ 0.000000] setup_percpu: NR_CPUS:512 nr_cpumask_bits:512
> >> nr_cpu_ids:1 nr_node_ids:1
> >> [ 0.000000] percpu: Embedded 37 pages/cpu @ffff96dd9e400000 s114392
> >> r8192 d28968 u2097152
> >> [ 0.000000] KVM setup async PF for cpu 0
> >> [ 0.000000] kvm-stealtime: cpu 0, msr 1e40d900
> >> [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 128872
> >> [ 0.000000] Policy zone: DMA32
> >> [ 0.000000] Kernel command line: root=/dev/ram0 console=ttyS0
> >> hung_task_panic=1 earlyprintk=ttyS0,115200
> >> [ 0.000000] PID hash table entries: 2048 (order: 2, 16384 bytes)
> >> [ 0.000000] Memory: 474480K/523768K available (7639K kernel code,
> >> 1005K rwdata, 2936K rodata, 1636K init, 688K bss, 49288K reserved, 0K
> >> cma-reserved)
> >> [ 0.000000] ftrace: allocating 27586 entries in 108 pages
> >> [ 0.004000] Hierarchical RCU implementation.
> >> [ 0.004000] RCU restricting CPUs from NR_CPUS=512 to nr_cpu_ids=1.
> >> [ 0.004000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
> >> [ 0.004000] NR_IRQS: 33024, nr_irqs: 256, preallocated irqs: 16
> >> [ 0.004000] Console: colour VGA+ 80x25
> >> [ 0.004000] console [ttyS0] enabled
> >> [ 0.004000] console [ttyS0] enabled
> >> [ 0.004000] bootconsole [earlyser0] disabled
> >> [ 0.004000] bootconsole [earlyser0] disabled
> >> [ 0.004000] clocksource: hpet: mask: 0xffffffff max_cycles:
> >> 0xffffffff, max_idle_ns: 19112604467 ns
> >> [ 0.004000] general protection fault: 0000 [#1] SMP
> >> [ 0.004000] Modules linked in:
> >> [ 0.004000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted
> >> 4.14.0-1-iniza-llvmlinux #1
> >> [ 0.004000] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
> >> BIOS 1.11.1-1 04/01/2014
> >> [ 0.004000] task: ffffffffaa610480 task.stack: ffffffffaa600000
> >> [ 0.004000] RIP: 0010:irq_work_tick+0x9d/0x110
> >> [ 0.004000] RSP: 0000:ffff96dd9e403e58 EFLAGS: 00010046
> >> [ 0.004000] RAX: 0000000000000082 RBX: ffff96dd9e411d80 RCX: adecc9cc04e2ca00
> >> [ 0.004000] RDX: 000000000001ba00 RSI: fffffffffffffed4 RDI: ffff96dd9e41ba38
> >> [ 0.004000] RBP: ffff96dd9e403e78 R08: 0000000000000000 R09: 0000000000000018
> >> [ 0.004000] R10: 0000000000000000 R11: 0000000000000018 R12: ffffffffaa61bd40
> >> [ 0.004000] R13: 0000000000000000 R14: ffffffffaa610480 R15: 0000000000000000
> >> [ 0.004000] FS: 0000000000000000(0000) GS:ffff96dd9e400000(0000)
> >> knlGS:0000000000000000
> >> [ 0.004000] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> >> [ 0.004000] CR2: 00000000ffffffff CR3: 0000000008a09000 CR4: 00000000000006b0
> >> [ 0.004000] Call Trace:
> >> [ 0.004000] <IRQ>
> >> [ 0.004000] update_process_times+0x6e/0xa0
> >> [ 0.004000] tick_periodic+0x78/0x90
> >> [ 0.004000] tick_handle_periodic+0x26/0x80
> >> [ 0.004000] timer_interrupt+0x13/0x20
> >> [ 0.004000] __handle_irq_event_percpu+0x106/0x230
> >> [ 0.004000] handle_irq_event+0x5a/0xc0
> >> [ 0.004000] handle_level_irq+0x11a/0x190
> >> [ 0.004000] handle_irq+0x1f/0x30
> >> [ 0.004000] do_IRQ+0x4b/0xd0
> >> [ 0.004000] common_interrupt+0x93/0x93
> >> [ 0.004000] </IRQ>
> >> [ 0.004000] RIP: 0010:native_restore_fl+0x12/0x20
> >> [ 0.004000] RSP: 0000:ffffffffaa603e10 EFLAGS: 00000286 ORIG_RAX:
> >> ffffffffffffffcf
> >> [ 0.004000] RAX: 0000000000000001 RBX: ffff96dd9d0c0608 RCX: 0000000000000000
> >> [ 0.004000] RDX: ffff96dd9d0c0400 RSI: 0000000000000286 RDI: 0000000000000286
> >> [ 0.004000] RBP: ffffffffaa603e18 R08: 0000000000000001 R09: 000000000000003f
> >> [ 0.004000] R10: 0000000000000286 R11: 0000000000000007 R12: ffff96dd9d0c0514
> >> [ 0.004000] R13: ffff96dd9d0c04e0 R14: ffffffffaa61bd40 R15: ffff96dd9d0c0400
> >> [ 0.004000] _raw_spin_unlock_irqrestore+0x1a/0x20
> >> [ 0.004000] __setup_irq+0x5d9/0x780
> >> [ 0.004000] setup_irq+0x5c/0x90
> >> [ 0.004000] hpet_time_init+0x32/0x40
> >> [ 0.004000] x86_late_time_init+0x10/0x20
> >> [ 0.004000] start_kernel+0x45d/0x580
> >> [ 0.004000] x86_64_start_kernel+0x30f/0x320
> >> [ 0.004000] secondary_startup_64+0xa5/0xa5
> >> [ 0.004000] Code: f0 f0 4c 0f b1 7b f8 4c 89 e3 4d 85 e4 75 cf 48
> >> c7 c1 88 5b 01 00 65 48 03 0c 25 d8 a1 00 00 ff 14 25 10 08 62 aa f6
> >> c4 02 75 4d <48> 83 39 00 74 3e 31 db 48 87 19 48 85 db 74 34 0f 1f 00
> >> 48 8d
> >> [ 0.004000] RIP: irq_work_tick+0x9d/0x110 RSP: ffff96dd9e403e58
> >> [ 0.004000] ---[ end trace 8c7851007fbc6b6f ]---
> >> [ 0.004000] Kernel panic - not syncing: Fatal exception in interrupt
> >> [ 0.004000] ---[ end Kernel panic - not syncing: Fatal exception in interrupt
> >>
> >> Is this saying someting to you?
> >
> > It doesn't ring a bell, but I can repro it (with different offsets):
> >
> > [ 0.004000] general protection fault: 0000 [#1] SMP
> > [ 0.004000] Modules linked in:
> > [ 0.004000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.14.0+ #10
> > [ 0.004000] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
> > [ 0.004000] task: ffffffffbb410480 task.stack: ffffffffbb400000
> > [ 0.004000] RIP: 0010:irq_work_tick+0xaf/0x120
> > [ 0.004000] RSP: 0000:ffff96fb5fc03e58 EFLAGS: 00010046
> > [ 0.004000] RAX: 0000000000000082 RBX: ffff96fb5fc11d80 RCX: 47e992c1bc778d00
> > [ 0.004000] RDX: 000000000001ba00 RSI: fffffffffffffed4 RDI: ffff96fb5fc1ba38
> > [ 0.004000] RBP: ffff96fb5fc03e78 R08: 0000000000000000 R09: 0000000000000018
> > [ 0.004000] R10: 0000000000000000 R11: 0000000000000018 R12: ffffffffbb41bd40
> > [ 0.004000] R13: 0000000000000000 R14: ffffffffbb410480 R15: 0000000000000000
> > [ 0.004000] FS: 0000000000000000(0000) GS:ffff96fb5fc00000(0000) knlGS:0000000000000000
> > [ 0.004000] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [ 0.004000] CR2: 00000000ffffffff CR3: 000000001ec09000 CR4: 00000000000006b0
> > [ 0.004000] Call Trace:
> > [ 0.004000] <IRQ>
> > [ 0.004000] update_process_times+0x6e/0xa0
> > [ 0.004000] tick_periodic+0x78/0x90
> > [ 0.004000] tick_handle_periodic+0x26/0x80
> > [ 0.004000] timer_interrupt+0x13/0x20
> > [ 0.004000] __handle_irq_event_percpu+0x106/0x230
> > [ 0.004000] handle_irq_event+0x5a/0xc0
> > [ 0.004000] handle_level_irq+0x11a/0x190
> > [ 0.004000] handle_irq+0x1f/0x30
> > [ 0.004000] do_IRQ+0x4b/0xd0
> > [ 0.004000] common_interrupt+0x93/0x93
> > [ 0.004000] </IRQ>
> > [ 0.004000] RIP: 0010:native_restore_fl+0xd/0x20
> > [ 0.004000] RSP: 0000:ffffffffbb403e08 EFLAGS: 00000282 ORIG_RAX: ffffffffffffffcf
> > [ 0.004000] RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000000
> > [ 0.004000] RDX: ffff96fb5f01b800 RSI: 0000000000000282 RDI: 0000000000000282
> > [ 0.004000] RBP: ffffffffbb403e10 R08: 0000000000000001 R09: 000000000000003f
> > [ 0.004000] R10: ffff96fb5f01b800 R11: 0000000000000007 R12: ffff96fb5f01ba08
> > [ 0.004000] R13: ffff96fb5f01b8e0 R14: ffffffffbb41bd40 R15: ffff96fb5f01b800
> > [ 0.004000] _raw_spin_unlock_irqrestore+0x1a/0x20
> > [ 0.004000] __setup_irq+0x610/0x7b0
> > [ 0.004000] setup_irq+0x5b/0x90
> > [ 0.004000] hpet_time_init+0x32/0x40
> > [ 0.004000] x86_late_time_init+0x10/0x20
> > [ 0.004000] start_kernel+0x460/0x580
> > [ 0.004000] x86_64_start_kernel+0x30f/0x320
> > [ 0.004000] secondary_startup_64+0xa5/0xa5
> > [ 0.004000] Code: 4c 0f b1 7b f8 4c 89 e3 4d 85 e4 75 cf 48 c7 c1 88 5b 01 00 65 48 03 0c 25 d8 a1 00 00 ff 14 25 10 08 42 bb a9 00 02 00 00 75 4b <48> 83 39 00 74 3c 31 db 48 87 19 48 85 db 74 32 90 48 8d 7b f8
> > [ 0.004000] RIP: irq_work_tick+0xaf/0x120 RSP: ffff96fb5fc03e58
> > [ 0.004000] ---[ end trace 08945838e05bf5b2 ]---
> > [ 0.004000] Kernel panic - not syncing: Fatal exception in interrupt
> > [ 0.004000] ---[ end Kernel panic - not syncing: Fatal exception in interrupt
> >
> > The exception occurs at 'irq_work_tick+0xaf'
> >
> > objdump -d -S --start-address=0x$(grep irq_work_tick System.map | sed -e
> > "s/ \+.*//") vmlinux | less
> >
> > ...
> > void irq_work_tick(void)
> > {
> > ffffffff81193200: 55 push %rbp
> > ...
> > static inline struct llist_node *llist_del_all(struct llist_head *head)
> > {
> > return xchg(&head->first, NULL);
> > ffffffff8119324a: 48 87 19 xchg %rbx,(%rcx)
> > while (llnode != NULL) {
> > ffffffff8119324d: 48 85 db test %rbx,%rbx
> > ffffffff81193250: 74 3f je ffffffff81193291 <irq_work_tick+0x91>
> > ffffffff81193252: 0f 1f 40 00 nopl 0x0(%rax)
> > ffffffff81193256: 66 2e 0f 1f 84 00 00 nopw %cs:0x0(%rax,%rax,1)
> > ffffffff8119325d: 00 00 00
> > work = llist_entry(llnode, struct irq_work, llnode);
> > ffffffff81193260: 48 8d 7b f8 lea -0x8(%rbx),%rdi
> > flags = work->flags & ~IRQ_WORK_PENDING;
> > ffffffff81193264: 4c 8b 7b f8 mov -0x8(%rbx),%r15
> > return node->next;
> > ffffffff81193268: 4c 8b 23 mov (%rbx),%r12
> > ffffffff8119326b: 4d 89 fe mov %r15,%r14
> > ffffffff8119326e: 49 83 e6 fe and $0xfffffffffffffffe,%r14
> > xchg(&work->flags, flags);
> > ffffffff81193272: 4c 89 f0 mov %r14,%rax
> > ffffffff81193275: 48 87 43 f8 xchg %rax,-0x8(%rbx)
> > work->func(work);
> > ffffffff81193279: ff 53 08 callq *0x8(%rbx)
> > (void)cmpxchg(&work->flags, flags, flags & ~IRQ_WORK_BUSY);
> > ffffffff8119327c: 49 83 e7 fc and $0xfffffffffffffffc,%r15
> > ffffffff81193280: 4c 89 f0 mov %r14,%rax
> > ffffffff81193283: f0 4c 0f b1 7b f8 lock cmpxchg %r15,-0x8(%rbx)
> > ffffffff81193289: 4c 89 e3 mov %r12,%rbx
> > while (llnode != NULL) {
> > ffffffff8119328c: 4d 85 e4 test %r12,%r12
> > ffffffff8119328f: 75 cf jne ffffffff81193260 <irq_work_tick+0x60>
> > irq_work_run_list(raised);
> > irq_work_run_list(this_cpu_ptr(&lazy_list));
> > ffffffff81193291: 48 c7 c1 88 5b 01 00 mov $0x15b88,%rcx
> > ffffffff81193298: 65 48 03 0c 25 d8 a1 add %gs:0xa1d8,%rcx
> > ffffffff8119329f: 00 00
> > ffffffff811932a1: ff 14 25 10 08 c2 81 callq *0xffffffff81c20810
> > BUG_ON(!irqs_disabled());
> > ffffffff811932a8: a9 00 02 00 00 test $0x200,%eax
> > ffffffff811932ad: 75 4b jne ffffffff811932fa <irq_work_tick+0xfa>
> > return ACCESS_ONCE(head->first) == NULL;
> > ffffffff811932af: 48 83 39 00 cmpq $0x0,(%rcx)
> >
> >
> > I'm no x86/assembly expert, anyway my interpretation is:
> >
> > 'irq_work_tick+0xaf' corresponds to address 0xffffffff811932af, so the
> > exception occurs during the execution of 'cmpq $0x0,(%rcx)' or
> > 'ACCESS_ONCE(head->first) == NULL'. The instruction checks if the
> > memory location in RCX contains 0. Supposedly the register should
> > contain the address of 'head->first', however the value is
> > 0x47e992c1bc778d00, which doesn't look like a valid address.
> >
> > RCX is set shortly before:
> >
> > ffffffff81193291: 48 c7 c1 88 5b 01 00 mov $0x15b88,%rcx
> > ffffffff81193298: 65 48 03 0c 25 d8 a1 add %gs:0xa1d8,%rcx
> >
> > As stated in https://www.kernel.org/doc/Documentation/this_cpu_ops.txt
> > the GS segment register is used by the kernel for per-cpu
> > variables. And effectively System.map tells us the offset 0x15b88
> > corresponds to the per-cpu variable lazy_list:
> >
> > 0000000000015b88 d lazy_list
> >
> > The register dump shows GS with a value of 0xffff96fb5fc00000, which
> > looks reasonable.
> >
> > But wait, right after setting RCX we do this:
> >
> > ffffffff811932a1: ff 14 25 10 08 c2 81 callq *0xffffffff81c20810
> > BUG_ON(!irqs_disabled());
> >
> > According to my (limited) understanding of x86 calling conventions RCX
> > is a caller-saved register, thus the caller should save it on the
> > stack to preserve its value across a function call.
> > (https://en.wikipedia.org/wiki/X86_calling_conventions)
> >
> > The call to '*0xffffffff81c20810' appears to be related with
> > KVM/paravirtualization:
> >
> > grep ffffffff81c20810 System.map
> > ffffffff81c20810 D pv_irq_ops
> >
> > Since there is no offset it must be calling the first function in the
> > structure, which is 'safe_fl' and points to 'native_save_fl'
> > (https://elixir.bootlin.com/linux/v4.14.36/source/arch/x86/kernel/paravirt.c#L316)
> >
> > objdump -d -S --start-address=0x$(grep native_save_fl System.map | sed
> > -e "s/ \+.*//") vmlinux | less
> >
> > static inline unsigned long native_save_fl(void)
> > {
> > ffffffff81060240: 55 push %rbp
> > ffffffff81060241: 48 89 e5 mov %rsp,%rbp
> > ffffffff81060244: 48 83 ec 10 sub $0x10,%rsp
> > ffffffff81060248: 65 48 8b 04 25 28 00 mov %gs:0x28,%rax
> > ffffffff8106024f: 00 00
> > ffffffff81060251: 48 89 45 f8 mov %rax,-0x8(%rbp)
> > /*
> > * "=rm" is safe here, because "pop" adjusts the stack before
> > * it evaluates its effective address -- this is part of the
> > * documented behavior of the "pop" instruction.
> > */
> > asm volatile("# __raw_save_flags\n\t"
> > ffffffff81060255: 9c pushfq
> > ffffffff81060256: 8f 45 f0 popq -0x10(%rbp)
> > "pushf ; pop %0"
> > : "=rm" (flags)
> > : /* no input */
> > : "memory");
> >
> > return flags;
> > ffffffff81060259: 48 8b 45 f0 mov -0x10(%rbp),%rax
> > ffffffff8106025d: 65 48 8b 0c 25 28 00 mov %gs:0x28,%rcx
> > ffffffff81060264: 00 00
> > ffffffff81060266: 48 3b 4d f8 cmp -0x8(%rbp),%rcx
> > ffffffff8106026a: 75 06 jne ffffffff81060272 <native_save_fl+0x32>
> > ffffffff8106026c: 48 83 c4 10 add $0x10,%rsp
> > ffffffff81060270: 5d pop %rbp
> > ffffffff81060271: c3 retq
> > ffffffff81060272: e8 09 ec 01 00 callq ffffffff8107ee80 <__stack_chk_fail>
> > ffffffff81060277: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> > ffffffff8106027e: 00 0
> >
> > At 0xffffffff8106025d this clobbers RCX! I don't know why clang
> > doesn't save the value on the stack before calling native_save_fl(),
> > but that seems to be the problem.
> >
> > Again, I'm not an expert in this area and ventured into territory
> > unknown to me, so please excuse if I got something totally wrong ...
> >
>
> Hi Matthias,
>
> again you helped me a lot.
>
> I recalled that I had success in the early days of llvmlinux with...
>
> CONFIG_PARAVIRT=n
>
> ...as suggested by Steve Rostedt.
>
> This needed some more hackish kconfig kungfu these days - patches attached.
>
> In the end I could boot into a 4.14.37-1-iniza-llvmlinux kernel
> (kernel-config and dmesg-log attached).
>
> Concerning your analysis...
>
> Is this a clang-bug?
> Or can this be solved in the linux-kernel code?
> What are your suggestion?
> And can you help on getting this fixed with the help of your Chromium
> team and/or bring it to the right persons?
Apparently stack-protector code is clobbering RCX, however the
paravirt code expects the register to be preserved:
struct pv_irq_ops {
/*
* ...
* NOTE: These functions callers expect the callee to preserve
* more registers than the standard C calling convention.
*/
struct paravirt_callee_save save_fl;
We think the solution is to mark native_save_fl with
'__nostackprotector', however this requires adding support for
such an attribute to clang. Our compiler team is working on this.
> IMHO an analysis like you did might help, too.
> I am not an expert in this area to say it is good/correct but I could
> follow and read it like a thriller.
> Where is Lieutenant Columbo when you need him :-)?
>
> Moreover, a test-case can be helpful, e.g. "Is clang clobbering RCX?"
> when it's a clang-bug.
>
> I tried to find some suitable test-case myself.
>
> The clang-source has a test-dir, but this is new to me.
>
> I tried to run a single test like test/Sema/asm.c in [1] with
> llvm-tools Debian-package which ships "lit.py" script.
>
> Not sure if this is a suitable for x86-64 (amd64) as the first line says...
>
> // RUN: %clang_cc1 %s -Wno-private-extern -triple i386-pc-linux-gnu
> -verify -fsyntax-only
>
> root# cd /path/to/clang-source
>
> root# python /usr/lib/llvm-7/build/utils/lit/lit.py -sv test/Sema
> lit.py: /usr/lib/llvm-7/build/utils/lit/lit/TestingConfig.py:101:
> fatal: unable to parse config file
> '/path/to/clang-source/test/lit.cfg.py', traceback: Traceback (most
> recent call last):
> File "/usr/lib/llvm-7/build/utils/lit/lit/TestingConfig.py", line
> 88, in load_from_path
> exec(compile(data, path, 'exec'), cfg_globals, None)
> File "/home/sdi/src/linux-kernel/clang/test/lit.cfg.py", line 25, in <module>
> config.test_format = lit.formats.ShTest(not llvm_config.use_lit_shell)
> AttributeError: 'NoneType' object has no attribute 'use_lit_shell'
>
> Not sure if you have any experiences in this area, but I think asking
> on llvm-dev is a good idea.
>
> If we have a reproducible test-case we could ask on llvm-dev?
>
> In the case of linux-kernel...
> Ask Linux/x86 folks for a test-case to see if the compiler clobbers RCX?
> Show your analysis to Linux/(x86)/KVM folks?
I'm not an expert, but I don't think there is a generic test case for
this special scenario involving paravirt and stack protector. It's
probably best to just fix the problem and move on.
On Mon, May 7, 2018 at 7:49 PM, Matthias Kaehlcke <[email protected]> wrote:
> On Sun, May 06, 2018 at 09:42:09AM +0200, Sedat Dilek wrote:
>> On Wed, Apr 25, 2018 at 1:06 AM, Matthias Kaehlcke <[email protected]> wrote:
>> > On Tue, Apr 24, 2018 at 01:54:29PM +0200, Sedat Dilek wrote:
>> >> Hi Matthias,
>> >>
>> >> a big thank you for giving all the informations!
>> >>
>> >> I used your mka/llvm/v4.14_ext Git tree...
>> >>
>> >> https://chromium.googlesource.com/chromiumos/third_party/kernel/+log/sandbox/mka/llvm/v4.14_ext
>> >>
>> >> ...and was able to compile with clang-6.0 from Debian/buster64 with...
>> >>
>> >> ...reverting the clang-3/clang-4 patches...
>> >>
>> >> user$ for p in 69e44656ae43 222b88977a00 cdfcf1e45537 26f14c9225a6
>> >> 0385a18e9995 68dab143c9b4 ; do echo [ $p ] ; LC_ALL=C git revert
>> >> --no-edit $p ; done
>> >>
>> >> 0001-Revert-CLANG3-core-clang-work-around-x86-regparm-int.patch
>> >> 0002-Revert-CLANG4-futex-don-t-optimize-futex_detect_cmpx.patch
>> >> 0003-Revert-CLANG4-Disable-lkdtm-when-ftrace-is-enabled.patch
>> >> 0004-Revert-CLANG4-arm64-prefetch-Use-__builtin_arm_prefe.patch
>> >> 0005-Revert-CLANG4-kbuild-Add-meabi-gnu-to-the-clang-para.patch
>> >> 0006-Revert-CLANG4-crypto-arm64-aes-ce-Explicitly-pass-th.patch
>> >>
>> >> ...and needed two additional patches from upstream:
>> >>
>> >> 0007-kbuild-clang-remove-crufty-HOSTCFLAGS.patch <--- Label with
>> >> BACKPORT (XXX: Backported; Required when using HOSTCC in make-line,
>> >> see below)
>> >> 0008-x86-xen-remove-the-use-of-VLAIS.patch <--- Label with UPSTREAM
>> >> (XXX: cherry-picked)
>> >>
>> >> These 3 patches in your Git branch are in Linux v4.14.36...
>> >>
>> >> 9af5ddf981ed BACKPORT: kbuild: disable clang's default use of
>> >> -fmerge-all-constants
>> >> f24088a3842c BACKPORT: kbuild: Set KBUILD_CFLAGS before incl. arch Makefile
>> >> d4dfe384346d UPSTREAM: kbuild: fix linker feature test macros when
>> >> cross compiling with Clang
>> >>
>> >> ...and can be dropped.
>> >> I do not know your work-flow/policy: Maybe you want to keep your
>> >> patch-stack against vanilla Linux v4.14 - without stables from
>> >> linux-stable-4.14.y.
>> >
>> > Yes, these patch stacks are based on vanilla Linux v4.14, one reason
>> > is that it isn't a moving target. Also I think it makes it easier for
>> > folks not merging LTS (though they probably should!) to locate all
>> > patches, and it's fairly trivial to skip the (currently) few patches
>> > not needed with the -stable tree.
>> >
>> >> For easy switching "mycompiler" I use a wrapper-script:
>> >>
>> >> root# cat /usr/bin/mycompiler
>> >> #!/bin/bash
>> >>
>> >> clang-6.0 "$@"
>> >> - EOF -
>> >>
>> >> user$ cp -v /boot/config-4.14.35-1-iniza-amd64 .config
>> >> user$ MAKE="make V=1" ; COMPILER="mycompiler" ;
>> >> MAKE_OPTS="CC=$COMPILER HOSTCC=$COMPILER"
>> >> user$ yes "" | $MAKE $MAKE_OPTS oldconfig && $MAKE $MAKE_OPTS
>> >> silentoldconfig < /dev/null
>> >>
>> >> The diffconfig to my current kernel-config-4.14 looks like this...
>> >>
>> >> user$ ./scripts/diffconfig /boot/config-4.14.35-1-iniza-amd64 .config
>> >> -ARCH_HAS_REFCOUNT y
>> >> -BPF_JIT_ALWAYS_ON n
>> >> -EXOFS_DEBUG n
>> >> -EXOFS_FS m
>> >> -GENERIC_CPU_VULNERABILITIES y
>> >> -KASAN n
>> >> -ORE m
>> >> -PAGE_TABLE_ISOLATION y
>> >> -RETPOLINE y
>> >> -UNWINDER_FRAME_POINTER n
>> >> -UNWINDER_GUESS n
>> >> -UNWINDER_ORC y
>> >> +FRAME_POINTER y
>> >> +FRAME_POINTER_UNWINDER y
>> >> +GUESS_UNWINDER n
>> >> +HAVE_ARCH_KMEMCHECK y
>> >> +HAVE_RELIABLE_STACKTRACE y
>> >> +ORC_UNWINDER n
>> >>
>> >> Unfortunately, I cannot boot into the generated kernel on bare metal.
>> >>
>> >> Checking with QEMU (version: 2.12~rc3) and catching earlyprintk, I see this...
>> >>
>> >> user$ echo $KPATH
>> >> $HOME/src/linux-kernel/important-files
>> >>
>> >> user$ ls -al
>> >> insgesamt 344916
>> >> drwxr-xr-x 2 sdi sdi 4096 Apr 24 13:15 .
>> >> drwxr-xr-x 20 sdi sdi 4096 Apr 24 13:13 ..
>> >> -rw-r--r-- 1 sdi sdi 4528416 Apr 24 12:42 bzImage
>> >> lrwxrwxrwx 1 sdi sdi 35 Apr 24 13:15 initrd.img ->
>> >> initrd.img-4.14.0-1-iniza-llvmlinux
>> >> -rw-r--r-- 1 sdi sdi 25572955 Apr 24 13:08
>> >> initrd.img-4.14.0-1-iniza-llvmlinux
>> >> -rw-r--r-- 1 sdi sdi 2887195 Apr 24 12:42 System.map
>> >> -rwxr-xr-x 1 sdi sdi 326116744 Apr 24 12:42 vmlinux
>> >>
>> >> user$ sudo qemu-system-x86_64 -enable-kvm -M pc -kernel $KPATH/bzImage
>> >> -initrd $KPATH/initrd.img -m 512 -net none -serial stdio -append
>> >> "root=/dev/ram0 console=ttyS0 hung_task_panic=1
>> >> earlyprintk=ttyS0,115200"
>> >>
>> >> Probing EDD (edd=off to disable)... ok
>> >> [ 0.000000] Linux version 4.14.0-1-iniza-llvmlinux
>> >> ([email protected]@iniza) (clang version 6.0.0-1
>> >> (tags/RELEASE_600/final)) #1 SMP Tue Apr 24 12:42:21 CEST 2018
>> >> [ 0.000000] Command line: root=/dev/ram0 console=ttyS0
>> >> hung_task_panic=1 earlyprintk=ttyS0,115200
>> >> [ 0.000000] x86/fpu: x87 FPU will use FXSAVE
>> >> [ 0.000000] e820: BIOS-provided physical RAM map:
>> >> [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
>> >> [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
>> >> [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
>> >> [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000001ffdffff] usable
>> >> [ 0.000000] BIOS-e820: [mem 0x000000001ffe0000-0x000000001fffffff] reserved
>> >> [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
>> >> [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
>> >> [ 0.000000] bootconsole [earlyser0] enabled
>> >> [ 0.000000] NX (Execute Disable) protection: active
>> >> [ 0.000000] random: fast init done
>> >> [ 0.000000] SMBIOS 2.8 present.
>> >> [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
>> >> 1.11.1-1 04/01/2014
>> >> [ 0.000000] Hypervisor detected: KVM
>> >> [ 0.000000] tsc: Fast TSC calibration using PIT
>> >> [ 0.000000] e820: last_pfn = 0x1ffe0 max_arch_pfn = 0x400000000
>> >> [ 0.000000] x86/PAT: PAT not supported by CPU.
>> >> [ 0.000000] x86/PAT: Configuration [0-7]: WB WT UC- UC WB WT UC- UC
>> >> Memory KASLR using RDTSC...
>> >> [ 0.000000] found SMP MP-table at [mem 0x000f5d60-0x000f5d6f]
>> >> mapped at [ffffffffff000d60]
>> >> [ 0.000000] RAMDISK: [mem 0x1e77c000-0x1ffdffff]
>> >> [ 0.000000] ACPI: Early table checksum verification disabled
>> >> [ 0.000000] ACPI: RSDP 0x00000000000F5B90 000014 (v00 BOCHS )
>> >> [ 0.000000] ACPI: RSDT 0x000000001FFE157C 000030 (v01 BOCHS
>> >> BXPCRSDT 00000001 BXPC 00000001)
>> >> [ 0.000000] ACPI: FACP 0x000000001FFE1458 000074 (v01 BOCHS
>> >> BXPCFACP 00000001 BXPC 00000001)
>> >> [ 0.000000] ACPI: DSDT 0x000000001FFE0040 001418 (v01 BOCHS
>> >> BXPCDSDT 00000001 BXPC 00000001)
>> >> [ 0.000000] ACPI: FACS 0x000000001FFE0000 000040
>> >> [ 0.000000] ACPI: APIC 0x000000001FFE14CC 000078 (v01 BOCHS
>> >> BXPCAPIC 00000001 BXPC 00000001)
>> >> [ 0.000000] ACPI: HPET 0x000000001FFE1544 000038 (v01 BOCHS
>> >> BXPCHPET 00000001 BXPC 00000001)
>> >> [ 0.000000] No NUMA configuration found
>> >> [ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000001ffdffff]
>> >> [ 0.000000] NODE_DATA(0) allocated [mem 0x1e777000-0x1e77bfff]
>> >> [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
>> >> [ 0.000000] kvm-clock: cpu 0, msr 0:1e76f001, primary cpu clock
>> >> [ 0.000000] kvm-clock: using sched offset of 528742140 cycles
>> >> [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff
>> >> max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
>> >> [ 0.000000] Zone ranges:
>> >> [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
>> >> [ 0.000000] DMA32 [mem 0x0000000001000000-0x000000001ffdffff]
>> >> [ 0.000000] Normal empty
>> >> [ 0.000000] Device empty
>> >> [ 0.000000] Movable zone start for each node
>> >> [ 0.000000] Early memory node ranges
>> >> [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff]
>> >> [ 0.000000] node 0: [mem 0x0000000000100000-0x000000001ffdffff]
>> >> [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x000000001ffdffff]
>> >> [ 0.000000] ACPI: PM-Timer IO Port: 0x608
>> >> [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
>> >> [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
>> >> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
>> >> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
>> >> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
>> >> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
>> >> [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
>> >> [ 0.000000] Using ACPI (MADT) for SMP configuration information
>> >> [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000
>> >> [ 0.000000] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
>> >> [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]
>> >> [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
>> >> [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff]
>> >> [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff]
>> >> [ 0.000000] e820: [mem 0x20000000-0xfeffbfff] available for PCI devices
>> >> [ 0.000000] Booting paravirtualized kernel on KVM
>> >> [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff
>> >> max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
>> >> [ 0.000000] setup_percpu: NR_CPUS:512 nr_cpumask_bits:512
>> >> nr_cpu_ids:1 nr_node_ids:1
>> >> [ 0.000000] percpu: Embedded 37 pages/cpu @ffff96dd9e400000 s114392
>> >> r8192 d28968 u2097152
>> >> [ 0.000000] KVM setup async PF for cpu 0
>> >> [ 0.000000] kvm-stealtime: cpu 0, msr 1e40d900
>> >> [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 128872
>> >> [ 0.000000] Policy zone: DMA32
>> >> [ 0.000000] Kernel command line: root=/dev/ram0 console=ttyS0
>> >> hung_task_panic=1 earlyprintk=ttyS0,115200
>> >> [ 0.000000] PID hash table entries: 2048 (order: 2, 16384 bytes)
>> >> [ 0.000000] Memory: 474480K/523768K available (7639K kernel code,
>> >> 1005K rwdata, 2936K rodata, 1636K init, 688K bss, 49288K reserved, 0K
>> >> cma-reserved)
>> >> [ 0.000000] ftrace: allocating 27586 entries in 108 pages
>> >> [ 0.004000] Hierarchical RCU implementation.
>> >> [ 0.004000] RCU restricting CPUs from NR_CPUS=512 to nr_cpu_ids=1.
>> >> [ 0.004000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
>> >> [ 0.004000] NR_IRQS: 33024, nr_irqs: 256, preallocated irqs: 16
>> >> [ 0.004000] Console: colour VGA+ 80x25
>> >> [ 0.004000] console [ttyS0] enabled
>> >> [ 0.004000] console [ttyS0] enabled
>> >> [ 0.004000] bootconsole [earlyser0] disabled
>> >> [ 0.004000] bootconsole [earlyser0] disabled
>> >> [ 0.004000] clocksource: hpet: mask: 0xffffffff max_cycles:
>> >> 0xffffffff, max_idle_ns: 19112604467 ns
>> >> [ 0.004000] general protection fault: 0000 [#1] SMP
>> >> [ 0.004000] Modules linked in:
>> >> [ 0.004000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted
>> >> 4.14.0-1-iniza-llvmlinux #1
>> >> [ 0.004000] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
>> >> BIOS 1.11.1-1 04/01/2014
>> >> [ 0.004000] task: ffffffffaa610480 task.stack: ffffffffaa600000
>> >> [ 0.004000] RIP: 0010:irq_work_tick+0x9d/0x110
>> >> [ 0.004000] RSP: 0000:ffff96dd9e403e58 EFLAGS: 00010046
>> >> [ 0.004000] RAX: 0000000000000082 RBX: ffff96dd9e411d80 RCX: adecc9cc04e2ca00
>> >> [ 0.004000] RDX: 000000000001ba00 RSI: fffffffffffffed4 RDI: ffff96dd9e41ba38
>> >> [ 0.004000] RBP: ffff96dd9e403e78 R08: 0000000000000000 R09: 0000000000000018
>> >> [ 0.004000] R10: 0000000000000000 R11: 0000000000000018 R12: ffffffffaa61bd40
>> >> [ 0.004000] R13: 0000000000000000 R14: ffffffffaa610480 R15: 0000000000000000
>> >> [ 0.004000] FS: 0000000000000000(0000) GS:ffff96dd9e400000(0000)
>> >> knlGS:0000000000000000
>> >> [ 0.004000] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> >> [ 0.004000] CR2: 00000000ffffffff CR3: 0000000008a09000 CR4: 00000000000006b0
>> >> [ 0.004000] Call Trace:
>> >> [ 0.004000] <IRQ>
>> >> [ 0.004000] update_process_times+0x6e/0xa0
>> >> [ 0.004000] tick_periodic+0x78/0x90
>> >> [ 0.004000] tick_handle_periodic+0x26/0x80
>> >> [ 0.004000] timer_interrupt+0x13/0x20
>> >> [ 0.004000] __handle_irq_event_percpu+0x106/0x230
>> >> [ 0.004000] handle_irq_event+0x5a/0xc0
>> >> [ 0.004000] handle_level_irq+0x11a/0x190
>> >> [ 0.004000] handle_irq+0x1f/0x30
>> >> [ 0.004000] do_IRQ+0x4b/0xd0
>> >> [ 0.004000] common_interrupt+0x93/0x93
>> >> [ 0.004000] </IRQ>
>> >> [ 0.004000] RIP: 0010:native_restore_fl+0x12/0x20
>> >> [ 0.004000] RSP: 0000:ffffffffaa603e10 EFLAGS: 00000286 ORIG_RAX:
>> >> ffffffffffffffcf
>> >> [ 0.004000] RAX: 0000000000000001 RBX: ffff96dd9d0c0608 RCX: 0000000000000000
>> >> [ 0.004000] RDX: ffff96dd9d0c0400 RSI: 0000000000000286 RDI: 0000000000000286
>> >> [ 0.004000] RBP: ffffffffaa603e18 R08: 0000000000000001 R09: 000000000000003f
>> >> [ 0.004000] R10: 0000000000000286 R11: 0000000000000007 R12: ffff96dd9d0c0514
>> >> [ 0.004000] R13: ffff96dd9d0c04e0 R14: ffffffffaa61bd40 R15: ffff96dd9d0c0400
>> >> [ 0.004000] _raw_spin_unlock_irqrestore+0x1a/0x20
>> >> [ 0.004000] __setup_irq+0x5d9/0x780
>> >> [ 0.004000] setup_irq+0x5c/0x90
>> >> [ 0.004000] hpet_time_init+0x32/0x40
>> >> [ 0.004000] x86_late_time_init+0x10/0x20
>> >> [ 0.004000] start_kernel+0x45d/0x580
>> >> [ 0.004000] x86_64_start_kernel+0x30f/0x320
>> >> [ 0.004000] secondary_startup_64+0xa5/0xa5
>> >> [ 0.004000] Code: f0 f0 4c 0f b1 7b f8 4c 89 e3 4d 85 e4 75 cf 48
>> >> c7 c1 88 5b 01 00 65 48 03 0c 25 d8 a1 00 00 ff 14 25 10 08 62 aa f6
>> >> c4 02 75 4d <48> 83 39 00 74 3e 31 db 48 87 19 48 85 db 74 34 0f 1f 00
>> >> 48 8d
>> >> [ 0.004000] RIP: irq_work_tick+0x9d/0x110 RSP: ffff96dd9e403e58
>> >> [ 0.004000] ---[ end trace 8c7851007fbc6b6f ]---
>> >> [ 0.004000] Kernel panic - not syncing: Fatal exception in interrupt
>> >> [ 0.004000] ---[ end Kernel panic - not syncing: Fatal exception in interrupt
>> >>
>> >> Is this saying someting to you?
>> >
>> > It doesn't ring a bell, but I can repro it (with different offsets):
>> >
>> > [ 0.004000] general protection fault: 0000 [#1] SMP
>> > [ 0.004000] Modules linked in:
>> > [ 0.004000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.14.0+ #10
>> > [ 0.004000] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
>> > [ 0.004000] task: ffffffffbb410480 task.stack: ffffffffbb400000
>> > [ 0.004000] RIP: 0010:irq_work_tick+0xaf/0x120
>> > [ 0.004000] RSP: 0000:ffff96fb5fc03e58 EFLAGS: 00010046
>> > [ 0.004000] RAX: 0000000000000082 RBX: ffff96fb5fc11d80 RCX: 47e992c1bc778d00
>> > [ 0.004000] RDX: 000000000001ba00 RSI: fffffffffffffed4 RDI: ffff96fb5fc1ba38
>> > [ 0.004000] RBP: ffff96fb5fc03e78 R08: 0000000000000000 R09: 0000000000000018
>> > [ 0.004000] R10: 0000000000000000 R11: 0000000000000018 R12: ffffffffbb41bd40
>> > [ 0.004000] R13: 0000000000000000 R14: ffffffffbb410480 R15: 0000000000000000
>> > [ 0.004000] FS: 0000000000000000(0000) GS:ffff96fb5fc00000(0000) knlGS:0000000000000000
>> > [ 0.004000] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> > [ 0.004000] CR2: 00000000ffffffff CR3: 000000001ec09000 CR4: 00000000000006b0
>> > [ 0.004000] Call Trace:
>> > [ 0.004000] <IRQ>
>> > [ 0.004000] update_process_times+0x6e/0xa0
>> > [ 0.004000] tick_periodic+0x78/0x90
>> > [ 0.004000] tick_handle_periodic+0x26/0x80
>> > [ 0.004000] timer_interrupt+0x13/0x20
>> > [ 0.004000] __handle_irq_event_percpu+0x106/0x230
>> > [ 0.004000] handle_irq_event+0x5a/0xc0
>> > [ 0.004000] handle_level_irq+0x11a/0x190
>> > [ 0.004000] handle_irq+0x1f/0x30
>> > [ 0.004000] do_IRQ+0x4b/0xd0
>> > [ 0.004000] common_interrupt+0x93/0x93
>> > [ 0.004000] </IRQ>
>> > [ 0.004000] RIP: 0010:native_restore_fl+0xd/0x20
>> > [ 0.004000] RSP: 0000:ffffffffbb403e08 EFLAGS: 00000282 ORIG_RAX: ffffffffffffffcf
>> > [ 0.004000] RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000000
>> > [ 0.004000] RDX: ffff96fb5f01b800 RSI: 0000000000000282 RDI: 0000000000000282
>> > [ 0.004000] RBP: ffffffffbb403e10 R08: 0000000000000001 R09: 000000000000003f
>> > [ 0.004000] R10: ffff96fb5f01b800 R11: 0000000000000007 R12: ffff96fb5f01ba08
>> > [ 0.004000] R13: ffff96fb5f01b8e0 R14: ffffffffbb41bd40 R15: ffff96fb5f01b800
>> > [ 0.004000] _raw_spin_unlock_irqrestore+0x1a/0x20
>> > [ 0.004000] __setup_irq+0x610/0x7b0
>> > [ 0.004000] setup_irq+0x5b/0x90
>> > [ 0.004000] hpet_time_init+0x32/0x40
>> > [ 0.004000] x86_late_time_init+0x10/0x20
>> > [ 0.004000] start_kernel+0x460/0x580
>> > [ 0.004000] x86_64_start_kernel+0x30f/0x320
>> > [ 0.004000] secondary_startup_64+0xa5/0xa5
>> > [ 0.004000] Code: 4c 0f b1 7b f8 4c 89 e3 4d 85 e4 75 cf 48 c7 c1 88 5b 01 00 65 48 03 0c 25 d8 a1 00 00 ff 14 25 10 08 42 bb a9 00 02 00 00 75 4b <48> 83 39 00 74 3c 31 db 48 87 19 48 85 db 74 32 90 48 8d 7b f8
>> > [ 0.004000] RIP: irq_work_tick+0xaf/0x120 RSP: ffff96fb5fc03e58
>> > [ 0.004000] ---[ end trace 08945838e05bf5b2 ]---
>> > [ 0.004000] Kernel panic - not syncing: Fatal exception in interrupt
>> > [ 0.004000] ---[ end Kernel panic - not syncing: Fatal exception in interrupt
>> >
>> > The exception occurs at 'irq_work_tick+0xaf'
>> >
>> > objdump -d -S --start-address=0x$(grep irq_work_tick System.map | sed -e
>> > "s/ \+.*//") vmlinux | less
>> >
>> > ...
>> > void irq_work_tick(void)
>> > {
>> > ffffffff81193200: 55 push %rbp
>> > ...
>> > static inline struct llist_node *llist_del_all(struct llist_head *head)
>> > {
>> > return xchg(&head->first, NULL);
>> > ffffffff8119324a: 48 87 19 xchg %rbx,(%rcx)
>> > while (llnode != NULL) {
>> > ffffffff8119324d: 48 85 db test %rbx,%rbx
>> > ffffffff81193250: 74 3f je ffffffff81193291 <irq_work_tick+0x91>
>> > ffffffff81193252: 0f 1f 40 00 nopl 0x0(%rax)
>> > ffffffff81193256: 66 2e 0f 1f 84 00 00 nopw %cs:0x0(%rax,%rax,1)
>> > ffffffff8119325d: 00 00 00
>> > work = llist_entry(llnode, struct irq_work, llnode);
>> > ffffffff81193260: 48 8d 7b f8 lea -0x8(%rbx),%rdi
>> > flags = work->flags & ~IRQ_WORK_PENDING;
>> > ffffffff81193264: 4c 8b 7b f8 mov -0x8(%rbx),%r15
>> > return node->next;
>> > ffffffff81193268: 4c 8b 23 mov (%rbx),%r12
>> > ffffffff8119326b: 4d 89 fe mov %r15,%r14
>> > ffffffff8119326e: 49 83 e6 fe and $0xfffffffffffffffe,%r14
>> > xchg(&work->flags, flags);
>> > ffffffff81193272: 4c 89 f0 mov %r14,%rax
>> > ffffffff81193275: 48 87 43 f8 xchg %rax,-0x8(%rbx)
>> > work->func(work);
>> > ffffffff81193279: ff 53 08 callq *0x8(%rbx)
>> > (void)cmpxchg(&work->flags, flags, flags & ~IRQ_WORK_BUSY);
>> > ffffffff8119327c: 49 83 e7 fc and $0xfffffffffffffffc,%r15
>> > ffffffff81193280: 4c 89 f0 mov %r14,%rax
>> > ffffffff81193283: f0 4c 0f b1 7b f8 lock cmpxchg %r15,-0x8(%rbx)
>> > ffffffff81193289: 4c 89 e3 mov %r12,%rbx
>> > while (llnode != NULL) {
>> > ffffffff8119328c: 4d 85 e4 test %r12,%r12
>> > ffffffff8119328f: 75 cf jne ffffffff81193260 <irq_work_tick+0x60>
>> > irq_work_run_list(raised);
>> > irq_work_run_list(this_cpu_ptr(&lazy_list));
>> > ffffffff81193291: 48 c7 c1 88 5b 01 00 mov $0x15b88,%rcx
>> > ffffffff81193298: 65 48 03 0c 25 d8 a1 add %gs:0xa1d8,%rcx
>> > ffffffff8119329f: 00 00
>> > ffffffff811932a1: ff 14 25 10 08 c2 81 callq *0xffffffff81c20810
>> > BUG_ON(!irqs_disabled());
>> > ffffffff811932a8: a9 00 02 00 00 test $0x200,%eax
>> > ffffffff811932ad: 75 4b jne ffffffff811932fa <irq_work_tick+0xfa>
>> > return ACCESS_ONCE(head->first) == NULL;
>> > ffffffff811932af: 48 83 39 00 cmpq $0x0,(%rcx)
>> >
>> >
>> > I'm no x86/assembly expert, anyway my interpretation is:
>> >
>> > 'irq_work_tick+0xaf' corresponds to address 0xffffffff811932af, so the
>> > exception occurs during the execution of 'cmpq $0x0,(%rcx)' or
>> > 'ACCESS_ONCE(head->first) == NULL'. The instruction checks if the
>> > memory location in RCX contains 0. Supposedly the register should
>> > contain the address of 'head->first', however the value is
>> > 0x47e992c1bc778d00, which doesn't look like a valid address.
>> >
>> > RCX is set shortly before:
>> >
>> > ffffffff81193291: 48 c7 c1 88 5b 01 00 mov $0x15b88,%rcx
>> > ffffffff81193298: 65 48 03 0c 25 d8 a1 add %gs:0xa1d8,%rcx
>> >
>> > As stated in https://www.kernel.org/doc/Documentation/this_cpu_ops.txt
>> > the GS segment register is used by the kernel for per-cpu
>> > variables. And effectively System.map tells us the offset 0x15b88
>> > corresponds to the per-cpu variable lazy_list:
>> >
>> > 0000000000015b88 d lazy_list
>> >
>> > The register dump shows GS with a value of 0xffff96fb5fc00000, which
>> > looks reasonable.
>> >
>> > But wait, right after setting RCX we do this:
>> >
>> > ffffffff811932a1: ff 14 25 10 08 c2 81 callq *0xffffffff81c20810
>> > BUG_ON(!irqs_disabled());
>> >
>> > According to my (limited) understanding of x86 calling conventions RCX
>> > is a caller-saved register, thus the caller should save it on the
>> > stack to preserve its value across a function call.
>> > (https://en.wikipedia.org/wiki/X86_calling_conventions)
>> >
>> > The call to '*0xffffffff81c20810' appears to be related with
>> > KVM/paravirtualization:
>> >
>> > grep ffffffff81c20810 System.map
>> > ffffffff81c20810 D pv_irq_ops
>> >
>> > Since there is no offset it must be calling the first function in the
>> > structure, which is 'safe_fl' and points to 'native_save_fl'
>> > (https://elixir.bootlin.com/linux/v4.14.36/source/arch/x86/kernel/paravirt.c#L316)
>> >
>> > objdump -d -S --start-address=0x$(grep native_save_fl System.map | sed
>> > -e "s/ \+.*//") vmlinux | less
>> >
>> > static inline unsigned long native_save_fl(void)
>> > {
>> > ffffffff81060240: 55 push %rbp
>> > ffffffff81060241: 48 89 e5 mov %rsp,%rbp
>> > ffffffff81060244: 48 83 ec 10 sub $0x10,%rsp
>> > ffffffff81060248: 65 48 8b 04 25 28 00 mov %gs:0x28,%rax
>> > ffffffff8106024f: 00 00
>> > ffffffff81060251: 48 89 45 f8 mov %rax,-0x8(%rbp)
>> > /*
>> > * "=rm" is safe here, because "pop" adjusts the stack before
>> > * it evaluates its effective address -- this is part of the
>> > * documented behavior of the "pop" instruction.
>> > */
>> > asm volatile("# __raw_save_flags\n\t"
>> > ffffffff81060255: 9c pushfq
>> > ffffffff81060256: 8f 45 f0 popq -0x10(%rbp)
>> > "pushf ; pop %0"
>> > : "=rm" (flags)
>> > : /* no input */
>> > : "memory");
>> >
>> > return flags;
>> > ffffffff81060259: 48 8b 45 f0 mov -0x10(%rbp),%rax
>> > ffffffff8106025d: 65 48 8b 0c 25 28 00 mov %gs:0x28,%rcx
>> > ffffffff81060264: 00 00
>> > ffffffff81060266: 48 3b 4d f8 cmp -0x8(%rbp),%rcx
>> > ffffffff8106026a: 75 06 jne ffffffff81060272 <native_save_fl+0x32>
>> > ffffffff8106026c: 48 83 c4 10 add $0x10,%rsp
>> > ffffffff81060270: 5d pop %rbp
>> > ffffffff81060271: c3 retq
>> > ffffffff81060272: e8 09 ec 01 00 callq ffffffff8107ee80 <__stack_chk_fail>
>> > ffffffff81060277: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
>> > ffffffff8106027e: 00 0
>> >
>> > At 0xffffffff8106025d this clobbers RCX! I don't know why clang
>> > doesn't save the value on the stack before calling native_save_fl(),
>> > but that seems to be the problem.
>> >
>> > Again, I'm not an expert in this area and ventured into territory
>> > unknown to me, so please excuse if I got something totally wrong ...
>> >
>>
>> Hi Matthias,
>>
>> again you helped me a lot.
>>
>> I recalled that I had success in the early days of llvmlinux with...
>>
>> CONFIG_PARAVIRT=n
>>
>> ...as suggested by Steve Rostedt.
>>
>> This needed some more hackish kconfig kungfu these days - patches attached.
>>
>> In the end I could boot into a 4.14.37-1-iniza-llvmlinux kernel
>> (kernel-config and dmesg-log attached).
>>
>> Concerning your analysis...
>>
>> Is this a clang-bug?
>> Or can this be solved in the linux-kernel code?
>> What are your suggestion?
>> And can you help on getting this fixed with the help of your Chromium
>> team and/or bring it to the right persons?
>
> Apparently stack-protector code is clobbering RCX, however the
> paravirt code expects the register to be preserved:
>
> struct pv_irq_ops {
> /*
> * ...
> * NOTE: These functions callers expect the callee to preserve
> * more registers than the standard C calling convention.
> */
> struct paravirt_callee_save save_fl;
>
> We think the solution is to mark native_save_fl with
> '__nostackprotector', however this requires adding support for
> such an attribute to clang. Our compiler team is working on this.
>
OK, cool.
Can you inform me when there is progress on this?
Thanks.
>> IMHO an analysis like you did might help, too.
>> I am not an expert in this area to say it is good/correct but I could
>> follow and read it like a thriller.
>> Where is Lieutenant Columbo when you need him :-)?
>>
>> Moreover, a test-case can be helpful, e.g. "Is clang clobbering RCX?"
>> when it's a clang-bug.
>>
>> I tried to find some suitable test-case myself.
>>
>> The clang-source has a test-dir, but this is new to me.
>>
>> I tried to run a single test like test/Sema/asm.c in [1] with
>> llvm-tools Debian-package which ships "lit.py" script.
>>
>> Not sure if this is a suitable for x86-64 (amd64) as the first line says...
>>
>> // RUN: %clang_cc1 %s -Wno-private-extern -triple i386-pc-linux-gnu
>> -verify -fsyntax-only
>>
>> root# cd /path/to/clang-source
>>
>> root# python /usr/lib/llvm-7/build/utils/lit/lit.py -sv test/Sema
>> lit.py: /usr/lib/llvm-7/build/utils/lit/lit/TestingConfig.py:101:
>> fatal: unable to parse config file
>> '/path/to/clang-source/test/lit.cfg.py', traceback: Traceback (most
>> recent call last):
>> File "/usr/lib/llvm-7/build/utils/lit/lit/TestingConfig.py", line
>> 88, in load_from_path
>> exec(compile(data, path, 'exec'), cfg_globals, None)
>> File "/home/sdi/src/linux-kernel/clang/test/lit.cfg.py", line 25, in <module>
>> config.test_format = lit.formats.ShTest(not llvm_config.use_lit_shell)
>> AttributeError: 'NoneType' object has no attribute 'use_lit_shell'
>>
>> Not sure if you have any experiences in this area, but I think asking
>> on llvm-dev is a good idea.
>>
>> If we have a reproducible test-case we could ask on llvm-dev?
>>
>> In the case of linux-kernel...
>> Ask Linux/x86 folks for a test-case to see if the compiler clobbers RCX?
>> Show your analysis to Linux/(x86)/KVM folks?
>
> I'm not an expert, but I don't think there is a generic test case for
> this special scenario involving paravirt and stack protector. It's
> probably best to just fix the problem and move on.
Yeah, I was able to run a single clang-test like Sema/asm.c.
There is a thread on llvm-dev "[clang] Running a single testcase"
where I had success with using the built clang binary etc.
A nice excursion in that area.
Thanks again.
- Sedat -
On Mon, May 7, 2018 at 7:49 PM, Matthias Kaehlcke <[email protected]> wrote:
[...]
>> Hi Matthias,
>>
>> again you helped me a lot.
>>
>> I recalled that I had success in the early days of llvmlinux with...
>>
>> CONFIG_PARAVIRT=n
>>
>> ...as suggested by Steve Rostedt.
>>
>> This needed some more hackish kconfig kungfu these days - patches attached.
>>
>> In the end I could boot into a 4.14.37-1-iniza-llvmlinux kernel
>> (kernel-config and dmesg-log attached).
>>
>> Concerning your analysis...
>>
>> Is this a clang-bug?
>> Or can this be solved in the linux-kernel code?
>> What are your suggestion?
>> And can you help on getting this fixed with the help of your Chromium
>> team and/or bring it to the right persons?
>
> Apparently stack-protector code is clobbering RCX, however the
> paravirt code expects the register to be preserved:
>
> struct pv_irq_ops {
> /*
> * ...
> * NOTE: These functions callers expect the callee to preserve
> * more registers than the standard C calling convention.
> */
> struct paravirt_callee_save save_fl;
>
> We think the solution is to mark native_save_fl with
> '__nostackprotector', however this requires adding support for
> such an attribute to clang. Our compiler team is working on this.
>
[ CC Steven Rostedt ]
Hi Matthias,
as a confirmation I could boot into a paravirtualized llvm-kernel
v4.14.40 with CC_STACKPROTECTOR_NONE=y.
$ ./scripts/diffconfig /boot/config-4.14.37-1-iniza-llvmlinux
/boot/config-4.14.40-1-iniza-llvmlinux | grep CC_STACKPROTECTOR
CC_STACKPROTECTOR y -> n
CC_STACKPROTECTOR_NONE n -> y
CC_STACKPROTECTOR_STRONG y -> n
$ grep -i paravirt qemu-log_4.14.40-1-iniza-llvmlinux.txt
[ 0.000000] Booting paravirtualized kernel on KVM
Both kernel-configs, the qemu-script and qemu-logs are attached.
For people following I can send you the used patch-series (see below).
Thanks.
Regards,
- Sedat -
P.S.: Patchset contains patchstack from llvm_v4.14_ext by Matthias &
coworkers with reverted stuff (mostly kbuild) being in
linux-stable-4.14.y plus some more patches (xen VLAIS, kbuild HOSTCC,
etc.)
$ git log --oneline v4.14.40..
590207f1e63c (HEAD -> 4.14.40-1-iniza-llvmlinux) Merge branch
'for-4.14/x86-xen-clang-fixes' into 4.14.40-1-iniza-llvmlinux
36a1c0f031f4 Merge branch 'for-4.14/kbuild-clang-fixes' into
4.14.40-1-iniza-llvmlinux
c3afa4921927 Merge branch
'for-llvm_v4.14_ext/revert-clang-kbuild-fixes' into
4.14.40-1-iniza-llvmlinux
32f3dd55b9c8 (for-4.14/x86-xen-clang-fixes) x86: xen: remove the use of VLAIS
6b78d2ecfae1 (for-4.14/kbuild-clang-fixes) kbuild: clang: remove
crufty HOSTCFLAGS
53da106d34a3 (for-llvm_v4.14_ext/revert-clang-kbuild-fixes) Revert
"UPSTREAM: kbuild: fix linker feature test macros when cross compiling
with Clang"
6136ef7f8eaf Revert "BACKPORT: kbuild: Set KBUILD_CFLAGS before incl.
arch Makefile"
73a00f4cfd62 Revert "BACKPORT: kbuild: disable clang's default use of
-fmerge-all-constants"
6d945349c761 Revert "CLANG4: crypto: arm64/aes-ce: Explicitly pass
through assembler options"
dc09abad371e Revert "CLANG4: kbuild: Add -meabi gnu to the clang parameters"
b4d76971bd02 Revert "CLANG4: arm64: prefetch: Use
__builtin_arm_prefetch() for clang"
c779654513fd Revert "CLANG4: Disable lkdtm when ftrace is enabled"
c52593cc5e62 Revert "CLANG4: futex: don't optimize
futex_detect_cmpxchg() on ARM64"
a7074de141fb Revert "CLANG3: core: clang: work around x86 regparm /
intrinsics bug"
c03a7ac96a18 (llvm_v4.14_ext) UPSTREAM: sched/sysctl: Fix attributes
of some extern declarations
fc0f14867820 UPSTREAM: x86/boot: Disable Clang warnings about GNU extensions
9a7e28aa6b4a HACK: arm64: Mark CONFIG_CPU_BIG_ENDIAN as broken
a0092455e775 HACK: arm64: Mark CONFIG_ARM64_LSE_ATOMICS as broken
9de3cf0e0462 HACK: media: camss: Mark as broken
82951cf7d32c HACK: kasan: Mark as broken
2732c83a4fa4 HACK: exofs: Mark as broken
6a3124b1d1f8 UPSTREAM: serial: sh-sci: Fix init data attribute for
struct 'port_cfg'
b3b270e93807 UPSTREAM: bcache: Avoid nested function definition
6098eb444464 UPSTREAM: nfp: don't depend on compiler constant propagation
e95e3974644d UPSTREAM: md: raid10: remove VLAIS
69e44656ae43 CLANG3: core: clang: work around x86 regparm / intrinsics bug
222b88977a00 CLANG4: futex: don't optimize futex_detect_cmpxchg() on ARM64
cdfcf1e45537 CLANG4: Disable lkdtm when ftrace is enabled
26f14c9225a6 CLANG4: arm64: prefetch: Use __builtin_arm_prefetch() for clang
0385a18e9995 CLANG4: kbuild: Add -meabi gnu to the clang parameters
68dab143c9b4 CLANG4: crypto: arm64/aes-ce: Explicitly pass through
assembler options
9af5ddf981ed (llvm_v4.14) BACKPORT: kbuild: disable clang's default
use of -fmerge-all-constants
b84e202d9cd6 UPSTREAM: crypto: arm64/aes-ce-cipher - move assembler
code to .S file
f24088a3842c BACKPORT: kbuild: Set KBUILD_CFLAGS before incl. arch Makefile
d4dfe384346d UPSTREAM: kbuild: fix linker feature test macros when
cross compiling with Clang
Sedat,
Thanks for the report. We have a fix ready in
https://bugs.llvm.org/show_bug.cgi?id=37512. Can you report what
version of clang you were using and if earlier versions of clang have
this issue?
Thanks,
~Nick
On Sat, May 19, 2018 at 12:54 AM, Nick Desaulniers
<[email protected]> wrote:
> Sedat,
> Thanks for the report. We have a fix ready in
> https://bugs.llvm.org/show_bug.cgi?id=37512. Can you report what
> version of clang you were using and if earlier versions of clang have
> this issue?
> Thanks,
Hi Nick,
yesterday was public holiday in Germany, so I answer today.
These are really good news, Thanks.
For my recent experiments with Linux-kernel v4.14.y LTS I used Clang...
#1: version 6.0 from Debian/testing repositories
#2: version 7 (svn330207) from <apt-llvm.org>
In these cases I had success (good means boot on bare metal) or it
failed (bad verified in Qemu):
bad: CONFIG_PARAVIRT=y and CONFIG_CC_STACKPROTECTOR_STRONG=y
good: CONFIG_PARAVIRT=n and CONFIG_CC_STACKPROTECTOR_STRONG=y
good: CONFIG_PARAVIRT=y and CONFIG_CC_STACKPROTECTOR_NONE=y
So, the issue is in both Clang versions and I appreciate to have your
fix in LLVM/Clang version 6.0.1 as I don't know which version will get
into upcoming Debian/buster (version 10, current status: testing).
Hope this helps.
Regards,
- Sedat -
On Sat, May 19, 2018 at 12:54 AM, Nick Desaulniers
<[email protected]> wrote:
> Sedat,
> Thanks for the report. We have a fix ready in
> https://bugs.llvm.org/show_bug.cgi?id=37512. Can you report what
> version of clang you were using and if earlier versions of clang have
> this issue?
> Thanks,
Can you give some advices on how to implement
"attribute((no_stack_protector))" for Clang?
I looked on how this is done for GCC...
[ include/linux/compiler-gcc.h ]
#if GCC_VERSION >= 40400
#define __optimize(level) __attribute__((__optimize__(level)))
#define __nostackprotector __optimize("no-stack-protector")
#endif /* GCC_VERSION >= 40400 */
[ include/linux/compiler_types.h ]
#ifndef __nostackprotector
# define __nostackprotector
#endif
LLVM-bug #37512 says:
"GCC option for this is: attribute((optimize("no-stack-protector")))
and the equivalent clang syntax would be: attribute((no_stack_protector))"
So, there should be a define in "include/linux/compiler-clang.h" for this?
[ include/linux/compiler-clang.h ]
#define __nostackprotector attribute((no_stack_protector))
Looks that good to you?
- Sedat -
On Tue, May 22, 2018 at 9:39 AM, Sedat Dilek <[email protected]> wrote:
> On Sat, May 19, 2018 at 12:54 AM, Nick Desaulniers
> <[email protected]> wrote:
>> Sedat,
>> Thanks for the report. We have a fix ready in
>> https://bugs.llvm.org/show_bug.cgi?id=37512. Can you report what
>> version of clang you were using and if earlier versions of clang have
>> this issue?
>> Thanks,
>
> Can you give some advices on how to implement
> "attribute((no_stack_protector))" for Clang?
>
> I looked on how this is done for GCC...
>
> [ include/linux/compiler-gcc.h ]
>
> #if GCC_VERSION >= 40400
> #define __optimize(level) __attribute__((__optimize__(level)))
> #define __nostackprotector __optimize("no-stack-protector")
> #endif /* GCC_VERSION >= 40400 */
>
> [ include/linux/compiler_types.h ]
>
> #ifndef __nostackprotector
> # define __nostackprotector
> #endif
>
> LLVM-bug #37512 says:
>
> "GCC option for this is: attribute((optimize("no-stack-protector")))
> and the equivalent clang syntax would be: attribute((no_stack_protector))"
>
> So, there should be a define in "include/linux/compiler-clang.h" for this?
>
> [ include/linux/compiler-clang.h ]
>
> #define __nostackprotector attribute((no_stack_protector))
>
> Looks that good to you?
>
I am trying with the attached files and...
CONFIG_PARAVIRT=y
CONFIG_CC_STACKPROTECTOR_STRONG=y
- Sedat -
On Tue, May 22, 2018 at 10:04 AM, Sedat Dilek <[email protected]> wrote:
> On Tue, May 22, 2018 at 9:39 AM, Sedat Dilek <[email protected]> wrote:
>> On Sat, May 19, 2018 at 12:54 AM, Nick Desaulniers
>> <[email protected]> wrote:
>>> Sedat,
>>> Thanks for the report. We have a fix ready in
>>> https://bugs.llvm.org/show_bug.cgi?id=37512. Can you report what
>>> version of clang you were using and if earlier versions of clang have
>>> this issue?
>>> Thanks,
>>
>> Can you give some advices on how to implement
>> "attribute((no_stack_protector))" for Clang?
>>
>> I looked on how this is done for GCC...
>>
>> [ include/linux/compiler-gcc.h ]
>>
>> #if GCC_VERSION >= 40400
>> #define __optimize(level) __attribute__((__optimize__(level)))
>> #define __nostackprotector __optimize("no-stack-protector")
>> #endif /* GCC_VERSION >= 40400 */
>>
>> [ include/linux/compiler_types.h ]
>>
>> #ifndef __nostackprotector
>> # define __nostackprotector
>> #endif
>>
>> LLVM-bug #37512 says:
>>
>> "GCC option for this is: attribute((optimize("no-stack-protector")))
>> and the equivalent clang syntax would be: attribute((no_stack_protector))"
>>
>> So, there should be a define in "include/linux/compiler-clang.h" for this?
>>
>> [ include/linux/compiler-clang.h ]
>>
>> #define __nostackprotector attribute((no_stack_protector))
>>
>> Looks that good to you?
>>
>
> I am trying with the attached files and...
>
> CONFIG_PARAVIRT=y
> CONFIG_CC_STACKPROTECTOR_STRONG=y
>
Hmmm, I get these errors in my build-log...
mycompiler -Wp,-MD,arch/x86/kernel/.asm-offsets.s.d -nostdinc
-isystem /usr/lib/llvm-7/lib/clang/7.0.0/include -I./arch/x86/include
-I./arch/x86/include/generated -I./include -I./arch/x86/include/uapi
-I./arch/x86/include/generated/uapi -I./include/uapi
-I./include/generated/uapi -include ./include/linux/kconfig.h
-D__KERNEL__ -Qunused-arguments -Wall -Wundef -Wstrict-prototypes
-Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wchar
-Werror-implicit-function-declaration -Wno-format-security -std=gnu89
-fno-PIE -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -m64
-mno-80387 -mstack-alignment=8 -mtune=generic -mno-red-zone
-mcmodel=kernel -funit-at-a-time -DCONFIG_AS_CFI=1
-DCONFIG_AS_CFI_SIGNAL_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1
-DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_SSSE3=1 -DCONFIG_AS_CRC32=1
-DCONFIG_AS_AVX=1 -DCONFIG_AS_AVX2=1 -DCONFIG_AS_AVX512=1
-DCONFIG_AS_SHA1_NI=1 -DCONFIG_AS_SHA256_NI=1 -pipe -Wno-sign-compare
-fno-asynchronous-unwind-tables -mretpoline-external-thunk -DRETPOLINE
-O2 -Wframe-larger-than=2048 -fstack-protector-strong
-Wno-unused-variable -Wno-format-invalid-specifier -Wno-gnu
-Wno-address-of-packed-member -Wno-tautological-compare
-mno-global-merge -no-integrated-as -g -pg -mfentry -DCC_USING_FENTRY
-Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow
-fno-merge-all-constants -fno-stack-check -Werror=implicit-int
-Werror=strict-prototypes -Werror=date-time
-Werror=incompatible-pointer-types -Wno-initializer-overrides
-Wno-unused-value -Wno-format -Wno-sign-compare
-Wno-format-zero-length -Wno-uninitialized
-DKBUILD_BASENAME='"asm_offsets"' -DKBUILD_MODNAME='"asm_offsets"'
-fverbose-asm -S -o arch/x86/kernel/asm-offsets.s
arch/x86/kernel/asm-offsets.c
In file included from arch/x86/kernel/asm-offsets.c:9:
In file included from ./include/linux/crypto.h:24:
In file included from ./include/linux/slab.h:15:
In file included from ./include/linux/gfp.h:6:
In file included from ./include/linux/mmzone.h:8:
In file included from ./include/linux/spinlock.h:51:
In file included from ./include/linux/preempt.h:81:
In file included from ./arch/x86/include/asm/preempt.h:7:
In file included from ./include/linux/thread_info.h:38:
In file included from ./arch/x86/include/asm/thread_info.h:53:
In file included from ./arch/x86/include/asm/cpufeature.h:5:
In file included from ./arch/x86/include/asm/processor.h:33:
In file included from ./include/linux/irqflags.h:16:
./arch/x86/include/asm/irqflags.h:16:29: warning: declaration
specifier missing, defaulting to 'int'
static inline unsigned long __nostackprotector native_save_fl(void)
^
./include/linux/compiler-clang.h:28:39: note: expanded from macro
'__nostackprotector'
#define __nostackprotector attribute((no_stack_protector))
^
In file included from arch/x86/kernel/asm-offsets.c:9:
In file included from ./include/linux/crypto.h:24:
In file included from ./include/linux/slab.h:15:
In file included from ./include/linux/gfp.h:6:
In file included from ./include/linux/mmzone.h:8:
In file included from ./include/linux/spinlock.h:51:
In file included from ./include/linux/preempt.h:81:
In file included from ./arch/x86/include/asm/preempt.h:7:
In file included from ./include/linux/thread_info.h:38:
In file included from ./arch/x86/include/asm/thread_info.h:53:
In file included from ./arch/x86/include/asm/cpufeature.h:5:
In file included from ./arch/x86/include/asm/processor.h:33:
In file included from ./include/linux/irqflags.h:16:
./arch/x86/include/asm/irqflags.h:16:48: error: expected function body
after function declarator
static inline unsigned long __nostackprotector native_save_fl(void)
^
In file included from arch/x86/kernel/asm-offsets.c:9:
In file included from ./include/linux/crypto.h:24:
In file included from ./include/linux/slab.h:15:
In file included from ./include/linux/gfp.h:6:
In file included from ./include/linux/mmzone.h:8:
In file included from ./include/linux/spinlock.h:51:
In file included from ./include/linux/preempt.h:81:
In file included from ./arch/x86/include/asm/preempt.h:7:
In file included from ./include/linux/thread_info.h:38:
In file included from ./arch/x86/include/asm/thread_info.h:53:
In file included from ./arch/x86/include/asm/cpufeature.h:5:
./arch/x86/include/asm/processor.h:76:37: error: use of undeclared
identifier 'NR_INFO'
extern u16 __read_mostly tlb_lli_4k[NR_INFO];
^
./arch/x86/include/asm/processor.h:77:37: error: use of undeclared
identifier 'NR_INFO'
extern u16 __read_mostly tlb_lli_2m[NR_INFO];
^
./arch/x86/include/asm/processor.h:78:37: error: use of undeclared
identifier 'NR_INFO'
extern u16 __read_mostly tlb_lli_4m[NR_INFO];
^
./arch/x86/include/asm/processor.h:79:37: error: use of undeclared
identifier 'NR_INFO'
extern u16 __read_mostly tlb_lld_4k[NR_INFO];
^
./arch/x86/include/asm/processor.h:80:37: error: use of undeclared
identifier 'NR_INFO'
extern u16 __read_mostly tlb_lld_2m[NR_INFO];
^
./arch/x86/include/asm/processor.h:81:37: error: use of undeclared
identifier 'NR_INFO'
extern u16 __read_mostly tlb_lld_4m[NR_INFO];
^
./arch/x86/include/asm/processor.h:82:37: error: use of undeclared
identifier 'NR_INFO'
extern u16 __read_mostly tlb_lld_1g[NR_INFO];
^
1 warning and 8 errors generated.
Kbuild:57: recipe for target 'arch/x86/kernel/asm-offsets.s' failed
make[3]: *** [arch/x86/kernel/asm-offsets.s] Error 1
Makefile:1085: recipe for target 'prepare0' failed
make[2]: *** [prepare0] Error 2
scripts/package/Makefile:91: recipe for target 'bindeb-pkg' failed
make[1]: *** [bindeb-pkg] Error 2
Makefile:1367: recipe for target 'bindeb-pkg' failed
make: *** [bindeb-pkg] Error 2
- Sedat -
Hi,
I looked again into the llvm-bug #37512 and mentioned patches, so I
give this a 2nd try (see attached patchset).
Regards,
- Sedat -
Hi,
this looks good with attached patchset...
$ egrep -i 'paravirt|stackprotector'
/boot/config-4.14.42-1-iniza-llvmlinux | grep ^CONFIG_
CONFIG_HAVE_CC_STACKPROTECTOR=y
CONFIG_CC_STACKPROTECTOR=y
CONFIG_CC_STACKPROTECTOR_STRONG=y
CONFIG_PARAVIRT=y
CONFIG_PARAVIRT_SPINLOCKS=y
CONFIG_PARAVIRT_CLOCK=y
$ egrep -i paravirt qemu-log.txt
[ 0.000000] Booting paravirtualized kernel on KVM
sdi@iniza:~/src/linux-kernel/important-files$ cat scripts/run_qemu.sh
KPATH=$(pwd)
sudo qemu-system-x86_64 -enable-kvm -M pc -kernel $KPATH/bzImage
-initrd $KPATH/initrd.img -m 512 -net none -serial stdio -append
"root=/dev/ram0 console=ttyS0 hung_task_panic=1
earlyprintk=ttyS0,115200"
sdi@iniza:~/src/linux-kernel/important-files$ ./scripts/run_qemu.sh
Probing EDD (edd=off to disable)... ok
[ 0.000000] Linux version 4.14.42-1-iniza-llvmlinux
([email protected]@iniza) (clang version 7.0.0- (trunk)) #1 SMP
Tue May 22 13:30:34 CEST 2018
[ 0.000000] Command line: root=/dev/ram0 console=ttyS0
hung_task_panic=1 earlyprintk=ttyS0,115200
[ 0.000000] x86/fpu: x87 FPU will use FXSAVE
[ 0.000000] e820: BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000001ffdffff] usable
[ 0.000000] BIOS-e820: [mem 0x000000001ffe0000-0x000000001fffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[ 0.000000] bootconsole [earlyser0] enabled
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 2.8 present.
[ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
1.11.1-1 04/01/2014
[ 0.000000] Hypervisor detected: KVM
[ 0.000000] tsc: Fast TSC calibration failed
[ 0.000000] tsc: Using PIT calibration value
[ 0.000000] e820: last_pfn = 0x1ffe0 max_arch_pfn = 0x400000000
[ 0.000000] x86/PAT: PAT not supported by CPU.
[ 0.000000] x86/PAT: Configuration [0-7]: WB WT UC- UC WB WT UC- UC
Memory KASLR using RDTSC...
[ 0.000000] found SMP MP-table at [mem 0x000f5d60-0x000f5d6f]
mapped at [ffffffffff200d60]
[ 0.000000] RAMDISK: [mem 0x1e61e000-0x1ffdffff]
[ 0.000000] ACPI: Early table checksum verification disabled
[ 0.000000] ACPI: RSDP 0x00000000000F5B90 000014 (v00 BOCHS )
[ 0.000000] ACPI: RSDT 0x000000001FFE157C 000030 (v01 BOCHS
BXPCRSDT 00000001 BXPC 00000001)
[ 0.000000] ACPI: FACP 0x000000001FFE1458 000074 (v01 BOCHS
BXPCFACP 00000001 BXPC 00000001)
[ 0.000000] ACPI: DSDT 0x000000001FFE0040 001418 (v01 BOCHS
BXPCDSDT 00000001 BXPC 00000001)
[ 0.000000] ACPI: FACS 0x000000001FFE0000 000040
[ 0.000000] ACPI: APIC 0x000000001FFE14CC 000078 (v01 BOCHS
BXPCAPIC 00000001 BXPC 00000001)
[ 0.000000] ACPI: HPET 0x000000001FFE1544 000038 (v01 BOCHS
BXPCHPET 00000001 BXPC 00000001)
[ 0.000000] No NUMA configuration found
[ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000001ffdffff]
[ 0.000000] NODE_DATA(0) allocated [mem 0x1e619000-0x1e61dfff]
[ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[ 0.000000] kvm-clock: cpu 0, msr 0:1e611001, primary cpu clock
[ 0.000000] kvm-clock: using sched offset of 1101155712 cycles
[ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff
max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.000000] DMA32 [mem 0x0000000001000000-0x000000001ffdffff]
[ 0.000000] Normal empty
[ 0.000000] Device empty
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.000000] node 0: [mem 0x0000000000100000-0x000000001ffdffff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x000000001ffdffff]
[ 0.000000] ACPI: PM-Timer IO Port: 0x608
[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[ 0.000000] Using ACPI (MADT) for SMP configuration information
[ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000
[ 0.000000] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
[ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]
[ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
[ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff]
[ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff]
[ 0.000000] e820: [mem 0x20000000-0xfeffbfff] available for PCI devices
[ 0.000000] Booting paravirtualized kernel on KVM
[ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff
max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[ 0.000000] random: get_random_bytes called from
start_kernel+0x87/0x590 with crng_init=0
[ 0.000000] setup_percpu: NR_CPUS:512 nr_cpumask_bits:512
nr_cpu_ids:1 nr_node_ids:1
[ 0.000000] percpu: Embedded 44 pages/cpu @ffff9d8c9e400000 s141464
r8192 d30568 u2097152
[ 0.000000] KVM setup async PF for cpu 0
[ 0.000000] kvm-stealtime: cpu 0, msr 1e416180
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 128872
[ 0.000000] Policy zone: DMA32
[ 0.000000] Kernel command line: root=/dev/ram0 console=ttyS0
hung_task_panic=1 earlyprintk=ttyS0,115200
[ 0.000000] PID hash table entries: 2048 (order: 2, 16384 bytes)
[ 0.000000] Memory: 466892K/523768K available (10252K kernel code,
1013K rwdata, 2796K rodata, 1676K init, 656K bss, 56876K reserved, 0K
cma-reserved)
[ 0.000000] Kernel/User page tables isolation: enabled
[ 0.000000] ftrace: allocating 26207 entries in 103 pages
[ 0.004000] Hierarchical RCU implementation.
[ 0.004000] RCU restricting CPUs from NR_CPUS=512 to nr_cpu_ids=1.
[ 0.004000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[ 0.004000] NR_IRQS: 33024, nr_irqs: 256, preallocated irqs: 16
[ 0.004000] Console: colour VGA+ 80x25
[ 0.004000] console [ttyS0] enabled
[ 0.004000] console [ttyS0] enabled
[ 0.004000] bootconsole [earlyser0] disabled
[ 0.004000] bootconsole [earlyser0] disabled
[ 0.004000] clocksource: hpet: mask: 0xffffffff max_cycles:
0xffffffff, max_idle_ns: 19112604467 ns
[ 0.004007] tsc: Detected 2904.000 MHz processor
[ 0.004655] Calibrating delay loop (skipped) preset value.. 5808.00
BogoMIPS (lpj=11616000)
[ 0.005174] pid_max: default: 32768 minimum: 301
[ 0.008032] ACPI: Core revision 20170728
[ 0.012383] ACPI: 1 ACPI AML tables successfully acquired and loaded
[ 0.016049] Security Framework initialized
[ 0.016764] Yama: becoming mindful.
[ 0.020019] AppArmor: AppArmor initialized
[ 0.020855] Dentry cache hash table entries: 65536 (order: 7, 524288 bytes)
[ 0.022036] Inode-cache hash table entries: 32768 (order: 6, 262144 bytes)
[ 0.024026] Mount-cache hash table entries: 1024 (order: 1, 8192 bytes)
[ 0.028013] Mountpoint-cache hash table entries: 1024 (order: 1, 8192 bytes)
[ 0.032169] CPU: Physical Processor ID: 0
[ 0.032936] mce: CPU supports 10 MCE banks
[ 0.033720] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
[ 0.034688] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
[ 0.036007] Spectre V2 : Mitigation: Full generic retpoline
[ 0.050620] Freeing SMP alternatives memory: 32K
[ 0.057607] smpboot: Max logical packages: 1
[ 0.058397] x2apic enabled
[ 0.059003] Switched APIC routing to physical x2apic.
[ 0.060790] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.064000] APIC calibration not consistent with PM-Timer: 112ms
instead of 100ms
[ 0.064000] APIC delta adjusted to PM-Timer: 6250337 (7058743)
[ 0.064000] smpboot: CPU0: Intel QEMU Virtual CPU version 2.5+
(family: 0x6, model: 0x6, stepping: 0x3)
[ 0.064000] Performance Events: PMU not available due to
virtualization, using software events only.
[ 0.064000] Hierarchical SRCU implementation.
[ 0.064000] NMI watchdog: Perf event create on CPU 0 failed with -2
[ 0.064004] NMI watchdog: Perf NMI watchdog permanently disabled
[ 0.065137] smp: Bringing up secondary CPUs ...
[ 0.065759] smp: Brought up 1 node, 1 CPU
[ 0.066306] smpboot: Total of 1 processors activated (5808.00 BogoMIPS)
[ 0.067534] devtmpfs: initialized
[ 0.068060] x86/mm: Memory block size: 128MB
[ 0.068875] clocksource: jiffies: mask: 0xffffffff max_cycles:
0xffffffff, max_idle_ns: 7645041785100000 ns
[ 0.070202] futex hash table entries: 256 (order: 2, 16384 bytes)
[ 0.071068] pinctrl core: initialized pinctrl subsystem
[ 0.072117] NET: Registered protocol family 16
[ 0.072891] cpuidle: using governor ladder
[ 0.073462] cpuidle: using governor menu
[ 0.074014] ACPI: bus type PCI registered
[ 0.074578] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[ 0.076132] PCI: Using configuration type 1 for base access
[ 0.077715] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[ 0.078843] ACPI: Added _OSI(Module Device)
[ 0.080006] ACPI: Added _OSI(Processor Device)
[ 0.080612] ACPI: Added _OSI(3.0 _SCP Extensions)
[ 0.081256] ACPI: Added _OSI(Processor Aggregator Device)
[ 0.084210] ACPI: Interpreter enabled
[ 0.084770] ACPI: (supports S0 S3 S4 S5)
[ 0.085304] ACPI: Using IOAPIC for interrupt routing
[ 0.085985] PCI: Using host bridge windows from ACPI; if necessary,
use "pci=nocrs" and report a bug
[ 0.087414] ACPI: Enabled 2 GPEs in block 00 to 0F
[ 0.090989] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 0.092009] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI]
[ 0.092991] acpi PNP0A03:00: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 0.093917] acpi PNP0A03:00: fail to add MMCONFIG information,
can't access extended PCI configuration space under this bridge.
[ 0.096384] acpiphp: Slot [3] registered
[ 0.097002] acpiphp: Slot [4] registered
[ 0.097624] acpiphp: Slot [5] registered
[ 0.098230] acpiphp: Slot [6] registered
[ 0.098874] acpiphp: Slot [7] registered
[ 0.099467] acpiphp: Slot [8] registered
[ 0.100045] acpiphp: Slot [9] registered
[ 0.100636] acpiphp: Slot [10] registered
[ 0.101250] acpiphp: Slot [11] registered
[ 0.101856] acpiphp: Slot [12] registered
[ 0.102460] acpiphp: Slot [13] registered
[ 0.103067] acpiphp: Slot [14] registered
[ 0.103672] acpiphp: Slot [15] registered
[ 0.104047] acpiphp: Slot [16] registered
[ 0.104654] acpiphp: Slot [17] registered
[ 0.106525] acpiphp: Slot [18] registered
[ 0.107145] acpiphp: Slot [19] registered
[ 0.108079] acpiphp: Slot [20] registered
[ 0.108698] acpiphp: Slot [21] registered
[ 0.109321] acpiphp: Slot [22] registered
[ 0.109930] acpiphp: Slot [23] registered
[ 0.110538] acpiphp: Slot [24] registered
[ 0.111147] acpiphp: Slot [25] registered
[ 0.111756] acpiphp: Slot [26] registered
[ 0.112059] acpiphp: Slot [27] registered
[ 0.112665] acpiphp: Slot [28] registered
[ 0.113273] acpiphp: Slot [29] registered
[ 0.113968] acpiphp: Slot [30] registered
[ 0.116056] acpiphp: Slot [31] registered
[ 0.116635] PCI host bridge to bus 0000:00
[ 0.117204] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]
[ 0.118159] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]
[ 0.119105] pci_bus 0000:00: root bus resource [mem
0x000a0000-0x000bffff window]
[ 0.120006] pci_bus 0000:00: root bus resource [mem
0x20000000-0xfebfffff window]
[ 0.121065] pci_bus 0000:00: root bus resource [mem
0x100000000-0x17fffffff window]
[ 0.122135] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.132032] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7]
[ 0.133111] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6]
[ 0.134072] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177]
[ 0.135020] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376]
[ 0.136676] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by
PIIX4 ACPI
[ 0.137665] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB
[ 0.148846] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
[ 0.149800] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
[ 0.150963] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
[ 0.152114] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
[ 0.152958] ACPI: PCI Interrupt Link [LNKS] (IRQs *9)
[ 0.153844] pci 0000:00:02.0: vgaarb: setting as boot VGA device
[ 0.154655] pci 0000:00:02.0: vgaarb: VGA device added:
decodes=io+mem,owns=io+mem,locks=none
[ 0.156005] pci 0000:00:02.0: vgaarb: bridge control possible
[ 0.156793] vgaarb: loaded
[ 0.157252] EDAC MC: Ver: 3.0.0
[ 0.157892] PCI: Using ACPI for IRQ routing
[ 0.158746] HPET: 3 timers in total, 0 timers will be used for per-cpu timer
[ 0.160030] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[ 0.160895] hpet0: 3 comparators, 64-bit 100.000000 MHz counter
[ 0.165067] clocksource: Switched to clocksource kvm-clock
[ 0.177496] VFS: Disk quotas dquot_6.6.0
[ 0.178085] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.179252] AppArmor: AppArmor Filesystem Enabled
[ 0.179948] pnp: PnP ACPI init
[ 0.180823] pnp: PnP ACPI: found 6 devices
[ 0.190306] clocksource: acpi_pm: mask: 0xffffff max_cycles:
0xffffff, max_idle_ns: 2085701024 ns
[ 0.191608] NET: Registered protocol family 2
[ 0.192578] TCP established hash table entries: 4096 (order: 3, 32768 bytes)
[ 0.194899] TCP bind hash table entries: 4096 (order: 4, 65536 bytes)
[ 0.196897] TCP: Hash tables configured (established 4096 bind 4096)
[ 0.197702] UDP hash table entries: 256 (order: 1, 8192 bytes)
[ 0.198521] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
[ 0.199379] NET: Registered protocol family 1
[ 0.200251] pci 0000:00:01.0: PIIX3: Enabling Passive Release
[ 0.201009] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
[ 0.201837] pci 0000:00:01.0: Activating ISA DMA hang workarounds
[ 0.202717] pci 0000:00:02.0: Video device with shadowed ROM at
[mem 0x000c0000-0x000dffff]
[ 0.203894] Unpacking initramfs...
[ 0.825651] Freeing initrd memory: 26376K
[ 0.826479] audit: initializing netlink subsys (disabled)
[ 0.827462] audit: type=2000 audit(1526989397.915:1):
state=initialized audit_enabled=0 res=1
[ 0.828635] workingset: timestamp_bits=40 max_order=17 bucket_order=0
[ 0.829495] zbud: loaded
[ 1.099037] Key type asymmetric registered
[ 1.099664] Asymmetric key parser 'x509' registered
[ 1.100392] Block layer SCSI generic (bsg) driver version 0.4
loaded (major 249)
[ 1.101431] io scheduler noop registered
[ 1.101973] io scheduler deadline registered
[ 1.102576] io scheduler cfq registered (default)
[ 1.103238] io scheduler mq-deadline registered
[ 1.104073] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[ 1.162484] 00:05: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200)
is a 16550A
[ 1.164108] Linux agpgart interface v0.103
[ 1.164899] AMD IOMMUv2 driver by Joerg Roedel <[email protected]>
[ 1.165982] AMD IOMMUv2 functionality not available on this system
[ 1.172057] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU]
at 0x60,0x64 irq 1,12
[ 1.174222] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 1.175121] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 1.176174] mousedev: PS/2 mouse device common for all mice
[ 1.177581] input: AT Translated Set 2 keyboard as
/devices/platform/i8042/serio0/input/input0
[ 1.189269] rtc_cmos 00:00: RTC can wake from S4
[ 1.190220] rtc_cmos 00:00: rtc core: registered rtc_cmos as rtc0
[ 1.191162] rtc_cmos 00:00: alarms up to one day, y3k, 114 bytes
nvram, hpet irqs
[ 1.201696] ledtrig-cpu: registered to indicate activity on CPUs
[ 1.202608] NET: Registered protocol family 10
[ 1.206534] Segment Routing with IPv6
[ 1.216158] mip6: Mobile IPv6
[ 1.216587] NET: Registered protocol family 17
[ 1.217203] mpls_gso: MPLS GSO support
[ 1.217807] sched_clock: Marking stable (1216061212,
0)->(1674647863, -458586651)
[ 1.218936] registered taskstats version 1
[ 1.219496] zswap: loaded using pool lzo/zbud
[ 1.220164] AppArmor: AppArmor sha1 policy hashing enabled
[ 1.220881] ima: No TPM chip found, activating TPM-bypass! (rc=-19)
[ 1.221880] rtc_cmos 00:00: setting system clock to 2018-05-22
11:43:17 UTC (1526989397)
[ 1.225092] Freeing unused kernel memory: 1676K
[ 1.225791] Write protecting the kernel read-only data: 16384k
[ 1.228780] Freeing unused kernel memory: 2008K
[ 1.233379] Freeing unused kernel memory: 1300K
[ 1.253169] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[ 1.254182] x86/mm: Checking user space page tables
[ 1.274301] x86/mm: Checked W+X mappings: passed, no W+X pages found.
Loading, please wait...
starting version 238
[ 1.561067] piix4_smbus 0000:00:01.3: SMBus Host Controller at
0x700, revision 0
[ 1.567888] SCSI subsystem initialized
[ 1.573479] Floppy drive(s): fd0 is 2.88M AMI BIOS
[ 1.580046] input: VirtualPS/2 VMware VMMouse as
/devices/platform/i8042/serio1/input/input3
[ 1.581403] input: VirtualPS/2 VMware VMMouse as
/devices/platform/i8042/serio1/input/input2
[ 1.592500] FDC 0 is a S82078B
[ 1.614004] scsi host0: ata_piix
[ 1.621934] scsi host1: ata_piix
[ 1.622504] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc000 irq 14
[ 1.623672] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc008 irq 15
[ 1.784894] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100
[ 1.786110] ata2.00: configured for MWDMA2
[ 1.787173] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM
2.5+ PQ: 0 ANSI: 5
[ 1.808558] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray
[ 1.809460] cdrom: Uniform CD-ROM driver Revision: 3.20
[ 1.856058] tsc: Refined TSC clocksource calibration: 2903.992 MHz
[ 1.856904] clocksource: tsc: mask: 0xffffffffffffffff max_cycles:
0x29dbfe32135, max_idle_ns: 440795254644 ns
Begin: Loading essential drivers ... [ 2.040021] raid6: sse2x1
gen() 5419 MB/s
[ 2.108023] raid6: sse2x1 xor() 4326 MB/s
[ 2.176021] raid6: sse2x2 gen() 7214 MB/s
[ 2.244029] raid6: sse2x2 xor() 4726 MB/s
[ 2.312140] raid6: sse2x4 gen() 3950 MB/s
[ 2.388025] raid6: sse2x4 xor() 5319 MB/s
[ 2.388661] raid6: using algorithm sse2x2 gen() 7214 MB/s
[ 2.389446] raid6: .... xor() 4726 MB/s, rmw enabled
[ 2.390129] raid6: using intx1 recovery algorithm
[ 2.393920] xor: measuring software checksum speed
[ 2.432020] prefetch64-sse: 13658.000 MB/sec
[ 2.488016] generic_sse: 13120.000 MB/sec
[ 2.488642] xor: using function: prefetch64-sse (13658.000 MB/sec)
[ 2.503807] async_tx: api initialized (async)
done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top
... [ 2.535575] device-mapper: uevent: version 1.0.3
[ 2.538873] device-mapper: ioctl: 4.37.0-ioctl (2017-09-20)
initialised: [email protected]
[ 2.559469] random: lvm: uninitialized urandom read (4 bytes read)
WARNING: Failed to connect to lvmetad. Falling back to device scanning.
Reading all physical volumes. This may take a while...
*** REQUIRED: DM-CRYPT (LUKS PASSWORD) ***
Thanks to all involved people!
Best regards,
- Sedat -
For the sake of completeness...
[ CLANG VERSION ]
# dpkg -l | grep clang-7
ii clang-7
1:7~svn332830-1~exp1+0~20180521091322.1776~1.gbp198359 amd64 C,
C++ and Objective-C compiler
[ OBJDUMP native_save_fl() ]
$ objdump -d -S --start-address=0x$(grep native_save_fl System.map |
sed -e "s/ \+.*//") vmlinux | less
vmlinux: file format elf64-x86-64
Disassembly of section .text:
ffffffff810633a0 <native_save_fl>:
/*
* Interrupt control:
*/
static inline __nostackprotector unsigned long native_save_fl(void)
{
ffffffff810633a0: 55 push %rbp
ffffffff810633a1: 48 89 e5 mov %rsp,%rbp
ffffffff810633a4: 50 push %rax
/*
* "=rm" is safe here, because "pop" adjusts the stack before
* it evaluates its effective address -- this is part of the
* documented behavior of the "pop" instruction.
*/
asm volatile("# __raw_save_flags\n\t"
ffffffff810633a5: 9c pushfq
ffffffff810633a6: 8f 45 f8 popq -0x8(%rbp)
"pushf ; pop %0"
: "=rm" (flags)
: /* no input */
: "memory");
return flags;
ffffffff810633a9: 48 8b 45 f8 mov -0x8(%rbp),%rax
ffffffff810633ad: 48 83 c4 08 add $0x8,%rsp
ffffffff810633b1: 5d pop %rbp
ffffffff810633b2: c3 retq
ffffffff810633b3: 0f 1f 00 nopl (%rax)
ffffffff810633b6: 66 2e 0f 1f 84 00 00 nopw %cs:0x0(%rax,%rax,1)
ffffffff810633bd: 00 00 00
ffffffff810633c0 <native_restore_fl>:
}
static inline void native_restore_fl(unsigned long flags)
{
...
- Sedat -
Just for the records...
[ OBJDUMP irq_work_tick() ]
$ objdump -d -S --start-address=0x$(grep irq_work_tick System.map |
sed -e "s/ \+.*//") vmlinux | less
[ OBJDUMP native_save_fl() ]
$ objdump -d -S --start-address=0x$(grep native_save_fl System.map |
sed -e "s/ \+.*//") vmlinux | less
- sedat -