2022-07-13 08:11:38

by kernel test robot

[permalink] [raw]
Subject: [x86,static_call] ee88d363d1: WARNING:at_arch/x86/kernel/alternative.c:#apply_returns



Greeting,

FYI, we noticed the following commit (built with gcc-11):

commit: ee88d363d15617ff50ac24fab0ffec11113b2aeb ("x86,static_call: Use alternative RET encoding")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master

in testcase: boot

on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 16G

caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):


please be noted we noticed there is fix commit:
c27c753ea6fd1 "x86/static_call: Serialize __static_call_fixup() properly"

but the issue still exists on it, and also exists on latest mainline commit
when this bisection finished:
72a8e05d4f66b "Merge tag 'ovl-fixes-5.19-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs"


If you fix the issue, kindly add following tag
Reported-by: kernel test robot <[email protected]>


[ 22.065014][ T0] ------------[ cut here ]------------
[ 22.066738][ T0] WARNING: CPU: 0 PID: 0 at arch/x86/kernel/alternative.c:557 apply_returns (arch/x86/kernel/alternative.c:557 (discriminator 1))
[ 22.069534][ T0] Modules linked in:
[ 22.070738][ T0] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.19.0-rc4-00008-gee88d363d156 #1
[ 22.072739][ T0] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.0-debian-1.16.0-4 04/01/2014
[ 22.074741][ T0] RIP: 0010:apply_returns (arch/x86/kernel/alternative.c:557 (discriminator 1))
[ 22.076739][ T0] Code: ff ff 74 cb 48 83 c5 04 49 39 ee 0f 87 81 fe ff ff e9 22 ff ff ff 0f 0b 48 83 c5 04 49 39 ee 0f 87 6d fe ff ff e9 0e ff ff ff <0f> 0b 48 83 c5 04 49 39 ee 0f 87 59 fe ff ff e9 fa fe ff ff 48 89
All code
========
0: ff (bad)
1: ff 74 cb 48 pushq 0x48(%rbx,%rcx,8)
5: 83 c5 04 add $0x4,%ebp
8: 49 39 ee cmp %rbp,%r14
b: 0f 87 81 fe ff ff ja 0xfffffffffffffe92
11: e9 22 ff ff ff jmpq 0xffffffffffffff38
16: 0f 0b ud2
18: 48 83 c5 04 add $0x4,%rbp
1c: 49 39 ee cmp %rbp,%r14
1f: 0f 87 6d fe ff ff ja 0xfffffffffffffe92
25: e9 0e ff ff ff jmpq 0xffffffffffffff38
2a:* 0f 0b ud2 <-- trapping instruction
2c: 48 83 c5 04 add $0x4,%rbp
30: 49 39 ee cmp %rbp,%r14
33: 0f 87 59 fe ff ff ja 0xfffffffffffffe92
39: e9 fa fe ff ff jmpq 0xffffffffffffff38
3e: 48 rex.W
3f: 89 .byte 0x89

Code starting with the faulting instruction
===========================================
0: 0f 0b ud2
2: 48 83 c5 04 add $0x4,%rbp
6: 49 39 ee cmp %rbp,%r14
9: 0f 87 59 fe ff ff ja 0xfffffffffffffe68
f: e9 fa fe ff ff jmpq 0xffffffffffffff0e
14: 48 rex.W
15: 89 .byte 0x89
[ 22.078738][ T0] RSP: 0000:ffffffffa2807dc0 EFLAGS: 00010202
[ 22.080737][ T0] RAX: 0000000000000000 RBX: ffffffffa1b8fe05 RCX: 0000000000000000
[ 22.082546][ T0] RDX: 000000000000000f RSI: ffffffffa184a3e0 RDI: ffffffffa1b8fe05
[ 22.083738][ T0] RBP: ffffffffa42851e8 R08: 0000000000000001 R09: ffffffffa1b8fe05
[ 22.086491][ T0] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffffa1b8fe00
[ 22.087738][ T0] R13: dffffc0000000000 R14: ffffffffa4299890 R15: 1ffffffff4500fbb
[ 22.089739][ T0] FS: 0000000000000000(0000) GS:ffff888396600000(0000) knlGS:0000000000000000
[ 22.091743][ T0] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 22.093738][ T0] CR2: ffff88843ffff000 CR3: 00000003a562a000 CR4: 00000000000006f0
[ 22.095753][ T0] Call Trace:
[ 22.097742][ T0] <TASK>
[ 22.098765][ T0] ? rwlock_bug+0xc0/0xc0
[ 22.100230][ T0] ? apply_retpolines (arch/x86/kernel/alternative.c:538)
[ 22.101791][ T0] ? int3_exception_notify (arch/x86/kernel/alternative.c:849)
[ 22.103261][ T0] ? check_bugs (arch/x86/kernel/cpu/bugs.c:149)
[ 22.104751][ T0] alternative_instructions (arch/x86/kernel/alternative.c:932)
[ 22.106766][ T0] check_bugs (arch/x86/kernel/cpu/bugs.c:159)
[ 22.108244][ T0] start_kernel (init/main.c:1132)
[ 22.109747][ T0] secondary_startup_64_no_verify (arch/x86/kernel/head_64.S:358)
[ 22.111300][ T0] </TASK>
[ 22.112742][ T0] irq event stamp: 87769
[ 22.113741][ T0] hardirqs last enabled at (87781): __up_console_sem (arch/x86/include/asm/irqflags.h:45 (discriminator 1) arch/x86/include/asm/irqflags.h:80 (discriminator 1) arch/x86/include/asm/irqflags.h:138 (discriminator 1) kernel/printk/printk.c:264 (discriminator 1))
[ 22.115740][ T0] hardirqs last disabled at (87794): __up_console_sem (kernel/printk/printk.c:262 (discriminator 1))
[ 22.117739][ T0] softirqs last enabled at (2774): cgroup_idr_alloc+0x5b/0x1c0
[ 22.119739][ T0] softirqs last disabled at (2772): cgroup_idr_alloc+0x2e/0x1c0
[ 22.121741][ T0] ---[ end trace 0000000000000000 ]---
[ 22.141235][ T0] Freeing SMP alternatives memory: 44K
[ 22.144654][ T1] smpboot: CPU0: Intel Xeon E312xx (Sandy Bridge) (family: 0x6, model: 0x2a, stepping: 0x1)
[ 22.149556][ T1] cblist_init_generic: Setting adjustable number of callback queues.
[ 22.149743][ T1] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 22.152024][ T1] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 22.153900][ T1] Running RCU-tasks wait API self tests
[ 22.155660][ T1] Performance Events: unsupported p6 CPU model 42 no PMU driver, software events only.
[ 22.157632][ T1] rcu: Hierarchical SRCU implementation.
[ 22.163849][ T12] Callback from call_rcu_tasks_trace() invoked.
[ 22.167856][ T1] NMI watchdog: Perf NMI watchdog permanently disabled
[ 22.171321][ T1] smp: Bringing up secondary CPUs ...
[ 22.175938][ T1] x86: Booting SMP configuration:
[ 22.177337][ T1] .... node #0, CPUs: #1
[ 0.223939][ T0] masked ExtINT on CPU#1
[ 22.184004][ T1] smp: Brought up 1 node, 2 CPUs
[ 22.185764][ T1] smpboot: Max logical packages: 1
[ 22.187285][ T1] smpboot: Total of 2 processors activated (11200.39 BogoMIPS)
[ 22.270174][ T11] Callback from call_rcu_tasks_rude() invoked.
[ 22.418970][ T23] node 0 deferred pages initialised in 228ms
[ 22.874410][ T1] allocated 268435456 bytes of page_ext
[ 22.875753][ T1] Node 0, zone DMA: page owner found early allocated 0 pages
[ 22.882035][ T1] Node 0, zone DMA32: page owner found early allocated 0 pages
[ 22.943324][ T1] Node 0, zone Normal: page owner found early allocated 66889 pages
[ 22.947637][ T1] devtmpfs: initialized
[ 22.950416][ T1] x86/mm: Memory block size: 128MB
[ 23.029610][ T1] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns
[ 23.030846][ T1] futex hash table entries: 512 (order: 4, 65536 bytes, linear)
[ 23.034332][ T1] pinctrl core: initialized pinctrl subsystem
[ 23.059364][ T1] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[ 23.064611][ T1] audit: initializing netlink subsys (disabled)
[ 23.065940][ T28] audit: type=2000 audit(1657669959.786:1): state=initialized audit_enabled=0 res=1
[ 23.070463][ T1] thermal_sys: Registered thermal governor 'fair_share'
[ 23.070479][ T1] thermal_sys: Registered thermal governor 'bang_bang'
[ 23.071374][ T1] thermal_sys: Registered thermal governor 'step_wise'
[ 23.072313][ T1] thermal_sys: Registered thermal governor 'user_space'
[ 23.073704][ T1] cpuidle: using governor menu
[ 23.077082][ T1] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[ 23.080623][ T1] PCI: Using configuration type 1 for base access
[ 23.180007][ T1] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[ 23.231960][ T1] HugeTLB: can optimize 7 vmemmap pages for hugepages-2048kB
[ 23.233551][ T1] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[ 23.238663][ T1] cryptd: max_cpu_qlen set to 1000
[ 23.246101][ T1] ACPI: Added _OSI(Module Device)
[ 23.247247][ T1] ACPI: Added _OSI(Processor Device)
[ 23.248255][ T1] ACPI: Added _OSI(3.0 _SCP Extensions)
[ 23.249269][ T1] ACPI: Added _OSI(Processor Aggregator Device)
[ 23.250344][ T1] ACPI: Added _OSI(Linux-Dell-Video)
[ 23.251186][ T1] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
[ 23.252204][ T1] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)
[ 23.324111][ T1] ACPI: 1 ACPI AML tables successfully acquired and loaded
[ 23.363631][ T1] ACPI: Interpreter enabled
[ 23.364833][ T1] ACPI: PM: (supports S0 S3 S4 S5)
[ 23.366349][ T1] ACPI: Using IOAPIC for interrupt routing
[ 23.367868][ T1] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[ 23.369654][ T1] PCI: Using E820 reservations for host bridge windows
[ 23.376215][ T1] ACPI: Enabled 2 GPEs in block 00 to 0F
[ 23.518867][ T1] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 23.520463][ T1] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3]
[ 23.521566][ T1] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI]
[ 23.523541][ T1] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
[ 23.540637][ T1] acpiphp: Slot [3] registered
[ 23.541844][ T1] acpiphp: Slot [4] registered
[ 23.543895][ T1] acpiphp: Slot [5] registered
[ 23.545794][ T1] acpiphp: Slot [6] registered
[ 23.547391][ T1] acpiphp: Slot [7] registered
[ 23.548701][ T1] acpiphp: Slot [8] registered
[ 23.549700][ T1] acpiphp: Slot [9] registered
[ 23.550720][ T1] acpiphp: Slot [10] registered
[ 23.552497][ T1] acpiphp: Slot [11] registered
[ 23.553839][ T1] acpiphp: Slot [12] registered
[ 23.555713][ T1] acpiphp: Slot [13] registered
[ 23.556757][ T1] acpiphp: Slot [14] registered
[ 23.558331][ T1] acpiphp: Slot [15] registered
[ 23.559715][ T1] acpiphp: Slot [16] registered
[ 23.560726][ T1] acpiphp: Slot [17] registered
[ 23.562788][ T1] acpiphp: Slot [18] registered
[ 23.564347][ T1] acpiphp: Slot [19] registered
[ 23.565738][ T1] acpiphp: Slot [20] registered
[ 23.567756][ T1] acpiphp: Slot [21] registered
[ 23.569297][ T1] acpiphp: Slot [22] registered
[ 23.570773][ T1] acpiphp: Slot [23] registered
[ 23.572670][ T1] acpiphp: Slot [24] registered
[ 23.573714][ T1] acpiphp: Slot [25] registered
[ 23.574727][ T1] acpiphp: Slot [26] registered
[ 23.576872][ T1] acpiphp: Slot [27] registered
[ 23.578840][ T1] acpiphp: Slot [28] registered
[ 23.580841][ T1] acpiphp: Slot [29] registered
[ 23.582789][ T1] acpiphp: Slot [30] registered
[ 23.584923][ T1] acpiphp: Slot [31] registered
[ 23.586584][ T1] PCI host bridge to bus 0000:00


To reproduce:

# build kernel
cd linux
cp config-5.19.0-rc4-00008-gee88d363d156 .config
make HOSTCC=gcc-11 CC=gcc-11 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage modules
make HOSTCC=gcc-11 CC=gcc-11 ARCH=x86_64 INSTALL_MOD_PATH=<mod-install-dir> modules_install
cd <mod-install-dir>
find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz


git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email

# if come across any failure that blocks the test,
# please remove ~/.lkp and /lkp dir to run from a clean state.



--
0-DAY CI Kernel Test Service
https://01.org/lkp



Attachments:
(No filename) (11.72 kB)
config-5.19.0-rc4-00008-gee88d363d156 (170.98 kB)
job-script (4.63 kB)
dmesg.xz (14.70 kB)
Download all attachments

2022-07-13 12:50:06

by Peter Zijlstra

[permalink] [raw]
Subject: Re: [x86,static_call] ee88d363d1: WARNING:at_arch/x86/kernel/alternative.c:#apply_returns

On Wed, Jul 13, 2022 at 04:02:15PM +0800, kernel test robot wrote:

> [ 22.065014][ T0] ------------[ cut here ]------------
> [ 22.066738][ T0] WARNING: CPU: 0 PID: 0 at arch/x86/kernel/alternative.c:557 apply_returns (arch/x86/kernel/alternative.c:557 (discriminator 1))
> [ 22.069534][ T0] Modules linked in:
> [ 22.070738][ T0] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.19.0-rc4-00008-gee88d363d156 #1
> [ 22.072739][ T0] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.0-debian-1.16.0-4 04/01/2014
> [ 22.074741][ T0] RIP: 0010:apply_returns (arch/x86/kernel/alternative.c:557 (discriminator 1))
> [ 22.076739][ T0] Code: ff ff 74 cb 48 83 c5 04 49 39 ee 0f 87 81 fe ff ff e9 22 ff ff ff 0f 0b 48 83 c5 04 49 39 ee 0f 87 6d fe ff ff e9 0e ff ff ff <0f> 0b 48 83 c5 04 49 39 ee 0f 87 59 fe ff ff e9 fa fe ff ff 48 89
> All code
> ========
> 0: ff (bad)
> 1: ff 74 cb 48 pushq 0x48(%rbx,%rcx,8)
> 5: 83 c5 04 add $0x4,%ebp
> 8: 49 39 ee cmp %rbp,%r14
> b: 0f 87 81 fe ff ff ja 0xfffffffffffffe92
> 11: e9 22 ff ff ff jmpq 0xffffffffffffff38
> 16: 0f 0b ud2
> 18: 48 83 c5 04 add $0x4,%rbp
> 1c: 49 39 ee cmp %rbp,%r14
> 1f: 0f 87 6d fe ff ff ja 0xfffffffffffffe92
> 25: e9 0e ff ff ff jmpq 0xffffffffffffff38
> 2a:* 0f 0b ud2 <-- trapping instruction
> 2c: 48 83 c5 04 add $0x4,%rbp
> 30: 49 39 ee cmp %rbp,%r14
> 33: 0f 87 59 fe ff ff ja 0xfffffffffffffe92
> 39: e9 fa fe ff ff jmpq 0xffffffffffffff38
> 3e: 48 rex.W
> 3f: 89 .byte 0x89
>
> Code starting with the faulting instruction
> ===========================================
> 0: 0f 0b ud2
> 2: 48 83 c5 04 add $0x4,%rbp
> 6: 49 39 ee cmp %rbp,%r14
> 9: 0f 87 59 fe ff ff ja 0xfffffffffffffe68
> f: e9 fa fe ff ff jmpq 0xffffffffffffff0e
> 14: 48 rex.W
> 15: 89 .byte 0x89
> [ 22.078738][ T0] RSP: 0000:ffffffffa2807dc0 EFLAGS: 00010202
> [ 22.080737][ T0] RAX: 0000000000000000 RBX: ffffffffa1b8fe05 RCX: 0000000000000000
> [ 22.082546][ T0] RDX: 000000000000000f RSI: ffffffffa184a3e0 RDI: ffffffffa1b8fe05
> [ 22.083738][ T0] RBP: ffffffffa42851e8 R08: 0000000000000001 R09: ffffffffa1b8fe05
> [ 22.086491][ T0] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffffa1b8fe00
> [ 22.087738][ T0] R13: dffffc0000000000 R14: ffffffffa4299890 R15: 1ffffffff4500fbb
> [ 22.089739][ T0] FS: 0000000000000000(0000) GS:ffff888396600000(0000) knlGS:0000000000000000
> [ 22.091743][ T0] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 22.093738][ T0] CR2: ffff88843ffff000 CR3: 00000003a562a000 CR4: 00000000000006f0
> [ 22.095753][ T0] Call Trace:
> [ 22.097742][ T0] <TASK>
> [ 22.098765][ T0] ? rwlock_bug+0xc0/0xc0
> [ 22.100230][ T0] ? apply_retpolines (arch/x86/kernel/alternative.c:538)
> [ 22.101791][ T0] ? int3_exception_notify (arch/x86/kernel/alternative.c:849)
> [ 22.103261][ T0] ? check_bugs (arch/x86/kernel/cpu/bugs.c:149)
> [ 22.104751][ T0] alternative_instructions (arch/x86/kernel/alternative.c:932)
> [ 22.106766][ T0] check_bugs (arch/x86/kernel/cpu/bugs.c:159)
> [ 22.108244][ T0] start_kernel (init/main.c:1132)
> [ 22.109747][ T0] secondary_startup_64_no_verify (arch/x86/kernel/head_64.S:358)
> [ 22.111300][ T0] </TASK>
> [ 22.112742][ T0] irq event stamp: 87769
> [ 22.113741][ T0] hardirqs last enabled at (87781): __up_console_sem (arch/x86/include/asm/irqflags.h:45 (discriminator 1) arch/x86/include/asm/irqflags.h:80 (discriminator 1) arch/x86/include/asm/irqflags.h:138 (discriminator 1) kernel/printk/printk.c:264 (discriminator 1))
> [ 22.115740][ T0] hardirqs last disabled at (87794): __up_console_sem (kernel/printk/printk.c:262 (discriminator 1))
> [ 22.117739][ T0] softirqs last enabled at (2774): cgroup_idr_alloc+0x5b/0x1c0
> [ 22.119739][ T0] softirqs last disabled at (2772): cgroup_idr_alloc+0x2e/0x1c0
> [ 22.121741][ T0] ---[ end trace 0000000000000000 ]---


diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c
index d6858533e6e5..ff309e829192 100644
--- a/arch/x86/kernel/alternative.c
+++ b/arch/x86/kernel/alternative.c
@@ -555,8 +555,10 @@ void __init_or_module noinline apply_returns(s32 *start, s32 *end)
dest = addr + insn.length + insn.immediate.value;

if (__static_call_fixup(addr, op, dest) ||
- WARN_ON_ONCE(dest != &__x86_return_thunk))
+ WARN_ON_ONCE(dest != &__x86_return_thunk)) {
+ printk("XXX: %pS %pS : %*ph", addr, dest, 5, addr);
continue;
+ }

DPRINTK("return thunk at: %pS (%px) len: %d to: %pS",
addr, addr, insn.length,

Gets me:

[ 2.559091][ T0] ---[ end trace 0000000000000000 ]---
[ 2.561092][ T0] XXX: lkdtm_rodata_do_nothing+0x0/0x1240 lkdtm_rodata_do_nothing+0x5/0x1240 : e9 00 00 00 00

which is a result of:

drivers/misc/lkdtm/Makefile:OBJCOPYFLAGS_rodata_objcopy.o := \
drivers/misc/lkdtm/Makefile: --rename-section .noinstr.text=.rodata,alloc,readonly,load,contents

which makes that:

0000000000000000 <lkdtm_rodata_do_nothing>:
0: e9 00 00 00 00 jmp 5 <lkdtm_rodata_do_nothing+0x5> 1: R_X86_64_PLT32 __x86_return_thunk-0x4

remains unresolved.

Kees, what's up with that thing, this is 'weird' at best.

2022-07-13 18:55:18

by Kees Cook

[permalink] [raw]
Subject: Re: [x86,static_call] ee88d363d1: WARNING:at_arch/x86/kernel/alternative.c:#apply_returns

On Wed, Jul 13, 2022 at 02:28:55PM +0200, Peter Zijlstra wrote:
> On Wed, Jul 13, 2022 at 04:02:15PM +0800, kernel test robot wrote:
>
> > [ 22.065014][ T0] ------------[ cut here ]------------
> > [ 22.066738][ T0] WARNING: CPU: 0 PID: 0 at arch/x86/kernel/alternative.c:557 apply_returns (arch/x86/kernel/alternative.c:557 (discriminator 1))
> > [ 22.069534][ T0] Modules linked in:
> > [ 22.070738][ T0] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.19.0-rc4-00008-gee88d363d156 #1
> > [ 22.072739][ T0] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.0-debian-1.16.0-4 04/01/2014
> > [ 22.074741][ T0] RIP: 0010:apply_returns (arch/x86/kernel/alternative.c:557 (discriminator 1))
> > [ 22.076739][ T0] Code: ff ff 74 cb 48 83 c5 04 49 39 ee 0f 87 81 fe ff ff e9 22 ff ff ff 0f 0b 48 83 c5 04 49 39 ee 0f 87 6d fe ff ff e9 0e ff ff ff <0f> 0b 48 83 c5 04 49 39 ee 0f 87 59 fe ff ff e9 fa fe ff ff 48 89
> > All code
> > ========
> > 0: ff (bad)
> > 1: ff 74 cb 48 pushq 0x48(%rbx,%rcx,8)
> > 5: 83 c5 04 add $0x4,%ebp
> > 8: 49 39 ee cmp %rbp,%r14
> > b: 0f 87 81 fe ff ff ja 0xfffffffffffffe92
> > 11: e9 22 ff ff ff jmpq 0xffffffffffffff38
> > 16: 0f 0b ud2
> > 18: 48 83 c5 04 add $0x4,%rbp
> > 1c: 49 39 ee cmp %rbp,%r14
> > 1f: 0f 87 6d fe ff ff ja 0xfffffffffffffe92
> > 25: e9 0e ff ff ff jmpq 0xffffffffffffff38
> > 2a:* 0f 0b ud2 <-- trapping instruction
> > 2c: 48 83 c5 04 add $0x4,%rbp
> > 30: 49 39 ee cmp %rbp,%r14
> > 33: 0f 87 59 fe ff ff ja 0xfffffffffffffe92
> > 39: e9 fa fe ff ff jmpq 0xffffffffffffff38
> > 3e: 48 rex.W
> > 3f: 89 .byte 0x89
> >
> > Code starting with the faulting instruction
> > ===========================================
> > 0: 0f 0b ud2
> > 2: 48 83 c5 04 add $0x4,%rbp
> > 6: 49 39 ee cmp %rbp,%r14
> > 9: 0f 87 59 fe ff ff ja 0xfffffffffffffe68
> > f: e9 fa fe ff ff jmpq 0xffffffffffffff0e
> > 14: 48 rex.W
> > 15: 89 .byte 0x89
> > [ 22.078738][ T0] RSP: 0000:ffffffffa2807dc0 EFLAGS: 00010202
> > [ 22.080737][ T0] RAX: 0000000000000000 RBX: ffffffffa1b8fe05 RCX: 0000000000000000
> > [ 22.082546][ T0] RDX: 000000000000000f RSI: ffffffffa184a3e0 RDI: ffffffffa1b8fe05
> > [ 22.083738][ T0] RBP: ffffffffa42851e8 R08: 0000000000000001 R09: ffffffffa1b8fe05
> > [ 22.086491][ T0] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffffa1b8fe00
> > [ 22.087738][ T0] R13: dffffc0000000000 R14: ffffffffa4299890 R15: 1ffffffff4500fbb
> > [ 22.089739][ T0] FS: 0000000000000000(0000) GS:ffff888396600000(0000) knlGS:0000000000000000
> > [ 22.091743][ T0] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [ 22.093738][ T0] CR2: ffff88843ffff000 CR3: 00000003a562a000 CR4: 00000000000006f0
> > [ 22.095753][ T0] Call Trace:
> > [ 22.097742][ T0] <TASK>
> > [ 22.098765][ T0] ? rwlock_bug+0xc0/0xc0
> > [ 22.100230][ T0] ? apply_retpolines (arch/x86/kernel/alternative.c:538)
> > [ 22.101791][ T0] ? int3_exception_notify (arch/x86/kernel/alternative.c:849)
> > [ 22.103261][ T0] ? check_bugs (arch/x86/kernel/cpu/bugs.c:149)
> > [ 22.104751][ T0] alternative_instructions (arch/x86/kernel/alternative.c:932)
> > [ 22.106766][ T0] check_bugs (arch/x86/kernel/cpu/bugs.c:159)
> > [ 22.108244][ T0] start_kernel (init/main.c:1132)
> > [ 22.109747][ T0] secondary_startup_64_no_verify (arch/x86/kernel/head_64.S:358)
> > [ 22.111300][ T0] </TASK>
> > [ 22.112742][ T0] irq event stamp: 87769
> > [ 22.113741][ T0] hardirqs last enabled at (87781): __up_console_sem (arch/x86/include/asm/irqflags.h:45 (discriminator 1) arch/x86/include/asm/irqflags.h:80 (discriminator 1) arch/x86/include/asm/irqflags.h:138 (discriminator 1) kernel/printk/printk.c:264 (discriminator 1))
> > [ 22.115740][ T0] hardirqs last disabled at (87794): __up_console_sem (kernel/printk/printk.c:262 (discriminator 1))
> > [ 22.117739][ T0] softirqs last enabled at (2774): cgroup_idr_alloc+0x5b/0x1c0
> > [ 22.119739][ T0] softirqs last disabled at (2772): cgroup_idr_alloc+0x2e/0x1c0
> > [ 22.121741][ T0] ---[ end trace 0000000000000000 ]---
>
>
> diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c
> index d6858533e6e5..ff309e829192 100644
> --- a/arch/x86/kernel/alternative.c
> +++ b/arch/x86/kernel/alternative.c
> @@ -555,8 +555,10 @@ void __init_or_module noinline apply_returns(s32 *start, s32 *end)
> dest = addr + insn.length + insn.immediate.value;
>
> if (__static_call_fixup(addr, op, dest) ||
> - WARN_ON_ONCE(dest != &__x86_return_thunk))
> + WARN_ON_ONCE(dest != &__x86_return_thunk)) {
> + printk("XXX: %pS %pS : %*ph", addr, dest, 5, addr);
> continue;
> + }
>
> DPRINTK("return thunk at: %pS (%px) len: %d to: %pS",
> addr, addr, insn.length,
>
> Gets me:
>
> [ 2.559091][ T0] ---[ end trace 0000000000000000 ]---
> [ 2.561092][ T0] XXX: lkdtm_rodata_do_nothing+0x0/0x1240 lkdtm_rodata_do_nothing+0x5/0x1240 : e9 00 00 00 00
>
> which is a result of:
>
> drivers/misc/lkdtm/Makefile:OBJCOPYFLAGS_rodata_objcopy.o := \
> drivers/misc/lkdtm/Makefile: --rename-section .noinstr.text=.rodata,alloc,readonly,load,contents
>
> which makes that:
>
> 0000000000000000 <lkdtm_rodata_do_nothing>:
> 0: e9 00 00 00 00 jmp 5 <lkdtm_rodata_do_nothing+0x5> 1: R_X86_64_PLT32 __x86_return_thunk-0x4
>
> remains unresolved.
>
> Kees, what's up with that thing, this is 'weird' at best.

Whee. Yeah, this is a regression test for validating that the .data
section is not executable. It's designed to be arch-agnostic to avoid
needing to know how to return from a function call.

Is there some way for this to opt out of the thunk and leave it a bare
"ret"?

--
Kees Cook