STACK_FRAME_NON_STANDARD results in inconsistent relocation types
depending on .c or .S usage:
Relocation section '.rela.discard.func_stack_frame_non_standard' at offset 0x3c01090 contains 5 entries:
Offset Info Type Symbol's Value Symbol's Name + Addend
0000000000000000 00020c2200000002 R_X86_64_PC32 0000000000047b40 do_suspend_lowlevel + 0
0000000000000008 0002461e00000001 R_X86_64_64 00000000000480a0 machine_real_restart + 0
0000000000000010 0000001400000001 R_X86_64_64 0000000000000000 .rodata + b3d4
0000000000000018 0002444600000002 R_X86_64_PC32 00000000000678a0 __efi64_thunk + 0
0000000000000020 0002659d00000001 R_X86_64_64 0000000000113160 __crash_kexec + 0
Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
---
include/linux/objtool.h | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
--- a/include/linux/objtool.h
+++ b/include/linux/objtool.h
@@ -137,7 +137,11 @@ struct unwind_hint {
.macro STACK_FRAME_NON_STANDARD func:req
.pushsection .discard.func_stack_frame_non_standard, "aw"
- .long \func - .
+#ifdef CONFIG_64BIT
+ .quad \func
+#else
+ .long \func
+#endif
.popsection
.endm
On Fri, Apr 29, 2022 at 11:20:24AM +0200, Peter Zijlstra wrote:
>
> STACK_FRAME_NON_STANDARD results in inconsistent relocation types
> depending on .c or .S usage:
>
> Relocation section '.rela.discard.func_stack_frame_non_standard' at offset 0x3c01090 contains 5 entries:
> Offset Info Type Symbol's Value Symbol's Name + Addend
> 0000000000000000 00020c2200000002 R_X86_64_PC32 0000000000047b40 do_suspend_lowlevel + 0
> 0000000000000008 0002461e00000001 R_X86_64_64 00000000000480a0 machine_real_restart + 0
> 0000000000000010 0000001400000001 R_X86_64_64 0000000000000000 .rodata + b3d4
> 0000000000000018 0002444600000002 R_X86_64_PC32 00000000000678a0 __efi64_thunk + 0
> 0000000000000020 0002659d00000001 R_X86_64_64 0000000000113160 __crash_kexec + 0
So that weird .rodata entry is optprobe_template_func.
It being in .rodata also means it's not validated and there is no ORC
data generated, is that all intentional? The changelog for:
877b145f0f47 ("x86/kprobes: Move trampoline code into RODATA")
doesn't really say anything useful about any of that :/
I also don't see any kprobe/optprobe hooks in unwind.h, so what happens
if we hit an optprobe?
On Fri, Apr 29, 2022 at 02:00:44PM +0200, Peter Zijlstra wrote:
> On Fri, Apr 29, 2022 at 11:20:24AM +0200, Peter Zijlstra wrote:
> >
> > STACK_FRAME_NON_STANDARD results in inconsistent relocation types
> > depending on .c or .S usage:
> >
> > Relocation section '.rela.discard.func_stack_frame_non_standard' at offset 0x3c01090 contains 5 entries:
> > Offset Info Type Symbol's Value Symbol's Name + Addend
> > 0000000000000000 00020c2200000002 R_X86_64_PC32 0000000000047b40 do_suspend_lowlevel + 0
> > 0000000000000008 0002461e00000001 R_X86_64_64 00000000000480a0 machine_real_restart + 0
> > 0000000000000010 0000001400000001 R_X86_64_64 0000000000000000 .rodata + b3d4
> > 0000000000000018 0002444600000002 R_X86_64_PC32 00000000000678a0 __efi64_thunk + 0
> > 0000000000000020 0002659d00000001 R_X86_64_64 0000000000113160 __crash_kexec + 0
>
> So that weird .rodata entry is optprobe_template_func.
>
> It being in .rodata also means it's not validated and there is no ORC
> data generated, is that all intentional? The changelog for:
>
> 877b145f0f47 ("x86/kprobes: Move trampoline code into RODATA")
>
> doesn't really say anything useful about any of that :/
>
> I also don't see any kprobe/optprobe hooks in unwind.h, so what happens
> if we hit an optprobe?
Same as for any other generated code, the unwinder will try to fall back
to frame pointers, and if that doesn't work, the unwind stops.
That commit didn't change anything since it was already not being
directly executed anyway, but rather used to generate code on the fly.
And before that commit it was being ignored by ORC anyway, thanks to
STACK_FRAME_NON_STANDARD. Which can now be removed since this code is
now data and objtool will no longer try to understand it.
--
Josh
On Sat, Apr 30, 2022 at 01:44:00PM +0200, Peter Zijlstra wrote:
> > > I also don't see any kprobe/optprobe hooks in unwind.h, so what happens
> > > if we hit an optprobe?
> >
> > Same as for any other generated code, the unwinder will try to fall back
> > to frame pointers, and if that doesn't work, the unwind stops.
> >
> > That commit didn't change anything since it was already not being
> > directly executed anyway, but rather used to generate code on the fly.
> >
> > And before that commit it was being ignored by ORC anyway, thanks to
> > STACK_FRAME_NON_STANDARD. Which can now be removed since this code is
> > now data and objtool will no longer try to understand it.
>
> Right; but I suppose I'm wondering if we should fix this. It seems a
> rather sub-optimal state of affairs.
Masami recently fixed some kprobes ORC issues but I don't know if this
one was fixed.
As to the whether it's worth fixing, I dunno. There are trade offs.
Depends on how common the stack trace is -- I'm guessing not very, since
I've never seen a bug report -- and how important it is to get to full
ORC coverage. If our goal is full coverage, we'd need a way for
generated code to add/remove ORC entries.
--
Josh
On Fri, Apr 29, 2022 at 03:56:36PM -0700, Josh Poimboeuf wrote:
> On Fri, Apr 29, 2022 at 02:00:44PM +0200, Peter Zijlstra wrote:
> > On Fri, Apr 29, 2022 at 11:20:24AM +0200, Peter Zijlstra wrote:
> > >
> > > STACK_FRAME_NON_STANDARD results in inconsistent relocation types
> > > depending on .c or .S usage:
> > >
> > > Relocation section '.rela.discard.func_stack_frame_non_standard' at offset 0x3c01090 contains 5 entries:
> > > Offset Info Type Symbol's Value Symbol's Name + Addend
> > > 0000000000000000 00020c2200000002 R_X86_64_PC32 0000000000047b40 do_suspend_lowlevel + 0
> > > 0000000000000008 0002461e00000001 R_X86_64_64 00000000000480a0 machine_real_restart + 0
> > > 0000000000000010 0000001400000001 R_X86_64_64 0000000000000000 .rodata + b3d4
> > > 0000000000000018 0002444600000002 R_X86_64_PC32 00000000000678a0 __efi64_thunk + 0
> > > 0000000000000020 0002659d00000001 R_X86_64_64 0000000000113160 __crash_kexec + 0
> >
> > So that weird .rodata entry is optprobe_template_func.
> >
> > It being in .rodata also means it's not validated and there is no ORC
> > data generated, is that all intentional? The changelog for:
> >
> > 877b145f0f47 ("x86/kprobes: Move trampoline code into RODATA")
> >
> > doesn't really say anything useful about any of that :/
> >
> > I also don't see any kprobe/optprobe hooks in unwind.h, so what happens
> > if we hit an optprobe?
>
> Same as for any other generated code, the unwinder will try to fall back
> to frame pointers, and if that doesn't work, the unwind stops.
>
> That commit didn't change anything since it was already not being
> directly executed anyway, but rather used to generate code on the fly.
>
> And before that commit it was being ignored by ORC anyway, thanks to
> STACK_FRAME_NON_STANDARD. Which can now be removed since this code is
> now data and objtool will no longer try to understand it.
Right; but I suppose I'm wondering if we should fix this. It seems a
rather sub-optimal state of affairs.
On Fri, Apr 29, 2022 at 11:20:24AM +0200, Peter Zijlstra wrote:
>
> STACK_FRAME_NON_STANDARD results in inconsistent relocation types
> depending on .c or .S usage:
>
> Relocation section '.rela.discard.func_stack_frame_non_standard' at offset 0x3c01090 contains 5 entries:
> Offset Info Type Symbol's Value Symbol's Name + Addend
> 0000000000000000 00020c2200000002 R_X86_64_PC32 0000000000047b40 do_suspend_lowlevel + 0
> 0000000000000008 0002461e00000001 R_X86_64_64 00000000000480a0 machine_real_restart + 0
> 0000000000000010 0000001400000001 R_X86_64_64 0000000000000000 .rodata + b3d4
> 0000000000000018 0002444600000002 R_X86_64_PC32 00000000000678a0 __efi64_thunk + 0
> 0000000000000020 0002659d00000001 R_X86_64_64 0000000000113160 __crash_kexec + 0
>
> Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
> ---
> include/linux/objtool.h | 6 +++++-
> 1 file changed, 5 insertions(+), 1 deletion(-)
>
> --- a/include/linux/objtool.h
> +++ b/include/linux/objtool.h
> @@ -137,7 +137,11 @@ struct unwind_hint {
>
> .macro STACK_FRAME_NON_STANDARD func:req
> .pushsection .discard.func_stack_frame_non_standard, "aw"
> - .long \func - .
> +#ifdef CONFIG_64BIT
> + .quad \func
> +#else
> + .long \func
> +#endif
> .popsection
> .endm
Can use _ASM_PTR here, and objtool.h needs synced to tools.
--
Josh
On Mon, 2 May 2022 10:59:21 -0700
Josh Poimboeuf <[email protected]> wrote:
> On Sat, Apr 30, 2022 at 01:44:00PM +0200, Peter Zijlstra wrote:
> > > > I also don't see any kprobe/optprobe hooks in unwind.h, so what happens
> > > > if we hit an optprobe?
> > >
> > > Same as for any other generated code, the unwinder will try to fall back
> > > to frame pointers, and if that doesn't work, the unwind stops.
> > >
> > > That commit didn't change anything since it was already not being
> > > directly executed anyway, but rather used to generate code on the fly.
Ah, OK. So ORC will not work on the dynamically generated trampoline code.
Can we generate ORC information entry dynamically?
(E.g. copying ORC data from the original code)
> > >
> > > And before that commit it was being ignored by ORC anyway, thanks to
> > > STACK_FRAME_NON_STANDARD. Which can now be removed since this code is
> > > now data and objtool will no longer try to understand it.
> >
> > Right; but I suppose I'm wondering if we should fix this. It seems a
> > rather sub-optimal state of affairs.
>
> Masami recently fixed some kprobes ORC issues but I don't know if this
> one was fixed.
I've fixed the kretprobe ORC unwinder issue. I need to check the optprobe
case too.
>
> As to the whether it's worth fixing, I dunno. There are trade offs.
>
> Depends on how common the stack trace is -- I'm guessing not very, since
> I've never seen a bug report -- and how important it is to get to full
> ORC coverage. If our goal is full coverage, we'd need a way for
> generated code to add/remove ORC entries.
Agreed, if I can copy the ORC entries for the original code to the entries
for generated code, I can fix it.
Thank you,
>
> --
> Josh
>
--
Masami Hiramatsu <[email protected]>
On Fri, 29 Apr 2022 14:00:44 +0200
Peter Zijlstra <[email protected]> wrote:
> On Fri, Apr 29, 2022 at 11:20:24AM +0200, Peter Zijlstra wrote:
> >
> > STACK_FRAME_NON_STANDARD results in inconsistent relocation types
> > depending on .c or .S usage:
> >
> > Relocation section '.rela.discard.func_stack_frame_non_standard' at offset 0x3c01090 contains 5 entries:
> > Offset Info Type Symbol's Value Symbol's Name + Addend
> > 0000000000000000 00020c2200000002 R_X86_64_PC32 0000000000047b40 do_suspend_lowlevel + 0
> > 0000000000000008 0002461e00000001 R_X86_64_64 00000000000480a0 machine_real_restart + 0
> > 0000000000000010 0000001400000001 R_X86_64_64 0000000000000000 .rodata + b3d4
> > 0000000000000018 0002444600000002 R_X86_64_PC32 00000000000678a0 __efi64_thunk + 0
> > 0000000000000020 0002659d00000001 R_X86_64_64 0000000000113160 __crash_kexec + 0
>
> So that weird .rodata entry is optprobe_template_func.
>
> It being in .rodata also means it's not validated and there is no ORC
> data generated, is that all intentional? The changelog for:
>
> 877b145f0f47 ("x86/kprobes: Move trampoline code into RODATA")
>
> doesn't really say anything useful about any of that :/
This commit was introduced just for reducing attack surface (the
trampoline code is NOT executed but just copied into trampoline
buffers), but if the ORC unwinder doesn't work correctly, please
revert it.
I think there is no functional change.
Thanks,
--
Masami Hiramatsu <[email protected]>