Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <Ys7pLq+tQk5xEa/B@dev-arch.thelio-3990X> <CAKwvOd=kqfAZoywsOFbvvzUJD+zVgFgmDAAcb7h0U8LGzfG33A@mail.gmail.com>
 <Ys7vKYJ9tyih8Cng@dev-arch.thelio-3990X> <Ys7yBcfBRWXPLsLq@hirez.programming.kicks-ass.net>
 <CAKwvOdmcU3SxgMGsX4MRfR6mskYu5kPYNwUNuNLKeSeskMV5Qg@mail.gmail.com>
In-Reply-To: <CAKwvOdmcU3SxgMGsX4MRfR6mskYu5kPYNwUNuNLKeSeskMV5Qg@mail.gmail.com>
From:   Nick Desaulniers <ndesaulniers@google.com>
Date:   Wed, 13 Jul 2022 15:05:32 -0700
Message-ID: <CAKwvOdkfz9aYDb9vzdttVucJDYX+98=O58sLE+6aTmCORwX0YA@mail.gmail.com>
Subject: Re: objtool "'naked' return found in RETHUNK build" with clang + CONFIG_K{A,C}SAN=y
To:     Nathan Chancellor <nathan@kernel.org>
Cc:     Josh Poimboeuf <jpoimboe@kernel.org>, linux-kernel@vger.kernel.org,
        llvm@lists.linux.dev, Peter Zijlstra <peterz@infradead.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Wed, Jul 13, 2022 at 9:42 AM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> On Wed, Jul 13, 2022 at 9:25 AM Peter Zijlstra <peterz@infradead.org> wrote:
> >
> > On Wed, Jul 13, 2022 at 09:13:29AM -0700, Nathan Chancellor wrote:
> > > On Wed, Jul 13, 2022 at 09:01:58AM -0700, Nick Desaulniers wrote:
> > > > On Wed, Jul 13, 2022 at 8:48 AM Nathan Chancellor <nathan@kernel.org> wrote:
> > > > >
> > > > > Hi Josh and Peter,
> > > > >
> > > > > After commit 9bb2ec608a20 ("objtool: Update Retpoline validation") and
> > > > > commit f43b9876e857 ("x86/retbleed: Add fine grained Kconfig knobs"), I
> > > > > see a spew of objtool warnings when building certain configurations with
> > > > > clang. Examples below, they appear to trigger in every single file.
> > > > >
> > > > > With x86_64_defconfig + CONFIG_KASAN=y:
> > > > >
> > > > > arch/x86/ia32/audit.o: warning: objtool: asan.module_ctor+0x11: 'naked' return found in RETHUNK build
> > > > > arch/x86/ia32/audit.o: warning: objtool: asan.module_dtor+0x11: 'naked' return found in RETHUNK build
> > > > > init/version.o: warning: objtool: asan.module_ctor+0x11: 'naked' return found in RETHUNK build
> > > > > init/version.o: warning: objtool: asan.module_dtor+0x11: 'naked' return found in RETHUNK build
> > > > > arch/x86/kernel/cpu/rdrand.o: warning: objtool: asan.module_ctor+0x11: 'naked' return found in RETHUNK build
> > > > > arch/x86/kernel/cpu/rdrand.o: warning: objtool: asan.module_dtor+0x11: 'naked' return found in RETHUNK build
> > > > > arch/x86/kernel/fpu/bugs.o: warning: objtool: asan.module_ctor+0x11: 'naked' return found in RETHUNK build
> > > > > arch/x86/kernel/fpu/bugs.o: warning: objtool: asan.module_dtor+0x11: 'naked' return found in RETHUNK build
> > > > > arch/x86/kernel/platform-quirks.o: warning: objtool: asan.module_ctor+0x0: 'naked' return found in RETHUNK build
> > > > > init/calibrate.o: warning: objtool: asan.module_ctor+0x11: 'naked' return found in RETHUNK build
> > > > > init/calibrate.o: warning: objtool: asan.module_dtor+0x11: 'naked' return found in RETHUNK build
> > > > >
> > > > > With x86_64_defconfig + CONFIG_KCSAN=y:
> > > > >
> > > > > arch/x86/ia32/audit.o: warning: objtool: tsan.module_ctor+0x5: 'naked' return found in RETHUNK build
> > > > > init/calibrate.o: warning: objtool: tsan.module_ctor+0x5: 'naked' return found in RETHUNK build
> > > > > init/version.o: warning: objtool: tsan.module_ctor+0x5: 'naked' return found in RETHUNK build
> > > > > arch/x86/kernel/fpu/bugs.o: warning: objtool: tsan.module_ctor+0x5: 'naked' return found in RETHUNK build
> > > > > arch/x86/kernel/cpu/rdrand.o: warning: objtool: tsan.module_ctor+0x5: 'naked' return found in RETHUNK build
> > > > > arch/x86/events/probe.o: warning: objtool: tsan.module_ctor+0x5: 'naked' return found in RETHUNK build
> > > > > arch/x86/kernel/apic/ipi.o: warning: objtool: tsan.module_ctor+0x5: 'naked' return found in RETHUNK build

It's not just the sanitizers, `CONFIG_GCOV_KERNEL=y` +
`CONFIG_GCOV_PROFILE_ALL=y` also has this issue:
```
...
drivers/base/platform.o: warning: objtool: __llvm_gcov_reset+0xa94:
'naked' return found in RETHUNK build
net/core/netpoll.o: warning: objtool: __llvm_gcov_reset+0xde3: 'naked'
return found in RETHUNK build
net/netfilter/nf_conntrack_netlink.o: warning: objtool:
__llvm_gcov_reset+0x1b94: 'naked' return found in RETHUNK build
fs/fs_parser.o: warning: objtool: __llvm_gcov_reset+0x344: 'naked'
return found in RETHUNK build
drivers/acpi/debugfs.o: warning: objtool: __llvm_gcov_reset+0xb:
'naked' return found in RETHUNK build
mm/pagewalk.o: warning: objtool: __llvm_gcov_reset+0x77f: 'naked'
return found in RETHUNK build
drivers/tty/serial/8250/8250_pnp.o: warning: objtool:
__llvm_gcov_reset+0x231: 'naked' return found in RETHUNK build
fs/init.o: warning: objtool: __llvm_gcov_reset+0x2f7: 'naked' return
found in RETHUNK build
drivers/gpu/drm/drm_fourcc.o: warning: objtool:
__llvm_gcov_reset+0x1ce: 'naked' return found in RETHUNK build
...
```

> > > >
> > > > Thanks for the report. I wonder if this might be a compiler bug; it
> > > > seems like the {a|t}san.module_{c|d}tor functions are being emitted
> > > > with ret instructions?
> > > >
> > > > If you have one of these builds lying around still, can you provide:
> > > >
> > > > $ llvm-objdump -dr --disassemble-symbols=tsan.module_ctor
> > > > arch/x86/kernel/fpu/bugs.o
> > >
> > > Sure thing.
> > >
> > > With KASAN:
> > >
> > > $ llvm-objdump -r --disassemble-symbols=asan.module_ctor,asan.module_dtor build/arch/x86/kernel/fpu/bugs.o
> > >
> > > build/arch/x86/kernel/fpu/bugs.o:     file format elf64-x86-64
> > >
> > > Disassembly of section .text.asan.module_ctor:
> > >
> > > 0000000000000000 <asan.module_ctor>:
> > >        0: be 01 00 00 00                      movl    $1, %esi
> > >        5: 48 c7 c7 00 00 00 00                movq    $0, %rdi
> > >               0000000000000008:  R_X86_64_32S .data
> > >        c: e8 00 00 00 00                      callq   0x11 <asan.module_ctor+0x11>
> > >               000000000000000d:  R_X86_64_PLT32       __asan_register_globals-0x4
> > >       11: c3                                  retq
> >
> > As Nick said; this should be: 'jmp __x86_return_thunk', on those builds.
>
> https://github.com/llvm/llvm-project/issues/56514
>
> --
> Thanks,
> ~Nick Desaulniers


-- 
Thanks,
~Nick Desaulniers