2022-09-26 15:57:40

by Naresh Kamboju

[permalink] [raw]
Subject: arch/arm/probes/kprobes/core.c:409:30: error: .fnstart must precede .save or .vsave directives

Following build warnings / errors noticed on arm with clang-13 / 14
on Linux next-20220923.

Reported-by: Linux Kernel Functional Testing <[email protected]>

Regressions found on arm:

- build-clang-13-bcm2835_defconfig
- build-clang-nightly-imx_v6_v7_defconfig
- build-clang-nightly-orion5x_defconfig
- build-clang-13-keystone_defconfig
- build-clang-13-omap2plus_defconfig
- build-clang-14-imx_v6_v7_defconfig
- build-clang-nightly-omap2plus_defconfig
- build-clang-nightly-multi_v5_defconfig
- build-clang-nightly-bcm2835_defconfig
- build-clang-13-imx_v6_v7_defconfig
- build-clang-13-imx_v4_v5_defconfig
- build-clang-14-imx_v4_v5_defconfig
- build-clang-13-orion5x_defconfig
- build-clang-14-multi_v5_defconfig-65236a87
- build-clang-14-lkftconfig
- build-clang-nightly-imx_v4_v5_defconfig
- build-clang-13-multi_v5_defconfig
- build-clang-13-lkftconfig
- build-clang-nightly-keystone_defconfig
- build-clang-14-multi_v5_defconfig
- build-clang-14-orion5x_defconfig
- build-clang-14-omap2plus_defconfig
- build-clang-nightly-multi_v5_defconfig-65236a87
- build-clang-14-bcm2835_defconfig
- build-clang-14-keystone_defconfig
- build-clang-nightly-lkftconfig

arch/arm/probes/kprobes/core.c:409:30: error: .fnstart must precede
.save or .vsave directives
"stmdb sp, {sp, lr, pc} \n\t"
^
<inline asm>:3:2: note: instantiated into assembly here
.save {sp, lr, pc}
^
/builds/linux/arch/arm/probes/kprobes/core.c:412:29: error: .fnstart
must precede .pad directive
"stmdb sp!, {r0 - r11} \n\t"
^
<inline asm>:6:2: note: instantiated into assembly here
.pad #52
^
2 errors generated.
make[5]: *** [/builds/linux/scripts/Makefile.build:250:
arch/arm/probes/kprobes/core.o] Error 1

build log:
https://builds.tuxbuild.com/2FAyD1qcTlzjIYE7mjrugjCsxu1/

--
Linaro LKFT
https://lkft.linaro.org


2022-09-26 16:53:58

by Nathan Chancellor

[permalink] [raw]
Subject: Re: arch/arm/probes/kprobes/core.c:409:30: error: .fnstart must precede .save or .vsave directives

Hi Naresh,

On Mon, Sep 26, 2022 at 06:57:00PM +0530, Naresh Kamboju wrote:
> Following build warnings / errors noticed on arm with clang-13 / 14
> on Linux next-20220923.
>
> Reported-by: Linux Kernel Functional Testing <[email protected]>
>
> Regressions found on arm:
>
> - build-clang-13-bcm2835_defconfig
> - build-clang-nightly-imx_v6_v7_defconfig
> - build-clang-nightly-orion5x_defconfig
> - build-clang-13-keystone_defconfig
> - build-clang-13-omap2plus_defconfig
> - build-clang-14-imx_v6_v7_defconfig
> - build-clang-nightly-omap2plus_defconfig
> - build-clang-nightly-multi_v5_defconfig
> - build-clang-nightly-bcm2835_defconfig
> - build-clang-13-imx_v6_v7_defconfig
> - build-clang-13-imx_v4_v5_defconfig
> - build-clang-14-imx_v4_v5_defconfig
> - build-clang-13-orion5x_defconfig
> - build-clang-14-multi_v5_defconfig-65236a87
> - build-clang-14-lkftconfig
> - build-clang-nightly-imx_v4_v5_defconfig
> - build-clang-13-multi_v5_defconfig
> - build-clang-13-lkftconfig
> - build-clang-nightly-keystone_defconfig
> - build-clang-14-multi_v5_defconfig
> - build-clang-14-orion5x_defconfig
> - build-clang-14-omap2plus_defconfig
> - build-clang-nightly-multi_v5_defconfig-65236a87
> - build-clang-14-bcm2835_defconfig
> - build-clang-14-keystone_defconfig
> - build-clang-nightly-lkftconfig
>
> arch/arm/probes/kprobes/core.c:409:30: error: .fnstart must precede
> .save or .vsave directives
> "stmdb sp, {sp, lr, pc} \n\t"
> ^
> <inline asm>:3:2: note: instantiated into assembly here
> .save {sp, lr, pc}
> ^
> /builds/linux/arch/arm/probes/kprobes/core.c:412:29: error: .fnstart
> must precede .pad directive
> "stmdb sp!, {r0 - r11} \n\t"
> ^
> <inline asm>:6:2: note: instantiated into assembly here
> .pad #52
> ^
> 2 errors generated.
> make[5]: *** [/builds/linux/scripts/Makefile.build:250:
> arch/arm/probes/kprobes/core.o] Error 1
>
> build log:
> https://builds.tuxbuild.com/2FAyD1qcTlzjIYE7mjrugjCsxu1/

Thank you for the testing and report! I brought this up on GitHub on
Friday as I noticed this as well:

https://github.com/ClangBuiltLinux/linux/issues/1718

It sounds like we can avoid this by rewriting __kretprobe_trampoline()
in out of line assembly but I have not had a chance to sit down and try
it.

Cheers,
Nathan

2022-09-26 17:21:13

by Russell King (Oracle)

[permalink] [raw]
Subject: Re: arch/arm/probes/kprobes/core.c:409:30: error: .fnstart must precede .save or .vsave directives

Looks to me like a failure caused by Chen Zhongjin's change:
"Recover kretprobes return address for EABI stack unwinder"

I will drop this patch; please submit a replacement that builds.

Thanks.

On Mon, Sep 26, 2022 at 06:57:00PM +0530, Naresh Kamboju wrote:
> Following build warnings / errors noticed on arm with clang-13 / 14
> on Linux next-20220923.
>
> Reported-by: Linux Kernel Functional Testing <[email protected]>
>
> Regressions found on arm:
>
> - build-clang-13-bcm2835_defconfig
> - build-clang-nightly-imx_v6_v7_defconfig
> - build-clang-nightly-orion5x_defconfig
> - build-clang-13-keystone_defconfig
> - build-clang-13-omap2plus_defconfig
> - build-clang-14-imx_v6_v7_defconfig
> - build-clang-nightly-omap2plus_defconfig
> - build-clang-nightly-multi_v5_defconfig
> - build-clang-nightly-bcm2835_defconfig
> - build-clang-13-imx_v6_v7_defconfig
> - build-clang-13-imx_v4_v5_defconfig
> - build-clang-14-imx_v4_v5_defconfig
> - build-clang-13-orion5x_defconfig
> - build-clang-14-multi_v5_defconfig-65236a87
> - build-clang-14-lkftconfig
> - build-clang-nightly-imx_v4_v5_defconfig
> - build-clang-13-multi_v5_defconfig
> - build-clang-13-lkftconfig
> - build-clang-nightly-keystone_defconfig
> - build-clang-14-multi_v5_defconfig
> - build-clang-14-orion5x_defconfig
> - build-clang-14-omap2plus_defconfig
> - build-clang-nightly-multi_v5_defconfig-65236a87
> - build-clang-14-bcm2835_defconfig
> - build-clang-14-keystone_defconfig
> - build-clang-nightly-lkftconfig
>
> arch/arm/probes/kprobes/core.c:409:30: error: .fnstart must precede
> .save or .vsave directives
> "stmdb sp, {sp, lr, pc} \n\t"
> ^
> <inline asm>:3:2: note: instantiated into assembly here
> .save {sp, lr, pc}
> ^
> /builds/linux/arch/arm/probes/kprobes/core.c:412:29: error: .fnstart
> must precede .pad directive
> "stmdb sp!, {r0 - r11} \n\t"
> ^
> <inline asm>:6:2: note: instantiated into assembly here
> .pad #52
> ^
> 2 errors generated.
> make[5]: *** [/builds/linux/scripts/Makefile.build:250:
> arch/arm/probes/kprobes/core.o] Error 1
>
> build log:
> https://builds.tuxbuild.com/2FAyD1qcTlzjIYE7mjrugjCsxu1/
>
> --
> Linaro LKFT
> https://lkft.linaro.org
>

--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!

2022-09-26 17:57:50

by Nathan Chancellor

[permalink] [raw]
Subject: Re: arch/arm/probes/kprobes/core.c:409:30: error: .fnstart must precede .save or .vsave directives

+ our mailing list, I should have added it with that message.

On Mon, Sep 26, 2022 at 08:44:05AM -0700, Nathan Chancellor wrote:
> Hi Naresh,
>
> On Mon, Sep 26, 2022 at 06:57:00PM +0530, Naresh Kamboju wrote:
> > Following build warnings / errors noticed on arm with clang-13 / 14
> > on Linux next-20220923.
> >
> > Reported-by: Linux Kernel Functional Testing <[email protected]>
> >
> > Regressions found on arm:
> >
> > - build-clang-13-bcm2835_defconfig
> > - build-clang-nightly-imx_v6_v7_defconfig
> > - build-clang-nightly-orion5x_defconfig
> > - build-clang-13-keystone_defconfig
> > - build-clang-13-omap2plus_defconfig
> > - build-clang-14-imx_v6_v7_defconfig
> > - build-clang-nightly-omap2plus_defconfig
> > - build-clang-nightly-multi_v5_defconfig
> > - build-clang-nightly-bcm2835_defconfig
> > - build-clang-13-imx_v6_v7_defconfig
> > - build-clang-13-imx_v4_v5_defconfig
> > - build-clang-14-imx_v4_v5_defconfig
> > - build-clang-13-orion5x_defconfig
> > - build-clang-14-multi_v5_defconfig-65236a87
> > - build-clang-14-lkftconfig
> > - build-clang-nightly-imx_v4_v5_defconfig
> > - build-clang-13-multi_v5_defconfig
> > - build-clang-13-lkftconfig
> > - build-clang-nightly-keystone_defconfig
> > - build-clang-14-multi_v5_defconfig
> > - build-clang-14-orion5x_defconfig
> > - build-clang-14-omap2plus_defconfig
> > - build-clang-nightly-multi_v5_defconfig-65236a87
> > - build-clang-14-bcm2835_defconfig
> > - build-clang-14-keystone_defconfig
> > - build-clang-nightly-lkftconfig
> >
> > arch/arm/probes/kprobes/core.c:409:30: error: .fnstart must precede
> > .save or .vsave directives
> > "stmdb sp, {sp, lr, pc} \n\t"
> > ^
> > <inline asm>:3:2: note: instantiated into assembly here
> > .save {sp, lr, pc}
> > ^
> > /builds/linux/arch/arm/probes/kprobes/core.c:412:29: error: .fnstart
> > must precede .pad directive
> > "stmdb sp!, {r0 - r11} \n\t"
> > ^
> > <inline asm>:6:2: note: instantiated into assembly here
> > .pad #52
> > ^
> > 2 errors generated.
> > make[5]: *** [/builds/linux/scripts/Makefile.build:250:
> > arch/arm/probes/kprobes/core.o] Error 1
> >
> > build log:
> > https://builds.tuxbuild.com/2FAyD1qcTlzjIYE7mjrugjCsxu1/
>
> Thank you for the testing and report! I brought this up on GitHub on
> Friday as I noticed this as well:
>
> https://github.com/ClangBuiltLinux/linux/issues/1718
>
> It sounds like we can avoid this by rewriting __kretprobe_trampoline()
> in out of line assembly but I have not had a chance to sit down and try
> it.
>
> Cheers,
> Nathan

2022-09-26 18:22:38

by Russell King (Oracle)

[permalink] [raw]
Subject: Re: arch/arm/probes/kprobes/core.c:409:30: error: .fnstart must precede .save or .vsave directives

On Mon, Sep 26, 2022 at 10:42:45AM -0700, Nick Desaulniers wrote:
> On Mon, Sep 26, 2022 at 8:46 AM Nathan Chancellor <[email protected]> wrote:
> >
> > + our mailing list, I should have added it with that message.
> >
> > On Mon, Sep 26, 2022 at 08:44:05AM -0700, Nathan Chancellor wrote:
> > > Hi Naresh,
> > >
> > > On Mon, Sep 26, 2022 at 06:57:00PM +0530, Naresh Kamboju wrote:
> > > > Following build warnings / errors noticed on arm with clang-13 / 14
> > > > on Linux next-20220923.
> > > >
> > > > Reported-by: Linux Kernel Functional Testing <[email protected]>
> > > >
> > > > Regressions found on arm:
> > > >
> > > > - build-clang-13-bcm2835_defconfig
> > > > - build-clang-nightly-imx_v6_v7_defconfig
> > > > - build-clang-nightly-orion5x_defconfig
> > > > - build-clang-13-keystone_defconfig
> > > > - build-clang-13-omap2plus_defconfig
> > > > - build-clang-14-imx_v6_v7_defconfig
> > > > - build-clang-nightly-omap2plus_defconfig
> > > > - build-clang-nightly-multi_v5_defconfig
> > > > - build-clang-nightly-bcm2835_defconfig
> > > > - build-clang-13-imx_v6_v7_defconfig
> > > > - build-clang-13-imx_v4_v5_defconfig
> > > > - build-clang-14-imx_v4_v5_defconfig
> > > > - build-clang-13-orion5x_defconfig
> > > > - build-clang-14-multi_v5_defconfig-65236a87
> > > > - build-clang-14-lkftconfig
> > > > - build-clang-nightly-imx_v4_v5_defconfig
> > > > - build-clang-13-multi_v5_defconfig
> > > > - build-clang-13-lkftconfig
> > > > - build-clang-nightly-keystone_defconfig
> > > > - build-clang-14-multi_v5_defconfig
> > > > - build-clang-14-orion5x_defconfig
> > > > - build-clang-14-omap2plus_defconfig
> > > > - build-clang-nightly-multi_v5_defconfig-65236a87
> > > > - build-clang-14-bcm2835_defconfig
> > > > - build-clang-14-keystone_defconfig
> > > > - build-clang-nightly-lkftconfig
> > > >
> > > > arch/arm/probes/kprobes/core.c:409:30: error: .fnstart must precede
> > > > .save or .vsave directives
> > > > "stmdb sp, {sp, lr, pc} \n\t"
> > > > ^
> > > > <inline asm>:3:2: note: instantiated into assembly here
> > > > .save {sp, lr, pc}
> > > > ^
> > > > /builds/linux/arch/arm/probes/kprobes/core.c:412:29: error: .fnstart
> > > > must precede .pad directive
> > > > "stmdb sp!, {r0 - r11} \n\t"
> > > > ^
> > > > <inline asm>:6:2: note: instantiated into assembly here
> > > > .pad #52
> > > > ^
> > > > 2 errors generated.
> > > > make[5]: *** [/builds/linux/scripts/Makefile.build:250:
> > > > arch/arm/probes/kprobes/core.o] Error 1
> > > >
> > > > build log:
> > > > https://builds.tuxbuild.com/2FAyD1qcTlzjIYE7mjrugjCsxu1/
> > >
> > > Thank you for the testing and report! I brought this up on GitHub on
> > > Friday as I noticed this as well:
> > >
> > > https://github.com/ClangBuiltLinux/linux/issues/1718
>
> Thanks for the reports. I'll take a look at filing additional bug
> reports against clang, then moving the definition of
> __kretprobe_trampoline to out of line assembler.

Are you saying that .save should be accepted without a .fnstart?

--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!

2022-09-26 18:27:56

by Nick Desaulniers

[permalink] [raw]
Subject: Re: arch/arm/probes/kprobes/core.c:409:30: error: .fnstart must precede .save or .vsave directives

On Mon, Sep 26, 2022 at 8:46 AM Nathan Chancellor <[email protected]> wrote:
>
> + our mailing list, I should have added it with that message.
>
> On Mon, Sep 26, 2022 at 08:44:05AM -0700, Nathan Chancellor wrote:
> > Hi Naresh,
> >
> > On Mon, Sep 26, 2022 at 06:57:00PM +0530, Naresh Kamboju wrote:
> > > Following build warnings / errors noticed on arm with clang-13 / 14
> > > on Linux next-20220923.
> > >
> > > Reported-by: Linux Kernel Functional Testing <[email protected]>
> > >
> > > Regressions found on arm:
> > >
> > > - build-clang-13-bcm2835_defconfig
> > > - build-clang-nightly-imx_v6_v7_defconfig
> > > - build-clang-nightly-orion5x_defconfig
> > > - build-clang-13-keystone_defconfig
> > > - build-clang-13-omap2plus_defconfig
> > > - build-clang-14-imx_v6_v7_defconfig
> > > - build-clang-nightly-omap2plus_defconfig
> > > - build-clang-nightly-multi_v5_defconfig
> > > - build-clang-nightly-bcm2835_defconfig
> > > - build-clang-13-imx_v6_v7_defconfig
> > > - build-clang-13-imx_v4_v5_defconfig
> > > - build-clang-14-imx_v4_v5_defconfig
> > > - build-clang-13-orion5x_defconfig
> > > - build-clang-14-multi_v5_defconfig-65236a87
> > > - build-clang-14-lkftconfig
> > > - build-clang-nightly-imx_v4_v5_defconfig
> > > - build-clang-13-multi_v5_defconfig
> > > - build-clang-13-lkftconfig
> > > - build-clang-nightly-keystone_defconfig
> > > - build-clang-14-multi_v5_defconfig
> > > - build-clang-14-orion5x_defconfig
> > > - build-clang-14-omap2plus_defconfig
> > > - build-clang-nightly-multi_v5_defconfig-65236a87
> > > - build-clang-14-bcm2835_defconfig
> > > - build-clang-14-keystone_defconfig
> > > - build-clang-nightly-lkftconfig
> > >
> > > arch/arm/probes/kprobes/core.c:409:30: error: .fnstart must precede
> > > .save or .vsave directives
> > > "stmdb sp, {sp, lr, pc} \n\t"
> > > ^
> > > <inline asm>:3:2: note: instantiated into assembly here
> > > .save {sp, lr, pc}
> > > ^
> > > /builds/linux/arch/arm/probes/kprobes/core.c:412:29: error: .fnstart
> > > must precede .pad directive
> > > "stmdb sp!, {r0 - r11} \n\t"
> > > ^
> > > <inline asm>:6:2: note: instantiated into assembly here
> > > .pad #52
> > > ^
> > > 2 errors generated.
> > > make[5]: *** [/builds/linux/scripts/Makefile.build:250:
> > > arch/arm/probes/kprobes/core.o] Error 1
> > >
> > > build log:
> > > https://builds.tuxbuild.com/2FAyD1qcTlzjIYE7mjrugjCsxu1/
> >
> > Thank you for the testing and report! I brought this up on GitHub on
> > Friday as I noticed this as well:
> >
> > https://github.com/ClangBuiltLinux/linux/issues/1718

Thanks for the reports. I'll take a look at filing additional bug
reports against clang, then moving the definition of
__kretprobe_trampoline to out of line assembler.

> >
> > It sounds like we can avoid this by rewriting __kretprobe_trampoline()
> > in out of line assembly but I have not had a chance to sit down and try
> > it.
> >
> > Cheers,
> > Nathan
>


--
Thanks,
~Nick Desaulniers

2022-09-26 19:04:07

by Nick Desaulniers

[permalink] [raw]
Subject: Re: arch/arm/probes/kprobes/core.c:409:30: error: .fnstart must precede .save or .vsave directives

On Mon, Sep 26, 2022 at 11:02 AM Russell King (Oracle)
<[email protected]> wrote:
>
> On Mon, Sep 26, 2022 at 10:42:45AM -0700, Nick Desaulniers wrote:
> > > On Mon, Sep 26, 2022 at 08:44:05AM -0700, Nathan Chancellor wrote:
> > > > Thank you for the testing and report! I brought this up on GitHub on
> > > > Friday as I noticed this as well:
> > > >
> > > > https://github.com/ClangBuiltLinux/linux/issues/1718
> >
> > Thanks for the reports. I'll take a look at filing additional bug
> > reports against clang, then moving the definition of
> > __kretprobe_trampoline to out of line assembler.
>
> Are you saying that .save should be accepted without a .fnstart?

No. It's just a bug in clang's inline assembler. But it does make
sense to just move it to out of line assembler anyways; having it be
inline provides little to no benefit.

Should I be using UNWIND from arch/arm/include/asm/unwind.h on these
.fnstart/.save/.pad/.fnend directives?
--
Thanks,
~Nick Desaulniers