LinuxLists.cc - [PATCH 1/2] ARM: Remove '-p' from LDFLAGS

[permalink] [raw]

Subject: Re: [PATCH 2/2] ARM: Wrap '--pic-veneer' with ld-option

On Wed, 5 Dec 2018 at 02:42, Nathan Chancellor <[email protected]> wrote:
>
> This flag is not supported by lld:
>
> ld.lld: error: unknown argument: --pic-veneer
>
> Signed-off-by: Nathan Chancellor <[email protected]>

Hi Nate,

Does this mean ld.lld is guaranteed to produce position independent
veneers if you build kernels that are bigger than the typical range of
a relative branch?

> ---
> arch/arm/Makefile | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/arm/Makefile b/arch/arm/Makefile
> index e2a0baf36766..4fab2aa29570 100644
> --- a/arch/arm/Makefile
> +++ b/arch/arm/Makefile
> @@ -10,7 +10,7 @@
> #
> # Copyright (C) 1995-2001 by Russell King
>
> -LDFLAGS_vmlinux := --no-undefined -X --pic-veneer
> +LDFLAGS_vmlinux := --no-undefined -X $(call ld-option,--pic-veneer)
> ifeq ($(CONFIG_CPU_ENDIAN_BE8),y)
> LDFLAGS_vmlinux += --be8
> KBUILD_LDFLAGS_MODULE += --be8
> --
> 2.20.0.rc1
>

2018-12-05 07:38:48

[permalink] [raw]

Subject: Re: [PATCH 1/2] ARM: Remove '-p' from LDFLAGS

On Wed, 5 Dec 2018 at 02:42, Nathan Chancellor <[email protected]> wrote:
>
> This option is not supported by lld:
>
> ld.lld: error: unknown argument: -p
>
> This has been a no-op in binutils since 2004 (see commit dea514f51da1 in
> that tree). Given that the lowest officially supported of binutils for
> the kernel is 2.20, which was released in 2009, nobody needs this flag
> around so just remove it. Commit 1a381d4a0a9a ("arm64: remove no-op -p
> linker flag") did the same for arm64.
>
> Signed-off-by: Nathan Chancellor <[email protected]>

Acked-by: Ard Biesheuvel <[email protected]>

> ---
> arch/arm/Makefile | 2 +-
> arch/arm/boot/compressed/Makefile | 2 --
> 2 files changed, 1 insertion(+), 3 deletions(-)
>
> diff --git a/arch/arm/Makefile b/arch/arm/Makefile
> index 05a91d8b89f3..e2a0baf36766 100644
> --- a/arch/arm/Makefile
> +++ b/arch/arm/Makefile
> @@ -10,7 +10,7 @@
> #
> # Copyright (C) 1995-2001 by Russell King
>
> -LDFLAGS_vmlinux :=-p --no-undefined -X --pic-veneer
> +LDFLAGS_vmlinux := --no-undefined -X --pic-veneer
> ifeq ($(CONFIG_CPU_ENDIAN_BE8),y)
> LDFLAGS_vmlinux += --be8
> KBUILD_LDFLAGS_MODULE += --be8
> diff --git a/arch/arm/boot/compressed/Makefile b/arch/arm/boot/compressed/Makefile
> index 1f5a5ffe7fcf..dcd07bd24b85 100644
> --- a/arch/arm/boot/compressed/Makefile
> +++ b/arch/arm/boot/compressed/Makefile
> @@ -131,8 +131,6 @@ endif
> ifeq ($(CONFIG_CPU_ENDIAN_BE8),y)
> LDFLAGS_vmlinux += --be8
> endif
> -# ?
> -LDFLAGS_vmlinux += -p
> # Report unresolved symbol references
> LDFLAGS_vmlinux += --no-undefined
> # Delete all temporary local symbols
> --
> 2.20.0.rc1
>

2018-12-05 08:07:43

[permalink] [raw]

Subject: Re: [PATCH 2/2] ARM: Wrap '--pic-veneer' with ld-option

On Wed, Dec 05, 2018 at 08:37:05AM +0100, Ard Biesheuvel wrote:
> On Wed, 5 Dec 2018 at 02:42, Nathan Chancellor <[email protected]> wrote:
> >
> > This flag is not supported by lld:
> >
> > ld.lld: error: unknown argument: --pic-veneer
> >
> > Signed-off-by: Nathan Chancellor <[email protected]>
>
> Hi Nate,
>
> Does this mean ld.lld is guaranteed to produce position independent
> veneers if you build kernels that are bigger than the typical range of
> a relative branch?
>

Hi Ard,

Honestly, I'm not quite sure. I saw your commit that introduced this
flag and I wasn't quite sure what to make of it for lld. What
configuration would I use to verify and what would I check for?

Additionally, I have filed an LLVM bug for the lld developers to
check and see if this is a flag they should support:

https://bugs.llvm.org/show_bug.cgi?id=39886

Thanks for the quick reply,
Nathan

> > ---
> > arch/arm/Makefile | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/arch/arm/Makefile b/arch/arm/Makefile
> > index e2a0baf36766..4fab2aa29570 100644
> > --- a/arch/arm/Makefile
> > +++ b/arch/arm/Makefile
> > @@ -10,7 +10,7 @@
> > #
> > # Copyright (C) 1995-2001 by Russell King
> >
> > -LDFLAGS_vmlinux := --no-undefined -X --pic-veneer
> > +LDFLAGS_vmlinux := --no-undefined -X $(call ld-option,--pic-veneer)
> > ifeq ($(CONFIG_CPU_ENDIAN_BE8),y)
> > LDFLAGS_vmlinux += --be8
> > KBUILD_LDFLAGS_MODULE += --be8
> > --
> > 2.20.0.rc1
> >

2018-12-05 08:11:08

[permalink] [raw]

Subject: Re: [PATCH 2/2] ARM: Wrap '--pic-veneer' with ld-option

(+ Arnd)

On Wed, 5 Dec 2018 at 09:06, Nathan Chancellor <[email protected]> wrote:
>
> On Wed, Dec 05, 2018 at 08:37:05AM +0100, Ard Biesheuvel wrote:
> > On Wed, 5 Dec 2018 at 02:42, Nathan Chancellor <[email protected]> wrote:
> > >
> > > This flag is not supported by lld:
> > >
> > > ld.lld: error: unknown argument: --pic-veneer
> > >
> > > Signed-off-by: Nathan Chancellor <[email protected]>
> >
> > Hi Nate,
> >
> > Does this mean ld.lld is guaranteed to produce position independent
> > veneers if you build kernels that are bigger than the typical range of
> > a relative branch?
> >
>
> Hi Ard,
>
> Honestly, I'm not quite sure. I saw your commit that introduced this
> flag and I wasn't quite sure what to make of it for lld. What
> configuration would I use to verify and what would I check for?
>

Try building allyesconfig, and check the resulting binary for veneers
(which have 'veneer' in the symbol name, at least when ld.bfd emits
them). These veneers should not take the [virtual] address of the
branch target directly, but take a PC relative offset (as in the
example in the commit log of that patch you are referring to)

> Additionally, I have filed an LLVM bug for the lld developers to
> check and see if this is a flag they should support:
>
> https://bugs.llvm.org/show_bug.cgi?id=39886
>
> Thanks for the quick reply,
> Nathan
>
> > > ---
> > > arch/arm/Makefile | 2 +-
> > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git a/arch/arm/Makefile b/arch/arm/Makefile
> > > index e2a0baf36766..4fab2aa29570 100644
> > > --- a/arch/arm/Makefile
> > > +++ b/arch/arm/Makefile
> > > @@ -10,7 +10,7 @@
> > > #
> > > # Copyright (C) 1995-2001 by Russell King
> > >
> > > -LDFLAGS_vmlinux := --no-undefined -X --pic-veneer
> > > +LDFLAGS_vmlinux := --no-undefined -X $(call ld-option,--pic-veneer)
> > > ifeq ($(CONFIG_CPU_ENDIAN_BE8),y)
> > > LDFLAGS_vmlinux += --be8
> > > KBUILD_LDFLAGS_MODULE += --be8
> > > --
> > > 2.20.0.rc1
> > >

2018-12-05 16:39:34

by Nicolas Pitre

[permalink] [raw]

Subject: Re: [PATCH 1/2] ARM: Remove '-p' from LDFLAGS

On Tue, 4 Dec 2018, Nathan Chancellor wrote:

> This option is not supported by lld:
>
> ld.lld: error: unknown argument: -p
>
> This has been a no-op in binutils since 2004 (see commit dea514f51da1 in
> that tree). Given that the lowest officially supported of binutils for
> the kernel is 2.20, which was released in 2009, nobody needs this flag
> around so just remove it. Commit 1a381d4a0a9a ("arm64: remove no-op -p
> linker flag") did the same for arm64.
>
> Signed-off-by: Nathan Chancellor <[email protected]>

Acked-by: Nicolas Pitre <[email protected]>

> ---
> arch/arm/Makefile | 2 +-
> arch/arm/boot/compressed/Makefile | 2 --
> 2 files changed, 1 insertion(+), 3 deletions(-)
>
> diff --git a/arch/arm/Makefile b/arch/arm/Makefile
> index 05a91d8b89f3..e2a0baf36766 100644
> --- a/arch/arm/Makefile
> +++ b/arch/arm/Makefile
> @@ -10,7 +10,7 @@
> #
> # Copyright (C) 1995-2001 by Russell King
>
> -LDFLAGS_vmlinux :=-p --no-undefined -X --pic-veneer
> +LDFLAGS_vmlinux := --no-undefined -X --pic-veneer
> ifeq ($(CONFIG_CPU_ENDIAN_BE8),y)
> LDFLAGS_vmlinux += --be8
> KBUILD_LDFLAGS_MODULE += --be8
> diff --git a/arch/arm/boot/compressed/Makefile b/arch/arm/boot/compressed/Makefile
> index 1f5a5ffe7fcf..dcd07bd24b85 100644
> --- a/arch/arm/boot/compressed/Makefile
> +++ b/arch/arm/boot/compressed/Makefile
> @@ -131,8 +131,6 @@ endif
> ifeq ($(CONFIG_CPU_ENDIAN_BE8),y)
> LDFLAGS_vmlinux += --be8
> endif
> -# ?
> -LDFLAGS_vmlinux += -p
> # Report unresolved symbol references
> LDFLAGS_vmlinux += --no-undefined
> # Delete all temporary local symbols
> --
> 2.20.0.rc1
>
>

2018-12-05 17:51:01

[permalink] [raw]

Subject: Re: [PATCH 1/2] ARM: Remove '-p' from LDFLAGS

On Tue, Dec 4, 2018 at 5:42 PM Nathan Chancellor
<[email protected]> wrote:
>
> This option is not supported by lld:
>
> ld.lld: error: unknown argument: -p
>
> This has been a no-op in binutils since 2004 (see commit dea514f51da1 in
> that tree). Given that the lowest officially supported of binutils for
> the kernel is 2.20, which was released in 2009, nobody needs this flag
> around so just remove it. Commit 1a381d4a0a9a ("arm64: remove no-op -p
> linker flag") did the same for arm64.

While investigating the context of 1a381d4a0a9a for review, I found
that bfd had removed -p's implementation a long time ago, but kept the
flag as a no-op for compatibility. I asked that the patch be extended
to arm as well, but don't think a v2 was ever cut.

Thanks for sending this Nathan.
Reviewed-by: Nick Desaulniers <[email protected]>

>
> Signed-off-by: Nathan Chancellor <[email protected]>
> ---
> arch/arm/Makefile | 2 +-
> arch/arm/boot/compressed/Makefile | 2 --
> 2 files changed, 1 insertion(+), 3 deletions(-)
>
> diff --git a/arch/arm/Makefile b/arch/arm/Makefile
> index 05a91d8b89f3..e2a0baf36766 100644
> --- a/arch/arm/Makefile
> +++ b/arch/arm/Makefile
> @@ -10,7 +10,7 @@
> #
> # Copyright (C) 1995-2001 by Russell King
>
> -LDFLAGS_vmlinux :=-p --no-undefined -X --pic-veneer
> +LDFLAGS_vmlinux := --no-undefined -X --pic-veneer
> ifeq ($(CONFIG_CPU_ENDIAN_BE8),y)
> LDFLAGS_vmlinux += --be8
> KBUILD_LDFLAGS_MODULE += --be8
> diff --git a/arch/arm/boot/compressed/Makefile b/arch/arm/boot/compressed/Makefile
> index 1f5a5ffe7fcf..dcd07bd24b85 100644
> --- a/arch/arm/boot/compressed/Makefile
> +++ b/arch/arm/boot/compressed/Makefile
> @@ -131,8 +131,6 @@ endif
> ifeq ($(CONFIG_CPU_ENDIAN_BE8),y)
> LDFLAGS_vmlinux += --be8
> endif
> -# ?
> -LDFLAGS_vmlinux += -p
> # Report unresolved symbol references
> LDFLAGS_vmlinux += --no-undefined
> # Delete all temporary local symbols
> --
> 2.20.0.rc1
>

--
Thanks,
~Nick Desaulniers

2018-12-05 18:22:53

[permalink] [raw]

Subject: Re: [PATCH 2/2] ARM: Wrap '--pic-veneer' with ld-option

On Wed, Dec 5, 2018 at 12:10 AM Ard Biesheuvel
<[email protected]> wrote:
>
> (+ Arnd)
>
> On Wed, 5 Dec 2018 at 09:06, Nathan Chancellor <[email protected]> wrote:
> >
> > On Wed, Dec 05, 2018 at 08:37:05AM +0100, Ard Biesheuvel wrote:
> > > On Wed, 5 Dec 2018 at 02:42, Nathan Chancellor <[email protected]> wrote:
> > > >
> > > > This flag is not supported by lld:
> > > >
> > > > ld.lld: error: unknown argument: --pic-veneer
> > > >
> > > > Signed-off-by: Nathan Chancellor <[email protected]>
> > >
> > > Hi Nate,
> > >
> > > Does this mean ld.lld is guaranteed to produce position independent
> > > veneers if you build kernels that are bigger than the typical range of
> > > a relative branch?

(+ Peter) who might be able to answer this.

> > >
> >
> > Hi Ard,
> >
> > Honestly, I'm not quite sure. I saw your commit that introduced this
> > flag and I wasn't quite sure what to make of it for lld. What
> > configuration would I use to verify and what would I check for?
> >
>
> Try building allyesconfig, and check the resulting binary for veneers
> (which have 'veneer' in the symbol name, at least when ld.bfd emits
> them). These veneers should not take the [virtual] address of the
> branch target directly, but take a PC relative offset (as in the
> example in the commit log of that patch you are referring to)

Not sure clang can compile an arm32 allyesconfig (haven't tried
personally, yet, TODO), but we could try gcc+lld.

>
> > Additionally, I have filed an LLVM bug for the lld developers to
> > check and see if this is a flag they should support:
> >
> > https://bugs.llvm.org/show_bug.cgi?id=39886
> >
> > Thanks for the quick reply,
> > Nathan
> >
> > > > ---
> > > > arch/arm/Makefile | 2 +-
> > > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > > >
> > > > diff --git a/arch/arm/Makefile b/arch/arm/Makefile
> > > > index e2a0baf36766..4fab2aa29570 100644
> > > > --- a/arch/arm/Makefile
> > > > +++ b/arch/arm/Makefile
> > > > @@ -10,7 +10,7 @@
> > > > #
> > > > # Copyright (C) 1995-2001 by Russell King
> > > >
> > > > -LDFLAGS_vmlinux := --no-undefined -X --pic-veneer
> > > > +LDFLAGS_vmlinux := --no-undefined -X $(call ld-option,--pic-veneer)
> > > > ifeq ($(CONFIG_CPU_ENDIAN_BE8),y)
> > > > LDFLAGS_vmlinux += --be8
> > > > KBUILD_LDFLAGS_MODULE += --be8
> > > > --
> > > > 2.20.0.rc1
> > > >

--
Thanks,
~Nick Desaulniers

2018-12-05 18:37:10

[permalink] [raw]

Subject: Re: [PATCH 2/2] ARM: Wrap '--pic-veneer' with ld-option

On Wed, Dec 05, 2018 at 09:09:56AM +0100, Ard Biesheuvel wrote:
> (+ Arnd)
>
> On Wed, 5 Dec 2018 at 09:06, Nathan Chancellor <[email protected]> wrote:
> >
> > On Wed, Dec 05, 2018 at 08:37:05AM +0100, Ard Biesheuvel wrote:
> > > On Wed, 5 Dec 2018 at 02:42, Nathan Chancellor <[email protected]> wrote:
> > > >
> > > > This flag is not supported by lld:
> > > >
> > > > ld.lld: error: unknown argument: --pic-veneer
> > > >
> > > > Signed-off-by: Nathan Chancellor <[email protected]>
> > >
> > > Hi Nate,
> > >
> > > Does this mean ld.lld is guaranteed to produce position independent
> > > veneers if you build kernels that are bigger than the typical range of
> > > a relative branch?
> > >
> >
> > Hi Ard,
> >
> > Honestly, I'm not quite sure. I saw your commit that introduced this
> > flag and I wasn't quite sure what to make of it for lld. What
> > configuration would I use to verify and what would I check for?
> >
>
> Try building allyesconfig, and check the resulting binary for veneers
> (which have 'veneer' in the symbol name, at least when ld.bfd emits
> them). These veneers should not take the [virtual] address of the
> branch target directly, but take a PC relative offset (as in the
> example in the commit log of that patch you are referring to)
>

Alright, compiling with allyesconfig is a little rough at the moment
(bug reports I will file in due time) but I was able to do it. Here's
the disassembly specifically for the functions you had in your commit,
my assembly knowledge is pretty much non-existent unfortunately so I am
not sure what to make of it (it doesn't look like there is a virtual
address for pc in that mix?). I am happy to provide any more information
that is needed.

c03030cc <__enable_mmu>:
c03030cc: e3c00002 bic r0, r0, #2
c03030d0: e3c00b02 bic r0, r0, #2048 ; 0x800
c03030d4: e3c00a01 bic r0, r0, #4096 ; 0x1000
c03030d8: e3a05051 mov r5, #81 ; 0x51
c03030dc: ee035f10 mcr 15, 0, r5, cr3, cr0, {0}
c03030e0: ee024f10 mcr 15, 0, r4, cr2, cr0, {0}
c03030e4: eafff3c5 b c0300000 <__turn_mmu_on>
c03030e8: e320f000 nop {0}
c03030ec: e320f000 nop {0}
c03030f0: e320f000 nop {0}
c03030f4: e320f000 nop {0}
c03030f8: e320f000 nop {0}
c03030fc: e320f000 nop {0}

c0300000 <__turn_mmu_on>:
c0300000: e1a00000 nop ; (mov r0, r0)
c0300004: ee070f95 mcr 15, 0, r0, cr7, cr5, {4}
c0300008: ee010f10 mcr 15, 0, r0, cr1, cr0, {0}
c030000c: ee103f10 mrc 15, 0, r3, cr0, cr0, {0}
c0300010: ee070f95 mcr 15, 0, r0, cr7, cr5, {4}
c0300014: e1a03003 mov r3, r3
c0300018: e1a0300d mov r3, sp
c030001c: e1a0f003 mov pc, r3

Thanks,
Nathan

> > Additionally, I have filed an LLVM bug for the lld developers to
> > check and see if this is a flag they should support:
> >
> > https://bugs.llvm.org/show_bug.cgi?id=39886
> >
> > Thanks for the quick reply,
> > Nathan
> >
> > > > ---
> > > > arch/arm/Makefile | 2 +-
> > > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > > >
> > > > diff --git a/arch/arm/Makefile b/arch/arm/Makefile
> > > > index e2a0baf36766..4fab2aa29570 100644
> > > > --- a/arch/arm/Makefile
> > > > +++ b/arch/arm/Makefile
> > > > @@ -10,7 +10,7 @@
> > > > #
> > > > # Copyright (C) 1995-2001 by Russell King
> > > >
> > > > -LDFLAGS_vmlinux := --no-undefined -X --pic-veneer
> > > > +LDFLAGS_vmlinux := --no-undefined -X $(call ld-option,--pic-veneer)
> > > > ifeq ($(CONFIG_CPU_ENDIAN_BE8),y)
> > > > LDFLAGS_vmlinux += --be8
> > > > KBUILD_LDFLAGS_MODULE += --be8
> > > > --
> > > > 2.20.0.rc1
> > > >

2018-12-05 18:40:58

[permalink] [raw]

Subject: Re: [PATCH 2/2] ARM: Wrap '--pic-veneer' with ld-option

On Wed, 5 Dec 2018 at 19:36, Nathan Chancellor <[email protected]> wrote:
>
> On Wed, Dec 05, 2018 at 09:09:56AM +0100, Ard Biesheuvel wrote:
> > (+ Arnd)
> >
> > On Wed, 5 Dec 2018 at 09:06, Nathan Chancellor <[email protected]> wrote:
> > >
> > > On Wed, Dec 05, 2018 at 08:37:05AM +0100, Ard Biesheuvel wrote:
> > > > On Wed, 5 Dec 2018 at 02:42, Nathan Chancellor <[email protected]> wrote:
> > > > >
> > > > > This flag is not supported by lld:
> > > > >
> > > > > ld.lld: error: unknown argument: --pic-veneer
> > > > >
> > > > > Signed-off-by: Nathan Chancellor <[email protected]>
> > > >
> > > > Hi Nate,
> > > >
> > > > Does this mean ld.lld is guaranteed to produce position independent
> > > > veneers if you build kernels that are bigger than the typical range of
> > > > a relative branch?
> > > >
> > >
> > > Hi Ard,
> > >
> > > Honestly, I'm not quite sure. I saw your commit that introduced this
> > > flag and I wasn't quite sure what to make of it for lld. What
> > > configuration would I use to verify and what would I check for?
> > >
> >
> > Try building allyesconfig, and check the resulting binary for veneers
> > (which have 'veneer' in the symbol name, at least when ld.bfd emits
> > them). These veneers should not take the [virtual] address of the
> > branch target directly, but take a PC relative offset (as in the
> > example in the commit log of that patch you are referring to)
> >
>
> Alright, compiling with allyesconfig is a little rough at the moment
> (bug reports I will file in due time) but I was able to do it. Here's
> the disassembly specifically for the functions you had in your commit,
> my assembly knowledge is pretty much non-existent unfortunately so I am
> not sure what to make of it (it doesn't look like there is a virtual
> address for pc in that mix?). I am happy to provide any more information
> that is needed.
>
> c03030cc <__enable_mmu>:
> c03030cc: e3c00002 bic r0, r0, #2
> c03030d0: e3c00b02 bic r0, r0, #2048 ; 0x800
> c03030d4: e3c00a01 bic r0, r0, #4096 ; 0x1000
> c03030d8: e3a05051 mov r5, #81 ; 0x51
> c03030dc: ee035f10 mcr 15, 0, r5, cr3, cr0, {0}
> c03030e0: ee024f10 mcr 15, 0, r4, cr2, cr0, {0}
> c03030e4: eafff3c5 b c0300000 <__turn_mmu_on>
> c03030e8: e320f000 nop {0}
> c03030ec: e320f000 nop {0}
> c03030f0: e320f000 nop {0}
> c03030f4: e320f000 nop {0}
> c03030f8: e320f000 nop {0}
> c03030fc: e320f000 nop {0}
>
> c0300000 <__turn_mmu_on>:
> c0300000: e1a00000 nop ; (mov r0, r0)
> c0300004: ee070f95 mcr 15, 0, r0, cr7, cr5, {4}
> c0300008: ee010f10 mcr 15, 0, r0, cr1, cr0, {0}
> c030000c: ee103f10 mrc 15, 0, r3, cr0, cr0, {0}
> c0300010: ee070f95 mcr 15, 0, r0, cr7, cr5, {4}
> c0300014: e1a03003 mov r3, r3
> c0300018: e1a0300d mov r3, sp
> c030001c: e1a0f003 mov pc, r3
>

Thanks Nate.

So these functions no longer appear to reside far away from each
other, so there no veneer has been emitted.

What we're looking for are veneers, i.e., snippets inserted by the
linker that serve as a trampoline so a branch target that is far away
can be reached.

If no symbols exist with 'veneer' in their name *, it might make sense
to rebuild the kernel as Thumb2, which has a branching range of only 8
MB (as opposed to 16 MB for ARM mode)

* I have no idea whether lld names its veneers like this, or even at all

2018-12-05 18:42:09

[permalink] [raw]

Subject: Re: [PATCH 2/2] ARM: Wrap '--pic-veneer' with ld-option

On Wed, Dec 5, 2018 at 10:36 AM Nathan Chancellor
<[email protected]> wrote:
>
> On Wed, Dec 05, 2018 at 09:09:56AM +0100, Ard Biesheuvel wrote:
> > (+ Arnd)
> >
> > On Wed, 5 Dec 2018 at 09:06, Nathan Chancellor <[email protected]> wrote:
> > >
> > > On Wed, Dec 05, 2018 at 08:37:05AM +0100, Ard Biesheuvel wrote:
> > > > On Wed, 5 Dec 2018 at 02:42, Nathan Chancellor <[email protected]> wrote:
> > > > >
> > > > > This flag is not supported by lld:
> > > > >
> > > > > ld.lld: error: unknown argument: --pic-veneer
> > > > >
> > > > > Signed-off-by: Nathan Chancellor <[email protected]>
> > > >
> > > > Hi Nate,
> > > >
> > > > Does this mean ld.lld is guaranteed to produce position independent
> > > > veneers if you build kernels that are bigger than the typical range of
> > > > a relative branch?
> > > >
> > >
> > > Hi Ard,
> > >
> > > Honestly, I'm not quite sure. I saw your commit that introduced this
> > > flag and I wasn't quite sure what to make of it for lld. What
> > > configuration would I use to verify and what would I check for?
> > >
> >
> > Try building allyesconfig, and check the resulting binary for veneers
> > (which have 'veneer' in the symbol name, at least when ld.bfd emits
> > them). These veneers should not take the [virtual] address of the
> > branch target directly, but take a PC relative offset (as in the
> > example in the commit log of that patch you are referring to)
> >
>
> Alright, compiling with allyesconfig is a little rough at the moment
> (bug reports I will file in due time) but I was able to do it. Here's
> the disassembly specifically for the functions you had in your commit,
> my assembly knowledge is pretty much non-existent unfortunately so I am
> not sure what to make of it (it doesn't look like there is a virtual
> address for pc in that mix?). I am happy to provide any more information
> that is needed.

Nathan,
Thanks for getting a build working! I think Ard is looking for the
presence of symbols with `veneer` in the name.

Something like `llvm-objdump -t vmlinux | grep veneer` or `llvm-nm
vmlinux | grep veneer` should tell you if such symbols exist.

If they do, Ard was then looking for the disassembly of those labels
(like you provided for __enable_mmu/__turn_mmu_on).

>
> c03030cc <__enable_mmu>:
> c03030cc: e3c00002 bic r0, r0, #2
> c03030d0: e3c00b02 bic r0, r0, #2048 ; 0x800
> c03030d4: e3c00a01 bic r0, r0, #4096 ; 0x1000
> c03030d8: e3a05051 mov r5, #81 ; 0x51
> c03030dc: ee035f10 mcr 15, 0, r5, cr3, cr0, {0}
> c03030e0: ee024f10 mcr 15, 0, r4, cr2, cr0, {0}
> c03030e4: eafff3c5 b c0300000 <__turn_mmu_on>
> c03030e8: e320f000 nop {0}
> c03030ec: e320f000 nop {0}
> c03030f0: e320f000 nop {0}
> c03030f4: e320f000 nop {0}
> c03030f8: e320f000 nop {0}
> c03030fc: e320f000 nop {0}
>
> c0300000 <__turn_mmu_on>:
> c0300000: e1a00000 nop ; (mov r0, r0)
> c0300004: ee070f95 mcr 15, 0, r0, cr7, cr5, {4}
> c0300008: ee010f10 mcr 15, 0, r0, cr1, cr0, {0}
> c030000c: ee103f10 mrc 15, 0, r3, cr0, cr0, {0}
> c0300010: ee070f95 mcr 15, 0, r0, cr7, cr5, {4}
> c0300014: e1a03003 mov r3, r3
> c0300018: e1a0300d mov r3, sp
> c030001c: e1a0f003 mov pc, r3
>
> Thanks,
> Nathan
>
> > > Additionally, I have filed an LLVM bug for the lld developers to
> > > check and see if this is a flag they should support:
> > >
> > > https://bugs.llvm.org/show_bug.cgi?id=39886
> > >
> > > Thanks for the quick reply,
> > > Nathan
> > >
> > > > > ---
> > > > > arch/arm/Makefile | 2 +-
> > > > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > > > >
> > > > > diff --git a/arch/arm/Makefile b/arch/arm/Makefile
> > > > > index e2a0baf36766..4fab2aa29570 100644
> > > > > --- a/arch/arm/Makefile
> > > > > +++ b/arch/arm/Makefile
> > > > > @@ -10,7 +10,7 @@
> > > > > #
> > > > > # Copyright (C) 1995-2001 by Russell King
> > > > >
> > > > > -LDFLAGS_vmlinux := --no-undefined -X --pic-veneer
> > > > > +LDFLAGS_vmlinux := --no-undefined -X $(call ld-option,--pic-veneer)
> > > > > ifeq ($(CONFIG_CPU_ENDIAN_BE8),y)
> > > > > LDFLAGS_vmlinux += --be8
> > > > > KBUILD_LDFLAGS_MODULE += --be8
> > > > > --
> > > > > 2.20.0.rc1
> > > > >

--
Thanks,
~Nick Desaulniers

2018-12-05 18:43:55

[permalink] [raw]

Subject: Re: [PATCH 2/2] ARM: Wrap '--pic-veneer' with ld-option

On Wed, Dec 5, 2018 at 10:40 AM Ard Biesheuvel
<[email protected]> wrote:
>
> On Wed, 5 Dec 2018 at 19:36, Nathan Chancellor <[email protected]> wrote:
> >
> > On Wed, Dec 05, 2018 at 09:09:56AM +0100, Ard Biesheuvel wrote:
> > > (+ Arnd)
> > >
> > > On Wed, 5 Dec 2018 at 09:06, Nathan Chancellor <[email protected]> wrote:
> > > >
> > > > On Wed, Dec 05, 2018 at 08:37:05AM +0100, Ard Biesheuvel wrote:
> > > > > On Wed, 5 Dec 2018 at 02:42, Nathan Chancellor <[email protected]> wrote:
> > > > > >
> > > > > > This flag is not supported by lld:
> > > > > >
> > > > > > ld.lld: error: unknown argument: --pic-veneer
> > > > > >
> > > > > > Signed-off-by: Nathan Chancellor <[email protected]>
> > > > >
> > > > > Hi Nate,
> > > > >
> > > > > Does this mean ld.lld is guaranteed to produce position independent
> > > > > veneers if you build kernels that are bigger than the typical range of
> > > > > a relative branch?
> > > > >
> > > >
> > > > Hi Ard,
> > > >
> > > > Honestly, I'm not quite sure. I saw your commit that introduced this
> > > > flag and I wasn't quite sure what to make of it for lld. What
> > > > configuration would I use to verify and what would I check for?
> > > >
> > >
> > > Try building allyesconfig, and check the resulting binary for veneers
> > > (which have 'veneer' in the symbol name, at least when ld.bfd emits
> > > them). These veneers should not take the [virtual] address of the
> > > branch target directly, but take a PC relative offset (as in the
> > > example in the commit log of that patch you are referring to)
> > >
> >
> > Alright, compiling with allyesconfig is a little rough at the moment
> > (bug reports I will file in due time) but I was able to do it. Here's
> > the disassembly specifically for the functions you had in your commit,
> > my assembly knowledge is pretty much non-existent unfortunately so I am
> > not sure what to make of it (it doesn't look like there is a virtual
> > address for pc in that mix?). I am happy to provide any more information
> > that is needed.
> >
> > c03030cc <__enable_mmu>:
> > c03030cc: e3c00002 bic r0, r0, #2
> > c03030d0: e3c00b02 bic r0, r0, #2048 ; 0x800
> > c03030d4: e3c00a01 bic r0, r0, #4096 ; 0x1000
> > c03030d8: e3a05051 mov r5, #81 ; 0x51
> > c03030dc: ee035f10 mcr 15, 0, r5, cr3, cr0, {0}
> > c03030e0: ee024f10 mcr 15, 0, r4, cr2, cr0, {0}
> > c03030e4: eafff3c5 b c0300000 <__turn_mmu_on>
> > c03030e8: e320f000 nop {0}
> > c03030ec: e320f000 nop {0}
> > c03030f0: e320f000 nop {0}
> > c03030f4: e320f000 nop {0}
> > c03030f8: e320f000 nop {0}
> > c03030fc: e320f000 nop {0}
> >
> > c0300000 <__turn_mmu_on>:
> > c0300000: e1a00000 nop ; (mov r0, r0)
> > c0300004: ee070f95 mcr 15, 0, r0, cr7, cr5, {4}
> > c0300008: ee010f10 mcr 15, 0, r0, cr1, cr0, {0}
> > c030000c: ee103f10 mrc 15, 0, r3, cr0, cr0, {0}
> > c0300010: ee070f95 mcr 15, 0, r0, cr7, cr5, {4}
> > c0300014: e1a03003 mov r3, r3
> > c0300018: e1a0300d mov r3, sp
> > c030001c: e1a0f003 mov pc, r3
> >
>
> Thanks Nate.
>
> So these functions no longer appear to reside far away from each
> other, so there no veneer has been emitted.
>
> What we're looking for are veneers, i.e., snippets inserted by the
> linker that serve as a trampoline so a branch target that is far away
> can be reached.
>
> If no symbols exist with 'veneer' in their name *, it might make sense
> to rebuild the kernel as Thumb2, which has a branching range of only 8
> MB (as opposed to 16 MB for ARM mode)

Heh, Arnd and I were just talking about this yesterday. Is there a
config that sets Thumb2 mode for the kernel?

>
> * I have no idea whether lld names its veneers like this, or even at all

--
Thanks,
~Nick Desaulniers

2018-12-05 19:31:34

by Peter Smith

[permalink] [raw]

Subject: Re: [PATCH 2/2] ARM: Wrap '--pic-veneer' with ld-option

On Wed, 5 Dec 2018 at 18:22, Nick Desaulniers <[email protected]> wrote:
>
> On Wed, Dec 5, 2018 at 12:10 AM Ard Biesheuvel
> <[email protected]> wrote:
> >
> > (+ Arnd)
> >
> > On Wed, 5 Dec 2018 at 09:06, Nathan Chancellor <[email protected]> wrote:
> > >
> > > On Wed, Dec 05, 2018 at 08:37:05AM +0100, Ard Biesheuvel wrote:
> > > > On Wed, 5 Dec 2018 at 02:42, Nathan Chancellor <[email protected]> wrote:
> > > > >
> > > > > This flag is not supported by lld:
> > > > >
> > > > > ld.lld: error: unknown argument: --pic-veneer
> > > > >
> > > > > Signed-off-by: Nathan Chancellor <[email protected]>
> > > >
> > > > Hi Nate,
> > > >
> > > > Does this mean ld.lld is guaranteed to produce position independent
> > > > veneers if you build kernels that are bigger than the typical range of
> > > > a relative branch?
>
> (+ Peter) who might be able to answer this.
>

LLD will only use position independent veneers when --shared or --pie
is used, otherwise it will use veneers with absolute addresses. If the
kernel must use position independent veneers even when the rest of the
code isn't then we'll have to implement --pic-veneer.

Peter

> > > >
> > >
> > > Hi Ard,
> > >
> > > Honestly, I'm not quite sure. I saw your commit that introduced this
> > > flag and I wasn't quite sure what to make of it for lld. What
> > > configuration would I use to verify and what would I check for?
> > >
> >
> > Try building allyesconfig, and check the resulting binary for veneers
> > (which have 'veneer' in the symbol name, at least when ld.bfd emits
> > them). These veneers should not take the [virtual] address of the
> > branch target directly, but take a PC relative offset (as in the
> > example in the commit log of that patch you are referring to)
>
> Not sure clang can compile an arm32 allyesconfig (haven't tried
> personally, yet, TODO), but we could try gcc+lld.
>
> >
> > > Additionally, I have filed an LLVM bug for the lld developers to
> > > check and see if this is a flag they should support:
> > >
> > > https://bugs.llvm.org/show_bug.cgi?id=39886
> > >
> > > Thanks for the quick reply,
> > > Nathan
> > >
> > > > > ---
> > > > > arch/arm/Makefile | 2 +-
> > > > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > > > >
> > > > > diff --git a/arch/arm/Makefile b/arch/arm/Makefile
> > > > > index e2a0baf36766..4fab2aa29570 100644
> > > > > --- a/arch/arm/Makefile
> > > > > +++ b/arch/arm/Makefile
> > > > > @@ -10,7 +10,7 @@
> > > > > #
> > > > > # Copyright (C) 1995-2001 by Russell King
> > > > >
> > > > > -LDFLAGS_vmlinux := --no-undefined -X --pic-veneer
> > > > > +LDFLAGS_vmlinux := --no-undefined -X $(call ld-option,--pic-veneer)
> > > > > ifeq ($(CONFIG_CPU_ENDIAN_BE8),y)
> > > > > LDFLAGS_vmlinux += --be8
> > > > > KBUILD_LDFLAGS_MODULE += --be8
> > > > > --
> > > > > 2.20.0.rc1
> > > > >
>
>
>
> --
> Thanks,
> ~Nick Desaulniers

2018-12-05 19:47:43

[permalink] [raw]

Subject: Re: [PATCH 2/2] ARM: Wrap '--pic-veneer' with ld-option

On Wed, Dec 05, 2018 at 07:39:55PM +0100, Ard Biesheuvel wrote:
> On Wed, 5 Dec 2018 at 19:36, Nathan Chancellor <[email protected]> wrote:
> >
> > On Wed, Dec 05, 2018 at 09:09:56AM +0100, Ard Biesheuvel wrote:
> > > (+ Arnd)
> > >
> > > On Wed, 5 Dec 2018 at 09:06, Nathan Chancellor <[email protected]> wrote:
> > > >
> > > > On Wed, Dec 05, 2018 at 08:37:05AM +0100, Ard Biesheuvel wrote:
> > > > > On Wed, 5 Dec 2018 at 02:42, Nathan Chancellor <[email protected]> wrote:
> > > > > >
> > > > > > This flag is not supported by lld:
> > > > > >
> > > > > > ld.lld: error: unknown argument: --pic-veneer
> > > > > >
> > > > > > Signed-off-by: Nathan Chancellor <[email protected]>
> > > > >
> > > > > Hi Nate,
> > > > >
> > > > > Does this mean ld.lld is guaranteed to produce position independent
> > > > > veneers if you build kernels that are bigger than the typical range of
> > > > > a relative branch?
> > > > >
> > > >
> > > > Hi Ard,
> > > >
> > > > Honestly, I'm not quite sure. I saw your commit that introduced this
> > > > flag and I wasn't quite sure what to make of it for lld. What
> > > > configuration would I use to verify and what would I check for?
> > > >
> > >
> > > Try building allyesconfig, and check the resulting binary for veneers
> > > (which have 'veneer' in the symbol name, at least when ld.bfd emits
> > > them). These veneers should not take the [virtual] address of the
> > > branch target directly, but take a PC relative offset (as in the
> > > example in the commit log of that patch you are referring to)
> > >
> >
> > Alright, compiling with allyesconfig is a little rough at the moment
> > (bug reports I will file in due time) but I was able to do it. Here's
> > the disassembly specifically for the functions you had in your commit,
> > my assembly knowledge is pretty much non-existent unfortunately so I am
> > not sure what to make of it (it doesn't look like there is a virtual
> > address for pc in that mix?). I am happy to provide any more information
> > that is needed.
> >
> > c03030cc <__enable_mmu>:
> > c03030cc: e3c00002 bic r0, r0, #2
> > c03030d0: e3c00b02 bic r0, r0, #2048 ; 0x800
> > c03030d4: e3c00a01 bic r0, r0, #4096 ; 0x1000
> > c03030d8: e3a05051 mov r5, #81 ; 0x51
> > c03030dc: ee035f10 mcr 15, 0, r5, cr3, cr0, {0}
> > c03030e0: ee024f10 mcr 15, 0, r4, cr2, cr0, {0}
> > c03030e4: eafff3c5 b c0300000 <__turn_mmu_on>
> > c03030e8: e320f000 nop {0}
> > c03030ec: e320f000 nop {0}
> > c03030f0: e320f000 nop {0}
> > c03030f4: e320f000 nop {0}
> > c03030f8: e320f000 nop {0}
> > c03030fc: e320f000 nop {0}
> >
> > c0300000 <__turn_mmu_on>:
> > c0300000: e1a00000 nop ; (mov r0, r0)
> > c0300004: ee070f95 mcr 15, 0, r0, cr7, cr5, {4}
> > c0300008: ee010f10 mcr 15, 0, r0, cr1, cr0, {0}
> > c030000c: ee103f10 mrc 15, 0, r3, cr0, cr0, {0}
> > c0300010: ee070f95 mcr 15, 0, r0, cr7, cr5, {4}
> > c0300014: e1a03003 mov r3, r3
> > c0300018: e1a0300d mov r3, sp
> > c030001c: e1a0f003 mov pc, r3
> >
>
> Thanks Nate.
>
> So these functions no longer appear to reside far away from each
> other, so there no veneer has been emitted.
>
> What we're looking for are veneers, i.e., snippets inserted by the
> linker that serve as a trampoline so a branch target that is far away
> can be reached.
>
> If no symbols exist with 'veneer' in their name *, it might make sense
> to rebuild the kernel as Thumb2, which has a branching range of only 8
> MB (as opposed to 16 MB for ARM mode)
>
> * I have no idea whether lld names its veneers like this, or even at all

Thanks Ard, I understand now, I appreciate that.

I compiled with CONFIG_THUMB2_KERNEL (config attached) and I am still
not seeing any veneers or thunks as Peter said they would be called for
lld in the LLVM bug report linked earlier in the thread. Peter did note
that the branch ranges were 32MB and 16MB for ARM and Thumb2
respectively, which could be playing into this.

c03028d0 <__enable_mmu>:
c03028d0: f020 0002 bic.w r0, r0, #2
c03028d4: f420 6000 bic.w r0, r0, #2048 ; 0x800
c03028d8: f420 5080 bic.w r0, r0, #4096 ; 0x1000
c03028dc: f04f 0551 mov.w r5, #81 ; 0x51
c03028e0: ee03 5f10 mcr 15, 0, r5, cr3, cr0, {0}
c03028e4: ee02 4f10 mcr 15, 0, r4, cr2, cr0, {0}
c03028e8: f7fd bb8a b.w c0300000 <__turn_mmu_on>
c03028ec: f3af 8000 nop.w
c03028f0: f3af 8000 nop.w
c03028f4: f3af 8000 nop.w
c03028f8: f3af 8000 nop.w
c03028fc: f3af 8000 nop.w

c0300000 <__turn_mmu_on>:
c0300000: 4600 mov r0, r0
c0300002: f3bf 8f6f isb sy
c0300006: ee01 0f10 mcr 15, 0, r0, cr1, cr0, {0}
c030000a: ee10 3f10 mrc 15, 0, r3, cr0, cr0, {0}
c030000e: f3bf 8f6f isb sy
c0300012: 461b mov r3, r3
c0300014: 466b mov r3, sp
c0300016: 469f mov pc, r3

Thanks for all the insight you've given!
Nathan

Attachments:

(No filename) (5.43 kB)
.config (285.17 kB)
Download all attachments

2018-12-05 21:04:54

[permalink] [raw]

Subject: Re: [PATCH 2/2] ARM: Wrap '--pic-veneer' with ld-option

On Wed, 5 Dec 2018 at 20:45, Nathan Chancellor <[email protected]> wrote:
>
> On Wed, Dec 05, 2018 at 07:39:55PM +0100, Ard Biesheuvel wrote:
> > On Wed, 5 Dec 2018 at 19:36, Nathan Chancellor <[email protected]> wrote:
> > >
> > > On Wed, Dec 05, 2018 at 09:09:56AM +0100, Ard Biesheuvel wrote:
> > > > (+ Arnd)
> > > >
> > > > On Wed, 5 Dec 2018 at 09:06, Nathan Chancellor <[email protected]> wrote:
> > > > >
> > > > > On Wed, Dec 05, 2018 at 08:37:05AM +0100, Ard Biesheuvel wrote:
> > > > > > On Wed, 5 Dec 2018 at 02:42, Nathan Chancellor <[email protected]> wrote:
> > > > > > >
> > > > > > > This flag is not supported by lld:
> > > > > > >
> > > > > > > ld.lld: error: unknown argument: --pic-veneer
> > > > > > >
> > > > > > > Signed-off-by: Nathan Chancellor <[email protected]>
> > > > > >
> > > > > > Hi Nate,
> > > > > >
> > > > > > Does this mean ld.lld is guaranteed to produce position independent
> > > > > > veneers if you build kernels that are bigger than the typical range of
> > > > > > a relative branch?
> > > > > >
> > > > >
> > > > > Hi Ard,
> > > > >
> > > > > Honestly, I'm not quite sure. I saw your commit that introduced this
> > > > > flag and I wasn't quite sure what to make of it for lld. What
> > > > > configuration would I use to verify and what would I check for?
> > > > >
> > > >
> > > > Try building allyesconfig, and check the resulting binary for veneers
> > > > (which have 'veneer' in the symbol name, at least when ld.bfd emits
> > > > them). These veneers should not take the [virtual] address of the
> > > > branch target directly, but take a PC relative offset (as in the
> > > > example in the commit log of that patch you are referring to)
> > > >
> > >
> > > Alright, compiling with allyesconfig is a little rough at the moment
> > > (bug reports I will file in due time) but I was able to do it. Here's
> > > the disassembly specifically for the functions you had in your commit,
> > > my assembly knowledge is pretty much non-existent unfortunately so I am
> > > not sure what to make of it (it doesn't look like there is a virtual
> > > address for pc in that mix?). I am happy to provide any more information
> > > that is needed.
> > >
> > > c03030cc <__enable_mmu>:
> > > c03030cc: e3c00002 bic r0, r0, #2
> > > c03030d0: e3c00b02 bic r0, r0, #2048 ; 0x800
> > > c03030d4: e3c00a01 bic r0, r0, #4096 ; 0x1000
> > > c03030d8: e3a05051 mov r5, #81 ; 0x51
> > > c03030dc: ee035f10 mcr 15, 0, r5, cr3, cr0, {0}
> > > c03030e0: ee024f10 mcr 15, 0, r4, cr2, cr0, {0}
> > > c03030e4: eafff3c5 b c0300000 <__turn_mmu_on>
> > > c03030e8: e320f000 nop {0}
> > > c03030ec: e320f000 nop {0}
> > > c03030f0: e320f000 nop {0}
> > > c03030f4: e320f000 nop {0}
> > > c03030f8: e320f000 nop {0}
> > > c03030fc: e320f000 nop {0}
> > >
> > > c0300000 <__turn_mmu_on>:
> > > c0300000: e1a00000 nop ; (mov r0, r0)
> > > c0300004: ee070f95 mcr 15, 0, r0, cr7, cr5, {4}
> > > c0300008: ee010f10 mcr 15, 0, r0, cr1, cr0, {0}
> > > c030000c: ee103f10 mrc 15, 0, r3, cr0, cr0, {0}
> > > c0300010: ee070f95 mcr 15, 0, r0, cr7, cr5, {4}
> > > c0300014: e1a03003 mov r3, r3
> > > c0300018: e1a0300d mov r3, sp
> > > c030001c: e1a0f003 mov pc, r3
> > >
> >
> > Thanks Nate.
> >
> > So these functions no longer appear to reside far away from each
> > other, so there no veneer has been emitted.
> >
> > What we're looking for are veneers, i.e., snippets inserted by the
> > linker that serve as a trampoline so a branch target that is far away
> > can be reached.
> >
> > If no symbols exist with 'veneer' in their name *, it might make sense
> > to rebuild the kernel as Thumb2, which has a branching range of only 8
> > MB (as opposed to 16 MB for ARM mode)
> >
> > * I have no idea whether lld names its veneers like this, or even at all
>
> Thanks Ard, I understand now, I appreciate that.
>
> I compiled with CONFIG_THUMB2_KERNEL (config attached) and I am still
> not seeing any veneers or thunks as Peter said they would be called for
> lld in the LLVM bug report linked earlier in the thread. Peter did note
> that the branch ranges were 32MB and 16MB for ARM and Thumb2
> respectively, which could be playing into this.
>

Ah, indeed. Off by one :-)

However, even when building in ARM rather than Thumb2 mode,
allyesconfig gives me

ard@harold:~/linux-build-arm$ size vmlinux
text data bss dec hex filename
107708906 50335363 15258568 173302837 a546435 vmlinux
ard@harold:~/linux-build-arm$ grep -cE _veneer$ System.map
16199

so surely, lld is emitting veneers but perhaps it doesn't emit symbols
for them in the same way as ld.bfd does.

Actually, most of those veneers are fine, given that the vast majority
of the code will only run at its proper link time address. However,
some pieces run with the MMU off, either at early boot or around
suspend/resume, and so we need veneers to be position independent.

2018-12-05 23:00:46

by Stefan Agner

[permalink] [raw]

Subject: Re: [PATCH 2/2] ARM: Wrap '--pic-veneer' with ld-option

On 05.12.2018 19:41, Nick Desaulniers wrote:
> On Wed, Dec 5, 2018 at 10:40 AM Ard Biesheuvel
> <[email protected]> wrote:
>>
>> On Wed, 5 Dec 2018 at 19:36, Nathan Chancellor <[email protected]> wrote:
>> >
>> > On Wed, Dec 05, 2018 at 09:09:56AM +0100, Ard Biesheuvel wrote:
>> > > (+ Arnd)
>> > >
>> > > On Wed, 5 Dec 2018 at 09:06, Nathan Chancellor <[email protected]> wrote:
>> > > >
>> > > > On Wed, Dec 05, 2018 at 08:37:05AM +0100, Ard Biesheuvel wrote:
>> > > > > On Wed, 5 Dec 2018 at 02:42, Nathan Chancellor <[email protected]> wrote:
>> > > > > >
>> > > > > > This flag is not supported by lld:
>> > > > > >
>> > > > > > ld.lld: error: unknown argument: --pic-veneer
>> > > > > >
>> > > > > > Signed-off-by: Nathan Chancellor <[email protected]>
>> > > > >
>> > > > > Hi Nate,
>> > > > >
>> > > > > Does this mean ld.lld is guaranteed to produce position independent
>> > > > > veneers if you build kernels that are bigger than the typical range of
>> > > > > a relative branch?
>> > > > >
>> > > >
>> > > > Hi Ard,
>> > > >
>> > > > Honestly, I'm not quite sure. I saw your commit that introduced this
>> > > > flag and I wasn't quite sure what to make of it for lld. What
>> > > > configuration would I use to verify and what would I check for?
>> > > >
>> > >
>> > > Try building allyesconfig, and check the resulting binary for veneers
>> > > (which have 'veneer' in the symbol name, at least when ld.bfd emits
>> > > them). These veneers should not take the [virtual] address of the
>> > > branch target directly, but take a PC relative offset (as in the
>> > > example in the commit log of that patch you are referring to)
>> > >
>> >
>> > Alright, compiling with allyesconfig is a little rough at the moment
>> > (bug reports I will file in due time) but I was able to do it. Here's
>> > the disassembly specifically for the functions you had in your commit,
>> > my assembly knowledge is pretty much non-existent unfortunately so I am
>> > not sure what to make of it (it doesn't look like there is a virtual
>> > address for pc in that mix?). I am happy to provide any more information
>> > that is needed.
>> >
>> > c03030cc <__enable_mmu>:
>> > c03030cc: e3c00002 bic r0, r0, #2
>> > c03030d0: e3c00b02 bic r0, r0, #2048 ; 0x800
>> > c03030d4: e3c00a01 bic r0, r0, #4096 ; 0x1000
>> > c03030d8: e3a05051 mov r5, #81 ; 0x51
>> > c03030dc: ee035f10 mcr 15, 0, r5, cr3, cr0, {0}
>> > c03030e0: ee024f10 mcr 15, 0, r4, cr2, cr0, {0}
>> > c03030e4: eafff3c5 b c0300000 <__turn_mmu_on>
>> > c03030e8: e320f000 nop {0}
>> > c03030ec: e320f000 nop {0}
>> > c03030f0: e320f000 nop {0}
>> > c03030f4: e320f000 nop {0}
>> > c03030f8: e320f000 nop {0}
>> > c03030fc: e320f000 nop {0}
>> >
>> > c0300000 <__turn_mmu_on>:
>> > c0300000: e1a00000 nop ; (mov r0, r0)
>> > c0300004: ee070f95 mcr 15, 0, r0, cr7, cr5, {4}
>> > c0300008: ee010f10 mcr 15, 0, r0, cr1, cr0, {0}
>> > c030000c: ee103f10 mrc 15, 0, r3, cr0, cr0, {0}
>> > c0300010: ee070f95 mcr 15, 0, r0, cr7, cr5, {4}
>> > c0300014: e1a03003 mov r3, r3
>> > c0300018: e1a0300d mov r3, sp
>> > c030001c: e1a0f003 mov pc, r3
>> >
>>
>> Thanks Nate.
>>
>> So these functions no longer appear to reside far away from each
>> other, so there no veneer has been emitted.
>>
>> What we're looking for are veneers, i.e., snippets inserted by the
>> linker that serve as a trampoline so a branch target that is far away
>> can be reached.
>>
>> If no symbols exist with 'veneer' in their name *, it might make sense
>> to rebuild the kernel as Thumb2, which has a branching range of only 8
>> MB (as opposed to 16 MB for ARM mode)
>
> Heh, Arnd and I were just talking about this yesterday. Is there a
> config that sets Thumb2 mode for the kernel?
>

Yes there is CONFIG_THUMB2_KERNEL, and it works also with LLVM/Clang.

However, it sometimes leads to surprising issues, like I just
encountered a few days ago:

https://lore.kernel.org/linux-pci/[email protected]/

--
Stefan

>>
>> * I have no idea whether lld names its veneers like this, or even at all

2018-12-05 23:20:30

[permalink] [raw]

Subject: Re: [PATCH 2/2] ARM: Wrap '--pic-veneer' with ld-option

On Wed, Dec 5, 2018 at 2:59 PM Stefan Agner <[email protected]> wrote:
>
> On 05.12.2018 19:41, Nick Desaulniers wrote:
> > On Wed, Dec 5, 2018 at 10:40 AM Ard Biesheuvel
> > <[email protected]> wrote:
> >>
> >> On Wed, 5 Dec 2018 at 19:36, Nathan Chancellor <[email protected]> wrote:
> >> >
> >> > On Wed, Dec 05, 2018 at 09:09:56AM +0100, Ard Biesheuvel wrote:
> >> > > (+ Arnd)
> >> > >
> >> > > On Wed, 5 Dec 2018 at 09:06, Nathan Chancellor <[email protected]> wrote:
> >> > > >
> >> > > > On Wed, Dec 05, 2018 at 08:37:05AM +0100, Ard Biesheuvel wrote:
> >> > > > > On Wed, 5 Dec 2018 at 02:42, Nathan Chancellor <[email protected]> wrote:
> >> > > > > >
> >> > > > > > This flag is not supported by lld:
> >> > > > > >
> >> > > > > > ld.lld: error: unknown argument: --pic-veneer
> >> > > > > >
> >> > > > > > Signed-off-by: Nathan Chancellor <[email protected]>
> >> > > > >
> >> > > > > Hi Nate,
> >> > > > >
> >> > > > > Does this mean ld.lld is guaranteed to produce position independent
> >> > > > > veneers if you build kernels that are bigger than the typical range of
> >> > > > > a relative branch?
> >> > > > >
> >> > > >
> >> > > > Hi Ard,
> >> > > >
> >> > > > Honestly, I'm not quite sure. I saw your commit that introduced this
> >> > > > flag and I wasn't quite sure what to make of it for lld. What
> >> > > > configuration would I use to verify and what would I check for?
> >> > > >
> >> > >
> >> > > Try building allyesconfig, and check the resulting binary for veneers
> >> > > (which have 'veneer' in the symbol name, at least when ld.bfd emits
> >> > > them). These veneers should not take the [virtual] address of the
> >> > > branch target directly, but take a PC relative offset (as in the
> >> > > example in the commit log of that patch you are referring to)
> >> > >
> >> >
> >> > Alright, compiling with allyesconfig is a little rough at the moment
> >> > (bug reports I will file in due time) but I was able to do it. Here's
> >> > the disassembly specifically for the functions you had in your commit,
> >> > my assembly knowledge is pretty much non-existent unfortunately so I am
> >> > not sure what to make of it (it doesn't look like there is a virtual
> >> > address for pc in that mix?). I am happy to provide any more information
> >> > that is needed.
> >> >
> >> > c03030cc <__enable_mmu>:
> >> > c03030cc: e3c00002 bic r0, r0, #2
> >> > c03030d0: e3c00b02 bic r0, r0, #2048 ; 0x800
> >> > c03030d4: e3c00a01 bic r0, r0, #4096 ; 0x1000
> >> > c03030d8: e3a05051 mov r5, #81 ; 0x51
> >> > c03030dc: ee035f10 mcr 15, 0, r5, cr3, cr0, {0}
> >> > c03030e0: ee024f10 mcr 15, 0, r4, cr2, cr0, {0}
> >> > c03030e4: eafff3c5 b c0300000 <__turn_mmu_on>
> >> > c03030e8: e320f000 nop {0}
> >> > c03030ec: e320f000 nop {0}
> >> > c03030f0: e320f000 nop {0}
> >> > c03030f4: e320f000 nop {0}
> >> > c03030f8: e320f000 nop {0}
> >> > c03030fc: e320f000 nop {0}
> >> >
> >> > c0300000 <__turn_mmu_on>:
> >> > c0300000: e1a00000 nop ; (mov r0, r0)
> >> > c0300004: ee070f95 mcr 15, 0, r0, cr7, cr5, {4}
> >> > c0300008: ee010f10 mcr 15, 0, r0, cr1, cr0, {0}
> >> > c030000c: ee103f10 mrc 15, 0, r3, cr0, cr0, {0}
> >> > c0300010: ee070f95 mcr 15, 0, r0, cr7, cr5, {4}
> >> > c0300014: e1a03003 mov r3, r3
> >> > c0300018: e1a0300d mov r3, sp
> >> > c030001c: e1a0f003 mov pc, r3
> >> >
> >>
> >> Thanks Nate.
> >>
> >> So these functions no longer appear to reside far away from each
> >> other, so there no veneer has been emitted.
> >>
> >> What we're looking for are veneers, i.e., snippets inserted by the
> >> linker that serve as a trampoline so a branch target that is far away
> >> can be reached.
> >>
> >> If no symbols exist with 'veneer' in their name *, it might make sense
> >> to rebuild the kernel as Thumb2, which has a branching range of only 8
> >> MB (as opposed to 16 MB for ARM mode)
> >
> > Heh, Arnd and I were just talking about this yesterday. Is there a
> > config that sets Thumb2 mode for the kernel?
> >
>
> Yes there is CONFIG_THUMB2_KERNEL, and it works also with LLVM/Clang.

Sounds like something we should put under CI?
https://github.com/ClangBuiltLinux/continuous-integration/issues/94

>
> However, it sometimes leads to surprising issues, like I just
> encountered a few days ago:
>
> https://lore.kernel.org/linux-pci/[email protected]/
>
> --
> Stefan
>
> >>
> >> * I have no idea whether lld names its veneers like this, or even at all

--
Thanks,
~Nick Desaulniers

2018-12-29 19:08:34

by Stefan Agner

[permalink] [raw]

Subject: Re: [PATCH 2/2] ARM: Wrap '--pic-veneer' with ld-option

On 05.12.2018 20:45, Nathan Chancellor wrote:
> On Wed, Dec 05, 2018 at 07:39:55PM +0100, Ard Biesheuvel wrote:
>> On Wed, 5 Dec 2018 at 19:36, Nathan Chancellor <[email protected]> wrote:
>> >
>> > On Wed, Dec 05, 2018 at 09:09:56AM +0100, Ard Biesheuvel wrote:
>> > > (+ Arnd)
>> > >
>> > > On Wed, 5 Dec 2018 at 09:06, Nathan Chancellor <[email protected]> wrote:
>> > > >
>> > > > On Wed, Dec 05, 2018 at 08:37:05AM +0100, Ard Biesheuvel wrote:
>> > > > > On Wed, 5 Dec 2018 at 02:42, Nathan Chancellor <[email protected]> wrote:
>> > > > > >
>> > > > > > This flag is not supported by lld:
>> > > > > >
>> > > > > > ld.lld: error: unknown argument: --pic-veneer
>> > > > > >
>> > > > > > Signed-off-by: Nathan Chancellor <[email protected]>
>> > > > >
>> > > > > Hi Nate,
>> > > > >
>> > > > > Does this mean ld.lld is guaranteed to produce position independent
>> > > > > veneers if you build kernels that are bigger than the typical range of
>> > > > > a relative branch?
>> > > > >
>> > > >
>> > > > Hi Ard,
>> > > >
>> > > > Honestly, I'm not quite sure. I saw your commit that introduced this
>> > > > flag and I wasn't quite sure what to make of it for lld. What
>> > > > configuration would I use to verify and what would I check for?
>> > > >
>> > >
>> > > Try building allyesconfig, and check the resulting binary for veneers
>> > > (which have 'veneer' in the symbol name, at least when ld.bfd emits
>> > > them). These veneers should not take the [virtual] address of the
>> > > branch target directly, but take a PC relative offset (as in the
>> > > example in the commit log of that patch you are referring to)
>> > >
>> >
>> > Alright, compiling with allyesconfig is a little rough at the moment
>> > (bug reports I will file in due time) but I was able to do it. Here's
>> > the disassembly specifically for the functions you had in your commit,
>> > my assembly knowledge is pretty much non-existent unfortunately so I am
>> > not sure what to make of it (it doesn't look like there is a virtual
>> > address for pc in that mix?). I am happy to provide any more information
>> > that is needed.
>> >
>> > c03030cc <__enable_mmu>:
>> > c03030cc: e3c00002 bic r0, r0, #2
>> > c03030d0: e3c00b02 bic r0, r0, #2048 ; 0x800
>> > c03030d4: e3c00a01 bic r0, r0, #4096 ; 0x1000
>> > c03030d8: e3a05051 mov r5, #81 ; 0x51
>> > c03030dc: ee035f10 mcr 15, 0, r5, cr3, cr0, {0}
>> > c03030e0: ee024f10 mcr 15, 0, r4, cr2, cr0, {0}
>> > c03030e4: eafff3c5 b c0300000 <__turn_mmu_on>
>> > c03030e8: e320f000 nop {0}
>> > c03030ec: e320f000 nop {0}
>> > c03030f0: e320f000 nop {0}
>> > c03030f4: e320f000 nop {0}
>> > c03030f8: e320f000 nop {0}
>> > c03030fc: e320f000 nop {0}
>> >
>> > c0300000 <__turn_mmu_on>:
>> > c0300000: e1a00000 nop ; (mov r0, r0)
>> > c0300004: ee070f95 mcr 15, 0, r0, cr7, cr5, {4}
>> > c0300008: ee010f10 mcr 15, 0, r0, cr1, cr0, {0}
>> > c030000c: ee103f10 mrc 15, 0, r3, cr0, cr0, {0}
>> > c0300010: ee070f95 mcr 15, 0, r0, cr7, cr5, {4}
>> > c0300014: e1a03003 mov r3, r3
>> > c0300018: e1a0300d mov r3, sp
>> > c030001c: e1a0f003 mov pc, r3
>> >
>>
>> Thanks Nate.
>>
>> So these functions no longer appear to reside far away from each
>> other, so there no veneer has been emitted.
>>
>> What we're looking for are veneers, i.e., snippets inserted by the
>> linker that serve as a trampoline so a branch target that is far away
>> can be reached.
>>
>> If no symbols exist with 'veneer' in their name *, it might make sense
>> to rebuild the kernel as Thumb2, which has a branching range of only 8
>> MB (as opposed to 16 MB for ARM mode)
>>
>> * I have no idea whether lld names its veneers like this, or even at all
>
> Thanks Ard, I understand now, I appreciate that.
>
> I compiled with CONFIG_THUMB2_KERNEL (config attached) and I am still
> not seeing any veneers or thunks as Peter said they would be called for
> lld in the LLVM bug report linked earlier in the thread. Peter did note
> that the branch ranges were 32MB and 16MB for ARM and Thumb2
> respectively, which could be playing into this.
>
> c03028d0 <__enable_mmu>:
> c03028d0: f020 0002 bic.w r0, r0, #2
> c03028d4: f420 6000 bic.w r0, r0, #2048 ; 0x800
> c03028d8: f420 5080 bic.w r0, r0, #4096 ; 0x1000
> c03028dc: f04f 0551 mov.w r5, #81 ; 0x51
> c03028e0: ee03 5f10 mcr 15, 0, r5, cr3, cr0, {0}
> c03028e4: ee02 4f10 mcr 15, 0, r4, cr2, cr0, {0}
> c03028e8: f7fd bb8a b.w c0300000 <__turn_mmu_on>
> c03028ec: f3af 8000 nop.w
> c03028f0: f3af 8000 nop.w
> c03028f4: f3af 8000 nop.w
> c03028f8: f3af 8000 nop.w
> c03028fc: f3af 8000 nop.w
>
> c0300000 <__turn_mmu_on>:
> c0300000: 4600 mov r0, r0
> c0300002: f3bf 8f6f isb sy
> c0300006: ee01 0f10 mcr 15, 0, r0, cr1, cr0, {0}
> c030000a: ee10 3f10 mrc 15, 0, r3, cr0, cr0, {0}
> c030000e: f3bf 8f6f isb sy
> c0300012: 461b mov r3, r3
> c0300014: 466b mov r3, sp
> c0300016: 469f mov pc, r3
>
> Thanks for all the insight you've given!
> Nathan

I was able to reproduce the issue by using allyesconfig, removing
--pic-veneer and reverting eb765c1ceb27 ("ARM: 8317/1: move the
.idmap.text section closer to .head.text"). I guess due to that fix the
issue currently does not appear in practice (or at least not in that
particular case we are looking at).

Using ld.bfd:
$ arm-buildroot-linux-gnueabihf-objdump -d vmlinux | grep -C 4
turn_mmu_on
c020815c: e12fff1e bx lr
c0208160: cc3c2857 .word 0xcc3c2857
c0208164: fd883000 .word 0xfd883000

c0208168 <__turn_mmu_on_loc>:
c0208168: c0208168 .word 0xc0208168
c020816c: c7b11230 .word 0xc7b11230
c0208170: c7b11250 .word 0xc7b11250

--
c0303304: e3c00a01 bic r0, r0, #4096 ; 0x1000
c0303308: e3a05051 mov r5, #81 ; 0x51
c030330c: ee035f10 mcr 15, 0, r5, cr3, cr0, {0}
c0303310: ee024f10 mcr 15, 0, r4, cr2, cr0, {0}
c0303314: ea0fd9c7 b c06f9a38
<____turn_mmu_on_veneer>
c0303318: e320f000 nop {0}
c030331c: e320f000 nop {0}

c0303320 <__do_fixup_smp_on_up>:
--
c06f9a30 <__bus_register_notifier_veneer>:
c06f9a30: e51ff004 ldr pc, [pc, #-4] ; c06f9a34
<__bus_register_notifier_veneer+0x4>
c06f9a34: c29752c0 .word 0xc29752c0

c06f9a38 <____turn_mmu_on_veneer>:
c06f9a38: e51ff004 ldr pc, [pc, #-4] ; c06f9a3c
<____turn_mmu_on_veneer+0x4>
c06f9a3c: c7b11230 .word 0xc7b11230

c06f9a40 <__rb_replace_node_veneer>:
c06f9a40: e51ff004 ldr pc, [pc, #-4] ; c06f9a44
<__rb_replace_node_veneer+0x4>
--

c7b1122c <__kprobes_text_end>:
c7b1122c: 00000000 .word 0x00000000

c7b11230 <__turn_mmu_on>:
c7b11230: e1a00000 nop ; (mov r0, r0)
c7b11234: ee070f95 mcr 15, 0, r0, cr7, cr5, {4}
c7b11238: ee010f10 mcr 15, 0, r0, cr1, cr0, {0}
c7b1123c: ee103f10 mrc 15, 0, r3, cr0, cr0, {0}

And with ld.lld:

$ arm-buildroot-linux-gnueabihf-objdump -d vmlinux | grep -C 4
turn_mmu_on
c0208158: e12fff1e bx lr
c020815c: cb2cb747 .word 0xcb2cb747
c0208160: fd883000 .word 0xfd883000

c0208164 <__turn_mmu_on_loc>:
c0208164: c0208164 .word 0xc0208164
c0208168: c6c3ca38 .word 0xc6c3ca38
c020816c: c6c3ca58 .word 0xc6c3ca58

--
c0301fb4: e3c00a01 bic r0, r0, #4096 ; 0x1000
c0301fb8: e3a05051 mov r5, #81 ; 0x51
c0301fbc: ee035f10 mcr 15, 0, r5, cr3, cr0, {0}
c0301fc0: ee024f10 mcr 15, 0, r4, cr2, cr0, {0}
c0301fc4: ea3f3e38 b c12d18ac
<__ARMv7ABSLongThunk___turn_mmu_on>
c0301fc8: e320f000 nop {0}
c0301fcc: e320f000 nop {0}
c0301fd0: e320f000 nop {0}
c0301fd4: e320f000 nop {0}
--
c12d18a0: e3a00001 mov r0, #1
c12d18a4: e28dd004 add sp, sp, #4
c12d18a8: e8bd8ff0 pop {r4, r5, r6, r7, r8, r9, sl, fp,
pc}

c12d18ac <__ARMv7ABSLongThunk___turn_mmu_on>:
c12d18ac: e30cca38 movw ip, #51768 ; 0xca38
c12d18b0: e34cc6c3 movt ip, #50883 ; 0xc6c3
c12d18b4: e12fff1c bx ip

--
c6c3ca2c: e309c90c movw ip, #39180 ; 0x990c
c6c3ca30: e34cc032 movt ip, #49202 ; 0xc032
c6c3ca34: e12fff1c bx ip

c6c3ca38 <__turn_mmu_on>:
c6c3ca38: e1a00000 nop ; (mov r0, r0)
c6c3ca3c: f57ff06f isb sy
c6c3ca40: ee010f10 mcr 15, 0, r0, cr1, cr0, {0}
c6c3ca44: ee103f10 mrc 15, 0, r3, cr0, cr0, {0}

With --pic-veneer and ld.bfd:

$ arm-buildroot-linux-gnueabihf-objdump -d vmlinux | grep -C 4
turn_mmu_on
c0208158: e12fff1e bx lr
c020815c: cb1cb747 .word 0xcb1cb747
c0208160: fd883000 .word 0xfd883000

c0208164 <__turn_mmu_on_loc>:
c0208164: c0208164 .word 0xc0208164
c0208168: c6c514e0 .word 0xc6c514e0
c020816c: c6c51500 .word 0xc6c51500

--
c0301fb4: e3c00a01 bic r0, r0, #4096 ; 0x1000
c0301fb8: e3a05051 mov r5, #81 ; 0x51
c0301fbc: ee035f10 mcr 15, 0, r5, cr3, cr0, {0}
c0301fc0: ee024f10 mcr 15, 0, r4, cr2, cr0, {0}
c0301fc4: ea0fd5a4 b c06f765c
<____turn_mmu_on_veneer>
c0301fc8: e320f000 nop {0}
c0301fcc: e320f000 nop {0}
c0301fd0: e320f000 nop {0}
c0301fd4: e320f000 nop {0}
--
c06f7650: e59fc000 ldr ip, [pc] ; c06f7658
<____xa_set_mark_veneer+0x8>
c06f7654: e08ff00c add pc, pc, ip
c06f7658: 0654cebc .word 0x0654cebc

c06f765c <____turn_mmu_on_veneer>:
c06f765c: e59fc000 ldr ip, [pc] ; c06f7664
<____turn_mmu_on_veneer+0x8>
c06f7660: e08ff00c add pc, pc, ip
c06f7664: 06559e78 .word 0x06559e78

--
c6c514d4: e1a00006 mov r0, r6
c6c514d8: e8bd4070 pop {r4, r5, r6, lr}
c6c514dc: e12fff13 bx r3

c6c514e0 <__turn_mmu_on>:
c6c514e0: e1a00000 nop ; (mov r0, r0)
c6c514e4: f57ff06f isb sy
c6c514e8: ee010f10 mcr 15, 0, r0, cr1, cr0, {0}
c6c514ec: ee103f10 mrc 15, 0, r3, cr0, cr0, {0}

--
Stefan

2018-12-29 19:54:34

by Stefan Agner

[permalink] [raw]

Subject: Re: [PATCH 1/2] ARM: Remove '-p' from LDFLAGS

On 05.12.2018 02:42, Nathan Chancellor wrote:
> This option is not supported by lld:
>
> ld.lld: error: unknown argument: -p
>
> This has been a no-op in binutils since 2004 (see commit dea514f51da1 in
> that tree). Given that the lowest officially supported of binutils for
> the kernel is 2.20, which was released in 2009, nobody needs this flag
> around so just remove it. Commit 1a381d4a0a9a ("arm64: remove no-op -p
> linker flag") did the same for arm64.
>
> Signed-off-by: Nathan Chancellor <[email protected]>

There is another -p in arch/arm/boot/bootp/Makefile.

With that fixed:

Reviewed-by: Stefan Agner <[email protected]>

I guess you can send a v2 with this patch only, which then can make it
in. Patch 2 seems to get fixed by adding --pic-veneer support in lld.

--
Stefan

> ---
> arch/arm/Makefile | 2 +-
> arch/arm/boot/compressed/Makefile | 2 --
> 2 files changed, 1 insertion(+), 3 deletions(-)
>
> diff --git a/arch/arm/Makefile b/arch/arm/Makefile
> index 05a91d8b89f3..e2a0baf36766 100644
> --- a/arch/arm/Makefile
> +++ b/arch/arm/Makefile
> @@ -10,7 +10,7 @@
> #
> # Copyright (C) 1995-2001 by Russell King
>
> -LDFLAGS_vmlinux :=-p --no-undefined -X --pic-veneer
> +LDFLAGS_vmlinux := --no-undefined -X --pic-veneer
> ifeq ($(CONFIG_CPU_ENDIAN_BE8),y)
> LDFLAGS_vmlinux += --be8
> KBUILD_LDFLAGS_MODULE += --be8
> diff --git a/arch/arm/boot/compressed/Makefile
> b/arch/arm/boot/compressed/Makefile
> index 1f5a5ffe7fcf..dcd07bd24b85 100644
> --- a/arch/arm/boot/compressed/Makefile
> +++ b/arch/arm/boot/compressed/Makefile
> @@ -131,8 +131,6 @@ endif
> ifeq ($(CONFIG_CPU_ENDIAN_BE8),y)
> LDFLAGS_vmlinux += --be8
> endif
> -# ?
> -LDFLAGS_vmlinux += -p
> # Report unresolved symbol references
> LDFLAGS_vmlinux += --no-undefined
> # Delete all temporary local symbols

2018-12-31 19:31:54

[permalink] [raw]

Subject: [PATCH v2] ARM: Remove '-p' from LDFLAGS

This option is not supported by lld:

ld.lld: error: unknown argument: -p

This has been a no-op in binutils since 2004 (see commit dea514f51da1 in
that tree). Given that the lowest officially supported of binutils for
the kernel is 2.20, which was released in 2009, nobody needs this flag
around so just remove it. Commit 1a381d4a0a9a ("arm64: remove no-op -p
linker flag") did the same for arm64.

Signed-off-by: Nathan Chancellor <[email protected]>
Acked-by: Ard Biesheuvel <[email protected]>
Acked-by: Nicolas Pitre <[email protected]>
Reviewed-by: Nick Desaulniers <[email protected]>
Reviewed-by: Stefan Agner <[email protected]>
---

v1 -> v2:

* Collect tags

* Remove an additional occurence of '-p' in the bootp Makefile, thank
you to Stefan for pointing it out

* Drop patch 2/2, it will be fixed properly in lld: https://reviews.llvm.org/D55505

arch/arm/Makefile | 2 +-
arch/arm/boot/bootp/Makefile | 2 +-
arch/arm/boot/compressed/Makefile | 2 --
3 files changed, 2 insertions(+), 4 deletions(-)

diff --git a/arch/arm/Makefile b/arch/arm/Makefile
index 0436002d5091..1d48c8ed5f89 100644
--- a/arch/arm/Makefile
+++ b/arch/arm/Makefile
@@ -10,7 +10,7 @@
#
# Copyright (C) 1995-2001 by Russell King

-LDFLAGS_vmlinux :=-p --no-undefined -X --pic-veneer
+LDFLAGS_vmlinux := --no-undefined -X --pic-veneer
ifeq ($(CONFIG_CPU_ENDIAN_BE8),y)
LDFLAGS_vmlinux += --be8
KBUILD_LDFLAGS_MODULE += --be8
diff --git a/arch/arm/boot/bootp/Makefile b/arch/arm/boot/bootp/Makefile
index 83e1a076a5d6..981a8d03f064 100644
--- a/arch/arm/boot/bootp/Makefile
+++ b/arch/arm/boot/bootp/Makefile
@@ -8,7 +8,7 @@

GCOV_PROFILE := n

-LDFLAGS_bootp :=-p --no-undefined -X \
+LDFLAGS_bootp := --no-undefined -X \
--defsym initrd_phys=$(INITRD_PHYS) \
--defsym params_phys=$(PARAMS_PHYS) -T
AFLAGS_initrd.o :=-DINITRD=\"$(INITRD)\"
diff --git a/arch/arm/boot/compressed/Makefile b/arch/arm/boot/compressed/Makefile
index 01bf2585a0fa..ecfa4ca320a7 100644
--- a/arch/arm/boot/compressed/Makefile
+++ b/arch/arm/boot/compressed/Makefile
@@ -132,8 +132,6 @@ endif
ifeq ($(CONFIG_CPU_ENDIAN_BE8),y)
LDFLAGS_vmlinux += --be8
endif
-# ?
-LDFLAGS_vmlinux += -p
# Report unresolved symbol references
LDFLAGS_vmlinux += --no-undefined
# Delete all temporary local symbols
--
2.20.1

2019-01-05 16:37:25

by Nicolas Pitre

[permalink] [raw]

Subject: Re: [PATCH v2] ARM: Remove '-p' from LDFLAGS

On Mon, 31 Dec 2018, Nathan Chancellor wrote:

> This option is not supported by lld:
>
> ld.lld: error: unknown argument: -p
>
> This has been a no-op in binutils since 2004 (see commit dea514f51da1 in
> that tree). Given that the lowest officially supported of binutils for
> the kernel is 2.20, which was released in 2009, nobody needs this flag
> around so just remove it. Commit 1a381d4a0a9a ("arm64: remove no-op -p
> linker flag") did the same for arm64.
>
> Signed-off-by: Nathan Chancellor <[email protected]>
> Acked-by: Ard Biesheuvel <[email protected]>
> Acked-by: Nicolas Pitre <[email protected]>
> Reviewed-by: Nick Desaulniers <[email protected]>
> Reviewed-by: Stefan Agner <[email protected]>

You may now submit your patch to RMK's patch system:
http://www.arm.linux.org.uk/developer/patches/

> ---
>
> v1 -> v2:
>
> * Collect tags
>
> * Remove an additional occurence of '-p' in the bootp Makefile, thank
> you to Stefan for pointing it out
>
> * Drop patch 2/2, it will be fixed properly in lld: https://reviews.llvm.org/D55505
>
> arch/arm/Makefile | 2 +-
> arch/arm/boot/bootp/Makefile | 2 +-
> arch/arm/boot/compressed/Makefile | 2 --
> 3 files changed, 2 insertions(+), 4 deletions(-)
>
> diff --git a/arch/arm/Makefile b/arch/arm/Makefile
> index 0436002d5091..1d48c8ed5f89 100644
> --- a/arch/arm/Makefile
> +++ b/arch/arm/Makefile
> @@ -10,7 +10,7 @@
> #
> # Copyright (C) 1995-2001 by Russell King
>
> -LDFLAGS_vmlinux :=-p --no-undefined -X --pic-veneer
> +LDFLAGS_vmlinux := --no-undefined -X --pic-veneer
> ifeq ($(CONFIG_CPU_ENDIAN_BE8),y)
> LDFLAGS_vmlinux += --be8
> KBUILD_LDFLAGS_MODULE += --be8
> diff --git a/arch/arm/boot/bootp/Makefile b/arch/arm/boot/bootp/Makefile
> index 83e1a076a5d6..981a8d03f064 100644
> --- a/arch/arm/boot/bootp/Makefile
> +++ b/arch/arm/boot/bootp/Makefile
> @@ -8,7 +8,7 @@
>
> GCOV_PROFILE := n
>
> -LDFLAGS_bootp :=-p --no-undefined -X \
> +LDFLAGS_bootp := --no-undefined -X \
> --defsym initrd_phys=$(INITRD_PHYS) \
> --defsym params_phys=$(PARAMS_PHYS) -T
> AFLAGS_initrd.o :=-DINITRD=\"$(INITRD)\"
> diff --git a/arch/arm/boot/compressed/Makefile b/arch/arm/boot/compressed/Makefile
> index 01bf2585a0fa..ecfa4ca320a7 100644
> --- a/arch/arm/boot/compressed/Makefile
> +++ b/arch/arm/boot/compressed/Makefile
> @@ -132,8 +132,6 @@ endif
> ifeq ($(CONFIG_CPU_ENDIAN_BE8),y)
> LDFLAGS_vmlinux += --be8
> endif
> -# ?
> -LDFLAGS_vmlinux += -p
> # Report unresolved symbol references
> LDFLAGS_vmlinux += --no-undefined
> # Delete all temporary local symbols
> --
> 2.20.1
>
>

2019-01-07 19:44:16

[permalink] [raw]

Subject: Re: [PATCH 2/2] ARM: Wrap '--pic-veneer' with ld-option

On Sat, Dec 29, 2018 at 4:27 AM Stefan Agner <[email protected]> wrote:
>
> On 05.12.2018 20:45, Nathan Chancellor wrote:
> > On Wed, Dec 05, 2018 at 07:39:55PM +0100, Ard Biesheuvel wrote:
> >> On Wed, 5 Dec 2018 at 19:36, Nathan Chancellor <[email protected]> wrote:
> >> >
> >> > On Wed, Dec 05, 2018 at 09:09:56AM +0100, Ard Biesheuvel wrote:
> >> > > (+ Arnd)
> >> > >
> >> > > On Wed, 5 Dec 2018 at 09:06, Nathan Chancellor <[email protected]> wrote:
> >> > > >
> >> > > > On Wed, Dec 05, 2018 at 08:37:05AM +0100, Ard Biesheuvel wrote:
> >> > > > > On Wed, 5 Dec 2018 at 02:42, Nathan Chancellor <[email protected]> wrote:
> >> > > > > >
> >> > > > > > This flag is not supported by lld:
> >> > > > > >
> >> > > > > > ld.lld: error: unknown argument: --pic-veneer
> >> > > > > >
> >> > > > > > Signed-off-by: Nathan Chancellor <[email protected]>
> >> > > > >
> >> > > > > Hi Nate,
> >> > > > >
> >> > > > > Does this mean ld.lld is guaranteed to produce position independent
> >> > > > > veneers if you build kernels that are bigger than the typical range of
> >> > > > > a relative branch?

Just for the list, 2 patches to lld:
https://github.com/ClangBuiltLinux/linux/issues/286#issuecomment-452027802

--
Thanks,
~Nick Desaulniers

2019-01-07 19:59:32