Hi all,
After merging the final tree, today's linux-next build (powerpc
allyesconfig) failed like this:
powerpc64-linux-ld: arch/powerpc/net/built-in.o: In function `bpf_slow_path_word':
(.text+0x90): sibling call optimization to `skb_copy_bits' does not allow automatic multiple TOCs; recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `skb_copy_bits' extern
powerpc64-linux-ld: arch/powerpc/net/built-in.o: In function `bpf_slow_path_half':
(.text+0xe0): sibling call optimization to `skb_copy_bits' does not allow automatic multiple TOCs; recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `skb_copy_bits' extern
powerpc64-linux-ld: arch/powerpc/net/built-in.o: In function `bpf_slow_path_byte':
(.text+0x130): sibling call optimization to `skb_copy_bits' does not allow automatic multiple TOCs; recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `skb_copy_bits' extern
powerpc64-linux-ld: arch/powerpc/net/built-in.o: In function `bpf_slow_path_byte_msh':
(.text+0x180): sibling call optimization to `skb_copy_bits' does not allow automatic multiple TOCs; recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `skb_copy_bits' extern
powerpc64-linux-ld: arch/powerpc/net/built-in.o: In function `sk_load_word_negative_offset':
(.text+0x1dc): sibling call optimization to `bpf_internal_load_pointer_neg_helper' does not allow automatic multiple TOCs; recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `bpf_internal_load_pointer_neg_helper' extern
powerpc64-linux-ld: arch/powerpc/net/built-in.o: In function `sk_load_half_negative_offset':
(.text+0x238): sibling call optimization to `bpf_internal_load_pointer_neg_helper' does not allow automatic multiple TOCs; recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `bpf_internal_load_pointer_neg_helper' extern
powerpc64-linux-ld: arch/powerpc/net/built-in.o: In function `sk_load_byte_negative_offset':
(.text+0x294): sibling call optimization to `bpf_internal_load_pointer_neg_helper' does not allow automatic multiple TOCs; recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `bpf_internal_load_pointer_neg_helper' extern
powerpc64-linux-ld: arch/powerpc/net/built-in.o: In function `sk_load_byte_msh_negative_offset':
(.text+0x2f0): sibling call optimization to `bpf_internal_load_pointer_neg_helper' does not allow automatic multiple TOCs; recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `bpf_internal_load_pointer_neg_helper' extern
powerpc64-linux-ld: final link failed: Bad value
I started building with gcc 4.6.3/binutils 2.22 today. gcc
4.6.0/binutils 2.21 do not produce this error, it produces this instead
(which has been happening for a long time):
powerpc64-linux-ld: TOC section size exceeds 64k
--
Cheers,
Stephen Rothwell [email protected]
On Wed, 2012-06-20 at 17:50 +1000, Stephen Rothwell wrote:
> Hi all,
>
> After merging the final tree, today's linux-next build (powerpc
> allyesconfig) failed like this:
I hate our ABI is a good answer ? :-)
I'll see what I can do tomorrow. Poke me when I'm in the office.
Cheers,
Ben.
> powerpc64-linux-ld: arch/powerpc/net/built-in.o: In function `bpf_slow_path_word':
> (.text+0x90): sibling call optimization to `skb_copy_bits' does not allow automatic multiple TOCs; recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `skb_copy_bits' extern
> powerpc64-linux-ld: arch/powerpc/net/built-in.o: In function `bpf_slow_path_half':
> (.text+0xe0): sibling call optimization to `skb_copy_bits' does not allow automatic multiple TOCs; recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `skb_copy_bits' extern
> powerpc64-linux-ld: arch/powerpc/net/built-in.o: In function `bpf_slow_path_byte':
> (.text+0x130): sibling call optimization to `skb_copy_bits' does not allow automatic multiple TOCs; recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `skb_copy_bits' extern
> powerpc64-linux-ld: arch/powerpc/net/built-in.o: In function `bpf_slow_path_byte_msh':
> (.text+0x180): sibling call optimization to `skb_copy_bits' does not allow automatic multiple TOCs; recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `skb_copy_bits' extern
> powerpc64-linux-ld: arch/powerpc/net/built-in.o: In function `sk_load_word_negative_offset':
> (.text+0x1dc): sibling call optimization to `bpf_internal_load_pointer_neg_helper' does not allow automatic multiple TOCs; recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `bpf_internal_load_pointer_neg_helper' extern
> powerpc64-linux-ld: arch/powerpc/net/built-in.o: In function `sk_load_half_negative_offset':
> (.text+0x238): sibling call optimization to `bpf_internal_load_pointer_neg_helper' does not allow automatic multiple TOCs; recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `bpf_internal_load_pointer_neg_helper' extern
> powerpc64-linux-ld: arch/powerpc/net/built-in.o: In function `sk_load_byte_negative_offset':
> (.text+0x294): sibling call optimization to `bpf_internal_load_pointer_neg_helper' does not allow automatic multiple TOCs; recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `bpf_internal_load_pointer_neg_helper' extern
> powerpc64-linux-ld: arch/powerpc/net/built-in.o: In function `sk_load_byte_msh_negative_offset':
> (.text+0x2f0): sibling call optimization to `bpf_internal_load_pointer_neg_helper' does not allow automatic multiple TOCs; recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `bpf_internal_load_pointer_neg_helper' extern
> powerpc64-linux-ld: final link failed: Bad value
>
> I started building with gcc 4.6.3/binutils 2.22 today. gcc
> 4.6.0/binutils 2.21 do not produce this error, it produces this instead
> (which has been happening for a long time):
>
> powerpc64-linux-ld: TOC section size exceeds 64k
>
On Wed, 2012-06-20 at 17:50 +1000, Stephen Rothwell wrote:
> Hi all,
>
> After merging the final tree, today's linux-next build (powerpc
> allyesconfig) failed like this:
>
> powerpc64-linux-ld: arch/powerpc/net/built-in.o: In function `bpf_slow_path_word':
> (.text+0x90): sibling call optimization to `skb_copy_bits' does not allow automatic multiple TOCs; recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `skb_copy_bits' extern
Those seem to be caused because we don't have a nop after the call,
meaning we can't patch the TOC pointer on the way back. Adding a nop
fixes those.
But, then I get 32,410 variants of this:
powerpc64-linux-ld: /src/next/net/openvswitch/vport-netdev.c:189:(.text+0x89b990):
sibling call optimization to `_restgpr0_28' does not allow automatic multiple TOCs;
recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `_restgpr0_28' extern
And those are generated calls so I don't see how we can fix them.
> I started building with gcc 4.6.3/binutils 2.22 today. gcc
> 4.6.0/binutils 2.21 do not produce this error, it produces this instead
> (which has been happening for a long time):
>
> powerpc64-linux-ld: TOC section size exceeds 64k
So presumably there's some new error checking that we're hitting, I
imagine it was always broken, but now it's being more explicit.
I think we need some help from the toolchain experts, hi Alan :)
cheers
On Thu, 2012-06-21 at 15:36 +1000, Michael Ellerman wrote:
>
> powerpc64-linux-ld: /src/next/net/openvswitch/vport-netdev.c:189:(.text+0x89b990):
> sibling call optimization to `_restgpr0_28' does not allow automatic multiple TOCs;
> recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `_restgpr0_28' extern
>
>
> And those are generated calls so I don't see how we can fix them.
Is this a module ? We should really be linking that stuff directly with the module....
The interesting thing is that we do build everything except a handful of
files with -mminimal-toc unless something's wrong with our main Makefile....
Can you show the full build command that triggers the above ?
Cheers,
Ben.
On Thu, 2012-06-21 at 16:24 +1000, Benjamin Herrenschmidt wrote:
> On Thu, 2012-06-21 at 15:36 +1000, Michael Ellerman wrote:
> >
> > powerpc64-linux-ld: /src/next/net/openvswitch/vport-netdev.c:189:(.text+0x89b990):
> > sibling call optimization to `_restgpr0_28' does not allow automatic multiple TOCs;
> > recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `_restgpr0_28' extern
> >
> >
> > And those are generated calls so I don't see how we can fix them.
>
> Is this a module ? We should really be linking that stuff directly with the module....
No, it's builtin.
> The interesting thing is that we do build everything except a handful of
> files with -mminimal-toc unless something's wrong with our main Makefile....
Yeah, the top arch Makefile sets it, though we do override it in a few
places. I tried removing those overrides and it didn't seem to make any
difference.
> Can you show the full build command that triggers the above ?
Well that would be a few MBs of log, but here's an excerpt:
make -f scripts/Makefile.build obj=net/openvswitch
/opt/cross/gcc-4.6-nolibc/powerpc64-linux/bin/powerpc64-linux-gcc -m64 -Wp,-MD,net/openvswitch/.vport-netdev.o.d -nostdinc -isystem /opt/cross/gcc-4.6.3-nolibc/powerpc64-linux/lib/gcc/powerpc64-linux/4.6.3/include -I/home/michael/src/kmk/next/arch/powerpc/include -Iarch/powerpc/include/generated -Iinclude -include /home/michael/src/kmk/next/include/linux/kconfig.h -D__KERNEL__ -Iarch/powerpc -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -fno-delete-null-pointer-checks -Os -msoft-float -pipe -Iarch/powerpc -mminimal-toc -mtraceback=no -mcall-aixdesc -mtune=power7 -mtune=cell -mno-altivec -mno-vsx -mno-spe -mspe=no -funit-at-a-time -fno-dwarf2-cfi-asm -mno-string -mno-sched-epilog -Wa,-maltivec -fno-reorder-blocks -fno-ipa-cp-clone -fno-partial-inlining -Wframe-larger-than=2048 -fno-stack-protector -Wno-unused-but-set-variable -g -femit-struct-debug-baseonly -pg -fno-inline-functions-called-once -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fconserve-stack -DCC_HAVE_ASM_GOTO -fprofile-arcs -ftest-coverage -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(vport_netdev)" -D"KBUILD_MODNAME=KBUILD_STR(openvswitch)" -c -o net/openvswitch/.tmp_vport-netdev.o net/openvswitch/vport-netdev.c
if [ "-pg" = "-pg" ]; then set -e ; perl /home/michael/src/kmk/next/scripts/recordmcount.pl "powerpc" "little" "64" "/opt/cross/gcc-4.6-nolibc/powerpc64-linux/bin/powerpc64-linux-objdump" "/opt/cross/gcc-4.6-nolibc/powerpc64-linux/bin/powerpc64-linux-objcopy" "/opt/cross/gcc-4.6-nolibc/powerpc64-linux/bin/powerpc64-linux-gcc -m64 -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -fno-delete-null-pointer-checks -Os -msoft-float -pipe -Iarch/powerpc -mminimal-toc -mtraceback=no -mcall-aixdesc -mtune=power7 -mtune=cell -mno-altivec -mno-vsx -mno-spe -mspe=no -funit-at-a-time -fno-dwarf2-cfi-asm -mno-string -mno-sched-epilog -Wa,-maltivec -fno-reorder-blocks -fno-ipa-cp-clone -fno-partial-inlining -Wframe-larger-than=2048 -fno-stack-protector -Wno-unused-but-set-variable -g -femit-struct-debug-baseonly -pg -fno-inline-functions-called-once -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fconserve-stack -DCC_HAVE_ASM_GOTO" "/opt/cross/gcc-4.6-nolibc/powerpc64-linux/bin/powerpc64-linux-ld -m elf64ppc" "/opt/cross/gcc-4.6-nolibc/powerpc64-linux/bin/powerpc64-linux-nm --synthetic" "" "" "0" "net/openvswitch/vport-netdev.o"; fi;
And then a whole bunch of calls to ld.
So we are at least building that file with -mminimal-toc.
cheers
On Thu, 2012-06-21 at 17:07 +1000, Michael Ellerman wrote:
> On Thu, 2012-06-21 at 16:24 +1000, Benjamin Herrenschmidt wrote:
> > On Thu, 2012-06-21 at 15:36 +1000, Michael Ellerman wrote:
> > >
> > > powerpc64-linux-ld: /src/next/net/openvswitch/vport-netdev.c:189:(.text+0x89b990):
> > > sibling call optimization to `_restgpr0_28' does not allow automatic multiple TOCs;
> > > recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `_restgpr0_28' extern
> > >
> > Can you show the full build command that triggers the above ?
>
> Well that would be a few MBs of log, but here's an excerpt:
>
> make -f scripts/Makefile.build obj=net/openvswitch
> /opt/cross/gcc-4.6-nolibc/powerpc64-linux/bin/powerpc64-linux-gcc -m64 -Wp,-MD,net/openvswitch/.vport-netdev.o.d -nostdinc -isystem /opt/cross/gcc-4.6.3-nolibc/powerpc64-linux/lib/gcc/powerpc64-linux/4.6.3/include -I/home/michael/src/kmk/next/arch/powerpc/include -Iarch/powerpc/include/generated -Iinclude -include /home/michael/src/kmk/next/include/linux/kconfig.h -D__KERNEL__ -Iarch/powerpc -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -fno-delete-null-pointer-checks -Os -msoft-float -pipe -Iarch/powerpc -mminimal-toc -mtraceback=no -mcall-aixdesc -mtune=power7 -mtune=cell -mno-altivec -mno-vsx -mno-spe -mspe=no -funit-at-a-time -fno-dwarf2-cfi-asm -mno-string -mno-sched-epilog -Wa,-maltivec -fno-reorder-blocks -fno-ipa-cp-clone -fno-partial-inlining -Wframe-larger-than=2048 -fno-stack-protector -Wno-unused-but-set-variable -g -femit-struct-debug-baseonly -pg -fno-inline-functions-called-once -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fconserve-stack -DCC_HAVE_ASM_GOTO -fprofile-arcs -ftest-coverage -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(vport_netdev)" -D"KBUILD_MODNAME=KBUILD_STR(openvswitch)" -c -o net/openvswitch/.tmp_vport-netdev.o net/openvswitch/vport-netdev.c
> if [ "-pg" = "-pg" ]; then set -e ; perl /home/michael/src/kmk/next/scripts/recordmcount.pl "powerpc" "little" "64" "/opt/cross/gcc-4.6-nolibc/powerpc64-linux/bin/powerpc64-linux-objdump" "/opt/cross/gcc-4.6-nolibc/powerpc64-linux/bin/powerpc64-linux-objcopy" "/opt/cross/gcc-4.6-nolibc/powerpc64-linux/bin/powerpc64-linux-gcc -m64 -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -fno-delete-null-pointer-checks -Os -msoft-float -pipe -Iarch/powerpc -mminimal-toc -mtraceback=no -mcall-aixdesc -mtune=power7 -mtune=cell -mno-altivec -mno-vsx -mno-spe -mspe=no -funit-at-a-time -fno-dwarf2-cfi-asm -mno-string -mno-sched-epilog -Wa,-maltivec -fno-reorder-blocks -fno-ipa-cp-clone -fno-partial-inlining -Wframe-larger-than=2048 -fno-stack-protector -Wno-unused-but-set-variable -g -femit-struct-debug-baseonly -pg -fno-inline-functions-called-once -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fconserve-stack -DCC_HAVE_ASM_GOTO" "/opt/cross/gcc-4.6-nolibc/powerpc64-linux/bin/powerpc64-linux-ld -m elf64ppc" "/opt/cross/gcc-4.6-nolibc/powerpc64-linux/bin/powerpc64-linux-nm --synthetic" "" "" "0" "net/openvswitch/vport-netdev.o"; fi;
Just for the record it's not the FTRACE stuff (-pg):
/opt/cross/gcc-4.6-nolibc/powerpc64-linux/bin/powerpc64-linux-gcc -m64 -Wp,-MD,net/openvswitch/.vport-netdev.o.d -nostdinc -isystem /opt/cross/gcc-4.6.3-nolibc/powerpc64-linux/lib/gcc/powerpc64-linux/4.6.3/include -I/home/michael/src/kmk/next/arch/powerpc/include -Iarch/powerpc/include/generated -Iinclude -include /home/michael/src/kmk/next/include/linux/kconfig.h -D__KERNEL__ -Iarch/powerpc -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -fno-delete-null-pointer-checks -Os -msoft-float -pipe -Iarch/powerpc -mminimal-toc -mtraceback=no -mcall-aixdesc -mtune=power7 -mtune=cell -mno-altivec -mno-vsx -mno-spe -mspe=no -funit-at-a-time -fno-dwarf2-cfi-asm -mno-string -Wa,-maltivec -fno-reorder-blocks -fno-ipa-cp-clone -fno-partial-inlining -Wframe-larger-than=2048 -fno-stack-protector -Wno-unused-but-set-variable -fomit-frame-pointer -g -femit-struct-debug-baseonly -fno-inline-functions-called-once -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fconserve-stack -DCC_HAVE_ASM_GOTO -fprofile-arcs -ftest-coverage -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(vport_netdev)" -D"KBUILD_MODNAME=KBUILD_STR(openvswitch)" -c -o net/openvswitch/.tmp_vport-netdev.o net/openvswitch/vport-netdev.c
And same error.
cheers
On Thu, Jun 21, 2012 at 03:36:01PM +1000, Michael Ellerman wrote:
> On Wed, 2012-06-20 at 17:50 +1000, Stephen Rothwell wrote:
> > Hi all,
> >
> > After merging the final tree, today's linux-next build (powerpc
> > allyesconfig) failed like this:
> >
> > powerpc64-linux-ld: arch/powerpc/net/built-in.o: In function `bpf_slow_path_word':
> > (.text+0x90): sibling call optimization to `skb_copy_bits' does not allow automatic multiple TOCs; recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `skb_copy_bits' extern
>
>
> Those seem to be caused because we don't have a nop after the call,
> meaning we can't patch the TOC pointer on the way back. Adding a nop
> fixes those.
>
> But, then I get 32,410 variants of this:
>
> powerpc64-linux-ld: /src/next/net/openvswitch/vport-netdev.c:189:(.text+0x89b990):
> sibling call optimization to `_restgpr0_28' does not allow automatic multiple TOCs;
> recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `_restgpr0_28' extern
>
>
These functions should not need a TOC in the first place. There is
code in the linker (for 64 bit only: bfd/elf64-ppc.c) to automatically
generate them whenever they are needed.
I suspect you compile with -Os. But I don't think you can use
these functions when doing a sibling call since restgpr0_nn
implies a return to the caller. restgpr1_nn would be different...
> And those are generated calls so I don't see how we can fix them.
>
> > I started building with gcc 4.6.3/binutils 2.22 today. gcc
> > 4.6.0/binutils 2.21 do not produce this error, it produces this instead
> > (which has been happening for a long time):
> >
> > powerpc64-linux-ld: TOC section size exceeds 64k
>
>
> So presumably there's some new error checking that we're hitting, I
> imagine it was always broken, but now it's being more explicit.
I'm not so sure. I suspect gcc, but upgrading gcc and binutils at the
same time may not be the wisest...
Gabriel
On Thu, Jun 21, 2012 at 05:38:27PM +1000, Michael Ellerman wrote:
> On Thu, 2012-06-21 at 17:07 +1000, Michael Ellerman wrote:
> > On Thu, 2012-06-21 at 16:24 +1000, Benjamin Herrenschmidt wrote:
> > > On Thu, 2012-06-21 at 15:36 +1000, Michael Ellerman wrote:
> > > >
> > > > powerpc64-linux-ld: /src/next/net/openvswitch/vport-netdev.c:189:(.text+0x89b990):
> > > > sibling call optimization to `_restgpr0_28' does not allow automatic multiple TOCs;
> > > > recompile with -mminimal-toc or -fno-optimize-sibling-calls, or make `_restgpr0_28' extern
> > > >
Linker bug. That's not a sibling call, but a normal function return
via an out-of-line register restore function. Will fix. I'm a bit
surprised to see this with gcc-4.6 though. Or does this gcc-4.6 have
some of my recent mainline gcc patches enabling out-of-line
save/restore functions for -Os?
--
Alan Modra
Australia Development Lab, IBM
On Thu, Jun 21, 2012 at 08:18:39PM +0930, Alan Modra wrote:
> Linker bug. That's not a sibling call, but a normal function return
> via an out-of-line register restore function.
I couldn't see how this might be occurring, then I remembered the
kernel has this horrible practise of using ld -r to package object
files. So linker generated functions might be munged together with
other functions. Does this help? (It won't if the kernel is
providing its own save/restore functions.)
Index: bfd/elf64-ppc.c
===================================================================
RCS file: /cvs/src/src/bfd/elf64-ppc.c,v
retrieving revision 1.387
diff -u -p -r1.387 elf64-ppc.c
@@ -6494,9 +6494,10 @@ ppc64_elf_func_desc_adjust (bfd *obfd AT
/* Provide any missing _save* and _rest* functions. */
htab->sfpr->size = 0;
- for (i = 0; i < sizeof (funcs) / sizeof (funcs[0]); i++)
- if (!sfpr_define (info, &funcs[i]))
- return FALSE;
+ if (!info->relocatable)
+ for (i = 0; i < sizeof (funcs) / sizeof (funcs[0]); i++)
+ if (!sfpr_define (info, &funcs[i]))
+ return FALSE;
elf_link_hash_traverse (&htab->elf, func_desc_adjust, info);
--
Alan Modra
Australia Development Lab, IBM
On Thu, 2012-06-21 at 21:13 +0930, Alan Modra wrote:
> On Thu, Jun 21, 2012 at 08:18:39PM +0930, Alan Modra wrote:
> > Linker bug. That's not a sibling call, but a normal function return
> > via an out-of-line register restore function.
>
> I couldn't see how this might be occurring, then I remembered the
> kernel has this horrible practise of using ld -r to package object
> files. So linker generated functions might be munged together with
> other functions. Does this help? (It won't if the kernel is
> providing its own save/restore functions.)
The kernel does provide its own AIUI.
cheers