Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <20181210222635.80886-1-trong@android.com> <CAKwvOdkb0_hxwnO2T_F34i-MYN_MJksg973Yu1hfb8HeRCXMMQ@mail.gmail.com>
 <CAKwvOd=0=r7oNVZW6VHNjB8PvMF0Etn6fgRbrFdaQKeKHXyexQ@mail.gmail.com>
In-Reply-To: <CAKwvOd=0=r7oNVZW6VHNjB8PvMF0Etn6fgRbrFdaQKeKHXyexQ@mail.gmail.com>
From:   Tri Vo <trong@android.com>
Date:   Wed, 12 Dec 2018 11:48:51 -0800
Message-ID: <CANA+-vBcxF7P1C+zddnqshKwMGuCSb97JC-=3==TwFeEGZP05w@mail.gmail.com>
Subject: Re: [RFC PATCH] x86_64: Add "-m elf_i386" when linking i386 object files.
To:     Nick Desaulniers <ndesaulniers@google.com>
Cc:     bp@alien8.de, x86@kernel.org, tglx@linutronix.de, mingo@redhat.com,
        grimar@accesssoftek.com, Dmitry Golovin <dima@golovin.in>,
        Bill Wendling <morbo@google.com>, hpa@zytor.com,
        linux-kernel@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Adding appropriate people, lists.
On Tue, Dec 11, 2018 at 1:07 PM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> On Mon, Dec 10, 2018 at 2:50 PM Nick Desaulniers
> <ndesaulniers@google.com> wrote:
> >
> > On Mon, Dec 10, 2018 at 2:26 PM Tri Vo <trong@android.com> wrote:
> > >
> > > From: George Rimar <grimar@accesssoftek.com>
> > >
> > > Linux kernel uses OUTPUT_FORMAT in it's linker scripts. Most of the time
> > > -m option is passed to the linker with correct architecture, but
> > > sometimes (at least for x86_64) the -m option contradicts OUTPUT_FORMAT
> > > directive. Specifically, arch/x86/boot and arch/x86/realmode/rm modules
> > > have i386 object files, but are linked with -m elf_x86_64 linker flag
> > > when building for x86_64.
> > >
> > > BFD and Gold linkers are OK with this, but lld fails:
> > > ld.lld: error: arch/x86/realmode/rm/header.o is incompatible with elf_x86_64
> > >
> > > Suggested fix: just add correct -m after incorrect one (it overrides
> > > it), so the linker invocation looks like this: ld -m elf_x86_64 -z
> > > max-page-size=0x200000 -m elf_i386 --emit-relocs -T realmode.lds
> > > header.o trampoline_64.o stack.o reboot.o -o realmode.elf (it will also
> > > work with GNU ld, because it supports OUTPUT_FORMAT and just ignores -m
> > > options if this directive is in the linker script).
> > >
> > > Tested by building x86_64 kernel with GNU gcc/ld toolchain and booting
> > > it in QEMU.
> > >
> > > Suggested-by: Dmitry Golovin <dima@golovin.in>
> > > Signed-off-by: Tri Vo <trong@android.com>
> > > Tested-by: Tri Vo <trong@android.com>
> >
> > This fixes the following linkage error I observe when linking an x86
> > kernel with LLD:
> >
> > ```
> >   LD      arch/x86/realmode/rm/realmode.elf
> > ld.lld: error: arch/x86/realmode/rm/header.o is incompatible with elf_x86_64
> > ld.lld: error: arch/x86/realmode/rm/trampoline_64.o is incompatible
> > with elf_x86_64
> > ld.lld: error: arch/x86/realmode/rm/stack.o is incompatible with elf_x86_64
> > ld.lld: error: arch/x86/realmode/rm/reboot.o is incompatible with elf_x86_64
> > ld.lld: error: arch/x86/realmode/rm/wakeup_asm.o is incompatible with elf_x86_64
> > ld.lld: error: arch/x86/realmode/rm/wakemain.o is incompatible with elf_x86_64
> > ld.lld: error: arch/x86/realmode/rm/video-mode.o is incompatible with elf_x86_64
> > ld.lld: error: arch/x86/realmode/rm/copy.o is incompatible with elf_x86_64
> > ld.lld: error: arch/x86/realmode/rm/bioscall.o is incompatible with elf_x86_64
> > ld.lld: error: arch/x86/realmode/rm/regs.o is incompatible with elf_x86_64
> > ld.lld: error: arch/x86/realmode/rm/video-vga.o is incompatible with elf_x86_64
> > ld.lld: error: arch/x86/realmode/rm/video-vesa.o is incompatible with elf_x86_64
> > ld.lld: error: arch/x86/realmode/rm/video-bios.o is incompatible with elf_x86_64
> > arch/x86/realmode/rm/Makefile:55: recipe for target
> > 'arch/x86/realmode/rm/realmode.elf' failed
> > ```
> > Tested-by: Nick Desaulniers <ndesaulniers@google.com>
> >
> > Looks like we still have a few other (unrelated) issues to track down
> > with LLD, but this gets us one step closer.
> >
> > Thanks for sending this, Tri!
>
> Just some additional thoughts on this:
>
> The kernel is adding -m elf_x86_64 to the linker command line
> parameters, but then relying on binutils tie-breaking-behavior by
> specifying a different architecture via linker script. So it's
> ambiguous which arch the kernel is trying to link for, and the kernel
> gets lucky/relies on the way ld.bfd happens to resolve this ambiguity
> without warning. An alternative fix may be to just filter out/remove
> the -m elf_x86_64 for this target's linker flags.
>
> >
> > > ---
> > >  arch/x86/boot/Makefile        | 2 +-
> > >  arch/x86/realmode/rm/Makefile | 2 +-
> > >  2 files changed, 2 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/arch/x86/boot/Makefile b/arch/x86/boot/Makefile
> > > index 9b5adae9cc40..e2839b5c246c 100644
> > > --- a/arch/x86/boot/Makefile
> > > +++ b/arch/x86/boot/Makefile
> > > @@ -100,7 +100,7 @@ $(obj)/zoffset.h: $(obj)/compressed/vmlinux FORCE
> > >  AFLAGS_header.o += -I$(objtree)/$(obj)
> > >  $(obj)/header.o: $(obj)/zoffset.h
> > >
> > > -LDFLAGS_setup.elf      := -T
> > > +LDFLAGS_setup.elf      := -m elf_i386 -T
> > >  $(obj)/setup.elf: $(src)/setup.ld $(SETUP_OBJS) FORCE
> > >         $(call if_changed,ld)
> > >
> > > diff --git a/arch/x86/realmode/rm/Makefile b/arch/x86/realmode/rm/Makefile
> > > index 4463fa72db94..96cb20de08af 100644
> > > --- a/arch/x86/realmode/rm/Makefile
> > > +++ b/arch/x86/realmode/rm/Makefile
> > > @@ -47,7 +47,7 @@ $(obj)/pasyms.h: $(REALMODE_OBJS) FORCE
> > >  targets += realmode.lds
> > >  $(obj)/realmode.lds: $(obj)/pasyms.h
> > >
> > > -LDFLAGS_realmode.elf := --emit-relocs -T
> > > +LDFLAGS_realmode.elf := -m elf_i386 --emit-relocs -T
> > >  CPPFLAGS_realmode.lds += -P -C -I$(objtree)/$(obj)
> > >
> > >  targets += realmode.elf
> > > --
> > > 2.20.0.rc2.403.gdbc3b29805-goog
> > >
> >
> >
> > --
> > Thanks,
> > ~Nick Desaulniers
>
>
>
> --
> Thanks,
> ~Nick Desaulniers