Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20230527-hyperlink-doctrine-ef22cfcb508a@spud>
 <CAHVXubgx3uBEjMLHXTxr0192ZHbSb=qK4NggZyWQTDfgrJt2-g@mail.gmail.com>
 <20230528-darkness-grandly-6cb9e014391d@spud> <cf0d2d2a-c407-7b3d-a5ab-ea5c19e7b890@ghiti.fr>
 <20230528-uneatable-earpiece-3f8673548863@spud> <b71dc2f5-fdc0-2a8d-e1f9-696cd9a1529e@ghiti.fr>
 <20230529-skillet-quarters-3fbc3b6edb3a@spud> <41e57bb9-ce0c-7772-abeb-5c01d5ec19bb@ghiti.fr>
 <20230530-polka-trifle-7ccd7a093099@wendy> <92d4aaa8-a1ed-74e1-3a22-df9be1ca1e4a@ghiti.fr>
 <20230530-hatchery-unifier-64d7a2ffe0d0@spud>
In-Reply-To: <20230530-hatchery-unifier-64d7a2ffe0d0@spud>
From:   Alexandre Ghiti <alexghiti@rivosinc.com>
Date:   Tue, 30 May 2023 20:04:17 +0200
Message-ID: <CAHVXubgG31moSNOe3fRqFzUSJK9tRWKH1KPP_BO7wRNC4WzxTQ@mail.gmail.com>
Subject: Re: [PATCH -fixes] riscv: Fix relocatable kernels with early
 alternatives using -fno-pie
To:     Conor Dooley <conor@kernel.org>
Cc:     Alexandre Ghiti <alex@ghiti.fr>,
        Conor Dooley <conor.dooley@microchip.com>,
        Paul Walmsley <paul.walmsley@sifive.com>,
        Palmer Dabbelt <palmer@dabbelt.com>,
        Albert Ou <aou@eecs.berkeley.edu>,
        Andreas Schwab <schwab@linux-m68k.org>,
        linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Tue, May 30, 2023 at 7:47=E2=80=AFPM Conor Dooley <conor@kernel.org> wro=
te:
>
> On Tue, May 30, 2023 at 04:33:45PM +0200, Alexandre Ghiti wrote:
> >
> > On 30/05/2023 13:27, Conor Dooley wrote:
> > > On Mon, May 29, 2023 at 09:37:28PM +0200, Alexandre Ghiti wrote:
> > > > On 29/05/2023 21:06, Conor Dooley wrote:
> > > > > On Mon, May 29, 2023 at 08:51:57PM +0200, Alexandre Ghiti wrote:
> > > > > > On 28/05/2023 15:56, Conor Dooley wrote:
> > > > > > > On Sun, May 28, 2023 at 03:42:59PM +0200, Alexandre Ghiti wro=
te:
> > > > > > > > Hmmm, it still works for me with both clang and gcc-9.
> > > > > > > gcc-9 is a bit of a relic, do you have more recent compilers =
lying
> > > > > > > around? If not, I can try some older compilers at some point.
> > > > > > >
> > > > > > > > You don't have to do that now but is there a way I could ge=
t your compiled
> > > > > > > > image? With the sha1 used to build it? Sorry, I don't see w=
hat happens, I
> > > > > > > > need to get my hands dirty in some debug!
> > > > > > > What do you mean by "sha1"? It falls with v6.4-rc1 which is a=
 stable
> > > > > > > hash, if that's what you're looking for.
> > > > > > >
> > > > > > > Otherwise,
> > > > > > > https://github.com/ConchuOD/riscv-env/releases/download/v2022=
.03/vmlinux.bin
> > > > > > > (ignore the release crap haha, too lazy to find a proper host=
ing
> > > > > > > mechanism)
> > > > > > Ok, I don't get much info without the symbols, can you also pro=
vide the
> > > > > > vmlinux please? But at least your image does not boot, not duri=
ng the early
> > > > > > boot though because the mmu is enabled.
> > > > > Do you see anything print when you try it? Cos I do not. Iff I ha=
ve time
> > > > > tomorrow, I'll go poking with gdb. I'm sorry I have not really do=
ne any
> > > > > investigating, I have been really busy this last week or so with
> > > > > dt-binding stuff but I should be freer again from tomorrow.
> > > > >
> > > > > https://github.com/ConchuOD/riscv-env/releases/download/v2022.03/=
vmlinux
> > > >
> > > > Better, the trap happens in kasan_early_init() when it tries to acc=
ess a
> > > > global symbol using the GOT but ends up with a NULL pointer, which =
is weird.
> > > > So to me, this is not related to kasan, it happens that kasan_early=
_init()
> > > > is the first function called after enabling the mmu, I think you ma=
y have an
> > > > issue with the filling of the relocations.
> > > Yeah, it reproduces without KASAN.
> > >
> > > > Sorry to bother you again, but if
> > > > at some point you can recompile with DEBUG_INFO enabled, that would=
 be
> > > > perfect! And also provide the vmlinux.relocs file. Sorry for all th=
at, too
> > > > bad I can't reproduce it.
> > > New vmlinux & vmlinux.relocs here:
> > > https://microchiptechnology-my.sharepoint.com/:u:/g/personal/conor_do=
oley_microchip_com/EZpFNxYYrnNAh5Z3c-rf0pUBBpdPGTLafqdtfcXRUUBkXw?e=3D7KKMH=
X
> > > They're pretty massive unfortunately & hopefully that is not some
> > > garbage internal-only link.
> > > .config is a wee bit different, cos different build machine, but the
> > > problem still manifests on a icicle. I've added it to the tarball jus=
t
> > > in case.
> >
> >
> > Ok so I had to recreate the Image from the files you gave me and it boo=
ts
> > fine using qemu: is that expected? Because you only mention the icicle
> > above.
>
> Unfortunately you sent this one right as I left work..
> I ssh'ed in though and ran the vmlinux.bin & had the same issues.
> Silly question perhaps - is it just not possible to boot something that
> has been hit with `objcopy -O binary vmlinux vmlinux.bin` with
> CONFIG_RELOCATABLE? At this point that's the main thing that sticks out
> to me as being different. You couldn't boot the vmlinux.bin that I sent
> you either.

Ahah, I think we found the culprit!

With CONFIG_RELOCATABLE, vmlinux is actually stripped from all the
relocations (so that it can be shipped) and vmlinux.relocs is what you
should use instead, since it is just a copy of vmlinux before the
removal of the relocations!

>
> Cheers,
> Conor.