2022-06-02 02:21:22

by Stephen Boyd

[permalink] [raw]
Subject: [PATCH] arm64: Initialize jump labels before setup_machine_fdt()

A static key warning splat appears during early boot on arm64 systems
that credit randomness from devicetrees that contain an "rng-seed"
property. This is because setup_machine_fdt() is called before
jump_label_init() during setup_arch(). Let's swap the order of these two
calls so that jump labels are initialized before the devicetree is
unflattened and the rng seed is credited.

static_key_enable_cpuslocked(): static key '0xffffffe51c6fcfc0' used before call to jump_label_init()
WARNING: CPU: 0 PID: 0 at kernel/jump_label.c:166 static_key_enable_cpuslocked+0xb0/0xb8
Modules linked in:
CPU: 0 PID: 0 Comm: swapper Not tainted 5.18.0+ #224 44b43e377bfc84bc99bb5ab885ff694984ee09ff
pstate: 600001c9 (nZCv dAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : static_key_enable_cpuslocked+0xb0/0xb8
lr : static_key_enable_cpuslocked+0xb0/0xb8
sp : ffffffe51c393cf0
x29: ffffffe51c393cf0 x28: 000000008185054c x27: 00000000f1042f10
x26: 0000000000000000 x25: 00000000f10302b2 x24: 0000002513200000
x23: 0000002513200000 x22: ffffffe51c1c9000 x21: fffffffdfdc00000
x20: ffffffe51c2f0831 x19: ffffffe51c6fcfc0 x18: 00000000ffff1020
x17: 00000000e1e2ac90 x16: 00000000000000e0 x15: ffffffe51b710708
x14: 0000000000000066 x13: 0000000000000018 x12: 0000000000000000
x11: 0000000000000000 x10: 00000000ffffffff x9 : 0000000000000000
x8 : 0000000000000000 x7 : 61632065726f6665 x6 : 6220646573752027
x5 : ffffffe51c641d25 x4 : ffffffe51c13142c x3 : ffff0a00ffffff05
x2 : 40000000ffffe003 x1 : 00000000000001c0 x0 : 0000000000000065
Call trace:
static_key_enable_cpuslocked+0xb0/0xb8
static_key_enable+0x2c/0x40
crng_set_ready+0x24/0x30
execute_in_process_context+0x80/0x90
_credit_init_bits+0x100/0x154
add_bootloader_randomness+0x64/0x78
early_init_dt_scan_chosen+0x140/0x184
early_init_dt_scan_nodes+0x28/0x4c
early_init_dt_scan+0x40/0x44
setup_machine_fdt+0x7c/0x120
setup_arch+0x74/0x1d8
start_kernel+0x84/0x44c
__primary_switched+0xc0/0xc8
---[ end trace 0000000000000000 ]---
random: crng init done
Machine model: Google Lazor (rev1 - 2) with LTE

Cc: Hsin-Yi Wang <[email protected]>
Cc: Douglas Anderson <[email protected]>
Cc: Ard Biesheuvel <[email protected]>
Cc: Steven Rostedt <[email protected]>
Cc: Jason A. Donenfeld <[email protected]>
Cc: Dominik Brodowski <[email protected]>
Fixes: f5bda35fba61 ("random: use static branch for crng_ready()")
Signed-off-by: Stephen Boyd <[email protected]>
---
arch/arm64/kernel/setup.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
index fea3223704b6..cf3a759f10d4 100644
--- a/arch/arm64/kernel/setup.c
+++ b/arch/arm64/kernel/setup.c
@@ -303,13 +303,14 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
early_fixmap_init();
early_ioremap_init();

- setup_machine_fdt(__fdt_pointer);
-
/*
* Initialise the static keys early as they may be enabled by the
- * cpufeature code and early parameters.
+ * cpufeature code, early parameters, and DT setup.
*/
jump_label_init();
+
+ setup_machine_fdt(__fdt_pointer);
+
parse_early_param();

/*
--
https://chromeos.dev



2022-06-02 09:17:51

by Jason A. Donenfeld

[permalink] [raw]
Subject: Re: [PATCH] arm64: Initialize jump labels before setup_machine_fdt()

Hi Stephen,

On 6/2/22, Stephen Boyd <[email protected]> wrote:
> A static key warning splat appears during early boot on arm64 systems
> that credit randomness from devicetrees that contain an "rng-seed"
> property. This is because setup_machine_fdt() is called before
> jump_label_init() during setup_arch(). Let's swap the order of these two
> calls so that jump labels are initialized before the devicetree is
> unflattened and the rng seed is credited.
>
> static_key_enable_cpuslocked(): static key '0xffffffe51c6fcfc0' used before
> call to jump_label_init()
> WARNING: CPU: 0 PID: 0 at kernel/jump_label.c:166
> static_key_enable_cpuslocked+0xb0/0xb8
> Modules linked in:
> CPU: 0 PID: 0 Comm: swapper Not tainted 5.18.0+ #224
> 44b43e377bfc84bc99bb5ab885ff694984ee09ff
> pstate: 600001c9 (nZCv dAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> pc : static_key_enable_cpuslocked+0xb0/0xb8
> lr : static_key_enable_cpuslocked+0xb0/0xb8
> sp : ffffffe51c393cf0
> x29: ffffffe51c393cf0 x28: 000000008185054c x27: 00000000f1042f10
> x26: 0000000000000000 x25: 00000000f10302b2 x24: 0000002513200000
> x23: 0000002513200000 x22: ffffffe51c1c9000 x21: fffffffdfdc00000
> x20: ffffffe51c2f0831 x19: ffffffe51c6fcfc0 x18: 00000000ffff1020
> x17: 00000000e1e2ac90 x16: 00000000000000e0 x15: ffffffe51b710708
> x14: 0000000000000066 x13: 0000000000000018 x12: 0000000000000000
> x11: 0000000000000000 x10: 00000000ffffffff x9 : 0000000000000000
> x8 : 0000000000000000 x7 : 61632065726f6665 x6 : 6220646573752027
> x5 : ffffffe51c641d25 x4 : ffffffe51c13142c x3 : ffff0a00ffffff05
> x2 : 40000000ffffe003 x1 : 00000000000001c0 x0 : 0000000000000065
> Call trace:
> static_key_enable_cpuslocked+0xb0/0xb8
> static_key_enable+0x2c/0x40
> crng_set_ready+0x24/0x30
> execute_in_process_context+0x80/0x90
> _credit_init_bits+0x100/0x154
> add_bootloader_randomness+0x64/0x78
> early_init_dt_scan_chosen+0x140/0x184
> early_init_dt_scan_nodes+0x28/0x4c
> early_init_dt_scan+0x40/0x44
> setup_machine_fdt+0x7c/0x120
> setup_arch+0x74/0x1d8
> start_kernel+0x84/0x44c
> __primary_switched+0xc0/0xc8
> ---[ end trace 0000000000000000 ]---
> random: crng init done
> Machine model: Google Lazor (rev1 - 2) with LTE
>
> Cc: Hsin-Yi Wang <[email protected]>
> Cc: Douglas Anderson <[email protected]>
> Cc: Ard Biesheuvel <[email protected]>
> Cc: Steven Rostedt <[email protected]>
> Cc: Jason A. Donenfeld <[email protected]>
> Cc: Dominik Brodowski <[email protected]>
> Fixes: f5bda35fba61 ("random: use static branch for crng_ready()")
> Signed-off-by: Stephen Boyd <[email protected]>

Thanks for fixing this.

Reviewed-by: Jason A. Donenfeld <[email protected]>

In looking at the matter more in depth, it looks like riscv does
things in the right order, but arm32 still does not. AFAICT, it uses
the logic in init/main.c, in which jump_label_init() is called before
setup_arch()->setup_machine_fdt(). Want to submit a patch for arm32?

Jason

2022-06-03 18:04:44

by Catalin Marinas

[permalink] [raw]
Subject: Re: [PATCH] arm64: Initialize jump labels before setup_machine_fdt()

On Wed, 1 Jun 2022 19:21:09 -0700, Stephen Boyd wrote:
> A static key warning splat appears during early boot on arm64 systems
> that credit randomness from devicetrees that contain an "rng-seed"
> property. This is because setup_machine_fdt() is called before
> jump_label_init() during setup_arch(). Let's swap the order of these two
> calls so that jump labels are initialized before the devicetree is
> unflattened and the rng seed is credited.
>
> [...]

Applied to arm64 (for-next/core), thanks!

[1/1] arm64: Initialize jump labels before setup_machine_fdt()
https://git.kernel.org/arm64/c/73e2d827a501

--
Catalin

2022-06-03 21:49:50

by Jason A. Donenfeld

[permalink] [raw]
Subject: Re: [PATCH] arm64: Initialize jump labels before setup_machine_fdt()

On Thu, Jun 2, 2022 at 9:46 PM Catalin Marinas <[email protected]> wrote:
>
> Hi Jason,
>
> On Thu, Jun 02, 2022 at 09:42:49PM +0200, Jason A. Donenfeld wrote:
> > On Thu, Jun 2, 2022 at 9:41 PM Catalin Marinas <[email protected]> wrote:
> > > On Wed, 1 Jun 2022 19:21:09 -0700, Stephen Boyd wrote:
> > > > A static key warning splat appears during early boot on arm64 systems
> > > > that credit randomness from devicetrees that contain an "rng-seed"
> > > > property. This is because setup_machine_fdt() is called before
> > > > jump_label_init() during setup_arch(). Let's swap the order of these two
> > > > calls so that jump labels are initialized before the devicetree is
> > > > unflattened and the rng seed is credited.
> > > >
> > > > [...]
> > >
> > > Applied to arm64 (for-next/core), thanks!
> > >
> > > [1/1] arm64: Initialize jump labels before setup_machine_fdt()
> > > https://git.kernel.org/arm64/c/73e2d827a501
> >
> > I'm not sure precisely what your for-next tag corresponds to, but just
> > FYI, this should probably go into 5.19 asap and be marked for stable@.
>
> The for-next/core branch should go in 5.19-rc1 (I'll send a pull request
> tomorrow). There's no need to cc stable since the fixed commit just went
> in (unless you mean something else by "marked for stable").

It is actually in stable, so please do CC. Otherwise I'll just forward manually.

Jason

2022-06-04 12:06:07

by Jason A. Donenfeld

[permalink] [raw]
Subject: Re: [PATCH] arm64: Initialize jump labels before setup_machine_fdt()

Hi Catalin,

On Thu, Jun 2, 2022 at 9:41 PM Catalin Marinas <[email protected]> wrote:
>
> On Wed, 1 Jun 2022 19:21:09 -0700, Stephen Boyd wrote:
> > A static key warning splat appears during early boot on arm64 systems
> > that credit randomness from devicetrees that contain an "rng-seed"
> > property. This is because setup_machine_fdt() is called before
> > jump_label_init() during setup_arch(). Let's swap the order of these two
> > calls so that jump labels are initialized before the devicetree is
> > unflattened and the rng seed is credited.
> >
> > [...]
>
> Applied to arm64 (for-next/core), thanks!
>
> [1/1] arm64: Initialize jump labels before setup_machine_fdt()
> https://git.kernel.org/arm64/c/73e2d827a501

I'm not sure precisely what your for-next tag corresponds to, but just
FYI, this should probably go into 5.19 asap and be marked for stable@.

Jason

2022-06-06 04:45:57

by Jason A. Donenfeld

[permalink] [raw]
Subject: Re: [PATCH] arm64: Initialize jump labels before setup_machine_fdt()

+Greg

Hi Catalin,

On Thu, Jun 2, 2022 at 11:08 PM Catalin Marinas <[email protected]> wrote:
>
> On Thu, Jun 02, 2022 at 10:18:04PM +0200, Jason A. Donenfeld wrote:
> > On Thu, Jun 2, 2022 at 9:46 PM Catalin Marinas <[email protected]> wrote:
> > > On Thu, Jun 02, 2022 at 09:42:49PM +0200, Jason A. Donenfeld wrote:
> > > > On Thu, Jun 2, 2022 at 9:41 PM Catalin Marinas <[email protected]> wrote:
> > > > > On Wed, 1 Jun 2022 19:21:09 -0700, Stephen Boyd wrote:
> > > > > > A static key warning splat appears during early boot on arm64 systems
> > > > > > that credit randomness from devicetrees that contain an "rng-seed"
> > > > > > property. This is because setup_machine_fdt() is called before
> > > > > > jump_label_init() during setup_arch(). Let's swap the order of these two
> > > > > > calls so that jump labels are initialized before the devicetree is
> > > > > > unflattened and the rng seed is credited.
> > > > > >
> > > > > > [...]
> > > > >
> > > > > Applied to arm64 (for-next/core), thanks!
> > > > >
> > > > > [1/1] arm64: Initialize jump labels before setup_machine_fdt()
> > > > > https://git.kernel.org/arm64/c/73e2d827a501
> > > >
> > > > I'm not sure precisely what your for-next tag corresponds to, but just
> > > > FYI, this should probably go into 5.19 asap and be marked for stable@.
> > >
> > > The for-next/core branch should go in 5.19-rc1 (I'll send a pull request
> > > tomorrow). There's no need to cc stable since the fixed commit just went
> > > in (unless you mean something else by "marked for stable").
> >
> > It is actually in stable, so please do CC. Otherwise I'll just forward manually.
>
> Ah, I didn't know this (the fixed commit doesn't have a cc stable
> either). So please forward it manually once Linus pulls it.

Will do.

Jason

2022-06-06 05:05:12

by Catalin Marinas

[permalink] [raw]
Subject: Re: [PATCH] arm64: Initialize jump labels before setup_machine_fdt()

Hi Jason,

On Thu, Jun 02, 2022 at 09:42:49PM +0200, Jason A. Donenfeld wrote:
> On Thu, Jun 2, 2022 at 9:41 PM Catalin Marinas <[email protected]> wrote:
> > On Wed, 1 Jun 2022 19:21:09 -0700, Stephen Boyd wrote:
> > > A static key warning splat appears during early boot on arm64 systems
> > > that credit randomness from devicetrees that contain an "rng-seed"
> > > property. This is because setup_machine_fdt() is called before
> > > jump_label_init() during setup_arch(). Let's swap the order of these two
> > > calls so that jump labels are initialized before the devicetree is
> > > unflattened and the rng seed is credited.
> > >
> > > [...]
> >
> > Applied to arm64 (for-next/core), thanks!
> >
> > [1/1] arm64: Initialize jump labels before setup_machine_fdt()
> > https://git.kernel.org/arm64/c/73e2d827a501
>
> I'm not sure precisely what your for-next tag corresponds to, but just
> FYI, this should probably go into 5.19 asap and be marked for stable@.

The for-next/core branch should go in 5.19-rc1 (I'll send a pull request
tomorrow). There's no need to cc stable since the fixed commit just went
in (unless you mean something else by "marked for stable").

--
Catalin

2022-06-06 06:19:09

by Catalin Marinas

[permalink] [raw]
Subject: Re: [PATCH] arm64: Initialize jump labels before setup_machine_fdt()

On Thu, Jun 02, 2022 at 10:18:04PM +0200, Jason A. Donenfeld wrote:
> On Thu, Jun 2, 2022 at 9:46 PM Catalin Marinas <[email protected]> wrote:
> > On Thu, Jun 02, 2022 at 09:42:49PM +0200, Jason A. Donenfeld wrote:
> > > On Thu, Jun 2, 2022 at 9:41 PM Catalin Marinas <[email protected]> wrote:
> > > > On Wed, 1 Jun 2022 19:21:09 -0700, Stephen Boyd wrote:
> > > > > A static key warning splat appears during early boot on arm64 systems
> > > > > that credit randomness from devicetrees that contain an "rng-seed"
> > > > > property. This is because setup_machine_fdt() is called before
> > > > > jump_label_init() during setup_arch(). Let's swap the order of these two
> > > > > calls so that jump labels are initialized before the devicetree is
> > > > > unflattened and the rng seed is credited.
> > > > >
> > > > > [...]
> > > >
> > > > Applied to arm64 (for-next/core), thanks!
> > > >
> > > > [1/1] arm64: Initialize jump labels before setup_machine_fdt()
> > > > https://git.kernel.org/arm64/c/73e2d827a501
> > >
> > > I'm not sure precisely what your for-next tag corresponds to, but just
> > > FYI, this should probably go into 5.19 asap and be marked for stable@.
> >
> > The for-next/core branch should go in 5.19-rc1 (I'll send a pull request
> > tomorrow). There's no need to cc stable since the fixed commit just went
> > in (unless you mean something else by "marked for stable").
>
> It is actually in stable, so please do CC. Otherwise I'll just forward manually.

Ah, I didn't know this (the fixed commit doesn't have a cc stable
either). So please forward it manually once Linus pulls it.

Thanks.

--
Catalin