2023-11-19 05:35:15

by Masahiro Yamada

[permalink] [raw]
Subject: [PATCH] arm64: add dependency between vmlinuz.efi and Image

A common issue in Makefile is a race in parallel building.

You need to be careful to prevent multiple threads from writing to the
same file simultaneously.

Commit 3939f3345050 ("ARM: 8418/1: add boot image dependencies to not
generate invalid images") addressed such a bad scenario.

A similar symptom occurs with the following command:

$ make -j$(nproc) ARCH=arm64 Image vmlinuz.efi
[ snip ]
SORTTAB vmlinux
OBJCOPY arch/arm64/boot/Image
OBJCOPY arch/arm64/boot/Image
AS arch/arm64/boot/zboot-header.o
PAD arch/arm64/boot/vmlinux.bin
GZIP arch/arm64/boot/vmlinuz
OBJCOPY arch/arm64/boot/vmlinuz.o
LD arch/arm64/boot/vmlinuz.efi.elf
OBJCOPY arch/arm64/boot/vmlinuz.efi

The log "OBJCOPY arch/arm64/boot/Image" is displayed twice.

It indicates that two threads simultaneously enter arch/arm64/boot/
and write to arch/arm64/boot/Image.

It occasionally leads to a build failure:

$ make -j$(nproc) ARCH=arm64 Image vmlinuz.efi
[ snip ]
SORTTAB vmlinux
OBJCOPY arch/arm64/boot/Image
PAD arch/arm64/boot/vmlinux.bin
truncate: Invalid number: 'arch/arm64/boot/vmlinux.bin'
make[2]: *** [drivers/firmware/efi/libstub/Makefile.zboot:13:
arch/arm64/boot/vmlinux.bin] Error 1
make[2]: *** Deleting file 'arch/arm64/boot/vmlinux.bin'
make[1]: *** [arch/arm64/Makefile:163: vmlinuz.efi] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:234: __sub-make] Error 2

vmlinuz.efi depends on Image, but such a dependency is not specified
in arch/arm64/Makefile.

Signed-off-by: Masahiro Yamada <[email protected]>
---

arch/arm64/Makefile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 4a1ad3248c2d..47ecc4cff9d2 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -158,7 +158,7 @@ endif

all: $(notdir $(KBUILD_IMAGE))

-
+vmlinuz.efi: Image
Image vmlinuz.efi: vmlinux
$(Q)$(MAKE) $(build)=$(boot) $(boot)/$@

--
2.40.1


2023-11-21 16:37:25

by Ard Biesheuvel

[permalink] [raw]
Subject: Re: [PATCH] arm64: add dependency between vmlinuz.efi and Image

On Sun, 19 Nov 2023 at 00:32, Masahiro Yamada <[email protected]> wrote:
>
> A common issue in Makefile is a race in parallel building.
>
> You need to be careful to prevent multiple threads from writing to the
> same file simultaneously.
>
> Commit 3939f3345050 ("ARM: 8418/1: add boot image dependencies to not
> generate invalid images") addressed such a bad scenario.
>
> A similar symptom occurs with the following command:
>
> $ make -j$(nproc) ARCH=arm64 Image vmlinuz.efi
> [ snip ]
> SORTTAB vmlinux
> OBJCOPY arch/arm64/boot/Image
> OBJCOPY arch/arm64/boot/Image
> AS arch/arm64/boot/zboot-header.o
> PAD arch/arm64/boot/vmlinux.bin
> GZIP arch/arm64/boot/vmlinuz
> OBJCOPY arch/arm64/boot/vmlinuz.o
> LD arch/arm64/boot/vmlinuz.efi.elf
> OBJCOPY arch/arm64/boot/vmlinuz.efi
>
> The log "OBJCOPY arch/arm64/boot/Image" is displayed twice.
>
> It indicates that two threads simultaneously enter arch/arm64/boot/
> and write to arch/arm64/boot/Image.
>
> It occasionally leads to a build failure:
>
> $ make -j$(nproc) ARCH=arm64 Image vmlinuz.efi
> [ snip ]
> SORTTAB vmlinux
> OBJCOPY arch/arm64/boot/Image
> PAD arch/arm64/boot/vmlinux.bin
> truncate: Invalid number: 'arch/arm64/boot/vmlinux.bin'
> make[2]: *** [drivers/firmware/efi/libstub/Makefile.zboot:13:
> arch/arm64/boot/vmlinux.bin] Error 1
> make[2]: *** Deleting file 'arch/arm64/boot/vmlinux.bin'
> make[1]: *** [arch/arm64/Makefile:163: vmlinuz.efi] Error 2
> make[1]: *** Waiting for unfinished jobs....
> make: *** [Makefile:234: __sub-make] Error 2
>
> vmlinuz.efi depends on Image, but such a dependency is not specified
> in arch/arm64/Makefile.
>

To clarify, this dependency /is/ specified in
arch/arm64/boot/Makefile, which is consumed by explicit make
invocations from arch/arm64/Makefile, and these may end up racing with
each other.


> Signed-off-by: Masahiro Yamada <[email protected]>

Acked-by: Ard Biesheuvel <[email protected]>

> ---
>
> arch/arm64/Makefile | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
> index 4a1ad3248c2d..47ecc4cff9d2 100644
> --- a/arch/arm64/Makefile
> +++ b/arch/arm64/Makefile
> @@ -158,7 +158,7 @@ endif
>
> all: $(notdir $(KBUILD_IMAGE))
>
> -
> +vmlinuz.efi: Image
> Image vmlinuz.efi: vmlinux
> $(Q)$(MAKE) $(build)=$(boot) $(boot)/$@
>
> --
> 2.40.1
>

2023-11-21 22:11:12

by Simon Glass

[permalink] [raw]
Subject: Re: [PATCH] arm64: add dependency between vmlinuz.efi and Image

On Sat, 18 Nov 2023 at 22:32, Masahiro Yamada <[email protected]> wrote:
>
> A common issue in Makefile is a race in parallel building.
>
> You need to be careful to prevent multiple threads from writing to the
> same file simultaneously.
>
> Commit 3939f3345050 ("ARM: 8418/1: add boot image dependencies to not
> generate invalid images") addressed such a bad scenario.
>
> A similar symptom occurs with the following command:
>
> $ make -j$(nproc) ARCH=arm64 Image vmlinuz.efi
> [ snip ]
> SORTTAB vmlinux
> OBJCOPY arch/arm64/boot/Image
> OBJCOPY arch/arm64/boot/Image
> AS arch/arm64/boot/zboot-header.o
> PAD arch/arm64/boot/vmlinux.bin
> GZIP arch/arm64/boot/vmlinuz
> OBJCOPY arch/arm64/boot/vmlinuz.o
> LD arch/arm64/boot/vmlinuz.efi.elf
> OBJCOPY arch/arm64/boot/vmlinuz.efi
>
> The log "OBJCOPY arch/arm64/boot/Image" is displayed twice.
>
> It indicates that two threads simultaneously enter arch/arm64/boot/
> and write to arch/arm64/boot/Image.
>
> It occasionally leads to a build failure:
>
> $ make -j$(nproc) ARCH=arm64 Image vmlinuz.efi
> [ snip ]
> SORTTAB vmlinux
> OBJCOPY arch/arm64/boot/Image
> PAD arch/arm64/boot/vmlinux.bin
> truncate: Invalid number: 'arch/arm64/boot/vmlinux.bin'
> make[2]: *** [drivers/firmware/efi/libstub/Makefile.zboot:13:
> arch/arm64/boot/vmlinux.bin] Error 1
> make[2]: *** Deleting file 'arch/arm64/boot/vmlinux.bin'
> make[1]: *** [arch/arm64/Makefile:163: vmlinuz.efi] Error 2
> make[1]: *** Waiting for unfinished jobs....
> make: *** [Makefile:234: __sub-make] Error 2
>
> vmlinuz.efi depends on Image, but such a dependency is not specified
> in arch/arm64/Makefile.
>
> Signed-off-by: Masahiro Yamada <[email protected]>
> ---
>
> arch/arm64/Makefile | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)

Reviewed-by: SImon Glass <[email protected]>

2023-11-23 19:30:15

by Catalin Marinas

[permalink] [raw]
Subject: Re: [PATCH] arm64: add dependency between vmlinuz.efi and Image

On Sun, 19 Nov 2023 14:32:34 +0900, Masahiro Yamada wrote:
> A common issue in Makefile is a race in parallel building.
>
> You need to be careful to prevent multiple threads from writing to the
> same file simultaneously.
>
> Commit 3939f3345050 ("ARM: 8418/1: add boot image dependencies to not
> generate invalid images") addressed such a bad scenario.
>
> [...]

Applied to arm64 (for-next/fixes), thanks!

[1/1] arm64: add dependency between vmlinuz.efi and Image
https://git.kernel.org/arm64/c/c0a857420405

--
Catalin