2023-09-16 12:06:23

by Thomas Weißschuh

[permalink] [raw]
Subject: [PATCH RFC 0/3] selftests/nolibc: avoid spurious kernel relinks

Currently the nolibc testsuite embeds the test executable into a kernel
image with CONFIG_INITRAMFS_SOURCE.
This forces a full kernel relink everytime the test executable is
updated.

This relinking step dominates the test cycle.
It is slower than building and running the test in qemu together.

With a bit of Makefile-shuffling the relinking can be avoided.

Signed-off-by: Thomas Weißschuh <[email protected]>
---
Thomas Weißschuh (3):
kbuild: add toplevel target for usr/gen_init_cpio
selftests/nolibc: don't embed initramfs into kernel image
selftests/nolibc: drop target "rerun"

Makefile | 4 +++
tools/testing/selftests/nolibc/Makefile | 50 +++++++++++++++------------------
2 files changed, 27 insertions(+), 27 deletions(-)
---
base-commit: 3f79a57865b33f49fdae6655510bd27c8e6610e0
change-id: 20230916-nolibc-initramfs-4fd00eac3256

Best regards,
--
Thomas Weißschuh <[email protected]>


2023-09-16 12:31:13

by Thomas Weißschuh

[permalink] [raw]
Subject: [PATCH RFC 2/3] selftests/nolibc: don't embed initramfs into kernel image

When the initramfs is embedded into the kernel each rebuild of it will
trigger a full kernel relink and all the expensive postprocessing steps.

Currently nolibc-test and therefore the initramfs are always rebuild,
even without source changes, leading to lots of slow kernel relinks.

Instead of linking the initramfs into the kernel assemble it manually
and pass it explicitly to qemu.
This avoids all of the kernel relinks.

Signed-off-by: Thomas Weißschuh <[email protected]>
---
tools/testing/selftests/nolibc/Makefile | 47 +++++++++++++++++----------------
1 file changed, 24 insertions(+), 23 deletions(-)

diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
index 689658f81a19..c62efc62a060 100644
--- a/tools/testing/selftests/nolibc/Makefile
+++ b/tools/testing/selftests/nolibc/Makefile
@@ -131,18 +131,18 @@ REPORT ?= awk '/\[OK\][\r]*$$/{p++} /\[FAIL\][\r]*$$/{if (!f) printf("\n"); f++

help:
@echo "Supported targets under selftests/nolibc:"
- @echo " all call the \"run\" target below"
- @echo " help this help"
- @echo " sysroot create the nolibc sysroot here (uses \$$ARCH)"
- @echo " nolibc-test build the executable (uses \$$CC and \$$CROSS_COMPILE)"
- @echo " libc-test build an executable using the compiler's default libc instead"
- @echo " run-user runs the executable under QEMU (uses \$$XARCH, \$$TEST)"
- @echo " initramfs prepare the initramfs with nolibc-test"
- @echo " defconfig create a fresh new default config (uses \$$XARCH)"
- @echo " kernel (re)build the kernel with the initramfs (uses \$$XARCH)"
- @echo " run runs the kernel in QEMU after building it (uses \$$XARCH, \$$TEST)"
- @echo " rerun runs a previously prebuilt kernel in QEMU (uses \$$XARCH, \$$TEST)"
- @echo " clean clean the sysroot, initramfs, build and output files"
+ @echo " all call the \"run\" target below"
+ @echo " help this help"
+ @echo " sysroot create the nolibc sysroot here (uses \$$ARCH)"
+ @echo " nolibc-test build the executable (uses \$$CC and \$$CROSS_COMPILE)"
+ @echo " libc-test build an executable using the compiler's default libc instead"
+ @echo " run-user runs the executable under QEMU (uses \$$XARCH, \$$TEST)"
+ @echo " initramfs.cpio prepare the initramfs with nolibc-test"
+ @echo " defconfig create a fresh new default config (uses \$$XARCH)"
+ @echo " kernel (re)build the kernel with the initramfs (uses \$$XARCH)"
+ @echo " run runs the kernel in QEMU after building it (uses \$$XARCH, \$$TEST)"
+ @echo " rerun runs a previously prebuilt kernel in QEMU (uses \$$XARCH, \$$TEST)"
+ @echo " clean clean the sysroot, initramfs, build and output files"
@echo ""
@echo "The output file is \"run.out\". Test ranges may be passed using \$$TEST."
@echo ""
@@ -195,25 +195,26 @@ run-user: nolibc-test
$(Q)qemu-$(QEMU_ARCH) ./nolibc-test > "$(CURDIR)/run.out" || :
$(Q)$(REPORT) $(CURDIR)/run.out

-initramfs: nolibc-test
- $(QUIET_MKDIR)mkdir -p initramfs
- $(call QUIET_INSTALL, initramfs/init)
- $(Q)cp nolibc-test initramfs/init
+$(srctree)/usr/gen_init_cpio:
+ $(Q)$(MAKE) -C $(srctree) usr_gen_init_cpio
+
+initramfs.cpio: $(srctree)/usr/gen_init_cpio nolibc-test
+ $(QUIET_GEN)echo 'file /init nolibc-test 755 0 0' | $(srctree)/usr/gen_init_cpio - > initramfs.cpio

defconfig:
$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper $(DEFCONFIG) prepare

-kernel: initramfs
- $(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) $(IMAGE_NAME) CONFIG_INITRAMFS_SOURCE=$(CURDIR)/initramfs
+kernel:
+ $(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) $(IMAGE_NAME)

# run the tests after building the kernel
-run: kernel
- $(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out"
+run: kernel initramfs.cpio
+ qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -initrd initramfs.cpio -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out"
$(Q)$(REPORT) $(CURDIR)/run.out

# re-run the tests from an existing kernel
rerun:
- $(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out"
+ $(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -initrd initramfs.cpio -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out"
$(Q)$(REPORT) $(CURDIR)/run.out

# report with existing test log
@@ -227,8 +228,8 @@ clean:
$(Q)rm -f nolibc-test
$(call QUIET_CLEAN, libc-test)
$(Q)rm -f libc-test
- $(call QUIET_CLEAN, initramfs)
- $(Q)rm -rf initramfs
+ $(call QUIET_CLEAN, initramfs.cpio)
+ $(Q)rm -rf initramfs.cpio
$(call QUIET_CLEAN, run.out)
$(Q)rm -rf run.out


--
2.42.0

2023-09-16 16:00:11

by Thomas Weißschuh

[permalink] [raw]
Subject: [PATCH RFC 1/3] kbuild: add toplevel target for usr/gen_init_cpio

The nolibc testsuite wants to generate an initrams without linking it
into the kernel for which it needs access to gen_init_cpio.
Add a new toplevel target for it so it can be built standalone.

Signed-off-by: Thomas Weißschuh <[email protected]>

---

Kbuild maintainers:

If there is a way that does not require modification of the toplevel
Makefile that would be nice, too.
I followed the example of the "scripts_unifdef" target.
---
Makefile | 4 ++++
1 file changed, 4 insertions(+)

diff --git a/Makefile b/Makefile
index ceb23eed4dce..1caa4429eceb 100644
--- a/Makefile
+++ b/Makefile
@@ -1812,6 +1812,10 @@ endif

endif # KBUILD_EXTMOD

+PHONY += usr_gen_init_cpio
+usr_gen_init_cpio:
+ $(Q)$(MAKE) $(build)=usr usr/gen_init_cpio
+
# ---------------------------------------------------------------------------
# Modules


--
2.42.0

2023-09-17 03:35:08

by Willy Tarreau

[permalink] [raw]
Subject: Re: [PATCH RFC 0/3] selftests/nolibc: avoid spurious kernel relinks

On Sat, Sep 16, 2023 at 09:13:26AM +0200, Thomas Wei?schuh wrote:
> Currently the nolibc testsuite embeds the test executable into a kernel
> image with CONFIG_INITRAMFS_SOURCE.
> This forces a full kernel relink everytime the test executable is
> updated.
>
> This relinking step dominates the test cycle.
> It is slower than building and running the test in qemu together.
>
> With a bit of Makefile-shuffling the relinking can be avoided.

That's pretty nice as indeed it still takes a while to relink it into
the kernel. I agree that for running nolibc-test in qemu we don't need
a unified image. However I've seldom used it on real hardware and I
find it significantly more convenient to use as a single image in this
case. Maybe we should just rename targets so that everything qemu-related
just uses two distinct images while a "unified-image" (or anything else)
still assembles the image into the kernel (BTW the help on the "kernel"
target still mentions initramfs).

Note that we don't need to modify anything in the build system to create
an initrd, I usually make them by hand using "cpio -o -H newc", we don't
need anything else here.

Regarding rerun, I'd rather keep it for the sole reason that I've used
it to check for randomly failing errors (typically the timing-based
ones). It's convenient to run the same image 100 times if needed. I'm
not strongly attached to it, but if "make run" is slower, then we can
keep it. However if you really want to drop it, it also needs to be
dropped from the help message ;-)

Cheers,
Willy

2023-09-17 23:51:32

by Thomas Weißschuh

[permalink] [raw]
Subject: Re: [PATCH RFC 0/3] selftests/nolibc: avoid spurious kernel relinks

On 2023-09-17 05:22:19+0200, Willy Tarreau wrote:
> On Sat, Sep 16, 2023 at 09:13:26AM +0200, Thomas Weißschuh wrote:
> > Currently the nolibc testsuite embeds the test executable into a kernel
> > image with CONFIG_INITRAMFS_SOURCE.
> > This forces a full kernel relink everytime the test executable is
> > updated.
> >
> > This relinking step dominates the test cycle.
> > It is slower than building and running the test in qemu together.
> >
> > With a bit of Makefile-shuffling the relinking can be avoided.
>
> That's pretty nice as indeed it still takes a while to relink it into
> the kernel. I agree that for running nolibc-test in qemu we don't need
> a unified image. However I've seldom used it on real hardware and I
> find it significantly more convenient to use as a single image in this
> case. Maybe we should just rename targets so that everything qemu-related
> just uses two distinct images while a "unified-image" (or anything else)
> still assembles the image into the kernel (BTW the help on the "kernel"
> target still mentions initramfs).

Sounds good, "unified-image" is a bit close to "unified kernel image"
(UKI) which is similar but different.

What about kernel-standalone?

> Note that we don't need to modify anything in the build system to create
> an initrd, I usually make them by hand using "cpio -o -H newc", we don't
> need anything else here.

I'd like to keep the build self-contained. But actually the kernel build
will always build a minimal initramfs if CONFIG_BLK_DEV_INITRD is set.
This config is needed for the nolibc testsuite in any case.

The fact that the initrd is always build means that usr/gen_init_cpio
is also always build.
So we can add "kernel" as a prerequisite to "initramfs.cpio" and
everything should work out without any buildsystem modifications.

> Regarding rerun, I'd rather keep it for the sole reason that I've used
> it to check for randomly failing errors (typically the timing-based
> ones). It's convenient to run the same image 100 times if needed. I'm
> not strongly attached to it, but if "make run" is slower, then we can
> keep it. However if you really want to drop it, it also needs to be
> dropped from the help message ;-)

Fine for me, let's keep it :-)