2023-07-19 13:50:39

by Zhangjin Wu

[permalink] [raw]
Subject: [PATCH v2 08/14] selftests/nolibc: string the core targets

To avoid run targets one by one manually and boringly, let's string them
with IMAGE and .config, the MAKE command will trigger the dependencies
for us.

Note, to allow do menuconfig before extconfig manually, only trigger
defconfig while the .config is not there, it means only trigger
defconfig for the first run or after a mrproper.

Signed-off-by: Zhangjin Wu <[email protected]>
---
tools/testing/selftests/nolibc/Makefile | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
index 83cb4b017bef..541f3565e584 100644
--- a/tools/testing/selftests/nolibc/Makefile
+++ b/tools/testing/selftests/nolibc/Makefile
@@ -150,6 +150,7 @@ all: run

sysroot: sysroot/$(ARCH)/include

+PHONY = sysroot/$(ARCH)/include
sysroot/$(ARCH)/include:
$(Q)rm -rf sysroot/$(ARCH) sysroot/sysroot
$(QUIET_MKDIR)mkdir -p sysroot
@@ -205,21 +206,28 @@ mrproper:
defconfig:
$(Q)$(MAKE_KERNEL) $(DEFCONFIG) prepare

-menuconfig:
+PHONY += $(KERNEL_CONFIG)
+$(KERNEL_CONFIG):
+ $(Q)if [ ! -f "$(KERNEL_CONFIG)" ]; then $(MAKE) --no-print-directory defconfig; fi
+
+menuconfig: $(KERNEL_CONFIG)
$(Q)$(MAKE_KERNEL) menuconfig

-extconfig:
+extconfig: $(KERNEL_CONFIG)
$(Q)$(srctree)/scripts/kconfig/merge_config.sh -O "$(objtree)" -m "$(KERNEL_CONFIG)" $(foreach c,$(EXTCONFIG),$(wildcard $(CURDIR)/configs/$c))
$(Q)$(MAKE_KERNEL) KCONFIG_ALLCONFIG="$(KERNEL_CONFIG)" allnoconfig

-kernel: initramfs
+kernel: extconfig
+ $(Q)$(MAKE) --no-print-directory initramfs
$(Q)$(MAKE_KERNEL) $(IMAGE_NAME) CONFIG_INITRAMFS_SOURCE=$(CURDIR)/initramfs

# common macros for qemu run/rerun targets
QEMU_SYSTEM_RUN = qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(KERNEL_IMAGE)" -serial stdio $(QEMU_ARGS)

# run the tests after building the kernel
-run: kernel
+PHONY += $(KERNEL_IMAGE)
+$(KERNEL_IMAGE): kernel
+run: $(KERNEL_IMAGE)
$(Q)$(QEMU_SYSTEM_RUN) $(LOG_OUT)
$(Q)$(REPORT_RUN_OUT)

@@ -244,4 +252,4 @@ clean:
$(call QUIET_CLEAN, run.out)
$(Q)rm -rf run.out

-.PHONY: sysroot/$(ARCH)/include
+.PHONY: $(PHONY)
--
2.25.1



2023-07-22 14:08:25

by Willy Tarreau

[permalink] [raw]
Subject: Re: [PATCH v2 08/14] selftests/nolibc: string the core targets

On Wed, Jul 19, 2023 at 09:26:01PM +0800, Zhangjin Wu wrote:
> To avoid run targets one by one manually and boringly, let's string them
> with IMAGE and .config, the MAKE command will trigger the dependencies
> for us.
>
> Note, to allow do menuconfig before extconfig manually, only trigger
> defconfig while the .config is not there, it means only trigger
> defconfig for the first run or after a mrproper.
>
> Signed-off-by: Zhangjin Wu <[email protected]>
> ---
> tools/testing/selftests/nolibc/Makefile | 18 +++++++++++++-----
> 1 file changed, 13 insertions(+), 5 deletions(-)
>
> diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
> index 83cb4b017bef..541f3565e584 100644
> --- a/tools/testing/selftests/nolibc/Makefile
> +++ b/tools/testing/selftests/nolibc/Makefile
(...)
> -extconfig:
> +extconfig: $(KERNEL_CONFIG)
> $(Q)$(srctree)/scripts/kconfig/merge_config.sh -O "$(objtree)" -m "$(KERNEL_CONFIG)" $(foreach c,$(EXTCONFIG),$(wildcard $(CURDIR)/configs/$c))
> $(Q)$(MAKE_KERNEL) KCONFIG_ALLCONFIG="$(KERNEL_CONFIG)" allnoconfig
>
> -kernel: initramfs
> +kernel: extconfig
> + $(Q)$(MAKE) --no-print-directory initramfs

There seems to be something wrong here. From what I'm seeing, now if I
run "make kernel" it will run extconfig and possibly change the config
I just edited.

Or am I missing something ? I must confess all of this is becoming more
and more obscure :-(

Willy

2023-07-25 14:30:31

by Zhangjin Wu

[permalink] [raw]
Subject: Re: [PATCH v2 08/14] selftests/nolibc: string the core targets

> On Wed, Jul 19, 2023 at 09:26:01PM +0800, Zhangjin Wu wrote:
> > To avoid run targets one by one manually and boringly, let's string them
> > with IMAGE and .config, the MAKE command will trigger the dependencies
> > for us.
> >
> > Note, to allow do menuconfig before extconfig manually, only trigger
> > defconfig while the .config is not there, it means only trigger
> > defconfig for the first run or after a mrproper.
> >
> > Signed-off-by: Zhangjin Wu <[email protected]>
> > ---
> > tools/testing/selftests/nolibc/Makefile | 18 +++++++++++++-----
> > 1 file changed, 13 insertions(+), 5 deletions(-)
> >
> > diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
> > index 83cb4b017bef..541f3565e584 100644
> > --- a/tools/testing/selftests/nolibc/Makefile
> > +++ b/tools/testing/selftests/nolibc/Makefile
> (...)
> > -extconfig:
> > +extconfig: $(KERNEL_CONFIG)
> > $(Q)$(srctree)/scripts/kconfig/merge_config.sh -O "$(objtree)" -m "$(KERNEL_CONFIG)" $(foreach c,$(EXTCONFIG),$(wildcard $(CURDIR)/configs/$c))
> > $(Q)$(MAKE_KERNEL) KCONFIG_ALLCONFIG="$(KERNEL_CONFIG)" allnoconfig
> >
> > -kernel: initramfs
> > +kernel: extconfig
> > + $(Q)$(MAKE) --no-print-directory initramfs
>
> There seems to be something wrong here. From what I'm seeing, now if I
> run "make kernel" it will run extconfig and possibly change the config
> I just edited.
>

Yeah, extconfig will run for every 'make kernel', it is ok for us to
let kernel depends on $(KERNEL_CONFIG), but this requires users to run
extconfig explictly, the solution may be:

- move extconfig operations to defconfig target and future tinyconfig target (it looks cleaner ...)

- but it is not convenient to trigger additional changes introduced by
extconfig, not necessary, but really helpful, something like 'menuconfig'

- let users run extconfig manually after a defconfig or tinyconfig (it is a little complex)

- this make users hard to learn what to do, should give up this method

- run extconfig for every 'make kernel' as it currently do

- this may change something after a menuconfig, but it only trigger minimal
required options, the 'hurt' should be minimal, but of course, it may confuse sometimes ;-(

As a summary, let's remove 'extconfig' and move its operations to the defconfig
and the future tinyconfig targets? 'extconfig' should be a 'default' config
action, let users apply additional ones manually from the top-level 'make
menuconfig', what's your idea?

> Or am I missing something ? I must confess all of this is becoming more
> and more obscure :-(

Yeah ... let's find a better solution ;-)

Best regards,
Zhangjin

>
> Willy

2023-07-29 09:13:18

by Willy Tarreau

[permalink] [raw]
Subject: Re: [PATCH v2 08/14] selftests/nolibc: string the core targets

On Tue, Jul 25, 2023 at 10:20:17PM +0800, Zhangjin Wu wrote:
> > On Wed, Jul 19, 2023 at 09:26:01PM +0800, Zhangjin Wu wrote:
> > > To avoid run targets one by one manually and boringly, let's string them
> > > with IMAGE and .config, the MAKE command will trigger the dependencies
> > > for us.
> > >
> > > Note, to allow do menuconfig before extconfig manually, only trigger
> > > defconfig while the .config is not there, it means only trigger
> > > defconfig for the first run or after a mrproper.
> > >
> > > Signed-off-by: Zhangjin Wu <[email protected]>
> > > ---
> > > tools/testing/selftests/nolibc/Makefile | 18 +++++++++++++-----
> > > 1 file changed, 13 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
> > > index 83cb4b017bef..541f3565e584 100644
> > > --- a/tools/testing/selftests/nolibc/Makefile
> > > +++ b/tools/testing/selftests/nolibc/Makefile
> > (...)
> > > -extconfig:
> > > +extconfig: $(KERNEL_CONFIG)
> > > $(Q)$(srctree)/scripts/kconfig/merge_config.sh -O "$(objtree)" -m "$(KERNEL_CONFIG)" $(foreach c,$(EXTCONFIG),$(wildcard $(CURDIR)/configs/$c))
> > > $(Q)$(MAKE_KERNEL) KCONFIG_ALLCONFIG="$(KERNEL_CONFIG)" allnoconfig
> > >
> > > -kernel: initramfs
> > > +kernel: extconfig
> > > + $(Q)$(MAKE) --no-print-directory initramfs
> >
> > There seems to be something wrong here. From what I'm seeing, now if I
> > run "make kernel" it will run extconfig and possibly change the config
> > I just edited.
> >
>
> Yeah, extconfig will run for every 'make kernel', it is ok for us to
> let kernel depends on $(KERNEL_CONFIG), but this requires users to run
> extconfig explictly, the solution may be:
>
> - move extconfig operations to defconfig target and future tinyconfig target (it looks cleaner ...)
>
> - but it is not convenient to trigger additional changes introduced by
> extconfig, not necessary, but really helpful, something like 'menuconfig'
>
> - let users run extconfig manually after a defconfig or tinyconfig (it is a little complex)
>
> - this make users hard to learn what to do, should give up this method
>
> - run extconfig for every 'make kernel' as it currently do
>
> - this may change something after a menuconfig, but it only trigger minimal
> required options, the 'hurt' should be minimal, but of course, it may confuse sometimes ;-(
>
> As a summary, let's remove 'extconfig' and move its operations to the defconfig
> and the future tinyconfig targets? 'extconfig' should be a 'default' config
> action, let users apply additional ones manually from the top-level 'make
> menuconfig', what's your idea?

Well, it's important to apply the principle of least surprise for the
user. You're a developer who spent time working on your config, you
believe it's OK and you just remind that you've heard about that nolibc
test thing recently and you think "why not give it a try in case it spots
something I forgot in my config". Then you just run the test there and
once done your config was secretly modified. This is exactly an example
of what *not* to do. We should never modify user's config nor files in
general without an explicit request from the user. If the user types
"make defconfig", they're implicitly requesting to replace the config,
so we can do what we want with it. If they type "make kernel", they
expect to make a kernel based on this config, not to mollest this
precious config file and then make a kernel out of it.

So I'm fine with the idea of adding config snippets on top of defconfig
and tinyconfig to allow the user to generate a working config for a
given architecture, but not for modifying their config without their
consent.

Willy

2023-07-29 11:24:35

by Zhangjin Wu

[permalink] [raw]
Subject: Re: [PATCH v2 08/14] selftests/nolibc: string the core targets

> On Tue, Jul 25, 2023 at 10:20:17PM +0800, Zhangjin Wu wrote:
> > > On Wed, Jul 19, 2023 at 09:26:01PM +0800, Zhangjin Wu wrote:
> > > > To avoid run targets one by one manually and boringly, let's string them
> > > > with IMAGE and .config, the MAKE command will trigger the dependencies
> > > > for us.
> > > >
> > > > Note, to allow do menuconfig before extconfig manually, only trigger
> > > > defconfig while the .config is not there, it means only trigger
> > > > defconfig for the first run or after a mrproper.
> > > >
> > > > Signed-off-by: Zhangjin Wu <[email protected]>
> > > > ---
> > > > tools/testing/selftests/nolibc/Makefile | 18 +++++++++++++-----
> > > > 1 file changed, 13 insertions(+), 5 deletions(-)
> > > >
> > > > diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
> > > > index 83cb4b017bef..541f3565e584 100644
> > > > --- a/tools/testing/selftests/nolibc/Makefile
> > > > +++ b/tools/testing/selftests/nolibc/Makefile
> > > (...)
> > > > -extconfig:
> > > > +extconfig: $(KERNEL_CONFIG)
> > > > $(Q)$(srctree)/scripts/kconfig/merge_config.sh -O "$(objtree)" -m "$(KERNEL_CONFIG)" $(foreach c,$(EXTCONFIG),$(wildcard $(CURDIR)/configs/$c))
> > > > $(Q)$(MAKE_KERNEL) KCONFIG_ALLCONFIG="$(KERNEL_CONFIG)" allnoconfig
> > > >
> > > > -kernel: initramfs
> > > > +kernel: extconfig
> > > > + $(Q)$(MAKE) --no-print-directory initramfs
> > >
> > > There seems to be something wrong here. From what I'm seeing, now if I
> > > run "make kernel" it will run extconfig and possibly change the config
> > > I just edited.
> > >
> >
> > Yeah, extconfig will run for every 'make kernel', it is ok for us to
> > let kernel depends on $(KERNEL_CONFIG), but this requires users to run
> > extconfig explictly, the solution may be:
> >
> > - move extconfig operations to defconfig target and future tinyconfig target (it looks cleaner ...)
> >
> > - but it is not convenient to trigger additional changes introduced by
> > extconfig, not necessary, but really helpful, something like 'menuconfig'
> >
> > - let users run extconfig manually after a defconfig or tinyconfig (it is a little complex)
> >
> > - this make users hard to learn what to do, should give up this method
> >
> > - run extconfig for every 'make kernel' as it currently do
> >
> > - this may change something after a menuconfig, but it only trigger minimal
> > required options, the 'hurt' should be minimal, but of course, it may confuse sometimes ;-(
> >
> > As a summary, let's remove 'extconfig' and move its operations to the defconfig
> > and the future tinyconfig targets? 'extconfig' should be a 'default' config
> > action, let users apply additional ones manually from the top-level 'make
> > menuconfig', what's your idea?
>
> Well, it's important to apply the principle of least surprise for the
> user. You're a developer who spent time working on your config, you
> believe it's OK and you just remind that you've heard about that nolibc
> test thing recently and you think "why not give it a try in case it spots
> something I forgot in my config". Then you just run the test there and
> once done your config was secretly modified. This is exactly an example
> of what *not* to do. We should never modify user's config nor files in
> general without an explicit request from the user. If the user types
> "make defconfig", they're implicitly requesting to replace the config,
> so we can do what we want with it. If they type "make kernel", they
> expect to make a kernel based on this config, not to mollest this
> precious config file and then make a kernel out of it.
>
> So I'm fine with the idea of adding config snippets on top of defconfig
> and tinyconfig to allow the user to generate a working config for a
> given architecture, but not for modifying their config without their
> consent.
>

Agree, seems our additional config snippets are minimal and 'necessary'
for boot and print, so, I ignored the override issue before.

What about the version in v3 here:
https://lore.kernel.org/lkml/9b52e26748eda1ac108d569207bf428bf37b3bbc.1690489039.git.falcon@tinylab.org/

The 'defconfig' will only be triggered while there is no .config there,
I do think it is important, at the first time of using nolibc, I
directly run kernel but it fails for it has a manual defconfig
requirement every time, so, I do think a default defconfig for kernel
for the first run or after a mrproper is helpful, it doesn't modify any
.config for there is no one there.

+PHONY += $(KERNEL_CONFIG)
+$(KERNEL_CONFIG):
+ $(Q)if [ ! -f "$(KERNEL_CONFIG)" ]; then $(MAKE) --no-print-directory defconfig; fi
+
+kernel: $(KERNEL_CONFIG)
+ $(Q)$(MAKE) --no-print-directory initramfs
$(Q)$(MAKE_KERNEL) $(IMAGE_NAME) CONFIG_INITRAMFS_SOURCE=$(CURDIR)/initramfs

Thanks,
Zhangjin

> Willy

2023-07-29 18:45:29

by Willy Tarreau

[permalink] [raw]
Subject: Re: [PATCH v2 08/14] selftests/nolibc: string the core targets

On Sat, Jul 29, 2023 at 05:54:47PM +0800, Zhangjin Wu wrote:
> The 'defconfig' will only be triggered while there is no .config there,
> I do think it is important, at the first time of using nolibc, I
> directly run kernel but it fails for it has a manual defconfig
> requirement every time, so, I do think a default defconfig for kernel
> for the first run or after a mrproper is helpful, it doesn't modify any
> .config for there is no one there.

On the opposite, that's yet another example of automatic stuff that for
me adds zero value and just doubts in the user's head: "is it safe to
call this with my own config or should I keep a safe copy of it?",
"what will it use for the config?", "will the arch be correct if my
current config references 32BIT and the generated default one switches
it to 64?" etc.

Please let's not add unneeded dependencies and chaining. It does not
help and makes it harder to restart at one specific step, thus lowers
the overall value.

Willy

2023-07-29 18:52:42

by Zhangjin Wu

[permalink] [raw]
Subject: Re: [PATCH v2 08/14] selftests/nolibc: string the core targets

> On Sat, Jul 29, 2023 at 05:54:47PM +0800, Zhangjin Wu wrote:
> > The 'defconfig' will only be triggered while there is no .config there,
> > I do think it is important, at the first time of using nolibc, I
> > directly run kernel but it fails for it has a manual defconfig
> > requirement every time, so, I do think a default defconfig for kernel
> > for the first run or after a mrproper is helpful, it doesn't modify any
> > .config for there is no one there.
>
> On the opposite, that's yet another example of automatic stuff that for
> me adds zero value and just doubts in the user's head: "is it safe to
> call this with my own config or should I keep a safe copy of it?",
> "what will it use for the config?", "will the arch be correct if my
> current config references 32BIT and the generated default one switches
> it to 64?" etc.
>
> Please let's not add unneeded dependencies and chaining. It does not
> help and makes it harder to restart at one specific step, thus lowers
> the overall value.
>

Ok, let's do subtraction, drop this one and the timeout two.

Thanks,
Zhangjin

> Willy