Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 5 Jun 2023 06:18:22 +0200
From:   Willy Tarreau <w@1wt.eu>
To:     Zhangjin Wu <falcon@tinylab.org>
Cc:     arnd@arndb.de, linux-kernel@vger.kernel.org,
        linux-kselftest@vger.kernel.org, linux-riscv@lists.infradead.org,
        thomas@t-8ch.de
Subject: Re: [PATCH 1/4] selftests/nolibc: add a test-report target
Message-ID: <ZH1iDne+PiyZmIIB@1wt.eu>
References: <cover.1685936428.git.falcon@tinylab.org>
 <291c5437db94057a3b045a6f036b02658380b05b.1685936428.git.falcon@tinylab.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <291c5437db94057a3b045a6f036b02658380b05b.1685936428.git.falcon@tinylab.org>
Precedence: bulk

On Mon, Jun 05, 2023 at 11:48:52AM +0800, Zhangjin Wu wrote:
> A standalone test-report target is added to let the run, run-user and
> rerun targets share them.
> 
> Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
> ---
>  tools/testing/selftests/nolibc/Makefile | 26 ++++++++++++-------------
>  1 file changed, 13 insertions(+), 13 deletions(-)
> 
> diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
> index be4159837494..8149ace2938a 100644
> --- a/tools/testing/selftests/nolibc/Makefile
> +++ b/tools/testing/selftests/nolibc/Makefile
> @@ -127,14 +127,18 @@ nolibc-test: nolibc-test.c sysroot/$(ARCH)/include
>  libc-test: nolibc-test.c
>  	$(QUIET_CC)$(CC) -o $@ $<
>  
> -# qemu user-land test
> -run-user: nolibc-test
> -	$(Q)qemu-$(QEMU_ARCH) ./nolibc-test > "$(CURDIR)/run.out" || :
> +test-report:
>  	$(Q)awk '/\[OK\]$$/{p++} /\[FAIL\]$$/{f++} /\[SKIPPED\]$$/{s++} \
>  	         END{ printf("%d test(s) passed, %d skipped, %d failed.", p, s, f); \
>  	         if (s+f > 0) printf(" See all results in %s\n", ARGV[1]); else print; }' \
>  	         $(CURDIR)/run.out
>  
> +# qemu user-land test
> +_run-user: nolibc-test
> +	$(Q)qemu-$(QEMU_ARCH) ./nolibc-test > "$(CURDIR)/run.out" || :
> +
> +run-user: _run-user test-report
> +

This will not reliably work, there's no ordering here, nothing guarantees
that test-report will run *after* _run-user (e.g. make -j). Another
approach is needed if you want to factor this, but in general creating
sequences in makefiles is difficult and often more painful than having
3 times the same 3 lines.

Willy