Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Zhangjin Wu <falcon@tinylab.org>
To:     thomas@t-8ch.de
Cc:     falcon@tinylab.org, linux-kernel@vger.kernel.org,
        linux-kselftest@vger.kernel.org, w@1wt.eu
Subject: nolibc: KTAP output and test reports
Date:   Thu,  8 Jun 2023 18:48:28 +0800
Message-Id: <20230608104828.38797-1-falcon@tinylab.org>
In-Reply-To: <cc03380f-594f-431b-941d-aea879a0c084@t-8ch.de>
References: <cc03380f-594f-431b-941d-aea879a0c084@t-8ch.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Feedback-ID: bizesmtp:tinylab.org:qybglogicsvrsz:qybglogicsvrsz3a-3
Precedence: bulk

Hi, Thomas

> Hi Willy, Zhangjin,
> 
> after your recent discussions about the test output and report I
> wondered if it would make sense to switch nolibc-test to KTAP output
> format [0].

Just read the doc, it looks really good.

> 
> With this it would be possible to have a wrapper script run each
> architecture test as its own test subcomponent.
> A (K)TAP parser/runner could then directly recognize and report failing
> testcases, making it easier to validate.
>

Yeah, this is what can we benefit from the standard format.

> Also maybe we can hook it up into the regular kselftests setup and have
> the bots run it as part of that.
>

I did take a look at the other kselftests cases, seems lots of cases use
qemu to run the tests, perhaps we can share some of them. Not sure if
there are some libraries work on qemu test support, therefore, we can
reuse them.

> The kernel even includes a header-only library to implement the format [1].
> It also should be fairly easy to emit the format without a library.
>

Perhaps we can learn and discuss on how to use them at first, I'm a newbie to
both of them, but I'm really interested in running nolibc in the kselftest
framework ;-)

Thanks,
Zhangjin

> Thomas
> 
> [0] Documentation/dev-tools/ktap.rst
> [1] Documentation/dev-tools/kselftest.rst (Test harness)