In the middle of the "RFC - kernel test result specification (KTAP)" thread,
started in August 2021, Tim Bird made a suggestion to allow a prefix to the
KTAP data format:
> Just as a side note, in some Fuego tests, it was very useful to include an identifier
> in thethe prefix nested tests. The output looked like this:
>
> TAP version 13
> 1..2
> [batch_id 4] TAP version 13
> [batch_id 4] 1..2
> [batch_id 4] ok 1 - cyclictest with 1000 cycles
> [batch_id 4] # problem setting CLOCK_REALTIME
> [batch_id 4] not ok 2 - cyclictest with CLOCK_REALTIME
> not ok 1 - check realtime
> [batch_id 4] TAP version 13
> [batch_id 4] 1..1
> [batch_id 4] ok 1 - IOZone read/write 4k blocks
> ok 2 - check I/O performance
>
> Can I propose that the prefix not be fixed by the spec, but that the spec indicates that
> whatever the prefix is on the TAP version line, that prefix must be used with the output for
> all lines from the test (with the exception of unknown lines)?
The thread was discussing many other items, but this is the one that I want
to focus on in this new RFC thread.
Tim's original email was:
https://lore.kernel.org/r/BYAPR13MB2503A4B79074D8ED5579345DFDCB9@BYAPR13MB2503.namprd13.prod.outlook.com
There was one reply to this that commented on Tim's suggestion (and also many
other items in the thread) at:
https://lore.kernel.org/r/202108301226.800F3D6D4@keescook
> Oh, interesting. This would also allow parallel (unique) test execution
> to be parsable. That sounds workable. (Again, this needs LAVA patching
> again...)
I found Tim's original suggestion to be useful, so I have come up with
two possible ways to modify the KTAP specification to implement what Tim
was thinking about. I would not be surprised if someone else has a better
suggestion than mine, but I will reply to this email with my two alternatives
to start a discussion. My alternatives are not in the form of patches, but
if discussion leads to a good result then I will create a patch for review.
-Frank
On Wed, May 11, 2022 at 10:59 PM Frank Rowand <[email protected]> wrote:
>
> In the middle of the "RFC - kernel test result specification (KTAP)" thread,
> started in August 2021, Tim Bird made a suggestion to allow a prefix to the
> KTAP data format:
>
> > Just as a side note, in some Fuego tests, it was very useful to include an identifier
> > in thethe prefix nested tests. The output looked like this:
> >
> > TAP version 13
> > 1..2
> > [batch_id 4] TAP version 13
> > [batch_id 4] 1..2
> > [batch_id 4] ok 1 - cyclictest with 1000 cycles
> > [batch_id 4] # problem setting CLOCK_REALTIME
> > [batch_id 4] not ok 2 - cyclictest with CLOCK_REALTIME
> > not ok 1 - check realtime
> > [batch_id 4] TAP version 13
> > [batch_id 4] 1..1
> > [batch_id 4] ok 1 - IOZone read/write 4k blocks
> > ok 2 - check I/O performance
> >
> > Can I propose that the prefix not be fixed by the spec, but that the spec indicates that
> > whatever the prefix is on the TAP version line, that prefix must be used with the output for
> > all lines from the test (with the exception of unknown lines)?
Just chiming in since I didn't see it mentioned after a quick skim of
the original thread:
This is already basically the behavior of kunit.py's TAP parser since
commit afc63da64f1e5e41875c98707020e85050f8a0c5
Author: Heidi Fahim <[email protected]>
Date: Mon Mar 16 13:21:24 2020 -0700
kunit: kunit_parser: make parser more robust
Previously, kunit_parser did not properly handle kunit TAP output that
- had any prefixes (generated from different configs e.g.
CONFIG_PRINTK_TIME)
...
The notable difference is that only the prefix _length_ is fixed, not
the contents of the string itself.
So ignoring a dynamic prefix is a practical necessity if we want to
parse TAP from kernelspace/printk across a range of configs.
But I don't know if this dynamic version is worth including in the spec.
The static prefix makes more sense to me to formalize, and if we go
down that route, at least kunit.py will already be compliant :)
Daniel