MIME-Version: 1.0
References: <20211102073014.2901870-1-davidgow@google.com> <20211102073014.2901870-4-davidgow@google.com>
In-Reply-To: <20211102073014.2901870-4-davidgow@google.com>
From:   Brendan Higgins <brendanhiggins@google.com>
Date:   Tue, 7 Dec 2021 15:30:45 -0500
Message-ID: <CAFd5g445hxT4koknsF_D68LsgytzoQo8_aAG6JPGcrcizOeiFg@mail.gmail.com>
Subject: Re: [PATCH v4 4/4] kunit: Report test parameter results as (K)TAP subtests
To:     David Gow <davidgow@google.com>
Cc:     Rae Moar <rmr167@gmail.com>, Daniel Latypov <dlatypov@google.com>,
        Shuah Khan <skhan@linuxfoundation.org>,
        kunit-dev@googlegroups.com, linux-kselftest@vger.kernel.org,
        linux-kernel@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Tue, Nov 2, 2021 at 3:30 AM David Gow <davidgow@google.com> wrote:
>
> Currently, the results for individial parameters in a parameterised test
> are simply output as (K)TAP diagnostic lines.
>
> As kunit_tool now supports nested subtests, report each parameter as its
> own subtest.
>
> For example, here's what the output now looks like:
>         # Subtest: inode_test_xtimestamp_decoding
>         ok 1 - 1901-12-13 Lower bound of 32bit < 0 timestamp, no extra bits
>         ok 2 - 1969-12-31 Upper bound of 32bit < 0 timestamp, no extra bits
>         ok 3 - 1970-01-01 Lower bound of 32bit >=0 timestamp, no extra bits
>         ok 4 - 2038-01-19 Upper bound of 32bit >=0 timestamp, no extra bits
>         ok 5 - 2038-01-19 Lower bound of 32bit <0 timestamp, lo extra sec bit on
>         ok 6 - 2106-02-07 Upper bound of 32bit <0 timestamp, lo extra sec bit on
>         ok 7 - 2106-02-07 Lower bound of 32bit >=0 timestamp, lo extra sec bit on
>         ok 8 - 2174-02-25 Upper bound of 32bit >=0 timestamp, lo extra sec bit on
>         ok 9 - 2174-02-25 Lower bound of 32bit <0 timestamp, hi extra sec bit on
>         ok 10 - 2242-03-16 Upper bound of 32bit <0 timestamp, hi extra sec bit on
>         ok 11 - 2242-03-16 Lower bound of 32bit >=0 timestamp, hi extra sec bit on
>         ok 12 - 2310-04-04 Upper bound of 32bit >=0 timestamp, hi extra sec bit on
>         ok 13 - 2310-04-04 Upper bound of 32bit>=0 timestamp, hi extra sec bit 1. 1 ns
>         ok 14 - 2378-04-22 Lower bound of 32bit>= timestamp. Extra sec bits 1. Max ns
>         ok 15 - 2378-04-22 Lower bound of 32bit >=0 timestamp. All extra sec bits on
>         ok 16 - 2446-05-10 Upper bound of 32bit >=0 timestamp. All extra sec bits on
>         # inode_test_xtimestamp_decoding: pass:16 fail:0 skip:0 total:16
>         ok 1 - inode_test_xtimestamp_decoding
>
> Signed-off-by: David Gow <davidgow@google.com>
> Reviewed-by: Daniel Latypov <dlatypov@google.com>

Reviewed-by: Brendan Higgins <brendanhiggins@google.com>