Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20220615135824.15522-1-maira.canal@usp.br>
In-Reply-To: <20220615135824.15522-1-maira.canal@usp.br>
From:   David Gow <davidgow@google.com>
Date:   Thu, 16 Jun 2022 22:55:08 +0800
Message-ID: <CABVgOSkVOKx1EEYtg-Os+kui-UivfFLT9OSEB9+sDxcgcxh5Mg@mail.gmail.com>
Subject: Re: [PATCH 00/10] drm: selftest: Convert to KUnit
To:     =?UTF-8?B?TWHDrXJhIENhbmFs?= <maira.canal@usp.br>
Cc:     Isabella Basso <isabbasso@riseup.net>, magalilemes00@gmail.com,
        tales.aparecida@gmail.com, mwen@igalia.com, andrealmeid@riseup.net,
        Trevor Woerner <twoerner@gmail.com>,
        leandro.ribeiro@collabora.com, n@nfraprado.net,
        Daniel Vetter <daniel@ffwll.ch>,
        Shuah Khan <skhan@linuxfoundation.org>,
        David Airlie <airlied@linux.ie>,
        Maxime Ripard <mripard@kernel.org>,
        Thomas Zimmermann <tzimmermann@suse.de>,
        michal.winiarski@intel.com,
        Javier Martinez Canillas <javierm@redhat.com>,
        =?UTF-8?B?Sm9zw6kgRXhww7NzaXRv?= <jose.exposito89@gmail.com>,
        Daniel Latypov <dlatypov@google.com>,
        Brendan Higgins <brendanhiggins@google.com>,
        dri-devel@lists.freedesktop.org,
        "open list:KERNEL SELFTEST FRAMEWORK" 
        <linux-kselftest@vger.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        KUnit Development <kunit-dev@googlegroups.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Wed, Jun 15, 2022 at 9:59 PM Ma=C3=ADra Canal <maira.canal@usp.br> wrote=
:
>
> KUnit unifies the test structure and provides helper tools that simplify
> the development of tests. The basic use case allows running tests as regu=
lar
> processes, which makes it easier to run unit tests on a development machi=
ne
> and to integrate the tests into a CI system.
>
> That said, the conversion of selftests for DRM to KUnit tests is benefici=
al
> as it unifies the testing API by using the KUnit API.
>
> KUnit is beneficial for developers as it eases the process to run unit te=
sts.
> It is possible to run the tests by using the kunit-tool on userspace with=
 the
> following command:
>
> ./tools/testing/kunit/kunit.py run --kunitconfig=3Ddrivers/gpu/drm/tests =
--arch=3Dx86_64
>
> For CI system, it is possible to execute during the build. But, we also t=
hink
> about IGT: we are developing a patch to introduce KUnit to IGT.
>
> These patches were developed during a KUnit hackathon [0] last October. N=
ow,
> we believe that both the IGT side and the Kernel side are in good shape f=
or
> submission.
>
> If you are willing to check the output, here is the Pastebin with the out=
put
> and execution times [1].
>
> [0] https://groups.google.com/g/kunit-dev/c/YqFR1q2uZvk/m/IbvItSfHBAAJ
> [1] https://pastebin.com/FJjLPKsC
>
> - Arthur Grillo, Isabella Basso, and Ma=C3=ADra Canal

Great to see these going upstream!

I've tested them on my machine, both with x86_64 qemu and with UML using:
./tools/testing/kunit/kunit.py run
--kunitconfig=3Ddrivers/gpu/drm/tests/.kunitconfig \
--kconfig_add CONFIG_UML_PCI_OVER_VIRTIO=3Dy \
--kconfig_add CONFIG_VIRTIO_UML=3Dy

And all 114 tests pass, and everything looks good. My only minor notes
(from a quick look at the results, rather than a detailed review of
the code) are that the test names have a few small oddities:
- The suites all end in _tests (or _test, in the case of
drm_plane_helper_test). This is a bit redundant (and while there is
only one drm_plane_helper_test, the inconsistency with the others is a
bit awkward), so removing the suffix may be cleaner. (Or at least
being optimistic, and making drm_plane_helper_test plural.)
- The drm_cmdline_parser_tests suite's tests have some inconsistencies
name-wise: they're the only ones to start with drm_, not igt_, and
they have a few capital letters in some of the
'drm_cmdline_test_force_D_' tests. (It's also technically redundant to
start all of the test names with drm_cmdline_test, given the suite
name.)

Of course, if you're trying to keep compatibility with existing tests
or tooling, or there's some deeper reason they're named like this,
it's definitely not a dealbreaker.

Either way, this whole series is:

Tested-by: David Gow <davidgow@google.com>

Cheers,
-- David