Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <e87a70f7-9b7e-e5dc-bc00-8f8ba76f0476@riseup.net>
Date:   Wed, 26 Jul 2023 11:56:57 -0300
MIME-Version: 1.0
Subject: Re: [PATCH v3 0/9] kunit: Add test attributes API
Content-Language: en-US
To:     Rae Moar <rmoar@google.com>, shuah@kernel.org, davidgow@google.com,
        dlatypov@google.com, brendan.higgins@linux.dev
Cc:     linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com,
        linux-kernel@vger.kernel.org, keescook@chromium.org,
        linux-hardening@vger.kernel.org, jstultz@google.com,
        tglx@linutronix.de, sboyd@kernel.org
References: <20230725212522.1622716-1-rmoar@google.com>
From:   Arthur Grillo Queiroz Cabral <arthurgrillo@riseup.net>
In-Reply-To: <20230725212522.1622716-1-rmoar@google.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk


On 25/07/23 18:25, 'Rae Moar' via KUnit Development wrote:
> Hello everyone,
> 
> This patch series adds a test attributes framework to KUnit.
> 
> There has been interest in filtering out "slow" KUnit tests. Most notably,
> a new config, CONFIG_MEMCPY_SLOW_KUNIT_TEST, has been added to exclude a
> particularly slow memcpy test
> (https://lore.kernel.org/all/20230118200653.give.574-kees@kernel.org/).
> 
> This attributes framework can be used to save and access test associated
> data, including whether a test is slow. These attributes are reportable
> (via KTAP and command line output) and are also filterable.
> 
> This framework is designed to allow for the addition of other attributes in
> the future. These attributes could include whether the test can be run
> concurrently, test file path, etc.
> 
> To try out the framework I suggest running:
> "./tools/testing/kunit/kunit.py run --filter speed!=slow"
> 
> This patch series was originally sent out as an RFC. Here is a link to the
> RFC v2:
> https://lore.kernel.org/all/20230707210947.1208717-1-rmoar@google.com/
> 
> Thanks!
> Rae
> 

Great addition to the KUnit framework!

On all the patch set

Reviewed-by: Arthur Grillo <arthurgrillo@riseup.net>

Best Regards,
~Arthur Grillo

> Rae Moar (9):
>   kunit: Add test attributes API structure
>   kunit: Add speed attribute
>   kunit: Add module attribute
>   kunit: Add ability to filter attributes
>   kunit: tool: Add command line interface to filter and report
>     attributes
>   kunit: memcpy: Mark tests as slow using test attributes
>   kunit: time: Mark test as slow using test attributes
>   kunit: add tests for filtering attributes
>   kunit: Add documentation of KUnit test attributes
> 
>  Documentation/dev-tools/kunit/run_wrapper.rst |  12 +
>  .../dev-tools/kunit/running_tips.rst          | 166 +++++++
>  include/kunit/attributes.h                    |  50 +++
>  include/kunit/test.h                          |  70 ++-
>  kernel/time/time_test.c                       |   2 +-
>  lib/Kconfig.debug                             |   3 +
>  lib/kunit/Makefile                            |   3 +-
>  lib/kunit/attributes.c                        | 418 ++++++++++++++++++
>  lib/kunit/executor.c                          | 114 ++++-
>  lib/kunit/executor_test.c                     | 128 +++++-
>  lib/kunit/kunit-example-test.c                |   9 +
>  lib/kunit/test.c                              |  27 +-
>  lib/memcpy_kunit.c                            |   8 +-
>  tools/testing/kunit/kunit.py                  |  70 ++-
>  tools/testing/kunit/kunit_kernel.py           |   8 +-
>  tools/testing/kunit/kunit_parser.py           |  11 +-
>  tools/testing/kunit/kunit_tool_test.py        |  39 +-
>  17 files changed, 1062 insertions(+), 76 deletions(-)
>  create mode 100644 include/kunit/attributes.h
>  create mode 100644 lib/kunit/attributes.c
> 
> 
> base-commit: 64bd4641310c41a1ecf07c13c67bc0ed61045dfd