Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20210204173045.1138504-1-dlatypov@google.com> <20210204173045.1138504-2-dlatypov@google.com>
In-Reply-To: <20210204173045.1138504-2-dlatypov@google.com>
From:   Brendan Higgins <brendanhiggins@google.com>
Date:   Thu, 4 Feb 2021 11:43:35 -0800
Message-ID: <CAFd5g47N2ec7KTG9Lk-DgxqEnOEdqYjoQ1+V=aE6NTXbY0-MMg@mail.gmail.com>
Subject: Re: [PATCH v2 1/3] kunit: add kunit.filter_glob cmdline option to
 filter suites
To:     Daniel Latypov <dlatypov@google.com>
Cc:     David Gow <davidgow@google.com>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        "open list:KERNEL SELFTEST FRAMEWORK" 
        <linux-kselftest@vger.kernel.org>,
        Shuah Khan <skhan@linuxfoundation.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Thu, Feb 4, 2021 at 9:31 AM Daniel Latypov <dlatypov@google.com> wrote:
>
> E.g. specifying this would run suites with "list" in their name.
>   kunit.filter_glob=list*
>
> Note: the executor prints out a TAP header that includes the number of
> suites we intend to run.
> So unless we want to report empty results for filtered-out suites, we
> need to do the filtering here in the executor.
> It's also probably better in the executor since we most likely don't
> want any filtering to apply to tests built as modules.
>
> This code does add a CONFIG_GLOB=y dependency for CONFIG_KUNIT=y.
> But the code seems light enough that it shouldn't be an issue.
>
> For now, we only filter on suite names so we don't have to create copies
> of the suites themselves, just the array (of arrays) holding them.
>
> The name is rather generic since in the future, we could consider
> extending it to a syntax like:
>   kunit.filter_glob=<suite_glob>.<test_glob>
> E.g. to run all the del list tests
>   kunit.filter_glob=list-kunit-test.*del*
>
> But at the moment, it's far easier to manually comment out test cases in
> test files as opposed to messing with sets of Kconfig entries to select
> specific suites.
> So even just doing this makes using kunit far less annoying.
>
> Signed-off-by: Daniel Latypov <dlatypov@google.com>

One minor issue below, otherwise:

Reviewed-by: Brendan Higgins <brendanhiggins@google.com>

> ---
>  lib/kunit/Kconfig    |  1 +
>  lib/kunit/executor.c | 91 +++++++++++++++++++++++++++++++++++++++-----
>  2 files changed, 83 insertions(+), 9 deletions(-)
>
> diff --git a/lib/kunit/Kconfig b/lib/kunit/Kconfig
> index 00909e6a2443..0b5dfb001bac 100644
> --- a/lib/kunit/Kconfig
> +++ b/lib/kunit/Kconfig
> @@ -4,6 +4,7 @@
>
>  menuconfig KUNIT
>         tristate "KUnit - Enable support for unit tests"
> +       select GLOB if KUNIT=y
>         help
>           Enables support for kernel unit tests (KUnit), a lightweight unit
>           testing and mocking framework for the Linux kernel. These tests are
> diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c
> index a95742a4ece7..996efb80dba6 100644
> --- a/lib/kunit/executor.c
> +++ b/lib/kunit/executor.c
> @@ -1,6 +1,8 @@
>  // SPDX-License-Identifier: GPL-2.0
>
>  #include <kunit/test.h>
> +#include <linux/glob.h>
> +#include <linux/moduleparam.h>
>
>  /*
>   * These symbols point to the .kunit_test_suites section and are defined in
> @@ -11,14 +13,79 @@ extern struct kunit_suite * const * const __kunit_suites_end[];
>
>  #if IS_BUILTIN(CONFIG_KUNIT)
>
> -static void kunit_print_tap_header(void)
> +static char *filter_glob;
> +module_param(filter_glob, charp, 0);

You should probably also use MODULE_PARM_DESC().