Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20230610005149.1145665-1-rmoar@google.com> <20230610005149.1145665-4-rmoar@google.com>
 <202306131324.5B69ABC@keescook>
In-Reply-To: <202306131324.5B69ABC@keescook>
From:   Rae Moar <rmoar@google.com>
Date:   Tue, 13 Jun 2023 16:58:30 -0400
Message-ID: <CA+GJov6FjojVU6FhbkFfosGQveqFG2a2fuc4Pw83Cs7dbhuhDw@mail.gmail.com>
Subject: Re: [RFC v1 3/6] kunit: Add ability to filter attributes
To:     Kees Cook <keescook@chromium.org>
Cc:     shuah@kernel.org, davidgow@google.com, dlatypov@google.com,
        brendan.higgins@linux.dev, linux-kselftest@vger.kernel.org,
        kunit-dev@googlegroups.com, linux-kernel@vger.kernel.org,
        linux-hardening@vger.kernel.org, jstultz@google.com,
        tglx@linutronix.de, sboyd@kernel.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Tue, Jun 13, 2023 at 4:26=E2=80=AFPM Kees Cook <keescook@chromium.org> w=
rote:
>
> On Sat, Jun 10, 2023 at 12:51:46AM +0000, Rae Moar wrote:
> > Add filtering of test attributes. Users can filter tests using a
> > module_param_array called "filter". This functionality will be added to
> > kunit.py in the next patch.
> >
> > The filters will be imputed in the format:
> > "<attribute_name><operation><attribute_value>"
> >
> > Example: "speed>slow"
> >
> > Operations include: >, <, >=3D, <=3D, !=3D, and =3D. These operations d=
o not need
> > to act the same for every attribute.
>
> How is the "default" filter specified? Is explicitly unfiltered? (i.e.
> "slow" stuff will run by default?) Or should there be a default filter
> of "speed<slow" for the memcpy conversion?
>
> But yes, I'm a fan of this whole series! I would much prefer this to
> having one-off CONFIGs for slow tests. :)
>

Hello!

Great to hear that you are happy to see this series.

Currently if no filter is specified, tests will run unfiltered (so the
slow tests will run by default).

But I think the idea of having a "default" filter is really
interesting. I would definitely be open to adding a default filter
that only runs tests with speeds faster than slow, which could then be
overridden by any filter input.

This also means there could be suite specific default filters but that
may be a future implementation since we currently only have one
attribute.

Or alternatively we could have a file that includes a list of default
filters that could then be inputted and altered based on suite.

Thanks!
Rae

> --
> Kees Cook