Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20221028210256.3776835-1-dlatypov@google.com> <CABVgOS=uAVc_nKRFYRiQtWFykyfWH6hWASK-yd+ZnH5UNmRgsw@mail.gmail.com>
In-Reply-To: <CABVgOS=uAVc_nKRFYRiQtWFykyfWH6hWASK-yd+ZnH5UNmRgsw@mail.gmail.com>
From:   Daniel Latypov <dlatypov@google.com>
Date:   Mon, 31 Oct 2022 09:37:06 -0700
Message-ID: <CAGS_qxpghJUj+R74SFPjq=VDoNdVp7Wq2b+AcWwmHHeCQVjmPg@mail.gmail.com>
Subject: Re: [PATCH 1/2] kunit: tool: make unit test not print parsed testdata
 to stdout
To:     David Gow <davidgow@google.com>
Cc:     brendanhiggins@google.com, rmoar@google.com,
        linux-kernel@vger.kernel.org, kunit-dev@googlegroups.com,
        linux-kselftest@vger.kernel.org, skhan@linuxfoundation.org
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Sat, Oct 29, 2022 at 8:31 PM David Gow <davidgow@google.com> wrote:
>
> On Sat, Oct 29, 2022 at 5:03 AM Daniel Latypov <dlatypov@google.com> wrote:
> >
> > Currently, if you run
> > $ ./tools/testing/kunit/kunit_tool_test.py
> > you'll see a lot of output from the parser as we feed it testdata.
> >
> > This makes the output hard to read and fairly confusing, esp. since our
> > testdata includes example failures, which get printed out in red.
> >
> > Silence that output so real failures are easier to see.
> >
> > Signed-off-by: Daniel Latypov <dlatypov@google.com>
> > ---
>
> Thanks -- this has been annoying me for ages.
>
> That being said, this isn't a perfect fix, the "usage" text and
> "Reconfiguring .config"  still show up for me:
> ---
> davidgow@slicestar:~/Development/linux-kselftest$
> ./tools/testing/kunit/kunit_tool_test.py
> ..............................usage: kunit_tool_test.py run [-h]
> [--build_dir DIR] [--make_options X=Y] [--alltests] [--kunitconfig
> PATHS] [--kconfig_add CONFIG_X=Y] [--arch ARCH] [--cross_compile
> PREFIX] [--qemu_config FILE] [--qemu_ar
> gs] [--jobs N]
>                              [--timeout SECONDS] [--kernel_args]
> [--run_isolated {suite,test}] [--raw_output [{all,kunit}]] [--json
> [FILE]]
>                              [filter_glob]
> kunit_tool_test.py run: error: argument --raw_output: invalid choice:
> 'invalid' (choose from 'all', 'kunit')
> ..............................Generating .config ...
> .Regenerating .config ...
> .........
> ----------------------------------------------------------------------
> Ran 70 tests in 0.232s
>
> OK
> ---
>
> That's still a significant improvement on what we had before, though, so:

Yeah, I was originally going to make this patch silence all the output.
But I figured I would focus this patch on just the code using our
kunit_printer abstraction, aka the parser.

We could address the other ones, but we'd have to mock out the
built-in print function.
As someone who is guilty of print(f) debugging, doing that means I no
longer have an easy of way of adding debugging code to those test
cases :P