Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Sat, 2 Oct 2021 16:29:42 -0300
From:   Marcelo Schmitt <marcelo.schmitt1@gmail.com>
To:     Daniel Latypov <dlatypov@google.com>
Cc:     brendanhiggins@google.com, andersonreisrosa@gmail.com,
        linux-kernel@vger.kernel.org, kunit-dev@googlegroups.com
Subject: Re: [PATCH] kunit: mock: add support for function mocks with no
 parameters
Message-ID: <YVizJvi/7+e3n55Q@marsc.168.1.7>
References: <YUZBkZhQsF5SlcLb@marsc.168.1.7>
 <CAGS_qxrtD7wJ8oTcyMdYjUZubiJwD7CO+YFDy8ohho5jLeNa8g@mail.gmail.com>
 <YU943oP0ERX1Xh3Q@marsc.168.1.7>
 <CAGS_qxooHU+7SW9_GbvV2w2QAr3daBJcybx0zMqVFp3f3azrGg@mail.gmail.com>
 <CAGS_qxpbH6c3OvoYZC6TXFQomLpwZg5q7=EZ9B9k=Rw1mOz=0w@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAGS_qxpbH6c3OvoYZC6TXFQomLpwZg5q7=EZ9B9k=Rw1mOz=0w@mail.gmail.com>
Precedence: bulk

Removed Andy from the CC list as we're not talking about that bug anymore.
[...]

> > > We can work on something else besides mocking if it makes more sense to the
> > > project.
> >
> > Mocking doesn't feel like an area where we can expect to see progress right now.
> > In terms of other KUnit features we know would be useful now, I think
> > it's mostly just [1] and [2], which hopefully will land in 5.16.
> 
> To be clear, if anyone thinks up a useful feature, that'd be great.
> I personally am just out of ideas at the moment, and I think so are
> Brendan and David.
> 
> We'd want to prioritize features that can improve existing tests or
> unblock known new tests.
> Mocking in the alpha version of KUnit is a case where a feature
> sounded really good on paper and had a bunch of bells and whistles
> (e.g. strict/nice/naggy mocks support, etc.) but was perhaps
> overengineered and thus failed to find a home upstream.
> 
> But I just thought of a few more things we could do in the kunit.py script.
> I think we have more room for improvement there than in the in-kernel
> part of KUnit right now, but I assume it's the more boring part for
> most people.
> 
> One thing I'd really like to see is getting code coverage to work in
> kunit.py while using QEMU.
> We have a process for doing so under UML here:
> https://lore.kernel.org/linux-kselftest/20210901190623.315736-1-rmoar@google.com/
> UML actually uses a different coverage implementation than normal, so
> there's a few things that would need to change.
> 
> We can build and run against "normal" coverage kernels pretty easily:
> 
> $ cat >qemu_coverage_kunitconfig <<EOF
> CONFIG_KUNIT=y
> CONFIG_KUNIT_EXAMPLE_TEST=y
> CONFIG_GCOV_KERNEL=y
> CONFIG_DEBUG_FS=y
> CONFIG_GCOV_PROFILE_ALL=y
> EOF
> $ ./tools/testing/kunit/kunit.py run --arch=x86_64
> --kunitconfig=qemu_coverage_kunitconfig
> 
> The problem is we'd need to copy the coverage data off the VM instead
> of just letting it shutdown when tests are done.
> If we had a userspace running, we'd basically do something like
> $ scp -r user@vm:/sys/kernel/debug/gcov .
> <some stuff to get these files in the right spot under .kunit/>
> <then we'd run lcov and genhtml, just like we do for UML>
> 
> Normal KUnit tests definitely don't want to have to have the overhead
> of running a userspace, so the implementation might look like a
> "--qemu_coverage" flag, or maybe a set of generic flags that would
> give a user enough control over the VM to do this.
> Or maybe the right answer is to not involve kunit.py at all.
> 
> Not sure if that sounds interesting to you or anyone.
> 

Sounds cool. We'll try to make it.

Thanks

> >
> > I think right now we probably need more tests written to have a better
> > idea of what else we could/should do.
> > Partly because of that, David is trying to get the ball rolling on
> > testing ext4. We're also hopeful that it'll be easier to add tests if
> > adjacent code is already tested (sharing fakes, conventions, ability
> > to copy-paste, etc.).
> >
> > [1] https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/commit/?h=kunit&id=3b29021ddd10cfb6b2565c623595bd3b02036f33
> > [2] https://lore.kernel.org/linux-kselftest/20210909001037.2842954-1-dlatypov@google.com/
> >
> >