Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
In-Reply-To: <20180922125256.GB14042@bombadil.infradead.org>
References: <CAPhKKr_XFzGcBBuQ51w7HjZd_Kq3nTOB6uAFG2vP5JH+Vx_eng@mail.gmail.com>
 <20180922125256.GB14042@bombadil.infradead.org>
From:   Dmitry Vyukov <dvyukov@google.com>
Date:   Mon, 24 Sep 2018 15:42:27 +0200
Message-ID: <CACT4Y+ZN0OcTkY0ahvF0CGZv1bbLHGH2aEbCFuQzpC498cNojw@mail.gmail.com>
Subject: Re: [Announce] LPC 2018: Testing and Fuzzing Microconference
To:     Matthew Wilcox <willy@infradead.org>
Cc:     Dhaval Giani <dhaval.giani@gmail.com>,
        Sasha Levin <alexander.levin@microsoft.com>,
        LKML <linux-kernel@vger.kernel.org>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        alice.ferrazzi@gmail.com, Kevin Hilman <khilman@baylibre.com>,
        Tim Bird <tbird20d@gmail.com>,
        Laura Abbott <labbott@redhat.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        gustavo.padovan@collabora.co.uk,
        "Carpenter,Dan" <dan.carpenter@oracle.com>, knut.omang@oracle.com
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Sat, Sep 22, 2018 at 2:52 PM, Matthew Wilcox <willy@infradead.org> wrote:
> On Wed, Sep 19, 2018 at 10:13:15AM -0700, Dhaval Giani wrote:
>> Sasha and I are pleased to announce the Testing and Fuzzing track at
>> LPC [ 1 ]. We are planning to continue the discussions from last
>> year's microconference [2]. Many discussions from the Automated
>> Testing Summit [3] will also continue, and a final agenda will come up
>> only soon after that.
>>
>> Suggested Topics
>>
>> - Syzbot/syzkaller
>> - ATS
>> - Distro/stable testing
>> - kernelci
>> - kernelci auto bisection
>> - Unit testing framework
>>
>> We look forward to other interesting topics for this microconference
>> as a reply to this email.
>
> I would like to talk about the IDA test suite that was recently merged.
> See lib/test_ida.c.  It can be built as a module (CONFIG_TEST_IDA=m),
> built-in (=y) or built in userspace (as part of the radix tree test
> suite for historical reasons) along with the current kernel code.
>
> Being able to build the test suite in userspace allows for much more
> rapid development.  Building it in kernel space offers testing across
> a wide range of configurations that I don't have access to and can't
> necessarily simulate well in userspace.
>
> My userspace implementation of kmalloc() simulates failures (in a rather
> heavy-handed way; every non-GFP_KERNEL allocation fails).  That's rather
> harder to simulate in kernel space.  I'd like there to be a way for a
> kernel space test suite to ask for kmalloc failures so that the failure
> paths can be tested.

Hi Matthew,

kmalloc fault injection is already implemented with
CONFIG_FAULT_INJECTION. For original fault injection, you more-or-less
ask to fail X% of allocations at random. It's great for testing
servers for stability, but not so well suited for testing. Recently
I've added so-called "systematic" fault injection
(/proc/thread-self/fail-nth) which is perfect for unit testing. It
allows to ask to fail N-th allocation request in the current task. So
a unit test for a syscall can do:

for (i = 0;; i++) {
  write(/proc/thread-self/fail-nth, i);
  syscall_under_test();
  if (read(/proc/thread-self/fail-nth) != 0)
    break;
}

which allows to examine each failure site one-by-one systematically
(without making random processes on the machine fail too).


> I think the idea of building parts of the core kernel libraries in
> userspace and testing them there has greater generality than just the
> IDA, IDR, XArray & radix tree and might profitably be adopted by other
> parts of lib/.  The userspace simulation of other parts of the kernel
> may well need to be extended.

This would be great. Besides providing faster turn-around time, this
allow us to finally have something like:

make test [subsystem]

which will run all tests for the subsystem locally (in parallel,
giving final OK/FAIL). This is not possible to in-kernel tests,
because they require a machine and an image, and it's not possible to
support everything that people use.