From: ebiederm@xmission.com (Eric W. Biederman)
To: Dmitry Vyukov <dvyukov@google.com>
Cc: Pavel Machek <pavel@ucw.cz>, Mike Galbraith <efault@gmx.de>,
        LKML <linux-kernel@vger.kernel.org>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        syzkaller <syzkaller@googlegroups.com>
References: <CACT4Y+ZNJRzPxxPQNR88vcPxD54mHY-evhMwR39iMsJh_NwaEA@mail.gmail.com>
        <20180104092552.GA991@amd> <1515058705.7875.25.camel@gmx.de>
        <20180104095628.GA4407@atrey.karlin.mff.cuni.cz>
        <CACT4Y+YB_p_p4ORbke1oZfkLSC8rnAD81Ra_fprsPSugGAQy8g@mail.gmail.com>
Date: Thu, 04 Jan 2018 09:23:58 -0600
In-Reply-To: <CACT4Y+YB_p_p4ORbke1oZfkLSC8rnAD81Ra_fprsPSugGAQy8g@mail.gmail.com>
        (Dmitry Vyukov's message of "Thu, 4 Jan 2018 12:09:26 +0100")
Message-ID: <87inchsl4h.fsf@xmission.com>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
Subject: Re: LKML admins (syzbot emails are not delivered)
Sender: linux-kernel-owner@vger.kernel.org

Dmitry Vyukov <dvyukov@google.com> writes:

> Hi Pavel,
>
> I've answered this question here in full detail. In short, this is
> useful and actionable.
> https://groups.google.com/d/msg/syzkaller/2nVn_XkVhEE/GjjfISejCgAJ

*Snort*

If the information to solve an issue is not in the Oops syzbot is
useless.

The Oops isn't even mailed in plain text so I have to save the stupid
thing in a file to see the full text of the problem.

Further there is no place in the syzbot process to test fixes.

Then there is the issue of testing linux-next and reporting errors on
who knows what code configuration against code that hasn't changed in
linux-next.   Which presumably any sane person would assume the errors
are introduced by some other piece of new code.  But syzbot goes and
spams the people who wrote the function where the code is failing.

Bots can work.  We have all of the automatic testing infrastructure
against everyone's branches on kernel.org to prove it.

syzbot finds weird errors, so that makes the problem space more
difficult to deal with.

Still I compleltely don't see the people behind syzbot presumably you
Dmitry taking responsibility for syzbot failings.  Instead I see excuses
like you don't completely control some part of the code that syzbot is
built on so can't fix practical real world issues.  Like Content-type.

Bots can be the most horrible thing for a code base.  If there is not
someone or something going through an filtering out the false positives.
If there is not a process to ensure that issues are brought to the
proper peoples attention so things get fixed.  Bots can be completely
demoralizing or possibily desensitizing because you keep seeing issues,
and nothing you do ever makes the issues go away.

Given that no one seems to take any responsibility for syzbots failures
of any kind.  Not content-type in the emails.  Not the body of the
message (which has a massive disclaimer).  I don't find syzbot at all
useful.

Tools are for people, in this case kernel programmers.  syzbot has
serious usability issues.  That makes syzbot a bad tool.

Eric