Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Content-Type: text/plain;
        charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 11.1 \(3445.4.7\))
Subject: Re: [LTP] Towards 4.14 LTS
From:   Tom Gall <tom.gall@linaro.org>
In-Reply-To: <20171120161029.GA2188@rei>
Date:   Mon, 20 Nov 2017 10:48:58 -0600
Cc:     linux-kernel@vger.kernel.org,
        linux- stable <stable@vger.kernel.org>,
        torvalds@linux-foundation.org,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        linux-kselftest@vger.kernel.org, ltp@lists.linux.it,
        shuahkh@osg.samsung.com, Guenter Roeck <linux@roeck-us.net>
Content-Transfer-Encoding: quoted-printable
Message-Id: <AD4F87EA-602A-402D-8E72-4DFB52585A5D@linaro.org>
References: <5E1947BC-9D22-42EB-BA29-96EA15A9C817@linaro.org>
 <20171120161029.GA2188@rei>
To:     Cyril Hrubis <chrubis@suse.cz>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk


> On Nov 20, 2017, at 10:10 AM, Cyril Hrubis <chrubis@suse.cz> wrote:
>=20
> Hi!
>> So why didn???t we report these? As mentioned we???ve been tossing =
out dodgy
>> test cases to get to a clean baseline. We don???t need or want noise.=20=

>>=20
>> For LTS, I want the system when it detects a failure to enable a =
quick=20
>> bisect involving the affected test bucket. Given the nature of kernel=20=

>> bugs tho, there is that class of bug which only happens occasionally.
>=20
> =46rom my experience debugging kernel bugs requires an actuall human
> interaction and there is only certain level of automation that can be
> achieved. Don't take me wrong, automatic bisection and other bells and
> whistles are a nice to have, but at the end of the day you usually =
need
> someone to reproduce/look at the problem, possibly check the source
> code, report a bug, etc. Hence it does not make much sense to have an
> automated system without dedicated engineers assigned to review the =
test
> results.

You are entirely right automation only gets so far. We have a few lines
of defense that probably are worth a mention.

1) infra - sometimes results/runs need to be re-run for whatever reason.
2) triage - Crappy test case or something that is real?
3) kernel - bisecting etc

We don=E2=80=99t have huge dedicated teams for each category but =
likewise each=20
has a team.

> --=20
> Cyril Hrubis
> chrubis@suse.cz


From 1584602879088144614@xxx Mon Nov 20 16:24:27 +0000 2017
X-GM-THRID: 1584318260606186478
X-Gmail-Labels: Inbox,Category Forums,HistoricalUnread