Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Wed, 28 Aug 2019 11:28:02 -0500
From:   Dan Rue <dan.rue@linaro.org>
To:     Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc:     Naresh Kamboju <naresh.kamboju@linaro.org>,
        open list <linux-kernel@vger.kernel.org>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        Guenter Roeck <linux@roeck-us.net>,
        Shuah Khan <shuah@kernel.org>, patches@kernelci.org,
        Ben Hutchings <ben.hutchings@codethink.co.uk>,
        lkft-triage@lists.linaro.org,
        linux- stable <stable@vger.kernel.org>,
        Major Hayden <major@mhtx.net>
Subject: Re: [PATCH 5.2 000/162] 5.2.11-stable review
Message-ID: <20190828162802.jdbrylm3dai3kaw7@xps.therub.org>
Mail-Followup-To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        Naresh Kamboju <naresh.kamboju@linaro.org>,
        open list <linux-kernel@vger.kernel.org>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        Guenter Roeck <linux@roeck-us.net>, Shuah Khan <shuah@kernel.org>,
        patches@kernelci.org, Ben Hutchings <ben.hutchings@codethink.co.uk>,
        lkft-triage@lists.linaro.org,
        linux- stable <stable@vger.kernel.org>,
        Major Hayden <major@mhtx.net>
References: <20190827072738.093683223@linuxfoundation.org>
 <CA+G9fYtmHsr8XWvmSLy9QKvF37KfZ4v+T1VnRy2uhpE0HB4Ggg@mail.gmail.com>
 <20190828151608.GC9673@kroah.com>
 <20190828154718.nesaolp7gfxxh5o5@xps.therub.org>
 <20190828155252.GA3803@kroah.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20190828155252.GA3803@kroah.com>
User-Agent: NeoMutt/20180716
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, Aug 28, 2019 at 05:52:52PM +0200, Greg Kroah-Hartman wrote:
> On Wed, Aug 28, 2019 at 10:47:18AM -0500, Dan Rue wrote:
> > On Wed, Aug 28, 2019 at 05:16:08PM +0200, Greg Kroah-Hartman wrote:
> > > On Wed, Aug 28, 2019 at 10:30:09AM +0530, Naresh Kamboju wrote:
> > > > On Tue, 27 Aug 2019 at 13:30, Greg Kroah-Hartman
> > > > <gregkh@linuxfoundation.org> wrote:
> > > > >
> > > > > This is the start of the stable review cycle for the 5.2.11 release.
> > > > > There are 162 patches in this series, all will be posted as a response
> > > > > to this one.  If anyone has any issues with these being applied, please
> > > > > let me know.
> > > > >
> > > > > Responses should be made by Thu 29 Aug 2019 07:25:02 AM UTC.
> > > > > Anything received after that time might be too late.
> > > > >
> > > > > The whole patch series can be found in one patch at:
> > > > >         https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/patch-5.2.11-rc1.gz
> > > > > or in the git tree and branch at:
> > > > >         git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-5.2.y
> > > > > and the diffstat can be found below.
> > > > >
> > > > > thanks,
> > > > >
> > > > > greg k-h
> > > > 
> > > > Results from Linaro’s test farm.
> > > > No regressions on arm64, arm, x86_64, and i386.
> > > 
> > > Thanks for testing all of these and letting us know.
> > > 
> > > Also, how did you all not catch the things that the redhat ci system was
> > > catching that caused us to add another networking aptch?
> > 
> > Hi Greg -
> > 
> > I'll follow up with them off list. That said, I expect different CI
> > setups to find different issues - that's the point, after all. It would
> > be bad if we all ran the exact same things, and found the exact same
> > things, because then we'd also miss the exact same things. In the macro
> > sense, there is a lot to test, and I would rather see CI teams go after
> > areas that are weak, rather than areas that are well covered.
> 
> I totally agree, but here we actually have a known failure (for once!)
> so it would be nice to see why the very large test suite that you all
> run missed this.

OK, so it seems the following two tests failed for them[0]:

>          ❌ Networking socket: fuzz [9]
>          ❌ Networking sctp-auth: sockopts test [10]

fuzz[1] seems to be a redhat test that was added to tests-beaker on May
20th this year. Similarly, sockopts[2] was added on May 7th.

We don't run tests-beaker. Maybe we should?

Major is coming to Linaro Connect in a few weeks to talk about CKI[3].
Topics such as test coverage and test suites are on our agenda to
discuss.

Hope that helps,
Dan

[0] https://lore.kernel.org/stable/291770ce-273a-68aa-a4a2-7655cbea2bcc@mhtx.net
[1] https://github.com/CKI-project/tests-beaker/commit/a454fdeaada71f7f193ae00e505621bf4a8ed8c6
[2] https://github.com/CKI-project/tests-beaker/commit/4ebc85dae5042cc2b70c98a868f3b3501eec1e08
[3] https://linaroconnectsandiego.sched.com/event/Suco/san19-416-transforming-kernel-developer-workflows-with-cicd


> 
> thanks,
> 
> greg k-h

-- 
Linaro - Kernel Validation