Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Message-ID: <1550088875.2871.21.camel@HansenPartnership.com>
Subject: Re: [LSF/MM TOPIC] FS, MM, and stable trees
From:   James Bottomley <James.Bottomley@HansenPartnership.com>
To:     Greg KH <gregkh@linuxfoundation.org>,
        Sasha Levin <sashal@kernel.org>
Cc:     Amir Goldstein <amir73il@gmail.com>,
        Steve French <smfrench@gmail.com>,
        lsf-pc@lists.linux-foundation.org,
        linux-fsdevel <linux-fsdevel@vger.kernel.org>,
        linux-mm <linux-mm@kvack.org>,
        LKML <linux-kernel@vger.kernel.org>,
        "Luis R. Rodriguez" <mcgrof@kernel.org>
Date:   Wed, 13 Feb 2019 12:14:35 -0800
In-Reply-To: <20190213195232.GA10047@kroah.com>
References: <20190212170012.GF69686@sasha-vm>
         <CAH2r5mviqHxaXg5mtVe30s2OTiPW2ZYa9+wPajjzz3VOarAUfw@mail.gmail.com>
         <CAOQ4uxjMYWJPF8wFF_7J7yy7KCdGd8mZChfQc5GzNDcfqA7UAA@mail.gmail.com>
         <20190213073707.GA2875@kroah.com>
         <CAOQ4uxgQGCSbhppBfhHQmDDXS3TGmgB4m=Vp3nyyWTFiyv6z6g@mail.gmail.com>
         <20190213091803.GA2308@kroah.com> <20190213192512.GH69686@sasha-vm>
         <20190213195232.GA10047@kroah.com>
Content-Type: text/plain; charset="UTF-8"
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, 2019-02-13 at 20:52 +0100, Greg KH wrote:
> On Wed, Feb 13, 2019 at 02:25:12PM -0500, Sasha Levin wrote:
> > On Wed, Feb 13, 2019 at 10:18:03AM +0100, Greg KH wrote:
> > > On Wed, Feb 13, 2019 at 11:01:25AM +0200, Amir Goldstein wrote:
> > > > Best effort testing in timely manner is good, but a good way to
> > > > improve confidence in stable kernel releases is a publicly
> > > > available list of tests that the release went through.
> > > 
> > > We have that, you aren't noticing them...
> > 
> > This is one of the biggest things I want to address: there is a
> > disconnect between the stable kernel testing story and the tests
> > the fs/ and mm/ folks expect to see here.
> > 
> > On one had, the stable kernel folks see these kernels go through
> > entire suites of testing by multiple individuals and organizations,
> > receiving way more coverage than any of Linus's releases.
> > 
> > On the other hand, things like LTP and selftests tend to barely
> > scratch the surface of our mm/ and fs/ code, and the maintainers of
> > these subsystems do not see LTP-like suites as something that adds
> > significant value and ignore them. Instead, they have a
> > (convoluted) set of testing they do with different tools and
> > configurations that qualifies their code as being "tested".
> > 
> > So really, it sounds like a low hanging fruit: we don't really need
> > to write much more testing code code nor do we have to refactor
> > existing test suites. We just need to make sure the right tests are
> > running on stable kernels. I really want to clarify what each
> > subsystem sees as "sufficient" (and have that documented
> > somewhere).
> 
> kernel.ci and 0-day and Linaro are starting to add the fs and mm
> tests to their test suites to address these issues (I think 0-day
> already has many of them).  So this is happening, but not quite
> obvious.  I know I keep asking Linaro about this :(

0day has xfstests at least, but it's opt-in only (you have to request
that it be run on your trees).  When I did it for the SCSI tree, I had
to email Fenguangg directly, there wasn't any other way of getting it.

James