Date: Thu, 8 Jun 2017 18:44:10 +0200
From: Greg KH <gregkh@linuxfoundation.org>
To: Tom Gall <tom.gall@linaro.org>
Cc: stable@vger.kernel.org, linux-kernel@vger.kernel.org, shuah@kernel.org
Subject: Re: kselftest backports?
Message-ID: <20170608164410.GA10597@kroah.com>
References: <CAOfXc9FMwPUPVdgC-0f4qzUyjNaptGQGP9-ZZvAh2B+30bw4rg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAOfXc9FMwPUPVdgC-0f4qzUyjNaptGQGP9-ZZvAh2B+30bw4rg@mail.gmail.com>
User-Agent: Mutt/1.8.3 (2017-05-23)
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2060
Lines: 51

On Thu, Jun 08, 2017 at 10:37:55AM -0500, Tom Gall wrote:
> We've been running kselftests for ARM and x86 hardware in an effort to
> detect regressions in various kernels including LTS and candidate
> patches.
> 
> One general question we've had is what's the right thing to do when it
> comes to running kselftest on older LTS kernels. Let's pick on 4.4 for
> the purposes of this discussion tho obviously the example applies to
> other versions.
> 
> 1) Run current top of tree, kselftest on a 4.4 LTS?
> 2) Run kselftest from 4.4 on 4.4 LTS?
> 
> #1 gets latest greatest set of tests but obviously there can be
> breakage because of how the kernel evolves over time.

Really?  What breaks?

> #2 misses tests that are later developed but otherwise fine
> 
> Regardless of the right answer, some obvious questions but I'll ask anyway:
> -) Shouldn't new additions to kselftest that work on older kernels be
> backported and submitted on the stable list?

No, that would be a new feature, and new features are not allowed in the
stable kernel releases, right?

And as the kernel/user api is "stable", there should never be any
problem with running new tests from newer kernel releases on older
kernel, right?  If so, I think that would be a bug and the test should
be fixed.  Right now we have a lot of tests that properly "degrade" if
the requested feature is not present, so I think things already work
properly today.

Do you know of any kselftests from 4.12-rc4 that are failing on 4.4
because of bugs in the tests?

> -) In the case where some test is kernel version specific for whatever
> reason, I presume building in version detection into kselftest makes
> sense?

No, it's a feature detection issue, look at how lots of tests already
degrade if a specific syscall is not present.  That should never be a
kernel release number detection as that would break horribly on things
like the "enterprise" Linux distro kernels who backport all sorts of new
features, and yet keep an old kernel version to make people feel warm
and fuzzy.

thanks,

greg k-h