Reply-To: shuah@kernel.org
Subject: Re: LTS testing with latest kselftests - some failures
To: Sumit Semwal <sumit.semwal@linaro.org>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        LKML <linux-kernel@vger.kernel.org>, stable@vger.kernel.org,
        linux-kselftest@vger.kernel.org
Cc: Brian Norris <computersforpeace@gmail.com>, mcgrof@kernel.org,
        Kees Cook <keescook@chromium.org>, Shuah Khan <shuah@kernel.org>
References: <CAO_48GFNXyqTDN85y8FKLV3wj19JJ28sWXuX4Vex3LVzaxzGCw@mail.gmail.com>
From: Shuah Khan <shuah@kernel.org>
Message-ID: <757548e7-e7d6-9c0b-f094-7b838538fbbf@kernel.org>
Date: Thu, 15 Jun 2017 12:30:04 -0600
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.1.1
MIME-Version: 1.0
In-Reply-To: <CAO_48GFNXyqTDN85y8FKLV3wj19JJ28sWXuX4Vex3LVzaxzGCw@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2630
Lines: 76

Hi Sumit,

On 06/15/2017 11:56 AM, Sumit Semwal wrote:
> Hello Greg, Shuah,
> 
> While testing 4.4.y and 4.9.y LTS kernels with latest kselftest, we
> found a couple more test failures due to test-kernel mismatch:
> 
> 1. firmware tests: - linux 4.5 [1] and 4.10 [2] added a few updates to
> tests, and related updates to lib/test_firmware.c to improve the
> tests. Stable-4.4 misses these patches to lib/test_firmware.c. Stable
> 4.9 misses the second update.

I will take a look the commit list your provided and see if it makes sense
to back-port. As we discussed earlier in this thread, fixes are something
that can be back-ported to include in the stable releases. Updates will not
be.

> 2. Bitmap test - test got added in 4.5, fails if test_bitmap.ko isn't present.

If you can send a patch to have it exit gracefully and change the message
to say unsupported feature, I can pull that into mainline can mark it for
stable inclusion if it qualifies.

> 
> 3. 'seccomp ptrace hole closure' patches got added in 4.7 [3] -
> feature and test together.
> - This one also seems like a security hole being closed, and the
> 'feature' could be a candidate for stable backports, but Arnd tried
> that, and it was quite non-trivial. So perhaps  we'll need some help
> from the subsystem developers here.

This is something I need to look at and consult with Security maintainers.

> 
> For all the 3 listed above, we will try and update the tests to gracefully exit.

That is great.

> 
> 
> 4. bpf tests: These seem to have build failures in mainline as well -
> I also tried to build kselftest-next, but a simple 'make -C
> tools/testing/selftests/bpf' seems to error out. Are there any special
> instructions to build these? [I tried x86_64, arm64 cross-compile on x86_64]

Hmm. I didn't notice this one. I will check and see what's going on there.

> 
> 
> I will also individually request subsystem authors / mailing lists for
> each of these towards help in improving these tests if required, but
> wanted to use this thread as a converging point.

Please cc me and linux-kselftest mailing list.

Thanks for reporting the problems.

> 
> 
> [1]: https://lkml.org/lkml/2015/12/8/816
> Patches added via [1]:
> eb910947c82f (test: firmware_class: add asynchronous request trigger)
> be4a1326d12c (test: firmware_class: use kstrndup() where appropriate)
> 47e0bbb7fa98 (test: firmware_class: report errors properly on failure)
> 
> [2]: https://lkml.org/lkml/2017/1/23/440
> Patch added via [2]:
> 061132d2b9c9 (test_firmware: add test custom fallback trigger)
> 
> [3]: https://lkml.org/lkml/2016/6/9/627
> 
> 

thanks,
-- Shuah