Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751306AbaKEXXT (ORCPT ); Wed, 5 Nov 2014 18:23:19 -0500 Received: from mail-ob0-f178.google.com ([209.85.214.178]:62651 "EHLO mail-ob0-f178.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750867AbaKEXXP (ORCPT ); Wed, 5 Nov 2014 18:23:15 -0500 MIME-Version: 1.0 In-Reply-To: <5459650E.6070201@osg.samsung.com> References: <5459650E.6070201@osg.samsung.com> Date: Wed, 5 Nov 2014 15:23:14 -0800 X-Google-Sender-Auth: T-AfqlqiBjqvZkxkMW1PJUANfHk Message-ID: Subject: Re: [PATCH 00/20] kselftest install target feature From: Kees Cook To: Shuah Khan Cc: Greg KH , Andrew Morton , Michal Marek , "David S. Miller" , Phong Tran , David Herrmann , Hugh Dickins , pranith kumar , "Eric W. Biederman" , "Serge E. Hallyn" , linux-kbuild , LKML , Linux API , Network Development Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Nov 4, 2014 at 3:45 PM, Shuah Khan wrote: > On 11/04/2014 12:22 PM, Kees Cook wrote: >> On Tue, Nov 4, 2014 at 9:10 AM, Shuah Khan wrote: >>> This patch series adds a new kselftest_install make target >>> to enable selftest install. When make kselftest_install is >>> run, selftests are installed on the system. A new install >>> target is added to selftests Makefile which will install >>> targets for the tests that are specified in INSTALL_TARGETS. >>> During install, a script is generated to run tests that are >>> installed. This script will be installed in the selftest install >>> directory. Individual test Makefiles are changed to add to the >>> script. This will allow new tests to add install and run test >>> commands to the generated kselftest script. >> >> I'm all for making the self tests more available, but I don't think >> this is the right approach. My primary objection is that it creates a >> second way to run tests, and that means any changes and additions need >> to be updated in two places. I'd much rather just maintain the single >> "make" targets instead. Having "make" available on the target device >> doesn't seem too bad to me. Is there a reason that doesn't work for >> your situation? > > Kees, > > My primary objective is to provide a way to install selftests for a > specific kernel release. This will allow developers to run tests for > a specific release and look for regressions. Adding an install target > will also help support local execution of tests in a virtualized > environments. In some cases such as qemu, it is not practical to > expect the target to have support for "make". Once tests are installed > to be run outside the git environment, we need a master script that > can run the tests. Hence the need for a master script that can run > tests. > > We have the ability to run all tests via make kselftest target or > run a specific test using the individual test's run_tests target. > Both of above are necessary to support running tests from the tree. > Embedding run_tests logic in the makefiles doesn't work very well > in the long run. > > We also need a way to run them outside tree. I agree with you that > the way I added the script generation, duplicates the code in individual > run_tests targets and that changes/updates need to be made in both > places. > > Would you be ok with the approach if I fixed the duplicating > problem? I can address the duplication concern easily. Yeah, getting rid of duplication would be much preferred. Thanks! -Kees > >> >> I would, however, like to see some better standardization of the test >> "framework" that we've got in there already. (For example, some >> failures fail the "make", some don't, there are various reporting >> methods for success/failure depending on the test, etc.) > > This is being addressed and I have the framework in linux-kselftest > git next branch at the moment. I do think the above work is part of > addressing the larger framework issues such as being able to run tests > on a target system that might not have "make" support and makes it > easier to use. > > thanks, > -- Shuah > > > -- > Shuah Khan > Sr. Linux Kernel Developer > Samsung Research America (Silicon Valley) > shuahkh@osg.samsung.com | (970) 217-8978 -- Kees Cook Chrome OS Security -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/