Received-SPF: pass (google.com: domain of linux-kernel+bounces-87041-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
From: Laura Nao <laura.nao@collabora.com>
To: skhan@linuxfoundation.org
Cc: a.hindborg@samsung.com,
	alex.gaynor@gmail.com,
	aliceryhl@google.com,
	benno.lossin@proton.me,
	bjorn3_gh@protonmail.com,
	boqun.feng@gmail.com,
	gary@garyguo.net,
	kernel@collabora.com,
	kernel@valentinobst.de,
	laura.nao@collabora.com,
	linux-kernel@vger.kernel.org,
	linux-kselftest@vger.kernel.org,
	ojeda@kernel.org,
	rust-for-linux@vger.kernel.org,
	sergio.collado@gmail.com,
	shuah@kernel.org,
	usama.anjum@collabora.com,
	wedsonaf@gmail.com
Subject: Re: [PATCH v4] kselftest: Add basic test for probing the rust sample modules
Date: Thu, 29 Feb 2024 16:57:11 +0100
Message-Id: <20240229155711.264231-1-laura.nao@collabora.com>
In-Reply-To: <5df7d5b1-1387-41f5-a9e9-29c9aefa448e@linuxfoundation.org>
References: <5df7d5b1-1387-41f5-a9e9-29c9aefa448e@linuxfoundation.org>
Precedence: bulk
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Hi Shuah,

On 2/28/24 01:01, Shuah Khan wrote:
> Hi Laura,
> 
> On 2/26/24 03:16, Laura Nao wrote:
>> Add new basic kselftest that checks if the available rust sample modules
>> can be added and removed correctly.
>>
>> Signed-off-by: Laura Nao <laura.nao@collabora.com>
>> Reviewed-by: Sergio Gonzalez Collado <sergio.collado@gmail.com>
>> Reviewed-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
>> ---
>> Depends on:
>> - 
>> https://lore.kernel.org/all/20240102141528.169947-1-laura.nao@collabora.com/T/#u
>> - 
>> https://lore.kernel.org/all/20240131-ktap-sh-helpers-extend-v1-0-98ffb468712c@collabora.com/
>> Changes in v4:
>> - Added config file
>> Changes in v3:
>> - Removed useless KSFT_PASS, KSFT_FAIL, KSFT_SKIP constants
>> - Used ktap_finished to print the results summary and handle the 
>> return code
>> Changes in v2:
>> - Added missing SPDX line
>> - Edited test_probe_samples.sh script to use the common KTAP helpers file
>> ---
>>   MAINTAINERS                                   |  1 +
>>   tools/testing/selftests/Makefile              |  1 +
>>   tools/testing/selftests/rust/Makefile         |  4 +++
>>   tools/testing/selftests/rust/config           |  5 +++
>>   .../selftests/rust/test_probe_samples.sh      | 34 +++++++++++++++++++
>>   5 files changed, 45 insertions(+)
>>   create mode 100644 tools/testing/selftests/rust/Makefile
>>   create mode 100644 tools/testing/selftests/rust/config
>>   create mode 100755 tools/testing/selftests/rust/test_probe_samples.sh
>>
> 
> I ran test again and I still see the same. I would like to
> see the script to handle error conditions.
> 
>> diff --git a/MAINTAINERS b/MAINTAINERS
>>
>> +
>> +DIR="$(dirname "$(readlink -f "$0")")"
>> +
>> +source "${DIR}"/../kselftest/ktap_helpers.sh
> 
> It tries to source and keeps going. Why can't we test for
> the file to exist and skip gracefully without printing
> the following messages.
> 
>   ./test_probe_samples.sh: line 12: 
> /linux/linux_6.8/tools/testing/selftests/rust/../kselftest/ktap_helpers.sh: No such file or director
> # ./test_probe_samples.sh: line 16: ktap_print_header: command not found
> # ./test_probe_samples.sh: line 18: ktap_set_plan: command not found
> # ./test_probe_samples.sh: line 22: ktap_test_skip: command not found
> # ./test_probe_samples.sh: line 22: ktap_test_skip: command not found
> # ./test_probe_samples.sh: line 34: ktap_finished: command not found
> 
> 
> 
> not ok 1 selftests: rust: test_probe_samples.sh # exit=127
> 
> 

Sorry, I misunderstood the request in your previous reply on v3 - will 
do.

>> +
>> +rust_sample_modules=("rust_minimal" "rust_print")
>> +
>> +ktap_print_header
>> +
>> +ktap_set_plan "${#rust_sample_modules[@]}"
>> +
>> +for sample in "${rust_sample_modules[@]}"; do
>> +    if ! /sbin/modprobe -n -q "$sample"; then
>> +        ktap_test_skip "module $sample is not found in 
>> /lib/modules/$(uname -r)"
>> +        continue
> 
> Why are we continuing here? Isn't this skip condition?
>

At first, I hadn't planned for the kselftest to skip entirely if only
one of the two sample modules was missing. However, considering that 
this kselftest is designed to test all available sample modules, and 
given that both are enabled with the provided configuration file, I 
believe it's more logical to verify the presence of both modules before 
running the test. If either of them is missing, then we exit the test 
with a skip code. This also covers the case where rust is not available.

>> +    fi
>> +
>> +    if /sbin/modprobe -q "$sample"; then
>> +        /sbin/modprobe -q -r "$sample"
>> +        ktap_test_pass "$sample"
>> +    else
>> +        ktap_test_fail "$sample"
>> +    fi
>> +done
>> +
>> +ktap_finished
> 
> 
> I would like to see the test exit with skip code when RUST isn't
> enabled. Please refer to existing tests that do this properly.
> 

Sent a v5 with the changes mentioned above: https://lore.kernel.org/linux-kselftest/20240229155235.263157-1-laura.nao@collabora.com/T/#u

Thanks!

Laura