Received-SPF: pass (google.com: domain of linux-kernel+bounces-99290-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Message-ID: <f877ce53-a5ee-4447-a57c-339a1e9701f4@collabora.com>
Date: Mon, 11 Mar 2024 22:10:21 +0500
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Cc: Muhammad Usama Anjum <usama.anjum@collabora.com>, kernel@collabora.com,
 kernel-janitors@vger.kernel.org, linux-kselftest@vger.kernel.org,
 linux-mm@kvack.org, linux-kernel@vger.kernel.org,
 Kees Cook <keescook@chromium.org>
Subject: Re: [PATCH 1/3] selftests/exec: Add the overall result line
 accourding to TAP
Content-Language: en-US
To: Shuah Khan <shuah@kernel.org>, Eric Biederman <ebiederm@xmission.com>
References: <20240304155928.1818928-1-usama.anjum@collabora.com>
From: Muhammad Usama Anjum <usama.anjum@collabora.com>
In-Reply-To: <20240304155928.1818928-1-usama.anjum@collabora.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Soft reminder!

On 3/4/24 8:59 PM, Muhammad Usama Anjum wrote:
> The following line is missing from the test's execution. Add it to make
> it fully TAP conformant:
>   # Totals: pass:27 fail:0 xfail:0 xpass:0 skip:0 error:0
> 
> Signed-off-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
> ---
>  tools/testing/selftests/exec/binfmt_script.py | 10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/exec/binfmt_script.py b/tools/testing/selftests/exec/binfmt_script.py
> index 05f94a741c7aa..2c575a2c0eab4 100755
> --- a/tools/testing/selftests/exec/binfmt_script.py
> +++ b/tools/testing/selftests/exec/binfmt_script.py
> @@ -16,6 +16,8 @@ SIZE=256
>  NAME_MAX=int(subprocess.check_output(["getconf", "NAME_MAX", "."]))
>  
>  test_num=0
> +pass_num=0
> +fail_num=0
>  
>  code='''#!/usr/bin/perl
>  print "Executed interpreter! Args:\n";
> @@ -42,7 +44,7 @@ foreach my $a (@ARGV) {
>  # ...
>  def test(name, size, good=True, leading="", root="./", target="/perl",
>                       fill="A", arg="", newline="\n", hashbang="#!"):
> -    global test_num, tests, NAME_MAX
> +    global test_num, pass_num, fail_num, tests, NAME_MAX
>      test_num += 1
>      if test_num > tests:
>          raise ValueError("more binfmt_script tests than expected! (want %d, expected %d)"
> @@ -80,16 +82,20 @@ def test(name, size, good=True, leading="", root="./", target="/perl",
>          if good:
>              print("ok %d - binfmt_script %s (successful good exec)"
>                    % (test_num, name))
> +            pass_num += 1
>          else:
>              print("not ok %d - binfmt_script %s succeeded when it should have failed"
>                    % (test_num, name))
> +            fail_num = 1
>      else:
>          if good:
>              print("not ok %d - binfmt_script %s failed when it should have succeeded (rc:%d)"
>                    % (test_num, name, proc.returncode))
> +            fail_num = 1
>          else:
>              print("ok %d - binfmt_script %s (correctly failed bad exec)"
>                    % (test_num, name))
> +            pass_num += 1
>  
>      # Clean up crazy binaries
>      os.unlink(script)
> @@ -166,6 +172,8 @@ test(name="two-under-trunc-arg", size=int(SIZE/2), arg=" ")
>  test(name="two-under-leading",   size=int(SIZE/2), leading=" ")
>  test(name="two-under-lead-trunc-arg", size=int(SIZE/2), leading=" ", arg=" ")
>  
> +print("# Totals: pass:%d fail:%d xfail:0 xpass:0 skip:0 error:0" % (pass_num, fail_num))
> +
>  if test_num != tests:
>      raise ValueError("fewer binfmt_script tests than expected! (ran %d, expected %d"
>                       % (test_num, tests))

-- 
BR,
Muhammad Usama Anjum