Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates
 216.228.112.34 as permitted sender) receiver=protection.outlook.com;
 client-ip=216.228.112.34; helo=mail.nvidia.com;
References: <20210525113316.25416-1-po-hsu.lin@canonical.com>
User-agent: mu4e 1.4.10; emacs 27.1
From:   Petr Machata <petrm@nvidia.com>
To:     Po-Hsu Lin <po-hsu.lin@canonical.com>
CC:     <linux-kselftest@vger.kernel.org>, <netdev@vger.kernel.org>,
        <bpf@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
        <shuah@kernel.org>, <skhan@linuxfoundation.org>, <ast@kernel.org>,
        <daniel@iogearbox.net>, <andrii@kernel.org>, <kafai@fb.com>,
        <songliubraving@fb.com>, <yhs@fb.com>, <john.fastabend@gmail.com>,
        <kpsingh@kernel.org>, <davem@davemloft.net>, <kuba@kernel.org>,
        <hawk@kernel.org>, <nikolay@nvidia.com>, <gnault@redhat.com>,
        <vladimir.oltean@nxp.com>, <idosch@nvidia.com>,
        <baowen.zheng@corigine.com>, <danieller@nvidia.com>,
        <petrm@nvidia.com>,
        Willem de Bruijn <willemdebruijn.kernel@gmail.com>
Subject: Re: [PATCHv2] selftests: Use kselftest skip code for skipped tests
In-Reply-To: <20210525113316.25416-1-po-hsu.lin@canonical.com>
Date:   Wed, 26 May 2021 10:53:52 +0200
Message-ID: <87y2c1swnz.fsf@nvidia.com>
MIME-Version: 1.0
Content-Type: text/plain
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 May 2021 08:54:01.1113
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 0176b126-049f-4e81-d595-08d92023cea1
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.34];Helo=[mail.nvidia.com]
X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT022.eop-nam11.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4859
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


Po-Hsu Lin <po-hsu.lin@canonical.com> writes:

> There are several test cases still using exit 0 when they need to be
> skipped. Use kselftest framework skip code instead so it can help us
> to distinguish the proper return status.
>
> Criterion to filter out what should be fixed in selftests directory:
>   grep -r "exit 0" -B1 | grep -i skip
>
> This change might cause some false-positives if people are running
> these test scripts directly and only checking their return codes,
> which will change from 0 to 4. However I think the impact should be
> small as most of our scripts here are already using this skip code.
> And there will be no such issue if running them with the kselftest
> framework.
>
> V2: router_mpath_nh.sh and outer_mpath_nh_res.sh sources lib.sh,
> there is no need to assign ksft_skip value in these two.
>
> Signed-off-by: Po-Hsu Lin <po-hsu.lin@canonical.com>

I want to note that defining ksft_skip=4 in every test separately is the
current practice. I agree with Willem (in a parallel thread) that this
stuff should live in a library of its own, but there is none currently.
When there is, it looks like the conversion would be mechanical.

Which is to say, IMHO this patch makes sense on its own as an
incremental improvement.

Reviewed-by: Petr Machata <petrm@nvidia.com>