Received-SPF: pass (google.com: domain of linux-kernel+bounces-173797-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
From: John Hubbard <jhubbard@nvidia.com>
To: Shuah Khan <shuah@kernel.org>
Cc: Phil Sutter <phil@nwl.cc>,
	Pablo Neira Ayuso <pablo@netfilter.org>,
	Florian Westphal <fw@strlen.de>,
	Felix Huettner <felix.huettner@mail.schwarz>,
	Max Lamprecht <max.lamprecht@mail.schwarz>,
	Luca Czesla <luca.czesla@mail.schwarz>,
	Xin Long <lucien.xin@gmail.com>,
	"David S . Miller" <davem@davemloft.net>,
	Paul Moore <paul@paul-moore.com>,
	Richard Guy Briggs <rgb@redhat.com>,
	Boris Sukholitko <boris.sukholitko@broadcom.com>,
	Valentin Obst <kernel@valentinobst.de>,
	linux-kselftest@vger.kernel.org,
	LKML <linux-kernel@vger.kernel.org>,
	llvm@lists.linux.dev,
	John Hubbard <jhubbard@nvidia.com>
Subject: [PATCH] selftests/netfilter: formally SKIP() two cases of setup failure
Date: Wed,  8 May 2024 13:23:22 -0700
Message-ID: <20240508202322.275565-1-jhubbard@nvidia.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?rIBhhIPV16boendiENVWLWgqyZYuHfscwV9agpgs/J2p36gE6UHB7zbjid6y?=
 =?us-ascii?Q?eDCDXnfDcl4f3W5bN6P3ze7zwbbEpH7/jXaxB2KxHtEihSrMBhIvlv3BJ9y6?=
 =?us-ascii?Q?ON325OIbHg1mzV+ETNiSsY2Drg2sYlfpCkpUBvT93c6kbZrPmJvsTRDzpwZ5?=
 =?us-ascii?Q?MXZIrv9BKYGZ4z2RL8384h4kSvW8MqSvgQkipzdcaedbm/f6JM6jTZwYoMmK?=
 =?us-ascii?Q?QuiwlZ+T3R/rzFd41TOKmkmyRR+hML/ooiSxLd7H008S6iCwDgRc7kONMjxd?=
 =?us-ascii?Q?bAi89C96HGx03S0lCk1kOrRg0ICzdMbfuUIvdKQQHDjduhvxrLX07QJz4511?=
 =?us-ascii?Q?Lcr44blB9L8UyGY0y/Cl3cmk2gKAzPgU73Ak+Fw0uvu1yET/CXoPSuoq7JyL?=
 =?us-ascii?Q?L3BO2D6QlGbi7Y61XZVDpEXNQ6WUtVHI7S9VV/FeH1cVKc8BUkmnQ8al1VoF?=
 =?us-ascii?Q?kfpl054lTQ8QsrzIlR5lClTL9mFVB1Z1rRrvmIYPYnkZ8/ujqYfTFGI6cAo5?=
 =?us-ascii?Q?N5N9CtSVmXdHu17MtTFapBOsyMCY+b4sGL5p44Ymf0bHvp2vwEQzIhZZZ3fh?=
 =?us-ascii?Q?+qMHJWCArxIurKe5SUJ8QZ8wh8JRd17z4ZOrGhJfo3fcAdzJVKmRHKhGQ7Pc?=
 =?us-ascii?Q?QEeEJNh5cMU2eJgplcdhtreW6GSiufBGWnmaVdt691yJD8EkAidLr0DaFoYv?=
 =?us-ascii?Q?aYZaTBQglBQ9PpuSBv36BE6nZcRv0ANrGhUQqlFQwv3k9gbFF8GLb46CnjWR?=
 =?us-ascii?Q?JxegHu9QZ6qlZjvdaKJQmKGHV9igH4PFty1uo2i9x0B/maEtEBypHIKcHUg+?=
 =?us-ascii?Q?BNM0QEVWe2XmYLZd6TM9xIBQ2u+WQ2LF0vd3C/9WlW9LFuDm312zxGXyRiwB?=
 =?us-ascii?Q?fDHIzT9SHbgLswqkbIvVbl5t1RcyvvQd82k/WNaLfYdKFcVxaxi7n0EMiIQF?=
 =?us-ascii?Q?A1wQXAmKM78AkkxLBcUOzvDixl9FfJjU1Mbs1Dbpx/KzEak6sfP9CLwldwmv?=
 =?us-ascii?Q?oPyORQ/vCWFhjXoc2jkn9ndWika04yddexV1p+jtoEIXAF/mLDz2pbX2Cn/w?=
 =?us-ascii?Q?fqyHdemAKTv8rXHzwjbqNw0rPYQjDEUBKQLgk6rBCsognO8mbV/HEWlMxB2v?=
 =?us-ascii?Q?LeK5OkbDZJFh+VCrHco+B9e06HGf/0icyACjHiWxIvFq2JQa6UFufCKOkgvm?=
 =?us-ascii?Q?UcDiu6wMoSiEx/tacUDSGMEhMOLAMtVbFS8Ga+x8u4fKfQt1J2A1pG/rGHwP?=
 =?us-ascii?Q?rAyt3S5l/kYJDhdJpUI1s1MAZ3vPgPZxUtz0sEYTsYzkBn5vWnaFjvPFIb7q?=
 =?us-ascii?Q?gr8vyOCNAsh5VYBJmHSZm5PpFS7MoB1T9efczsvaFYmx78c6WdIb5jugq4EC?=
 =?us-ascii?Q?Ra0a699xo+v5YlOHP+o0ITj+bMlam/iLjFPb9uMplK7duGGrkbuk+JdnRq2R?=
 =?us-ascii?Q?Z5jD+60zDlyG3UeFk1rfVrkidfOVZhqWmjMgUpMyOD0MHiBmGLxGxT61hAO/?=
 =?us-ascii?Q?+e1RSVM+lTc+ncy/kB8EUCmWaueQL7iwcEvNyDAjwBksETU0NuofJgOF6n0Z?=
 =?us-ascii?Q?/FwaWDXHszhyc72n1gnKkweL/d1RGYx8asuawMIw?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e4b80253-b170-492d-07a1-08dc6f9cb6b6
X-MS-Exchange-CrossTenant-AuthSource: BY5PR12MB4130.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 20:23:25.1625
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: zII8W7hTieDqEZPnbjA83CLLuQBvnOQ5ExS704en6947hjPXLSo7sjWGJG84bHSKU14NncyvHR+cwsGckxJJ0Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB6237

If mnl_socket_open() or mnl_socket_bind() fails, it's generally due to
not having the user space parts fully installed and configured
correctly. This was previously ignored and reported as a test PASS, but
what really happened is that the tests were being skipped. This led to
generating inaccurate TAP output (I've omitted the leading '#'
character):

   Totals: pass:3 fail:0 xfail:0 xpass:0 skip:0 error:0

Fix this by using kselftest's SKIP() macro. The new output on the same
misconfigured system shows:

    Totals: pass:0 fail:0 xfail:0 xpass:0 skip:3 error:0

This was briefly discussed already with Felix Huettner [1].

[1] https://lore.kernel.org/all/edca87be-d8a7-4c62-b9c1-f9b3f5752595@nvidia.com/

Cc: Felix Huettner <felix.huettner@mail.schwarz>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
 tools/testing/selftests/netfilter/conntrack_dump_flush.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/netfilter/conntrack_dump_flush.c b/tools/testing/selftests/netfilter/conntrack_dump_flush.c
index e9df4ae14e16..4a73afad4de4 100644
--- a/tools/testing/selftests/netfilter/conntrack_dump_flush.c
+++ b/tools/testing/selftests/netfilter/conntrack_dump_flush.c
@@ -317,12 +317,12 @@ FIXTURE_SETUP(conntrack_dump_flush)
 	self->sock = mnl_socket_open(NETLINK_NETFILTER);
 	if (!self->sock) {
 		perror("mnl_socket_open");
-		exit(EXIT_FAILURE);
+		SKIP(exit(EXIT_FAILURE), "mnl_socket_open() failed");
 	}
 
 	if (mnl_socket_bind(self->sock, 0, MNL_SOCKET_AUTOPID) < 0) {
 		perror("mnl_socket_bind");
-		exit(EXIT_FAILURE);
+		SKIP(exit(EXIT_FAILURE), "mnl_socket_bind() failed");
 	}
 
 	ret = conntracK_count_zone(self->sock, TEST_ZONE_ID);

base-commit: 45db3ab70092637967967bfd8e6144017638563c
prerequisite-patch-id: b901ece2a5b78503e2fb5480f20e304d36a0ea27
prerequisite-patch-id: 9db2d20be98dc44731d8605a3da64ff118d2546d
prerequisite-patch-id: 72413b9b47d66666f20967a664470199892fe282
-- 
2.45.0