Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Matthieu Baerts <matthieu.baerts@tessares.net>
Subject: [PATCH net 00/14] selftests: mptcp: skip tests not supported by
 old kernels (part 2)
Date:   Thu, 08 Jun 2023 18:38:42 +0200
Message-Id: <20230608-upstream-net-20230608-mptcp-selftests-support-old-kernels-part-2-v1-0-20997a6fd841@tessares.net>
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
To:     mptcp@lists.linux.dev, Mat Martineau <martineau@kernel.org>,
        "David S. Miller" <davem@davemloft.net>,
        Eric Dumazet <edumazet@google.com>,
        Jakub Kicinski <kuba@kernel.org>,
        Paolo Abeni <pabeni@redhat.com>, Shuah Khan <shuah@kernel.org>,
        Davide Caratti <dcaratti@redhat.com>,
        Dmytro Shytyi <dmytro@shytyi.net>,
        Menglong Dong <imagedong@tencent.com>,
        Geliang Tang <geliang.tang@suse.com>,
        Kishen Maloor <kishen.maloor@intel.com>
Cc:     Florian Westphal <fw@strlen.de>, netdev@vger.kernel.org,
        linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org,
        Matthieu Baerts <matthieu.baerts@tessares.net>,
        stable@vger.kernel.org
Precedence: bulk

After a few years of increasing test coverage in the MPTCP selftests, we
realised [1] the last version of the selftests is supposed to run on old
kernels without issues.

Supporting older versions is not that easy for this MPTCP case: these
selftests are often validating the internals by checking packets that
are exchanged, when some MIB counters are incremented after some
actions, how connections are getting opened and closed in some cases,
etc. In other words, it is not limited to the socket interface between
the userspace and the kernelspace.

In addition to that, the current MPTCP selftests run a lot of different
sub-tests but the TAP13 protocol used in the selftests don't support
sub-tests: one failure in sub-tests implies that the whole selftest is
seen as failed at the end because sub-tests are not tracked. It is then
important to skip sub-tests not supported by old kernels.

To minimise the modifications and reduce the complexity to support old
versions, the idea is to look at external signs and skip the whole
selftests or just some sub-tests before starting them. This cannot be
applied in all cases.

This second part focuses on marking different sub-tests as skipped if
some MPTCP features are not supported. A few techniques are used here:

- Before starting some tests:

  - Check if a file (sysctl knob) is present: that's what patch 13/14 is
    doing for the userspace PM feature.

  - Check if a symbol is present in /proc/kallsyms: patch 1/14 adds some
    helpers in mptcp_lib.sh to ease its use. Then these helpers are used
    in patches 2, 3, 4, 10, 11 and 14/14.

  - Set a flag and get the status to check if a feature is supported:
    patch 8/14 is doing that with the 'fullmesh' flag.

- After having launched the tests:

  - Retrieve the counters after a test and check if they are different
    than 0. Similar to the check with the flag, that's not ideal but in
    this case, the counters were already present before the introduction
    of MPTCP but they have been supported by MPTCP sockets only later.
    Patches 5 and 6/14 are using this technique.

Before skipping tests, SELFTESTS_MPTCP_LIB_EXPECT_ALL_FEATURES env var
value is checked: if it is set to 1, the test is marked as "failed"
instead of "skipped". MPTCP public CI expects to have all features
supported and it sets this env var to 1 to catch regressions in these
new checks.

Patches 7/14 and 9/14 are a bit different because they don't skip tests:

- Patch 7/14 retrieves the default values instead of using hardcoded
  ones because these default values have been modified at some points.
  Then the comparisons are done with the default values.

- patch 9/14 relaxes the expected returned size from MPTCP's getsockopt
  because the different structures gathering various info can get new
  fields and get bigger over time. We cannot expect that the userspace
  is using the same structure as the kernel.

Patch 12/14 marks the test as "skipped" instead of "failed" if the "ip"
tool is not available.

In this second part, the "mptcp_join" selftest is not modified yet. This
will come soon after in the third part with quite a few patches.

Link: https://lore.kernel.org/stable/CA+G9fYtDGpgT4dckXD-y-N92nqUxuvue_7AtDdBcHrbOMsDZLg@mail.gmail.com/ [1]
Link: https://github.com/multipath-tcp/mptcp_net-next/issues/368
Signed-off-by: Matthieu Baerts <matthieu.baerts@tessares.net>
---
Matthieu Baerts (14):
      selftests: mptcp: lib: skip if missing symbol
      selftests: mptcp: connect: skip transp tests if not supported
      selftests: mptcp: connect: skip disconnect tests if not supported
      selftests: mptcp: connect: skip TFO tests if not supported
      selftests: mptcp: diag: skip listen tests if not supported
      selftests: mptcp: diag: skip inuse tests if not supported
      selftests: mptcp: pm nl: remove hardcoded default limits
      selftests: mptcp: pm nl: skip fullmesh flag checks if not supported
      selftests: mptcp: sockopt: relax expected returned size
      selftests: mptcp: sockopt: skip getsockopt checks if not supported
      selftests: mptcp: sockopt: skip TCP_INQ checks if not supported
      selftests: mptcp: userspace pm: skip if 'ip' tool is unavailable
      selftests: mptcp: userspace pm: skip if not supported
      selftests: mptcp: userspace pm: skip PM listener events tests if unavailable

 tools/testing/selftests/net/mptcp/config           |  1 +
 tools/testing/selftests/net/mptcp/diag.sh          | 42 +++++++++-------------
 tools/testing/selftests/net/mptcp/mptcp_connect.sh | 20 +++++++++++
 tools/testing/selftests/net/mptcp/mptcp_lib.sh     | 38 ++++++++++++++++++++
 tools/testing/selftests/net/mptcp/mptcp_sockopt.c  | 18 ++++++----
 tools/testing/selftests/net/mptcp/mptcp_sockopt.sh | 20 +++++++++--
 tools/testing/selftests/net/mptcp/pm_netlink.sh    | 27 ++++++++------
 tools/testing/selftests/net/mptcp/userspace_pm.sh  | 13 ++++++-
 8 files changed, 135 insertions(+), 44 deletions(-)
---
base-commit: 6c0ec7ab5aaff3706657dd4946798aed483b9471
change-id: 20230608-upstream-net-20230608-mptcp-selftests-support-old-kernels-part-2-6e337e1f047d

Best regards,
-- 
Matthieu Baerts <matthieu.baerts@tessares.net>