Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <cover.1690332693.git.yan@cloudflare.com> <9c4896b109a39c3fa088844addaa1737a84bbbb5.1690332693.git.yan@cloudflare.com>
 <791b919c-de82-6dc8-905a-520543f975cd@linux.dev> <87edkoflvx.fsf@cloudflare.com>
In-Reply-To: <87edkoflvx.fsf@cloudflare.com>
From:   Alexei Starovoitov <alexei.starovoitov@gmail.com>
Date:   Mon, 31 Jul 2023 11:46:56 -0700
Message-ID: <CAADnVQ+7akLU_rY7xe=LbvY=bLiN8C5Vk5m0iwbao6hVH9v88w@mail.gmail.com>
Subject: Re: [PATCH v4 bpf 2/2] bpf: selftests: add lwt redirect regression
 test cases
To:     Jakub Sitnicki <jakub@cloudflare.com>
Cc:     Martin KaFai Lau <martin.lau@linux.dev>,
        Yan Zhai <yan@cloudflare.com>, bpf <bpf@vger.kernel.org>,
        Alexei Starovoitov <ast@kernel.org>,
        Daniel Borkmann <daniel@iogearbox.net>,
        Andrii Nakryiko <andrii@kernel.org>,
        Song Liu <song@kernel.org>, Yonghong Song <yhs@fb.com>,
        John Fastabend <john.fastabend@gmail.com>,
        KP Singh <kpsingh@kernel.org>,
        Stanislav Fomichev <sdf@google.com>,
        Hao Luo <haoluo@google.com>, Jiri Olsa <jolsa@kernel.org>,
        "David S. Miller" <davem@davemloft.net>,
        Eric Dumazet <edumazet@google.com>,
        Jakub Kicinski <kuba@kernel.org>,
        Paolo Abeni <pabeni@redhat.com>,
        Mykola Lysenko <mykolal@fb.com>, Shuah Khan <shuah@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Network Development <netdev@vger.kernel.org>,
        "open list:KERNEL SELFTEST FRAMEWORK" 
        <linux-kselftest@vger.kernel.org>,
        kernel-team <kernel-team@cloudflare.com>,
        Jordan Griege <jgriege@cloudflare.com>,
        Markus Elfring <Markus.Elfring@web.de>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Mon, Jul 31, 2023 at 2:52=E2=80=AFAM Jakub Sitnicki <jakub@cloudflare.co=
m> wrote:
>
> On Fri, Jul 28, 2023 at 03:47 PM -07, Martin KaFai Lau wrote:
> > On 7/25/23 6:09 PM, Yan Zhai wrote:
>
> [...]
>
> >> diff --git a/tools/testing/selftests/bpf/test_lwt_redirect.sh
> >> b/tools/testing/selftests/bpf/test_lwt_redirect.sh
> >> new file mode 100755
> >> index 000000000000..1b7b78b48174
> >> --- /dev/null
> >> +++ b/tools/testing/selftests/bpf/test_lwt_redirect.sh
> >
> > This has to be written in the test_progs infrastructure in C. Only test=
_progs is
> > run by the BPF CI. Take a look at other tests in prog_tests/. For examp=
le,
> > tc_redirect.c and xdp_metadata.c which are having setup in netns/link/.=
..etc. It
> > currently has helpers to add tc qdisc and filter but not adding route y=
et which
> > could be a useful addition.
>
> Can we help make the BPF CI better so that it also runs other tests in
> addition test_progs?

Not really.
CI is not just running the test. It needs to understand the output,
pass it to UI, run in parallel, etc.
All the shell scripts are not suitable for long term CI exposure.

So I completely agree with Martin. No new shell scripts.
All selftests must be in test_progs.

> We have bpf selftests written in shell and even Python. These are
> sometimes the right tools for the job and make adding tests easier,
> IMHO. Network setup from C is verbose and tedious. Not to mention, hard
> to read through.

For comparison take a look at BPF CI code base and what it takes to run
the tests and process the output. There is plenty of work for CI ahead.