Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20210316211837.910506-1-songliubraving@fb.com> <20210316211837.910506-4-songliubraving@fb.com>
In-Reply-To: <20210316211837.910506-4-songliubraving@fb.com>
From:   Namhyung Kim <namhyung@kernel.org>
Date:   Thu, 18 Mar 2021 15:07:49 +0900
Message-ID: <CAM9d7ciqTGRx6kZ3c-+tXi3AwvOj5EG1ig4oTiUy9sDuTA=KEw@mail.gmail.com>
Subject: Re: [PATCH v2 3/3] perf-test: add a test for perf-stat --bpf-counters option
To:     Song Liu <songliubraving@fb.com>
Cc:     linux-kernel <linux-kernel@vger.kernel.org>,
        Kernel Team <kernel-team@fb.com>,
        Arnaldo Carvalho de Melo <acme@kernel.org>,
        Arnaldo Carvalho de Melo <acme@redhat.com>,
        Jiri Olsa <jolsa@kernel.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Wed, Mar 17, 2021 at 6:18 AM Song Liu <songliubraving@fb.com> wrote:
>
> Add a test to compare the output of perf-stat with and without option
> --bpf-counters. If the difference is more than 10%, the test is considered
> as failed.
>
> For stable results between two runs (w/ and w/o --bpf-counters), the test
> program should: 1) be long enough for better signal-noise-ratio; 2) not
> depend on the behavior of IO subsystem (for less noise from caching). So
> far, the best option we found is stressapptest.
>
> Signed-off-by: Song Liu <songliubraving@fb.com>
> ---
>  tools/perf/tests/shell/stat_bpf_counters.sh | 34 +++++++++++++++++++++
>  1 file changed, 34 insertions(+)
>  create mode 100755 tools/perf/tests/shell/stat_bpf_counters.sh
>
> diff --git a/tools/perf/tests/shell/stat_bpf_counters.sh b/tools/perf/tests/shell/stat_bpf_counters.sh
> new file mode 100755
> index 0000000000000..c0bcb38d6b53c
> --- /dev/null
> +++ b/tools/perf/tests/shell/stat_bpf_counters.sh
> @@ -0,0 +1,34 @@
> +#!/bin/sh
> +# perf stat --bpf-counters test
> +# SPDX-License-Identifier: GPL-2.0
> +
> +set -e
> +
> +# check whether $2 is within +/- 10% of $1
> +compare_number()
> +{
> +       first_num=$1
> +       second_num=$2
> +
> +       # upper bound is first_num * 110%
> +       upper=$(( $first_num + $first_num / 10 ))
> +       # lower bound is first_num * 90%
> +       lower=$(( $first_num - $first_num / 10 ))
> +
> +       if [ $second_num -gt $upper ] || [ $second_num -lt $lower ]; then
> +               echo "The difference between $first_num and $second_num are greater than 10%."
> +               exit 1
> +       fi
> +}
> +
> +# skip if --bpf-counters is not supported
> +perf stat --bpf-counters true > /dev/null 2>&1 || exit 2
> +
> +# skip if stressapptest is not available
> +stressapptest -s 1 -M 100 -m 1 > /dev/null 2>&1 || exit 2

I don't know how popular it is, but we can print some info
in case we miss it.

> +
> +base_cycles=$(perf stat --no-big-num -e cycles -- stressapptest -s 3 -M 100 -m 1 2>&1 | grep -e cycles | awk '{print $1}')
> +bpf_cycles=$(perf stat --no-big-num --bpf-counters -e cycles -- stressapptest -s 3 -M 100 -m 1 2>&1 | grep -e cycles | awk '{print $1}')

I think just awk '/cycles/ {print $1}' should work.

Thanks,
Namhyung


> +
> +compare_number $base_cycles $bpf_cycles
> +exit 0
> --
> 2.30.2
>