Received-SPF: pass (google.com: domain of linux-kernel+bounces-151595-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Precedence: bulk
MIME-Version: 1.0
References: <20240417183219.1208493-1-irogers@google.com> <c68bf005-af20-4995-86d5-a62e171763e3@linux.intel.com>
In-Reply-To: <c68bf005-af20-4995-86d5-a62e171763e3@linux.intel.com>
From: Ian Rogers <irogers@google.com>
Date: Fri, 19 Apr 2024 07:40:54 -0700
Message-ID: <CAP-5=fUG_b6e9Q=qMFMtuKpzvz7D1xoLXNxeyfqwyX8qSsBwsg@mail.gmail.com>
Subject: Re: [PATCH v1] perf test: Avoid hard coded metrics in stat std output test
To: "Liang, Kan" <kan.liang@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>, 
	Arnaldo Carvalho de Melo <acme@kernel.org>, Namhyung Kim <namhyung@kernel.org>, 
	Mark Rutland <mark.rutland@arm.com>, 
	Alexander Shishkin <alexander.shishkin@linux.intel.com>, Jiri Olsa <jolsa@kernel.org>, 
	Adrian Hunter <adrian.hunter@intel.com>, Yicong Yang <yangyicong@hisilicon.com>, 
	Athira Rajeev <atrajeev@linux.vnet.ibm.com>, linux-perf-users@vger.kernel.org, 
	linux-kernel@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, Apr 19, 2024 at 6:54=E2=80=AFAM Liang, Kan <kan.liang@linux.intel.c=
om> wrote:
>
>
>
> On 2024-04-17 2:32 p.m., Ian Rogers wrote:
> > Hard coded metric names fail on ARM testing.
> >
> > Signed-off-by: Ian Rogers <irogers@google.com>
> > ---
> >  tools/perf/tests/shell/stat+std_output.sh | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/tools/perf/tests/shell/stat+std_output.sh b/tools/perf/tes=
ts/shell/stat+std_output.sh
> > index cbf2894b2c84..845f83213855 100755
> > --- a/tools/perf/tests/shell/stat+std_output.sh
> > +++ b/tools/perf/tests/shell/stat+std_output.sh
> > @@ -13,7 +13,7 @@ stat_output=3D$(mktemp /tmp/__perf_test.stat_output.s=
td.XXXXX)
> >
> >  event_name=3D(cpu-clock task-clock context-switches cpu-migrations pag=
e-faults stalled-cycles-frontend stalled-cycles-backend cycles instructions=
 branches branch-misses)
> >  event_metric=3D("CPUs utilized" "CPUs utilized" "/sec" "/sec" "/sec" "=
frontend cycles idle" "backend cycles idle" "GHz" "insn per cycle" "/sec" "=
of all branches")
> > -skip_metric=3D("stalled cycles per insn" "tma_" "retiring" "frontend_b=
ound" "bad_speculation" "backend_bound")
> > +skip_metric=3D($(perf list --raw Default 2> /dev/null))
>
>
> The "perf list --raw Default" only gives the topdown metrics.
> The "stalled cycles per insn" is not covered.
> The check should skip the line of "stalled cycles per insn" as well.
>
>      3,856,436,920 stalled-cycles-frontend   #   74.09% frontend cycles i=
dle
>      1,600,790,871 stalled-cycles-backend    #   30.75% backend  cycles i=
dle
>      2,603,501,247 instructions              #    0.50  insns per cycle
>                                              #    1.48  stalled cycles
> per insn
>        484,357,498 branches                  #  283.455 M/sec
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/t=
ools/perf/builtin-stat.c#n24
>
> The newer Intel CPU doesn't have the stalled-cycles-* events. But it
> seems power and older x86 CPU have the events.

Oh, sigh. This test should really ignore lines like that. How much do
we care about these metrics? The RISC-V event parsing change:
https://lore.kernel.org/lkml/20240416061533.921723-1-irogers@google.com/
means that legacy hardware events will be uncommon and we need to
adapt the hard coded metrics in stat-shadow.c to json ones. Once they
are json metrics they will be in Default.

Thanks,
Ian

> Thanks,
> Kan
>
> >
> >  cleanup() {
> >    rm -f "${stat_output}"