Received-SPF: pass (google.com: domain of linux-kernel+bounces-151632-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Message-ID: <b48c2040-7ee9-40a4-a97e-54bcb82cfc3e@linux.intel.com>
Date: Fri, 19 Apr 2024 11:09:34 -0400
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v1] perf test: Avoid hard coded metrics in stat std output
 test
To: Ian Rogers <irogers@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>,
 Arnaldo Carvalho de Melo <acme@kernel.org>,
 Namhyung Kim <namhyung@kernel.org>, Mark Rutland <mark.rutland@arm.com>,
 Alexander Shishkin <alexander.shishkin@linux.intel.com>,
 Jiri Olsa <jolsa@kernel.org>, Adrian Hunter <adrian.hunter@intel.com>,
 Yicong Yang <yangyicong@hisilicon.com>,
 Athira Rajeev <atrajeev@linux.vnet.ibm.com>,
 linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org
References: <20240417183219.1208493-1-irogers@google.com>
 <c68bf005-af20-4995-86d5-a62e171763e3@linux.intel.com>
 <CAP-5=fUG_b6e9Q=qMFMtuKpzvz7D1xoLXNxeyfqwyX8qSsBwsg@mail.gmail.com>
Content-Language: en-US
From: "Liang, Kan" <kan.liang@linux.intel.com>
In-Reply-To: <CAP-5=fUG_b6e9Q=qMFMtuKpzvz7D1xoLXNxeyfqwyX8qSsBwsg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit


On 2024-04-19 10:40 a.m., Ian Rogers wrote:
> On Fri, Apr 19, 2024 at 6:54 AM Liang, Kan <kan.liang@linux.intel.com> wrote:
>>
>>
>>
>> On 2024-04-17 2:32 p.m., Ian Rogers wrote:
>>> Hard coded metric names fail on ARM testing.
>>>
>>> Signed-off-by: Ian Rogers <irogers@google.com>
>>> ---
>>>  tools/perf/tests/shell/stat+std_output.sh | 2 +-
>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/tools/perf/tests/shell/stat+std_output.sh b/tools/perf/tests/shell/stat+std_output.sh
>>> index cbf2894b2c84..845f83213855 100755
>>> --- a/tools/perf/tests/shell/stat+std_output.sh
>>> +++ b/tools/perf/tests/shell/stat+std_output.sh
>>> @@ -13,7 +13,7 @@ stat_output=$(mktemp /tmp/__perf_test.stat_output.std.XXXXX)
>>>
>>>  event_name=(cpu-clock task-clock context-switches cpu-migrations page-faults stalled-cycles-frontend stalled-cycles-backend cycles instructions branches branch-misses)
>>>  event_metric=("CPUs utilized" "CPUs utilized" "/sec" "/sec" "/sec" "frontend cycles idle" "backend cycles idle" "GHz" "insn per cycle" "/sec" "of all branches")
>>> -skip_metric=("stalled cycles per insn" "tma_" "retiring" "frontend_bound" "bad_speculation" "backend_bound")
>>> +skip_metric=($(perf list --raw Default 2> /dev/null))
>>
>>
>> The "perf list --raw Default" only gives the topdown metrics.
>> The "stalled cycles per insn" is not covered.
>> The check should skip the line of "stalled cycles per insn" as well.
>>
>>      3,856,436,920 stalled-cycles-frontend   #   74.09% frontend cycles idle
>>      1,600,790,871 stalled-cycles-backend    #   30.75% backend  cycles idle
>>      2,603,501,247 instructions              #    0.50  insns per cycle
>>                                              #    1.48  stalled cycles
>> per insn
>>        484,357,498 branches                  #  283.455 M/sec
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/perf/builtin-stat.c#n24
>>
>> The newer Intel CPU doesn't have the stalled-cycles-* events. But it
>> seems power and older x86 CPU have the events.
> 
> Oh, sigh. This test should really ignore lines like that. How much do
> we care about these metrics? The RISC-V event parsing change:
> https://lore.kernel.org/lkml/20240416061533.921723-1-irogers@google.com/
> means that legacy hardware events will be uncommon and we need to
> adapt the hard coded metrics in stat-shadow.c to json ones. Once they
> are json metrics they will be in Default.

It seems except the newer Intel CPU, all the other ARCHs support the two
stalled-cycles-* events and the metric. For Intel, there are Topdown
metrics. But it seems an important metrics for the other ARCHs.

RISC-V
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/perf/riscv_pmu_sbi.c#n134
Power
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/powerpc/perf/power9-pmu.c#n279
Arm
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/perf/arm_pmuv3.c#n53

So almost all json files have to be updated. I'm not sure if it's a
practical way to fix the issue.

Thanks,
Kan
> 
> Thanks,
> Ian
> 
>> Thanks,
>> Kan
>>
>>>
>>>  cleanup() {
>>>    rm -f "${stat_output}"
>