Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
From:   Naresh Kamboju <naresh.kamboju@linaro.org>
Date:   Fri, 25 Nov 2022 16:44:02 +0530
Message-ID: <CA+G9fYt9W2h31bSquJ7Jj7jsP=4OEhF9WMB=NwdR2WcavvPusA@mail.gmail.com>
Subject: perf: list of failures from LKFT CI
To:     linux-perf-users@vger.kernel.org,
        open list <linux-kernel@vger.kernel.org>
Cc:     Arnaldo Carvalho de Melo <acme@kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        =?UTF-8?B?RGFuaWVsIETDrWF6?= <daniel.diaz@linaro.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

Perf stat JSON output linter test and few more failed on next-20221125.

Reported-by: Linux Kernel Functional Testing <lkft@linaro.org>

Regressions found on qemu_arm64:
    perf/perf-stat-JSON-output-linter
    perf/perf-stat-CSV-output-linter
    perf/perf-all-PMU-test
    https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20221125/testrun/13178580/suite/perf/tests/

Regressions found on qemu_x86_64:
    perf/perf-stat-JSON-output-linter
    perf/Watchpoint-subtest-2
    perf/perf-all-metricgroups-test
    https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20221125/testrun/13178612/suite/perf/tests/

Regressions found on x86:
    perf/perf-all-metricgroups-test
    perf/perf-stat-JSON-output-linter
    https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20221125/testrun/13181129/suite/perf/tests/

Details of perf test failures.
Good case: Expected output,
Linux next-20221124

 92: perf stat JSON output linter                                    :
--- start ---
test child forked, pid 3067
Checking json output: no args [Success]
Checking json output: system wide [Success]
Checking json output: interval [Success]
Checking json output: event [Success]
Checking json output: per thread [Success]
Checking json output: per node [Success]
Checking json output: system wide Checking json output: system wide no
aggregation [Success]
Checking json output: per core [Success]
Checking json output: per die [Success]
Checking json output: per socket [Success]
test child finished with 0
---- end ----
perf stat JSON output linter: Ok

https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20221124/testrun/13170208/suite/perf/test/perf-stat-JSON-output-linter/log

Bad case: failed output.
 92: perf stat JSON output linter                                    :
--- start ---
test child forked, pid 2268
Checking json output: no args [   65.101948] audit: type=1701
audit(1669350899.408:21): auid=4294967295 uid=0 gid=0 ses=4294967295
pid=2277 comm=\"perf\" exe=\"/usr/bin/perf\" sig=6 res=1
Test failed for input:
{\"counter-value\" : \"0.531160\", \"unit\" : \"msec\", \"event\" :
\"task-clock\", \"event-runtime\" : 531160, \"pcnt-running\" : 100.00,
\"metric-value\" : 0.425336, \"metric-unit\" : \"CPUs utilized\"}

{\"counter-value\" : \"0.000000\", \"unit\" : \"\", \"event\" :
\"context-switches\", \"event-runtime\" : 531160, \"pcnt-running\" :
100.00, \"metric-value\" : 0.000000, \"metric-unit\" : \"/sec\"}

{\"counter-value\" : \"0.000000\", \"unit\" : \"\", \"event\" :
\"cpu-migrations\", \"event-runtime\" : 531160, \"pcnt-running\" :
100.00, \"metric-value\" : 0.000000, \"metric-unit\" : \"/sec\"}

{\"counter-value\" : \"43.000000\", \"unit\" : \"\", \"event\" :
\"page-faults\", \"event-runtime\" : 531160, \"pcnt-running\" :
100.00, \"metric-value\" : 80.954891, \"metric-unit\" : \"K/sec\"}

{\"counter-value\" : \"1264746.000000\", \"unit\" : \"\", \"event\" :
\"cycles\", \"event-runtime\" : 531160, \"pcnt-running\" : 100.00,
\"metric-value\" : 2.381102, \"metric-unit\" : \"GHz\"}

{\"counter-value\" : \"1711188.000000\", \"unit\" : \"\", \"event\" :
\"instructions\", \"event-runtime\" : 531160, \"pcnt-running\" :
100.00, \"metric-value\" : 1.352989, \"metric-unit\" : \"insn per
cycle\"}

{\"counter-value\" : \"<not supported>\", \"unit\" : \"\", \"event\" :
\"branches\", \"event-runtime\" : 0, \"pcnt-running\" : 100.00, Fatal
error: glibc detected an invalid stdio handle

Traceback (most recent call last):
  File \"/usr/libexec/perf-core/tests/shell/lib/perf_json_output_lint.py\",
line 93, in <module>
    check_json_output(expected_items)
  File \"/usr/libexec/perf-core/tests/shell/lib/perf_json_output_lint.py\",
line 54, in check_json_output
    raise RuntimeError(f'wrong number of fields. counted {count}
expected {expected_items}'
RuntimeError: wrong number of fields. counted 5 expected 6 in
'{\"counter-value\" : \"<not supported>\", \"unit\" : \"\", \"event\"
: \"branches\", \"event-runtime\" : 0, \"pcnt-running\" : 100.00,
Fatal error: glibc detected an invalid stdio handle
'
test child finished with -1
---- end ----
perf stat JSON output linter: FAILED!

https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20221125/testrun/13178580/suite/perf/test/perf-stat-JSON-output-linter/log

Test results comparison history:
https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20221125/testrun/13178580/suite/perf/test/perf-stat-JSON-output-linter/history/

--
Linaro LKFT
https://lkft.linaro.org