Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Tue, 2 Jun 2020 21:03:08 +0200
From:   Jiri Olsa <jolsa@redhat.com>
To:     Ian Rogers <irogers@google.com>
Cc:     Jiri Olsa <jolsa@kernel.org>,
        Arnaldo Carvalho de Melo <acme@kernel.org>,
        lkml <linux-kernel@vger.kernel.org>,
        Ingo Molnar <mingo@kernel.org>,
        Namhyung Kim <namhyung@kernel.org>,
        Alexander Shishkin <alexander.shishkin@linux.intel.com>,
        Peter Zijlstra <a.p.zijlstra@chello.nl>,
        Michael Petlan <mpetlan@redhat.com>,
        Stephane Eranian <eranian@google.com>,
        Andi Kleen <ak@linux.intel.com>
Subject: Re: [PATCH 04/13] perf tests: Add another metric parsing test
Message-ID: <20200602190308.GF1169993@krava>
References: <20200602115055.1168446-1-jolsa@kernel.org>
 <20200602115055.1168446-5-jolsa@kernel.org>
 <CAP-5=fXjQ0ZgtrsizH6F7y2VRZ6dnsu54BXRxszyM+mMAvP-cQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAP-5=fXjQ0ZgtrsizH6F7y2VRZ6dnsu54BXRxszyM+mMAvP-cQ@mail.gmail.com>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Tue, Jun 02, 2020 at 10:58:32AM -0700, Ian Rogers wrote:
> On Tue, Jun 2, 2020 at 4:51 AM Jiri Olsa <jolsa@kernel.org> wrote:
> >
> > The test goes through all metrics compiled for arch
> > within pmu events and try to parse them.
> >
> > This test is different from 'test_parsing' in that
> > we go through all the events in the current arch,
> > not just one defined for current CPU model. Using
> > 'fake_pmu' to parse events which do not have PMUs
> > defined in the system.
> >
> > Say there's bad change in ivybridge metrics file, like:
> >
> >   --- a/tools/perf/pmu-events/arch/x86/ivybridge/ivb-metrics.json
> >   +++ b/tools/perf/pmu-events/arch/x86/ivybridge/ivb-metrics.json
> >   @@ -8,7 +8,7 @@
> >   -        "MetricExpr": "IDQ_UOPS_NOT_DELIVERED.CORE / (4 * ((
> >   +        "MetricExpr": "IDQ_UOPS_NOT_DELIVERED.CORE / / (4 *
> >
> > the test fails with (on my kabylake laptop):
> >
> >   $ perf test 'Parsing of PMU event table metrics with fake PMUs' -v
> >   parsing 'idq_uops_not_delivered.core / / (4 * (( ( cpu_clk_unh...
> >   syntax error, line 1
> >   expr__parse failed
> >   test child finished with -1
> >   ...
> 
> For this example as the problem is the expression, presumably this was
> "passing" with test_parsing due to returning TEST_SKIP? I did this
> initially so that we could get the test merged and then the metrics
> fixed. I'd prefer if test_parsing were returning TEST_FAIL.

it will fail:

	[jolsa@krava perf]$ git diff
	diff --git a/tools/perf/pmu-events/arch/x86/ivybridge/ivb-metrics.json b/tools/perf/pmu-events/arch/x86/ivybridge/ivb-metrics.json
	index 28e25447d3ef..0cad6b709f96 100644
	--- a/tools/perf/pmu-events/arch/x86/ivybridge/ivb-metrics.json
	+++ b/tools/perf/pmu-events/arch/x86/ivybridge/ivb-metrics.json
	@@ -1,7 +1,7 @@
	 [
	     {
		 "BriefDescription": "This category represents fraction of slots where the processor's Frontend undersupplies its Backend",
	-        "MetricExpr": "IDQ_UOPS_NOT_DELIVERED.CORE / (4 * cycles)",
	+        "MetricExpr": "IDQ_UOPS_NOT_DELIVERED.CORE / / (4 * cycles)",
		 "MetricGroup": "TopdownL1",
		 "MetricName": "Frontend_Bound",

	[jolsa@krava perf]$ ./perf test  'Parsing of PMU event table metrics with fake PMUs' 
	10: PMU events                                            :
	10.4: Parsing of PMU event table metrics with fake PMUs   : FAILED!

because the metric is malformed

jirka