Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20200501173333.227162-1-irogers@google.com> <20200507084445.GA2447905@krava>
 <20200511153207.GD28888@kernel.org> <CAP-5=fUEz6iBYfiwHgyGCuVNuPGAQN+Fgbcd4ZSn5cV_VAsFvw@mail.gmail.com>
In-Reply-To: <CAP-5=fUEz6iBYfiwHgyGCuVNuPGAQN+Fgbcd4ZSn5cV_VAsFvw@mail.gmail.com>
From:   Ian Rogers <irogers@google.com>
Date:   Tue, 12 May 2020 23:32:15 -0700
Message-ID: <CAP-5=fUj_htQNqqMkhqM6Az0iEh1ekU5AbLRPh_x4jRoagqzZA@mail.gmail.com>
Subject: Re: [PATCH v4 00/12] perf metric fixes and test
To:     Arnaldo Carvalho de Melo <arnaldo.melo@gmail.com>
Cc:     Jiri Olsa <jolsa@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Ingo Molnar <mingo@redhat.com>,
        Mark Rutland <mark.rutland@arm.com>,
        Alexander Shishkin <alexander.shishkin@linux.intel.com>,
        Namhyung Kim <namhyung@kernel.org>,
        Kan Liang <kan.liang@linux.intel.com>,
        Andi Kleen <ak@linux.intel.com>,
        Haiyan Song <haiyanx.song@intel.com>,
        Jin Yao <yao.jin@linux.intel.com>,
        Song Liu <songliubraving@fb.com>,
        Ravi Bangoria <ravi.bangoria@linux.ibm.com>,
        John Garry <john.garry@huawei.com>,
        Leo Yan <leo.yan@linaro.org>,
        Adrian Hunter <adrian.hunter@intel.com>,
        Paul Clarke <pc@us.ibm.com>,
        LKML <linux-kernel@vger.kernel.org>,
        kajoljain <kjain@linux.ibm.com>,
        linux-perf-users <linux-perf-users@vger.kernel.org>,
        Stephane Eranian <eranian@google.com>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Mon, May 11, 2020 at 9:12 AM Ian Rogers <irogers@google.com> wrote:
>
> On Mon, May 11, 2020 at 8:32 AM Arnaldo Carvalho de Melo
> <arnaldo.melo@gmail.com> wrote:
> >
> > Em Thu, May 07, 2020 at 10:44:45AM +0200, Jiri Olsa escreveu:
> > > On Fri, May 01, 2020 at 10:33:21AM -0700, Ian Rogers wrote:
> > > > Add a test that all perf metrics (for your architecture) are parsable
> > > > with the simple expression parser. Attempt to parse all events in
> > > > metrics but only fail if the metric is for the current CPU. Fix bugs
> > > > in the expr parser, x86 and powerpc metrics. Improve debug messages
> > > > around add PMU config term failures.
> > > >
> > > > v4 rebases after Kajol Jain's patches and fixes an asprintf warning.
> > > > v3 adds parse event testing of ids and improves debug messages for add
> > > >   PMU. These messages are paticular visible with 'perf test 10
> > > >   -vvv'. It moves the testing logic from tests/expr.c to
> > > >   tests/pmu-events.c as suggested by John Garry
> > > >   <john.garry@huawei.com>.
> > > > v2 adds Fixes tags to commit messages for when broken metrics were
> > > >   first added. Adds a debug warning for division by zero in expr, and
> > > >   adds a workaround for id values in the expr test necessary for
> > > >   powerpc. It also fixes broken power8 and power9 metrics.
> > > >
> > > > Ian Rogers (12):
> > > >   perf expr: unlimited escaped characters in a symbol
> > > >   perf metrics: fix parse errors in cascade lake metrics
> > > >   perf metrics: fix parse errors in skylake metrics
> > > >   perf expr: allow ',' to be an other token
> > > >   perf expr: increase max other
> > > >   perf expr: parse numbers as doubles
> > > >   perf expr: debug lex if debugging yacc
> > > >   perf metrics: fix parse errors in power8 metrics
> > > >   perf metrics: fix parse errors in power9 metrics
> > > >   perf expr: print a debug message for division by zero
> > >
> > > heya,
> > > could we please get the 1st 10 patches in? they are important,
> > > and let's not block them with new versions for patches 11/12
> > >
> > > Acked-by: Jiri Olsa <jolsa@redhat.com>
> >
> > The first ten patches are in, can we go with what Ian suggested for the
> > last two patches?

To follow up, I've resent the last two patches here:
https://lore.kernel.org/lkml/20200513001729.21155-1-irogers@google.com/
https://lore.kernel.org/lkml/20200513062752.3681-1-irogers@google.com/T/#t
The event parsing errors don't cause the test to fail in the latter.

Thanks,
Ian

> It seems sad not to make this test fail for broken event encodings, as
> perf test does a good job of swallowing the test's debug output
> (fork...). I was thinking that I'll split out the test and various
> fixes in:
> https://lore.kernel.org/lkml/20200508053629.210324-1-irogers@google.com/
> The test will only do something if there is a MetricExpr for an
> architecture. There are currently only MetricExpr for x86 and powerpc.
> I've tested power 9 with the test, which tests the expressions for
> power 8 but not the events, and everything passes. The area the test
> can fail is x86 and I should be able to get reasonable coverage there
> by grabbing cloud machines. When an x86 test fails it is hard to know
> what to do. In Jiri's Skylake case naively 'fixing' the expression to
> remove 'thresh=1' yields an expression that will only give the value
> 1. Perhaps a better 'fix' is just to remove the expressions in those
> cases and wait for someone like Intel to refresh them? If that
> expression will only work on a newer kernel then we need to identify a
> constraint, etc. However, we couldn't see thresh as an event parameter
> in a recent tree. Removing the expression seems like the most
> expedient fix to get the test in, so that new expressions don't break
> things :-)
>
> Thanks Arnaldo for following up on this!
>
> Ian
>
> > - Arnaldo
> >
> > > thanks,
> > > jirka
> > >
> > > >   perf parse-events: expand add PMU error/verbose messages
> > > >   perf test: improve pmu event metric testing
> > > >
> > > >  tools/perf/arch/x86/util/intel-pt.c           |  32 ++--
> > > >  .../arch/powerpc/power8/metrics.json          |   2 +-
> > > >  .../arch/powerpc/power9/metrics.json          |   2 +-
> > > >  .../arch/x86/cascadelakex/clx-metrics.json    |  10 +-
> > > >  .../arch/x86/skylakex/skx-metrics.json        |   4 +-
> > > >  tools/perf/tests/builtin-test.c               |   5 +
> > > >  tools/perf/tests/expr.c                       |   1 +
> > > >  tools/perf/tests/pmu-events.c                 | 156 +++++++++++++++++-
> > > >  tools/perf/tests/pmu.c                        |   4 +-
> > > >  tools/perf/tests/tests.h                      |   2 +
> > > >  tools/perf/util/expr.c                        |   1 +
> > > >  tools/perf/util/expr.h                        |   2 +-
> > > >  tools/perf/util/expr.l                        |  16 +-
> > > >  tools/perf/util/expr.y                        |  16 +-
> > > >  tools/perf/util/parse-events.c                |  29 +++-
> > > >  tools/perf/util/pmu.c                         |  33 ++--
> > > >  tools/perf/util/pmu.h                         |   2 +-
> > > >  17 files changed, 262 insertions(+), 55 deletions(-)
> > > >
> > > > --
> > > > 2.26.2.526.g744177e7f7-goog
> > > >
> > >
> >
> > --
> >
> > - Arnaldo