Received: by 2002:a05:6358:9144:b0:117:f937:c515 with SMTP id r4csp3033871rwr; Fri, 28 Apr 2023 22:37:26 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7iUUMaWxq7l0waBjaMdcVUJ3OS2JSy49ol8Zl6er3FmUxtNApfE1swfRB3CQQacO3bSr3M X-Received: by 2002:a17:902:ce91:b0:1a9:8769:3697 with SMTP id f17-20020a170902ce9100b001a987693697mr9016118plg.4.1682746646234; Fri, 28 Apr 2023 22:37:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682746646; cv=none; d=google.com; s=arc-20160816; b=yxW7JngqN0009ozIUUnZ0bH5MGA6rC297GmFE0WPIQU7nE1ZzkQAPu6DOWgWv2CYtL Ac+8QeI91ODf4AUErR2QEhgFJMF0FgXB6QhFDh2IOTKrhuqD+H/S43ogxLF86GNtUBq1 hurQpbuFgEbYghQT7eXt+cyrG9PcEdOT9QKt2wz7DTdrdibcTT6y9eGq9bUpOsAz42z4 6V/XogpNcdp5gsAxpajB4ul2XXShNY2W4c2CAK1HIsXEP7tcT/T2l9XpJjhV97QtJsZU ugKtAGLwV7egJq67Yfk6lPlqWB1wJ0vxEPyVUMkC7xDzliWSV4HbFeHfAZnhDJKK/WVS T1Zg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:mime-version:message-id:date :dkim-signature; bh=pSO6TaQceat5s7+w8tdwXDUT+UMck7zKZJz2rmLR/is=; b=jMq38dfhmVfIez4mwXy9vWee7L2k//95MH3K/oJuv9dzANCCw179rCPWv4ntuE8KG3 1Ryc4+NN4royM9fqCoY2VWOOwFFt6hgxIobUXJoYCyluYQkZwK/F+XAoVEPtzxNZ3peA Nh9Zw4l1dMNk/BhvS26JQkZQRcIPDmGvFN1vLtm0yP2CSqgX6Kx56VRTwIEWUPAbD0cJ oybV2l1s8lnRJ5b5S7Af78yYn2trMtkMx/foGixoazAblmZfuBsW76EumXosZzacLRqB qz11pK/LfFPx8RtLGqaBvBAyy3LDsKU0E+lBwkxIRkmWflqeaxHkBdWo8JOMFXn/IkUo Pkrg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=mH7Z6fOb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q17-20020a635c11000000b0051909d663desi22265492pgb.481.2023.04.28.22.37.12; Fri, 28 Apr 2023 22:37:26 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=mH7Z6fOb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229437AbjD2FfV (ORCPT + 99 others); Sat, 29 Apr 2023 01:35:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36502 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229497AbjD2FfU (ORCPT ); Sat, 29 Apr 2023 01:35:20 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8C6E9271E for ; Fri, 28 Apr 2023 22:35:18 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-b922aa3725fso1018003276.0 for ; Fri, 28 Apr 2023 22:35:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1682746517; x=1685338517; h=cc:to:from:subject:mime-version:message-id:date:from:to:cc:subject :date:message-id:reply-to; bh=pSO6TaQceat5s7+w8tdwXDUT+UMck7zKZJz2rmLR/is=; b=mH7Z6fObbNpehAF50RREY5bm5FazivHNYBujzFqv6IwF9F6EGuSydLvI7BK0lkS4bC TWRyUgmtoxhEK8N6DmEFTZwtCzRJ2HIOIIG+YeP6Lv2/35NC5MI7T0L6F+BlrWinhHsL VfRnl9GZP1RCX3LJ7V4B1pveiTaw/jemNZAe4jJdGnr3M00cRxHUFSgnkX7TIqDQU5w5 2UeUNVD2iZA8P8papiWK/+Nppm7k/2lIt9Q0MwnnIdnb3qtws6/RJaHDc+qmRO6xkgq5 9Ic8ukBVvARqVM/7jrfK0ewtOkwVyffZS8goJhg2q33qGf0vAtnmw5SOXpqDYjFk/ILZ xkQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682746517; x=1685338517; h=cc:to:from:subject:mime-version:message-id:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=pSO6TaQceat5s7+w8tdwXDUT+UMck7zKZJz2rmLR/is=; b=KhFLEOoQ0dsIIqWBub86ypyHyLpVpaknuKXtnwVHpcr4hDezPADIUrGKGHqsyRGdau 5zJriO3svDsY/A/png3nVLVCgu7ymKeggXWRvOJaUY3mxY0IaeFpk9mzgUdQ5lSkup5K IZwa1wC+1BxWP3ZVTgW6/tWnlFosF4ZtkmBWWpycSx5y6X8iZ6foxKZ0Bx1fpZsVCLrN HjQM8F/wr5jJepZN82L8JUdNaneY08OyyglgvQaIm2uTpce3r3J7k+p+ualg5mWKweAp HQfsFhOCzn0BJVey+xHDu1gJoRKgpv6VHUes6jDC0CMT+Vyjx4Y+bgPrrHMAuzLswnOp HEGQ== X-Gm-Message-State: AC+VfDwNzMlHX6twE6kIsZUuoHuD0in1Pg9urBEmmrocZvyJEGlsUhVZ 23k/aFAISiHZqgURJA7wtIEyfRHL3nzt X-Received: from irogers.svl.corp.google.com ([2620:15c:2d4:203:c563:7e28:fb7c:bce3]) (user=irogers job=sendgmr) by 2002:a25:6b05:0:b0:b8b:f5fb:5986 with SMTP id g5-20020a256b05000000b00b8bf5fb5986mr4397240ybc.10.1682746517408; Fri, 28 Apr 2023 22:35:17 -0700 (PDT) Date: Fri, 28 Apr 2023 22:34:20 -0700 Message-Id: <20230429053506.1962559-1-irogers@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.40.1.495.gc816e09b53d-goog Subject: [PATCH v3 00/46] Fix perf on Intel hybrid CPUs From: Ian Rogers To: Arnaldo Carvalho de Melo , Kan Liang , Ahmad Yasin , Peter Zijlstra , Ingo Molnar , Stephane Eranian , Andi Kleen , Perry Taylor , Samantha Alt , Caleb Biggers , Weilin Wang , Edward Baker , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Adrian Hunter , Florian Fischer , Rob Herring , Zhengjun Xing , John Garry , Kajol Jain , Sumanth Korikkar , Thomas Richter , Tiezhu Yang , Ravi Bangoria , Leo Yan , Yang Jihong , James Clark , Suzuki Poulouse , Kang Minchul , Athira Rajeev , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ian Rogers Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org TL;DR: hybrid doesn't crash, json metrics work on hybrid on both PMUs or individually, event parsing doesn't always scan all PMUs, more and new tests that also run without hybrid, less code. The first 3 patches are proposed to address Linux 6.4 issues with the remainder to be the basis for 6.5. The first patch is a 1 liner, plus comment, workaround for Intel hybrid platforms crashing perf stat when running with no events/metrics. The second patch works around an issue with platforms like Skylake where certain events fail to open at higher paranoia levels even if kernel and hypervisor profiling is disabled. Events are marked as skippable and not placed in the output, while remaining events are and metrics are computed. The third patch makes it so that TopdownL1 and TopdownL2 json metric groups don't group events on Intel, this is done via a new json flag. Adding the json flag in each of the json files is two-thirds of the 300 lines of these patches. Patch 42 removes the disabling of TopdownL1 on hybrid as the underlying problems are fixed. The json file updates are created with the usual script updated in: https://github.com/intel/perfmon/pull/73 The next 5 patches avoid grouping events for metrics where is could never succeed and were previously posted as: "perf vendor events intel: Add xxx metric constraints" https://lore.kernel.org/all/20230419005423.343862-1-irogers@google.com/ Next are some general and test improvements. Next event parsing is rewritten to not scan all PMUs for the benefit of raw and legacy cache parsing, instead these are handled by the lexer and a new term type. This ultimately removes the need for the event parser for hybrid to be recursive as legacy cache can be just a term. Tests are re-enabled for events with hyphens, so AMD's branch-brs event is now parsable. The cputype option is made a generic pmu filter flag and is tested even on non-hybrid systems. The final patches address specific json metric issues on hybrid, in both the json metrics and the metric code. The patches add slightly more code than they remove, in areas like better json metric constraints and tests, but in the core util code, the removal of hybrid is a net reduction: 20 files changed, 641 insertions(+), 960 deletions(-) Sample output is contained in the v1 patch set: https://lore.kernel.org/lkml/bff481ba-e60a-763f-0aa0-3ee53302c480@linux.intel.com/ Tested on Tigerlake, Skylake and Alderlake CPUs. The v3 patch set: - for Linux 6.4 the first 5 patches are recommended: - changes skippable evsels to always print in case short running benchmarks meet the 0 enable and 0 count condition (suggested by Stephane). - changes metrics to show "nan" so that it is more obvious they couldn't be computed (suggested by Stephane). - fixes a reordering issue that reordered events that lacked a group, especially when the core PMU isn't "cpu". - for Linux 6.5 added extra hybrid type tests in the x86 hybrid test as suggested by Kan. - note, the patches aren't rebased against the tmp.perf-tools branch meaning patches 1 and 11 should be dropped there. The v2 patch set: - restructures the 3 Linux 6.4 patches first; - makes it so that is_event_supported isn't called during core event parsing; - displays skippable 0 count events that were enabled; - addresses output formatting review comments; - addresses some test issues and an uninitialized variable use in strchr; - addresses checkpatch.pl reported issues; - adds reviewed-by to some patches. Ian Rogers (46): perf stat: Disable TopdownL1 on hybrid perf metric: Change divide by zero and !support events behavior perf stat: Introduce skippable evsels perf metric: Json flag to not group events if gathering a metric group perf parse-events: Don't reorder ungrouped events by pmu perf vendor events intel: Add alderlake metric constraints perf vendor events intel: Add icelake metric constraints perf vendor events intel: Add icelakex metric constraints perf vendor events intel: Add sapphirerapids metric constraints perf vendor events intel: Add tigerlake metric constraints perf stat: Avoid segv on counter->name perf test: Test more sysfs events perf test: Use valid for PMU tests perf test: Mask config then test perf test: Test more with config_cache perf test: Roundtrip name, don't assume 1 event per name perf parse-events: Set attr.type to PMU type early perf parse-events: Set pmu_name whenever a pmu is given perf print-events: Avoid unnecessary strlist perf parse-events: Avoid scanning PMUs before parsing perf evsel: Modify group pmu name for software events perf test: Move x86 hybrid tests to arch/x86 perf test x86 hybrid: Update test expectations perf test x86 hybrid: Add hybrid extended type checks perf parse-events: Support PMUs for legacy cache events perf parse-events: Wildcard legacy cache events perf print-events: Print legacy cache events for each PMU perf parse-events: Support wildcards on raw events perf parse-events: Remove now unused hybrid logic perf parse-events: Minor type safety cleanup perf parse-events: Add pmu filter perf stat: Make cputype filter generic perf test: Add cputype testing to perf stat perf test: Fix parse-events tests for >1 core PMU perf parse-events: Support hardware events as terms perf parse-events: Avoid error when assigning a term perf parse-events: Avoid error when assigning a legacy cache term perf parse-events: Don't auto merge hybrid wildcard events perf parse-events: Don't reorder atom cpu events perf metrics: Be PMU specific for referenced metrics. perf stat: Command line PMU metric filtering perf vendor events intel: Correct alderlake metrics perf jevents: Don't rewrite metrics across PMUs perf metrics: Be PMU specific in event match perf stat: Don't disable TopdownL1 metric on hybrid perf parse-events: Reduce scope of is_event_supported tools/perf/arch/x86/include/arch-tests.h | 1 + tools/perf/arch/x86/tests/Build | 1 + tools/perf/arch/x86/tests/arch-tests.c | 10 + tools/perf/arch/x86/tests/hybrid.c | 287 ++++++ tools/perf/arch/x86/util/evlist.c | 4 +- tools/perf/builtin-list.c | 19 +- tools/perf/builtin-record.c | 13 +- tools/perf/builtin-stat.c | 73 +- tools/perf/builtin-top.c | 5 +- tools/perf/builtin-trace.c | 5 +- .../arch/x86/alderlake/adl-metrics.json | 275 +++--- .../arch/x86/alderlaken/adln-metrics.json | 20 +- .../arch/x86/broadwell/bdw-metrics.json | 12 + .../arch/x86/broadwellde/bdwde-metrics.json | 12 + .../arch/x86/broadwellx/bdx-metrics.json | 12 + .../arch/x86/cascadelakex/clx-metrics.json | 12 + .../arch/x86/haswell/hsw-metrics.json | 12 + .../arch/x86/haswellx/hsx-metrics.json | 12 + .../arch/x86/icelake/icl-metrics.json | 23 + .../arch/x86/icelakex/icx-metrics.json | 23 + .../arch/x86/ivybridge/ivb-metrics.json | 12 + .../arch/x86/ivytown/ivt-metrics.json | 12 + .../arch/x86/jaketown/jkt-metrics.json | 12 + .../arch/x86/sandybridge/snb-metrics.json | 12 + .../arch/x86/sapphirerapids/spr-metrics.json | 23 + .../arch/x86/skylake/skl-metrics.json | 12 + .../arch/x86/skylakex/skx-metrics.json | 12 + .../arch/x86/tigerlake/tgl-metrics.json | 23 + tools/perf/pmu-events/jevents.py | 10 +- tools/perf/pmu-events/metric.py | 28 +- tools/perf/pmu-events/metric_test.py | 6 +- tools/perf/pmu-events/pmu-events.h | 2 + tools/perf/tests/evsel-roundtrip-name.c | 119 +-- tools/perf/tests/expr.c | 3 +- tools/perf/tests/parse-events.c | 836 +++++++++--------- tools/perf/tests/parse-metric.c | 1 + tools/perf/tests/pmu-events.c | 12 +- tools/perf/tests/shell/stat.sh | 44 + tools/perf/util/Build | 1 - tools/perf/util/evlist.h | 1 - tools/perf/util/evsel.c | 30 +- tools/perf/util/evsel.h | 1 + tools/perf/util/expr.y | 6 +- tools/perf/util/metricgroup.c | 111 ++- tools/perf/util/metricgroup.h | 3 +- tools/perf/util/parse-events-hybrid.c | 214 ----- tools/perf/util/parse-events-hybrid.h | 25 - tools/perf/util/parse-events.c | 720 ++++++--------- tools/perf/util/parse-events.h | 63 +- tools/perf/util/parse-events.l | 108 +-- tools/perf/util/parse-events.y | 222 ++--- tools/perf/util/pmu-hybrid.c | 20 - tools/perf/util/pmu-hybrid.h | 1 - tools/perf/util/pmu.c | 16 +- tools/perf/util/pmu.h | 3 + tools/perf/util/pmus.c | 25 +- tools/perf/util/pmus.h | 3 + tools/perf/util/print-events.c | 124 ++- tools/perf/util/stat-display.c | 4 +- tools/perf/util/stat-shadow.c | 25 +- 60 files changed, 2045 insertions(+), 1691 deletions(-) create mode 100644 tools/perf/arch/x86/tests/hybrid.c delete mode 100644 tools/perf/util/parse-events-hybrid.c delete mode 100644 tools/perf/util/parse-events-hybrid.h -- 2.40.1.495.gc816e09b53d-goog