LinuxLists.cc - [RFC PATCH 00/25] Perf stat metric grouping with hardware information

2023-09-25 08:22:34

Subject: [RFC PATCH 00/25] Perf stat metric grouping with hardware information

From: Weilin Wang <[email protected]>

Perf stat metric grouping generates event groups that are provided to kernel for
data collection using the hardware counters. Sometimes, the grouping might fail
and kernel has to retry the groups because generated groups do not fit in the
hardware counters correctly. In some other cases, the groupings are collected
correctly, however, they left some hardware counters unused.

To improve these inefficiencies, we would like to propose a hardware aware
grouping method that does metric/event grouping based on event counter
restriction rules and the availability of hardware counters in the system. This
method is generic as long as all the restriction rules could be provided from
the pmu-event JSON files.

This patch set includes code that does hardware aware grouping and updated
pmu-event JSON files for four platforms (SapphireRapids, Icelakex, Cascadelakex,
and Tigerlake) for your testing and experimenting. We've successfully tested
these patches on three platforms (SapphireRapids, Icelakex, and Cascadelakex)
with topdown metrics from TopdownL1 to TopdownL6.

There are some optimization opportunities that we might implement in the future:
1) Better NMI hanlding: when NMI watchdog is enabled, we reduce the default_core
total counter size by one. This could be improved to better utilize the counter.
2) Fill important events into unused counter for better counter utlization:
there might be some unused counters scattered in the groups. We could consider
to add important events in this slots if necessary. This could help increase the
multiplexing percentage and help improve accuracy if the event is critical.

Remaining questions for dicussion:
3) Where to start grouping from? The current implementation start grouping by
combining all the events into a single list. This step deduplicates events. But
it does not maintain the relationship of events according to the metrics, i.e.
events required by one metric may not be collected at the same time. Another
type of starting point would be grouping each individual metric and then try to
merge the groups.
4) Any comments, suggestions, new ideas?
5) If you are interested to test the patch out and the pmu-event JSON files of
your testing platform is not provided here, please let me know so that I could
provide you the files.

Weilin Wang (25):
perf stat: Add hardware-grouping cmd option to perf stat
perf stat: Add basic functions for the hardware-grouping stat cmd
option
perf pmu-events: Add functions in jevent.py
perf pmu-events: Add counter info into JSON files for SapphireRapids
perf pmu-events: Add event counter data for Cascadelakex
perf pmu-events: Add event counter data for Icelakex
perf stat: Add helper functions for hardware-grouping method
perf stat: Add functions to get counter info
perf stat: Add helper functions for hardware-grouping method
perf stat: Add helper functions to hardware-grouping method
perf stat: Add utility functions to hardware-grouping method
perf stat: Add more functions for hardware-grouping method
perf stat: Add functions to hardware-grouping method
perf stat: Add build string function and topdown events handling in
hardware-grouping
perf stat: Add function to combine metrics for hardware-grouping
perf stat: Update keyword core to default_core to adjust to the
changes for events with no unit
perf stat: Handle taken alone in hardware-grouping
perf stat: Handle NMI in hardware-grouping
perf stat: Handle grouping method fall back in hardware-grouping
perf stat: Code refactoring in hardware-grouping
perf stat: Add tool events support in hardware-grouping
perf stat: Add TSC support in hardware-grouping
perf stat: Fix a return error issue in hardware-grouping
perf stat: Add check to ensure correctness in platform that does not
support hardware-grouping
perf pmu-events: Add event counter data for Tigerlake

tools/lib/bitmap.c | 20 +
tools/perf/builtin-stat.c | 7 +
.../arch/x86/cascadelakex/cache.json | 1237 ++++++++++++
.../arch/x86/cascadelakex/counter.json | 17 +
.../arch/x86/cascadelakex/floating-point.json | 16 +
.../arch/x86/cascadelakex/frontend.json | 68 +
.../arch/x86/cascadelakex/memory.json | 751 ++++++++
.../arch/x86/cascadelakex/other.json | 168 ++
.../arch/x86/cascadelakex/pipeline.json | 102 +
.../arch/x86/cascadelakex/uncore-cache.json | 1138 +++++++++++
.../x86/cascadelakex/uncore-interconnect.json | 1272 +++++++++++++
.../arch/x86/cascadelakex/uncore-io.json | 394 ++++
.../arch/x86/cascadelakex/uncore-memory.json | 509 +++++
.../arch/x86/cascadelakex/uncore-power.json | 25 +
.../arch/x86/cascadelakex/virtual-memory.json | 28 +
.../pmu-events/arch/x86/icelakex/cache.json | 98 +
.../pmu-events/arch/x86/icelakex/counter.json | 17 +
.../arch/x86/icelakex/floating-point.json | 13 +
.../arch/x86/icelakex/frontend.json | 55 +
.../pmu-events/arch/x86/icelakex/memory.json | 53 +
.../pmu-events/arch/x86/icelakex/other.json | 52 +
.../arch/x86/icelakex/pipeline.json | 92 +
.../arch/x86/icelakex/uncore-cache.json | 965 ++++++++++
.../x86/icelakex/uncore-interconnect.json | 1667 +++++++++++++++++
.../arch/x86/icelakex/uncore-io.json | 966 ++++++++++
.../arch/x86/icelakex/uncore-memory.json | 186 ++
.../arch/x86/icelakex/uncore-power.json | 26 +
.../arch/x86/icelakex/virtual-memory.json | 22 +
.../arch/x86/sapphirerapids/cache.json | 104 +
.../arch/x86/sapphirerapids/counter.json | 17 +
.../x86/sapphirerapids/floating-point.json | 25 +
.../arch/x86/sapphirerapids/frontend.json | 98 +-
.../arch/x86/sapphirerapids/memory.json | 44 +
.../arch/x86/sapphirerapids/other.json | 40 +
.../arch/x86/sapphirerapids/pipeline.json | 118 ++
.../arch/x86/sapphirerapids/uncore-cache.json | 534 +++++-
.../arch/x86/sapphirerapids/uncore-cxl.json | 56 +
.../sapphirerapids/uncore-interconnect.json | 476 +++++
.../arch/x86/sapphirerapids/uncore-io.json | 373 ++++
.../x86/sapphirerapids/uncore-memory.json | 391 ++++
.../arch/x86/sapphirerapids/uncore-power.json | 24 +
.../x86/sapphirerapids/virtual-memory.json | 20 +
.../pmu-events/arch/x86/tigerlake/cache.json | 65 +
.../arch/x86/tigerlake/counter.json | 7 +
.../arch/x86/tigerlake/floating-point.json | 13 +
.../arch/x86/tigerlake/frontend.json | 56 +
.../pmu-events/arch/x86/tigerlake/memory.json | 31 +
.../pmu-events/arch/x86/tigerlake/other.json | 4 +
.../arch/x86/tigerlake/pipeline.json | 96 +
.../x86/tigerlake/uncore-interconnect.json | 11 +
.../arch/x86/tigerlake/uncore-memory.json | 6 +
.../arch/x86/tigerlake/uncore-other.json | 1 +
.../arch/x86/tigerlake/virtual-memory.json | 20 +
tools/perf/pmu-events/jevents.py | 179 +-
tools/perf/pmu-events/pmu-events.h | 26 +-
tools/perf/util/metricgroup.c | 927 +++++++++
tools/perf/util/metricgroup.h | 82 +
tools/perf/util/pmu.c | 5 +
tools/perf/util/pmu.h | 1 +
tools/perf/util/stat.h | 1 +
60 files changed, 13790 insertions(+), 25 deletions(-)
create mode 100644 tools/perf/pmu-events/arch/x86/cascadelakex/counter.json
create mode 100644 tools/perf/pmu-events/arch/x86/icelakex/counter.json
create mode 100644 tools/perf/pmu-events/arch/x86/sapphirerapids/counter.json
create mode 100644 tools/perf/pmu-events/arch/x86/tigerlake/counter.json

--
2.39.3

2023-09-25 08:58:07

by Wang, Weilin

[permalink] [raw]

On 2023-09-25 2:18 a.m., [email protected] wrote:
> From: Weilin Wang <[email protected]>
>
> Hardware counter and event information could be used to help creating event
> groups that better utilize hardware counters and improve multiplexing.
>
> Add a cmd option to allow user to choose this new metric grouping method.
>

It's better to mention this option is only available with -M.

> $ perf stat -M TopdownL1 -a --hardware-grouping
>

I think it should be counter aware grouping. Hardware grouping sounds
like the group is created by hardware.

Eventually we should make it default and let the default grouping code
take the counter information into account.

Thanks,
Kan

> Signed-off-by: Weilin Wang <[email protected]>
> ---
> tools/perf/builtin-stat.c | 7 +++++++
> tools/perf/util/metricgroup.c | 5 +++++
> tools/perf/util/metricgroup.h | 1 +
> tools/perf/util/stat.h | 1 +
> 4 files changed, 14 insertions(+)
>
> diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c
> index 07b48f6df..1a7dbfff5 100644
> --- a/tools/perf/builtin-stat.c
> +++ b/tools/perf/builtin-stat.c
> @@ -1256,6 +1256,8 @@ static struct option stat_options[] = {
> "don't try to share events between metrics in a group"),
> OPT_BOOLEAN(0, "metric-no-threshold", &stat_config.metric_no_threshold,
> "don't try to share events between metrics in a group "),
> + OPT_BOOLEAN(0, "hardware-grouping", &stat_config.hardware_aware_grouping,
> + "Use hardware aware metric grouping method"),
> OPT_BOOLEAN(0, "topdown", &topdown_run,
> "measure top-down statistics"),
> OPT_UINTEGER(0, "td-level", &stat_config.topdown_level,
> @@ -2061,6 +2063,7 @@ static int add_default_attributes(void)
> stat_config.metric_no_threshold,
> stat_config.user_requested_cpu_list,
> stat_config.system_wide,
> + stat_config.hardware_aware_grouping,
> &stat_config.metric_events);
> }
>
> @@ -2094,6 +2097,7 @@ static int add_default_attributes(void)
> stat_config.metric_no_threshold,
> stat_config.user_requested_cpu_list,
> stat_config.system_wide,
> + stat_config.hardware_aware_grouping,
> &stat_config.metric_events);
> }
>
> @@ -2128,6 +2132,7 @@ static int add_default_attributes(void)
> /*metric_no_threshold=*/true,
> stat_config.user_requested_cpu_list,
> stat_config.system_wide,
> + stat_config.hardware_aware_grouping,
> &stat_config.metric_events) < 0)
> return -1;
> }
> @@ -2169,6 +2174,7 @@ static int add_default_attributes(void)
> /*metric_no_threshold=*/true,
> stat_config.user_requested_cpu_list,
> stat_config.system_wide,
> + stat_config.hardware_aware_grouping,
> &stat_config.metric_events) < 0)
> return -1;
>
> @@ -2702,6 +2708,7 @@ int cmd_stat(int argc, const char **argv)
> stat_config.metric_no_threshold,
> stat_config.user_requested_cpu_list,
> stat_config.system_wide,
> + stat_config.hardware_aware_grouping,
> &stat_config.metric_events);
> zfree(&metrics);
> }
> diff --git a/tools/perf/util/metricgroup.c b/tools/perf/util/metricgroup.c
> index 6231044a4..b08af6860 100644
> --- a/tools/perf/util/metricgroup.c
> +++ b/tools/perf/util/metricgroup.c
> @@ -1690,12 +1690,17 @@ int metricgroup__parse_groups(struct evlist *perf_evlist,
> bool metric_no_threshold,
> const char *user_requested_cpu_list,
> bool system_wide,
> + bool hardware_aware_grouping,
> struct rblist *metric_events)
> {
> const struct pmu_metrics_table *table = pmu_metrics_table__find();
>
> if (!table)
> return -EINVAL;
> + if (hardware_aware_grouping) {
> + pr_debug("Use hardware aware grouping instead of traditional metric grouping method\n");
> + }
> +
>
> return parse_groups(perf_evlist, pmu, str, metric_no_group, metric_no_merge,
> metric_no_threshold, user_requested_cpu_list, system_wide,
> diff --git a/tools/perf/util/metricgroup.h b/tools/perf/util/metricgroup.h
> index d5325c6ec..779f6ede1 100644
> --- a/tools/perf/util/metricgroup.h
> +++ b/tools/perf/util/metricgroup.h
> @@ -77,6 +77,7 @@ int metricgroup__parse_groups(struct evlist *perf_evlist,
> bool metric_no_threshold,
> const char *user_requested_cpu_list,
> bool system_wide,
> + bool hardware_aware_grouping,
> struct rblist *metric_events);
> int metricgroup__parse_groups_test(struct evlist *evlist,
> const struct pmu_metrics_table *table,
> diff --git a/tools/perf/util/stat.h b/tools/perf/util/stat.h
> index 325d0fad1..f97d6282b 100644
> --- a/tools/perf/util/stat.h
> +++ b/tools/perf/util/stat.h
> @@ -87,6 +87,7 @@ struct perf_stat_config {
> bool metric_no_group;
> bool metric_no_merge;
> bool metric_no_threshold;
> + bool hardware_aware_grouping;
> bool stop_read_counter;
> bool iostat_run;
> char *user_requested_cpu_list;