Received: by 2002:a05:7412:d8a:b0:e2:908c:2ebd with SMTP id b10csp1315115rdg; Fri, 13 Oct 2023 18:52:50 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHIC/tiiN6iS2X4Wca4TE0l1uGSMX3N+iASxxLNB4woY/SlDhrU2QqXUDrxPjwhn2yBSeC5 X-Received: by 2002:a17:90a:c718:b0:27d:6b5:9e07 with SMTP id o24-20020a17090ac71800b0027d06b59e07mr8741704pjt.1.1697248369972; Fri, 13 Oct 2023 18:52:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697248369; cv=none; d=google.com; s=arc-20160816; b=xnFSpDhXO20pEZyfcIcjn9JyaqcbzT5iSS84WBPEU4nPL3ewBbQxlWliVWlGWngzBm dunbHWy+9HUghd7kJ3uOrfLzt4lZOKQylzWmO0BPmwiEyVnuIIoIeK0T4RfSXkBFWQqF y+KIhxbwcDKS/v2k/9Am+t+bEEh55/bXzDn/9BHQmjSPHXPZKZSxcbTf0pOgt1JEozym k9oPoydp/RG+bVzgBCQi2m0QYXXs8cFrpXXXPZZBf69wskMxHpZiGS6fgdYerdr3LGGw /xc1xWG5nfDLyJZdNyV/FvNpjmTqUJzCXNdgTwZu/1SkeESt7t2Z0afqOHs3DVJndjKz FEtQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=xfdWolEpwbi9iMh/bdaRVmKY9mDXsD2gtTqlTQQMMoE=; fh=fjecWn2tsXHKF5pPtw+gI2dQE/iSXdFsWSc8KUP/zmc=; b=NpQ5dktHeb8SaxtZVmhffoiHBI2oFZA/IcsdEk5krZcHkeTmLGThI0QkRLmV+ZGNHN RLJGFxxGAjnNSz6AciUJkFfKHHM0RMc+VD1YfqF2/HcZ8at8wiphpLGEuO33akgNawl1 LXDEwqSbeyi56vArsCTcbllWoJlzyblKjXsLU3JKbVDPKhJzPXmX0LC47sqjAgIr7Zyk bz2SNpNPpBtr00DG+TaBGS91Fm9ymTHZbnq675PQaC+WmGOTWbnyYfBBCljnzcu46jBs kcV5hr1qpywh+Xa5fRtPJgURi9+cv4AQ31kPhve+NgzOTIDnwknFYXSWQkEqbP4O4L0f zp3w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=BLVlCezU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id i1-20020a635841000000b00578b8d202b0si5996198pgm.536.2023.10.13.18.52.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 18:52:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=BLVlCezU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 0E6168350FE2; Fri, 13 Oct 2023 18:52:49 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232750AbjJNBw2 (ORCPT + 99 others); Fri, 13 Oct 2023 21:52:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39636 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232698AbjJNBwW (ORCPT ); Fri, 13 Oct 2023 21:52:22 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A5992C9; Fri, 13 Oct 2023 18:52:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697248339; x=1728784339; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=FdSzq34zoHI6D8snY8m3i2h6zKnXHCpA5Ta+mHr9V8A=; b=BLVlCezUGNc1O1Z3t7YaCU79sRImJ+tPm4B3WrcYp/Dwz+BGcGOWSpLb mofmOUDUQr4wZfqekf2DhG26ZicDKs1K3IV/LASVc6ViQHMWr4/vIKVCh mwtoZLH5Ypm3pfDqNTR6Ar6279dtc/iWys/ByEZeo8mwUD7gfwTueemM5 V2phL3drPUrLtmMcmqdiG4TPv3s0CzChohBRgTB5GlZ3zZhPqHj7R68rp /1Ul2sxlajBIZYJVNFtfKkB4CTDTA8PoOItcvgq77pT553N4nWTSuCRXC bKNv6090ohywfPbwBxNtTpDr5KShmUSZfw4zvr/Ca/15PLSEyAlejAGjh Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10862"; a="389154679" X-IronPort-AV: E=Sophos;i="6.03,223,1694761200"; d="scan'208";a="389154679" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Oct 2023 18:52:15 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10862"; a="731565660" X-IronPort-AV: E=Sophos;i="6.03,223,1694761200"; d="scan'208";a="731565660" Received: from b49691a75598.jf.intel.com ([10.54.34.22]) by orsmga006.jf.intel.com with ESMTP; 13 Oct 2023 18:52:15 -0700 From: weilin.wang@intel.com To: weilin.wang@intel.com, Ian Rogers , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Adrian Hunter , Kan Liang Cc: linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, Perry Taylor , Samantha Alt , Caleb Biggers , Mark Rutland , Yang Jihong Subject: [RFC PATCH v2 03/17] perf pmu-events: Add functions in jevent.py to parse counter and event info for hardware aware grouping Date: Fri, 13 Oct 2023 18:51:48 -0700 Message-Id: <20231014015202.1175377-4-weilin.wang@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20231014015202.1175377-1-weilin.wang@intel.com> References: <20231014015202.1175377-1-weilin.wang@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Fri, 13 Oct 2023 18:52:49 -0700 (PDT) From: Weilin Wang These functions are added to parse event counter restrictions and counter availability info from json files so that the metric grouping method could do grouping based on the counter restriction of events and the counters that are available on the system. Signed-off-by: Weilin Wang --- tools/perf/pmu-events/jevents.py | 172 ++++++++++++++++++++++++++++- tools/perf/pmu-events/pmu-events.h | 25 ++++- 2 files changed, 189 insertions(+), 8 deletions(-) diff --git a/tools/perf/pmu-events/jevents.py b/tools/perf/pmu-events/jevents.py index a7e883322..1b3776c92 100755 --- a/tools/perf/pmu-events/jevents.py +++ b/tools/perf/pmu-events/jevents.py @@ -23,6 +23,8 @@ _metric_tables = [] _sys_metric_tables = [] # Mapping between sys event table names and sys metric table names. _sys_event_table_to_metric_table_mapping = {} +# List of regular PMU counter layout tables. +_pmu_layouts_tables = [] # Map from an event name to an architecture standard # JsonEvent. Architecture standard events are in json files in the top # f'{_args.starting_dir}/{_args.arch}' directory. @@ -31,6 +33,9 @@ _arch_std_events = {} _pending_events = [] # Name of events table to be written out _pending_events_tblname = None +# PMU counter layout to write out when the table is closed +_pending_pmu_counts = [] # Name of PMU counter layout table to be written out +_pending_pmu_counts_tblname = None # Metrics to write out when the table is closed _pending_metrics = [] # Name of metrics table to be written out @@ -47,10 +52,17 @@ _json_event_attributes = [ 'event', # Short things in alphabetical order. 'compat', 'deprecated', 'perpkg', 'unit', + # Counter this event could use + 'counter', # Longer things (the last won't be iterated over during decompress). 'long_desc' ] +# Attributes that are in pmu_unit_layout. +_json_layout_attributes = [ + 'pmu', 'desc', 'size', 'fixed_size' +] + # Attributes that are in pmu_metric rather than pmu_event. _json_metric_attributes = [ 'metric_name', 'metric_group', 'metric_expr', 'metric_threshold', @@ -58,7 +70,9 @@ _json_metric_attributes = [ 'default_metricgroup_name', 'aggr_mode', 'event_grouping' ] # Attributes that are bools or enum int values, encoded as '0', '1',... -_json_enum_attributes = ['aggr_mode', 'deprecated', 'event_grouping', 'perpkg'] +_json_enum_attributes = ['aggr_mode', 'deprecated', 'event_grouping', 'perpkg', + 'size', 'fixed_size' +] def removesuffix(s: str, suffix: str) -> str: """Remove the suffix from a string @@ -289,6 +303,7 @@ class JsonEvent: 'cpu_core': 'cpu_core', 'cpu_atom': 'cpu_atom', 'ali_drw': 'ali_drw', + 'Core': 'default_core', } return table[unit] if unit in table else f'uncore_{unit.lower()}' @@ -314,6 +329,9 @@ class JsonEvent: if 'Errata' in jd: extra_desc += ' Spec update: ' + jd['Errata'] self.pmu = unit_to_pmu(jd.get('Unit')) + self.counter = jd.get('Counter') + self.size = jd.get('Size') + self.fixed_size = jd.get('FixedSize') filter = jd.get('Filter') self.unit = jd.get('ScaleUnit') self.perpkg = jd.get('PerPkg') @@ -377,8 +395,16 @@ class JsonEvent: s += f'\t{attr} = {value},\n' return s + '}' - def build_c_string(self, metric: bool) -> str: + def build_c_string(self, metric: bool, layout: bool = False) -> str: s = '' + if layout: + for attr in _json_layout_attributes: + x = getattr(self, attr) + if attr in _json_enum_attributes: + s += x if x else '0' + else: + s += f'{x}\\000' if x else '\\000' + return s for attr in _json_metric_attributes if metric else _json_event_attributes: x = getattr(self, attr) if metric and x and attr == 'metric_expr': @@ -393,10 +419,10 @@ class JsonEvent: s += f'{x}\\000' if x else '\\000' return s - def to_c_string(self, metric: bool) -> str: + def to_c_string(self, metric: bool, layout: bool = False) -> str: """Representation of the event as a C struct initializer.""" - s = self.build_c_string(metric) + s = self.build_c_string(metric, layout) return f'{{ { _bcs.offsets[s] } }}, /* {s} */\n' @@ -433,6 +459,8 @@ def preprocess_arch_std_files(archpath: str) -> None: _arch_std_events[event.name.lower()] = event if event.metric_name: _arch_std_events[event.metric_name.lower()] = event + if event.size: + _arch_std_events[event.pmu.lower()] = event def add_events_table_entries(item: os.DirEntry, topic: str) -> None: @@ -442,6 +470,8 @@ def add_events_table_entries(item: os.DirEntry, topic: str) -> None: _pending_events.append(e) if e.metric_name: _pending_metrics.append(e) + if e.size: + _pending_pmu_counts.append(e) def print_pending_events() -> None: @@ -555,6 +585,33 @@ const struct pmu_table_entry {_pending_metrics_tblname}[] = {{ """) _args.output_file.write('};\n\n') +def print_pending_pmu_counts() -> None: + + def pmu_counts_cmp_key(j: JsonEvent) -> Tuple[bool, str, str]: + def fix_none(s: Optional[str]) -> str: + if s is None: + return '' + return s + + return (j.desc is not None, fix_none(j.pmu), fix_none(j.size)) + + global _pending_pmu_counts + if not _pending_pmu_counts: + return + + global _pending_pmu_counts_tblname + global pmu_layouts_tables + _pmu_layouts_tables.append(_pending_pmu_counts_tblname) + + _args.output_file.write( + f'static const struct compact_pmu_event {_pending_pmu_counts_tblname}[] = {{\n') + + for pmu_layout in sorted(_pending_pmu_counts, key=pmu_counts_cmp_key): + _args.output_file.write(pmu_layout.to_c_string(metric=False, layout=True)) + _pending_pmu_counts = [] + + _args.output_file.write('};\n\n') + def get_topic(topic: str) -> str: if topic.endswith('metrics.json'): return 'metrics' @@ -595,6 +652,8 @@ def preprocess_one_file(parents: Sequence[str], item: os.DirEntry) -> None: if event.metric_name: _bcs.add(pmu_name, metric=True) _bcs.add(event.build_c_string(metric=True), metric=True) + if event.size: + _bcs.add(event.build_c_string(metric=False, layout=True), metric=False) def process_one_file(parents: Sequence[str], item: os.DirEntry) -> None: """Process a JSON file during the main walk.""" @@ -608,11 +667,14 @@ def process_one_file(parents: Sequence[str], item: os.DirEntry) -> None: if item.is_dir() and is_leaf_dir(item.path): print_pending_events() print_pending_metrics() + print_pending_pmu_counts() global _pending_events_tblname _pending_events_tblname = file_name_to_table_name('pmu_events_', parents, item.name) global _pending_metrics_tblname _pending_metrics_tblname = file_name_to_table_name('pmu_metrics_', parents, item.name) + global _pending_pmu_counts_tblname + _pending_pmu_counts_tblname = file_name_to_table_name('pmu_layouts_', parents, item.name) if item.name == 'sys': _sys_event_table_to_metric_table_mapping[_pending_events_tblname] = _pending_metrics_tblname @@ -646,6 +708,12 @@ struct pmu_metrics_table { uint32_t num_pmus; }; +/* Struct used to make the PMU counter layout table implementation opaque to callers. */ +struct pmu_layouts_table { + const struct compact_pmu_event *entries; + size_t length; +}; + /* * Map a CPU to its table of PMU events. The CPU is identified by the * cpuid field, which is an arch-specific identifier for the CPU. @@ -659,6 +727,7 @@ struct pmu_events_map { const char *cpuid; struct pmu_events_table event_table; struct pmu_metrics_table metric_table; + struct pmu_layouts_table layout_table; }; /* @@ -703,6 +772,12 @@ const struct pmu_events_map pmu_events_map[] = { metric_size = '0' if event_size == '0' and metric_size == '0': continue + layout_tblname = file_name_to_table_name('pmu_layouts_', [], row[2].replace('/', '_')) + if layout_tblname in _pmu_layouts_tables: + layout_size = f'ARRAY_SIZE({layout_tblname})' + else: + layout_tblname = 'NULL' + layout_size = '0' cpuid = row[0].replace('\\', '\\\\') _args.output_file.write(f"""{{ \t.arch = "{arch}", @@ -714,6 +789,10 @@ const struct pmu_events_map pmu_events_map[] = { \t.metric_table = {{ \t\t.pmus = {metric_tblname}, \t\t.num_pmus = {metric_size} +\t}}, +\t.layout_table = {{ +\t\t.entries = {layout_tblname}, +\t\t.length = {layout_size} \t}} }}, """) @@ -724,6 +803,7 @@ const struct pmu_events_map pmu_events_map[] = { \t.cpuid = 0, \t.event_table = { 0, 0 }, \t.metric_table = { 0, 0 }, +\t.layout_table = { 0, 0 }, } }; """) @@ -812,6 +892,24 @@ static void decompress_metric(int offset, struct pmu_metric *pm) _args.output_file.write('\twhile (*p++);') _args.output_file.write("""} +static void decompress_layout(int offset, struct pmu_layout *pm) +{ +\tconst char *p = &big_c_string[offset]; +""") + for attr in _json_layout_attributes: + _args.output_file.write(f'\n\tpm->{attr} = ') + if attr in _json_enum_attributes: + _args.output_file.write("*p - '0';\n") + else: + _args.output_file.write("(*p == '\\0' ? NULL : p);\n") + if attr == _json_layout_attributes[-1]: + continue + if attr in _json_enum_attributes: + _args.output_file.write('\tp++;') + else: + _args.output_file.write('\twhile (*p++);') + _args.output_file.write("""} + static int pmu_events_table__for_each_event_pmu(const struct pmu_events_table *table, const struct pmu_table_entry *pmu, pmu_event_iter_fn fn, @@ -967,6 +1065,21 @@ int pmu_metrics_table__for_each_metric(const struct pmu_metrics_table *table, return 0; } +int pmu_layouts_table__for_each_layout(const struct pmu_layouts_table *table, + pmu_layout_iter_fn fn, + void *data) { + for (size_t i = 0; i < table->length; i++) { + struct pmu_layout pm; + int ret; + + decompress_layout(table->entries[i].offset, &pm); + ret = fn(&pm, data); + if (ret) + return ret; + } + return 0; +} + const struct pmu_events_table *perf_pmu__find_events_table(struct perf_pmu *pmu) { const struct pmu_events_table *table = NULL; @@ -1031,6 +1144,33 @@ const struct pmu_metrics_table *perf_pmu__find_metrics_table(struct perf_pmu *pm return table; } +const struct pmu_layouts_table *perf_pmu__find_layouts_table(struct perf_pmu *pmu) +{ + const struct pmu_layouts_table *table = NULL; + char *cpuid = perf_pmu__getcpuid(pmu); + int i; + + /* on some platforms which uses cpus map, cpuid can be NULL for + * PMUs other than CORE PMUs. + */ + if (!cpuid) + return NULL; + + i = 0; + for (;;) { + const struct pmu_events_map *map = &pmu_events_map[i++]; + if (!map->arch) + break; + + if (!strcmp_cpuid_str(map->cpuid, cpuid)) { + table = &map->layout_table; + break; + } + } + free(cpuid); + return table; +} + const struct pmu_events_table *find_core_events_table(const char *arch, const char *cpuid) { for (const struct pmu_events_map *tables = &pmu_events_map[0]; @@ -1052,6 +1192,16 @@ const struct pmu_metrics_table *find_core_metrics_table(const char *arch, const } return NULL; } +const struct pmu_layouts_table *find_core_layouts_table(const char *arch, const char *cpuid) +{ + for (const struct pmu_events_map *tables = &pmu_events_map[0]; + tables->arch; + tables++) { + if (!strcmp(tables->arch, arch) && !strcmp_cpuid_str(tables->cpuid, cpuid)) + return &tables->layout_table; + } + return NULL; +} int pmu_for_each_core_event(pmu_event_iter_fn fn, void *data) { @@ -1080,6 +1230,19 @@ int pmu_for_each_core_metric(pmu_metric_iter_fn fn, void *data) return 0; } +int pmu_for_each_core_layout(pmu_layout_iter_fn fn, void *data) +{ + for (const struct pmu_events_map *tables = &pmu_events_map[0]; + tables->arch; + tables++) { + int ret = pmu_layouts_table__for_each_layout(&tables->layout_table, fn, data); + + if (ret) + return ret; + } + return 0; +} + const struct pmu_events_table *find_sys_events_table(const char *name) { for (const struct pmu_sys_events *tables = &pmu_sys_event_tables[0]; @@ -1236,6 +1399,7 @@ struct pmu_table_entry { ftw(arch_path, [], process_one_file) print_pending_events() print_pending_metrics() + print_pending_pmu_counts() print_mapping_table(archs) print_system_mapping_table() diff --git a/tools/perf/pmu-events/pmu-events.h b/tools/perf/pmu-events/pmu-events.h index f5aa96f16..65e0c5dd8 100644 --- a/tools/perf/pmu-events/pmu-events.h +++ b/tools/perf/pmu-events/pmu-events.h @@ -45,6 +45,7 @@ struct pmu_event { const char *desc; const char *topic; const char *long_desc; + const char *counter; const char *pmu; const char *unit; bool perpkg; @@ -67,8 +68,16 @@ struct pmu_metric { enum metric_event_groups event_grouping; }; +struct pmu_layout { + const char *pmu; + const char *desc; + int size; + int fixed_size; +}; + struct pmu_events_table; struct pmu_metrics_table; +struct pmu_layouts_table; typedef int (*pmu_event_iter_fn)(const struct pmu_event *pe, const struct pmu_events_table *table, @@ -78,15 +87,20 @@ typedef int (*pmu_metric_iter_fn)(const struct pmu_metric *pm, const struct pmu_metrics_table *table, void *data); +typedef int (*pmu_layout_iter_fn)(const struct pmu_layout *pm, + void *data); + int pmu_events_table__for_each_event(const struct pmu_events_table *table, struct perf_pmu *pmu, pmu_event_iter_fn fn, void *data); int pmu_events_table__find_event(const struct pmu_events_table *table, - struct perf_pmu *pmu, - const char *name, - pmu_event_iter_fn fn, - void *data); + struct perf_pmu *pmu, + const char *name, + pmu_event_iter_fn fn, + void *data); +int pmu_layouts_table__for_each_layout(const struct pmu_layouts_table *table, pmu_layout_iter_fn fn, + void *data); size_t pmu_events_table__num_events(const struct pmu_events_table *table, struct perf_pmu *pmu); @@ -95,10 +109,13 @@ int pmu_metrics_table__for_each_metric(const struct pmu_metrics_table *table, pm const struct pmu_events_table *perf_pmu__find_events_table(struct perf_pmu *pmu); const struct pmu_metrics_table *perf_pmu__find_metrics_table(struct perf_pmu *pmu); +const struct pmu_layouts_table *perf_pmu__find_layouts_table(struct perf_pmu *pmu); const struct pmu_events_table *find_core_events_table(const char *arch, const char *cpuid); const struct pmu_metrics_table *find_core_metrics_table(const char *arch, const char *cpuid); +const struct pmu_layouts_table *find_core_layouts_table(const char *arch, const char *cpuid); int pmu_for_each_core_event(pmu_event_iter_fn fn, void *data); int pmu_for_each_core_metric(pmu_metric_iter_fn fn, void *data); +int pmu_for_each_core_layout(pmu_layout_iter_fn fn, void *data); const struct pmu_events_table *find_sys_events_table(const char *name); const struct pmu_metrics_table *find_sys_metrics_table(const char *name); -- 2.39.3