2021-02-06 15:10:31

by Leo Yan

[permalink] [raw]
Subject: [PATCH v3 0/8] coresight: etm-perf: Fix pid tracing with VHE

This patch series is to support PID tracing with Virtualization Host
Extensions (VHE).

To be backward compatibility, and can both support PID tracing for the
kernel is running at either EL1 or EL2, the two new PMU formats
"contextid1" and "contextid2" are introduced, which works as switches
to trace PID for EL1 kernel and EL2 kernel respectively.

The existed PMU format "contextid" needs to be backward compatible for
users, it's changed as an alias for "contextid1" on EL1 kernel and for
"contextid2" on EL2 kernel. Therefore, even without setting "contextid"
config, the perf tool can dynamically pick up the config for PID
tracing, the user doesn't have to set the "contexid" config manually.

This patch series can be cleanly applied on perf/core branch:

commit cd07e536b020 ("Merge remote-tracking branch 'torvalds/master' into perf/core")

... and applied on the mainline kernel:

commit 1e0d27fce010 ("Merge branch 'akpm' (patches from Andrew)")

The patch series has been tested on Arm Juno-r2 board. Verified the
kernel with EL1 and didn't find issue; after some hacking in kernel
driver and tool to emulate the code paths for kernel on EL2, can see
the code path is hit without failure.


Changes from v2:
* Split into two patches for clarification comment on perf options, one
patch is for kernel change and the another one is for tools' change
(Suzuki);
* Simplified cs_etm__get_pid_fmt() to return ETM_OPT_CTXTID2 OR
ETM_OPT_CTXTID, but not both (Suzuki);
* Cached "pid_fmt" in cs_etm_decoder__set_tid() (Suzuki);
* Refined documentation for more clear description for PMU format usages
(Mike);
* Added Suzuki's and Mike's Reviewed tags.

Changes from v1:
* Refactored PMU formats, added formats "contextid1"/"contextid2", and
reworked format "contextid" (Suzuki/Mathieu);
* Refined the comments for perf configs (Leo/Mike);
* Added patch 07/07 for description PID tracing in docs;
* Found the issue for bitmap for option, extracted patch 03/07 for the
fixing.

Changes from RFC:
* Added comments to clarify cases requested (Leo);
* Explain the change to generic flags for cs_etm_set_option() in the
commit description;
* Stored PID format in metadata and passed it to decoder (Leo);
* Enhanced cs-etm for backward compatibility (Denis Nikitin).


Leo Yan (4):
coresight: etm-perf: Clarify comment on perf options
tools headers UAPI: Update tools' copy of linux/coresight-pmu.h
perf cs-etm: Add helper cs_etm__get_pid_fmt()
Documentation: coresight: Add PID tracing description

Suzuki K Poulose (4):
coresight: etm-perf: Support PID tracing for kernel at EL2
perf cs-etm: Fix bitmap for option
perf cs-etm: Support PID tracing in config
perf cs-etm: Detect pid in VMID for kernel running at EL2

Documentation/trace/coresight/coresight.rst | 32 ++++++++
.../hwtracing/coresight/coresight-etm-perf.c | 32 +++++++-
.../coresight/coresight-etm4x-core.c | 13 ++++
include/linux/coresight-pmu.h | 20 +++--
tools/include/linux/coresight-pmu.h | 20 +++--
tools/perf/arch/arm/util/cs-etm.c | 73 +++++++++++++++----
.../perf/util/cs-etm-decoder/cs-etm-decoder.c | 38 +++++++++-
tools/perf/util/cs-etm.c | 42 +++++++++++
tools/perf/util/cs-etm.h | 1 +
9 files changed, 239 insertions(+), 32 deletions(-)

--
2.25.1


2021-02-06 15:12:09

by Leo Yan

[permalink] [raw]
Subject: [PATCH v3 1/8] coresight: etm-perf: Clarify comment on perf options

In theory, the options should be arbitrary values and are neutral for
any ETM version; so far perf tool uses ETMv3.5/PTM ETMCR config bits
except for register's bit definitions, also uses as options.

This can introduce confusion, especially if we want to add a new option
but the new option is not supported by ETMv3.5/PTM ETMCR. But on the
other hand, we cannot change options since these options are generic
CoreSight PMU ABI.

For easier maintenance and avoid confusion, this patch refines the
comment to clarify perf options, and gives out the background info for
these bits are coming from ETMv3.5/PTM. Afterwards, we should take
these options as general knobs, and if there have any confliction with
ETMv3.5/PTM, should consider to define saperate macros for ETMv3.5/PTM
ETMCR config bits.

Suggested-by: Suzuki K Poulose <[email protected]>
Signed-off-by: Leo Yan <[email protected]>
Reviewed-by: Suzuki K Poulose <[email protected]>
---
.../hwtracing/coresight/coresight-etm-perf.c | 5 ++++-
include/linux/coresight-pmu.h | 17 ++++++++++++-----
2 files changed, 16 insertions(+), 6 deletions(-)

diff --git a/drivers/hwtracing/coresight/coresight-etm-perf.c b/drivers/hwtracing/coresight/coresight-etm-perf.c
index bdc34ca449f7..465ef1aa8c82 100644
--- a/drivers/hwtracing/coresight/coresight-etm-perf.c
+++ b/drivers/hwtracing/coresight/coresight-etm-perf.c
@@ -27,7 +27,10 @@ static bool etm_perf_up;
static DEFINE_PER_CPU(struct perf_output_handle, ctx_handle);
static DEFINE_PER_CPU(struct coresight_device *, csdev_src);

-/* ETMv3.5/PTM's ETMCR is 'config' */
+/*
+ * The PMU formats were orignally for ETMv3.5/PTM's ETMCR 'config';
+ * now take them as general formats and apply on all ETMs.
+ */
PMU_FORMAT_ATTR(cycacc, "config:" __stringify(ETM_OPT_CYCACC));
PMU_FORMAT_ATTR(contextid, "config:" __stringify(ETM_OPT_CTXTID));
PMU_FORMAT_ATTR(timestamp, "config:" __stringify(ETM_OPT_TS));
diff --git a/include/linux/coresight-pmu.h b/include/linux/coresight-pmu.h
index b0e35eec6499..5dc47cfdcf07 100644
--- a/include/linux/coresight-pmu.h
+++ b/include/linux/coresight-pmu.h
@@ -10,11 +10,18 @@
#define CORESIGHT_ETM_PMU_NAME "cs_etm"
#define CORESIGHT_ETM_PMU_SEED 0x10

-/* ETMv3.5/PTM's ETMCR config bit */
-#define ETM_OPT_CYCACC 12
-#define ETM_OPT_CTXTID 14
-#define ETM_OPT_TS 28
-#define ETM_OPT_RETSTK 29
+/*
+ * Below are the definition of bit offsets for perf option, and works as
+ * arbitrary values for all ETM versions.
+ *
+ * Most of them are orignally from ETMv3.5/PTM's ETMCR config, therefore,
+ * ETMv3.5/PTM doesn't define ETMCR config bits with prefix "ETM3_" and
+ * directly use below macros as config bits.
+ */
+#define ETM_OPT_CYCACC 12
+#define ETM_OPT_CTXTID 14
+#define ETM_OPT_TS 28
+#define ETM_OPT_RETSTK 29

/* ETMv4 CONFIGR programming bits for the ETM OPTs */
#define ETM4_CFG_BIT_CYCACC 4
--
2.25.1

2021-02-06 15:12:29

by Leo Yan

[permalink] [raw]
Subject: [PATCH v3 4/8] perf cs-etm: Fix bitmap for option

From: Suzuki K Poulose <[email protected]>

When set option with macros ETM_OPT_CTXTID and ETM_OPT_TS, it wrongly
takes these two values (14 and 28 prespectively) as bit masks, but
actually both are the offset for bits. But this doesn't lead to
further failure due to the AND logic operation will be always true for
ETM_OPT_CTXTID / ETM_OPT_TS.

This patch defines new independent macros (rather than using the
"config" bits) for requesting the "contextid" and "timestamp" for
cs_etm_set_option().

[leoy: Extract the change as a separate patch for easier review]
Signed-off-by: Suzuki K Poulose <[email protected]>
Signed-off-by: Leo Yan <[email protected]>
Reviewed-by: Mike Leach <[email protected]>
---
tools/perf/arch/arm/util/cs-etm.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/tools/perf/arch/arm/util/cs-etm.c b/tools/perf/arch/arm/util/cs-etm.c
index bd446aba64f7..c25c878fd06c 100644
--- a/tools/perf/arch/arm/util/cs-etm.c
+++ b/tools/perf/arch/arm/util/cs-etm.c
@@ -156,6 +156,10 @@ static int cs_etm_set_timestamp(struct auxtrace_record *itr,
return err;
}

+#define ETM_SET_OPT_CTXTID (1 << 0)
+#define ETM_SET_OPT_TS (1 << 1)
+#define ETM_SET_OPT_MASK (ETM_SET_OPT_CTXTID | ETM_SET_OPT_TS)
+
static int cs_etm_set_option(struct auxtrace_record *itr,
struct evsel *evsel, u32 option)
{
@@ -169,17 +173,17 @@ static int cs_etm_set_option(struct auxtrace_record *itr,
!cpu_map__has(online_cpus, i))
continue;

- if (option & ETM_OPT_CTXTID) {
+ if (option & ETM_SET_OPT_CTXTID) {
err = cs_etm_set_context_id(itr, evsel, i);
if (err)
goto out;
}
- if (option & ETM_OPT_TS) {
+ if (option & ETM_SET_OPT_TS) {
err = cs_etm_set_timestamp(itr, evsel, i);
if (err)
goto out;
}
- if (option & ~(ETM_OPT_CTXTID | ETM_OPT_TS))
+ if (option & ~(ETM_SET_OPT_MASK))
/* Nothing else is currently supported */
goto out;
}
@@ -406,7 +410,7 @@ static int cs_etm_recording_options(struct auxtrace_record *itr,
evsel__set_sample_bit(cs_etm_evsel, CPU);

err = cs_etm_set_option(itr, cs_etm_evsel,
- ETM_OPT_CTXTID | ETM_OPT_TS);
+ ETM_SET_OPT_CTXTID | ETM_SET_OPT_TS);
if (err)
goto out;
}
--
2.25.1

2021-02-06 15:13:27

by Leo Yan

[permalink] [raw]
Subject: [PATCH v3 6/8] perf cs-etm: Add helper cs_etm__get_pid_fmt()

This patch adds helper function cs_etm__get_pid_fmt(), by passing
parameter "traceID", it returns the PID format.

Signed-off-by: Leo Yan <[email protected]>
---
tools/perf/util/cs-etm.c | 42 ++++++++++++++++++++++++++++++++++++++++
tools/perf/util/cs-etm.h | 1 +
2 files changed, 43 insertions(+)

diff --git a/tools/perf/util/cs-etm.c b/tools/perf/util/cs-etm.c
index a2a369e2fbb6..b9c1d329a7f1 100644
--- a/tools/perf/util/cs-etm.c
+++ b/tools/perf/util/cs-etm.c
@@ -7,6 +7,7 @@
*/

#include <linux/bitops.h>
+#include <linux/coresight-pmu.h>
#include <linux/err.h>
#include <linux/kernel.h>
#include <linux/log2.h>
@@ -156,6 +157,47 @@ int cs_etm__get_cpu(u8 trace_chan_id, int *cpu)
return 0;
}

+/*
+ * The returned PID format is presented by two bits:
+ *
+ * Bit ETM_OPT_CTXTID: CONTEXTIDR or CONTEXTIDR_EL1 is traced;
+ * Bit ETM_OPT_CTXTID2: CONTEXTIDR_EL2 is traced.
+ *
+ * It's possible that the two bits ETM_OPT_CTXTID and ETM_OPT_CTXTID2
+ * are enabled at the same time when the session runs on an EL2 kernel.
+ * This means the CONTEXTIDR_EL1 and CONTEXTIDR_EL2 both will be
+ * recorded in the trace data, the tool will selectively use
+ * CONTEXTIDR_EL2 as PID.
+ */
+int cs_etm__get_pid_fmt(u8 trace_chan_id, u64 *pid_fmt)
+{
+ struct int_node *inode;
+ u64 *metadata, val;
+
+ inode = intlist__find(traceid_list, trace_chan_id);
+ if (!inode)
+ return -EINVAL;
+
+ metadata = inode->priv;
+
+ if (metadata[CS_ETM_MAGIC] == __perf_cs_etmv3_magic) {
+ val = metadata[CS_ETM_ETMCR];
+ /* CONTEXTIDR is traced */
+ if (val & BIT(ETM_OPT_CTXTID))
+ *pid_fmt = BIT(ETM_OPT_CTXTID);
+ } else {
+ val = metadata[CS_ETMV4_TRCCONFIGR];
+ /* CONTEXTIDR_EL2 is traced */
+ if (val & (BIT(ETM4_CFG_BIT_VMID) | BIT(ETM4_CFG_BIT_VMID_OPT)))
+ *pid_fmt = BIT(ETM_OPT_CTXTID2);
+ /* CONTEXTIDR_EL1 is traced */
+ else if (val & BIT(ETM4_CFG_BIT_CTXTID))
+ *pid_fmt = BIT(ETM_OPT_CTXTID);
+ }
+
+ return 0;
+}
+
void cs_etm__etmq_set_traceid_queue_timestamp(struct cs_etm_queue *etmq,
u8 trace_chan_id)
{
diff --git a/tools/perf/util/cs-etm.h b/tools/perf/util/cs-etm.h
index 4ad925d6d799..7cc3bba0017d 100644
--- a/tools/perf/util/cs-etm.h
+++ b/tools/perf/util/cs-etm.h
@@ -173,6 +173,7 @@ struct cs_etm_packet_queue {
int cs_etm__process_auxtrace_info(union perf_event *event,
struct perf_session *session);
int cs_etm__get_cpu(u8 trace_chan_id, int *cpu);
+int cs_etm__get_pid_fmt(u8 trace_chan_id, u64 *pid_fmt);
int cs_etm__etmq_set_tid(struct cs_etm_queue *etmq,
pid_t tid, u8 trace_chan_id);
bool cs_etm__etmq_is_timeless(struct cs_etm_queue *etmq);
--
2.25.1

2021-02-06 15:14:00

by Leo Yan

[permalink] [raw]
Subject: [PATCH v3 5/8] perf cs-etm: Support PID tracing in config

From: Suzuki K Poulose <[email protected]>

If the kernel is running at EL2, the pid of a task is exposed via VMID
instead of the CONTEXTID. Add support for this in the perf tool.

This patch respects user setting if user has specified any configs
from "contextid", "contextid1" or "contextid2"; otherwise, it
dynamically sets config based on PMU format "contextid".

Cc: Mike Leach <[email protected]>
Cc: Mathieu Poirier <[email protected]>
Cc: Al Grant <[email protected]>
Signed-off-by: Suzuki K Poulose <[email protected]>
Co-developed-by: Leo Yan <[email protected]>
Signed-off-by: Leo Yan <[email protected]>
Reviewed-by: Mike Leach <[email protected]>
---
tools/include/linux/coresight-pmu.h | 3 ++
tools/perf/arch/arm/util/cs-etm.c | 61 +++++++++++++++++++++++------
2 files changed, 52 insertions(+), 12 deletions(-)

diff --git a/tools/include/linux/coresight-pmu.h b/tools/include/linux/coresight-pmu.h
index 5dc47cfdcf07..4ac5c081af93 100644
--- a/tools/include/linux/coresight-pmu.h
+++ b/tools/include/linux/coresight-pmu.h
@@ -20,14 +20,17 @@
*/
#define ETM_OPT_CYCACC 12
#define ETM_OPT_CTXTID 14
+#define ETM_OPT_CTXTID2 15
#define ETM_OPT_TS 28
#define ETM_OPT_RETSTK 29

/* ETMv4 CONFIGR programming bits for the ETM OPTs */
#define ETM4_CFG_BIT_CYCACC 4
#define ETM4_CFG_BIT_CTXTID 6
+#define ETM4_CFG_BIT_VMID 7
#define ETM4_CFG_BIT_TS 11
#define ETM4_CFG_BIT_RETSTK 12
+#define ETM4_CFG_BIT_VMID_OPT 15

static inline int coresight_get_trace_id(int cpu)
{
diff --git a/tools/perf/arch/arm/util/cs-etm.c b/tools/perf/arch/arm/util/cs-etm.c
index c25c878fd06c..fa6f91a7c8a1 100644
--- a/tools/perf/arch/arm/util/cs-etm.c
+++ b/tools/perf/arch/arm/util/cs-etm.c
@@ -67,6 +67,7 @@ static int cs_etm_set_context_id(struct auxtrace_record *itr,
char path[PATH_MAX];
int err = -EINVAL;
u32 val;
+ u64 contextid;

ptr = container_of(itr, struct cs_etm_recording, itr);
cs_etm_pmu = ptr->cs_etm_pmu;
@@ -86,25 +87,59 @@ static int cs_etm_set_context_id(struct auxtrace_record *itr,
goto out;
}

+ /* User has configured for PID tracing, respects it. */
+ contextid = evsel->core.attr.config &
+ (BIT(ETM_OPT_CTXTID) | BIT(ETM_OPT_CTXTID2));
+
/*
- * TRCIDR2.CIDSIZE, bit [9-5], indicates whether contextID tracing
- * is supported:
- * 0b00000 Context ID tracing is not supported.
- * 0b00100 Maximum of 32-bit Context ID size.
- * All other values are reserved.
+ * If user doesn't configure the contextid format, parse PMU format and
+ * enable PID tracing according to the "contextid" format bits:
+ *
+ * If bit ETM_OPT_CTXTID is set, trace CONTEXTIDR_EL1;
+ * If bit ETM_OPT_CTXTID2 is set, trace CONTEXTIDR_EL2.
*/
- val = BMVAL(val, 5, 9);
- if (!val || val != 0x4) {
- err = -EINVAL;
- goto out;
+ if (!contextid)
+ contextid = perf_pmu__format_bits(&cs_etm_pmu->format,
+ "contextid");
+
+ if (contextid & BIT(ETM_OPT_CTXTID)) {
+ /*
+ * TRCIDR2.CIDSIZE, bit [9-5], indicates whether contextID
+ * tracing is supported:
+ * 0b00000 Context ID tracing is not supported.
+ * 0b00100 Maximum of 32-bit Context ID size.
+ * All other values are reserved.
+ */
+ val = BMVAL(val, 5, 9);
+ if (!val || val != 0x4) {
+ pr_err("%s: CONTEXTIDR_EL1 isn't supported\n",
+ CORESIGHT_ETM_PMU_NAME);
+ err = -EINVAL;
+ goto out;
+ }
+ }
+
+ if (contextid & BIT(ETM_OPT_CTXTID2)) {
+ /*
+ * TRCIDR2.VMIDOPT[30:29] != 0 and
+ * TRCIDR2.VMIDSIZE[14:10] == 0b00100 (32bit virtual contextid)
+ * We can't support CONTEXTIDR in VMID if the size of the
+ * virtual context id is < 32bit.
+ * Any value of VMIDSIZE >= 4 (i.e, > 32bit) is fine for us.
+ */
+ if (!BMVAL(val, 29, 30) || BMVAL(val, 10, 14) < 4) {
+ pr_err("%s: CONTEXTIDR_EL2 isn't supported\n",
+ CORESIGHT_ETM_PMU_NAME);
+ err = -EINVAL;
+ goto out;
+ }
}

/* All good, let the kernel know */
- evsel->core.attr.config |= (1 << ETM_OPT_CTXTID);
+ evsel->core.attr.config |= contextid;
err = 0;

out:
-
return err;
}

@@ -489,7 +524,9 @@ static u64 cs_etmv4_get_config(struct auxtrace_record *itr)
config |= BIT(ETM4_CFG_BIT_TS);
if (config_opts & BIT(ETM_OPT_RETSTK))
config |= BIT(ETM4_CFG_BIT_RETSTK);
-
+ if (config_opts & BIT(ETM_OPT_CTXTID2))
+ config |= BIT(ETM4_CFG_BIT_VMID) |
+ BIT(ETM4_CFG_BIT_VMID_OPT);
return config;
}

--
2.25.1

2021-02-06 15:14:19

by Leo Yan

[permalink] [raw]
Subject: [PATCH v3 3/8] coresight: etm-perf: Support PID tracing for kernel at EL2

From: Suzuki K Poulose <[email protected]>

When the kernel is running at EL2, the PID is stored in CONTEXTIDR_EL2.
So, tracing CONTEXTIDR_EL1 doesn't give us the pid of the process.
Thus we should trace the VMID with VMIDOPT set to trace CONTEXTIDR_EL2
instead of CONTEXTIDR_EL1. Given that we have an existing config
option "contextid" and this will be useful for tracing virtual machines
(when we get to support virtualization).

So instead, this patch extends option CTXTID with an extra bit
ETM_OPT_CTXTID2 (bit 15), thus on an EL2 kernel, we will have another
bit available for the perf tool: ETM_OPT_CTXTID is for kernel running in
EL1, ETM_OPT_CTXTID2 is used when kernel runs in EL2 with VHE enabled.

The tool must be backward compatible for users, i.e, "contextid" today
traces PID and that should remain the same; for this purpose, the perf
tool is updated to automatically set corresponding bit for the
"contextid" config, therefore, the user doesn't have to bother which EL
the kernel is running.

i.e, perf record -e cs_etm/contextid/u --

will always do the "pid" tracing, independent of the kernel EL.

The driver parses the format "contextid", which traces CONTEXTIDR_EL1
for ETM_OPT_CTXTID (on EL1 kernel) and traces CONTEXTIDR_EL2 for
ETM_OPT_CTXTID2 (on EL2 kernel).

Besides the enhancement for format "contexid", extra two formats are
introduced: "contextid1" and "contextid2". This considers to support
tracing both CONTEXTIDR_EL1 and CONTEXTIDR_EL2 when the kernel is
running at EL2. Finally, the PMU formats are defined as follow:

"contextid1": Available on both EL1 kernel and EL2 kernel. When the
kernel is running at EL1, "contextid1" enables the PID
tracing; when the kernel is running at EL2, this enables
tracing the PID of guest applications.

"contextid2": Only usable when the kernel is running at EL2. When
selected, enables PID tracing on EL2 kernel.

"contextid": Will be an alias for the option that enables PID
tracing. I.e,
contextid == contextid1, on EL1 kernel.
contextid == contextid2, on EL2 kernel.

Cc: Mathieu Poirier <[email protected]>
Cc: Al Grant <[email protected]>
Cc: Mike Leach <[email protected]>
Cc: Leo Yan <[email protected]>
Signed-off-by: Suzuki K Poulose <[email protected]>
[ Added two config formats: contextid1, contextid2 ]
Signed-off-by: Leo Yan <[email protected]>
Reviewed-by: Mike Leach <[email protected]>
---
.../hwtracing/coresight/coresight-etm-perf.c | 27 ++++++++++++++++++-
.../coresight/coresight-etm4x-core.c | 13 +++++++++
include/linux/coresight-pmu.h | 3 +++
3 files changed, 42 insertions(+), 1 deletion(-)

diff --git a/drivers/hwtracing/coresight/coresight-etm-perf.c b/drivers/hwtracing/coresight/coresight-etm-perf.c
index 465ef1aa8c82..0f603b4094f2 100644
--- a/drivers/hwtracing/coresight/coresight-etm-perf.c
+++ b/drivers/hwtracing/coresight/coresight-etm-perf.c
@@ -32,15 +32,40 @@ static DEFINE_PER_CPU(struct coresight_device *, csdev_src);
* now take them as general formats and apply on all ETMs.
*/
PMU_FORMAT_ATTR(cycacc, "config:" __stringify(ETM_OPT_CYCACC));
-PMU_FORMAT_ATTR(contextid, "config:" __stringify(ETM_OPT_CTXTID));
+/* contextid1 enables tracing CONTEXTIDR_EL1 for ETMv4 */
+PMU_FORMAT_ATTR(contextid1, "config:" __stringify(ETM_OPT_CTXTID));
+/* contextid2 enables tracing CONTEXTIDR_EL2 for ETMv4 */
+PMU_FORMAT_ATTR(contextid2, "config:" __stringify(ETM_OPT_CTXTID2));
PMU_FORMAT_ATTR(timestamp, "config:" __stringify(ETM_OPT_TS));
PMU_FORMAT_ATTR(retstack, "config:" __stringify(ETM_OPT_RETSTK));
/* Sink ID - same for all ETMs */
PMU_FORMAT_ATTR(sinkid, "config2:0-31");

+/*
+ * contextid always traces the "PID". The PID is in CONTEXTIDR_EL1
+ * when the kernel is running at EL1; when the kernel is at EL2,
+ * the PID is in CONTEXTIDR_EL2.
+ */
+static ssize_t format_attr_contextid_show(struct device *dev,
+ struct device_attribute *attr,
+ char *page)
+{
+ int pid_fmt = ETM_OPT_CTXTID;
+
+#if defined(CONFIG_CORESIGHT_SOURCE_ETM4X)
+ pid_fmt = is_kernel_in_hyp_mode() ? ETM_OPT_CTXTID2 : ETM_OPT_CTXTID;
+#endif
+ return sprintf(page, "config:%d\n", pid_fmt);
+}
+
+struct device_attribute format_attr_contextid =
+ __ATTR(contextid, 0444, format_attr_contextid_show, NULL);
+
static struct attribute *etm_config_formats_attr[] = {
&format_attr_cycacc.attr,
&format_attr_contextid.attr,
+ &format_attr_contextid1.attr,
+ &format_attr_contextid2.attr,
&format_attr_timestamp.attr,
&format_attr_retstack.attr,
&format_attr_sinkid.attr,
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c
index b20b6ff17cf6..8681c225b0ba 100644
--- a/drivers/hwtracing/coresight/coresight-etm4x-core.c
+++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c
@@ -477,6 +477,19 @@ static int etm4_parse_event_config(struct etmv4_drvdata *drvdata,
/* bit[6], Context ID tracing bit */
config->cfg |= BIT(ETM4_CFG_BIT_CTXTID);

+ /*
+ * If set bit ETM_OPT_CTXTID2 in perf config, this asks to trace VMID
+ * for recording CONTEXTIDR_EL2. Do not enable VMID tracing if the
+ * kernel is not running in EL2.
+ */
+ if (attr->config & BIT(ETM_OPT_CTXTID2)) {
+ if (!is_kernel_in_hyp_mode()) {
+ ret = -EINVAL;
+ goto out;
+ }
+ config->cfg |= BIT(ETM4_CFG_BIT_VMID) | BIT(ETM4_CFG_BIT_VMID_OPT);
+ }
+
/* return stack - enable if selected and supported */
if ((attr->config & BIT(ETM_OPT_RETSTK)) && drvdata->retstack)
/* bit[12], Return stack enable bit */
diff --git a/include/linux/coresight-pmu.h b/include/linux/coresight-pmu.h
index 5dc47cfdcf07..4ac5c081af93 100644
--- a/include/linux/coresight-pmu.h
+++ b/include/linux/coresight-pmu.h
@@ -20,14 +20,17 @@
*/
#define ETM_OPT_CYCACC 12
#define ETM_OPT_CTXTID 14
+#define ETM_OPT_CTXTID2 15
#define ETM_OPT_TS 28
#define ETM_OPT_RETSTK 29

/* ETMv4 CONFIGR programming bits for the ETM OPTs */
#define ETM4_CFG_BIT_CYCACC 4
#define ETM4_CFG_BIT_CTXTID 6
+#define ETM4_CFG_BIT_VMID 7
#define ETM4_CFG_BIT_TS 11
#define ETM4_CFG_BIT_RETSTK 12
+#define ETM4_CFG_BIT_VMID_OPT 15

static inline int coresight_get_trace_id(int cpu)
{
--
2.25.1

2021-02-06 15:14:49

by Leo Yan

[permalink] [raw]
Subject: [PATCH v3 8/8] Documentation: coresight: Add PID tracing description

After support the PID tracing for the kernel in EL1 or EL2, the usage
gets more complicated.

This patch gives description for the PMU formats of contextID configs,
this can help users to understand how to control the knobs for PID
tracing when the kernel is in different ELs.

Signed-off-by: Leo Yan <[email protected]>
---
Documentation/trace/coresight/coresight.rst | 32 +++++++++++++++++++++
1 file changed, 32 insertions(+)

diff --git a/Documentation/trace/coresight/coresight.rst b/Documentation/trace/coresight/coresight.rst
index 0b73acb44efa..169749efd8d1 100644
--- a/Documentation/trace/coresight/coresight.rst
+++ b/Documentation/trace/coresight/coresight.rst
@@ -512,6 +512,38 @@ The --itrace option controls the type and frequency of synthesized events
Note that only 64-bit programs are currently supported - further work is
required to support instruction decode of 32-bit Arm programs.

+2.2) Tracing PID
+
+The kernel can be built to write the PID value into the PE ContextID registers.
+For a kernel running at EL1, the PID is stored in CONTEXTIDR_EL1. A PE may
+implement Arm Virtualization Host Extensions (VHE), which the kernel can
+run at EL2 as a virtualisation host; in this case, the PID value is stored in
+CONTEXTIDR_EL2.
+
+perf provides PMU formats that program the ETM to insert these values into the
+trace data; the PMU formats are defined as below:
+
+ "contextid1": Available on both EL1 kernel and EL2 kernel. When the
+ kernel is running at EL1, "contextid1" enables the PID
+ tracing; when the kernel is running at EL2, this enables
+ tracing the PID of guest applications.
+
+ "contextid2": Only usable when the kernel is running at EL2. When
+ selected, enables PID tracing on EL2 kernel.
+
+ "contextid": Will be an alias for the option that enables PID
+ tracing. I.e,
+ contextid == contextid1, on EL1 kernel.
+ contextid == contextid2, on EL2 kernel.
+
+perf will always enable PID tracing at the relevant EL, this is accomplished by
+automatically enable the "contextid" config - but for EL2 it is possible to make
+specific adjustments using configs "contextid1" and "contextid2", E.g. if a user
+wants to trace PIDs for both host and guest, the two configs "contextid1" and
+"contextid2" can be set at the same time:
+
+ perf record -e cs_etm/contextid1,contextid2/u -- vm
+

Generating coverage files for Feedback Directed Optimization: AutoFDO
---------------------------------------------------------------------
--
2.25.1

2021-02-06 16:25:04

by Leo Yan

[permalink] [raw]
Subject: [PATCH v3 2/8] tools headers UAPI: Update tools' copy of linux/coresight-pmu.h

To get the changes in the commit:

"coresight: etm-perf: Clarify comment on perf options".

Signed-off-by: Leo Yan <[email protected]>
Reviewed-by: Suzuki K Poulose <[email protected]>
---
tools/include/linux/coresight-pmu.h | 17 ++++++++++++-----
1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/tools/include/linux/coresight-pmu.h b/tools/include/linux/coresight-pmu.h
index b0e35eec6499..5dc47cfdcf07 100644
--- a/tools/include/linux/coresight-pmu.h
+++ b/tools/include/linux/coresight-pmu.h
@@ -10,11 +10,18 @@
#define CORESIGHT_ETM_PMU_NAME "cs_etm"
#define CORESIGHT_ETM_PMU_SEED 0x10

-/* ETMv3.5/PTM's ETMCR config bit */
-#define ETM_OPT_CYCACC 12
-#define ETM_OPT_CTXTID 14
-#define ETM_OPT_TS 28
-#define ETM_OPT_RETSTK 29
+/*
+ * Below are the definition of bit offsets for perf option, and works as
+ * arbitrary values for all ETM versions.
+ *
+ * Most of them are orignally from ETMv3.5/PTM's ETMCR config, therefore,
+ * ETMv3.5/PTM doesn't define ETMCR config bits with prefix "ETM3_" and
+ * directly use below macros as config bits.
+ */
+#define ETM_OPT_CYCACC 12
+#define ETM_OPT_CTXTID 14
+#define ETM_OPT_TS 28
+#define ETM_OPT_RETSTK 29

/* ETMv4 CONFIGR programming bits for the ETM OPTs */
#define ETM4_CFG_BIT_CYCACC 4
--
2.25.1

2021-02-08 19:05:02

by Mathieu Poirier

[permalink] [raw]
Subject: Re: [PATCH v3 2/8] tools headers UAPI: Update tools' copy of linux/coresight-pmu.h

On Sat, Feb 06, 2021 at 11:08:27PM +0800, Leo Yan wrote:
> To get the changes in the commit:
>
> "coresight: etm-perf: Clarify comment on perf options".
>
> Signed-off-by: Leo Yan <[email protected]>
> Reviewed-by: Suzuki K Poulose <[email protected]>

Reviewed-by: Mathieu Poirier <[email protected]>

> ---
> tools/include/linux/coresight-pmu.h | 17 ++++++++++++-----
> 1 file changed, 12 insertions(+), 5 deletions(-)
>
> diff --git a/tools/include/linux/coresight-pmu.h b/tools/include/linux/coresight-pmu.h
> index b0e35eec6499..5dc47cfdcf07 100644
> --- a/tools/include/linux/coresight-pmu.h
> +++ b/tools/include/linux/coresight-pmu.h
> @@ -10,11 +10,18 @@
> #define CORESIGHT_ETM_PMU_NAME "cs_etm"
> #define CORESIGHT_ETM_PMU_SEED 0x10
>
> -/* ETMv3.5/PTM's ETMCR config bit */
> -#define ETM_OPT_CYCACC 12
> -#define ETM_OPT_CTXTID 14
> -#define ETM_OPT_TS 28
> -#define ETM_OPT_RETSTK 29
> +/*
> + * Below are the definition of bit offsets for perf option, and works as
> + * arbitrary values for all ETM versions.
> + *
> + * Most of them are orignally from ETMv3.5/PTM's ETMCR config, therefore,
> + * ETMv3.5/PTM doesn't define ETMCR config bits with prefix "ETM3_" and
> + * directly use below macros as config bits.
> + */
> +#define ETM_OPT_CYCACC 12
> +#define ETM_OPT_CTXTID 14
> +#define ETM_OPT_TS 28
> +#define ETM_OPT_RETSTK 29
>
> /* ETMv4 CONFIGR programming bits for the ETM OPTs */
> #define ETM4_CFG_BIT_CYCACC 4
> --
> 2.25.1
>

2021-02-08 20:23:51

by Mathieu Poirier

[permalink] [raw]
Subject: Re: [PATCH v3 5/8] perf cs-etm: Support PID tracing in config

On Sat, Feb 06, 2021 at 11:08:30PM +0800, Leo Yan wrote:
> From: Suzuki K Poulose <[email protected]>
>
> If the kernel is running at EL2, the pid of a task is exposed via VMID
> instead of the CONTEXTID. Add support for this in the perf tool.
>
> This patch respects user setting if user has specified any configs
> from "contextid", "contextid1" or "contextid2"; otherwise, it
> dynamically sets config based on PMU format "contextid".
>
> Cc: Mike Leach <[email protected]>
> Cc: Mathieu Poirier <[email protected]>
> Cc: Al Grant <[email protected]>
> Signed-off-by: Suzuki K Poulose <[email protected]>
> Co-developed-by: Leo Yan <[email protected]>
> Signed-off-by: Leo Yan <[email protected]>
> Reviewed-by: Mike Leach <[email protected]>

Reviewed-by: Mathieu Poirier <[email protected]>

> ---
> tools/include/linux/coresight-pmu.h | 3 ++
> tools/perf/arch/arm/util/cs-etm.c | 61 +++++++++++++++++++++++------
> 2 files changed, 52 insertions(+), 12 deletions(-)
>
> diff --git a/tools/include/linux/coresight-pmu.h b/tools/include/linux/coresight-pmu.h
> index 5dc47cfdcf07..4ac5c081af93 100644
> --- a/tools/include/linux/coresight-pmu.h
> +++ b/tools/include/linux/coresight-pmu.h
> @@ -20,14 +20,17 @@
> */
> #define ETM_OPT_CYCACC 12
> #define ETM_OPT_CTXTID 14
> +#define ETM_OPT_CTXTID2 15
> #define ETM_OPT_TS 28
> #define ETM_OPT_RETSTK 29
>
> /* ETMv4 CONFIGR programming bits for the ETM OPTs */
> #define ETM4_CFG_BIT_CYCACC 4
> #define ETM4_CFG_BIT_CTXTID 6
> +#define ETM4_CFG_BIT_VMID 7
> #define ETM4_CFG_BIT_TS 11
> #define ETM4_CFG_BIT_RETSTK 12
> +#define ETM4_CFG_BIT_VMID_OPT 15
>
> static inline int coresight_get_trace_id(int cpu)
> {
> diff --git a/tools/perf/arch/arm/util/cs-etm.c b/tools/perf/arch/arm/util/cs-etm.c
> index c25c878fd06c..fa6f91a7c8a1 100644
> --- a/tools/perf/arch/arm/util/cs-etm.c
> +++ b/tools/perf/arch/arm/util/cs-etm.c
> @@ -67,6 +67,7 @@ static int cs_etm_set_context_id(struct auxtrace_record *itr,
> char path[PATH_MAX];
> int err = -EINVAL;
> u32 val;
> + u64 contextid;
>
> ptr = container_of(itr, struct cs_etm_recording, itr);
> cs_etm_pmu = ptr->cs_etm_pmu;
> @@ -86,25 +87,59 @@ static int cs_etm_set_context_id(struct auxtrace_record *itr,
> goto out;
> }
>
> + /* User has configured for PID tracing, respects it. */
> + contextid = evsel->core.attr.config &
> + (BIT(ETM_OPT_CTXTID) | BIT(ETM_OPT_CTXTID2));
> +
> /*
> - * TRCIDR2.CIDSIZE, bit [9-5], indicates whether contextID tracing
> - * is supported:
> - * 0b00000 Context ID tracing is not supported.
> - * 0b00100 Maximum of 32-bit Context ID size.
> - * All other values are reserved.
> + * If user doesn't configure the contextid format, parse PMU format and
> + * enable PID tracing according to the "contextid" format bits:
> + *
> + * If bit ETM_OPT_CTXTID is set, trace CONTEXTIDR_EL1;
> + * If bit ETM_OPT_CTXTID2 is set, trace CONTEXTIDR_EL2.
> */
> - val = BMVAL(val, 5, 9);
> - if (!val || val != 0x4) {
> - err = -EINVAL;
> - goto out;
> + if (!contextid)
> + contextid = perf_pmu__format_bits(&cs_etm_pmu->format,
> + "contextid");
> +
> + if (contextid & BIT(ETM_OPT_CTXTID)) {
> + /*
> + * TRCIDR2.CIDSIZE, bit [9-5], indicates whether contextID
> + * tracing is supported:
> + * 0b00000 Context ID tracing is not supported.
> + * 0b00100 Maximum of 32-bit Context ID size.
> + * All other values are reserved.
> + */
> + val = BMVAL(val, 5, 9);
> + if (!val || val != 0x4) {
> + pr_err("%s: CONTEXTIDR_EL1 isn't supported\n",
> + CORESIGHT_ETM_PMU_NAME);
> + err = -EINVAL;
> + goto out;
> + }
> + }
> +
> + if (contextid & BIT(ETM_OPT_CTXTID2)) {
> + /*
> + * TRCIDR2.VMIDOPT[30:29] != 0 and
> + * TRCIDR2.VMIDSIZE[14:10] == 0b00100 (32bit virtual contextid)
> + * We can't support CONTEXTIDR in VMID if the size of the
> + * virtual context id is < 32bit.
> + * Any value of VMIDSIZE >= 4 (i.e, > 32bit) is fine for us.
> + */
> + if (!BMVAL(val, 29, 30) || BMVAL(val, 10, 14) < 4) {
> + pr_err("%s: CONTEXTIDR_EL2 isn't supported\n",
> + CORESIGHT_ETM_PMU_NAME);
> + err = -EINVAL;
> + goto out;
> + }
> }
>
> /* All good, let the kernel know */
> - evsel->core.attr.config |= (1 << ETM_OPT_CTXTID);
> + evsel->core.attr.config |= contextid;
> err = 0;
>
> out:
> -
> return err;
> }
>
> @@ -489,7 +524,9 @@ static u64 cs_etmv4_get_config(struct auxtrace_record *itr)
> config |= BIT(ETM4_CFG_BIT_TS);
> if (config_opts & BIT(ETM_OPT_RETSTK))
> config |= BIT(ETM4_CFG_BIT_RETSTK);
> -
> + if (config_opts & BIT(ETM_OPT_CTXTID2))
> + config |= BIT(ETM4_CFG_BIT_VMID) |
> + BIT(ETM4_CFG_BIT_VMID_OPT);
> return config;
> }
>
> --
> 2.25.1
>

2021-02-08 21:30:36

by Mathieu Poirier

[permalink] [raw]
Subject: Re: [PATCH v3 6/8] perf cs-etm: Add helper cs_etm__get_pid_fmt()

On Sat, Feb 06, 2021 at 11:08:31PM +0800, Leo Yan wrote:
> This patch adds helper function cs_etm__get_pid_fmt(), by passing
> parameter "traceID", it returns the PID format.
>
> Signed-off-by: Leo Yan <[email protected]>

Reviewed-by: Mathieu Poirier <[email protected]>

> ---
> tools/perf/util/cs-etm.c | 42 ++++++++++++++++++++++++++++++++++++++++
> tools/perf/util/cs-etm.h | 1 +
> 2 files changed, 43 insertions(+)
>
> diff --git a/tools/perf/util/cs-etm.c b/tools/perf/util/cs-etm.c
> index a2a369e2fbb6..b9c1d329a7f1 100644
> --- a/tools/perf/util/cs-etm.c
> +++ b/tools/perf/util/cs-etm.c
> @@ -7,6 +7,7 @@
> */
>
> #include <linux/bitops.h>
> +#include <linux/coresight-pmu.h>
> #include <linux/err.h>
> #include <linux/kernel.h>
> #include <linux/log2.h>
> @@ -156,6 +157,47 @@ int cs_etm__get_cpu(u8 trace_chan_id, int *cpu)
> return 0;
> }
>
> +/*
> + * The returned PID format is presented by two bits:
> + *
> + * Bit ETM_OPT_CTXTID: CONTEXTIDR or CONTEXTIDR_EL1 is traced;
> + * Bit ETM_OPT_CTXTID2: CONTEXTIDR_EL2 is traced.
> + *
> + * It's possible that the two bits ETM_OPT_CTXTID and ETM_OPT_CTXTID2
> + * are enabled at the same time when the session runs on an EL2 kernel.
> + * This means the CONTEXTIDR_EL1 and CONTEXTIDR_EL2 both will be
> + * recorded in the trace data, the tool will selectively use
> + * CONTEXTIDR_EL2 as PID.
> + */
> +int cs_etm__get_pid_fmt(u8 trace_chan_id, u64 *pid_fmt)
> +{
> + struct int_node *inode;
> + u64 *metadata, val;
> +
> + inode = intlist__find(traceid_list, trace_chan_id);
> + if (!inode)
> + return -EINVAL;
> +
> + metadata = inode->priv;
> +
> + if (metadata[CS_ETM_MAGIC] == __perf_cs_etmv3_magic) {
> + val = metadata[CS_ETM_ETMCR];
> + /* CONTEXTIDR is traced */
> + if (val & BIT(ETM_OPT_CTXTID))
> + *pid_fmt = BIT(ETM_OPT_CTXTID);
> + } else {
> + val = metadata[CS_ETMV4_TRCCONFIGR];
> + /* CONTEXTIDR_EL2 is traced */
> + if (val & (BIT(ETM4_CFG_BIT_VMID) | BIT(ETM4_CFG_BIT_VMID_OPT)))
> + *pid_fmt = BIT(ETM_OPT_CTXTID2);
> + /* CONTEXTIDR_EL1 is traced */
> + else if (val & BIT(ETM4_CFG_BIT_CTXTID))
> + *pid_fmt = BIT(ETM_OPT_CTXTID);
> + }
> +
> + return 0;
> +}
> +
> void cs_etm__etmq_set_traceid_queue_timestamp(struct cs_etm_queue *etmq,
> u8 trace_chan_id)
> {
> diff --git a/tools/perf/util/cs-etm.h b/tools/perf/util/cs-etm.h
> index 4ad925d6d799..7cc3bba0017d 100644
> --- a/tools/perf/util/cs-etm.h
> +++ b/tools/perf/util/cs-etm.h
> @@ -173,6 +173,7 @@ struct cs_etm_packet_queue {
> int cs_etm__process_auxtrace_info(union perf_event *event,
> struct perf_session *session);
> int cs_etm__get_cpu(u8 trace_chan_id, int *cpu);
> +int cs_etm__get_pid_fmt(u8 trace_chan_id, u64 *pid_fmt);
> int cs_etm__etmq_set_tid(struct cs_etm_queue *etmq,
> pid_t tid, u8 trace_chan_id);
> bool cs_etm__etmq_is_timeless(struct cs_etm_queue *etmq);
> --
> 2.25.1
>

2021-02-08 21:37:31

by Mathieu Poirier

[permalink] [raw]
Subject: Re: [PATCH v3 4/8] perf cs-etm: Fix bitmap for option

On Sat, Feb 06, 2021 at 11:08:29PM +0800, Leo Yan wrote:
> From: Suzuki K Poulose <[email protected]>
>
> When set option with macros ETM_OPT_CTXTID and ETM_OPT_TS, it wrongly
> takes these two values (14 and 28 prespectively) as bit masks, but
> actually both are the offset for bits. But this doesn't lead to
> further failure due to the AND logic operation will be always true for
> ETM_OPT_CTXTID / ETM_OPT_TS.
>
> This patch defines new independent macros (rather than using the
> "config" bits) for requesting the "contextid" and "timestamp" for
> cs_etm_set_option().
>
> [leoy: Extract the change as a separate patch for easier review]

This should go just above your name - see below.

> Signed-off-by: Suzuki K Poulose <[email protected]>
> Signed-off-by: Leo Yan <[email protected]>
> Reviewed-by: Mike Leach <[email protected]>

Signed-off-by: Suzuki K Poulose <[email protected]>
[Extract the change as a separate patch for easier review]
Signed-off-by: Leo Yan <[email protected]>
Reviewed-by: Mike Leach <[email protected]>

> ---
> tools/perf/arch/arm/util/cs-etm.c | 12 ++++++++----
> 1 file changed, 8 insertions(+), 4 deletions(-)
>
> diff --git a/tools/perf/arch/arm/util/cs-etm.c b/tools/perf/arch/arm/util/cs-etm.c
> index bd446aba64f7..c25c878fd06c 100644
> --- a/tools/perf/arch/arm/util/cs-etm.c
> +++ b/tools/perf/arch/arm/util/cs-etm.c
> @@ -156,6 +156,10 @@ static int cs_etm_set_timestamp(struct auxtrace_record *itr,
> return err;
> }
>
> +#define ETM_SET_OPT_CTXTID (1 << 0)
> +#define ETM_SET_OPT_TS (1 << 1)
> +#define ETM_SET_OPT_MASK (ETM_SET_OPT_CTXTID | ETM_SET_OPT_TS)
> +

I would much rather see this fixed with the BIT() macro as it is done in the
rest of this set than defining new constant.

With the above:

Reviewed-by: Mathieu Poirier <[email protected]>

I have picked up the kernel portion of this set. I suggest you fix the above
and send another revision to Arnaldo with my RBs.

> static int cs_etm_set_option(struct auxtrace_record *itr,
> struct evsel *evsel, u32 option)
> {
> @@ -169,17 +173,17 @@ static int cs_etm_set_option(struct auxtrace_record *itr,
> !cpu_map__has(online_cpus, i))
> continue;
>
> - if (option & ETM_OPT_CTXTID) {
> + if (option & ETM_SET_OPT_CTXTID) {
> err = cs_etm_set_context_id(itr, evsel, i);
> if (err)
> goto out;
> }
> - if (option & ETM_OPT_TS) {
> + if (option & ETM_SET_OPT_TS) {
> err = cs_etm_set_timestamp(itr, evsel, i);
> if (err)
> goto out;
> }
> - if (option & ~(ETM_OPT_CTXTID | ETM_OPT_TS))
> + if (option & ~(ETM_SET_OPT_MASK))
> /* Nothing else is currently supported */
> goto out;
> }
> @@ -406,7 +410,7 @@ static int cs_etm_recording_options(struct auxtrace_record *itr,
> evsel__set_sample_bit(cs_etm_evsel, CPU);
>
> err = cs_etm_set_option(itr, cs_etm_evsel,
> - ETM_OPT_CTXTID | ETM_OPT_TS);
> + ETM_SET_OPT_CTXTID | ETM_SET_OPT_TS);
> if (err)
> goto out;
> }
> --
> 2.25.1
>

2021-02-08 21:40:57

by Mathieu Poirier

[permalink] [raw]
Subject: Re: [PATCH v3 8/8] Documentation: coresight: Add PID tracing description

On Sat, Feb 06, 2021 at 11:08:33PM +0800, Leo Yan wrote:
> After support the PID tracing for the kernel in EL1 or EL2, the usage
> gets more complicated.
>
> This patch gives description for the PMU formats of contextID configs,
> this can help users to understand how to control the knobs for PID
> tracing when the kernel is in different ELs.
>
> Signed-off-by: Leo Yan <[email protected]>

As I indicated I have picked up the kernel patches in this set and there should
be a new patchset sent to Arnaldo. The only thing left is this patch and I will
give time to Mike and Suzuki to look at it before I add it to my tree.

Thanks,
Mathieu

> ---
> Documentation/trace/coresight/coresight.rst | 32 +++++++++++++++++++++
> 1 file changed, 32 insertions(+)
>
> diff --git a/Documentation/trace/coresight/coresight.rst b/Documentation/trace/coresight/coresight.rst
> index 0b73acb44efa..169749efd8d1 100644
> --- a/Documentation/trace/coresight/coresight.rst
> +++ b/Documentation/trace/coresight/coresight.rst
> @@ -512,6 +512,38 @@ The --itrace option controls the type and frequency of synthesized events
> Note that only 64-bit programs are currently supported - further work is
> required to support instruction decode of 32-bit Arm programs.
>
> +2.2) Tracing PID
> +
> +The kernel can be built to write the PID value into the PE ContextID registers.
> +For a kernel running at EL1, the PID is stored in CONTEXTIDR_EL1. A PE may
> +implement Arm Virtualization Host Extensions (VHE), which the kernel can
> +run at EL2 as a virtualisation host; in this case, the PID value is stored in
> +CONTEXTIDR_EL2.
> +
> +perf provides PMU formats that program the ETM to insert these values into the
> +trace data; the PMU formats are defined as below:
> +
> + "contextid1": Available on both EL1 kernel and EL2 kernel. When the
> + kernel is running at EL1, "contextid1" enables the PID
> + tracing; when the kernel is running at EL2, this enables
> + tracing the PID of guest applications.
> +
> + "contextid2": Only usable when the kernel is running at EL2. When
> + selected, enables PID tracing on EL2 kernel.
> +
> + "contextid": Will be an alias for the option that enables PID
> + tracing. I.e,
> + contextid == contextid1, on EL1 kernel.
> + contextid == contextid2, on EL2 kernel.
> +
> +perf will always enable PID tracing at the relevant EL, this is accomplished by
> +automatically enable the "contextid" config - but for EL2 it is possible to make
> +specific adjustments using configs "contextid1" and "contextid2", E.g. if a user
> +wants to trace PIDs for both host and guest, the two configs "contextid1" and
> +"contextid2" can be set at the same time:
> +
> + perf record -e cs_etm/contextid1,contextid2/u -- vm
> +
>
> Generating coverage files for Feedback Directed Optimization: AutoFDO
> ---------------------------------------------------------------------
> --
> 2.25.1
>

2021-02-08 21:47:51

by Mike Leach

[permalink] [raw]
Subject: Re: [PATCH v3 8/8] Documentation: coresight: Add PID tracing description

I think Suzuki may be on holiday for two weeks. But this does appear
to pick up both mine and his suggestions.
Suzuki gave his conditional reviewed by in the last patch.

Reviewed-by: Mike Leach <[email protected]>


On Mon, 8 Feb 2021 at 20:50, Mathieu Poirier <[email protected]> wrote:
>
> On Sat, Feb 06, 2021 at 11:08:33PM +0800, Leo Yan wrote:
> > After support the PID tracing for the kernel in EL1 or EL2, the usage
> > gets more complicated.
> >
> > This patch gives description for the PMU formats of contextID configs,
> > this can help users to understand how to control the knobs for PID
> > tracing when the kernel is in different ELs.
> >
> > Signed-off-by: Leo Yan <[email protected]>
>
> As I indicated I have picked up the kernel patches in this set and there should
> be a new patchset sent to Arnaldo. The only thing left is this patch and I will
> give time to Mike and Suzuki to look at it before I add it to my tree.
>
> Thanks,
> Mathieu
>
> > ---
> > Documentation/trace/coresight/coresight.rst | 32 +++++++++++++++++++++
> > 1 file changed, 32 insertions(+)
> >
> > diff --git a/Documentation/trace/coresight/coresight.rst b/Documentation/trace/coresight/coresight.rst
> > index 0b73acb44efa..169749efd8d1 100644
> > --- a/Documentation/trace/coresight/coresight.rst
> > +++ b/Documentation/trace/coresight/coresight.rst
> > @@ -512,6 +512,38 @@ The --itrace option controls the type and frequency of synthesized events
> > Note that only 64-bit programs are currently supported - further work is
> > required to support instruction decode of 32-bit Arm programs.
> >
> > +2.2) Tracing PID
> > +
> > +The kernel can be built to write the PID value into the PE ContextID registers.
> > +For a kernel running at EL1, the PID is stored in CONTEXTIDR_EL1. A PE may
> > +implement Arm Virtualization Host Extensions (VHE), which the kernel can
> > +run at EL2 as a virtualisation host; in this case, the PID value is stored in
> > +CONTEXTIDR_EL2.
> > +
> > +perf provides PMU formats that program the ETM to insert these values into the
> > +trace data; the PMU formats are defined as below:
> > +
> > + "contextid1": Available on both EL1 kernel and EL2 kernel. When the
> > + kernel is running at EL1, "contextid1" enables the PID
> > + tracing; when the kernel is running at EL2, this enables
> > + tracing the PID of guest applications.
> > +
> > + "contextid2": Only usable when the kernel is running at EL2. When
> > + selected, enables PID tracing on EL2 kernel.
> > +
> > + "contextid": Will be an alias for the option that enables PID
> > + tracing. I.e,
> > + contextid == contextid1, on EL1 kernel.
> > + contextid == contextid2, on EL2 kernel.
> > +
> > +perf will always enable PID tracing at the relevant EL, this is accomplished by
> > +automatically enable the "contextid" config - but for EL2 it is possible to make
> > +specific adjustments using configs "contextid1" and "contextid2", E.g. if a user
> > +wants to trace PIDs for both host and guest, the two configs "contextid1" and
> > +"contextid2" can be set at the same time:
> > +
> > + perf record -e cs_etm/contextid1,contextid2/u -- vm
> > +
> >
> > Generating coverage files for Feedback Directed Optimization: AutoFDO
> > ---------------------------------------------------------------------
> > --
> > 2.25.1
> >



--
Mike Leach
Principal Engineer, ARM Ltd.
Manchester Design Centre. UK

2021-02-09 02:03:06

by Leo Yan

[permalink] [raw]
Subject: Re: [PATCH v3 4/8] perf cs-etm: Fix bitmap for option

On Mon, Feb 08, 2021 at 01:46:41PM -0700, Mathieu Poirier wrote:
> On Sat, Feb 06, 2021 at 11:08:29PM +0800, Leo Yan wrote:
> > From: Suzuki K Poulose <[email protected]>
> >
> > When set option with macros ETM_OPT_CTXTID and ETM_OPT_TS, it wrongly
> > takes these two values (14 and 28 prespectively) as bit masks, but
> > actually both are the offset for bits. But this doesn't lead to
> > further failure due to the AND logic operation will be always true for
> > ETM_OPT_CTXTID / ETM_OPT_TS.
> >
> > This patch defines new independent macros (rather than using the
> > "config" bits) for requesting the "contextid" and "timestamp" for
> > cs_etm_set_option().
> >
> > [leoy: Extract the change as a separate patch for easier review]
>
> This should go just above your name - see below.
>
> > Signed-off-by: Suzuki K Poulose <[email protected]>
> > Signed-off-by: Leo Yan <[email protected]>
> > Reviewed-by: Mike Leach <[email protected]>
>
> Signed-off-by: Suzuki K Poulose <[email protected]>
> [Extract the change as a separate patch for easier review]
> Signed-off-by: Leo Yan <[email protected]>
> Reviewed-by: Mike Leach <[email protected]>
>
> > ---
> > tools/perf/arch/arm/util/cs-etm.c | 12 ++++++++----
> > 1 file changed, 8 insertions(+), 4 deletions(-)
> >
> > diff --git a/tools/perf/arch/arm/util/cs-etm.c b/tools/perf/arch/arm/util/cs-etm.c
> > index bd446aba64f7..c25c878fd06c 100644
> > --- a/tools/perf/arch/arm/util/cs-etm.c
> > +++ b/tools/perf/arch/arm/util/cs-etm.c
> > @@ -156,6 +156,10 @@ static int cs_etm_set_timestamp(struct auxtrace_record *itr,
> > return err;
> > }
> >
> > +#define ETM_SET_OPT_CTXTID (1 << 0)
> > +#define ETM_SET_OPT_TS (1 << 1)
> > +#define ETM_SET_OPT_MASK (ETM_SET_OPT_CTXTID | ETM_SET_OPT_TS)
> > +
>
> I would much rather see this fixed with the BIT() macro as it is done in the
> rest of this set than defining new constant.
>
> With the above:
>
> Reviewed-by: Mathieu Poirier <[email protected]>
>
> I have picked up the kernel portion of this set. I suggest you fix the above
> and send another revision to Arnaldo with my RBs.

Will do this. Thanks for suggestion, Mathieu.

Leo

[...]

2021-02-11 13:03:14

by Suzuki K Poulose

[permalink] [raw]
Subject: Re: [PATCH v3 6/8] perf cs-etm: Add helper cs_etm__get_pid_fmt()

On 2/6/21 3:08 PM, Leo Yan wrote:
> This patch adds helper function cs_etm__get_pid_fmt(), by passing
> parameter "traceID", it returns the PID format.
>
> Signed-off-by: Leo Yan <[email protected]>

Reviewed-by: Suzuki K Poulose <[email protected]>

2021-02-11 13:09:38

by Suzuki K Poulose

[permalink] [raw]
Subject: Re: [PATCH v3 8/8] Documentation: coresight: Add PID tracing description

On 2/6/21 3:08 PM, Leo Yan wrote:
> After support the PID tracing for the kernel in EL1 or EL2, the usage
> gets more complicated.
>
> This patch gives description for the PMU formats of contextID configs,
> this can help users to understand how to control the knobs for PID
> tracing when the kernel is in different ELs.
>
> Signed-off-by: Leo Yan <[email protected]>
> ---
> Documentation/trace/coresight/coresight.rst | 32 +++++++++++++++++++++
> 1 file changed, 32 insertions(+)
>
> diff --git a/Documentation/trace/coresight/coresight.rst b/Documentation/trace/coresight/coresight.rst
> index 0b73acb44efa..169749efd8d1 100644
> --- a/Documentation/trace/coresight/coresight.rst
> +++ b/Documentation/trace/coresight/coresight.rst
> @@ -512,6 +512,38 @@ The --itrace option controls the type and frequency of synthesized events
> Note that only 64-bit programs are currently supported - further work is
> required to support instruction decode of 32-bit Arm programs.
>
> +2.2) Tracing PID
> +
> +The kernel can be built to write the PID value into the PE ContextID registers.
> +For a kernel running at EL1, the PID is stored in CONTEXTIDR_EL1. A PE may
> +implement Arm Virtualization Host Extensions (VHE), which the kernel can
> +run at EL2 as a virtualisation host; in this case, the PID value is stored in
> +CONTEXTIDR_EL2.
> +
> +perf provides PMU formats that program the ETM to insert these values into the
> +trace data; the PMU formats are defined as below:
> +
> + "contextid1": Available on both EL1 kernel and EL2 kernel. When the
> + kernel is running at EL1, "contextid1" enables the PID
> + tracing; when the kernel is running at EL2, this enables
> + tracing the PID of guest applications.
> +
> + "contextid2": Only usable when the kernel is running at EL2. When
> + selected, enables PID tracing on EL2 kernel.
> +
> + "contextid": Will be an alias for the option that enables PID
> + tracing. I.e,
> + contextid == contextid1, on EL1 kernel.
> + contextid == contextid2, on EL2 kernel.
> +
> +perf will always enable PID tracing at the relevant EL, this is accomplished by
> +automatically enable the "contextid" config - but for EL2 it is possible to make
> +specific adjustments using configs "contextid1" and "contextid2", E.g. if a user
> +wants to trace PIDs for both host and guest, the two configs "contextid1" and
> +"contextid2" can be set at the same time:
> +
> + perf record -e cs_etm/contextid1,contextid2/u -- vm
> +
>


Reviewed-by: Suzuki K Poulose <[email protected]>

2021-03-05 17:31:17

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: [PATCH v3 4/8] perf cs-etm: Fix bitmap for option

Em Tue, Feb 09, 2021 at 09:58:55AM +0800, Leo Yan escreveu:
> On Mon, Feb 08, 2021 at 01:46:41PM -0700, Mathieu Poirier wrote:
> > On Sat, Feb 06, 2021 at 11:08:29PM +0800, Leo Yan wrote:
> > > From: Suzuki K Poulose <[email protected]>
> > >
> > > When set option with macros ETM_OPT_CTXTID and ETM_OPT_TS, it wrongly
> > > takes these two values (14 and 28 prespectively) as bit masks, but
> > > actually both are the offset for bits. But this doesn't lead to
> > > further failure due to the AND logic operation will be always true for
> > > ETM_OPT_CTXTID / ETM_OPT_TS.
> > >
> > > This patch defines new independent macros (rather than using the
> > > "config" bits) for requesting the "contextid" and "timestamp" for
> > > cs_etm_set_option().
> > >
> > > [leoy: Extract the change as a separate patch for easier review]
> >
> > This should go just above your name - see below.

I fixed this up and added this patch to my perf/urgent branch, for
v5.12, since the kernel bits are upstream and this is a fix.

Looking at the other patches in the series.

- Arnaldo

> > > Signed-off-by: Suzuki K Poulose <[email protected]>
> > > Signed-off-by: Leo Yan <[email protected]>
> > > Reviewed-by: Mike Leach <[email protected]>
> >
> > Signed-off-by: Suzuki K Poulose <[email protected]>
> > [Extract the change as a separate patch for easier review]
> > Signed-off-by: Leo Yan <[email protected]>
> > Reviewed-by: Mike Leach <[email protected]>
> >
> > > ---
> > > tools/perf/arch/arm/util/cs-etm.c | 12 ++++++++----
> > > 1 file changed, 8 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/tools/perf/arch/arm/util/cs-etm.c b/tools/perf/arch/arm/util/cs-etm.c
> > > index bd446aba64f7..c25c878fd06c 100644
> > > --- a/tools/perf/arch/arm/util/cs-etm.c
> > > +++ b/tools/perf/arch/arm/util/cs-etm.c
> > > @@ -156,6 +156,10 @@ static int cs_etm_set_timestamp(struct auxtrace_record *itr,
> > > return err;
> > > }
> > >
> > > +#define ETM_SET_OPT_CTXTID (1 << 0)
> > > +#define ETM_SET_OPT_TS (1 << 1)
> > > +#define ETM_SET_OPT_MASK (ETM_SET_OPT_CTXTID | ETM_SET_OPT_TS)
> > > +
> >
> > I would much rather see this fixed with the BIT() macro as it is done in the
> > rest of this set than defining new constant.
> >
> > With the above:
> >
> > Reviewed-by: Mathieu Poirier <[email protected]>
> >
> > I have picked up the kernel portion of this set. I suggest you fix the above
> > and send another revision to Arnaldo with my RBs.
>
> Will do this. Thanks for suggestion, Mathieu.
>
> Leo
>
> [...]

--

- Arnaldo

2021-03-05 17:33:21

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: [PATCH v3 5/8] perf cs-etm: Support PID tracing in config

Em Sat, Feb 06, 2021 at 11:08:30PM +0800, Leo Yan escreveu:
> From: Suzuki K Poulose <[email protected]>
>
> If the kernel is running at EL2, the pid of a task is exposed via VMID
> instead of the CONTEXTID. Add support for this in the perf tool.
>
> This patch respects user setting if user has specified any configs
> from "contextid", "contextid1" or "contextid2"; otherwise, it
> dynamically sets config based on PMU format "contextid".

Since the merge window is closed, I'll continue processing this from
here onwards on my perf/core branch, as soon as I get what I have in
perf/urgent sent upstream and merged.

- Arnaldo

> Cc: Mike Leach <[email protected]>
> Cc: Mathieu Poirier <[email protected]>
> Cc: Al Grant <[email protected]>
> Signed-off-by: Suzuki K Poulose <[email protected]>
> Co-developed-by: Leo Yan <[email protected]>
> Signed-off-by: Leo Yan <[email protected]>
> Reviewed-by: Mike Leach <[email protected]>
> ---
> tools/include/linux/coresight-pmu.h | 3 ++
> tools/perf/arch/arm/util/cs-etm.c | 61 +++++++++++++++++++++++------
> 2 files changed, 52 insertions(+), 12 deletions(-)
>
> diff --git a/tools/include/linux/coresight-pmu.h b/tools/include/linux/coresight-pmu.h
> index 5dc47cfdcf07..4ac5c081af93 100644
> --- a/tools/include/linux/coresight-pmu.h
> +++ b/tools/include/linux/coresight-pmu.h
> @@ -20,14 +20,17 @@
> */
> #define ETM_OPT_CYCACC 12
> #define ETM_OPT_CTXTID 14
> +#define ETM_OPT_CTXTID2 15
> #define ETM_OPT_TS 28
> #define ETM_OPT_RETSTK 29
>
> /* ETMv4 CONFIGR programming bits for the ETM OPTs */
> #define ETM4_CFG_BIT_CYCACC 4
> #define ETM4_CFG_BIT_CTXTID 6
> +#define ETM4_CFG_BIT_VMID 7
> #define ETM4_CFG_BIT_TS 11
> #define ETM4_CFG_BIT_RETSTK 12
> +#define ETM4_CFG_BIT_VMID_OPT 15
>
> static inline int coresight_get_trace_id(int cpu)
> {
> diff --git a/tools/perf/arch/arm/util/cs-etm.c b/tools/perf/arch/arm/util/cs-etm.c
> index c25c878fd06c..fa6f91a7c8a1 100644
> --- a/tools/perf/arch/arm/util/cs-etm.c
> +++ b/tools/perf/arch/arm/util/cs-etm.c
> @@ -67,6 +67,7 @@ static int cs_etm_set_context_id(struct auxtrace_record *itr,
> char path[PATH_MAX];
> int err = -EINVAL;
> u32 val;
> + u64 contextid;
>
> ptr = container_of(itr, struct cs_etm_recording, itr);
> cs_etm_pmu = ptr->cs_etm_pmu;
> @@ -86,25 +87,59 @@ static int cs_etm_set_context_id(struct auxtrace_record *itr,
> goto out;
> }
>
> + /* User has configured for PID tracing, respects it. */
> + contextid = evsel->core.attr.config &
> + (BIT(ETM_OPT_CTXTID) | BIT(ETM_OPT_CTXTID2));
> +
> /*
> - * TRCIDR2.CIDSIZE, bit [9-5], indicates whether contextID tracing
> - * is supported:
> - * 0b00000 Context ID tracing is not supported.
> - * 0b00100 Maximum of 32-bit Context ID size.
> - * All other values are reserved.
> + * If user doesn't configure the contextid format, parse PMU format and
> + * enable PID tracing according to the "contextid" format bits:
> + *
> + * If bit ETM_OPT_CTXTID is set, trace CONTEXTIDR_EL1;
> + * If bit ETM_OPT_CTXTID2 is set, trace CONTEXTIDR_EL2.
> */
> - val = BMVAL(val, 5, 9);
> - if (!val || val != 0x4) {
> - err = -EINVAL;
> - goto out;
> + if (!contextid)
> + contextid = perf_pmu__format_bits(&cs_etm_pmu->format,
> + "contextid");
> +
> + if (contextid & BIT(ETM_OPT_CTXTID)) {
> + /*
> + * TRCIDR2.CIDSIZE, bit [9-5], indicates whether contextID
> + * tracing is supported:
> + * 0b00000 Context ID tracing is not supported.
> + * 0b00100 Maximum of 32-bit Context ID size.
> + * All other values are reserved.
> + */
> + val = BMVAL(val, 5, 9);
> + if (!val || val != 0x4) {
> + pr_err("%s: CONTEXTIDR_EL1 isn't supported\n",
> + CORESIGHT_ETM_PMU_NAME);
> + err = -EINVAL;
> + goto out;
> + }
> + }
> +
> + if (contextid & BIT(ETM_OPT_CTXTID2)) {
> + /*
> + * TRCIDR2.VMIDOPT[30:29] != 0 and
> + * TRCIDR2.VMIDSIZE[14:10] == 0b00100 (32bit virtual contextid)
> + * We can't support CONTEXTIDR in VMID if the size of the
> + * virtual context id is < 32bit.
> + * Any value of VMIDSIZE >= 4 (i.e, > 32bit) is fine for us.
> + */
> + if (!BMVAL(val, 29, 30) || BMVAL(val, 10, 14) < 4) {
> + pr_err("%s: CONTEXTIDR_EL2 isn't supported\n",
> + CORESIGHT_ETM_PMU_NAME);
> + err = -EINVAL;
> + goto out;
> + }
> }
>
> /* All good, let the kernel know */
> - evsel->core.attr.config |= (1 << ETM_OPT_CTXTID);
> + evsel->core.attr.config |= contextid;
> err = 0;
>
> out:
> -
> return err;
> }
>
> @@ -489,7 +524,9 @@ static u64 cs_etmv4_get_config(struct auxtrace_record *itr)
> config |= BIT(ETM4_CFG_BIT_TS);
> if (config_opts & BIT(ETM_OPT_RETSTK))
> config |= BIT(ETM4_CFG_BIT_RETSTK);
> -
> + if (config_opts & BIT(ETM_OPT_CTXTID2))
> + config |= BIT(ETM4_CFG_BIT_VMID) |
> + BIT(ETM4_CFG_BIT_VMID_OPT);
> return config;
> }
>
> --
> 2.25.1
>

--

- Arnaldo

2021-03-06 03:09:14

by Leo Yan

[permalink] [raw]
Subject: Re: [PATCH v3 4/8] perf cs-etm: Fix bitmap for option

Hi Arnaldo,

On Fri, Mar 05, 2021 at 02:29:44PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Tue, Feb 09, 2021 at 09:58:55AM +0800, Leo Yan escreveu:
> > On Mon, Feb 08, 2021 at 01:46:41PM -0700, Mathieu Poirier wrote:
> > > On Sat, Feb 06, 2021 at 11:08:29PM +0800, Leo Yan wrote:
> > > > From: Suzuki K Poulose <[email protected]>
> > > >
> > > > When set option with macros ETM_OPT_CTXTID and ETM_OPT_TS, it wrongly
> > > > takes these two values (14 and 28 prespectively) as bit masks, but
> > > > actually both are the offset for bits. But this doesn't lead to
> > > > further failure due to the AND logic operation will be always true for
> > > > ETM_OPT_CTXTID / ETM_OPT_TS.
> > > >
> > > > This patch defines new independent macros (rather than using the
> > > > "config" bits) for requesting the "contextid" and "timestamp" for
> > > > cs_etm_set_option().
> > > >
> > > > [leoy: Extract the change as a separate patch for easier review]
> > >
> > > This should go just above your name - see below.
>
> I fixed this up and added this patch to my perf/urgent branch, for
> v5.12, since the kernel bits are upstream and this is a fix.

Yeah, it makes sense to pick this patch into perf/urgent branch since
it's a fixing patch.

Actually, this patch has been merged into the tmp.perf/core branch [1],
after you move it to the perf/urgent branch, I can confirm all other
patches for perf tool in this series have been merged into the
tmp.perf/core branch.

Thanks,
Leo

[1] https://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git/commit/?h=tmp.perf/core&id=8c559e8d68630d64d932bada633705f6551427df