2022-09-09 15:34:16

by Carsten Haitzler

[permalink] [raw]
Subject: [PATCH v9 00/13] perf: test: Add trace data quality tests for CoreSight

From: Carsten Haitzler <[email protected]>

This series adds more test cases to perf test as well as new
infrastructure for testing coresight data quality to ensure the data
coming out of the kernel via perf actually contains useful information.

Signed-off-by: Carsten Haitzler <[email protected]>


Carsten Haitzler (13):
perf test: Add CoreSight shell lib shared code for future tests
perf test: Add build infra for perf test tools for CoreSight tests
perf test: Add asm pureloop test tool
perf test: Add asm pureloop test shell script
perf test: Add git ignore for perf data generated by the CoreSight
tests
perf test: Add memcpy thread test tool
perf test: Add memcpy thread test shell script
perf test: Add thread loop test tool
perf test: Add thread loop test shell scripts
perf test: Add unroll thread test tool
perf test: Add unroll thread test shell script
perf test: Add git ignore for tmp and output files of CoreSight tests
perf test: Add relevant documentation about CoreSight testing

.../trace/coresight/coresight-perf.rst | 158 ++++++++++++++++++
MAINTAINERS | 1 +
tools/perf/.gitignore | 6 +-
.../perf/Documentation/perf-arm-coresight.txt | 5 +
tools/perf/Makefile.config | 2 +
tools/perf/Makefile.perf | 15 +-
tools/perf/tests/shell/coresight/Makefile | 29 ++++
.../tests/shell/coresight/Makefile.miniconfig | 14 ++
.../tests/shell/coresight/asm_pure_loop.sh | 18 ++
.../shell/coresight/asm_pure_loop/.gitignore | 1 +
.../shell/coresight/asm_pure_loop/Makefile | 34 ++++
.../coresight/asm_pure_loop/asm_pure_loop.S | 28 ++++
.../shell/coresight/memcpy_thread/.gitignore | 1 +
.../shell/coresight/memcpy_thread/Makefile | 33 ++++
.../coresight/memcpy_thread/memcpy_thread.c | 79 +++++++++
.../shell/coresight/memcpy_thread_16k_10.sh | 18 ++
.../shell/coresight/thread_loop/.gitignore | 1 +
.../shell/coresight/thread_loop/Makefile | 33 ++++
.../shell/coresight/thread_loop/thread_loop.c | 86 ++++++++++
.../coresight/thread_loop_check_tid_10.sh | 19 +++
.../coresight/thread_loop_check_tid_2.sh | 19 +++
.../coresight/unroll_loop_thread/.gitignore | 1 +
.../coresight/unroll_loop_thread/Makefile | 33 ++++
.../unroll_loop_thread/unroll_loop_thread.c | 74 ++++++++
.../shell/coresight/unroll_loop_thread_10.sh | 18 ++
tools/perf/tests/shell/lib/coresight.sh | 132 +++++++++++++++
26 files changed, 854 insertions(+), 4 deletions(-)
create mode 100644 Documentation/trace/coresight/coresight-perf.rst
create mode 100644 tools/perf/Documentation/perf-arm-coresight.txt
create mode 100644 tools/perf/tests/shell/coresight/Makefile
create mode 100644 tools/perf/tests/shell/coresight/Makefile.miniconfig
create mode 100755 tools/perf/tests/shell/coresight/asm_pure_loop.sh
create mode 100644 tools/perf/tests/shell/coresight/asm_pure_loop/.gitignore
create mode 100644 tools/perf/tests/shell/coresight/asm_pure_loop/Makefile
create mode 100644 tools/perf/tests/shell/coresight/asm_pure_loop/asm_pure_loop.S
create mode 100644 tools/perf/tests/shell/coresight/memcpy_thread/.gitignore
create mode 100644 tools/perf/tests/shell/coresight/memcpy_thread/Makefile
create mode 100644 tools/perf/tests/shell/coresight/memcpy_thread/memcpy_thread.c
create mode 100755 tools/perf/tests/shell/coresight/memcpy_thread_16k_10.sh
create mode 100644 tools/perf/tests/shell/coresight/thread_loop/.gitignore
create mode 100644 tools/perf/tests/shell/coresight/thread_loop/Makefile
create mode 100644 tools/perf/tests/shell/coresight/thread_loop/thread_loop.c
create mode 100755 tools/perf/tests/shell/coresight/thread_loop_check_tid_10.sh
create mode 100755 tools/perf/tests/shell/coresight/thread_loop_check_tid_2.sh
create mode 100644 tools/perf/tests/shell/coresight/unroll_loop_thread/.gitignore
create mode 100644 tools/perf/tests/shell/coresight/unroll_loop_thread/Makefile
create mode 100644 tools/perf/tests/shell/coresight/unroll_loop_thread/unroll_loop_thread.c
create mode 100755 tools/perf/tests/shell/coresight/unroll_loop_thread_10.sh
create mode 100644 tools/perf/tests/shell/lib/coresight.sh

--
2.32.0


2022-09-09 15:34:47

by Carsten Haitzler

[permalink] [raw]
Subject: [PATCH v9 05/13] perf test: Add git ignore for perf data generated by the CoreSight tests

From: Carsten Haitzler <[email protected]>

Ignore perf output data files generated by perf tests for cleaner
git status.

Signed-off-by: Carsten Haitzler <[email protected]>
---
tools/perf/.gitignore | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/perf/.gitignore b/tools/perf/.gitignore
index 4b9c71faa01a..faa23b5d32f5 100644
--- a/tools/perf/.gitignore
+++ b/tools/perf/.gitignore
@@ -15,8 +15,8 @@ perf*.1
perf*.xml
perf*.html
common-cmds.h
-perf.data
-perf.data.old
+perf*.data
+perf*.data.old
output.svg
perf-archive
perf-iostat
--
2.32.0

2022-09-09 15:35:07

by Carsten Haitzler

[permalink] [raw]
Subject: [PATCH v9 06/13] perf test: Add memcpy thread test tool

From: Carsten Haitzler <[email protected]>

Add test tool to be driven by further test scripts. This is a simple C
based memcpy with threads test to drive from scripts.

Signed-off-by: Carsten Haitzler <[email protected]>
---
tools/perf/tests/shell/coresight/Makefile | 3 +-
.../shell/coresight/memcpy_thread/.gitignore | 1 +
.../shell/coresight/memcpy_thread/Makefile | 33 ++++++++
.../coresight/memcpy_thread/memcpy_thread.c | 79 +++++++++++++++++++
4 files changed, 115 insertions(+), 1 deletion(-)
create mode 100644 tools/perf/tests/shell/coresight/memcpy_thread/.gitignore
create mode 100644 tools/perf/tests/shell/coresight/memcpy_thread/Makefile
create mode 100644 tools/perf/tests/shell/coresight/memcpy_thread/memcpy_thread.c

diff --git a/tools/perf/tests/shell/coresight/Makefile b/tools/perf/tests/shell/coresight/Makefile
index 0c192ee253e3..9a8526a74267 100644
--- a/tools/perf/tests/shell/coresight/Makefile
+++ b/tools/perf/tests/shell/coresight/Makefile
@@ -5,7 +5,8 @@ include ../../../../../tools/scripts/Makefile.arch
include ../../../../../tools/scripts/utilities.mak

SUBDIRS = \
- asm_pure_loop
+ asm_pure_loop \
+ memcpy_thread

all: $(SUBDIRS)
$(SUBDIRS):
diff --git a/tools/perf/tests/shell/coresight/memcpy_thread/.gitignore b/tools/perf/tests/shell/coresight/memcpy_thread/.gitignore
new file mode 100644
index 000000000000..f8217e56091e
--- /dev/null
+++ b/tools/perf/tests/shell/coresight/memcpy_thread/.gitignore
@@ -0,0 +1 @@
+memcpy_thread
diff --git a/tools/perf/tests/shell/coresight/memcpy_thread/Makefile b/tools/perf/tests/shell/coresight/memcpy_thread/Makefile
new file mode 100644
index 000000000000..2db637eb2c26
--- /dev/null
+++ b/tools/perf/tests/shell/coresight/memcpy_thread/Makefile
@@ -0,0 +1,33 @@
+# SPDX-License-Identifier: GPL-2.0
+# Carsten Haitzler <[email protected]>, 2021
+include ../Makefile.miniconfig
+
+# Binary to produce
+BIN=memcpy_thread
+# Any linking/libraries needed for the binary - empty if none needed
+LIB=-pthread
+
+all: $(BIN)
+
+$(BIN): $(BIN).c
+ifdef CORESIGHT
+ifeq ($(ARCH),arm64)
+# Build line
+ $(Q)$(CC) $(BIN).c -o $(BIN) $(LIB)
+endif
+endif
+
+install-tests: all
+ifdef CORESIGHT
+ifeq ($(ARCH),arm64)
+# Install the test tool in the right place
+ $(call QUIET_INSTALL, tests) \
+ $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/$(INSTDIR_SUB)/$(BIN)'; \
+ $(INSTALL) $(BIN) '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/$(INSTDIR_SUB)/$(BIN)/$(BIN)'
+endif
+endif
+
+clean:
+ $(Q)$(RM) -f $(BIN)
+
+.PHONY: all clean install-tests
diff --git a/tools/perf/tests/shell/coresight/memcpy_thread/memcpy_thread.c b/tools/perf/tests/shell/coresight/memcpy_thread/memcpy_thread.c
new file mode 100644
index 000000000000..a7e169d1bf64
--- /dev/null
+++ b/tools/perf/tests/shell/coresight/memcpy_thread/memcpy_thread.c
@@ -0,0 +1,79 @@
+// SPDX-License-Identifier: GPL-2.0
+// Carsten Haitzler <[email protected]>, 2021
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <string.h>
+#include <pthread.h>
+
+struct args {
+ unsigned long loops;
+ unsigned long size;
+ pthread_t th;
+ void *ret;
+};
+
+static void *thrfn(void *arg)
+{
+ struct args *a = arg;
+ unsigned long i, len = a->loops;
+ unsigned char *src, *dst;
+
+ src = malloc(a->size * 1024);
+ dst = malloc(a->size * 1024);
+ if ((!src) || (!dst)) {
+ printf("ERR: Can't allocate memory\n");
+ exit(1);
+ }
+ for (i = 0; i < len; i++)
+ memcpy(dst, src, a->size * 1024);
+}
+
+static pthread_t new_thr(void *(*fn) (void *arg), void *arg)
+{
+ pthread_t t;
+ pthread_attr_t attr;
+
+ pthread_attr_init(&attr);
+ pthread_create(&t, &attr, fn, arg);
+ return t;
+}
+
+int main(int argc, char **argv)
+{
+ unsigned long i, len, size, thr;
+ pthread_t threads[256];
+ struct args args[256];
+ long long v;
+
+ if (argc < 4) {
+ printf("ERR: %s [copysize Kb] [numthreads] [numloops (hundreds)]\n", argv[0]);
+ exit(1);
+ }
+
+ v = atoll(argv[1]);
+ if ((v < 1) || (v > (1024 * 1024))) {
+ printf("ERR: max memory 1GB (1048576 KB)\n");
+ exit(1);
+ }
+ size = v;
+ thr = atol(argv[2]);
+ if ((thr < 1) || (thr > 256)) {
+ printf("ERR: threads 1-256\n");
+ exit(1);
+ }
+ v = atoll(argv[3]);
+ if ((v < 1) || (v > 40000000000ll)) {
+ printf("ERR: loops 1-40000000000 (hundreds)\n");
+ exit(1);
+ }
+ len = v * 100;
+ for (i = 0; i < thr; i++) {
+ args[i].loops = len;
+ args[i].size = size;
+ args[i].th = new_thr(thrfn, &(args[i]));
+ }
+ for (i = 0; i < thr; i++)
+ pthread_join(args[i].th, &(args[i].ret));
+ return 0;
+}
--
2.32.0

2022-09-09 15:35:32

by Carsten Haitzler

[permalink] [raw]
Subject: [PATCH v9 11/13] perf test: Add unroll thread test shell script

From: Carsten Haitzler <[email protected]>

This adds scripts to drive the unroll thread tests to compare perf
output against a minimum bar of content/quality.

Signed-off-by: Carsten Haitzler <[email protected]>
---
.../shell/coresight/unroll_loop_thread_10.sh | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
create mode 100755 tools/perf/tests/shell/coresight/unroll_loop_thread_10.sh

diff --git a/tools/perf/tests/shell/coresight/unroll_loop_thread_10.sh b/tools/perf/tests/shell/coresight/unroll_loop_thread_10.sh
new file mode 100755
index 000000000000..f48c85230b15
--- /dev/null
+++ b/tools/perf/tests/shell/coresight/unroll_loop_thread_10.sh
@@ -0,0 +1,18 @@
+#!/bin/sh -e
+# CoreSight / Unroll Loop Thread 10
+
+# SPDX-License-Identifier: GPL-2.0
+# Carsten Haitzler <[email protected]>, 2021
+
+TEST="unroll_loop_thread"
+. $(dirname $0)/../lib/coresight.sh
+ARGS="10"
+DATV="10"
+DATA="$DATD/perf-$TEST-$DATV.data"
+
+perf record $PERFRECOPT -o "$DATA" "$BIN" $ARGS
+
+perf_dump_aux_verify "$DATA" 10 10 10
+
+err=$?
+exit $err
--
2.32.0

2022-09-09 15:45:39

by Carsten Haitzler

[permalink] [raw]
Subject: [PATCH v9 01/13] perf test: Add CoreSight shell lib shared code for future tests

From: Carsten Haitzler <[email protected]>

This adds a library of shell "code" to be shared and used by future
tests that target quality testing for Arm CoreSight support in perf
and the Linux kernel.

Signed-off-by: Carsten Haitzler <[email protected]>
---
tools/perf/tests/shell/lib/coresight.sh | 132 ++++++++++++++++++++++++
1 file changed, 132 insertions(+)
create mode 100644 tools/perf/tests/shell/lib/coresight.sh

diff --git a/tools/perf/tests/shell/lib/coresight.sh b/tools/perf/tests/shell/lib/coresight.sh
new file mode 100644
index 000000000000..45a1477256b6
--- /dev/null
+++ b/tools/perf/tests/shell/lib/coresight.sh
@@ -0,0 +1,132 @@
+# SPDX-License-Identifier: GPL-2.0
+# Carsten Haitzler <[email protected]>, 2021
+
+# This is sourced from a driver script so no need for #!/bin... etc. at the
+# top - the assumption below is that it runs as part of sourcing after the
+# test sets up some basic env vars to say what it is.
+
+# This currently works with ETMv4 / ETF not any other packet types at thi
+# point. This will need changes if that changes.
+
+# perf record options for the perf tests to use
+PERFRECMEM="-m ,16M"
+PERFRECOPT="$PERFRECMEM -e cs_etm//u"
+
+TOOLS=$(dirname $0)
+DIR="$TOOLS/$TEST"
+BIN="$DIR/$TEST"
+# If the test tool/binary does not exist and is executable then skip the test
+if ! test -x "$BIN"; then exit 2; fi
+DATD="."
+# If the data dir env is set then make the data dir use that instead of ./
+if test -n "$PERF_TEST_CORESIGHT_DATADIR"; then
+ DATD="$PERF_TEST_CORESIGHT_DATADIR";
+fi
+# If the stat dir env is set then make the data dir use that instead of ./
+STATD="."
+if test -n "$PERF_TEST_CORESIGHT_STATDIR"; then
+ STATD="$PERF_TEST_CORESIGHT_STATDIR";
+fi
+
+# Called if the test fails - error code 1
+err() {
+ echo "$1"
+ exit 1
+}
+
+# Check that some statistics from our perf
+check_val_min() {
+ STATF="$4"
+ if test "$2" -lt "$3"; then
+ echo ", FAILED" >> "$STATF"
+ err "Sanity check number of $1 is too low ($2 < $3)"
+ fi
+}
+
+perf_dump_aux_verify() {
+ # Some basic checking that the AUX chunk contains some sensible data
+ # to see that we are recording something and at least a minimum
+ # amount of it. We should almost always see Fn packets in just about
+ # anything but certainly we will see some trace info and async
+ # packets
+ DUMP="$DATD/perf-tmp-aux-dump.txt"
+ perf report --stdio --dump -i "$1" | \
+ grep -o -e I_ATOM_F -e I_ASYNC -e I_TRACE_INFO > "$DUMP"
+ # Simply count how many of these packets we find to see that we are
+ # producing a reasonable amount of data - exact checks are not sane
+ # as this is a lossy process where we may lose some blocks and the
+ # compiler may produce different code depending on the compiler and
+ # optimization options, so this is rough just to see if we're
+ # either missing almost all the data or all of it
+ ATOM_FX_NUM=`grep I_ATOM_F "$DUMP" | wc -l`
+ ASYNC_NUM=`grep I_ASYNC "$DUMP" | wc -l`
+ TRACE_INFO_NUM=`grep I_TRACE_INFO "$DUMP" | wc -l`
+ rm -f "$DUMP"
+
+ # Arguments provide minimums for a pass
+ CHECK_FX_MIN="$2"
+ CHECK_ASYNC_MIN="$3"
+ CHECK_TRACE_INFO_MIN="$4"
+
+ # Write out statistics, so over time you can track results to see if
+ # there is a pattern - for example we have less "noisy" results that
+ # produce more consistent amounts of data each run, to see if over
+ # time any techinques to minimize data loss are having an effect or
+ # not
+ STATF="$STATD/stats-$TEST-$DATV.csv"
+ if ! test -f "$STATF"; then
+ echo "ATOM Fx Count, Minimum, ASYNC Count, Minimum, TRACE INFO Count, Minimum" > "$STATF"
+ fi
+ echo -n "$ATOM_FX_NUM, $CHECK_FX_MIN, $ASYNC_NUM, $CHECK_ASYNC_MIN, $TRACE_INFO_NUM, $CHECK_TRACE_INFO_MIN" >> "$STATF"
+
+ # Actually check to see if we passed or failed.
+ check_val_min "ATOM_FX" "$ATOM_FX_NUM" "$CHECK_FX_MIN" "$STATF"
+ check_val_min "ASYNC" "$ASYNC_NUM" "$CHECK_ASYNC_MIN" "$STATF"
+ check_val_min "TRACE_INFO" "$TRACE_INFO_NUM" "$CHECK_TRACE_INFO_MIN" "$STATF"
+ echo ", Ok" >> "$STATF"
+}
+
+perf_dump_aux_tid_verify() {
+ # Specifically crafted test will produce a list of Tread ID's to
+ # stdout that need to be checked to see that they have had trace
+ # info collected in AUX blocks in the perf data. This will go
+ # through all the TID's that are listed as CID=0xabcdef and see
+ # that all the Thread IDs the test tool reports are in the perf
+ # data AUX chunks
+
+ # The TID test tools will print a TID per stdout line that are being
+ # tested
+ TIDS=`cat "$2"`
+ # Scan the perf report to find the TIDs that are actually CID in hex
+ # and build a list of the ones found
+ FOUND_TIDS=`perf report --stdio --dump -i "$1" | \
+ grep -o "CID=0x[0-9a-z]\+" | sed 's/CID=//g' | \
+ uniq | sort | uniq`
+ # No CID=xxx found - maybe your kernel is reporting these as
+ # VMID=xxx so look there
+ if test -z "$FOUND_TIDS"; then
+ FOUND_TIDS=`perf report --stdio --dump -i "$1" | \
+ grep -o "VMID=0x[0-9a-z]\+" | sed 's/VMID=//g' | \
+ uniq | sort | uniq`
+ fi
+
+ # Iterate over the list of TIDs that the test says it has and find
+ # them in the TIDs found in the perf report
+ MISSING=""
+ for TID2 in $TIDS; do
+ FOUND=""
+ for TIDHEX in $FOUND_TIDS; do
+ TID=`printf "%i" $TIDHEX`
+ if test "$TID" -eq "$TID2"; then
+ FOUND="y"
+ break
+ fi
+ done
+ if test -z "$FOUND"; then
+ MISSING="$MISSING $TID"
+ fi
+ done
+ if test -n "$MISSING"; then
+ err "Thread IDs $MISSING not found in perf AUX data"
+ fi
+}
--
2.32.0

2022-09-09 15:46:06

by Carsten Haitzler

[permalink] [raw]
Subject: [PATCH v9 02/13] perf test: Add build infra for perf test tools for CoreSight tests

From: Carsten Haitzler <[email protected]>

This adds the initial build infrastructure (makefiles maintainers
information) for adding follow-on tests for CoreSight.

Signed-off-by: Carsten Haitzler <[email protected]>
---
MAINTAINERS | 1 +
tools/perf/Makefile.config | 2 ++
tools/perf/Makefile.perf | 15 +++++++++--
tools/perf/tests/shell/coresight/Makefile | 25 +++++++++++++++++++
.../tests/shell/coresight/Makefile.miniconfig | 14 +++++++++++
5 files changed, 55 insertions(+), 2 deletions(-)
create mode 100644 tools/perf/tests/shell/coresight/Makefile
create mode 100644 tools/perf/tests/shell/coresight/Makefile.miniconfig

diff --git a/MAINTAINERS b/MAINTAINERS
index 9d7f64dc0efe..c8ae5a6638b6 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2029,6 +2029,7 @@ F: drivers/hwtracing/coresight/*
F: include/dt-bindings/arm/coresight-cti-dt.h
F: include/linux/coresight*
F: samples/coresight/*
+F: tools/perf/tests/shell/coresight/*
F: tools/perf/arch/arm/util/auxtrace.c
F: tools/perf/arch/arm/util/cs-etm.c
F: tools/perf/arch/arm/util/cs-etm.h
diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config
index 0661a1cf9855..e3e28b3481f9 100644
--- a/tools/perf/Makefile.config
+++ b/tools/perf/Makefile.config
@@ -1291,6 +1291,8 @@ perf_examples_instdir_SQ = $(subst ','\'',$(perf_examples_instdir))
STRACE_GROUPS_INSTDIR_SQ = $(subst ','\'',$(STRACE_GROUPS_INSTDIR))
tip_instdir_SQ = $(subst ','\'',$(tip_instdir))

+export perfexec_instdir_SQ
+
# If we install to $(HOME) we keep the traceevent default:
# $(HOME)/.traceevent/plugins
# Otherwise we install plugins into the global $(libdir).
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index e5921b347153..cef2a06c8f54 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -629,7 +629,15 @@ sync_file_range_tbls := $(srctree)/tools/perf/trace/beauty/sync_file_range.sh
$(sync_file_range_arrays): $(linux_uapi_dir)/fs.h $(sync_file_range_tbls)
$(Q)$(SHELL) '$(sync_file_range_tbls)' $(linux_uapi_dir) > $@

-all: shell_compatibility_test $(ALL_PROGRAMS) $(LANG_BINDINGS) $(OTHER_PROGRAMS)
+TESTS_CORESIGHT_DIR := $(srctree)/tools/perf/tests/shell/coresight
+
+tests-coresight-targets: FORCE
+ $(Q)$(MAKE) -C $(TESTS_CORESIGHT_DIR)
+
+tests-coresight-targets-clean:
+ $(Q)$(MAKE) -C $(TESTS_CORESIGHT_DIR) clean
+
+all: shell_compatibility_test $(ALL_PROGRAMS) $(LANG_BINDINGS) $(OTHER_PROGRAMS) tests-coresight-targets

# Create python binding output directory if not already present
_dummy := $(shell [ -d '$(OUTPUT)python' ] || mkdir -p '$(OUTPUT)python')
@@ -1007,6 +1015,9 @@ install-tests: all install-gtk
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
$(INSTALL) tests/shell/lib/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
$(INSTALL) tests/shell/lib/*.py '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'
+ $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'; \
+ $(INSTALL) tests/shell/coresight/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
+ $(Q)$(MAKE) -C tests/shell/coresight install-tests

install-bin: install-tools install-tests install-traceevent-plugins

@@ -1077,7 +1088,7 @@ endif # BUILD_BPF_SKEL
bpf-skel-clean:
$(call QUIET_CLEAN, bpf-skel) $(RM) -r $(SKEL_TMP_OUT) $(SKELETONS)

-clean:: $(LIBTRACEEVENT)-clean $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clean $(LIBPERF)-clean fixdep-clean python-clean bpf-skel-clean
+clean:: $(LIBTRACEEVENT)-clean $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clean $(LIBPERF)-clean fixdep-clean python-clean bpf-skel-clean tests-coresight-targets-clean
$(call QUIET_CLEAN, core-objs) $(RM) $(LIBPERF_A) $(OUTPUT)perf-archive $(OUTPUT)perf-iostat $(LANG_BINDINGS)
$(Q)find $(or $(OUTPUT),.) -name '*.o' -delete -o -name '\.*.cmd' -delete -o -name '\.*.d' -delete
$(Q)$(RM) $(OUTPUT).config-detected
diff --git a/tools/perf/tests/shell/coresight/Makefile b/tools/perf/tests/shell/coresight/Makefile
new file mode 100644
index 000000000000..3fee05cfcb0e
--- /dev/null
+++ b/tools/perf/tests/shell/coresight/Makefile
@@ -0,0 +1,25 @@
+# SPDX-License-Identifier: GPL-2.0-only
+# Carsten Haitzler <[email protected]>, 2021
+include ../../../../../tools/scripts/Makefile.include
+include ../../../../../tools/scripts/Makefile.arch
+include ../../../../../tools/scripts/utilities.mak
+
+SUBDIRS =
+
+all: $(SUBDIRS)
+$(SUBDIRS):
+ $(Q)$(MAKE) -C $@
+
+INSTALLDIRS = $(SUBDIRS:%=install-%)
+
+install-tests: $(INSTALLDIRS)
+$(INSTALLDIRS):
+ $(Q)$(MAKE) -C $(@:install-%=%) install-tests
+
+CLEANDIRS = $(SUBDIRS:%=clean-%)
+
+clean: $(CLEANDIRS)
+$(CLEANDIRS):
+ $(Q)$(MAKE) -C $(@:clean-%=%) clean >/dev/null
+
+.PHONY: all clean $(SUBDIRS) $(CLEANDIRS) $(INSTALLDIRS)
diff --git a/tools/perf/tests/shell/coresight/Makefile.miniconfig b/tools/perf/tests/shell/coresight/Makefile.miniconfig
new file mode 100644
index 000000000000..5f72a9cb43f3
--- /dev/null
+++ b/tools/perf/tests/shell/coresight/Makefile.miniconfig
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: GPL-2.0-only
+# Carsten Haitzler <[email protected]>, 2021
+
+ifndef DESTDIR
+prefix ?= $(HOME)
+endif
+
+DESTDIR_SQ = $(subst ','\'',$(DESTDIR))
+INSTALL = install
+INSTDIR_SUB = tests/shell/coresight
+
+include ../../../../../scripts/Makefile.include
+include ../../../../../scripts/Makefile.arch
+include ../../../../../scripts/utilities.mak
--
2.32.0

2022-09-09 15:48:16

by Carsten Haitzler

[permalink] [raw]
Subject: [PATCH v9 10/13] perf test: Add unroll thread test tool

From: Carsten Haitzler <[email protected]>

Add test tool to be driven by further test scripts. This is a simple C
based test that is for arm64 with some inline ASM to manually unroll a
lot of code to have a very long sequence of commands.

Signed-off-by: Carsten Haitzler <[email protected]>
---
tools/perf/tests/shell/coresight/Makefile | 3 +-
.../coresight/unroll_loop_thread/.gitignore | 1 +
.../coresight/unroll_loop_thread/Makefile | 33 +++++++++
.../unroll_loop_thread/unroll_loop_thread.c | 74 +++++++++++++++++++
4 files changed, 110 insertions(+), 1 deletion(-)
create mode 100644 tools/perf/tests/shell/coresight/unroll_loop_thread/.gitignore
create mode 100644 tools/perf/tests/shell/coresight/unroll_loop_thread/Makefile
create mode 100644 tools/perf/tests/shell/coresight/unroll_loop_thread/unroll_loop_thread.c

diff --git a/tools/perf/tests/shell/coresight/Makefile b/tools/perf/tests/shell/coresight/Makefile
index 9330e4f62656..c8e3b2b6d5b5 100644
--- a/tools/perf/tests/shell/coresight/Makefile
+++ b/tools/perf/tests/shell/coresight/Makefile
@@ -7,7 +7,8 @@ include ../../../../../tools/scripts/utilities.mak
SUBDIRS = \
asm_pure_loop \
memcpy_thread \
- thread_loop
+ thread_loop \
+ unroll_loop_thread

all: $(SUBDIRS)
$(SUBDIRS):
diff --git a/tools/perf/tests/shell/coresight/unroll_loop_thread/.gitignore b/tools/perf/tests/shell/coresight/unroll_loop_thread/.gitignore
new file mode 100644
index 000000000000..2cb4e996dbf3
--- /dev/null
+++ b/tools/perf/tests/shell/coresight/unroll_loop_thread/.gitignore
@@ -0,0 +1 @@
+unroll_loop_thread
diff --git a/tools/perf/tests/shell/coresight/unroll_loop_thread/Makefile b/tools/perf/tests/shell/coresight/unroll_loop_thread/Makefile
new file mode 100644
index 000000000000..6264c4e3abd1
--- /dev/null
+++ b/tools/perf/tests/shell/coresight/unroll_loop_thread/Makefile
@@ -0,0 +1,33 @@
+# SPDX-License-Identifier: GPL-2.0
+# Carsten Haitzler <[email protected]>, 2021
+include ../Makefile.miniconfig
+
+# Binary to produce
+BIN=unroll_loop_thread
+# Any linking/libraries needed for the binary - empty if none needed
+LIB=-pthread
+
+all: $(BIN)
+
+$(BIN): $(BIN).c
+ifdef CORESIGHT
+ifeq ($(ARCH),arm64)
+# Build line
+ $(Q)$(CC) $(BIN).c -o $(BIN) $(LIB)
+endif
+endif
+
+install-tests: all
+ifdef CORESIGHT
+ifeq ($(ARCH),arm64)
+# Install the test tool in the right place
+ $(call QUIET_INSTALL, tests) \
+ $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/$(INSTDIR_SUB)/$(BIN)'; \
+ $(INSTALL) $(BIN) '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/$(INSTDIR_SUB)/$(BIN)/$(BIN)'
+endif
+endif
+
+clean:
+ $(Q)$(RM) -f $(BIN)
+
+.PHONY: all clean install-tests
diff --git a/tools/perf/tests/shell/coresight/unroll_loop_thread/unroll_loop_thread.c b/tools/perf/tests/shell/coresight/unroll_loop_thread/unroll_loop_thread.c
new file mode 100644
index 000000000000..8f6d384208ed
--- /dev/null
+++ b/tools/perf/tests/shell/coresight/unroll_loop_thread/unroll_loop_thread.c
@@ -0,0 +1,74 @@
+// SPDX-License-Identifier: GPL-2.0
+// Carsten Haitzler <[email protected]>, 2021
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <string.h>
+#include <pthread.h>
+
+struct args {
+ pthread_t th;
+ unsigned int in;
+ void *ret;
+};
+
+static void *thrfn(void *arg)
+{
+ struct args *a = arg;
+ unsigned int i, in = a->in;
+
+ for (i = 0; i < 10000; i++) {
+ asm volatile (
+// force an unroll of thia add instruction so we can test long runs of code
+#define SNIP1 "add %[in], %[in], #1\n"
+// 10
+#define SNIP2 SNIP1 SNIP1 SNIP1 SNIP1 SNIP1 SNIP1 SNIP1 SNIP1 SNIP1 SNIP1
+// 100
+#define SNIP3 SNIP2 SNIP2 SNIP2 SNIP2 SNIP2 SNIP2 SNIP2 SNIP2 SNIP2 SNIP2
+// 1000
+#define SNIP4 SNIP3 SNIP3 SNIP3 SNIP3 SNIP3 SNIP3 SNIP3 SNIP3 SNIP3 SNIP3
+// 10000
+#define SNIP5 SNIP4 SNIP4 SNIP4 SNIP4 SNIP4 SNIP4 SNIP4 SNIP4 SNIP4 SNIP4
+// 100000
+ SNIP5 SNIP5 SNIP5 SNIP5 SNIP5 SNIP5 SNIP5 SNIP5 SNIP5 SNIP5
+ : /* out */
+ : /* in */ [in] "r" (in)
+ : /* clobber */
+ );
+ }
+}
+
+static pthread_t new_thr(void *(*fn) (void *arg), void *arg)
+{
+ pthread_t t;
+ pthread_attr_t attr;
+
+ pthread_attr_init(&attr);
+ pthread_create(&t, &attr, fn, arg);
+ return t;
+}
+
+int main(int argc, char **argv)
+{
+ unsigned int i, thr;
+ pthread_t threads[256];
+ struct args args[256];
+
+ if (argc < 2) {
+ printf("ERR: %s [numthreads]\n", argv[0]);
+ exit(1);
+ }
+
+ thr = atoi(argv[1]);
+ if ((thr > 256) || (thr < 1)) {
+ printf("ERR: threads 1-256\n");
+ exit(1);
+ }
+ for (i = 0; i < thr; i++) {
+ args[i].in = rand();
+ args[i].th = new_thr(thrfn, &(args[i]));
+ }
+ for (i = 0; i < thr; i++)
+ pthread_join(args[i].th, &(args[i].ret));
+ return 0;
+}
--
2.32.0

2022-09-09 15:49:44

by Carsten Haitzler

[permalink] [raw]
Subject: [PATCH v9 12/13] perf test: Add git ignore for tmp and output files of CoreSight tests

From: Carsten Haitzler <[email protected]>

Ignore other output files of the new CoreSight tests so they don't
fill git status with noise we don't need or want.

Signed-off-by: Carsten Haitzler <[email protected]>
---
tools/perf/.gitignore | 2 ++
1 file changed, 2 insertions(+)

diff --git a/tools/perf/.gitignore b/tools/perf/.gitignore
index faa23b5d32f5..a653311d9693 100644
--- a/tools/perf/.gitignore
+++ b/tools/perf/.gitignore
@@ -22,6 +22,7 @@ perf-archive
perf-iostat
tags
TAGS
+stats-*.csv
cscope*
config.mak
config.mak.autogen
@@ -29,6 +30,7 @@ config.mak.autogen
*-flex.*
*.pyc
*.pyo
+*.stdout
.config-detected
util/intel-pt-decoder/inat-tables.c
arch/*/include/generated/
--
2.32.0

2022-09-09 15:57:00

by Carsten Haitzler

[permalink] [raw]
Subject: [PATCH v9 04/13] perf test: Add asm pureloop test shell script

From: Carsten Haitzler <[email protected]>

Add a script to drive the asm pureloop test for arm64/CoreSight that
gathers data so it passes a minimum bar for amount and quality of
content that we extract from the kernel's perf support.

Signed-off-by: Carsten Haitzler <[email protected]>
---
.../tests/shell/coresight/asm_pure_loop.sh | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
create mode 100755 tools/perf/tests/shell/coresight/asm_pure_loop.sh

diff --git a/tools/perf/tests/shell/coresight/asm_pure_loop.sh b/tools/perf/tests/shell/coresight/asm_pure_loop.sh
new file mode 100755
index 000000000000..569e9d46162b
--- /dev/null
+++ b/tools/perf/tests/shell/coresight/asm_pure_loop.sh
@@ -0,0 +1,18 @@
+#!/bin/sh -e
+# CoreSight / ASM Pure Loop
+
+# SPDX-License-Identifier: GPL-2.0
+# Carsten Haitzler <[email protected]>, 2021
+
+TEST="asm_pure_loop"
+. $(dirname $0)/../lib/coresight.sh
+ARGS=""
+DATV="out"
+DATA="$DATD/perf-$TEST-$DATV.data"
+
+perf record $PERFRECOPT -o "$DATA" "$BIN" $ARGS
+
+perf_dump_aux_verify "$DATA" 10 10 10
+
+err=$?
+exit $err
--
2.32.0

2022-09-09 15:57:31

by Carsten Haitzler

[permalink] [raw]
Subject: [PATCH v9 09/13] perf test: Add thread loop test shell scripts

From: Carsten Haitzler <[email protected]>

Add a script to drive the thread loop test that gathers data so
it passes a minimum bar (in this case do we get any perf context data
for every thread).

Signed-off-by: Carsten Haitzler <[email protected]>
---
.../coresight/thread_loop_check_tid_10.sh | 19 +++++++++++++++++++
.../coresight/thread_loop_check_tid_2.sh | 19 +++++++++++++++++++
2 files changed, 38 insertions(+)
create mode 100755 tools/perf/tests/shell/coresight/thread_loop_check_tid_10.sh
create mode 100755 tools/perf/tests/shell/coresight/thread_loop_check_tid_2.sh

diff --git a/tools/perf/tests/shell/coresight/thread_loop_check_tid_10.sh b/tools/perf/tests/shell/coresight/thread_loop_check_tid_10.sh
new file mode 100755
index 000000000000..7c13636fc778
--- /dev/null
+++ b/tools/perf/tests/shell/coresight/thread_loop_check_tid_10.sh
@@ -0,0 +1,19 @@
+#!/bin/sh -e
+# CoreSight / Thread Loop 10 Threads - Check TID
+
+# SPDX-License-Identifier: GPL-2.0
+# Carsten Haitzler <[email protected]>, 2021
+
+TEST="thread_loop"
+. $(dirname $0)/../lib/coresight.sh
+ARGS="10 1"
+DATV="check-tid-10th"
+DATA="$DATD/perf-$TEST-$DATV.data"
+STDO="$DATD/perf-$TEST-$DATV.stdout"
+
+SHOW_TID=1 perf record -s $PERFRECOPT -o "$DATA" "$BIN" $ARGS > $STDO
+
+perf_dump_aux_tid_verify "$DATA" "$STDO"
+
+err=$?
+exit $err
diff --git a/tools/perf/tests/shell/coresight/thread_loop_check_tid_2.sh b/tools/perf/tests/shell/coresight/thread_loop_check_tid_2.sh
new file mode 100755
index 000000000000..a067145af43c
--- /dev/null
+++ b/tools/perf/tests/shell/coresight/thread_loop_check_tid_2.sh
@@ -0,0 +1,19 @@
+#!/bin/sh -e
+# CoreSight / Thread Loop 2 Threads - Check TID
+
+# SPDX-License-Identifier: GPL-2.0
+# Carsten Haitzler <[email protected]>, 2021
+
+TEST="thread_loop"
+. $(dirname $0)/../lib/coresight.sh
+ARGS="2 20"
+DATV="check-tid-2th"
+DATA="$DATD/perf-$TEST-$DATV.data"
+STDO="$DATD/perf-$TEST-$DATV.stdout"
+
+SHOW_TID=1 perf record -s $PERFRECOPT -o "$DATA" "$BIN" $ARGS > $STDO
+
+perf_dump_aux_tid_verify "$DATA" "$STDO"
+
+err=$?
+exit $err
--
2.32.0

2022-09-09 15:58:31

by Carsten Haitzler

[permalink] [raw]
Subject: [PATCH v9 13/13] perf test: Add relevant documentation about CoreSight testing

From: Carsten Haitzler <[email protected]>

Add/improve documentation helping people get started with CoreSight and
perf as well as describe the testing and how it works.

Cc: [email protected]
Signed-off-by: Carsten Haitzler <[email protected]>
---
.../trace/coresight/coresight-perf.rst | 158 ++++++++++++++++++
.../perf/Documentation/perf-arm-coresight.txt | 5 +
2 files changed, 163 insertions(+)
create mode 100644 Documentation/trace/coresight/coresight-perf.rst
create mode 100644 tools/perf/Documentation/perf-arm-coresight.txt

diff --git a/Documentation/trace/coresight/coresight-perf.rst b/Documentation/trace/coresight/coresight-perf.rst
new file mode 100644
index 000000000000..d087aae7d492
--- /dev/null
+++ b/Documentation/trace/coresight/coresight-perf.rst
@@ -0,0 +1,158 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+================
+CoreSight - Perf
+================
+
+ :Author: Carsten Haitzler <[email protected]>
+ :Date: June 29th, 2022
+
+Perf is able to locally access CoreSight trace data and store it to the
+output perf data files. This data can then be later decoded to give the
+instructions that were traced for debugging or profiling purposes. You
+can log such data with a perf record command like::
+
+ perf record -e cs_etm//u testbinary
+
+This would run some test binary (testbinary) until it exits and record
+a perf.data trace file. That file would have AUX sections if CoreSight
+is working correctly. You can dump the content of this file as
+readable text with a command like::
+
+ perf report --stdio --dump -i perf.data
+
+You should find some sections of this file have AUX data blocks like::
+
+ 0x1e78 [0x30]: PERF_RECORD_AUXTRACE size: 0x11dd0 offset: 0 ref: 0x1b614fc1061b0ad1 idx: 0 tid: 531230 cpu: -1
+
+ . ... CoreSight ETM Trace data: size 73168 bytes
+ Idx:0; ID:10; I_ASYNC : Alignment Synchronisation.
+ Idx:12; ID:10; I_TRACE_INFO : Trace Info.; INFO=0x0 { CC.0 }
+ Idx:17; ID:10; I_ADDR_L_64IS0 : Address, Long, 64 bit, IS0.; Addr=0x0000000000000000;
+ Idx:26; ID:10; I_TRACE_ON : Trace On.
+ Idx:27; ID:10; I_ADDR_CTXT_L_64IS0 : Address & Context, Long, 64 bit, IS0.; Addr=0x0000FFFFB6069140; Ctxt: AArch64,EL0, NS;
+ Idx:38; ID:10; I_ATOM_F6 : Atom format 6.; EEEEEEEEEEEEEEEEEEEEEEEE
+ Idx:39; ID:10; I_ATOM_F6 : Atom format 6.; EEEEEEEEEEEEEEEEEEEEEEEE
+ Idx:40; ID:10; I_ATOM_F6 : Atom format 6.; EEEEEEEEEEEEEEEEEEEEEEEE
+ Idx:41; ID:10; I_ATOM_F6 : Atom format 6.; EEEEEEEEEEEN
+ ...
+
+If you see these above, then your system is tracing CoreSight data
+correctly.
+
+To compile perf with CoreSight support in the tools/perf directory do::
+
+ make CORESIGHT=1
+
+This requires OpenCSD to build. You may install distribution packages
+for the support such as libopencsd and libopencsd-dev or download it
+and build yourself. Upstream OpenCSD is located at:
+
+ https://github.com/Linaro/OpenCSD
+
+For complete information on building perf with CoreSight support and
+more extensive usage look at:
+
+ https://github.com/Linaro/OpenCSD/blob/master/HOWTO.md
+
+
+Kernel CoreSight Support
+------------------------
+
+You will also want CoreSight support enabled in your kernel config.
+Ensure it is enabled with::
+
+ CONFIG_CORESIGHT=y
+
+There are various other CoreSight options you probably also want
+enabled like::
+
+ CONFIG_CORESIGHT_LINKS_AND_SINKS=y
+ CONFIG_CORESIGHT_LINK_AND_SINK_TMC=y
+ CONFIG_CORESIGHT_CATU=y
+ CONFIG_CORESIGHT_SINK_TPIU=y
+ CONFIG_CORESIGHT_SINK_ETBV10=y
+ CONFIG_CORESIGHT_SOURCE_ETM4X=y
+ CONFIG_CORESIGHT_CTI=y
+ CONFIG_CORESIGHT_CTI_INTEGRATION_REGS=y
+
+Please refer to the kernel configuration help for more information.
+
+Perf test - Verify kernel and userspace perf CoreSight work
+-----------------------------------------------------------
+
+When you run perf test, it will do a lot of self tests. Some of those
+tests will cover CoreSight (only if enabled and on ARM64). You
+generally would run perf test from the tools/perf directory in the
+kernel tree. Some tests will check some internal perf support like:
+
+ Check Arm CoreSight trace data recording and synthesized samples
+ Check Arm SPE trace data recording and synthesized samples
+
+Some others will actually use perf record and some test binaries that
+are in tests/shell/coresight and will collect traces to ensure a
+minimum level of functionality is met. The scripts that launch these
+tests are in the same directory. These will all look like:
+
+ CoreSight / ASM Pure Loop
+ CoreSight / Memcpy 16k 10 Threads
+ CoreSight / Thread Loop 10 Threads - Check TID
+ etc.
+
+These perf record tests will not run if the tool binaries do not exist
+in tests/shell/coresight/\*/ and will be skipped. If you do not have
+CoreSight support in hardware then either do not build perf with
+CoreSight support or remove these binaries in order to not have these
+tests fail and have them skip instead.
+
+These tests will log historical results in the current working
+directory (e.g. tools/perf) and will be named stats-\*.csv like:
+
+ stats-asm_pure_loop-out.csv
+ stats-memcpy_thread-16k_10.csv
+ ...
+
+These statistic files log some aspects of the AUX data sections in
+the perf data output counting some numbers of certain encodings (a
+good way to know that it's working in a very simple way). One problem
+with CoreSight is that given a large enough amount of data needing to
+be logged, some of it can be lost due to the processor not waking up
+in time to read out all the data from buffers etc.. You will notice
+that the amount of data collected can vary a lot per run of perf test.
+If you wish to see how this changes over time, simply run perf test
+multiple times and all these csv files will have more and more data
+appended to it that you can later examine, graph and otherwise use to
+figure out if things have become worse or better.
+
+This means sometimes these tests fail as they don't capture all the
+data needed. This is about tracking quality and amount of data
+produced over time and to see when changes to the Linux kernel improve
+quality of traces.
+
+Be aware that some of these tests take quite a while to run, specifically
+in processing the perf data file and dumping contents to then examine what
+is inside.
+
+You can change where these csv logs are stored by setting the
+PERF_TEST_CORESIGHT_STATDIR environment variable before running perf
+test like::
+
+ export PERF_TEST_CORESIGHT_STATDIR=/var/tmp
+ perf test
+
+They will also store resulting perf output data in the current
+directory for later inspection like::
+
+ perf-asm_pure_loop-out.data
+ perf-memcpy_thread-16k_10.data
+ ...
+
+You can alter where the perf data files are stored by setting the
+PERF_TEST_CORESIGHT_DATADIR environment variable such as::
+
+ PERF_TEST_CORESIGHT_DATADIR=/var/tmp
+ perf test
+
+You may wish to set these above environment variables if you wish to
+keep the output of tests outside of the current working directory for
+longer term storage and examination.
diff --git a/tools/perf/Documentation/perf-arm-coresight.txt b/tools/perf/Documentation/perf-arm-coresight.txt
new file mode 100644
index 000000000000..c117fc50a2a9
--- /dev/null
+++ b/tools/perf/Documentation/perf-arm-coresight.txt
@@ -0,0 +1,5 @@
+Arm CoreSight Support
+=====================
+
+For full documentation, see Documentation/trace/coresight/coresight-perf.rst
+in the kernel tree.
--
2.32.0

2022-09-09 15:58:57

by Carsten Haitzler

[permalink] [raw]
Subject: [PATCH v9 08/13] perf test: Add thread loop test tool

From: Carsten Haitzler <[email protected]>

Add test tool to be driven by further test scripts. This is a simple C
based loop with threads test to drive from scripts that can output TIDs
for tracking/checking.

Signed-off-by: Carsten Haitzler <[email protected]>
---
tools/perf/tests/shell/coresight/Makefile | 3 +-
.../shell/coresight/thread_loop/.gitignore | 1 +
.../shell/coresight/thread_loop/Makefile | 33 +++++++
.../shell/coresight/thread_loop/thread_loop.c | 86 +++++++++++++++++++
4 files changed, 122 insertions(+), 1 deletion(-)
create mode 100644 tools/perf/tests/shell/coresight/thread_loop/.gitignore
create mode 100644 tools/perf/tests/shell/coresight/thread_loop/Makefile
create mode 100644 tools/perf/tests/shell/coresight/thread_loop/thread_loop.c

diff --git a/tools/perf/tests/shell/coresight/Makefile b/tools/perf/tests/shell/coresight/Makefile
index 9a8526a74267..9330e4f62656 100644
--- a/tools/perf/tests/shell/coresight/Makefile
+++ b/tools/perf/tests/shell/coresight/Makefile
@@ -6,7 +6,8 @@ include ../../../../../tools/scripts/utilities.mak

SUBDIRS = \
asm_pure_loop \
- memcpy_thread
+ memcpy_thread \
+ thread_loop

all: $(SUBDIRS)
$(SUBDIRS):
diff --git a/tools/perf/tests/shell/coresight/thread_loop/.gitignore b/tools/perf/tests/shell/coresight/thread_loop/.gitignore
new file mode 100644
index 000000000000..6d4c33eaa9e8
--- /dev/null
+++ b/tools/perf/tests/shell/coresight/thread_loop/.gitignore
@@ -0,0 +1 @@
+thread_loop
diff --git a/tools/perf/tests/shell/coresight/thread_loop/Makefile b/tools/perf/tests/shell/coresight/thread_loop/Makefile
new file mode 100644
index 000000000000..ea846c038e7a
--- /dev/null
+++ b/tools/perf/tests/shell/coresight/thread_loop/Makefile
@@ -0,0 +1,33 @@
+# SPDX-License-Identifier: GPL-2.0
+# Carsten Haitzler <[email protected]>, 2021
+include ../Makefile.miniconfig
+
+# Binary to produce
+BIN=thread_loop
+# Any linking/libraries needed for the binary - empty if none needed
+LIB=-pthread
+
+all: $(BIN)
+
+$(BIN): $(BIN).c
+ifdef CORESIGHT
+ifeq ($(ARCH),arm64)
+# Build line
+ $(Q)$(CC) $(BIN).c -o $(BIN) $(LIB)
+endif
+endif
+
+install-tests: all
+ifdef CORESIGHT
+ifeq ($(ARCH),arm64)
+# Install the test tool in the right place
+ $(call QUIET_INSTALL, tests) \
+ $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/$(INSTDIR_SUB)/$(BIN)'; \
+ $(INSTALL) $(BIN) '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/$(INSTDIR_SUB)/$(BIN)/$(BIN)'
+endif
+endif
+
+clean:
+ $(Q)$(RM) -f $(BIN)
+
+.PHONY: all clean install-tests
diff --git a/tools/perf/tests/shell/coresight/thread_loop/thread_loop.c b/tools/perf/tests/shell/coresight/thread_loop/thread_loop.c
new file mode 100644
index 000000000000..c0158fac7d0b
--- /dev/null
+++ b/tools/perf/tests/shell/coresight/thread_loop/thread_loop.c
@@ -0,0 +1,86 @@
+// SPDX-License-Identifier: GPL-2.0
+// Carsten Haitzler <[email protected]>, 2021
+
+// define this for gettid()
+#define _GNU_SOURCE
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <string.h>
+#include <pthread.h>
+#include <sys/syscall.h>
+#ifndef SYS_gettid
+// gettid is 178 on arm64
+# define SYS_gettid 178
+#endif
+#define gettid() syscall(SYS_gettid)
+
+struct args {
+ unsigned int loops;
+ pthread_t th;
+ void *ret;
+};
+
+static void *thrfn(void *arg)
+{
+ struct args *a = arg;
+ int i = 0, len = a->loops;
+
+ if (getenv("SHOW_TID")) {
+ unsigned long long tid = gettid();
+
+ printf("%llu\n", tid);
+ }
+ asm volatile(
+ "loop:\n"
+ "add %[i], %[i], #1\n"
+ "cmp %[i], %[len]\n"
+ "blt loop\n"
+ : /* out */
+ : /* in */ [i] "r" (i), [len] "r" (len)
+ : /* clobber */
+ );
+ return (void *)(long)i;
+}
+
+static pthread_t new_thr(void *(*fn) (void *arg), void *arg)
+{
+ pthread_t t;
+ pthread_attr_t attr;
+
+ pthread_attr_init(&attr);
+ pthread_create(&t, &attr, fn, arg);
+ return t;
+}
+
+int main(int argc, char **argv)
+{
+ unsigned int i, len, thr;
+ pthread_t threads[256];
+ struct args args[256];
+
+ if (argc < 3) {
+ printf("ERR: %s [numthreads] [numloops (millions)]\n", argv[0]);
+ exit(1);
+ }
+
+ thr = atoi(argv[1]);
+ if ((thr < 1) || (thr > 256)) {
+ printf("ERR: threads 1-256\n");
+ exit(1);
+ }
+ len = atoi(argv[2]);
+ if ((len < 1) || (len > 4000)) {
+ printf("ERR: max loops 4000 (millions)\n");
+ exit(1);
+ }
+ len *= 1000000;
+ for (i = 0; i < thr; i++) {
+ args[i].loops = len;
+ args[i].th = new_thr(thrfn, &(args[i]));
+ }
+ for (i = 0; i < thr; i++)
+ pthread_join(args[i].th, &(args[i].ret));
+ return 0;
+}
--
2.32.0

2022-09-09 16:24:59

by Carsten Haitzler

[permalink] [raw]
Subject: [PATCH v9 07/13] perf test: Add memcpy thread test shell script

From: Carsten Haitzler <[email protected]>

Add a script to drive the threaded memcpy test that gathers data so
it passes a minimum bar for amount and quality of content that we
extract from the kernel's perf support.

Signed-off-by: Carsten Haitzler <[email protected]>
---
.../shell/coresight/memcpy_thread_16k_10.sh | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
create mode 100755 tools/perf/tests/shell/coresight/memcpy_thread_16k_10.sh

diff --git a/tools/perf/tests/shell/coresight/memcpy_thread_16k_10.sh b/tools/perf/tests/shell/coresight/memcpy_thread_16k_10.sh
new file mode 100755
index 000000000000..d21ba8545938
--- /dev/null
+++ b/tools/perf/tests/shell/coresight/memcpy_thread_16k_10.sh
@@ -0,0 +1,18 @@
+#!/bin/sh -e
+# CoreSight / Memcpy 16k 10 Threads
+
+# SPDX-License-Identifier: GPL-2.0
+# Carsten Haitzler <[email protected]>, 2021
+
+TEST="memcpy_thread"
+. $(dirname $0)/../lib/coresight.sh
+ARGS="16 10 1"
+DATV="16k_10"
+DATA="$DATD/perf-$TEST-$DATV.data"
+
+perf record $PERFRECOPT -o "$DATA" "$BIN" $ARGS
+
+perf_dump_aux_verify "$DATA" 10 10 10
+
+err=$?
+exit $err
--
2.32.0

2022-10-06 11:55:14

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: [PATCH v9 02/13] perf test: Add build infra for perf test tools for CoreSight tests

Em Thu, Oct 06, 2022 at 08:26:00AM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Fri, Sep 09, 2022 at 04:27:52PM +0100, [email protected] escreveu:
> > +all: shell_compatibility_test $(ALL_PROGRAMS) $(LANG_BINDINGS) $(OTHER_PROGRAMS) tests-coresight-targets
> >
> > # Create python binding output directory if not already present
> > _dummy := $(shell [ -d '$(OUTPUT)python' ] || mkdir -p '$(OUTPUT)python')
> > @@ -1007,6 +1015,9 @@ install-tests: all install-gtk
> > $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > $(INSTALL) tests/shell/lib/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > $(INSTALL) tests/shell/lib/*.py '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'
> > + $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'; \
> > + $(INSTALL) tests/shell/coresight/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
> > + $(Q)$(MAKE) -C tests/shell/coresight install-tests
>
> Added a missing ;\ on the line installing the *.py and adjusted use -m
> 644 for the .sh files to follow what was done in:
>
> 0a9eaf616f29ca32 ("perf tools: Don't install data files with x permissions")
>
> @@ -1006,7 +1014,10 @@ install-tests: all install-gtk
> $(INSTALL) tests/shell/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell'; \
> $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> $(INSTALL) tests/shell/lib/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> - $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'
> + $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> + $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'; \
> + $(INSTALL) tests/shell/coresight/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
> + $(Q)$(MAKE) -C tests/shell/coresight install-tests
>

Also had to remove:

$(INSTALL) tests/shell/coresight/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'

from this patch, as it makes install fail at this point in the patchset:

DESCEND plugins
GEN /tmp/build/perf/python/perf.so
INSTALL trace_plugins
INSTALL binaries
INSTALL tests
install: cannot stat 'tests/shell/coresight/*.sh': No such file or directory
make[2]: *** [Makefile.perf:1007: install-tests] Error 1
make[2]: *** Waiting for unfinished jobs....

I'll add it back when the first .sh gets added to tests/shell/coresight/

- Arnaldo

2022-10-06 12:28:41

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: [PATCH v9 02/13] perf test: Add build infra for perf test tools for CoreSight tests

Em Fri, Sep 09, 2022 at 04:27:52PM +0100, [email protected] escreveu:
> From: Carsten Haitzler <[email protected]>
>
> This adds the initial build infrastructure (makefiles maintainers
> information) for adding follow-on tests for CoreSight.
>
> Signed-off-by: Carsten Haitzler <[email protected]>
> ---
> MAINTAINERS | 1 +
> tools/perf/Makefile.config | 2 ++
> tools/perf/Makefile.perf | 15 +++++++++--
> tools/perf/tests/shell/coresight/Makefile | 25 +++++++++++++++++++
> .../tests/shell/coresight/Makefile.miniconfig | 14 +++++++++++
> 5 files changed, 55 insertions(+), 2 deletions(-)
> create mode 100644 tools/perf/tests/shell/coresight/Makefile
> create mode 100644 tools/perf/tests/shell/coresight/Makefile.miniconfig
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 9d7f64dc0efe..c8ae5a6638b6 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -2029,6 +2029,7 @@ F: drivers/hwtracing/coresight/*
> F: include/dt-bindings/arm/coresight-cti-dt.h
> F: include/linux/coresight*
> F: samples/coresight/*
> +F: tools/perf/tests/shell/coresight/*
> F: tools/perf/arch/arm/util/auxtrace.c
> F: tools/perf/arch/arm/util/cs-etm.c
> F: tools/perf/arch/arm/util/cs-etm.h
> diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config
> index 0661a1cf9855..e3e28b3481f9 100644
> --- a/tools/perf/Makefile.config
> +++ b/tools/perf/Makefile.config
> @@ -1291,6 +1291,8 @@ perf_examples_instdir_SQ = $(subst ','\'',$(perf_examples_instdir))
> STRACE_GROUPS_INSTDIR_SQ = $(subst ','\'',$(STRACE_GROUPS_INSTDIR))
> tip_instdir_SQ = $(subst ','\'',$(tip_instdir))
>
> +export perfexec_instdir_SQ
> +
> # If we install to $(HOME) we keep the traceevent default:
> # $(HOME)/.traceevent/plugins
> # Otherwise we install plugins into the global $(libdir).
> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index e5921b347153..cef2a06c8f54 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -629,7 +629,15 @@ sync_file_range_tbls := $(srctree)/tools/perf/trace/beauty/sync_file_range.sh
> $(sync_file_range_arrays): $(linux_uapi_dir)/fs.h $(sync_file_range_tbls)
> $(Q)$(SHELL) '$(sync_file_range_tbls)' $(linux_uapi_dir) > $@
>
> -all: shell_compatibility_test $(ALL_PROGRAMS) $(LANG_BINDINGS) $(OTHER_PROGRAMS)
> +TESTS_CORESIGHT_DIR := $(srctree)/tools/perf/tests/shell/coresight
> +
> +tests-coresight-targets: FORCE
> + $(Q)$(MAKE) -C $(TESTS_CORESIGHT_DIR)
> +
> +tests-coresight-targets-clean:
> + $(Q)$(MAKE) -C $(TESTS_CORESIGHT_DIR) clean
> +
> +all: shell_compatibility_test $(ALL_PROGRAMS) $(LANG_BINDINGS) $(OTHER_PROGRAMS) tests-coresight-targets
>
> # Create python binding output directory if not already present
> _dummy := $(shell [ -d '$(OUTPUT)python' ] || mkdir -p '$(OUTPUT)python')
> @@ -1007,6 +1015,9 @@ install-tests: all install-gtk
> $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> $(INSTALL) tests/shell/lib/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> $(INSTALL) tests/shell/lib/*.py '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'
> + $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'; \
> + $(INSTALL) tests/shell/coresight/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
> + $(Q)$(MAKE) -C tests/shell/coresight install-tests

Added a missing ;\ on the line installing the *.py and adjusted use -m
644 for the .sh files to follow what was done in:

0a9eaf616f29ca32 ("perf tools: Don't install data files with x permissions")

@@ -1006,7 +1014,10 @@ install-tests: all install-gtk
$(INSTALL) tests/shell/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell'; \
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
$(INSTALL) tests/shell/lib/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
- $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'
+ $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
+ $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'; \
+ $(INSTALL) tests/shell/coresight/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
+ $(Q)$(MAKE) -C tests/shell/coresight install-tests


> install-bin: install-tools install-tests install-traceevent-plugins
>
> @@ -1077,7 +1088,7 @@ endif # BUILD_BPF_SKEL
> bpf-skel-clean:
> $(call QUIET_CLEAN, bpf-skel) $(RM) -r $(SKEL_TMP_OUT) $(SKELETONS)
>
> -clean:: $(LIBTRACEEVENT)-clean $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clean $(LIBPERF)-clean fixdep-clean python-clean bpf-skel-clean
> +clean:: $(LIBTRACEEVENT)-clean $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clean $(LIBPERF)-clean fixdep-clean python-clean bpf-skel-clean tests-coresight-targets-clean
> $(call QUIET_CLEAN, core-objs) $(RM) $(LIBPERF_A) $(OUTPUT)perf-archive $(OUTPUT)perf-iostat $(LANG_BINDINGS)
> $(Q)find $(or $(OUTPUT),.) -name '*.o' -delete -o -name '\.*.cmd' -delete -o -name '\.*.d' -delete
> $(Q)$(RM) $(OUTPUT).config-detected
> diff --git a/tools/perf/tests/shell/coresight/Makefile b/tools/perf/tests/shell/coresight/Makefile
> new file mode 100644
> index 000000000000..3fee05cfcb0e
> --- /dev/null
> +++ b/tools/perf/tests/shell/coresight/Makefile
> @@ -0,0 +1,25 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +# Carsten Haitzler <[email protected]>, 2021
> +include ../../../../../tools/scripts/Makefile.include
> +include ../../../../../tools/scripts/Makefile.arch
> +include ../../../../../tools/scripts/utilities.mak
> +
> +SUBDIRS =
> +
> +all: $(SUBDIRS)
> +$(SUBDIRS):
> + $(Q)$(MAKE) -C $@
> +
> +INSTALLDIRS = $(SUBDIRS:%=install-%)
> +
> +install-tests: $(INSTALLDIRS)
> +$(INSTALLDIRS):
> + $(Q)$(MAKE) -C $(@:install-%=%) install-tests
> +
> +CLEANDIRS = $(SUBDIRS:%=clean-%)
> +
> +clean: $(CLEANDIRS)
> +$(CLEANDIRS):
> + $(Q)$(MAKE) -C $(@:clean-%=%) clean >/dev/null
> +
> +.PHONY: all clean $(SUBDIRS) $(CLEANDIRS) $(INSTALLDIRS)
> diff --git a/tools/perf/tests/shell/coresight/Makefile.miniconfig b/tools/perf/tests/shell/coresight/Makefile.miniconfig
> new file mode 100644
> index 000000000000..5f72a9cb43f3
> --- /dev/null
> +++ b/tools/perf/tests/shell/coresight/Makefile.miniconfig
> @@ -0,0 +1,14 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +# Carsten Haitzler <[email protected]>, 2021
> +
> +ifndef DESTDIR
> +prefix ?= $(HOME)
> +endif
> +
> +DESTDIR_SQ = $(subst ','\'',$(DESTDIR))
> +INSTALL = install
> +INSTDIR_SUB = tests/shell/coresight
> +
> +include ../../../../../scripts/Makefile.include
> +include ../../../../../scripts/Makefile.arch
> +include ../../../../../scripts/utilities.mak
> --
> 2.32.0

--

- Arnaldo

2022-10-06 13:16:58

by Leo Yan

[permalink] [raw]
Subject: Re: [PATCH v9 02/13] perf test: Add build infra for perf test tools for CoreSight tests

Hi Arnaldo,

On Thu, Oct 06, 2022 at 08:45:18AM -0300, Arnaldo Carvalho de Melo wrote:
> Em Thu, Oct 06, 2022 at 08:26:00AM -0300, Arnaldo Carvalho de Melo escreveu:
> > Em Fri, Sep 09, 2022 at 04:27:52PM +0100, [email protected] escreveu:
> > > +all: shell_compatibility_test $(ALL_PROGRAMS) $(LANG_BINDINGS) $(OTHER_PROGRAMS) tests-coresight-targets
> > >
> > > # Create python binding output directory if not already present
> > > _dummy := $(shell [ -d '$(OUTPUT)python' ] || mkdir -p '$(OUTPUT)python')
> > > @@ -1007,6 +1015,9 @@ install-tests: all install-gtk
> > > $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > > $(INSTALL) tests/shell/lib/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > > $(INSTALL) tests/shell/lib/*.py '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'
> > > + $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'; \
> > > + $(INSTALL) tests/shell/coresight/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
> > > + $(Q)$(MAKE) -C tests/shell/coresight install-tests
> >
> > Added a missing ;\ on the line installing the *.py and adjusted use -m
> > 644 for the .sh files to follow what was done in:
> >
> > 0a9eaf616f29ca32 ("perf tools: Don't install data files with x permissions")
> >
> > @@ -1006,7 +1014,10 @@ install-tests: all install-gtk
> > $(INSTALL) tests/shell/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell'; \
> > $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > $(INSTALL) tests/shell/lib/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > - $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'
> > + $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > + $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'; \
> > + $(INSTALL) tests/shell/coresight/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
> > + $(Q)$(MAKE) -C tests/shell/coresight install-tests
> >
>
> Also had to remove:
>
> $(INSTALL) tests/shell/coresight/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
>
> from this patch, as it makes install fail at this point in the patchset:

Thanks a lot!

James and me had found the merging conflict and planned to send out
email for reminding it, the right change for above section would be:

@@ -1006,7 +1014,10 @@ install-tests: all install-gtk
$(INSTALL) tests/shell/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell'; \
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
$(INSTALL) tests/shell/lib/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
- $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'
+ $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
+ $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'; \
+ $(INSTALL) tests/shell/coresight/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
+ $(Q)$(MAKE) -C tests/shell/coresight install-tests

It's deliberately to not add '-m 644' when install coresight test
shell scripts under the folder tests/shell/coresight/ so that we can
keep the executable permission for these scripts. On the other flip,
we set 644 mode for the scripts under tests/shell/lib/ so these
scripts will not be executed directly by perf test framework.

@Carsten, if have chance could you confirm for above change?

> DESCEND plugins
> GEN /tmp/build/perf/python/perf.so
> INSTALL trace_plugins
> INSTALL binaries
> INSTALL tests
> install: cannot stat 'tests/shell/coresight/*.sh': No such file or directory
> make[2]: *** [Makefile.perf:1007: install-tests] Error 1
> make[2]: *** Waiting for unfinished jobs....
>
> I'll add it back when the first .sh gets added to tests/shell/coresight/

Sorry for introducing extra efforts for you and thanks!

Leo

2022-10-06 14:50:16

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: [PATCH v9 02/13] perf test: Add build infra for perf test tools for CoreSight tests

Em Thu, Oct 06, 2022 at 08:58:42PM +0800, Leo Yan escreveu:
> Hi Arnaldo,
>
> On Thu, Oct 06, 2022 at 08:45:18AM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Thu, Oct 06, 2022 at 08:26:00AM -0300, Arnaldo Carvalho de Melo escreveu:
> > > Em Fri, Sep 09, 2022 at 04:27:52PM +0100, [email protected] escreveu:
> > > > +all: shell_compatibility_test $(ALL_PROGRAMS) $(LANG_BINDINGS) $(OTHER_PROGRAMS) tests-coresight-targets
> > > >
> > > > # Create python binding output directory if not already present
> > > > _dummy := $(shell [ -d '$(OUTPUT)python' ] || mkdir -p '$(OUTPUT)python')
> > > > @@ -1007,6 +1015,9 @@ install-tests: all install-gtk
> > > > $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > > > $(INSTALL) tests/shell/lib/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > > > $(INSTALL) tests/shell/lib/*.py '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'
> > > > + $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'; \
> > > > + $(INSTALL) tests/shell/coresight/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
> > > > + $(Q)$(MAKE) -C tests/shell/coresight install-tests
> > >
> > > Added a missing ;\ on the line installing the *.py and adjusted use -m
> > > 644 for the .sh files to follow what was done in:
> > >
> > > 0a9eaf616f29ca32 ("perf tools: Don't install data files with x permissions")
> > >
> > > @@ -1006,7 +1014,10 @@ install-tests: all install-gtk
> > > $(INSTALL) tests/shell/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell'; \
> > > $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > > $(INSTALL) tests/shell/lib/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > > - $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'
> > > + $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > > + $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'; \
> > > + $(INSTALL) tests/shell/coresight/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
> > > + $(Q)$(MAKE) -C tests/shell/coresight install-tests
> > >
> >
> > Also had to remove:
> >
> > $(INSTALL) tests/shell/coresight/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
> >
> > from this patch, as it makes install fail at this point in the patchset:
>
> Thanks a lot!
>
> James and me had found the merging conflict and planned to send out
> email for reminding it, the right change for above section would be:

Not at this patch, in this patch there are no tests/shell/coresight/*.sh
which makes 'make install' to fail _at this point_, for bisectability we
can't just test at the end of the patchkit.

> @@ -1006,7 +1014,10 @@ install-tests: all install-gtk
> $(INSTALL) tests/shell/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell'; \
> $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> $(INSTALL) tests/shell/lib/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> - $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'
> + $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> + $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'; \
> + $(INSTALL) tests/shell/coresight/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
> + $(Q)$(MAKE) -C tests/shell/coresight install-tests
>
> It's deliberately to not add '-m 644' when install coresight test
> shell scripts under the folder tests/shell/coresight/ so that we can
> keep the executable permission for these scripts. On the other flip,
> we set 644 mode for the scripts under tests/shell/lib/ so these
> scripts will not be executed directly by perf test framework.
>
> @Carsten, if have chance could you confirm for above change?

If you need for some .sh, coresight test's to have x set while all of
the others don't have, and you have a good reason for that, ok, we just
add a comment.

> > DESCEND plugins
> > GEN /tmp/build/perf/python/perf.so
> > INSTALL trace_plugins
> > INSTALL binaries
> > INSTALL tests
> > install: cannot stat 'tests/shell/coresight/*.sh': No such file or directory
> > make[2]: *** [Makefile.perf:1007: install-tests] Error 1
> > make[2]: *** Waiting for unfinished jobs....
> >
> > I'll add it back when the first .sh gets added to tests/shell/coresight/
>
> Sorry for introducing extra efforts for you and thanks!

This also has to go, I'll check and fix this loop that doesn't follow
the existing output style, happens with ´make install', etc too:

⬢[acme@toolbox perf]$ make -C tools/perf clean
make: Entering directory '/var/home/acme/git/perf/tools/perf'
CLEAN x86
CLEAN libtraceevent
CLEAN libapi
CLEAN libbpf
CLEAN libsubcmd
CLEAN libperf
CLEAN fixdep
CLEAN feature-detect
CLEAN python
CLEAN bpf-skel
make -C asm_pure_loop clean >/dev/null <-------------------
make -C memcpy_thread clean >/dev/null <-------------------
make -C thread_loop clean >/dev/null <-------------------
make -C unroll_loop_thread clean >/dev/null <-------------------
CLEAN core-objs
CLEAN core-progs
CLEAN core-gen
CLEAN Documentation
make: Leaving directory '/var/home/acme/git/perf/tools/perf'
⬢[acme@toolbox perf]$

2022-10-06 15:41:52

by Leo Yan

[permalink] [raw]
Subject: Re: [PATCH v9 02/13] perf test: Add build infra for perf test tools for CoreSight tests

On Thu, Oct 06, 2022 at 11:22:13AM -0300, Arnaldo Carvalho de Melo wrote:

[...]

> > @@ -1006,7 +1014,10 @@ install-tests: all install-gtk
> > $(INSTALL) tests/shell/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell'; \
> > $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > $(INSTALL) tests/shell/lib/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > - $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'
> > + $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > + $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'; \
> > + $(INSTALL) tests/shell/coresight/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
> > + $(Q)$(MAKE) -C tests/shell/coresight install-tests
> >
> > It's deliberately to not add '-m 644' when install coresight test
> > shell scripts under the folder tests/shell/coresight/ so that we can
> > keep the executable permission for these scripts. On the other flip,
> > we set 644 mode for the scripts under tests/shell/lib/ so these
> > scripts will not be executed directly by perf test framework.
> >
> > @Carsten, if have chance could you confirm for above change?
>
> If you need for some .sh, coresight test's to have x set while all of
> the others don't have, and you have a good reason for that, ok, we just
> add a comment.

Simply to say, all lib scripts are not needed for execute permission.
Yes, it's good to add a comment for the installation steps.

> > > DESCEND plugins
> > > GEN /tmp/build/perf/python/perf.so
> > > INSTALL trace_plugins
> > > INSTALL binaries
> > > INSTALL tests
> > > install: cannot stat 'tests/shell/coresight/*.sh': No such file or directory
> > > make[2]: *** [Makefile.perf:1007: install-tests] Error 1
> > > make[2]: *** Waiting for unfinished jobs....
> > >
> > > I'll add it back when the first .sh gets added to tests/shell/coresight/
> >
> > Sorry for introducing extra efforts for you and thanks!
>
> This also has to go, I'll check and fix this loop that doesn't follow
> the existing output style, happens with ´make install', etc too:
>
> ⬢[acme@toolbox perf]$ make -C tools/perf clean
> make: Entering directory '/var/home/acme/git/perf/tools/perf'
> CLEAN x86
> CLEAN libtraceevent
> CLEAN libapi
> CLEAN libbpf
> CLEAN libsubcmd
> CLEAN libperf
> CLEAN fixdep
> CLEAN feature-detect
> CLEAN python
> CLEAN bpf-skel
> make -C asm_pure_loop clean >/dev/null <-------------------
> make -C memcpy_thread clean >/dev/null <-------------------
> make -C thread_loop clean >/dev/null <-------------------
> make -C unroll_loop_thread clean >/dev/null <-------------------

Ouch, if we can update the file tests/shell/coresight/Makefile as below:

$(CLEANDIRS):
$(call QUIET_CLEAN, test-$(@:clean-%=%)) $(Q)$(MAKE) -C $(@:clean-%=%) clean >/dev/null

Then we can get output as:

# make clean
CLEAN arm64
CLEAN libtraceevent
CLEAN libapi
CLEAN libbpf
CLEAN libsubcmd
CLEAN libperf
CLEAN fixdep
CLEAN feature-detect
CLEAN python
CLEAN bpf-skel
CLEAN test-asm_pure_loop
CLEAN test-memcpy_thread
CLEAN test-thread_loop
CLEAN test-unroll_loop_thread
CLEAN core-objs
CLEAN core-progs
CLEAN core-gen
CLEAN Documentation

Sorry for inconvenience.

Thanks,
Leo

2022-10-06 17:15:45

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: [PATCH v9 02/13] perf test: Add build infra for perf test tools for CoreSight tests

Em Thu, Oct 06, 2022 at 11:07:19PM +0800, Leo Yan escreveu:
> On Thu, Oct 06, 2022 at 11:22:13AM -0300, Arnaldo Carvalho de Melo wrote:
>
> [...]
>
> > > @@ -1006,7 +1014,10 @@ install-tests: all install-gtk
> > > $(INSTALL) tests/shell/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell'; \
> > > $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > > $(INSTALL) tests/shell/lib/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > > - $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'
> > > + $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > > + $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'; \
> > > + $(INSTALL) tests/shell/coresight/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
> > > + $(Q)$(MAKE) -C tests/shell/coresight install-tests
> > >
> > > It's deliberately to not add '-m 644' when install coresight test
> > > shell scripts under the folder tests/shell/coresight/ so that we can
> > > keep the executable permission for these scripts. On the other flip,
> > > we set 644 mode for the scripts under tests/shell/lib/ so these
> > > scripts will not be executed directly by perf test framework.
> > >
> > > @Carsten, if have chance could you confirm for above change?
> >
> > If you need for some .sh, coresight test's to have x set while all of
> > the others don't have, and you have a good reason for that, ok, we just
> > add a comment.
>
> Simply to say, all lib scripts are not needed for execute permission.
> Yes, it's good to add a comment for the installation steps.
>
> > > > DESCEND plugins
> > > > GEN /tmp/build/perf/python/perf.so
> > > > INSTALL trace_plugins
> > > > INSTALL binaries
> > > > INSTALL tests
> > > > install: cannot stat 'tests/shell/coresight/*.sh': No such file or directory
> > > > make[2]: *** [Makefile.perf:1007: install-tests] Error 1
> > > > make[2]: *** Waiting for unfinished jobs....
> > > >
> > > > I'll add it back when the first .sh gets added to tests/shell/coresight/
> > >
> > > Sorry for introducing extra efforts for you and thanks!
> >
> > This also has to go, I'll check and fix this loop that doesn't follow
> > the existing output style, happens with ´make install', etc too:
> >
> > ⬢[acme@toolbox perf]$ make -C tools/perf clean
> > make: Entering directory '/var/home/acme/git/perf/tools/perf'
> > CLEAN x86
> > CLEAN libtraceevent
> > CLEAN libapi
> > CLEAN libbpf
> > CLEAN libsubcmd
> > CLEAN libperf
> > CLEAN fixdep
> > CLEAN feature-detect
> > CLEAN python
> > CLEAN bpf-skel
> > make -C asm_pure_loop clean >/dev/null <-------------------
> > make -C memcpy_thread clean >/dev/null <-------------------
> > make -C thread_loop clean >/dev/null <-------------------
> > make -C unroll_loop_thread clean >/dev/null <-------------------
>
> Ouch, if we can update the file tests/shell/coresight/Makefile as below:
>
> $(CLEANDIRS):
> $(call QUIET_CLEAN, test-$(@:clean-%=%)) $(Q)$(MAKE) -C $(@:clean-%=%) clean >/dev/null

I had to do it, so far, at the time that the above line is first added
as:

diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index a20b4adb5e3451f1..194e582e70c2c766 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -635,7 +635,8 @@ tests-coresight-targets: FORCE
$(Q)$(MAKE) -C $(TESTS_CORESIGHT_DIR)

tests-coresight-targets-clean:
- $(Q)$(MAKE) -C $(TESTS_CORESIGHT_DIR) clean
+ $(call QUIET_CLEAN, coresight)
+ $(Q)$(MAKE) -C $(TESTS_CORESIGHT_DIR) O=$(OUTPUT) clean >/dev/null

all: shell_compatibility_test $(ALL_PROGRAMS) $(LANG_BINDINGS) $(OTHER_PROGRAMS) tests-coresight-targets

diff --git a/tools/perf/tests/shell/coresight/Makefile b/tools/perf/tests/shell/coresight/Makefile
index 3fee05cfcb0e73f6..6baab5cbaa9aa3b1 100644
--- a/tools/perf/tests/shell/coresight/Makefile
+++ b/tools/perf/tests/shell/coresight/Makefile
@@ -20,6 +20,6 @@ CLEANDIRS = $(SUBDIRS:%=clean-%)

clean: $(CLEANDIRS)
$(CLEANDIRS):
- $(Q)$(MAKE) -C $(@:clean-%=%) clean >/dev/null
+ $(call QUIET_CLEAN, test-$(@:clean-%=%)) $(Q)$(MAKE) -C $(@:clean-%=%) clean >/dev/null

.PHONY: all clean $(SUBDIRS) $(CLEANDIRS) $(INSTALLDIRS)

> Then we can get output as:
>
> # make clean
> CLEAN arm64
> CLEAN libtraceevent
> CLEAN libapi
> CLEAN libbpf
> CLEAN libsubcmd
> CLEAN libperf
> CLEAN fixdep
> CLEAN feature-detect
> CLEAN python
> CLEAN bpf-skel
> CLEAN test-asm_pure_loop
> CLEAN test-memcpy_thread
> CLEAN test-thread_loop
> CLEAN test-unroll_loop_thread
> CLEAN core-objs
> CLEAN core-progs
> CLEAN core-gen
> CLEAN Documentation
>
> Sorry for inconvenience.
>
> Thanks,
> Leo

--

- Arnaldo

2022-10-06 18:14:41

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: [PATCH v9 02/13] perf test: Add build infra for perf test tools for CoreSight tests

Em Thu, Oct 06, 2022 at 02:12:11PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Thu, Oct 06, 2022 at 11:07:19PM +0800, Leo Yan escreveu:
> > On Thu, Oct 06, 2022 at 11:22:13AM -0300, Arnaldo Carvalho de Melo wrote:
> >
> > [...]
> >
> > > > @@ -1006,7 +1014,10 @@ install-tests: all install-gtk
> > > > $(INSTALL) tests/shell/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell'; \
> > > > $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > > > $(INSTALL) tests/shell/lib/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > > > - $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'
> > > > + $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > > > + $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'; \
> > > > + $(INSTALL) tests/shell/coresight/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
> > > > + $(Q)$(MAKE) -C tests/shell/coresight install-tests
> > > >
> > > > It's deliberately to not add '-m 644' when install coresight test
> > > > shell scripts under the folder tests/shell/coresight/ so that we can
> > > > keep the executable permission for these scripts. On the other flip,
> > > > we set 644 mode for the scripts under tests/shell/lib/ so these
> > > > scripts will not be executed directly by perf test framework.
> > > >
> > > > @Carsten, if have chance could you confirm for above change?
> > >
> > > If you need for some .sh, coresight test's to have x set while all of
> > > the others don't have, and you have a good reason for that, ok, we just
> > > add a comment.
> >
> > Simply to say, all lib scripts are not needed for execute permission.
> > Yes, it's good to add a comment for the installation steps.
> >
> > > > > DESCEND plugins
> > > > > GEN /tmp/build/perf/python/perf.so
> > > > > INSTALL trace_plugins
> > > > > INSTALL binaries
> > > > > INSTALL tests
> > > > > install: cannot stat 'tests/shell/coresight/*.sh': No such file or directory
> > > > > make[2]: *** [Makefile.perf:1007: install-tests] Error 1
> > > > > make[2]: *** Waiting for unfinished jobs....
> > > > >
> > > > > I'll add it back when the first .sh gets added to tests/shell/coresight/
> > > >
> > > > Sorry for introducing extra efforts for you and thanks!
> > >
> > > This also has to go, I'll check and fix this loop that doesn't follow
> > > the existing output style, happens with ´make install', etc too:
> > >
> > > ⬢[acme@toolbox perf]$ make -C tools/perf clean
> > > make: Entering directory '/var/home/acme/git/perf/tools/perf'
> > > CLEAN x86
> > > CLEAN libtraceevent
> > > CLEAN libapi
> > > CLEAN libbpf
> > > CLEAN libsubcmd
> > > CLEAN libperf
> > > CLEAN fixdep
> > > CLEAN feature-detect
> > > CLEAN python
> > > CLEAN bpf-skel
> > > make -C asm_pure_loop clean >/dev/null <-------------------
> > > make -C memcpy_thread clean >/dev/null <-------------------
> > > make -C thread_loop clean >/dev/null <-------------------
> > > make -C unroll_loop_thread clean >/dev/null <-------------------
> >
> > Ouch, if we can update the file tests/shell/coresight/Makefile as below:
> >
> > $(CLEANDIRS):
> > $(call QUIET_CLEAN, test-$(@:clean-%=%)) $(Q)$(MAKE) -C $(@:clean-%=%) clean >/dev/null
>
> I had to do it, so far, at the time that the above line is first added
> as:
>
> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index a20b4adb5e3451f1..194e582e70c2c766 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -635,7 +635,8 @@ tests-coresight-targets: FORCE
> $(Q)$(MAKE) -C $(TESTS_CORESIGHT_DIR)
>
> tests-coresight-targets-clean:
> - $(Q)$(MAKE) -C $(TESTS_CORESIGHT_DIR) clean
> + $(call QUIET_CLEAN, coresight)
> + $(Q)$(MAKE) -C $(TESTS_CORESIGHT_DIR) O=$(OUTPUT) clean >/dev/null
>
> all: shell_compatibility_test $(ALL_PROGRAMS) $(LANG_BINDINGS) $(OTHER_PROGRAMS) tests-coresight-targets
>
> diff --git a/tools/perf/tests/shell/coresight/Makefile b/tools/perf/tests/shell/coresight/Makefile
> index 3fee05cfcb0e73f6..6baab5cbaa9aa3b1 100644
> --- a/tools/perf/tests/shell/coresight/Makefile
> +++ b/tools/perf/tests/shell/coresight/Makefile
> @@ -20,6 +20,6 @@ CLEANDIRS = $(SUBDIRS:%=clean-%)
>
> clean: $(CLEANDIRS)
> $(CLEANDIRS):
> - $(Q)$(MAKE) -C $(@:clean-%=%) clean >/dev/null
> + $(call QUIET_CLEAN, test-$(@:clean-%=%)) $(Q)$(MAKE) -C $(@:clean-%=%) clean >/dev/null
>
> .PHONY: all clean $(SUBDIRS) $(CLEANDIRS) $(INSTALLDIRS)

And for the normal build and install, somehow that $(Q) is not being
obeyed there, so I'm using straight @ for now so that we have a clean
output, someone more knowledgeable in the Makefile voodoo can then fix
it.

⬢[acme@toolbox perf]$ git log --oneline -1
e1700b67a41092a5 (HEAD) perf test: Add build infra for perf test tools for ARM CoreSight tests
⬢[acme@toolbox perf]$ git diff
diff --git a/tools/perf/tests/shell/coresight/Makefile b/tools/perf/tests/shell/coresight/Makefile
index 6baab5cbaa9aa3b1..6f3b3f07049c2d53 100644
--- a/tools/perf/tests/shell/coresight/Makefile
+++ b/tools/perf/tests/shell/coresight/Makefile
@@ -8,13 +8,13 @@ SUBDIRS =

all: $(SUBDIRS)
$(SUBDIRS):
- $(Q)$(MAKE) -C $@
+ @$(MAKE) -C $@

INSTALLDIRS = $(SUBDIRS:%=install-%)

install-tests: $(INSTALLDIRS)
$(INSTALLDIRS):
- $(Q)$(MAKE) -C $(@:install-%=%) install-tests
+ @$(MAKE) -C $(@:install-%=%) install-tests >/dev/null

CLEANDIRS = $(SUBDIRS:%=clean-%)

⬢[acme@toolbox perf]$

2022-10-07 07:07:05

by Jiri Slaby

[permalink] [raw]
Subject: Re: [PATCH v9 02/13] perf test: Add build infra for perf test tools for CoreSight tests

On 06. 10. 22, 19:42, Arnaldo Carvalho de Melo wrote:
> Em Thu, Oct 06, 2022 at 02:12:11PM -0300, Arnaldo Carvalho de Melo escreveu:
>> Em Thu, Oct 06, 2022 at 11:07:19PM +0800, Leo Yan escreveu:
>>> On Thu, Oct 06, 2022 at 11:22:13AM -0300, Arnaldo Carvalho de Melo wrote:
>>>
>>> [...]
>>>
>>>>> @@ -1006,7 +1014,10 @@ install-tests: all install-gtk
>>>>> $(INSTALL) tests/shell/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell'; \
>>>>> $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
>>>>> $(INSTALL) tests/shell/lib/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
>>>>> - $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'
>>>>> + $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
>>>>> + $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'; \
>>>>> + $(INSTALL) tests/shell/coresight/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
>>>>> + $(Q)$(MAKE) -C tests/shell/coresight install-tests
>>>>>
>>>>> It's deliberately to not add '-m 644' when install coresight test
>>>>> shell scripts under the folder tests/shell/coresight/ so that we can
>>>>> keep the executable permission for these scripts. On the other flip,
>>>>> we set 644 mode for the scripts under tests/shell/lib/ so these
>>>>> scripts will not be executed directly by perf test framework.
>>>>>
>>>>> @Carsten, if have chance could you confirm for above change?
>>>>
>>>> If you need for some .sh, coresight test's to have x set while all of
>>>> the others don't have, and you have a good reason for that, ok, we just
>>>> add a comment.
>>>
>>> Simply to say, all lib scripts are not needed for execute permission.
>>> Yes, it's good to add a comment for the installation steps.
>>>
>>>>>> DESCEND plugins
>>>>>> GEN /tmp/build/perf/python/perf.so
>>>>>> INSTALL trace_plugins
>>>>>> INSTALL binaries
>>>>>> INSTALL tests
>>>>>> install: cannot stat 'tests/shell/coresight/*.sh': No such file or directory
>>>>>> make[2]: *** [Makefile.perf:1007: install-tests] Error 1
>>>>>> make[2]: *** Waiting for unfinished jobs....
>>>>>>
>>>>>> I'll add it back when the first .sh gets added to tests/shell/coresight/
>>>>>
>>>>> Sorry for introducing extra efforts for you and thanks!
>>>>
>>>> This also has to go, I'll check and fix this loop that doesn't follow
>>>> the existing output style, happens with ´make install', etc too:
>>>>
>>>> ⬢[acme@toolbox perf]$ make -C tools/perf clean
>>>> make: Entering directory '/var/home/acme/git/perf/tools/perf'
>>>> CLEAN x86
>>>> CLEAN libtraceevent
>>>> CLEAN libapi
>>>> CLEAN libbpf
>>>> CLEAN libsubcmd
>>>> CLEAN libperf
>>>> CLEAN fixdep
>>>> CLEAN feature-detect
>>>> CLEAN python
>>>> CLEAN bpf-skel
>>>> make -C asm_pure_loop clean >/dev/null <-------------------
>>>> make -C memcpy_thread clean >/dev/null <-------------------
>>>> make -C thread_loop clean >/dev/null <-------------------
>>>> make -C unroll_loop_thread clean >/dev/null <-------------------
>>>
>>> Ouch, if we can update the file tests/shell/coresight/Makefile as below:
>>>
>>> $(CLEANDIRS):
>>> $(call QUIET_CLEAN, test-$(@:clean-%=%)) $(Q)$(MAKE) -C $(@:clean-%=%) clean >/dev/null
>>
>> I had to do it, so far, at the time that the above line is first added
>> as:
>>
>> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
>> index a20b4adb5e3451f1..194e582e70c2c766 100644
>> --- a/tools/perf/Makefile.perf
>> +++ b/tools/perf/Makefile.perf
>> @@ -635,7 +635,8 @@ tests-coresight-targets: FORCE
>> $(Q)$(MAKE) -C $(TESTS_CORESIGHT_DIR)
>>
>> tests-coresight-targets-clean:
>> - $(Q)$(MAKE) -C $(TESTS_CORESIGHT_DIR) clean
>> + $(call QUIET_CLEAN, coresight)
>> + $(Q)$(MAKE) -C $(TESTS_CORESIGHT_DIR) O=$(OUTPUT) clean >/dev/null
>>
>> all: shell_compatibility_test $(ALL_PROGRAMS) $(LANG_BINDINGS) $(OTHER_PROGRAMS) tests-coresight-targets
>>
>> diff --git a/tools/perf/tests/shell/coresight/Makefile b/tools/perf/tests/shell/coresight/Makefile
>> index 3fee05cfcb0e73f6..6baab5cbaa9aa3b1 100644
>> --- a/tools/perf/tests/shell/coresight/Makefile
>> +++ b/tools/perf/tests/shell/coresight/Makefile
>> @@ -20,6 +20,6 @@ CLEANDIRS = $(SUBDIRS:%=clean-%)
>>
>> clean: $(CLEANDIRS)
>> $(CLEANDIRS):
>> - $(Q)$(MAKE) -C $(@:clean-%=%) clean >/dev/null
>> + $(call QUIET_CLEAN, test-$(@:clean-%=%)) $(Q)$(MAKE) -C $(@:clean-%=%) clean >/dev/null
>>
>> .PHONY: all clean $(SUBDIRS) $(CLEANDIRS) $(INSTALLDIRS)
>
> And for the normal build and install, somehow that $(Q) is not being
> obeyed there, so I'm using straight @ for now so that we have a clean
> output, someone more knowledgeable in the Makefile voodoo can then fix
> it.
>
> ⬢[acme@toolbox perf]$ git log --oneline -1
> e1700b67a41092a5 (HEAD) perf test: Add build infra for perf test tools for ARM CoreSight tests
> ⬢[acme@toolbox perf]$ git diff
> diff --git a/tools/perf/tests/shell/coresight/Makefile b/tools/perf/tests/shell/coresight/Makefile
> index 6baab5cbaa9aa3b1..6f3b3f07049c2d53 100644
> --- a/tools/perf/tests/shell/coresight/Makefile
> +++ b/tools/perf/tests/shell/coresight/Makefile
> @@ -8,13 +8,13 @@ SUBDIRS =
>
> all: $(SUBDIRS)
> $(SUBDIRS):
> - $(Q)$(MAKE) -C $@
> + @$(MAKE) -C $@

DOn't you want to prefix this with + too (to preserve jobserver, i.e.
parallel builds)?

Re the Q -- is it exported? What is printed with echo $(Q).

>
> INSTALLDIRS = $(SUBDIRS:%=install-%)
>
> install-tests: $(INSTALLDIRS)
> $(INSTALLDIRS):
> - $(Q)$(MAKE) -C $(@:install-%=%) install-tests
> + @$(MAKE) -C $(@:install-%=%) install-tests >/dev/null
>
> CLEANDIRS = $(SUBDIRS:%=clean-%)
>
> ⬢[acme@toolbox perf]$

--
js

2022-10-07 11:49:18

by Carsten Haitzler

[permalink] [raw]
Subject: Re: [PATCH v9 02/13] perf test: Add build infra for perf test tools for CoreSight tests



On 10/6/22 13:58, Leo Yan wrote:
> Hi Arnaldo,
>
> On Thu, Oct 06, 2022 at 08:45:18AM -0300, Arnaldo Carvalho de Melo wrote:
>> Em Thu, Oct 06, 2022 at 08:26:00AM -0300, Arnaldo Carvalho de Melo escreveu:
>>> Em Fri, Sep 09, 2022 at 04:27:52PM +0100, [email protected] escreveu:
>>>> +all: shell_compatibility_test $(ALL_PROGRAMS) $(LANG_BINDINGS) $(OTHER_PROGRAMS) tests-coresight-targets
>>>>
>>>> # Create python binding output directory if not already present
>>>> _dummy := $(shell [ -d '$(OUTPUT)python' ] || mkdir -p '$(OUTPUT)python')
>>>> @@ -1007,6 +1015,9 @@ install-tests: all install-gtk
>>>> $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
>>>> $(INSTALL) tests/shell/lib/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
>>>> $(INSTALL) tests/shell/lib/*.py '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'
>>>> + $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'; \
>>>> + $(INSTALL) tests/shell/coresight/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
>>>> + $(Q)$(MAKE) -C tests/shell/coresight install-tests
>>>
>>> Added a missing ;\ on the line installing the *.py and adjusted use -m
>>> 644 for the .sh files to follow what was done in:
>>>
>>> 0a9eaf616f29ca32 ("perf tools: Don't install data files with x permissions")
>>>
>>> @@ -1006,7 +1014,10 @@ install-tests: all install-gtk
>>> $(INSTALL) tests/shell/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell'; \
>>> $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
>>> $(INSTALL) tests/shell/lib/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
>>> - $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'
>>> + $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
>>> + $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'; \
>>> + $(INSTALL) tests/shell/coresight/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
>>> + $(Q)$(MAKE) -C tests/shell/coresight install-tests
>>>
>>
>> Also had to remove:
>>
>> $(INSTALL) tests/shell/coresight/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
>>
>> from this patch, as it makes install fail at this point in the patchset:
>
> Thanks a lot!
>
> James and me had found the merging conflict and planned to send out
> email for reminding it, the right change for above section would be:
>
> @@ -1006,7 +1014,10 @@ install-tests: all install-gtk
> $(INSTALL) tests/shell/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell'; \
> $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> $(INSTALL) tests/shell/lib/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> - $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'
> + $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> + $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'; \
> + $(INSTALL) tests/shell/coresight/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
> + $(Q)$(MAKE) -C tests/shell/coresight install-tests
>
> It's deliberately to not add '-m 644' when install coresight test
> shell scripts under the folder tests/shell/coresight/ so that we can
> keep the executable permission for these scripts. On the other flip,
> we set 644 mode for the scripts under tests/shell/lib/ so these
> scripts will not be executed directly by perf test framework.
>
> @Carsten, if have chance could you confirm for above change?
>
>> DESCEND plugins
>> GEN /tmp/build/perf/python/perf.so
>> INSTALL trace_plugins
>> INSTALL binaries
>> INSTALL tests
>> install: cannot stat 'tests/shell/coresight/*.sh': No such file or directory
>> make[2]: *** [Makefile.perf:1007: install-tests] Error 1
>> make[2]: *** Waiting for unfinished jobs....
>>
>> I'll add it back when the first .sh gets added to tests/shell/coresight/
>
> Sorry for introducing extra efforts for you and thanks!

oh sorry - indeed i didn't see this problem coming after fixing the
conflicts. i've got an update of the patches that fix that. should i
just send through the 2 updates patches as a v10 or the whole series?

2022-10-07 12:48:08

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: [PATCH v9 02/13] perf test: Add build infra for perf test tools for CoreSight tests

Em Fri, Oct 07, 2022 at 12:34:51PM +0100, Carsten Haitzler escreveu:
> On 10/6/22 13:58, Leo Yan wrote:
> > On Thu, Oct 06, 2022 at 08:45:18AM -0300, Arnaldo Carvalho de Melo wrote:
> > > Em Thu, Oct 06, 2022 at 08:26:00AM -0300, Arnaldo Carvalho de Melo escreveu:
> > > Also had to remove:

> > > $(INSTALL) tests/shell/coresight/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'

> > > from this patch, as it makes install fail at this point in the patchset:

> > Thanks a lot!

> > James and me had found the merging conflict and planned to send out
> > email for reminding it, the right change for above section would be:

> > @@ -1006,7 +1014,10 @@ install-tests: all install-gtk
> > $(INSTALL) tests/shell/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell'; \
> > $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > $(INSTALL) tests/shell/lib/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > - $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'
> > + $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
> > + $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'; \
> > + $(INSTALL) tests/shell/coresight/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
> > + $(Q)$(MAKE) -C tests/shell/coresight install-tests

> > It's deliberately to not add '-m 644' when install coresight test
> > shell scripts under the folder tests/shell/coresight/ so that we can
> > keep the executable permission for these scripts. On the other
> > flip, we set 644 mode for the scripts under tests/shell/lib/ so
> > these scripts will not be executed directly by perf test framework.

> > @Carsten, if have chance could you confirm for above change?

> > > DESCEND plugins
> > > GEN /tmp/build/perf/python/perf.so
> > > INSTALL trace_plugins
> > > INSTALL binaries
> > > INSTALL tests
> > > install: cannot stat 'tests/shell/coresight/*.sh': No such file or directory
> > > make[2]: *** [Makefile.perf:1007: install-tests] Error 1
> > > make[2]: *** Waiting for unfinished jobs....

> > > I'll add it back when the first .sh gets added to tests/shell/coresight/

> > Sorry for introducing extra efforts for you and thanks!

> oh sorry - indeed i didn't see this problem coming after fixing the
> conflicts. i've got an update of the patches that fix that. should i just
> send through the 2 updates patches as a v10 or the whole series?

No need, I did some fixes taking into account the comments on this
thread, we can go on and fix things from what I have now at
acme/perf/core, which I'll send to Linus today.

- Arnaldo

2022-10-10 08:24:34

by Carsten Haitzler

[permalink] [raw]
Subject: Re: [PATCH v9 02/13] perf test: Add build infra for perf test tools for CoreSight tests



On 10/7/22 13:44, Arnaldo Carvalho de Melo wrote:
> Em Fri, Oct 07, 2022 at 12:34:51PM +0100, Carsten Haitzler escreveu:
>> On 10/6/22 13:58, Leo Yan wrote:
>>> On Thu, Oct 06, 2022 at 08:45:18AM -0300, Arnaldo Carvalho de Melo wrote:
>>>> Em Thu, Oct 06, 2022 at 08:26:00AM -0300, Arnaldo Carvalho de Melo escreveu:
>>>> Also had to remove:
>
>>>> $(INSTALL) tests/shell/coresight/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
>
>>>> from this patch, as it makes install fail at this point in the patchset:
>
>>> Thanks a lot!
>
>>> James and me had found the merging conflict and planned to send out
>>> email for reminding it, the right change for above section would be:
>
>>> @@ -1006,7 +1014,10 @@ install-tests: all install-gtk
>>> $(INSTALL) tests/shell/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell'; \
>>> $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
>>> $(INSTALL) tests/shell/lib/*.sh -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
>>> - $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'
>>> + $(INSTALL) tests/shell/lib/*.py -m 644 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/lib'; \
>>> + $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'; \
>>> + $(INSTALL) tests/shell/coresight/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
>>> + $(Q)$(MAKE) -C tests/shell/coresight install-tests
>
>>> It's deliberately to not add '-m 644' when install coresight test
>>> shell scripts under the folder tests/shell/coresight/ so that we can
>>> keep the executable permission for these scripts. On the other
>>> flip, we set 644 mode for the scripts under tests/shell/lib/ so
>>> these scripts will not be executed directly by perf test framework.
>
>>> @Carsten, if have chance could you confirm for above change?
>
>>>> DESCEND plugins
>>>> GEN /tmp/build/perf/python/perf.so
>>>> INSTALL trace_plugins
>>>> INSTALL binaries
>>>> INSTALL tests
>>>> install: cannot stat 'tests/shell/coresight/*.sh': No such file or directory
>>>> make[2]: *** [Makefile.perf:1007: install-tests] Error 1
>>>> make[2]: *** Waiting for unfinished jobs....
>
>>>> I'll add it back when the first .sh gets added to tests/shell/coresight/
>
>>> Sorry for introducing extra efforts for you and thanks!
>
>> oh sorry - indeed i didn't see this problem coming after fixing the
>> conflicts. i've got an update of the patches that fix that. should i just
>> send through the 2 updates patches as a v10 or the whole series?
>
> No need, I did some fixes taking into account the comments on this
> thread, we can go on and fix things from what I have now at
> acme/perf/core, which I'll send to Linus today.

oh cool. i'll let patches sit for now - let me know if there's anything
you want/need from me.

2022-10-10 14:47:28

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: [PATCH v9 02/13] perf test: Add build infra for perf test tools for CoreSight tests

Em Mon, Oct 10, 2022 at 08:02:29AM +0100, Carsten Haitzler escreveu:
> On 10/7/22 13:44, Arnaldo Carvalho de Melo wrote:
> > Em Fri, Oct 07, 2022 at 12:34:51PM +0100, Carsten Haitzler escreveu:
> > > On 10/6/22 13:58, Leo Yan wrote:
> > > oh sorry - indeed i didn't see this problem coming after fixing the
> > > conflicts. i've got an update of the patches that fix that. should i just
> > > send through the 2 updates patches as a v10 or the whole series?

> > No need, I did some fixes taking into account the comments on this
> > thread, we can go on and fix things from what I have now at
> > acme/perf/core, which I'll send to Linus today.

> oh cool. i'll let patches sit for now - let me know if there's anything you
> want/need from me.

Not right now, I'm now just waiting for Linus to merge what I sent, then
you can continue from upstream.

- Arnaldo

2022-10-11 08:54:00

by Sven Schnelle

[permalink] [raw]
Subject: Re: [PATCH v9 13/13] perf test: Add relevant documentation about CoreSight testing

Hi Carsten,

[email protected] writes:

> From: Carsten Haitzler <[email protected]>
>
> Add/improve documentation helping people get started with CoreSight and
> perf as well as describe the testing and how it works.
>
> Cc: [email protected]
> Signed-off-by: Carsten Haitzler <[email protected]>
> ---
> .../trace/coresight/coresight-perf.rst | 158 ++++++++++++++++++
> .../perf/Documentation/perf-arm-coresight.txt | 5 +
> 2 files changed, 163 insertions(+)
> create mode 100644 Documentation/trace/coresight/coresight-perf.rst
> create mode 100644 tools/perf/Documentation/perf-arm-coresight.txt
>

Investigating a rpm build failure i see the following error with next-20221011:

% cd tools/perf/Documentation
% make O=/home/svens/linux-build/ man V=1
rm -f /home/svens/linux-build/perf-arm-coresight.xml+ /home/svens/linux-build/perf-arm-coresight.xml && \
asciidoc -b docbook -d manpage \
--unsafe -f asciidoc.conf -aperf_version= \
-aperf_date=2022-10-06 \
-o /home/svens/linux-build/perf-arm-coresight.xml+ perf-arm-coresight.txt && \
mv /home/svens/linux-build/perf-arm-coresight.xml+ /home/svens/linux-build/perf-arm-coresight.xml
asciidoc: FAILED: manpage document title is mandatory
make: *** [Makefile:266: /home/svens/linux-build/perf-arm-coresight.xml] Error 1

Can you please fix this?

Thanks!

2022-10-19 08:17:02

by Carsten Haitzler

[permalink] [raw]
Subject: Re: [PATCH v9 02/13] perf test: Add build infra for perf test tools for CoreSight tests



On 10/10/22 14:54, Arnaldo Carvalho de Melo wrote:
> Em Mon, Oct 10, 2022 at 08:02:29AM +0100, Carsten Haitzler escreveu:
>> On 10/7/22 13:44, Arnaldo Carvalho de Melo wrote:
>>> Em Fri, Oct 07, 2022 at 12:34:51PM +0100, Carsten Haitzler escreveu:
>>>> On 10/6/22 13:58, Leo Yan wrote:
>>>> oh sorry - indeed i didn't see this problem coming after fixing the
>>>> conflicts. i've got an update of the patches that fix that. should i just
>>>> send through the 2 updates patches as a v10 or the whole series?
>
>>> No need, I did some fixes taking into account the comments on this
>>> thread, we can go on and fix things from what I have now at
>>> acme/perf/core, which I'll send to Linus today.
>
>> oh cool. i'll let patches sit for now - let me know if there's anything you
>> want/need from me.
>
> Not right now, I'm now just waiting for Linus to merge what I sent, then
> you can continue from upstream.

I'm not sure what you have sent to Linus? The patch series? Specific
patches?

2022-10-19 09:01:05

by Carsten Haitzler

[permalink] [raw]
Subject: Re: [PATCH v9 13/13] perf test: Add relevant documentation about CoreSight testing

This is weird. This shouldn't be making its way into any man page at
all. It's just a file pointing people at other parts of the kernel tree
for docs (because when I looked for docs - i found nothing for any kind
of getting started guide - I put something here as this was the obvious
place for perf related docs for testing - but was told to put the docs
elsewhere in core kernel documentation, but I put a reference here for
those who follow after to be able to more easily find it).

There is some rule somewhere it seems that makes anything perf-*.txt
into a man page. I need to rename this file it seems.

On 10/11/22 08:27, Sven Schnelle wrote:
> Hi Carsten,
>
> [email protected] writes:
>
>> From: Carsten Haitzler <[email protected]>
>>
>> Add/improve documentation helping people get started with CoreSight and
>> perf as well as describe the testing and how it works.
>>
>> Cc: [email protected]
>> Signed-off-by: Carsten Haitzler <[email protected]>
>> ---
>> .../trace/coresight/coresight-perf.rst | 158 ++++++++++++++++++
>> .../perf/Documentation/perf-arm-coresight.txt | 5 +
>> 2 files changed, 163 insertions(+)
>> create mode 100644 Documentation/trace/coresight/coresight-perf.rst
>> create mode 100644 tools/perf/Documentation/perf-arm-coresight.txt
>>
>
> Investigating a rpm build failure i see the following error with next-20221011:
>
> % cd tools/perf/Documentation
> % make O=/home/svens/linux-build/ man V=1
> rm -f /home/svens/linux-build/perf-arm-coresight.xml+ /home/svens/linux-build/perf-arm-coresight.xml && \
> asciidoc -b docbook -d manpage \
> --unsafe -f asciidoc.conf -aperf_version= \
> -aperf_date=2022-10-06 \
> -o /home/svens/linux-build/perf-arm-coresight.xml+ perf-arm-coresight.txt && \
> mv /home/svens/linux-build/perf-arm-coresight.xml+ /home/svens/linux-build/perf-arm-coresight.xml
> asciidoc: FAILED: manpage document title is mandatory
> make: *** [Makefile:266: /home/svens/linux-build/perf-arm-coresight.xml] Error 1
>
> Can you please fix this?
>
> Thanks!

2022-10-19 10:10:56

by Leo Yan

[permalink] [raw]
Subject: Re: [PATCH v9 02/13] perf test: Add build infra for perf test tools for CoreSight tests

On Wed, Oct 19, 2022 at 09:11:20AM +0100, Carsten Haitzler wrote:
>
>
> On 10/10/22 14:54, Arnaldo Carvalho de Melo wrote:
> > Em Mon, Oct 10, 2022 at 08:02:29AM +0100, Carsten Haitzler escreveu:
> > > On 10/7/22 13:44, Arnaldo Carvalho de Melo wrote:
> > > > Em Fri, Oct 07, 2022 at 12:34:51PM +0100, Carsten Haitzler escreveu:
> > > > > On 10/6/22 13:58, Leo Yan wrote:
> > > > > oh sorry - indeed i didn't see this problem coming after fixing the
> > > > > conflicts. i've got an update of the patches that fix that. should i just
> > > > > send through the 2 updates patches as a v10 or the whole series?
> >
> > > > No need, I did some fixes taking into account the comments on this
> > > > thread, we can go on and fix things from what I have now at
> > > > acme/perf/core, which I'll send to Linus today.
> > > oh cool. i'll let patches sit for now - let me know if there's anything you
> > > want/need from me.
> >
> > Not right now, I'm now just waiting for Linus to merge what I sent, then
> > you can continue from upstream.
>
> I'm not sure what you have sent to Linus? The patch series? Specific
> patches?

I think the whole patch set has been merged into Linux mainline;
after git pull the Linux mainline code and you could see:

$ git log --oneline --author="Carsten Haitzle"
dc2e0fb00bb2 perf test coresight: Add relevant documentation about ARM64 CoreSight testing
43c688cb3241 perf test: Add git ignore for tmp and output files of ARM CoreSight tests
b65c6477f6bb perf test coresight: Add unroll thread test shell script
fc0a0ea03980 perf test coresight: Add unroll thread test tool
74c62b8d6161 perf test coresight: Add thread loop test shell scripts
e9664b96c6c0 perf test coresight: Add thread loop test tool
b76692fea7f2 perf test coresight: Add memcpy thread test shell script
f1288bdb6d48 perf test coresight: Add memcpy thread test tool
6ea586b1e3dc perf test: Add git ignore for perf data generated by the ARM CoreSight tests
fdc25cc59c71 perf test: Add arm64 asm pureloop test shell script
8b97519711c3 perf test: Add asm pureloop test tool
34bec35cbbb2 perf test: Add build infra for perf test tools for ARM CoreSight tests
c63317ab14b0 perf test: Add CoreSight shell lib shared code for future tests
7391db645938 perf test: Refactor shell tests allowing subdirs

Thanks,
Leo

2022-10-19 10:29:48

by Leo Yan

[permalink] [raw]
Subject: Re: [PATCH v9 13/13] perf test: Add relevant documentation about CoreSight testing

Hi Carsten,

On Wed, Oct 19, 2022 at 09:16:03AM +0100, Carsten Haitzler wrote:
> This is weird. This shouldn't be making its way into any man page at all.
> It's just a file pointing people at other parts of the kernel tree for docs
> (because when I looked for docs - i found nothing for any kind of getting
> started guide - I put something here as this was the obvious place for perf
> related docs for testing - but was told to put the docs elsewhere in core
> kernel documentation, but I put a reference here for those who follow after
> to be able to more easily find it).
>
> There is some rule somewhere it seems that makes anything perf-*.txt into a
> man page. I need to rename this file it seems.

Adrian has sent a patch for renaming the file, see:
https://lore.kernel.org/lkml/[email protected]/

Thanks,
Leo

2022-10-19 16:06:12

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: [PATCH v9 02/13] perf test: Add build infra for perf test tools for CoreSight tests

Em Wed, Oct 19, 2022 at 04:51:25PM +0800, Leo Yan escreveu:
> On Wed, Oct 19, 2022 at 09:11:20AM +0100, Carsten Haitzler wrote:
> >
> >
> > On 10/10/22 14:54, Arnaldo Carvalho de Melo wrote:
> > > Em Mon, Oct 10, 2022 at 08:02:29AM +0100, Carsten Haitzler escreveu:
> > > > On 10/7/22 13:44, Arnaldo Carvalho de Melo wrote:
> > > > > Em Fri, Oct 07, 2022 at 12:34:51PM +0100, Carsten Haitzler escreveu:
> > > > > > On 10/6/22 13:58, Leo Yan wrote:
> > > > > > oh sorry - indeed i didn't see this problem coming after fixing the
> > > > > > conflicts. i've got an update of the patches that fix that. should i just
> > > > > > send through the 2 updates patches as a v10 or the whole series?
> > >
> > > > > No need, I did some fixes taking into account the comments on this
> > > > > thread, we can go on and fix things from what I have now at
> > > > > acme/perf/core, which I'll send to Linus today.
> > > > oh cool. i'll let patches sit for now - let me know if there's anything you
> > > > want/need from me.
> > >
> > > Not right now, I'm now just waiting for Linus to merge what I sent, then
> > > you can continue from upstream.
> >
> > I'm not sure what you have sent to Linus? The patch series? Specific
> > patches?
>
> I think the whole patch set has been merged into Linux mainline;
> after git pull the Linux mainline code and you could see:
>
> $ git log --oneline --author="Carsten Haitzle"
> dc2e0fb00bb2 perf test coresight: Add relevant documentation about ARM64 CoreSight testing
> 43c688cb3241 perf test: Add git ignore for tmp and output files of ARM CoreSight tests
> b65c6477f6bb perf test coresight: Add unroll thread test shell script
> fc0a0ea03980 perf test coresight: Add unroll thread test tool
> 74c62b8d6161 perf test coresight: Add thread loop test shell scripts
> e9664b96c6c0 perf test coresight: Add thread loop test tool
> b76692fea7f2 perf test coresight: Add memcpy thread test shell script
> f1288bdb6d48 perf test coresight: Add memcpy thread test tool
> 6ea586b1e3dc perf test: Add git ignore for perf data generated by the ARM CoreSight tests
> fdc25cc59c71 perf test: Add arm64 asm pureloop test shell script
> 8b97519711c3 perf test: Add asm pureloop test tool
> 34bec35cbbb2 perf test: Add build infra for perf test tools for ARM CoreSight tests
> c63317ab14b0 perf test: Add CoreSight shell lib shared code for future tests
> 7391db645938 perf test: Refactor shell tests allowing subdirs

Thanks, Leo, Carsten, are we on the same page now? :-)

- Arnaldo