2023-11-29 21:35:40

by Ian Rogers

[permalink] [raw]
Subject: [PATCH v2 1/4] perf build: Shellcheck support for OUTPUT directory

Migrate Makefile.tests to Build so that variables like rule_mkdir are
defined via Makefile.build (needed so the output directory can be
created). This requires SHELLCHECK being exported and the clean rule
tweaking to remove the files in find.

Change find "-perm -o=x" as it was failing on my Debian based Linux
kernel tree, switch to using "-executable".

Adding a filename prefix of "." to the shellcheck log files is a pain
and error prone in make, remove this prefix and just add the
shellcheck log files to .gitignore.

Fix the command echo so that running the test is displayed.

Fixes: 1638b11ef815 ("perf tools: Add perf binary dependent rule for shellcheck log in Makefile.perf")
Signed-off-by: Ian Rogers <[email protected]>
---
tools/perf/.gitignore | 3 +++
tools/perf/Makefile.perf | 30 ++++++++++--------------------
tools/perf/tests/Build | 14 ++++++++++++++
tools/perf/tests/Makefile.tests | 22 ----------------------
4 files changed, 27 insertions(+), 42 deletions(-)
delete mode 100644 tools/perf/tests/Makefile.tests

diff --git a/tools/perf/.gitignore b/tools/perf/.gitignore
index ee5c14f3b8b1..f5b81d439387 100644
--- a/tools/perf/.gitignore
+++ b/tools/perf/.gitignore
@@ -39,6 +39,9 @@ trace/beauty/generated/
pmu-events/pmu-events.c
pmu-events/jevents
pmu-events/metric_test.log
+tests/shell/*.shellcheck_log
+tests/shell/coresight/*.shellcheck_log
+tests/shell/lib/*.shellcheck_log
feature/
libapi/
libbpf/
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 824cbc0af7d7..1ab2a908f240 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -229,8 +229,15 @@ else
force_fixdep := $(config)
endif

+# Runs shellcheck on perf test shell scripts
+ifeq ($(NO_SHELLCHECK),1)
+ SHELLCHECK :=
+else
+ SHELLCHECK := $(shell which shellcheck 2> /dev/null)
+endif
+
export srctree OUTPUT RM CC CXX LD AR CFLAGS CXXFLAGS V BISON FLEX AWK
-export HOSTCC HOSTLD HOSTAR HOSTCFLAGS
+export HOSTCC HOSTLD HOSTAR HOSTCFLAGS SHELLCHECK

include $(srctree)/tools/build/Makefile.include

@@ -673,23 +680,7 @@ $(PERF_IN): prepare FORCE
$(PMU_EVENTS_IN): FORCE prepare
$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=pmu-events obj=pmu-events

-# Runs shellcheck on perf test shell scripts
-
-SHELLCHECK := $(shell which shellcheck 2> /dev/null)
-
-ifeq ($(NO_SHELLCHECK),1)
-SHELLCHECK :=
-endif
-
-ifneq ($(SHELLCHECK),)
-SHELLCHECK_TEST: FORCE prepare
- $(Q)$(MAKE) -f $(srctree)/tools/perf/tests/Makefile.tests
-else
-SHELLCHECK_TEST:
- @:
-endif
-
-$(OUTPUT)perf: $(PERFLIBS) $(PERF_IN) $(PMU_EVENTS_IN) SHELLCHECK_TEST
+$(OUTPUT)perf: $(PERFLIBS) $(PERF_IN) $(PMU_EVENTS_IN)
$(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) \
$(PERF_IN) $(PMU_EVENTS_IN) $(LIBS) -o $@

@@ -1152,9 +1143,8 @@ bpf-skel-clean:
$(call QUIET_CLEAN, bpf-skel) $(RM) -r $(SKEL_TMP_OUT) $(SKELETONS)

clean:: $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clean $(LIBSYMBOL)-clean $(LIBPERF)-clean fixdep-clean python-clean bpf-skel-clean tests-coresight-targets-clean
- $(Q)$(MAKE) -f $(srctree)/tools/perf/tests/Makefile.tests clean
$(call QUIET_CLEAN, core-objs) $(RM) $(LIBPERF_A) $(OUTPUT)perf-archive $(OUTPUT)perf-iostat $(LANG_BINDINGS)
- $(Q)find $(or $(OUTPUT),.) -name '*.o' -delete -o -name '\.*.cmd' -delete -o -name '\.*.d' -delete
+ $(Q)find $(or $(OUTPUT),.) -name '*.o' -delete -o -name '\.*.cmd' -delete -o -name '\.*.d' -delete -o -name '*.shellcheck_log' -delete
$(Q)$(RM) $(OUTPUT).config-detected
$(call QUIET_CLEAN, core-progs) $(RM) $(ALL_PROGRAMS) perf perf-read-vdso32 perf-read-vdsox32 $(OUTPUT)$(LIBJVMTI).so
$(call QUIET_CLEAN, core-gen) $(RM) *.spec *.pyc *.pyo */*.pyc */*.pyo $(OUTPUT)common-cmds.h TAGS tags cscope* $(OUTPUT)PERF-VERSION-FILE $(OUTPUT)FEATURE-DUMP $(OUTPUT)util/*-bison* $(OUTPUT)util/*-flex* \
diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
index 2b45ffa462a6..53ba9c3e20e0 100644
--- a/tools/perf/tests/Build
+++ b/tools/perf/tests/Build
@@ -77,3 +77,17 @@ CFLAGS_python-use.o += -DPYTHONPATH="BUILD_STR($(OUTPUT)python)" -DPYTHON="BUI
CFLAGS_dwarf-unwind.o += -fno-optimize-sibling-calls

perf-y += workloads/
+
+ifdef SHELLCHECK
+ SHELL_TESTS := $(shell find tests/shell -executable -type f -name '*.sh')
+ TEST_LOGS := $(SHELL_TESTS:tests/shell/%=shell/%.shellcheck_log)
+else
+ SHELL_TESTS :=
+ TEST_LOGS :=
+endif
+
+$(OUTPUT)%.shellcheck_log: %
+ $(call rule_mkdir)
+ $(Q)$(call echo-cmd,test)shellcheck -a -S warning "$<" > $@ || (cat $@ && rm $@ && false)
+
+perf-y += $(TEST_LOGS)
diff --git a/tools/perf/tests/Makefile.tests b/tools/perf/tests/Makefile.tests
deleted file mode 100644
index fdaca5f7a946..000000000000
--- a/tools/perf/tests/Makefile.tests
+++ /dev/null
@@ -1,22 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-# Athira Rajeev <[email protected]>, 2023
-
-PROGS := $(shell find tests/shell -perm -o=x -type f -name '*.sh')
-FILE_NAME := $(notdir $(PROGS))
-FILE_NAME := $(FILE_NAME:%=.%)
-LOGS := $(join $(dir $(PROGS)),$(FILE_NAME))
-LOGS := $(LOGS:%=%.shellcheck_log)
-
-.PHONY: all
-all: SHELLCHECK_RUN
- @:
-
-SHELLCHECK_RUN: $(LOGS)
-
-.%.shellcheck_log: %
- $(call rule_mkdir)
- $(Q)$(call frecho-cmd,test)@shellcheck -S warning "$<" > $@ || (cat $@ && rm $@ && false)
-
-clean:
- $(eval log_files := $(shell find . -name '.*.shellcheck_log'))
- @rm -rf $(log_files)
--
2.43.0.rc1.413.gea7ed67945-goog


2023-11-29 21:35:49

by Ian Rogers

[permalink] [raw]
Subject: [PATCH v2 4/4] perf test: Add basic list test

Test that json output produces valid json.

Signed-off-by: Ian Rogers <[email protected]>
---
tools/perf/tests/shell/list.sh | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
create mode 100755 tools/perf/tests/shell/list.sh

diff --git a/tools/perf/tests/shell/list.sh b/tools/perf/tests/shell/list.sh
new file mode 100755
index 000000000000..22b004f2b23e
--- /dev/null
+++ b/tools/perf/tests/shell/list.sh
@@ -0,0 +1,19 @@
+#!/bin/sh
+# perf list tests
+# SPDX-License-Identifier: GPL-2.0
+
+set -e
+err=0
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+test_list_json() {
+ echo "Json output test"
+ perf list -j | $PYTHON -m json.tool
+ echo "Json output test [Success]"
+}
+
+test_list_json
+exit $err
--
2.43.0.rc1.413.gea7ed67945-goog

2023-11-29 21:35:48

by Ian Rogers

[permalink] [raw]
Subject: [PATCH v2 3/4] perf test: Use common python setup library

Avoid replicated logic by having a common library to set the PYTHON
environment variable.

Signed-off-by: Ian Rogers <[email protected]>
---
tools/perf/tests/shell/lib/setup_python.sh | 16 ++++++++++++++++
tools/perf/tests/shell/stat+json_output.sh | 16 +++-------------
tools/perf/tests/shell/stat_metrics_values.sh | 14 ++++----------
.../tests/shell/test_perf_data_converter_json.sh | 13 +++----------
4 files changed, 26 insertions(+), 33 deletions(-)
create mode 100644 tools/perf/tests/shell/lib/setup_python.sh

diff --git a/tools/perf/tests/shell/lib/setup_python.sh b/tools/perf/tests/shell/lib/setup_python.sh
new file mode 100644
index 000000000000..c2fce1793538
--- /dev/null
+++ b/tools/perf/tests/shell/lib/setup_python.sh
@@ -0,0 +1,16 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+
+if [ "x$PYTHON" = "x" ]
+then
+ python3 --version >/dev/null 2>&1 && PYTHON=python3
+fi
+if [ "x$PYTHON" = "x" ]
+then
+ python --version >/dev/null 2>&1 && PYTHON=python
+fi
+if [ "x$PYTHON" = "x" ]
+then
+ echo Skipping test, python not detected please set environment variable PYTHON.
+ exit 2
+fi
diff --git a/tools/perf/tests/shell/stat+json_output.sh b/tools/perf/tests/shell/stat+json_output.sh
index 196e22672c50..3bc900533a5d 100755
--- a/tools/perf/tests/shell/stat+json_output.sh
+++ b/tools/perf/tests/shell/stat+json_output.sh
@@ -8,20 +8,10 @@ set -e

skip_test=0

+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
pythonchecker=$(dirname $0)/lib/perf_json_output_lint.py
-if [ "x$PYTHON" == "x" ]
-then
- if which python3 > /dev/null
- then
- PYTHON=python3
- elif which python > /dev/null
- then
- PYTHON=python
- else
- echo Skipping test, python not detected please set environment variable PYTHON.
- exit 2
- fi
-fi

stat_output=$(mktemp /tmp/__perf_test.stat_output.json.XXXXX)

diff --git a/tools/perf/tests/shell/stat_metrics_values.sh b/tools/perf/tests/shell/stat_metrics_values.sh
index ad94c936de7e..7ca172599aa6 100755
--- a/tools/perf/tests/shell/stat_metrics_values.sh
+++ b/tools/perf/tests/shell/stat_metrics_values.sh
@@ -1,16 +1,10 @@
#!/bin/bash
# perf metrics value validation
# SPDX-License-Identifier: GPL-2.0
-if [ "x$PYTHON" == "x" ]
-then
- if which python3 > /dev/null
- then
- PYTHON=python3
- else
- echo Skipping test, python3 not detected please set environment variable PYTHON.
- exit 2
- fi
-fi
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh

grep -q GenuineIntel /proc/cpuinfo || { echo Skipping non-Intel; exit 2; }

diff --git a/tools/perf/tests/shell/test_perf_data_converter_json.sh b/tools/perf/tests/shell/test_perf_data_converter_json.sh
index 6ded58f98f55..c4f1b59d116f 100755
--- a/tools/perf/tests/shell/test_perf_data_converter_json.sh
+++ b/tools/perf/tests/shell/test_perf_data_converter_json.sh
@@ -6,16 +6,9 @@ set -e

err=0

-if [ "$PYTHON" = "" ] ; then
- if which python3 > /dev/null ; then
- PYTHON=python3
- elif which python > /dev/null ; then
- PYTHON=python
- else
- echo Skipping test, python not detected please set environment variable PYTHON.
- exit 2
- fi
-fi
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh

perfdata=$(mktemp /tmp/__perf_test.perf.data.XXXXX)
result=$(mktemp /tmp/__perf_test.output.json.XXXXX)
--
2.43.0.rc1.413.gea7ed67945-goog

2023-11-30 05:50:10

by Athira Rajeev

[permalink] [raw]
Subject: Re: [PATCH v2 1/4] perf build: Shellcheck support for OUTPUT directory



> On 30-Nov-2023, at 3:04 AM, Ian Rogers <[email protected]> wrote:
>
> Migrate Makefile.tests to Build so that variables like rule_mkdir are
> defined via Makefile.build (needed so the output directory can be
> created). This requires SHELLCHECK being exported and the clean rule
> tweaking to remove the files in find.
>
> Change find "-perm -o=x" as it was failing on my Debian based Linux
> kernel tree, switch to using "-executable".
>
> Adding a filename prefix of "." to the shellcheck log files is a pain
> and error prone in make, remove this prefix and just add the
> shellcheck log files to .gitignore.
>
> Fix the command echo so that running the test is displayed.

Thanks for checking this Ian.
I will do testing in my environment and report back on the results

Thanks
Athira
>
> Fixes: 1638b11ef815 ("perf tools: Add perf binary dependent rule for shellcheck log in Makefile.perf")
> Signed-off-by: Ian Rogers <[email protected]>
> ---
> tools/perf/.gitignore | 3 +++
> tools/perf/Makefile.perf | 30 ++++++++++--------------------
> tools/perf/tests/Build | 14 ++++++++++++++
> tools/perf/tests/Makefile.tests | 22 ----------------------
> 4 files changed, 27 insertions(+), 42 deletions(-)
> delete mode 100644 tools/perf/tests/Makefile.tests
>
> diff --git a/tools/perf/.gitignore b/tools/perf/.gitignore
> index ee5c14f3b8b1..f5b81d439387 100644
> --- a/tools/perf/.gitignore
> +++ b/tools/perf/.gitignore
> @@ -39,6 +39,9 @@ trace/beauty/generated/
> pmu-events/pmu-events.c
> pmu-events/jevents
> pmu-events/metric_test.log
> +tests/shell/*.shellcheck_log
> +tests/shell/coresight/*.shellcheck_log
> +tests/shell/lib/*.shellcheck_log
> feature/
> libapi/
> libbpf/
> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index 824cbc0af7d7..1ab2a908f240 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -229,8 +229,15 @@ else
> force_fixdep := $(config)
> endif
>
> +# Runs shellcheck on perf test shell scripts
> +ifeq ($(NO_SHELLCHECK),1)
> + SHELLCHECK :=
> +else
> + SHELLCHECK := $(shell which shellcheck 2> /dev/null)
> +endif
> +
> export srctree OUTPUT RM CC CXX LD AR CFLAGS CXXFLAGS V BISON FLEX AWK
> -export HOSTCC HOSTLD HOSTAR HOSTCFLAGS
> +export HOSTCC HOSTLD HOSTAR HOSTCFLAGS SHELLCHECK
>
> include $(srctree)/tools/build/Makefile.include
>
> @@ -673,23 +680,7 @@ $(PERF_IN): prepare FORCE
> $(PMU_EVENTS_IN): FORCE prepare
> $(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=pmu-events obj=pmu-events
>
> -# Runs shellcheck on perf test shell scripts
> -
> -SHELLCHECK := $(shell which shellcheck 2> /dev/null)
> -
> -ifeq ($(NO_SHELLCHECK),1)
> -SHELLCHECK :=
> -endif
> -
> -ifneq ($(SHELLCHECK),)
> -SHELLCHECK_TEST: FORCE prepare
> - $(Q)$(MAKE) -f $(srctree)/tools/perf/tests/Makefile.tests
> -else
> -SHELLCHECK_TEST:
> - @:
> -endif
> -
> -$(OUTPUT)perf: $(PERFLIBS) $(PERF_IN) $(PMU_EVENTS_IN) SHELLCHECK_TEST
> +$(OUTPUT)perf: $(PERFLIBS) $(PERF_IN) $(PMU_EVENTS_IN)
> $(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) \
> $(PERF_IN) $(PMU_EVENTS_IN) $(LIBS) -o $@
>
> @@ -1152,9 +1143,8 @@ bpf-skel-clean:
> $(call QUIET_CLEAN, bpf-skel) $(RM) -r $(SKEL_TMP_OUT) $(SKELETONS)
>
> clean:: $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clean $(LIBSYMBOL)-clean $(LIBPERF)-clean fixdep-clean python-clean bpf-skel-clean tests-coresight-targets-clean
> - $(Q)$(MAKE) -f $(srctree)/tools/perf/tests/Makefile.tests clean
> $(call QUIET_CLEAN, core-objs) $(RM) $(LIBPERF_A) $(OUTPUT)perf-archive $(OUTPUT)perf-iostat $(LANG_BINDINGS)
> - $(Q)find $(or $(OUTPUT),.) -name '*.o' -delete -o -name '\.*.cmd' -delete -o -name '\.*.d' -delete
> + $(Q)find $(or $(OUTPUT),.) -name '*.o' -delete -o -name '\.*.cmd' -delete -o -name '\.*.d' -delete -o -name '*.shellcheck_log' -delete
> $(Q)$(RM) $(OUTPUT).config-detected
> $(call QUIET_CLEAN, core-progs) $(RM) $(ALL_PROGRAMS) perf perf-read-vdso32 perf-read-vdsox32 $(OUTPUT)$(LIBJVMTI).so
> $(call QUIET_CLEAN, core-gen) $(RM) *.spec *.pyc *.pyo */*.pyc */*.pyo $(OUTPUT)common-cmds.h TAGS tags cscope* $(OUTPUT)PERF-VERSION-FILE $(OUTPUT)FEATURE-DUMP $(OUTPUT)util/*-bison* $(OUTPUT)util/*-flex* \
> diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
> index 2b45ffa462a6..53ba9c3e20e0 100644
> --- a/tools/perf/tests/Build
> +++ b/tools/perf/tests/Build
> @@ -77,3 +77,17 @@ CFLAGS_python-use.o += -DPYTHONPATH="BUILD_STR($(OUTPUT)python)" -DPYTHON="BUI
> CFLAGS_dwarf-unwind.o += -fno-optimize-sibling-calls
>
> perf-y += workloads/
> +
> +ifdef SHELLCHECK
> + SHELL_TESTS := $(shell find tests/shell -executable -type f -name '*.sh')
> + TEST_LOGS := $(SHELL_TESTS:tests/shell/%=shell/%.shellcheck_log)
> +else
> + SHELL_TESTS :=
> + TEST_LOGS :=
> +endif
> +
> +$(OUTPUT)%.shellcheck_log: %
> + $(call rule_mkdir)
> + $(Q)$(call echo-cmd,test)shellcheck -a -S warning "$<" > $@ || (cat $@ && rm $@ && false)
> +
> +perf-y += $(TEST_LOGS)
> diff --git a/tools/perf/tests/Makefile.tests b/tools/perf/tests/Makefile.tests
> deleted file mode 100644
> index fdaca5f7a946..000000000000
> --- a/tools/perf/tests/Makefile.tests
> +++ /dev/null
> @@ -1,22 +0,0 @@
> -# SPDX-License-Identifier: GPL-2.0
> -# Athira Rajeev <[email protected]>, 2023
> -
> -PROGS := $(shell find tests/shell -perm -o=x -type f -name '*.sh')
> -FILE_NAME := $(notdir $(PROGS))
> -FILE_NAME := $(FILE_NAME:%=.%)
> -LOGS := $(join $(dir $(PROGS)),$(FILE_NAME))
> -LOGS := $(LOGS:%=%.shellcheck_log)
> -
> -.PHONY: all
> -all: SHELLCHECK_RUN
> - @:
> -
> -SHELLCHECK_RUN: $(LOGS)
> -
> -.%.shellcheck_log: %
> - $(call rule_mkdir)
> - $(Q)$(call frecho-cmd,test)@shellcheck -S warning "$<" > $@ || (cat $@ && rm $@ && false)
> -
> -clean:
> - $(eval log_files := $(shell find . -name '.*.shellcheck_log'))
> - @rm -rf $(log_files)
> --
> 2.43.0.rc1.413.gea7ed67945-goog
>

2023-12-01 06:22:40

by Athira Rajeev

[permalink] [raw]
Subject: Re: [PATCH v2 1/4] perf build: Shellcheck support for OUTPUT directory



> On 30-Nov-2023, at 3:04 AM, Ian Rogers <[email protected]> wrote:
>
> Migrate Makefile.tests to Build so that variables like rule_mkdir are
> defined via Makefile.build (needed so the output directory can be
> created). This requires SHELLCHECK being exported and the clean rule
> tweaking to remove the files in find.
>
> Change find "-perm -o=x" as it was failing on my Debian based Linux
> kernel tree, switch to using "-executable".
>
> Adding a filename prefix of "." to the shellcheck log files is a pain
> and error prone in make, remove this prefix and just add the
> shellcheck log files to .gitignore.
>
> Fix the command echo so that running the test is displayed.
>
> Fixes: 1638b11ef815 ("perf tools: Add perf binary dependent rule for shellcheck log in Makefile.perf")
> Signed-off-by: Ian Rogers <[email protected]>

Hi Ian,

Changes looks good to me.
Tested with make, make clean, make with shellcheck error, make with NO_SHELLCHECK

Reviewed-by: Athira Rajeev <[email protected]>

Thanks
Athira
> ---
> tools/perf/.gitignore | 3 +++
> tools/perf/Makefile.perf | 30 ++++++++++--------------------
> tools/perf/tests/Build | 14 ++++++++++++++
> tools/perf/tests/Makefile.tests | 22 ----------------------
> 4 files changed, 27 insertions(+), 42 deletions(-)
> delete mode 100644 tools/perf/tests/Makefile.tests
>
> diff --git a/tools/perf/.gitignore b/tools/perf/.gitignore
> index ee5c14f3b8b1..f5b81d439387 100644
> --- a/tools/perf/.gitignore
> +++ b/tools/perf/.gitignore
> @@ -39,6 +39,9 @@ trace/beauty/generated/
> pmu-events/pmu-events.c
> pmu-events/jevents
> pmu-events/metric_test.log
> +tests/shell/*.shellcheck_log
> +tests/shell/coresight/*.shellcheck_log
> +tests/shell/lib/*.shellcheck_log
> feature/
> libapi/
> libbpf/
> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index 824cbc0af7d7..1ab2a908f240 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -229,8 +229,15 @@ else
> force_fixdep := $(config)
> endif
>
> +# Runs shellcheck on perf test shell scripts
> +ifeq ($(NO_SHELLCHECK),1)
> + SHELLCHECK :=
> +else
> + SHELLCHECK := $(shell which shellcheck 2> /dev/null)
> +endif
> +
> export srctree OUTPUT RM CC CXX LD AR CFLAGS CXXFLAGS V BISON FLEX AWK
> -export HOSTCC HOSTLD HOSTAR HOSTCFLAGS
> +export HOSTCC HOSTLD HOSTAR HOSTCFLAGS SHELLCHECK
>
> include $(srctree)/tools/build/Makefile.include
>
> @@ -673,23 +680,7 @@ $(PERF_IN): prepare FORCE
> $(PMU_EVENTS_IN): FORCE prepare
> $(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=pmu-events obj=pmu-events
>
> -# Runs shellcheck on perf test shell scripts
> -
> -SHELLCHECK := $(shell which shellcheck 2> /dev/null)
> -
> -ifeq ($(NO_SHELLCHECK),1)
> -SHELLCHECK :=
> -endif
> -
> -ifneq ($(SHELLCHECK),)
> -SHELLCHECK_TEST: FORCE prepare
> - $(Q)$(MAKE) -f $(srctree)/tools/perf/tests/Makefile.tests
> -else
> -SHELLCHECK_TEST:
> - @:
> -endif
> -
> -$(OUTPUT)perf: $(PERFLIBS) $(PERF_IN) $(PMU_EVENTS_IN) SHELLCHECK_TEST
> +$(OUTPUT)perf: $(PERFLIBS) $(PERF_IN) $(PMU_EVENTS_IN)
> $(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) \
> $(PERF_IN) $(PMU_EVENTS_IN) $(LIBS) -o $@
>
> @@ -1152,9 +1143,8 @@ bpf-skel-clean:
> $(call QUIET_CLEAN, bpf-skel) $(RM) -r $(SKEL_TMP_OUT) $(SKELETONS)
>
> clean:: $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clean $(LIBSYMBOL)-clean $(LIBPERF)-clean fixdep-clean python-clean bpf-skel-clean tests-coresight-targets-clean
> - $(Q)$(MAKE) -f $(srctree)/tools/perf/tests/Makefile.tests clean
> $(call QUIET_CLEAN, core-objs) $(RM) $(LIBPERF_A) $(OUTPUT)perf-archive $(OUTPUT)perf-iostat $(LANG_BINDINGS)
> - $(Q)find $(or $(OUTPUT),.) -name '*.o' -delete -o -name '\.*.cmd' -delete -o -name '\.*.d' -delete
> + $(Q)find $(or $(OUTPUT),.) -name '*.o' -delete -o -name '\.*.cmd' -delete -o -name '\.*.d' -delete -o -name '*.shellcheck_log' -delete
> $(Q)$(RM) $(OUTPUT).config-detected
> $(call QUIET_CLEAN, core-progs) $(RM) $(ALL_PROGRAMS) perf perf-read-vdso32 perf-read-vdsox32 $(OUTPUT)$(LIBJVMTI).so
> $(call QUIET_CLEAN, core-gen) $(RM) *.spec *.pyc *.pyo */*.pyc */*.pyo $(OUTPUT)common-cmds.h TAGS tags cscope* $(OUTPUT)PERF-VERSION-FILE $(OUTPUT)FEATURE-DUMP $(OUTPUT)util/*-bison* $(OUTPUT)util/*-flex* \
> diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
> index 2b45ffa462a6..53ba9c3e20e0 100644
> --- a/tools/perf/tests/Build
> +++ b/tools/perf/tests/Build
> @@ -77,3 +77,17 @@ CFLAGS_python-use.o += -DPYTHONPATH="BUILD_STR($(OUTPUT)python)" -DPYTHON="BUI
> CFLAGS_dwarf-unwind.o += -fno-optimize-sibling-calls
>
> perf-y += workloads/
> +
> +ifdef SHELLCHECK
> + SHELL_TESTS := $(shell find tests/shell -executable -type f -name '*.sh')
> + TEST_LOGS := $(SHELL_TESTS:tests/shell/%=shell/%.shellcheck_log)
> +else
> + SHELL_TESTS :=
> + TEST_LOGS :=
> +endif
> +
> +$(OUTPUT)%.shellcheck_log: %
> + $(call rule_mkdir)
> + $(Q)$(call echo-cmd,test)shellcheck -a -S warning "$<" > $@ || (cat $@ && rm $@ && false)
> +
> +perf-y += $(TEST_LOGS)
> diff --git a/tools/perf/tests/Makefile.tests b/tools/perf/tests/Makefile.tests
> deleted file mode 100644
> index fdaca5f7a946..000000000000
> --- a/tools/perf/tests/Makefile.tests
> +++ /dev/null
> @@ -1,22 +0,0 @@
> -# SPDX-License-Identifier: GPL-2.0
> -# Athira Rajeev <[email protected]>, 2023
> -
> -PROGS := $(shell find tests/shell -perm -o=x -type f -name '*.sh')
> -FILE_NAME := $(notdir $(PROGS))
> -FILE_NAME := $(FILE_NAME:%=.%)
> -LOGS := $(join $(dir $(PROGS)),$(FILE_NAME))
> -LOGS := $(LOGS:%=%.shellcheck_log)
> -
> -.PHONY: all
> -all: SHELLCHECK_RUN
> - @:
> -
> -SHELLCHECK_RUN: $(LOGS)
> -
> -.%.shellcheck_log: %
> - $(call rule_mkdir)
> - $(Q)$(call frecho-cmd,test)@shellcheck -S warning "$<" > $@ || (cat $@ && rm $@ && false)
> -
> -clean:
> - $(eval log_files := $(shell find . -name '.*.shellcheck_log'))
> - @rm -rf $(log_files)
> --
> 2.43.0.rc1.413.gea7ed67945-goog
>

2023-12-04 19:45:53

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: [PATCH v2 1/4] perf build: Shellcheck support for OUTPUT directory

Em Fri, Dec 01, 2023 at 11:49:59AM +0530, Athira Rajeev escreveu:
>
>
> > On 30-Nov-2023, at 3:04 AM, Ian Rogers <[email protected]> wrote:
> >
> > Migrate Makefile.tests to Build so that variables like rule_mkdir are
> > defined via Makefile.build (needed so the output directory can be
> > created). This requires SHELLCHECK being exported and the clean rule
> > tweaking to remove the files in find.
> >
> > Change find "-perm -o=x" as it was failing on my Debian based Linux
> > kernel tree, switch to using "-executable".
> >
> > Adding a filename prefix of "." to the shellcheck log files is a pain
> > and error prone in make, remove this prefix and just add the
> > shellcheck log files to .gitignore.
> >
> > Fix the command echo so that running the test is displayed.
> >
> > Fixes: 1638b11ef815 ("perf tools: Add perf binary dependent rule for shellcheck log in Makefile.perf")
> > Signed-off-by: Ian Rogers <[email protected]>
>
> Hi Ian,
>
> Changes looks good to me.
> Tested with make, make clean, make with shellcheck error, make with NO_SHELLCHECK
>
> Reviewed-by: Athira Rajeev <[email protected]>

Next time please reply with your Reviewed-by to the cover letter, so
that b4 stamps your reviewed-by to all the patches and not just to the
patch that you replied to.

This time I'll take the plural in "Changes look good to me" to signify
that you reviewed the whole series, ok?

- Arnaldo

2023-12-05 07:33:52

by Athira Rajeev

[permalink] [raw]
Subject: Re: [PATCH v2 1/4] perf build: Shellcheck support for OUTPUT directory



> On 05-Dec-2023, at 1:15 AM, Arnaldo Carvalho de Melo <[email protected]> wrote:
>
> Em Fri, Dec 01, 2023 at 11:49:59AM +0530, Athira Rajeev escreveu:
>>
>>
>>> On 30-Nov-2023, at 3:04 AM, Ian Rogers <[email protected]> wrote:
>>>
>>> Migrate Makefile.tests to Build so that variables like rule_mkdir are
>>> defined via Makefile.build (needed so the output directory can be
>>> created). This requires SHELLCHECK being exported and the clean rule
>>> tweaking to remove the files in find.
>>>
>>> Change find "-perm -o=x" as it was failing on my Debian based Linux
>>> kernel tree, switch to using "-executable".
>>>
>>> Adding a filename prefix of "." to the shellcheck log files is a pain
>>> and error prone in make, remove this prefix and just add the
>>> shellcheck log files to .gitignore.
>>>
>>> Fix the command echo so that running the test is displayed.
>>>
>>> Fixes: 1638b11ef815 ("perf tools: Add perf binary dependent rule for shellcheck log in Makefile.perf")
>>> Signed-off-by: Ian Rogers <[email protected]>
>>
>> Hi Ian,
>>
>> Changes looks good to me.
>> Tested with make, make clean, make with shellcheck error, make with NO_SHELLCHECK
>>
>> Reviewed-by: Athira Rajeev <[email protected]>
>
> Next time please reply with your Reviewed-by to the cover letter, so
> that b4 stamps your reviewed-by to all the patches and not just to the
> patch that you replied to.
Hi Arnaldo,

Ok Sure.
>
> This time I'll take the plural in "Changes look good to me" to signify
> that you reviewed the whole series, ok?

Yes, please add my Reviewed-by for the whole series.

Thanks
Athira
>
> - Arnaldo