2023-10-30 11:17:08

by Yang Jihong

Subject: [PATCH 1/2] perf build: Add loading python binding check to python.so build

Add loading python binding check to python.so build so that problem can be
detected in advance in the build phase instead of being left to `perf test`
phase.

In normal scenarios, the original build is not affected:

$ cd tools/perf
$ rm -rf /tmp/perf; mkdir /tmp/perf; make O=/tmp/perf
$ echo $?
0
$ cd /tmp/perf
$ ./perf test python
19: 'import perf' in python : Ok

Create an error scenario, for example, delete util/rlimit.c from
util/python-ext-sources:

$ cd tools/perf
$ sed -i 's@util/rlimit.c@#util/rlimit.c@g' util/python-ext-sources
$ grep rlimit util/python-ext-sources
#util/rlimit.c
$ rm -rf /tmp/perf; mkdir /tmp/perf; make JOBS=1 O=/tmp/perf
<SNIP>
GEN /tmp/perf/python/perf.cpython-310-x86_64-linux-gnu.so
Error: Load python binding failed. See /tmp/perf/python_ext_build/lib//build.log for more details
make[2]: *** [Makefile.perf:644: /tmp/perf/python/perf.cpython-310-x86_64-linux-gnu.so] Error 1
make[1]: *** [Makefile.perf:242: sub-make] Error 2
make: *** [Makefile:70: all] Error 2
$ cat /tmp/perf/python_ext_build/lib//build.log
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: /tmp/perf/python_ext_build/lib/perf.cpython-310-x86_64-linux-gnu.so: undefined symbol: rlimit__increase_nofile

Signed-off-by: Yang Jihong <[email protected]>
---
tools/perf/Makefile.perf | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index d80dcaa5a1e3..a2449c4890ad 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -645,7 +645,13 @@ $(OUTPUT)python/perf$(PYTHON_EXTENSION_SUFFIX): $(PYTHON_EXT_SRCS) $(PYTHON_EXT_
CFLAGS='$(CFLAGS)' LDFLAGS='$(LDFLAGS)' \
$(PYTHON_WORD) util/setup.py \
--quiet build_ext; \
- cp $(PYTHON_EXTBUILD_LIB)perf*.so $(OUTPUT)python/
+ $(PYTHON_WORD) -c 'import sys; sys.path.insert(0, "$(PYTHON_EXTBUILD_LIB)"); import perf' 2>$(PYTHON_EXTBUILD_LIB)/build.log; \
+ if [ $$? -ne 0 ]; then \
+ echo "Error: Load python binding failed. See $(PYTHON_EXTBUILD_LIB)/build.log for more details"; \
+ exit 1; \
+ else \
+ cp $(PYTHON_EXTBUILD_LIB)perf*.so $(OUTPUT)python/; \
+ fi

python_perf_target:
@echo "Target is: $(OUTPUT)python/perf$(PYTHON_EXTENSION_SUFFIX)"
--
2.34.1

2023-11-01 04:52:02

by Namhyung Kim

Subject: Re: [PATCH 1/2] perf build: Add loading python binding check to python.so build

Hello,

On Mon, Oct 30, 2023 at 4:16 AM Yang Jihong <[email protected]> wrote:
>
> Add loading python binding check to python.so build so that problem can be
> detected in advance in the build phase instead of being left to `perf test`
> phase.
>
> In normal scenarios, the original build is not affected:
>
> $ cd tools/perf
> $ rm -rf /tmp/perf; mkdir /tmp/perf; make O=/tmp/perf
> $ echo $?
> 0
> $ cd /tmp/perf
> $ ./perf test python
> 19: 'import perf' in python : Ok
>
> Create an error scenario, for example, delete util/rlimit.c from
> util/python-ext-sources:
>
> $ cd tools/perf
> $ sed -i 's@util/rlimit.c@#util/rlimit.c@g' util/python-ext-sources
> $ grep rlimit util/python-ext-sources
> #util/rlimit.c
> $ rm -rf /tmp/perf; mkdir /tmp/perf; make JOBS=1 O=/tmp/perf
> <SNIP>
> GEN /tmp/perf/python/perf.cpython-310-x86_64-linux-gnu.so
> Error: Load python binding failed. See /tmp/perf/python_ext_build/lib//build.log for more details
> make[2]: *** [Makefile.perf:644: /tmp/perf/python/perf.cpython-310-x86_64-linux-gnu.so] Error 1
> make[1]: *** [Makefile.perf:242: sub-make] Error 2
> make: *** [Makefile:70: all] Error 2
> $ cat /tmp/perf/python_ext_build/lib//build.log
> Traceback (most recent call last):
> File "<string>", line 1, in <module>
> ImportError: /tmp/perf/python_ext_build/lib/perf.cpython-310-x86_64-linux-gnu.so: undefined symbol: rlimit__increase_nofile
>
> Signed-off-by: Yang Jihong <[email protected]>

For both patches,
Acked-by: Namhyung Kim <[email protected]>

Thanks,
Namhyung


> ---
> tools/perf/Makefile.perf | 8 +++++++-
> 1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index d80dcaa5a1e3..a2449c4890ad 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -645,7 +645,13 @@ $(OUTPUT)python/perf$(PYTHON_EXTENSION_SUFFIX): $(PYTHON_EXT_SRCS) $(PYTHON_EXT_
> CFLAGS='$(CFLAGS)' LDFLAGS='$(LDFLAGS)' \
> $(PYTHON_WORD) util/setup.py \
> --quiet build_ext; \
> - cp $(PYTHON_EXTBUILD_LIB)perf*.so $(OUTPUT)python/
> + $(PYTHON_WORD) -c 'import sys; sys.path.insert(0, "$(PYTHON_EXTBUILD_LIB)"); import perf' 2>$(PYTHON_EXTBUILD_LIB)/build.log; \
> + if [ $$? -ne 0 ]; then \
> + echo "Error: Load python binding failed. See $(PYTHON_EXTBUILD_LIB)/build.log for more details"; \
> + exit 1; \
> + else \
> + cp $(PYTHON_EXTBUILD_LIB)perf*.so $(OUTPUT)python/; \
> + fi
>
> python_perf_target:
> @echo "Target is: $(OUTPUT)python/perf$(PYTHON_EXTENSION_SUFFIX)"
> --
> 2.34.1
>