2020-02-13 18:22:19

by He Zhe

[permalink] [raw]
Subject: [PATCH 1/2] perf: Be compatible with all python versions when fetching ldflags

From: He Zhe <[email protected]>

Since Python v3.8.0, with the following commit
0a8e57248b91 ("bpo-36721: Add --embed option to python-config (GH-13500)"),
--embed option must be passed to "python3-config --ldflags --embed" or
"python3-config --libs --embed" to get "-lpython3.8".

To make it compatible with all Python versons, according to the suggestion
in the commit log, we try with --embed first and then witout it if fails.

Signed-off-by: He Zhe <[email protected]>
---
tools/perf/Makefile.config | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config
index 80e55e7..b2eabcf 100644
--- a/tools/perf/Makefile.config
+++ b/tools/perf/Makefile.config
@@ -229,7 +229,7 @@ strip-libs = $(filter-out -l%,$(1))
PYTHON_CONFIG_SQ := $(call shell-sq,$(PYTHON_CONFIG))

ifdef PYTHON_CONFIG
- PYTHON_EMBED_LDOPTS := $(shell $(PYTHON_CONFIG_SQ) --ldflags 2>/dev/null)
+ PYTHON_EMBED_LDOPTS := $(shell $(PYTHON_CONFIG_SQ) --ldflags --embed 2>/dev/null || $(PYTHON_CONFIG_SQ) --ldflags 2>/dev/null)
PYTHON_EMBED_LDFLAGS := $(call strip-libs,$(PYTHON_EMBED_LDOPTS))
PYTHON_EMBED_LIBADD := $(call grep-libs,$(PYTHON_EMBED_LDOPTS)) -lutil
PYTHON_EMBED_CCOPTS := $(shell $(PYTHON_CONFIG_SQ) --includes 2>/dev/null)
--
2.7.4


2020-02-13 18:24:12

by He Zhe

[permalink] [raw]
Subject: [PATCH 2/2] perf: Normalize gcc parameter when generating arch errno table

From: He Zhe <[email protected]>

The $(CC) passed to arch_errno_names.sh may include a series of parameters
along with gcc itself. To avoid overwriting the following parameters of
arch_errno_names.sh and break the build like below, we just pick up the
first word of the $(CC).

find: unknown predicate `-m64/arch'
x86_64-wrs-linux-gcc: warning: '-x c' after last input file has no effect
x86_64-wrs-linux-gcc: error: unrecognized command line option '-m64/include/uapi/asm-generic/errno.h'
x86_64-wrs-linux-gcc: fatal error: no input files

Signed-off-by: He Zhe <[email protected]>
---
tools/perf/Makefile.perf | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 3eda9d4..7114c11 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -573,7 +573,7 @@ arch_errno_hdr_dir := $(srctree)/tools
arch_errno_tbl := $(srctree)/tools/perf/trace/beauty/arch_errno_names.sh

$(arch_errno_name_array): $(arch_errno_tbl)
- $(Q)$(SHELL) '$(arch_errno_tbl)' $(CC) $(arch_errno_hdr_dir) > $@
+ $(Q)$(SHELL) '$(arch_errno_tbl)' $(firstword $(CC)) $(arch_errno_hdr_dir) > $@

sync_file_range_arrays := $(beauty_outdir)/sync_file_range_arrays.c
sync_file_range_tbls := $(srctree)/tools/perf/trace/beauty/sync_file_range.sh
--
2.7.4

2020-02-16 22:26:49

by Jiri Olsa

[permalink] [raw]
Subject: Re: [PATCH 1/2] perf: Be compatible with all python versions when fetching ldflags

On Fri, Feb 14, 2020 at 02:21:05AM +0800, [email protected] wrote:
> From: He Zhe <[email protected]>
>
> Since Python v3.8.0, with the following commit
> 0a8e57248b91 ("bpo-36721: Add --embed option to python-config (GH-13500)"),

we got similar change recently.. might have not been picked up yet

https://lore.kernel.org/lkml/[email protected]/

jirka

> --embed option must be passed to "python3-config --ldflags --embed" or
> "python3-config --libs --embed" to get "-lpython3.8".
>
> To make it compatible with all Python versons, according to the suggestion
> in the commit log, we try with --embed first and then witout it if fails.
>
> Signed-off-by: He Zhe <[email protected]>
> ---
> tools/perf/Makefile.config | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config
> index 80e55e7..b2eabcf 100644
> --- a/tools/perf/Makefile.config
> +++ b/tools/perf/Makefile.config
> @@ -229,7 +229,7 @@ strip-libs = $(filter-out -l%,$(1))
> PYTHON_CONFIG_SQ := $(call shell-sq,$(PYTHON_CONFIG))
>
> ifdef PYTHON_CONFIG
> - PYTHON_EMBED_LDOPTS := $(shell $(PYTHON_CONFIG_SQ) --ldflags 2>/dev/null)
> + PYTHON_EMBED_LDOPTS := $(shell $(PYTHON_CONFIG_SQ) --ldflags --embed 2>/dev/null || $(PYTHON_CONFIG_SQ) --ldflags 2>/dev/null)
> PYTHON_EMBED_LDFLAGS := $(call strip-libs,$(PYTHON_EMBED_LDOPTS))
> PYTHON_EMBED_LIBADD := $(call grep-libs,$(PYTHON_EMBED_LDOPTS)) -lutil
> PYTHON_EMBED_CCOPTS := $(shell $(PYTHON_CONFIG_SQ) --includes 2>/dev/null)
> --
> 2.7.4
>

2020-02-17 04:01:03

by He Zhe

[permalink] [raw]
Subject: Re: [PATCH 1/2] perf: Be compatible with all python versions when fetching ldflags



On 2/17/20 6:22 AM, Jiri Olsa wrote:
> On Fri, Feb 14, 2020 at 02:21:05AM +0800, [email protected] wrote:
>> From: He Zhe <[email protected]>
>>
>> Since Python v3.8.0, with the following commit
>> 0a8e57248b91 ("bpo-36721: Add --embed option to python-config (GH-13500)"),
> we got similar change recently.. might have not been picked up yet
>
> https://lore.kernel.org/lkml/[email protected]/

Thanks for pointing out.

Zhe

>
> jirka
>
>> --embed option must be passed to "python3-config --ldflags --embed" or
>> "python3-config --libs --embed" to get "-lpython3.8".
>>
>> To make it compatible with all Python versons, according to the suggestion
>> in the commit log, we try with --embed first and then witout it if fails.
>>
>> Signed-off-by: He Zhe <[email protected]>
>> ---
>> tools/perf/Makefile.config | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config
>> index 80e55e7..b2eabcf 100644
>> --- a/tools/perf/Makefile.config
>> +++ b/tools/perf/Makefile.config
>> @@ -229,7 +229,7 @@ strip-libs = $(filter-out -l%,$(1))
>> PYTHON_CONFIG_SQ := $(call shell-sq,$(PYTHON_CONFIG))
>>
>> ifdef PYTHON_CONFIG
>> - PYTHON_EMBED_LDOPTS := $(shell $(PYTHON_CONFIG_SQ) --ldflags 2>/dev/null)
>> + PYTHON_EMBED_LDOPTS := $(shell $(PYTHON_CONFIG_SQ) --ldflags --embed 2>/dev/null || $(PYTHON_CONFIG_SQ) --ldflags 2>/dev/null)
>> PYTHON_EMBED_LDFLAGS := $(call strip-libs,$(PYTHON_EMBED_LDOPTS))
>> PYTHON_EMBED_LIBADD := $(call grep-libs,$(PYTHON_EMBED_LDOPTS)) -lutil
>> PYTHON_EMBED_CCOPTS := $(shell $(PYTHON_CONFIG_SQ) --includes 2>/dev/null)
>> --
>> 2.7.4
>>

2020-03-25 13:08:27

by He Zhe

[permalink] [raw]
Subject: Re: [PATCH 2/2] perf: Normalize gcc parameter when generating arch errno table

Can this be considered for the moment?

Thanks,
Zhe

On 2/14/20 2:21 AM, [email protected] wrote:
> From: He Zhe <[email protected]>
>
> The $(CC) passed to arch_errno_names.sh may include a series of parameters
> along with gcc itself. To avoid overwriting the following parameters of
> arch_errno_names.sh and break the build like below, we just pick up the
> first word of the $(CC).
>
> find: unknown predicate `-m64/arch'
> x86_64-wrs-linux-gcc: warning: '-x c' after last input file has no effect
> x86_64-wrs-linux-gcc: error: unrecognized command line option '-m64/include/uapi/asm-generic/errno.h'
> x86_64-wrs-linux-gcc: fatal error: no input files
>
> Signed-off-by: He Zhe <[email protected]>
> ---
> tools/perf/Makefile.perf | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index 3eda9d4..7114c11 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -573,7 +573,7 @@ arch_errno_hdr_dir := $(srctree)/tools
> arch_errno_tbl := $(srctree)/tools/perf/trace/beauty/arch_errno_names.sh
>
> $(arch_errno_name_array): $(arch_errno_tbl)
> - $(Q)$(SHELL) '$(arch_errno_tbl)' $(CC) $(arch_errno_hdr_dir) > $@
> + $(Q)$(SHELL) '$(arch_errno_tbl)' $(firstword $(CC)) $(arch_errno_hdr_dir) > $@
>
> sync_file_range_arrays := $(beauty_outdir)/sync_file_range_arrays.c
> sync_file_range_tbls := $(srctree)/tools/perf/trace/beauty/sync_file_range.sh

2020-03-25 13:32:19

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: [PATCH 1/2] perf: Be compatible with all python versions when fetching ldflags

Em Mon, Feb 17, 2020 at 10:24:27AM +0800, He Zhe escreveu:
>
>
> On 2/17/20 6:22 AM, Jiri Olsa wrote:
> > On Fri, Feb 14, 2020 at 02:21:05AM +0800, [email protected] wrote:
> >> From: He Zhe <[email protected]>
> >>
> >> Since Python v3.8.0, with the following commit
> >> 0a8e57248b91 ("bpo-36721: Add --embed option to python-config (GH-13500)"),
> > we got similar change recently.. might have not been picked up yet
> >
> > https://lore.kernel.org/lkml/[email protected]/
>
> Thanks for pointing out.

So, just with your patch:

[acme@five perf]$ rm -rf /tmp/build/perf ; mkdir -p /tmp/build/perf
[acme@five perf]$ make PYTHON=python3 -C tools/perf O=/tmp/build/perf install-bin |& grep python
... libpython: [ OFF ]
Makefile.config:750: No 'Python.h' (for Python 2.x support) was found: disables Python support - please install python-devel/python-dev
CC /tmp/build/perf/tests/python-use.o
[acme@five perf]$

[acme@five perf]$ rpm -q python2-devel python3-devel python-devel
package python2-devel is not installed
python3-devel-3.7.6-2.fc31.x86_64
package python-devel is not installed
[acme@five perf]$

[acme@five perf]$ cat /tmp/build/perf/feature/test-libpython.make.output
/bin/sh: --configdir: command not found
[acme@five perf]$ cat /tmp/build/perf/feature/test-libpython
test-libpython.make.output test-libpython-version.make.output
[acme@five perf]$ cat /tmp/build/perf/feature/test-libpython-version.make.output
/bin/sh: --configdir: command not found
[acme@five perf]$


Without your patch:

[acme@five perf]$ rm -rf /tmp/build/perf ; mkdir -p /tmp/build/perf
[acme@five perf]$ make PYTHON=python3 -C tools/perf O=/tmp/build/perf install-bin |& grep python
... libpython: [ on ]
GEN /tmp/build/perf/python/perf.so
MKDIR /tmp/build/perf/scripts/python/Perf-Trace-Util/
CC /tmp/build/perf/scripts/python/Perf-Trace-Util/Context.o
LD /tmp/build/perf/scripts/python/Perf-Trace-Util/perf-in.o
CC /tmp/build/perf/tests/python-use.o
CC /tmp/build/perf/util/scripting-engines/trace-event-python.o
INSTALL python-scripts
[acme@five perf]$

[acme@five perf]$ ldd /tmp/build/perf/perf |& grep python
libpython3.7m.so.1.0 => /lib64/libpython3.7m.so.1.0 (0x00007f11dd1ee000)
[acme@five perf]$ perf -vv |& grep -i python
libpython: [ on ] # HAVE_LIBPYTHON_SUPPORT
[acme@five perf]$

What am I missing?


[acme@five perf]$ cat /etc/redhat-release
Fedora release 31 (Thirty One)
[acme@five perf]$

- Arnaldo

2020-03-25 14:42:51

by Sam Lunt

[permalink] [raw]
Subject: Re: [PATCH 1/2] perf: Be compatible with all python versions when fetching ldflags

On Wed, Mar 25, 2020 at 8:30 AM Arnaldo Carvalho de Melo
<[email protected]> wrote:
>
> Em Mon, Feb 17, 2020 at 10:24:27AM +0800, He Zhe escreveu:
> >
> >
> > On 2/17/20 6:22 AM, Jiri Olsa wrote:
> > > On Fri, Feb 14, 2020 at 02:21:05AM +0800, [email protected] wrote:
> > >> From: He Zhe <[email protected]>
> > >>
> > >> Since Python v3.8.0, with the following commit
> > >> 0a8e57248b91 ("bpo-36721: Add --embed option to python-config (GH-13500)"),
> > > we got similar change recently.. might have not been picked up yet
> > >
> > > https://lore.kernel.org/lkml/[email protected]/
> >
> > Thanks for pointing out.
>
> So, just with your patch:
>
> [acme@five perf]$ rm -rf /tmp/build/perf ; mkdir -p /tmp/build/perf
> [acme@five perf]$ make PYTHON=python3 -C tools/perf O=/tmp/build/perf install-bin |& grep python
> ... libpython: [ OFF ]
> Makefile.config:750: No 'Python.h' (for Python 2.x support) was found: disables Python support - please install python-devel/python-dev
> CC /tmp/build/perf/tests/python-use.o
> [acme@five perf]$
>
> [acme@five perf]$ rpm -q python2-devel python3-devel python-devel
> package python2-devel is not installed
> python3-devel-3.7.6-2.fc31.x86_64
> package python-devel is not installed
> [acme@five perf]$
>
> [acme@five perf]$ cat /tmp/build/perf/feature/test-libpython.make.output
> /bin/sh: --configdir: command not found
> [acme@five perf]$ cat /tmp/build/perf/feature/test-libpython
> test-libpython.make.output test-libpython-version.make.output
> [acme@five perf]$ cat /tmp/build/perf/feature/test-libpython-version.make.output
> /bin/sh: --configdir: command not found
> [acme@five perf]$
>
>
> Without your patch:
>
> [acme@five perf]$ rm -rf /tmp/build/perf ; mkdir -p /tmp/build/perf
> [acme@five perf]$ make PYTHON=python3 -C tools/perf O=/tmp/build/perf install-bin |& grep python
> ... libpython: [ on ]
> GEN /tmp/build/perf/python/perf.so
> MKDIR /tmp/build/perf/scripts/python/Perf-Trace-Util/
> CC /tmp/build/perf/scripts/python/Perf-Trace-Util/Context.o
> LD /tmp/build/perf/scripts/python/Perf-Trace-Util/perf-in.o
> CC /tmp/build/perf/tests/python-use.o
> CC /tmp/build/perf/util/scripting-engines/trace-event-python.o
> INSTALL python-scripts
> [acme@five perf]$
>
> [acme@five perf]$ ldd /tmp/build/perf/perf |& grep python
> libpython3.7m.so.1.0 => /lib64/libpython3.7m.so.1.0 (0x00007f11dd1ee000)
> [acme@five perf]$ perf -vv |& grep -i python
> libpython: [ on ] # HAVE_LIBPYTHON_SUPPORT
> [acme@five perf]$
>
> What am I missing?

It looks like you are using python3.7, but the change in behavior for
python-config happened in version 3.8

> [acme@five perf]$ cat /etc/redhat-release
> Fedora release 31 (Thirty One)
> [acme@five perf]$
>
> - Arnaldo

2020-03-25 19:27:58

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: [PATCH 1/2] perf: Be compatible with all python versions when fetching ldflags

Em Wed, Mar 25, 2020 at 09:40:34AM -0500, Sam Lunt escreveu:
> On Wed, Mar 25, 2020 at 8:30 AM Arnaldo Carvalho de Melo
> <[email protected]> wrote:
> >
> > Em Mon, Feb 17, 2020 at 10:24:27AM +0800, He Zhe escreveu:
> > >
> > >
> > > On 2/17/20 6:22 AM, Jiri Olsa wrote:
> > > > On Fri, Feb 14, 2020 at 02:21:05AM +0800, [email protected] wrote:
> > > >> From: He Zhe <[email protected]>
> > > >>
> > > >> Since Python v3.8.0, with the following commit
> > > >> 0a8e57248b91 ("bpo-36721: Add --embed option to python-config (GH-13500)"),
> > > > we got similar change recently.. might have not been picked up yet
> > > >
> > > > https://lore.kernel.org/lkml/[email protected]/
> > >
> > > Thanks for pointing out.
> >
> > So, just with your patch:
> >
> > [acme@five perf]$ rm -rf /tmp/build/perf ; mkdir -p /tmp/build/perf
> > [acme@five perf]$ make PYTHON=python3 -C tools/perf O=/tmp/build/perf install-bin |& grep python
> > ... libpython: [ OFF ]
> > Makefile.config:750: No 'Python.h' (for Python 2.x support) was found: disables Python support - please install python-devel/python-dev
> > CC /tmp/build/perf/tests/python-use.o
> > [acme@five perf]$
> >
> > [acme@five perf]$ rpm -q python2-devel python3-devel python-devel
> > package python2-devel is not installed
> > python3-devel-3.7.6-2.fc31.x86_64
> > package python-devel is not installed
> > [acme@five perf]$
> >
> > [acme@five perf]$ cat /tmp/build/perf/feature/test-libpython.make.output
> > /bin/sh: --configdir: command not found
> > [acme@five perf]$ cat /tmp/build/perf/feature/test-libpython
> > test-libpython.make.output test-libpython-version.make.output
> > [acme@five perf]$ cat /tmp/build/perf/feature/test-libpython-version.make.output
> > /bin/sh: --configdir: command not found
> > [acme@five perf]$
> >
> >
> > Without your patch:
> >
> > [acme@five perf]$ rm -rf /tmp/build/perf ; mkdir -p /tmp/build/perf
> > [acme@five perf]$ make PYTHON=python3 -C tools/perf O=/tmp/build/perf install-bin |& grep python
> > ... libpython: [ on ]
> > GEN /tmp/build/perf/python/perf.so
> > MKDIR /tmp/build/perf/scripts/python/Perf-Trace-Util/
> > CC /tmp/build/perf/scripts/python/Perf-Trace-Util/Context.o
> > LD /tmp/build/perf/scripts/python/Perf-Trace-Util/perf-in.o
> > CC /tmp/build/perf/tests/python-use.o
> > CC /tmp/build/perf/util/scripting-engines/trace-event-python.o
> > INSTALL python-scripts
> > [acme@five perf]$
> >
> > [acme@five perf]$ ldd /tmp/build/perf/perf |& grep python
> > libpython3.7m.so.1.0 => /lib64/libpython3.7m.so.1.0 (0x00007f11dd1ee000)
> > [acme@five perf]$ perf -vv |& grep -i python
> > libpython: [ on ] # HAVE_LIBPYTHON_SUPPORT
> > [acme@five perf]$
> >
> > What am I missing?
>
> It looks like you are using python3.7, but the change in behavior for
> python-config happened in version 3.8

Humm, but shouldn't this continue to work with python3.7?

- Arnaldo

> > [acme@five perf]$ cat /etc/redhat-release
> > Fedora release 31 (Thirty One)
> > [acme@five perf]$
> >
> > - Arnaldo

--

- Arnaldo

2020-03-25 20:32:19

by Sam Lunt

[permalink] [raw]
Subject: Re: [PATCH 1/2] perf: Be compatible with all python versions when fetching ldflags

On Wed, Mar 25, 2020 at 2:26 PM Arnaldo Carvalho de Melo
<[email protected]> wrote:
>
> Em Wed, Mar 25, 2020 at 09:40:34AM -0500, Sam Lunt escreveu:
> > On Wed, Mar 25, 2020 at 8:30 AM Arnaldo Carvalho de Melo
> > <[email protected]> wrote:
> > >
> > > Em Mon, Feb 17, 2020 at 10:24:27AM +0800, He Zhe escreveu:
> > > >
> > > >
> > > > On 2/17/20 6:22 AM, Jiri Olsa wrote:
> > > > > On Fri, Feb 14, 2020 at 02:21:05AM +0800, [email protected] wrote:
> > > > >> From: He Zhe <[email protected]>
> > > > >>
> > > > >> Since Python v3.8.0, with the following commit
> > > > >> 0a8e57248b91 ("bpo-36721: Add --embed option to python-config (GH-13500)"),
> > > > > we got similar change recently.. might have not been picked up yet
> > > > >
> > > > > https://lore.kernel.org/lkml/[email protected]/
> > > >
> > > > Thanks for pointing out.
> > >
> > > So, just with your patch:
> > >
> > > [acme@five perf]$ rm -rf /tmp/build/perf ; mkdir -p /tmp/build/perf
> > > [acme@five perf]$ make PYTHON=python3 -C tools/perf O=/tmp/build/perf install-bin |& grep python
> > > ... libpython: [ OFF ]
> > > Makefile.config:750: No 'Python.h' (for Python 2.x support) was found: disables Python support - please install python-devel/python-dev
> > > CC /tmp/build/perf/tests/python-use.o
> > > [acme@five perf]$
> > >
> > > [acme@five perf]$ rpm -q python2-devel python3-devel python-devel
> > > package python2-devel is not installed
> > > python3-devel-3.7.6-2.fc31.x86_64
> > > package python-devel is not installed
> > > [acme@five perf]$
> > >
> > > [acme@five perf]$ cat /tmp/build/perf/feature/test-libpython.make.output
> > > /bin/sh: --configdir: command not found
> > > [acme@five perf]$ cat /tmp/build/perf/feature/test-libpython
> > > test-libpython.make.output test-libpython-version.make.output
> > > [acme@five perf]$ cat /tmp/build/perf/feature/test-libpython-version.make.output
> > > /bin/sh: --configdir: command not found
> > > [acme@five perf]$
> > >
> > >
> > > Without your patch:
> > >
> > > [acme@five perf]$ rm -rf /tmp/build/perf ; mkdir -p /tmp/build/perf
> > > [acme@five perf]$ make PYTHON=python3 -C tools/perf O=/tmp/build/perf install-bin |& grep python
> > > ... libpython: [ on ]
> > > GEN /tmp/build/perf/python/perf.so
> > > MKDIR /tmp/build/perf/scripts/python/Perf-Trace-Util/
> > > CC /tmp/build/perf/scripts/python/Perf-Trace-Util/Context.o
> > > LD /tmp/build/perf/scripts/python/Perf-Trace-Util/perf-in.o
> > > CC /tmp/build/perf/tests/python-use.o
> > > CC /tmp/build/perf/util/scripting-engines/trace-event-python.o
> > > INSTALL python-scripts
> > > [acme@five perf]$
> > >
> > > [acme@five perf]$ ldd /tmp/build/perf/perf |& grep python
> > > libpython3.7m.so.1.0 => /lib64/libpython3.7m.so.1.0 (0x00007f11dd1ee000)
> > > [acme@five perf]$ perf -vv |& grep -i python
> > > libpython: [ on ] # HAVE_LIBPYTHON_SUPPORT
> > > [acme@five perf]$
> > >
> > > What am I missing?
> >
> > It looks like you are using python3.7, but the change in behavior for
> > python-config happened in version 3.8
>
> Humm, but shouldn't this continue to work with python3.7?

Oh, my mistake, I didn't read the output carefully. It should
obviously still work with old versions, yes. I actually submitted a
similar patch, and it seemed to work when I used python 3.7. I wonder
if the issue is the "||" operator in the subshell.

https://lore.kernel.org/lkml/[email protected]/


> - Arnaldo
>
> > > [acme@five perf]$ cat /etc/redhat-release
> > > Fedora release 31 (Thirty One)
> > > [acme@five perf]$
> > >
> > > - Arnaldo
>
> --
>
> - Arnaldo

2020-03-26 01:08:30

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: [PATCH 1/2] perf: Be compatible with all python versions when fetching ldflags



On March 25, 2020 5:31:15 PM GMT-03:00, Sam Lunt <[email protected]> wrote:
>On Wed, Mar 25, 2020 at 2:26 PM Arnaldo Carvalho de Melo
><[email protected]> wrote:
>>
>> Em Wed, Mar 25, 2020 at 09:40:34AM -0500, Sam Lunt escreveu:
>> > On Wed, Mar 25, 2020 at 8:30 AM Arnaldo Carvalho de Melo
>> > <[email protected]> wrote:
>> > >
>> > > Em Mon, Feb 17, 2020 at 10:24:27AM +0800, He Zhe escreveu:
>> > > >
>> > > >
>> > > > On 2/17/20 6:22 AM, Jiri Olsa wrote:
>> > > > > On Fri, Feb 14, 2020 at 02:21:05AM +0800,
>[email protected] wrote:
>> > > > >> From: He Zhe <[email protected]>
>> > > > >>
>> > > > >> Since Python v3.8.0, with the following commit
>> > > > >> 0a8e57248b91 ("bpo-36721: Add --embed option to
>python-config (GH-13500)"),
>> > > > > we got similar change recently.. might have not been picked
>up yet
>> > > > >
>> > > > >
>https://lore.kernel.org/lkml/[email protected]/
>> > > >
>> > > > Thanks for pointing out.
>> > >
>> > > So, just with your patch:
>> > >
>> > > [acme@five perf]$ rm -rf /tmp/build/perf ; mkdir -p
>/tmp/build/perf
>> > > [acme@five perf]$ make PYTHON=python3 -C tools/perf
>O=/tmp/build/perf install-bin |& grep python
>> > > ... libpython: [ OFF ]
>> > > Makefile.config:750: No 'Python.h' (for Python 2.x support) was
>found: disables Python support - please install python-devel/python-dev
>> > > CC /tmp/build/perf/tests/python-use.o
>> > > [acme@five perf]$
>> > >
>> > > [acme@five perf]$ rpm -q python2-devel python3-devel python-devel
>> > > package python2-devel is not installed
>> > > python3-devel-3.7.6-2.fc31.x86_64
>> > > package python-devel is not installed
>> > > [acme@five perf]$
>> > >
>> > > [acme@five perf]$ cat
>/tmp/build/perf/feature/test-libpython.make.output
>> > > /bin/sh: --configdir: command not found
>> > > [acme@five perf]$ cat /tmp/build/perf/feature/test-libpython
>> > > test-libpython.make.output
>test-libpython-version.make.output
>> > > [acme@five perf]$ cat
>/tmp/build/perf/feature/test-libpython-version.make.output
>> > > /bin/sh: --configdir: command not found
>> > > [acme@five perf]$
>> > >
>> > >
>> > > Without your patch:
>> > >
>> > > [acme@five perf]$ rm -rf /tmp/build/perf ; mkdir -p
>/tmp/build/perf
>> > > [acme@five perf]$ make PYTHON=python3 -C tools/perf
>O=/tmp/build/perf install-bin |& grep python
>> > > ... libpython: [ on ]
>> > > GEN /tmp/build/perf/python/perf.so
>> > > MKDIR /tmp/build/perf/scripts/python/Perf-Trace-Util/
>> > > CC
>/tmp/build/perf/scripts/python/Perf-Trace-Util/Context.o
>> > > LD
>/tmp/build/perf/scripts/python/Perf-Trace-Util/perf-in.o
>> > > CC /tmp/build/perf/tests/python-use.o
>> > > CC
>/tmp/build/perf/util/scripting-engines/trace-event-python.o
>> > > INSTALL python-scripts
>> > > [acme@five perf]$
>> > >
>> > > [acme@five perf]$ ldd /tmp/build/perf/perf |& grep python
>> > > libpython3.7m.so.1.0 => /lib64/libpython3.7m.so.1.0
>(0x00007f11dd1ee000)
>> > > [acme@five perf]$ perf -vv |& grep -i python
>> > > libpython: [ on ] # HAVE_LIBPYTHON_SUPPORT
>> > > [acme@five perf]$
>> > >
>> > > What am I missing?
>> >
>> > It looks like you are using python3.7, but the change in behavior
>for
>> > python-config happened in version 3.8
>>
>> Humm, but shouldn't this continue to work with python3.7?
>
>Oh, my mistake, I didn't read the output carefully. It should
>obviously still work with old versions, yes. I actually submitted a
>similar patch, and it seemed to work when I used python 3.7. I wonder
>if the issue is the "||" operator in the subshell.
>
>https://lore.kernel.org/lkml/[email protected]/


I'm aware of your path, even for confused by your comment here, will it try it tomorrow

>
>
>> - Arnaldo
>>
>> > > [acme@five perf]$ cat /etc/redhat-release
>> > > Fedora release 31 (Thirty One)
>> > > [acme@five perf]$
>> > >
>> > > - Arnaldo
>>
>> --
>>
>> - Arnaldo

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

2020-03-26 10:34:33

by He Zhe

[permalink] [raw]
Subject: Re: [PATCH 1/2] perf: Be compatible with all python versions when fetching ldflags



On 3/26/20 9:06 AM, Arnaldo Melo wrote:
>
> On March 25, 2020 5:31:15 PM GMT-03:00, Sam Lunt <[email protected]> wrote:
>> On Wed, Mar 25, 2020 at 2:26 PM Arnaldo Carvalho de Melo
>> <[email protected]> wrote:
>>> Em Wed, Mar 25, 2020 at 09:40:34AM -0500, Sam Lunt escreveu:
>>>> On Wed, Mar 25, 2020 at 8:30 AM Arnaldo Carvalho de Melo
>>>> <[email protected]> wrote:
>>>>> Em Mon, Feb 17, 2020 at 10:24:27AM +0800, He Zhe escreveu:
>>>>>>
>>>>>> On 2/17/20 6:22 AM, Jiri Olsa wrote:
>>>>>>> On Fri, Feb 14, 2020 at 02:21:05AM +0800,
>> [email protected] wrote:
>>>>>>>> From: He Zhe <[email protected]>
>>>>>>>>
>>>>>>>> Since Python v3.8.0, with the following commit
>>>>>>>> 0a8e57248b91 ("bpo-36721: Add --embed option to
>> python-config (GH-13500)"),
>>>>>>> we got similar change recently.. might have not been picked
>> up yet
>>>>>>>
>> https://lore.kernel.org/lkml/[email protected]/
>>>>>> Thanks for pointing out.
>>>>> So, just with your patch:
>>>>>
>>>>> [acme@five perf]$ rm -rf /tmp/build/perf ; mkdir -p
>> /tmp/build/perf
>>>>> [acme@five perf]$ make PYTHON=python3 -C tools/perf
>> O=/tmp/build/perf install-bin |& grep python
>>>>> ... libpython: [ OFF ]
>>>>> Makefile.config:750: No 'Python.h' (for Python 2.x support) was
>> found: disables Python support - please install python-devel/python-dev
>>>>> CC /tmp/build/perf/tests/python-use.o
>>>>> [acme@five perf]$
>>>>>
>>>>> [acme@five perf]$ rpm -q python2-devel python3-devel python-devel
>>>>> package python2-devel is not installed
>>>>> python3-devel-3.7.6-2.fc31.x86_64
>>>>> package python-devel is not installed
>>>>> [acme@five perf]$
>>>>>
>>>>> [acme@five perf]$ cat
>> /tmp/build/perf/feature/test-libpython.make.output
>>>>> /bin/sh: --configdir: command not found
>>>>> [acme@five perf]$ cat /tmp/build/perf/feature/test-libpython
>>>>> test-libpython.make.output
>> test-libpython-version.make.output
>>>>> [acme@five perf]$ cat
>> /tmp/build/perf/feature/test-libpython-version.make.output
>>>>> /bin/sh: --configdir: command not found
>>>>> [acme@five perf]$
>>>>>
>>>>>
>>>>> Without your patch:
>>>>>
>>>>> [acme@five perf]$ rm -rf /tmp/build/perf ; mkdir -p
>> /tmp/build/perf
>>>>> [acme@five perf]$ make PYTHON=python3 -C tools/perf
>> O=/tmp/build/perf install-bin |& grep python
>>>>> ... libpython: [ on ]
>>>>> GEN /tmp/build/perf/python/perf.so
>>>>> MKDIR /tmp/build/perf/scripts/python/Perf-Trace-Util/
>>>>> CC
>> /tmp/build/perf/scripts/python/Perf-Trace-Util/Context.o
>>>>> LD
>> /tmp/build/perf/scripts/python/Perf-Trace-Util/perf-in.o
>>>>> CC /tmp/build/perf/tests/python-use.o
>>>>> CC
>> /tmp/build/perf/util/scripting-engines/trace-event-python.o
>>>>> INSTALL python-scripts
>>>>> [acme@five perf]$
>>>>>
>>>>> [acme@five perf]$ ldd /tmp/build/perf/perf |& grep python
>>>>> libpython3.7m.so.1.0 => /lib64/libpython3.7m.so.1.0
>> (0x00007f11dd1ee000)
>>>>> [acme@five perf]$ perf -vv |& grep -i python
>>>>> libpython: [ on ] # HAVE_LIBPYTHON_SUPPORT
>>>>> [acme@five perf]$
>>>>>
>>>>> What am I missing?
>>>> It looks like you are using python3.7, but the change in behavior
>> for
>>>> python-config happened in version 3.8
>>> Humm, but shouldn't this continue to work with python3.7?
>> Oh, my mistake, I didn't read the output carefully. It should
>> obviously still work with old versions, yes. I actually submitted a
>> similar patch, and it seemed to work when I used python 3.7. I wonder
>> if the issue is the "||" operator in the subshell.
>>
>> https://lore.kernel.org/lkml/[email protected]/
>
> I'm aware of your path, even for confused by your comment here, will it try it tomorrow

Sorry for inconvenience.

Yes, it is due to that the command before "||" prints some usage on failure and
thus messes up the whole string.

And I've tested Sam's patch. It works.

BTW, my [2/2] may still make sense.


Thanks,
Zhe

>
>>
>>> - Arnaldo
>>>
>>>>> [acme@five perf]$ cat /etc/redhat-release
>>>>> Fedora release 31 (Thirty One)
>>>>> [acme@five perf]$
>>>>>
>>>>> - Arnaldo
>>> --
>>>
>>> - Arnaldo

Subject: [tip: perf/urgent] perf: Normalize gcc parameter when generating arch errno table

The following commit has been merged into the perf/urgent branch of tip:

Commit-ID: e4ffd066ff440a57097e9140fa9e16ceef905de8
Gitweb: https://git.kernel.org/tip/e4ffd066ff440a57097e9140fa9e16ceef905de8
Author: He Zhe <[email protected]>
AuthorDate: Fri, 14 Feb 2020 02:21:06 +08:00
Committer: Arnaldo Carvalho de Melo <[email protected]>
CommitterDate: Thu, 26 Mar 2020 11:04:01 -03:00

perf: Normalize gcc parameter when generating arch errno table

The $(CC) passed to arch_errno_names.sh may include a series of parameters
along with gcc itself. To avoid overwriting the following parameters of
arch_errno_names.sh and break the build like below, we just pick up the
first word of the $(CC).

find: unknown predicate `-m64/arch'
x86_64-wrs-linux-gcc: warning: '-x c' after last input file has no effect
x86_64-wrs-linux-gcc: error: unrecognized command line option '-m64/include/uapi/asm-generic/errno.h'
x86_64-wrs-linux-gcc: fatal error: no input files

Signed-off-by: He Zhe <[email protected]>
Cc: Alexander Shishkin <[email protected]>
Cc: Jiri Olsa <[email protected]>
Cc: Mark Rutland <[email protected]>
Cc: Namhyung Kim <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Link: http://lore.kernel.org/lkml/[email protected]
Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
---
tools/perf/Makefile.perf | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index a02aca9..d15a311 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -574,7 +574,7 @@ arch_errno_hdr_dir := $(srctree)/tools
arch_errno_tbl := $(srctree)/tools/perf/trace/beauty/arch_errno_names.sh

$(arch_errno_name_array): $(arch_errno_tbl)
- $(Q)$(SHELL) '$(arch_errno_tbl)' $(CC) $(arch_errno_hdr_dir) > $@
+ $(Q)$(SHELL) '$(arch_errno_tbl)' $(firstword $(CC)) $(arch_errno_hdr_dir) > $@

sync_file_range_arrays := $(beauty_outdir)/sync_file_range_arrays.c
sync_file_range_tbls := $(srctree)/tools/perf/trace/beauty/sync_file_range.sh

Subject: [tip: perf/urgent] perf tools: Support Python 3.8+ in Makefile

The following commit has been merged into the perf/urgent branch of tip:

Commit-ID: b9c9ce4e598e012ca7c1813fae2f4d02395807de
Gitweb: https://git.kernel.org/tip/b9c9ce4e598e012ca7c1813fae2f4d02395807de
Author: Sam Lunt <[email protected]>
AuthorDate: Fri, 31 Jan 2020 12:11:23 -06:00
Committer: Arnaldo Carvalho de Melo <[email protected]>
CommitterDate: Fri, 03 Apr 2020 10:03:44 -03:00

perf tools: Support Python 3.8+ in Makefile

Python 3.8 changed the output of 'python-config --ldflags' to no longer
include the '-lpythonX.Y' flag (this apparently fixed an issue loading
modules with a statically linked Python executable). The libpython
feature check in linux/build/feature fails if the Python library is not
included in FEATURE_CHECK_LDFLAGS-libpython variable.

This adds a check in the Makefile to determine if PYTHON_CONFIG accepts
the '--embed' flag and passes that flag alongside '--ldflags' if so.

tools/perf is the only place the libpython feature check is used.

Signed-off-by: Sam Lunt <[email protected]>
Tested-by: He Zhe <[email protected]>
Link: http://lore.kernel.org/lkml/[email protected]
Acked-by: Jiri Olsa <[email protected]>
Cc: Alexander Shishkin <[email protected]>
Cc: Mark Rutland <[email protected]>
Cc: Namhyung Kim <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: [email protected]
Cc: [email protected]
Link: http://lore.kernel.org/lkml/[email protected]
Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
---
tools/perf/Makefile.config | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config
index eb95c0c..12a8204 100644
--- a/tools/perf/Makefile.config
+++ b/tools/perf/Makefile.config
@@ -228,8 +228,17 @@ strip-libs = $(filter-out -l%,$(1))

PYTHON_CONFIG_SQ := $(call shell-sq,$(PYTHON_CONFIG))

+# Python 3.8 changed the output of `python-config --ldflags` to not include the
+# '-lpythonX.Y' flag unless '--embed' is also passed. The feature check for
+# libpython fails if that flag is not included in LDFLAGS
+ifeq ($(shell $(PYTHON_CONFIG_SQ) --ldflags --embed 2>&1 1>/dev/null; echo $$?), 0)
+ PYTHON_CONFIG_LDFLAGS := --ldflags --embed
+else
+ PYTHON_CONFIG_LDFLAGS := --ldflags
+endif
+
ifdef PYTHON_CONFIG
- PYTHON_EMBED_LDOPTS := $(shell $(PYTHON_CONFIG_SQ) --ldflags 2>/dev/null)
+ PYTHON_EMBED_LDOPTS := $(shell $(PYTHON_CONFIG_SQ) $(PYTHON_CONFIG_LDFLAGS) 2>/dev/null)
PYTHON_EMBED_LDFLAGS := $(call strip-libs,$(PYTHON_EMBED_LDOPTS))
PYTHON_EMBED_LIBADD := $(call grep-libs,$(PYTHON_EMBED_LDOPTS)) -lutil
PYTHON_EMBED_CCOPTS := $(shell $(PYTHON_CONFIG_SQ) --includes 2>/dev/null)