As mentioned here [1], I was going to try to attempt to improve Python
autodetection in the scenario where Python3 devtools are installed
alongside the Python2 runtime which is a pretty common scenario on a base
system after running "apt install python3-dev".
Thanks
James
[1]: https://lkml.org/lkml/2021/11/30/614
James Clark (1):
perf tools: Use Python devtools for version autodetection rather than
runtime
tools/perf/Makefile.config | 39 ++++++++++++++++++++++++++------------
1 file changed, 27 insertions(+), 12 deletions(-)
--
2.28.0
This fixes the issue where the build will fail if only the Python2
runtime is installed but the Python3 devtools are installed. Currently
the workaround is 'make PYTHON=python3'.
Fix it by autodetecting Python based on whether python[x]-config exists
rather than just python[x] because both are needed for the build. Then
-config is stripped to find the Python runtime.
Testing
=======
* Auto detect links with Python3 when the v3 devtools are installed
and only Python 2 runtime is installed
* Auto detect links with Python2 when both devtools are installed
* Sensible warning is printed if no Python devtools are installed
* 'make PYTHON=x' still automatically sets PYTHON_CONFIG=x-config
* 'make PYTHON=x' fails if x-config doesn't exist
* 'make PYTHON=python3' overrides Python2 devtools
* 'make PYTHON=python2' overrides Python3 devtools
* 'make PYTHON_CONFIG=x-config' works
* 'make PYTHON=x PYTHON_CONFIG=x' works
* 'make PYTHON=missing' reports an error
* 'make PYTHON_CONFIG=missing' reports an error
Fixes: 79373082fa9d ("perf python: Autodetect python3 binary")
Signed-off-by: James Clark <[email protected]>
---
tools/perf/Makefile.config | 39 ++++++++++++++++++++++++++------------
1 file changed, 27 insertions(+), 12 deletions(-)
diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config
index 96ad944ca6a8..b3fbb746e7f0 100644
--- a/tools/perf/Makefile.config
+++ b/tools/perf/Makefile.config
@@ -239,18 +239,33 @@ ifdef PARSER_DEBUG
endif
# Try different combinations to accommodate systems that only have
-# python[2][-config] in weird combinations but always preferring
-# python2 and python2-config as per pep-0394. If python2 or python
-# aren't found, then python3 is used.
-PYTHON_AUTO := python
-PYTHON_AUTO := $(if $(call get-executable,python3),python3,$(PYTHON_AUTO))
-PYTHON_AUTO := $(if $(call get-executable,python),python,$(PYTHON_AUTO))
-PYTHON_AUTO := $(if $(call get-executable,python2),python2,$(PYTHON_AUTO))
-override PYTHON := $(call get-executable-or-default,PYTHON,$(PYTHON_AUTO))
-PYTHON_AUTO_CONFIG := \
- $(if $(call get-executable,$(PYTHON)-config),$(PYTHON)-config,python-config)
-override PYTHON_CONFIG := \
- $(call get-executable-or-default,PYTHON_CONFIG,$(PYTHON_AUTO_CONFIG))
+# python[2][3]-config in weird combinations in the following order of
+# priority from lowest to highest:
+# * python3-config
+# * python-config
+# * python2-config as per pep-0394.
+# * $(PYTHON)-config (If PYTHON is user supplied but PYTHON_CONFIG isn't)
+#
+PYTHON_AUTO := python-config
+PYTHON_AUTO := $(if $(call get-executable,python3-config),python3-config,$(PYTHON_AUTO))
+PYTHON_AUTO := $(if $(call get-executable,python-config),python-config,$(PYTHON_AUTO))
+PYTHON_AUTO := $(if $(call get-executable,python2-config),python2-config,$(PYTHON_AUTO))
+
+# If PYTHON is defined but PYTHON_CONFIG isn't, then take $(PYTHON)-config as if it was the user
+# supplied value for PYTHON_CONFIG. Because it's "user supplied", error out if it doesn't exist.
+ifdef PYTHON
+ ifndef PYTHON_CONFIG
+ PYTHON_CONFIG_AUTO := $(call get-executable,$(PYTHON)-config)
+ PYTHON_CONFIG := $(if $(PYTHON_CONFIG_AUTO),$(PYTHON_CONFIG_AUTO),\
+ $(call $(error $(PYTHON)-config not found)))
+ endif
+endif
+
+# Select either auto detected python and python-config or use user supplied values if they are
+# defined. get-executable-or-default fails with an error if the first argument is supplied but
+# doesn't exist.
+override PYTHON_CONFIG := $(call get-executable-or-default,PYTHON_CONFIG,$(PYTHON_AUTO))
+override PYTHON := $(call get-executable-or-default,PYTHON,$(subst -config,,$(PYTHON_AUTO)))
grep-libs = $(filter -l%,$(1))
strip-libs = $(filter-out -l%,$(1))
--
2.28.0
Hi James,
On 09/03/2022 19:43, James Clark wrote:
> This fixes the issue where the build will fail if only the Python2
> runtime is installed but the Python3 devtools are installed. Currently
> the workaround is 'make PYTHON=python3'.
>
> Fix it by autodetecting Python based on whether python[x]-config exists
> rather than just python[x] because both are needed for the build. Then
> -config is stripped to find the Python runtime.
>
> Testing
> =======
>
> * Auto detect links with Python3 when the v3 devtools are installed
> and only Python 2 runtime is installed
> * Auto detect links with Python2 when both devtools are installed
> * Sensible warning is printed if no Python devtools are installed
> * 'make PYTHON=x' still automatically sets PYTHON_CONFIG=x-config
> * 'make PYTHON=x' fails if x-config doesn't exist
If x is a valid python but no x-config is found in the system, the build
fails, instead of printing a warning like before. If we use this approach
I think [1] in the Makefile is never hit and needs to be cleaned up.
Thanks,
German
[1] https://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git/tree/tools/perf/Makefile.config?h=perf/core#n826:
> * 'make PYTHON=python3' overrides Python2 devtools
> * 'make PYTHON=python2' overrides Python3 devtools
> * 'make PYTHON_CONFIG=x-config' works
> * 'make PYTHON=x PYTHON_CONFIG=x' works
> * 'make PYTHON=missing' reports an error
> * 'make PYTHON_CONFIG=missing' reports an error
>
> Fixes: 79373082fa9d ("perf python: Autodetect python3 binary")
> Signed-off-by: James Clark <[email protected]>
> ---
> tools/perf/Makefile.config | 39 ++++++++++++++++++++++++++------------
> 1 file changed, 27 insertions(+), 12 deletions(-)
>
> diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config
> index 96ad944ca6a8..b3fbb746e7f0 100644
> --- a/tools/perf/Makefile.config
> +++ b/tools/perf/Makefile.config
> @@ -239,18 +239,33 @@ ifdef PARSER_DEBUG
> endif
>
> # Try different combinations to accommodate systems that only have
> -# python[2][-config] in weird combinations but always preferring
> -# python2 and python2-config as per pep-0394. If python2 or python
> -# aren't found, then python3 is used.
> -PYTHON_AUTO := python
> -PYTHON_AUTO := $(if $(call get-executable,python3),python3,$(PYTHON_AUTO))
> -PYTHON_AUTO := $(if $(call get-executable,python),python,$(PYTHON_AUTO))
> -PYTHON_AUTO := $(if $(call get-executable,python2),python2,$(PYTHON_AUTO))
> -override PYTHON := $(call get-executable-or-default,PYTHON,$(PYTHON_AUTO))
> -PYTHON_AUTO_CONFIG := \
> - $(if $(call get-executable,$(PYTHON)-config),$(PYTHON)-config,python-config)
> -override PYTHON_CONFIG := \
> - $(call get-executable-or-default,PYTHON_CONFIG,$(PYTHON_AUTO_CONFIG))
> +# python[2][3]-config in weird combinations in the following order of
> +# priority from lowest to highest:
> +# * python3-config
> +# * python-config
> +# * python2-config as per pep-0394.
> +# * $(PYTHON)-config (If PYTHON is user supplied but PYTHON_CONFIG isn't)
> +#
> +PYTHON_AUTO := python-config
> +PYTHON_AUTO := $(if $(call get-executable,python3-config),python3-config,$(PYTHON_AUTO))
> +PYTHON_AUTO := $(if $(call get-executable,python-config),python-config,$(PYTHON_AUTO))
> +PYTHON_AUTO := $(if $(call get-executable,python2-config),python2-config,$(PYTHON_AUTO))
> +
> +# If PYTHON is defined but PYTHON_CONFIG isn't, then take $(PYTHON)-config as if it was the user
> +# supplied value for PYTHON_CONFIG. Because it's "user supplied", error out if it doesn't exist.
> +ifdef PYTHON
> + ifndef PYTHON_CONFIG
> + PYTHON_CONFIG_AUTO := $(call get-executable,$(PYTHON)-config)
> + PYTHON_CONFIG := $(if $(PYTHON_CONFIG_AUTO),$(PYTHON_CONFIG_AUTO),\
> + $(call $(error $(PYTHON)-config not found)))
> + endif
> +endif
> +
> +# Select either auto detected python and python-config or use user supplied values if they are
> +# defined. get-executable-or-default fails with an error if the first argument is supplied but
> +# doesn't exist.
> +override PYTHON_CONFIG := $(call get-executable-or-default,PYTHON_CONFIG,$(PYTHON_AUTO))
> +override PYTHON := $(call get-executable-or-default,PYTHON,$(subst -config,,$(PYTHON_AUTO)))
>
> grep-libs = $(filter -l%,$(1))
> strip-libs = $(filter-out -l%,$(1))
On 12/04/2022 16:30, German Gomez wrote:
> Hi James,
>
> On 09/03/2022 19:43, James Clark wrote:
>> This fixes the issue where the build will fail if only the Python2
>> runtime is installed but the Python3 devtools are installed. Currently
>> the workaround is 'make PYTHON=python3'.
>>
>> Fix it by autodetecting Python based on whether python[x]-config exists
>> rather than just python[x] because both are needed for the build. Then
>> -config is stripped to find the Python runtime.
>>
>> Testing
>> =======
>>
>> * Auto detect links with Python3 when the v3 devtools are installed
>> and only Python 2 runtime is installed
>> * Auto detect links with Python2 when both devtools are installed
>> * Sensible warning is printed if no Python devtools are installed
>> * 'make PYTHON=x' still automatically sets PYTHON_CONFIG=x-config
>> * 'make PYTHON=x' fails if x-config doesn't exist
> If x is a valid python but no x-config is found in the system, the build
> fails, instead of printing a warning like before. If we use this approach
> I think [1] in the Makefile is never hit and needs to be cleaned up.
Sorry, small correction:
The line in the Makefile is never hit when PYTHON is explicitly defined (probably not a big deal).
Still I wanted to point the small change in behaviour with the build warning vs. build fail.