On 28/08/2023 22:13, Nícolas F. R. A. Prado wrote:
> Introduce a new kselftest to detect devices that were declared in the
> Devicetree, and are expected to be probed by a driver, but weren't.
>
> The test uses two lists: a list of compatibles that can match a
> Devicetree device to a driver, and a list of compatibles that should be
> ignored. The first is automatically generated by the
> dt-extract-compatibles script, and is run as part of building this test.
> The list of compatibles to ignore is a hand-crafted list to capture the
> few exceptions of compatibles that are expected to match a driver but
> not be bound to it.
>
> Signed-off-by: Nícolas F. R. A. Prado <[email protected]>
> Reviewed-by: Shuah Khan <[email protected]>
>
Hi Nicolas,
Currently when building kselftest against next-master and
mainline-master the below build error is observed. A bisect (full log
below) identified this patch as introducing the failure.
Full log from a failure:
https://storage.kernelci.org/mainline/master/v6.6-9152-gdeefd5024f07/arm64/defconfig%2Bkselftest/gcc-10/logs/kselftest.log
make[4]: Entering directory '/tmp/kci/linux/tools/testing/selftests/dt'
/tmp/kci/linux/tools/testing/selftests/../../../scripts/dtc/dt-extract-compatibles
-d /tmp/kci/linux/tools/testing/selftests/../../.. >
/tmp/kci/linux/build/kselftest/dt/compatible_list
Traceback (most recent call last):
File
"/tmp/kci/linux/tools/testing/selftests/../../../scripts/dtc/dt-extract-compatibles",
line 107, in <module>
compat_ignore_list.extend(parse_compatibles_to_ignore(f))
File
"/tmp/kci/linux/tools/testing/selftests/../../../scripts/dtc/dt-extract-compatibles",
line 67, in parse_compatibles_to_ignore
with open(file, 'r', encoding='utf-8') as f:
OSError: [Errno 40] Too many levels of symbolic links:
'/tmp/kci/linux/tools/testing/selftests/../../../build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/tools/testing/selftests/powerpc/vphn/vphn.c'
make[4]: *** [Makefile:12:
/tmp/kci/linux/build/kselftest/dt/compatible_list] Error 1
make[4]: Leaving directory '/tmp/kci/linux/tools/testing/selftests/dt'
The bisect log:
git bisect start
# good: [f9a7eda4d73d44dc1d17d05cdc9aeb9fc5660740] Merge tag
'hwmon-for-v6.7' of
git://git.kernel.org/pub/scm/linux/kernel/git/groeck/linux-staging
git bisect good f9a7eda4d73d44dc1d17d05cdc9aeb9fc5660740
# bad: [8bc9e6515183935fa0cccaf67455c439afe4982b] Merge tag
'devicetree-for-6.7' of
git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux
git bisect bad 8bc9e6515183935fa0cccaf67455c439afe4982b
# good: [0a6d7f8275f255eda823c0f0b61d024f6f5b483d] Merge branch
'clk-cleanup' into clk-next
git bisect good 0a6d7f8275f255eda823c0f0b61d024f6f5b483d
# good: [fe4ae2fab00b4751265580c5865fdf23b62d80b3] Merge tag
'clk-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/clk/linux
git bisect good fe4ae2fab00b4751265580c5865fdf23b62d80b3
# good: [fc7b34ae1347f4eb36f065458e53d6065cd85928] power: supply:
wm831x_backup: Convert to platform remove callback returning void
git bisect good fc7b34ae1347f4eb36f065458e53d6065cd85928
# bad: [f2147371a83c6de1128093c163dc17bc61096362] dt-bindings: soc: fsl:
cpm_qe: cpm1-scc-qmc: Fix example property name
git bisect bad f2147371a83c6de1128093c163dc17bc61096362
# bad: [22c3888e55bf5c86be536a3d9e06a50e7bf3a39f] dt-bindings: watchdog:
atmel,at91rm9200-wdt: convert txt to yaml
git bisect bad 22c3888e55bf5c86be536a3d9e06a50e7bf3a39f
# bad: [780967feb626c6f4efa1e4b3532f1be83884cd76] dt-bindings: Add
Marantec vendor prefix
git bisect bad 780967feb626c6f4efa1e4b3532f1be83884cd76
# bad: [bc17fd92c1eb7589f1f3df1893e9f62bb35b8cc8] dt-bindings:
interrupt-controller: qcom,pdc: document qcom,sm4450-pdc
git bisect bad bc17fd92c1eb7589f1f3df1893e9f62bb35b8cc8
# good: [365ba0c7a73cce407bf40cdf9900b86b945d4acb] dt:
dt-extract-compatibles: Add flag for driver matching compatibles
git bisect good 365ba0c7a73cce407bf40cdf9900b86b945d4acb
# bad: [14571ab1ad213de59b3726a40aea7ca0365bf445] kselftest: Add new
test for detecting unprobed Devicetree devices
git bisect bad 14571ab1ad213de59b3726a40aea7ca0365bf445
# first bad commit: [14571ab1ad213de59b3726a40aea7ca0365bf445]
kselftest: Add new test for detecting unprobed Devicetree devices
Thanks,
Aishwarya
Hi Aishwarya,
On Thu, 2 Nov 2023 at 17:41, Aishwarya TCV <[email protected]> wrote:
>
>
>
> On 28/08/2023 22:13, Nícolas F. R. A. Prado wrote:
> > Introduce a new kselftest to detect devices that were declared in the
> > Devicetree, and are expected to be probed by a driver, but weren't.
> >
> > The test uses two lists: a list of compatibles that can match a
> > Devicetree device to a driver, and a list of compatibles that should be
> > ignored. The first is automatically generated by the
> > dt-extract-compatibles script, and is run as part of building this test.
> > The list of compatibles to ignore is a hand-crafted list to capture the
> > few exceptions of compatibles that are expected to match a driver but
> > not be bound to it.
> >
> > Signed-off-by: Nícolas F. R. A. Prado <[email protected]>
> > Reviewed-by: Shuah Khan <[email protected]>
> >
>
> Hi Nicolas,
>
> Currently when building kselftest against next-master and
> mainline-master the below build error is observed. A bisect (full log
> below) identified this patch as introducing the failure.
>
> Full log from a failure:
>
> https://storage.kernelci.org/mainline/master/v6.6-9152-gdeefd5024f07/arm64/defconfig%2Bkselftest/gcc-10/logs/kselftest.log
>
> make[4]: Entering directory '/tmp/kci/linux/tools/testing/selftests/dt'
> /tmp/kci/linux/tools/testing/selftests/../../../scripts/dtc/dt-extract-compatibles
> -d /tmp/kci/linux/tools/testing/selftests/../../.. >
> /tmp/kci/linux/build/kselftest/dt/compatible_list
> Traceback (most recent call last):
> File
> "/tmp/kci/linux/tools/testing/selftests/../../../scripts/dtc/dt-extract-compatibles",
> line 107, in <module>
> compat_ignore_list.extend(parse_compatibles_to_ignore(f))
> File
> "/tmp/kci/linux/tools/testing/selftests/../../../scripts/dtc/dt-extract-compatibles",
> line 67, in parse_compatibles_to_ignore
> with open(file, 'r', encoding='utf-8') as f:
> OSError: [Errno 40] Too many levels of symbolic links:
OSError: [Errno 40] Too many levels of symbolic links:
This is not related to selftests/dt tests build.
May be due to, A loop of symlinks that are pointing to self / same files ?
> '/tmp/kci/linux/tools/testing/selftests/../../../build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/tools/testing/selftests/powerpc/vphn/vphn.c'
> make[4]: *** [Makefile:12:
> /tmp/kci/linux/build/kselftest/dt/compatible_list] Error 1
> make[4]: Leaving directory '/tmp/kci/linux/tools/testing/selftests/dt'
Here is the log showing selftests/dt build pass for arm64 build from
Linux next master branch.
Links to the successful build and kselftest.tar file shared below [1].
Build log:
========
make[4]: Entering directory 'tools/testing/selftests/dt'
tools/testing/selftests/../../../scripts/dtc/dt-extract-compatibles -d
tools/testing/selftests/../../.. >
/home/tuxbuild/.cache/tuxmake/builds/1/build/kselftest/dt/compatible_list
make[4]: Leaving directory 'tools/testing/selftests/dt
Please build by using tuxmake and validate builds are working.
steps to reproduce:
======
#!/bin/sh
# TuxMake is a command line tool and Python library that provides
# portable and repeatable Linux kernel builds across a variety of
# architectures, toolchains, kernel configurations, and make targets.
#
# TuxMake supports the concept of runtimes.
# See https://docs.tuxmake.org/runtimes/, for that to work it requires
# that you install podman or docker on your system.
#
# To install tuxmake to your home directory at ~/.local/bin:
# pip3 install -U --user tuxmake
#
# Or install a deb/rpm depending on the running distribution
# See https://tuxmake.org/install-deb/ or
# https://tuxmake.org/install-rpm/
#
# See https://docs.tuxmake.org/ for complete documentation.
# Original tuxmake command with fragments listed below.
# tuxmake --runtime podman --target-arch arm64 --toolchain gcc-13
--kconfig defconfig --kconfig-add
https://raw.githubusercontent.com/Linaro/meta-lkft/kirkstone/meta/recipes-kernel/linux/files/systemd.config
--kconfig-add CONFIG_BCMGENET=y --kconfig-add
tools/testing/selftests/cgroup/config --kconfig-add
tools/testing/selftests/cpufreq/config --kconfig-add
tools/testing/selftests/efivarfs/config --kconfig-add
tools/testing/selftests/filesystems/binderfs/config --kconfig-add
tools/testing/selftests/filesystems/fat/config --kconfig-add
tools/testing/selftests/firmware/config --kconfig-add
tools/testing/selftests/ftrace/config --kconfig-add
tools/testing/selftests/gpio/config --kconfig-add
tools/testing/selftests/ipc/config --kconfig-add
tools/testing/selftests/memfd/config dtbs dtbs-legacy headers kernel
kselftest modules
tuxmake --runtime podman --target-arch arm64 --toolchain gcc-13
--kconfig https://storage.tuxsuite.com/public/linaro/lkft/builds/2XYjd2yxHiay3gVALCGpAch4G8o/config
dtbs dtbs-legacy headers kernel kselftest modules
Build links:
[1] https://storage.tuxsuite.com/public/linaro/lkft/builds/2XYjd2yxHiay3gVALCGpAch4G8o/
- Naresh
On Thu, Nov 02, 2023 at 07:15:58PM +0530, Naresh Kamboju wrote:
> On Thu, 2 Nov 2023 at 17:41, Aishwarya TCV <[email protected]> wrote:
> > https://storage.kernelci.org/mainline/master/v6.6-9152-gdeefd5024f07/arm64/defconfig%2Bkselftest/gcc-10/logs/kselftest.log
...
> May be due to, A loop of symlinks that are pointing to self / same files ?
Right, it does look like something bad is going on with symlinks:
> > '/tmp/kci/linux/tools/testing/selftests/../../../build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/tools/testing/selftests/powerpc/vphn/vphn.c'
> Please build by using tuxmake and validate builds are working.
Note that tuxmake does an in tree build of kselftest:
make --silent --keep-going --jobs=8 O=/home/tuxbuild/.cache/tuxmake/builds/1/build INSTALL_PATH=/home/tuxbuild/.cache/tuxmake/builds/1/build/kselftest_install ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- CROSS_COMPILE_COMPAT=arm-linux-gnueabihf- 'CC=sccache aarch64-linux-gnu-gcc' 'HOSTCC=sccache gcc' kselftest-install
and does it's own tarball build too, whereas kernelci does an out of
tree build and uses kselftest-gen_tar:
make KBUILD_BUILD_USER=KernelCI FORMAT=.xz ARCH=arm64 HOSTCC=gcc CROSS_COMPILE=aarch64-linux-gnu- CROSS_COMPILE_COMPAT=arm-linux-gnueabihf- CC="ccache aarch64-linux-gnu-gcc" O=/tmp/kci/linux/build -C/tmp/kci/linux -j10 kselftest-gen_tar
and that the error is in the dt-extract-compatibles program which is
part of the kernel (well, imported into the kernel from dtc upstream):
File "/tmp/kci/linux/tools/testing/selftests/../../../scripts/dtc/dt-extract-compatibles", line 107, in <module>
compat_ignore_list.extend(parse_compatibles_to_ignore(f))
This all suggests that something to do with how the build is set up is
resulting in the source symlink that gets created for out of tree builds
blowing up, I guess it's not specifically the DT stuff that's blowing it
up but rather that it's tripping over an existing bug. Really does look
like a legitimate bug though, the source link is set up by the in tree
kernel build infrastructure.
I did poke a bit at reproducing outside of the KernelCI scripts but
didn't manage to yet.
On Thu, Nov 2, 2023 at 12:36 PM Mark Brown <[email protected]> wrote:
>
> On Thu, Nov 02, 2023 at 07:15:58PM +0530, Naresh Kamboju wrote:
> > On Thu, 2 Nov 2023 at 17:41, Aishwarya TCV <[email protected]> wrote:
>
> > > https://storage.kernelci.org/mainline/master/v6.6-9152-gdeefd5024f07/arm64/defconfig%2Bkselftest/gcc-10/logs/kselftest.log
>
> ...
>
> > May be due to, A loop of symlinks that are pointing to self / same files ?
>
> Right, it does look like something bad is going on with symlinks:
>
> > > '/tmp/kci/linux/tools/testing/selftests/../../../build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/build/source/tools/testing/selftests/powerpc/vphn/vphn.c'
>
> > Please build by using tuxmake and validate builds are working.
>
> Note that tuxmake does an in tree build of kselftest:
>
> make --silent --keep-going --jobs=8 O=/home/tuxbuild/.cache/tuxmake/builds/1/build INSTALL_PATH=/home/tuxbuild/.cache/tuxmake/builds/1/build/kselftest_install ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- CROSS_COMPILE_COMPAT=arm-linux-gnueabihf- 'CC=sccache aarch64-linux-gnu-gcc' 'HOSTCC=sccache gcc' kselftest-install
>
> and does it's own tarball build too, whereas kernelci does an out of
> tree build and uses kselftest-gen_tar:
>
> make KBUILD_BUILD_USER=KernelCI FORMAT=.xz ARCH=arm64 HOSTCC=gcc CROSS_COMPILE=aarch64-linux-gnu- CROSS_COMPILE_COMPAT=arm-linux-gnueabihf- CC="ccache aarch64-linux-gnu-gcc" O=/tmp/kci/linux/build -C/tmp/kci/linux -j10 kselftest-gen_tar
>
> and that the error is in the dt-extract-compatibles program which is
> part of the kernel (well, imported into the kernel from dtc upstream):
>
> File "/tmp/kci/linux/tools/testing/selftests/../../../scripts/dtc/dt-extract-compatibles", line 107, in <module>
> compat_ignore_list.extend(parse_compatibles_to_ignore(f))
>
> This all suggests that something to do with how the build is set up is
> resulting in the source symlink that gets created for out of tree builds
> blowing up, I guess it's not specifically the DT stuff that's blowing it
> up but rather that it's tripping over an existing bug. Really does look
> like a legitimate bug though, the source link is set up by the in tree
> kernel build infrastructure.
>
> I did poke a bit at reproducing outside of the KernelCI scripts but
> didn't manage to yet.
I can repro with "make dt_compatible_check". The problem is with an
'out of tree' build within the tree. That's my normal setup, but the
difference is I have ".build" directories. If I use "build" instead,
then I can repro. The issue is the iglob will recurse into "build" but
not hidden directories (by default). There's no option to not follow
symlinks which would solve this (there is an open python issue since
2017 to add it). I don't see a simple solution in python other than
getting a full list with glob(), convert to absolute paths, and remove
duplicates. I imagine that will be somewhat slow.
A simple solution would be instead of passing the source tree root to
dt-extract-compatibles, pass 'arch', 'drivers', and 'sound' instead.
There shouldn't be compatibles anywhere else.
Rob