Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   =?UTF-8?q?N=C3=ADcolas=20F=2E=20R=2E=20A=2E=20Prado?= 
        <nfraprado@collabora.com>
To:     Rob Herring <robh+dt@kernel.org>,
        Frank Rowand <frowand.list@gmail.com>,
        Shuah Khan <shuah@kernel.org>
Cc:     cocci@inria.fr, Mark Brown <broonie@kernel.org>,
        Nicolas Palix <nicolas.palix@imag.fr>,
        kernelci@lists.linux.dev, Julia Lawall <Julia.Lawall@inria.fr>,
        Bjorn Andersson <andersson@kernel.org>, kernel@collabora.com,
        Guenter Roeck <groeck@chromium.org>,
        =?UTF-8?q?N=C3=ADcolas=20F=2E=20R=2E=20A=2E=20Prado?= 
        <nfraprado@collabora.com>, devicetree@vger.kernel.org,
        linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org
Subject: [RFC PATCH 0/2] Add a test to catch unprobed Devicetree devices
Date:   Thu, 10 Aug 2023 16:23:49 -0400
Message-ID: <20230810202413.1780286-1-nfraprado@collabora.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Precedence: bulk


Regressions that cause a device to no longer be probed by a driver can
have a big impact on the platform's functionality, and despite being
relatively common there isn't currently any generic test to detect them.
As an example, bootrr [1] does test for device probe, but it requires
defining the expected probed devices for each platform.

Given that the Devicetree already provides a static description of
devices on the system, it is a good basis for building such a test on
top.

This series introduces a test to catch regressions that prevent devices
from probing.

Patch 1 introduces a script to parse the kernel source using Coccinelle
and extract all compatibles that can be matched by a Devicetree node to
a driver. Patch 2 adds a kselftest that walks over the Devicetree nodes
on the current platform and compares the compatibles to the ones on the
list, and on an ignore list, to point out devices that failed to be
probed.

A compatible list is needed because not all compatibles that can show up
in a Devicetree node can be used to match to a driver, for example the
code for that compatible might use "OF_DECLARE" type macros and avoid
the driver framework, or the node might be controlled by a driver that
was bound to a different node.

An ignore list is needed for the few cases where it's common for a
driver to match a device but not probe, like for the "simple-mfd"
compatible, where the driver only probes if that compatible is the
node's first compatible.

Even though there's already scripts/dtc/dt-extract-compatibles that does
a similar job, it didn't seem to find all compatibles, returning ~3k,
while Coccinelle found ~11k. Besides that, Coccinelle actually parses
the C files, so it should be a more robust solution than relying on
regexes.

The reason for parsing the kernel source instead of relying on
information exposed by the kernel at runtime (say, looking at modaliases
or introducing some other mechanism), is to be able to catch issues
where a config was renamed or a driver moved across configs, and the
.config used by the kernel not updated accordingly. We need to parse the
source to find all compatibles present in the kernel independent of the
current config being run.

Feedback is very much welcome.

Thanks,
Nícolas

[1] https://github.com/kernelci/bootrr


Nícolas F. R. A. Prado (2):
  scripts/dtc: Add script to extract matchable DT compatibles
  kselftest: Add Devicetree unprobed devices test

 scripts/dtc/extract-matchable-dt-compatibles  | 33 +++++++++++
 scripts/dtc/matchable_dt_compatibles.cocci    | 58 +++++++++++++++++++
 tools/testing/selftests/Makefile              |  1 +
 tools/testing/selftests/dt/.gitignore         |  1 +
 tools/testing/selftests/dt/Makefile           | 17 ++++++
 .../selftests/dt/compatible_ignore_list       |  3 +
 .../selftests/dt/test_unprobed_devices.sh     | 58 +++++++++++++++++++
 7 files changed, 171 insertions(+)
 create mode 100755 scripts/dtc/extract-matchable-dt-compatibles
 create mode 100644 scripts/dtc/matchable_dt_compatibles.cocci
 create mode 100644 tools/testing/selftests/dt/.gitignore
 create mode 100644 tools/testing/selftests/dt/Makefile
 create mode 100644 tools/testing/selftests/dt/compatible_ignore_list
 create mode 100755 tools/testing/selftests/dt/test_unprobed_devices.sh

-- 
2.41.0