This series switches the kernel's DT schema validation from YAML encoded
DT files to using DTB files directly. See patch 2 for the full reasoning
of why. The diffstat also shows this is a nice simplification (at least
from the kernel side). Patch 1 is further reworking of how
DT_SCHEMA_FILES works and builds on [1].
Overall, the build time is about the same (still slow) though we do save
a dtc call for dtbs_check. Extracting the type information is not cached
in any way, so there's some opportunity for a slight optimization there.
Switching to DTB validation found various issues in the bindings and
examples. Patches for those issues have already been posted and applied
over the last month or so.
Rob
[1] https://lore.kernel.org/all/[email protected]/
Rob Herring (2):
dt-bindings: kbuild: Pass DT_SCHEMA_FILES to dt-validate
dt-bindings: kbuild: Use DTB files for validation
Documentation/devicetree/bindings/Makefile | 35 ++++---------------
.../devicetree/bindings/writing-schema.rst | 12 -------
scripts/Makefile.lib | 23 +++++-------
scripts/dtc/Makefile | 13 -------
scripts/dtc/update-dtc-source.sh | 2 +-
5 files changed, 15 insertions(+), 70 deletions(-)
--
2.32.0
In preparation for supporting validation of DTB files, the full
processed schema will always be needed in order to extract type
information from it. Therefore, the processed schema containing only
what DT_SCHEMA_FILES specifies won't work. Instead, dt-validate has
gained an option, -l or --limit, to specify which schema(s) to use for
validation.
As the command line option is new, we the minimum dtschema version must be
updated.
Cc: Masahiro Yamada <[email protected]>
Signed-off-by: Rob Herring <[email protected]>
---
Documentation/devicetree/bindings/Makefile | 28 +++-------------------
scripts/Makefile.lib | 3 +--
2 files changed, 4 insertions(+), 27 deletions(-)
diff --git a/Documentation/devicetree/bindings/Makefile b/Documentation/devicetree/bindings/Makefile
index 61ec18ecc931..246ba0ecab64 100644
--- a/Documentation/devicetree/bindings/Makefile
+++ b/Documentation/devicetree/bindings/Makefile
@@ -6,7 +6,7 @@ DT_MK_SCHEMA ?= dt-mk-schema
DT_SCHEMA_LINT := $(shell which yamllint || \
echo "warning: yamllint not installed, skipping. To install, run 'pip install yamllint'" >&2)
-DT_SCHEMA_MIN_VERSION = 2021.2.1
+DT_SCHEMA_MIN_VERSION = 2022.3
PHONY += check_dtschema_version
check_dtschema_version:
@@ -25,9 +25,6 @@ quiet_cmd_extract_ex = DTEX $@
$(obj)/%.example.dts: $(src)/%.yaml check_dtschema_version FORCE
$(call if_changed,extract_ex)
-# Use full schemas when checking %.example.dts
-DT_TMP_SCHEMA := $(obj)/processed-schema-examples.json
-
find_all_cmd = find $(srctree)/$(src) \( -name '*.yaml' ! \
-name 'processed-schema*' ! \
-name '*.example.dt.yaml' \)
@@ -70,29 +67,10 @@ override DTC_FLAGS := \
# Disable undocumented compatible checks until warning free
override DT_CHECKER_FLAGS ?=
-$(obj)/processed-schema-examples.json: $(DT_DOCS) $(src)/.yamllint check_dtschema_version FORCE
+$(obj)/processed-schema.json: $(DT_DOCS) $(src)/.yamllint check_dtschema_version FORCE
$(call if_changed_rule,chkdt)
-ifeq ($(DT_SCHEMA_FILES),)
-
-# Unless DT_SCHEMA_FILES is specified, use the full schema for dtbs_check too.
-# Just copy processed-schema-examples.json
-
-$(obj)/processed-schema.json: $(obj)/processed-schema-examples.json FORCE
- $(call if_changed,copy)
-
-else
-
-# If DT_SCHEMA_FILES is specified, use it for processed-schema.json
-
-$(obj)/processed-schema.json: DT_MK_SCHEMA_FLAGS := -u
-$(obj)/processed-schema.json: $(CHK_DT_DOCS) check_dtschema_version FORCE
- $(call if_changed,mk_schema)
-
-endif
-
-always-$(CHECK_DT_BINDING) += processed-schema-examples.json
-always-$(CHECK_DTBS) += processed-schema.json
+always-y += processed-schema.json
always-$(CHECK_DT_BINDING) += $(patsubst $(srctree)/$(src)/%.yaml,%.example.dts, $(CHK_DT_DOCS))
always-$(CHECK_DT_BINDING) += $(patsubst $(srctree)/$(src)/%.yaml,%.example.dt.yaml, $(CHK_DT_DOCS))
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 79be57fdd32a..9f1e8442564e 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -361,9 +361,8 @@ $(multi-dtb-y): FORCE
$(call multi_depend, $(multi-dtb-y), .dtb, -dtbs)
DT_CHECKER ?= dt-validate
-DT_CHECKER_FLAGS ?= $(if $(DT_SCHEMA_FILES),,-m)
+DT_CHECKER_FLAGS ?= $(if $(DT_SCHEMA_FILES),-l $(DT_SCHEMA_FILES),-m)
DT_BINDING_DIR := Documentation/devicetree/bindings
-# DT_TMP_SCHEMA may be overridden from Documentation/devicetree/bindings/Makefile
DT_TMP_SCHEMA ?= $(objtree)/$(DT_BINDING_DIR)/processed-schema.json
quiet_cmd_dtb_check = CHECK $@
--
2.32.0
On Thu, Mar 03, 2022 at 04:42:36PM -0600, Rob Herring wrote:
> In preparation for supporting validation of DTB files, the full
> processed schema will always be needed in order to extract type
> information from it. Therefore, the processed schema containing only
> what DT_SCHEMA_FILES specifies won't work. Instead, dt-validate has
> gained an option, -l or --limit, to specify which schema(s) to use for
> validation.
>
> As the command line option is new, we the minimum dtschema version must be
> updated.
>
> Cc: Masahiro Yamada <[email protected]>
> Signed-off-by: Rob Herring <[email protected]>
> ---
> Documentation/devicetree/bindings/Makefile | 28 +++-------------------
> scripts/Makefile.lib | 3 +--
> 2 files changed, 4 insertions(+), 27 deletions(-)
>
> diff --git a/Documentation/devicetree/bindings/Makefile b/Documentation/devicetree/bindings/Makefile
> index 61ec18ecc931..246ba0ecab64 100644
> --- a/Documentation/devicetree/bindings/Makefile
> +++ b/Documentation/devicetree/bindings/Makefile
> @@ -6,7 +6,7 @@ DT_MK_SCHEMA ?= dt-mk-schema
> DT_SCHEMA_LINT := $(shell which yamllint || \
> echo "warning: yamllint not installed, skipping. To install, run 'pip install yamllint'" >&2)
>
> -DT_SCHEMA_MIN_VERSION = 2021.2.1
> +DT_SCHEMA_MIN_VERSION = 2022.3
>
> PHONY += check_dtschema_version
> check_dtschema_version:
> @@ -25,9 +25,6 @@ quiet_cmd_extract_ex = DTEX $@
> $(obj)/%.example.dts: $(src)/%.yaml check_dtschema_version FORCE
> $(call if_changed,extract_ex)
>
> -# Use full schemas when checking %.example.dts
> -DT_TMP_SCHEMA := $(obj)/processed-schema-examples.json
> -
> find_all_cmd = find $(srctree)/$(src) \( -name '*.yaml' ! \
> -name 'processed-schema*' ! \
> -name '*.example.dt.yaml' \)
> @@ -70,29 +67,10 @@ override DTC_FLAGS := \
> # Disable undocumented compatible checks until warning free
> override DT_CHECKER_FLAGS ?=
>
> -$(obj)/processed-schema-examples.json: $(DT_DOCS) $(src)/.yamllint check_dtschema_version FORCE
> +$(obj)/processed-schema.json: $(DT_DOCS) $(src)/.yamllint check_dtschema_version FORCE
> $(call if_changed_rule,chkdt)
>
> -ifeq ($(DT_SCHEMA_FILES),)
> -
> -# Unless DT_SCHEMA_FILES is specified, use the full schema for dtbs_check too.
> -# Just copy processed-schema-examples.json
> -
> -$(obj)/processed-schema.json: $(obj)/processed-schema-examples.json FORCE
> - $(call if_changed,copy)
> -
> -else
> -
> -# If DT_SCHEMA_FILES is specified, use it for processed-schema.json
> -
> -$(obj)/processed-schema.json: DT_MK_SCHEMA_FLAGS := -u
> -$(obj)/processed-schema.json: $(CHK_DT_DOCS) check_dtschema_version FORCE
> - $(call if_changed,mk_schema)
> -
> -endif
> -
> -always-$(CHECK_DT_BINDING) += processed-schema-examples.json
> -always-$(CHECK_DTBS) += processed-schema.json
> +always-y += processed-schema.json
> always-$(CHECK_DT_BINDING) += $(patsubst $(srctree)/$(src)/%.yaml,%.example.dts, $(CHK_DT_DOCS))
> always-$(CHECK_DT_BINDING) += $(patsubst $(srctree)/$(src)/%.yaml,%.example.dt.yaml, $(CHK_DT_DOCS))
>
> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> index 79be57fdd32a..9f1e8442564e 100644
> --- a/scripts/Makefile.lib
> +++ b/scripts/Makefile.lib
> @@ -361,9 +361,8 @@ $(multi-dtb-y): FORCE
> $(call multi_depend, $(multi-dtb-y), .dtb, -dtbs)
>
> DT_CHECKER ?= dt-validate
> -DT_CHECKER_FLAGS ?= $(if $(DT_SCHEMA_FILES),,-m)
> +DT_CHECKER_FLAGS ?= $(if $(DT_SCHEMA_FILES),-l $(DT_SCHEMA_FILES),-m)
> DT_BINDING_DIR := Documentation/devicetree/bindings
> -# DT_TMP_SCHEMA may be overridden from Documentation/devicetree/bindings/Makefile
> DT_TMP_SCHEMA ?= $(objtree)/$(DT_BINDING_DIR)/processed-schema.json
This could now use := instead of ?=
Apart from the fact that 2022.3 hasn't been tagged yet as pointed out by
Geert, this looks fine to me (but I'm no expert in this area).
>
> quiet_cmd_dtb_check = CHECK $@
--
Regards,
Laurent Pinchart
On Fri, Mar 04, 2022 at 01:48:37PM +0200, Laurent Pinchart wrote:
> On Thu, Mar 03, 2022 at 04:42:36PM -0600, Rob Herring wrote:
> > In preparation for supporting validation of DTB files, the full
> > processed schema will always be needed in order to extract type
> > information from it. Therefore, the processed schema containing only
> > what DT_SCHEMA_FILES specifies won't work. Instead, dt-validate has
> > gained an option, -l or --limit, to specify which schema(s) to use for
> > validation.
> >
> > As the command line option is new, we the minimum dtschema version must be
> > updated.
> >
> > Cc: Masahiro Yamada <[email protected]>
> > Signed-off-by: Rob Herring <[email protected]>
> > ---
> > Documentation/devicetree/bindings/Makefile | 28 +++-------------------
> > scripts/Makefile.lib | 3 +--
> > 2 files changed, 4 insertions(+), 27 deletions(-)
> >
> > diff --git a/Documentation/devicetree/bindings/Makefile b/Documentation/devicetree/bindings/Makefile
> > index 61ec18ecc931..246ba0ecab64 100644
> > --- a/Documentation/devicetree/bindings/Makefile
> > +++ b/Documentation/devicetree/bindings/Makefile
> > @@ -6,7 +6,7 @@ DT_MK_SCHEMA ?= dt-mk-schema
> > DT_SCHEMA_LINT := $(shell which yamllint || \
> > echo "warning: yamllint not installed, skipping. To install, run 'pip install yamllint'" >&2)
> >
> > -DT_SCHEMA_MIN_VERSION = 2021.2.1
> > +DT_SCHEMA_MIN_VERSION = 2022.3
> >
> > PHONY += check_dtschema_version
> > check_dtschema_version:
> > @@ -25,9 +25,6 @@ quiet_cmd_extract_ex = DTEX $@
> > $(obj)/%.example.dts: $(src)/%.yaml check_dtschema_version FORCE
> > $(call if_changed,extract_ex)
> >
> > -# Use full schemas when checking %.example.dts
> > -DT_TMP_SCHEMA := $(obj)/processed-schema-examples.json
> > -
> > find_all_cmd = find $(srctree)/$(src) \( -name '*.yaml' ! \
> > -name 'processed-schema*' ! \
> > -name '*.example.dt.yaml' \)
> > @@ -70,29 +67,10 @@ override DTC_FLAGS := \
> > # Disable undocumented compatible checks until warning free
> > override DT_CHECKER_FLAGS ?=
> >
> > -$(obj)/processed-schema-examples.json: $(DT_DOCS) $(src)/.yamllint check_dtschema_version FORCE
> > +$(obj)/processed-schema.json: $(DT_DOCS) $(src)/.yamllint check_dtschema_version FORCE
> > $(call if_changed_rule,chkdt)
> >
> > -ifeq ($(DT_SCHEMA_FILES),)
> > -
> > -# Unless DT_SCHEMA_FILES is specified, use the full schema for dtbs_check too.
> > -# Just copy processed-schema-examples.json
> > -
> > -$(obj)/processed-schema.json: $(obj)/processed-schema-examples.json FORCE
> > - $(call if_changed,copy)
> > -
> > -else
> > -
> > -# If DT_SCHEMA_FILES is specified, use it for processed-schema.json
> > -
> > -$(obj)/processed-schema.json: DT_MK_SCHEMA_FLAGS := -u
> > -$(obj)/processed-schema.json: $(CHK_DT_DOCS) check_dtschema_version FORCE
> > - $(call if_changed,mk_schema)
> > -
> > -endif
> > -
> > -always-$(CHECK_DT_BINDING) += processed-schema-examples.json
> > -always-$(CHECK_DTBS) += processed-schema.json
> > +always-y += processed-schema.json
> > always-$(CHECK_DT_BINDING) += $(patsubst $(srctree)/$(src)/%.yaml,%.example.dts, $(CHK_DT_DOCS))
> > always-$(CHECK_DT_BINDING) += $(patsubst $(srctree)/$(src)/%.yaml,%.example.dt.yaml, $(CHK_DT_DOCS))
> >
> > diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> > index 79be57fdd32a..9f1e8442564e 100644
> > --- a/scripts/Makefile.lib
> > +++ b/scripts/Makefile.lib
> > @@ -361,9 +361,8 @@ $(multi-dtb-y): FORCE
> > $(call multi_depend, $(multi-dtb-y), .dtb, -dtbs)
> >
> > DT_CHECKER ?= dt-validate
> > -DT_CHECKER_FLAGS ?= $(if $(DT_SCHEMA_FILES),,-m)
> > +DT_CHECKER_FLAGS ?= $(if $(DT_SCHEMA_FILES),-l $(DT_SCHEMA_FILES),-m)
> > DT_BINDING_DIR := Documentation/devicetree/bindings
> > -# DT_TMP_SCHEMA may be overridden from Documentation/devicetree/bindings/Makefile
> > DT_TMP_SCHEMA ?= $(objtree)/$(DT_BINDING_DIR)/processed-schema.json
>
> This could now use := instead of ?=
Yes, though I think it is possible we may still want to override it.
Other than debugging perhaps, I don't have an immediate reason.
> Apart from the fact that 2022.3 hasn't been tagged yet as pointed out by
> Geert, this looks fine to me (but I'm no expert in this area).
It's now there.
Rob