Received-SPF: pass (google.com: domain of linux-kernel+bounces-212878-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Precedence: bulk
MIME-Version: 1.0
References: <20240605094843.4141730-1-wenst@chromium.org> <CAFLszTjX=ixC3pRRGJeaP=ie_yc+KcCRyQ06MBFeSZnBepaXaw@mail.gmail.com>
 <CAGXv+5EcEYGqXq2C1OCK4J4t1NusV7nWp16zb74P6_tCeLnSGw@mail.gmail.com> <CAK7LNAQ7XAGgzhBXQWPFVgqJwdBcO3mF5pmQ3mSsmdrZ0EBL9Q@mail.gmail.com>
In-Reply-To: <CAK7LNAQ7XAGgzhBXQWPFVgqJwdBcO3mF5pmQ3mSsmdrZ0EBL9Q@mail.gmail.com>
From: Chen-Yu Tsai <wenst@chromium.org>
Date: Thu, 13 Jun 2024 16:49:02 +0800
Message-ID: <CAGXv+5FpHkiJaNa=XRdnrmPJN01AdxsOYpTBur8zqM8tU=hFKQ@mail.gmail.com>
Subject: Re: [PATCH] scripts/make_fit: Support decomposing DTBs
To: Masahiro Yamada <masahiroy@kernel.org>
Cc: Simon Glass <sjg@chromium.org>, Nathan Chancellor <nathan@kernel.org>, 
	Nicolas Schier <nicolas@fjasle.eu>, linux-arm-kernel@lists.infradead.org, 
	linux-kernel@vger.kernel.org, linux-kbuild@vger.kernel.org, 
	Elliot Berman <quic_eberman@quicinc.com>, Devicetree List <devicetree@vger.kernel.org>, 
	Rob Herring <robh@kernel.org>, Krzysztof Kozlowski <krzk+dt@kernel.org>, Conor Dooley <conor+dt@kernel.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Jun 11, 2024 at 10:45=E2=80=AFPM Masahiro Yamada <masahiroy@kernel.=
org> wrote:
>
> On Tue, Jun 11, 2024 at 5:52=E2=80=AFPM Chen-Yu Tsai <wenst@chromium.org>=
 wrote:
> >
> > On Mon, Jun 10, 2024 at 11:16=E2=80=AFPM Simon Glass <sjg@chromium.org>=
 wrote:
> > >
> > > Hi Chen-Yu,
> > >
> > > On Wed, 5 Jun 2024 at 03:48, Chen-Yu Tsai <wenst@chromium.org> wrote:
> > > >
> > > > The kernel tree builds some "composite" DTBs, where the final DTB i=
s the
> > > > result of applying one or more DTB overlays on top of a base DTB wi=
th
> > > > fdtoverlay.
> > > >
> > > > The FIT image specification already supports configurations having =
one
> > > > base DTB and overlays applied on top. It is then up to the bootload=
er to
> > > > apply said overlays and either use or pass on the final result. Thi=
s
> > > > allows the FIT image builder to reuse the same FDT images for multi=
ple
> > > > configurations, if such cases exist.
> > > >
> > > > The decomposition function depends on the kernel build system, read=
ing
> > > > back the .cmd files for the to-be-packaged DTB files to check for t=
he
> > > > fdtoverlay command being called. This will not work outside the ker=
nel
> > > > tree. The function is off by default to keep compatibility with pos=
sible
> > > > existing users.
> > > >
> > > > To facilitate the decomposition and keep the code clean, the model =
and
> > > > compatitble string extraction have been moved out of the output_dtb
> > > > function. The FDT image description is replaced with the base file =
name
> > > > of the included image.
> > > >
> > > > Signed-off-by: Chen-Yu Tsai <wenst@chromium.org>
> > > > ---
> > > > This is a feature I alluded to in my replies to Simon's original
> > > > submission of the make_fit.py script [1].
> > > >
> > > > This is again made a runtime argument as not all firmware out there
> > > > that boot FIT images support applying overlays. Like my previous
> > > > submission for disabling compression for included FDT images, the
> > > > bootloader found in RK3399 and MT8173 Chromebooks do not support
> > > > applying overlays. Another case of this is U-boot shipped by develo=
pment
> > > > board vendors in binary form (without upstream) in an image or in
> > > > SPI flash on the board that were built with OF_LIBFDT_OVERLAY=3Dn.
> > > > These would fail to boot FIT images with DT overlays. One such
> > > > example is my Hummingboard Pulse. In these cases the firmware is
> > > > either not upgradable or very hard to upgrade.
> > > >
> > > > I believe there is value in supporting these cases. A common script
> > > > shipped with the kernel source that can be shared by distros means
> > > > the distro people don't have to reimplement this in their downstrea=
m
> > > > repos or meta-packages. For ChromeOS this means reducing the amount
> > > > of package code we have in shell script.
> > > >
> > > > [1] https://lore.kernel.org/linux-kbuild/20231207142723.GA3187877@g=
oogle.com/
> > > > [2]
> > > >
> > > >  scripts/Makefile.lib |  1 +
> > > >  scripts/make_fit.py  | 70 ++++++++++++++++++++++++++++++----------=
----
> > > >  2 files changed, 49 insertions(+), 22 deletions(-)
> > >
> > > This is a clever way to discover the included files. Does it need to
> > > rely on the Linux build information, or could this information someho=
w
> > > be in the .dtb files? I had expected some sort of overlay scheme in
> >
> > (+CC DT folks and mailing list)
> >
> > I suppose we could make the `fdtoverlay` program embed this data during
> > the kernel build. That would keep the information together, while also
> > having one source of truth (the kernel Makefiles). Whether it belongs
> > in the DTB or not is a separate matter.
>
>
> Some time ago, I asked a similar question
> with a similar motivation.
>
> https://lore.kernel.org/devicetree-compiler/CAK7LNARV8Bo-tBXMdOu55Wg9uZRX=
vNiRdkDJ4LH8PwVMnMp4cA@mail.gmail.com/

I think this discussion was geared towards "unapplying" overlays. What
we would like is for metadata to be available, and preferably not tied
to the kernel build system generated metadata file, so that with the same
bunch of DTB + DTBO files, we could figure out how to decompose them and
just bundle the components.

If the metadata is embedded within the composite DTB, then given a DTB
bundle (such as installed with `make dtbs_install`) the make_fit.py
script could go and do the decomposition without the *.cmd files
from the kernel build. This is assuming all the component parts are
installed together.

ChenYu

> >
> > > the source, but perhaps people have given up on that?
> >
> > I wouldn't say given up, since we haven't agreed on anything either.
> > Elliot had some concerns when I brought this up earlier [1] though.
> >
> > ChenYu
> >
> > [1] https://lore.kernel.org/linux-mediatek/20240314113908471-0700.eberm=
an@hu-eberman-lv.qualcomm.com/
> >
>
> --
> Best Regards
> Masahiro Yamada