by Laurent Pinchart

[permalink] [raw]

Subject: Re: [PATCH 2/2] arm64: dts: imx8mp: Add overlays for ISP instances

On Thu, Nov 30, 2023 at 04:34:11PM +0100, Alexander Stein wrote:
> Am Donnerstag, 30. November 2023, 15:20:48 CET schrieb Laurent Pinchart:
> > On Thu, Nov 30, 2023 at 10:51:22AM +0100, Alexander Stein wrote:
> > > Am Mittwoch, 29. November 2023, 16:16:37 CET schrieb Laurent Pinchart:
> > > > On Wed, Nov 29, 2023 at 11:20:07AM +0100, Alexander Stein wrote:
> > > > > Am Mittwoch, 29. November 2023, 10:31:13 CET schrieb Paul Elder:
> > > > > > From: Laurent Pinchart <[email protected]>
> > > > > >
> > > > > > Add two overlay to enable each ISP instance. The ISP is wired
> > > > > > directly
> > > > > > to the CSIS for now, bypassing the ISI completely.
> > > > >
> > > > > I'm not sure if this is worth adding in a separate overlay.
> > > >
> > > > The trouble is that, at this point, selection between the ISP and the
> > > > ISI can only be performed through DT :-S That's why this is implemented
> > > > as an overlay.
> > >
> > > I feel a better place would be the overlay which actually adds the sensor.
> > > This knows best whether ISI or ISP should be used.
> >
> > Any sensor could be used with either the ISI or the ISP, so I don't
> > think the camera module overlay would be the best place for this. Unless
> > you want to duplicate all camera module overlays, with an ISI version
> > and an ISP version :-)
>
> True, that's a really good argument for having these small overlays.
> But how to deal with dtc warnings?
> > imx8mp-isp1.dtbo: Warning (graph_port): /fragment@2: graph port node name
> should be 'port'
> > imx8mp-isp1.dtso:34.17-36.3: Warning (graph_endpoint): /fragment@2/
> __overlay__: graph endpoint node name should be 'endpoint'
> > imx8mp-isp1.dtso:34.17-36.3: Warning (graph_endpoint): /fragment@2/
> __overlay__: graph connection to node '/fragment@1/__overlay__/ports/port@1/
> endpoint' is not bidirectional

See below :-)

> But for the small overlay itself:
> Tested-by: Alexander Stein <[email protected]>
>
> > > > > > Signed-off-by: Laurent Pinchart <[email protected]>
> > > > > > Signed-off-by: Paul Elder <[email protected]>
> > > > > > ---
> > > > > >
> > > > > > arch/arm64/boot/dts/freescale/Makefile | 2 ++
> > > > > > .../arm64/boot/dts/freescale/imx8mp-isp1.dtso | 36 +++++++++++++++++++
> > > > > > .../arm64/boot/dts/freescale/imx8mp-isp2.dtso | 36 +++++++++++++++++++
> > > > > > 3 files changed, 74 insertions(+)
> > > > > > create mode 100644 arch/arm64/boot/dts/freescale/imx8mp-isp1.dtso
> > > > > > create mode 100644 arch/arm64/boot/dts/freescale/imx8mp-isp2.dtso
> > > > > >
> > > > > > diff --git a/arch/arm64/boot/dts/freescale/Makefile
> > > > > > b/arch/arm64/boot/dts/freescale/Makefile index
> > > > > > 300049037eb0..f97dfac11189
> > > > > > 100644
> > > > > > --- a/arch/arm64/boot/dts/freescale/Makefile
> > > > > > +++ b/arch/arm64/boot/dts/freescale/Makefile
> > > > > > @@ -113,6 +113,8 @@ dtb-$(CONFIG_ARCH_MXC) += imx8mp-dhcom-pdk2.dtb
> > > > > >
> > > > > > dtb-$(CONFIG_ARCH_MXC) += imx8mp-dhcom-pdk3.dtb
> > > > > > dtb-$(CONFIG_ARCH_MXC) += imx8mp-evk.dtb
> > > > > > dtb-$(CONFIG_ARCH_MXC) += imx8mp-icore-mx8mp-edimm2.2.dtb
> > > > > >
> > > > > > +dtb-$(CONFIG_ARCH_MXC) += imx8mp-isp1.dtbo
> > > > > > +dtb-$(CONFIG_ARCH_MXC) += imx8mp-isp2.dtbo

Overlays need to be validated in the context of a base DT on which they
apply. For instance, in the same Makefile, we have

--------
dtb-$(CONFIG_ARCH_MXC) += imx8mm-tqma8mqml-mba8mx.dtb
...
imx8mm-tqma8mqml-mba8mx-lvds-tm070jvhg33-dtbs += imx8mm-tqma8mqml-mba8mx.dtb imx8mm-tqma8mqml-mba8mx-lvds-tm070jvhg33.dtbo
dtb-$(CONFIG_ARCH_MXC) += imx8mm-tqma8mqml-mba8mx-lvds-tm070jvhg33.dtb
--------

imx8mm-tqma8mqml-mba8mx.dts is the base board DT, and
imx8mm-tqma8mqml-mba8mx-lvds-tm070jvhg33.dtso the overlay. As far as I
understand, when compiling dtbs, the build system will compile
imx8mm-tqma8mqml-mba8mx.dtb and
imx8mm-tqma8mqml-mba8mx-lvds-tm070jvhg33-dtb. To create the latter, it
will compile imx8mm-tqma8mqml-mba8mx-lvds-tm070jvhg33.dtbo and apply it
to imx8mm-tqma8mqml-mba8mx.dtb. Then, it will validate the DTBs
specified as part of dtb-$(CONFIG_ARCH_MXC), which are
imx8mm-tqma8mqml-mba8mx.dtb standlone, and through
imx8mm-tqma8mqml-mba8mx-lvds-tm070jvhg33-dtbs
imx8mm-tqma8mqml-mba8mx.dtb with the overlay applied.

TL;DR: a v2 of this patch should fix the Makefile, and be compile-tested
with make dtbs.

Rob, Conor or Krzysztof can correct me if I'm wrong.

> > > > > >
> > > > > > dtb-$(CONFIG_ARCH_MXC) += imx8mp-msc-sm2s-ep1.dtb
> > > > > > dtb-$(CONFIG_ARCH_MXC) += imx8mp-phyboard-pollux-rdk.dtb
> > > > > > dtb-$(CONFIG_ARCH_MXC) += imx8mp-tqma8mpql-mba8mpxl.dtb
> > > > > >
> > > > > > diff --git a/arch/arm64/boot/dts/freescale/imx8mp-isp1.dtso
> > > > > > b/arch/arm64/boot/dts/freescale/imx8mp-isp1.dtso new file mode
> > > > > > 100644
> > > > > > index 000000000000..cf394ed224ab
> > > > > > --- /dev/null
> > > > > > +++ b/arch/arm64/boot/dts/freescale/imx8mp-isp1.dtso
> > > > > > @@ -0,0 +1,36 @@
> > > > > > +// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
> > > > > > +/*
> > > > > > + * Copyright 2022 Ideas on Board Oy
> > > > > > + */
> > > > > > +
> > > > > > +/dts-v1/;
> > > > > > +/plugin/;
> > > > > > +
> > > > > > +#include <dt-bindings/media/video-interfaces.h>
> > > > > > +
> > > > > > +&isi_0 {
> > > > > > + status = "disabled";
> > > > >
> > > > > ISI is disabled by default. What is your intention here?
> > > >
> > > > It could be enabled by an overlay for a camera module. Ideally we want
> > > > to be able to enable both the ISI and ISP at runtime, but that's not
> > > > possible yet and will require a very large amount of work.
> > >
> > > Again IMHO this is part of sensor setup, in a very specific overlay. To
> > > put it into different words: I barely see the gain of this small overlay.
> > >
> > > Runtime switching would require a combined media controller including both
> > > ISI and ISP, no?
> >
> > Correct, that's the hard part.
> >
> > > > > > +
> > > > > > + ports {
> > > > > > + port@0 {
> > > > > > + /delete-node/ endpoint;
> > > > >
> > > > > This doesn't work in overlays. See [1]. Otherwise the OF graph
> > > > > connections
> > > > > look fine to me. I'm using the same in my local overlay.
> > > >
> > > > Interesting, I wasn't aware of that. Maybe we should fix it :-)
> > > >
> > > > > [1] https://lore.kernel.org/all/CAMuHMdWu4KZbBkvEofUV2wuA1g2S=XHHM3RUN1cNrcZBkhsPZA@mail.gmail.com/
> > > > >
> > > > > > + };
> > > > > > + };
> > > > > > +};
> > > > > > +
> > > > > > +&isp_0 {
> > > > > > + status = "okay";
> > > > > > +
> > > > > > + ports {
> > > > > > + port@1 {
> > > > > > + isp0_in: endpoint {
> > > > > > + bus-type = <MEDIA_BUS_TYPE_PARALLEL>;
> > > > > > + remote-endpoint = <&mipi_csi_0_out>;
> > > > > > + };
> > > > > > + };
> > > > > > + };
> > > > > > +};
> > > > > > +
> > > > > > +&mipi_csi_0_out {
> > > > > > + remote-endpoint = <&isp0_in>;
> > > > > > +};
> > > > > > diff --git a/arch/arm64/boot/dts/freescale/imx8mp-isp2.dtso
> > > > > > b/arch/arm64/boot/dts/freescale/imx8mp-isp2.dtso new file mode
> > > > > > 100644
> > > > > > index 000000000000..14e2e7b2617f
> > > > > > --- /dev/null
> > > > > > +++ b/arch/arm64/boot/dts/freescale/imx8mp-isp2.dtso
> > > > > > @@ -0,0 +1,36 @@
> > > > > > +// SPDX-License-Identifier: (GPL-2.0+ OR MIT)
> > > > > > +/*
> > > > > > + * Copyright 2022 Ideas on Board Oy
> > > > > > + */
> > > > > > +
> > > > > > +/dts-v1/;
> > > > > > +/plugin/;
> > > > > > +
> > > > > > +#include <dt-bindings/media/video-interfaces.h>
> > > > > > +
> > > > > > +&isi_0 {
> > > > > > + status = "disabled";
> > > > > > +
> > > > > > + ports {
> > > > > > + port@1 {
> > > > > > + /delete-node/ endpoint;
> > > > > > + };
> > > > > > + };
> > > > > > +};
> > > > > > +
> > > > > > +&isp_1 {
> > > > > > + status = "okay";
> > > > > > +
> > > > > > + ports {
> > > > > > + port@1 {
> > > > > > + isp1_in: endpoint {
> > > > > > + bus-type = <MEDIA_BUS_TYPE_PARALLEL>;
> > > > > > + remote-endpoint = <&mipi_csi_1_out>;
> > > > > > + };
> > > > > > + };
> > > > > > + };
> > > > > > +};
> > > > > > +
> > > > > > +&mipi_csi_1_out {
> > > > > > + remote-endpoint = <&isp1_in>;
> > > > > > +};

--
Regards,

Laurent Pinchart