2022-03-09 15:09:59

by Alexander Stein

[permalink] [raw]
Subject: [PATCH v3 0/8] imx7/imx8mm media / csi patches

Hey everyone,

thanks for the feedback in v2.

Changes in v3:
* Dropped PATCH v2 5/9, applied in commit a7a52e146d0e
* Added R-b Laurent Pinchart to PATCH 1 & 4
* Fix bpp check in PATCH 3

Below is the original message slightly modified to match the v3 changes.

This is a set of patch for imx7 media drivers based on next-20220308. With
this set I was able to capture video frames from a MIPI CSI-2 camera in my
TQMa8MQML + MBA8MX hardware. The actual camera used is a Vision Components
'VC MIPI IMX327 C' camera. IMX327 is compatible to IMX290. Patch 8 shows the
DT overlay I'm using, not suitable for merging.
Please ignore the FPGA part, this is mainly for power supply and GPIO reset
line. This is currently a custom driver I'm working on, but I do not want to
focus on in this series.

Please note I tested this only on this imx8 platform.

First thanks to Dorota for the patchset at [1] (patches 1-4) which is necessary
to capture correct images.

Starting from patch 5 there are small fixes which allows me to configure my
media device.

Device configuration:
```
media-ctl -l "'imx290 2-001a':0->'csis-32e30000.mipi-csi':0 [1]"
media-ctl -V "'imx290 2-001a':0 [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw]"
media-ctl -V "'csi':0 [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw]"
v4l2-ctl -d0 --set-fmt-video width=1920,height=1080,pixelformat='RG10',field=none
media-ctl -p
```

The media-ctl topology is:
```
$ media-ctl -p
Media controller API version 5.17.0

Media device information
------------------------
driver imx7-csi
model imx-media
serial
bus info platform:32e20000.csi
hw revision 0x0
driver version 5.17.0

Device topology
- entity 1: csi (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev0
pad0: Sink
[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range]
<- "csis-32e30000.mipi-csi":1 [ENABLED,IMMUTABLE]
pad1: Source
[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range]
-> "csi capture":0 [ENABLED,IMMUTABLE]

- entity 4: csi capture (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "csi":1 [ENABLED,IMMUTABLE]

- entity 10: csis-32e30000.mipi-csi (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw]
<- "imx290 2-001a":0 [ENABLED]
pad1: Source
[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw]
-> "csi":0 [ENABLED,IMMUTABLE]

- entity 15: imx290 2-001a (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev2
pad0: Source
[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw]
-> "csis-32e30000.mipi-csi":0 [ENABLED]
```

Note: MIPI CSI settle times are not calculated correctly right now and need a
manual overwrite:
echo 13 > /sys/kernel/debug/32e30000.mipi-csi/ths_settle
echo 2 > /sys/kernel/debug/32e30000.mipi-csi/tclk_settle

I did not focus on v4l2-compliance tool, this is a future task.

Regards,
Alexander

[1] https://patchwork.linuxtv.org/project/linux-media/patch/[email protected]/

Alexander Stein (4):
media: imx: imx7_media-csi: Add support for additional Bayer patterns
media: imx: utils: Add more Bayer formats
media: imx: utils: initialize local variable
[DNI] arm64: dts: tqma8mqml: add IMX327 MIPI-CSI overlay

Dorota Czaplejewicz (4):
media: imx: Store the type of hardware implementation
media: imx: Forward type of hardware implementation
media: imx: Use dedicated format handler for i.MX7/8
media: imx: Fail conversion if pixel format not supported

arch/arm64/boot/dts/freescale/Makefile | 4 +
.../imx8mm-tqma8mqml-mba8mx-imx327.dts | 95 ++++++++++++
drivers/staging/media/imx/imx-ic-prpencvf.c | 3 +-
drivers/staging/media/imx/imx-media-capture.c | 20 ++-
drivers/staging/media/imx/imx-media-csi.c | 3 +-
drivers/staging/media/imx/imx-media-utils.c | 136 +++++++++++++++++-
drivers/staging/media/imx/imx-media.h | 11 +-
drivers/staging/media/imx/imx7-media-csi.c | 15 +-
8 files changed, 271 insertions(+), 16 deletions(-)
create mode 100644 arch/arm64/boot/dts/freescale/imx8mm-tqma8mqml-mba8mx-imx327.dts

--
2.25.1


2022-03-09 15:44:57

by Alexander Stein

[permalink] [raw]
Subject: [PATCH v3 5/8] media: imx: imx7_media-csi: Add support for additional Bayer patterns

imx7_csi_configure() allows configuring these Bayer patterns when
starting a stream. So allow these in link_validate() as well.

Reviewed-by: Laurent Pinchart <[email protected]>
Signed-off-by: Alexander Stein <[email protected]>
---
Changes in v3:
* None

drivers/staging/media/imx/imx7-media-csi.c | 12 ++++++++++++
1 file changed, 12 insertions(+)

diff --git a/drivers/staging/media/imx/imx7-media-csi.c b/drivers/staging/media/imx/imx7-media-csi.c
index cf74960dee84..82627bd0c9ed 100644
--- a/drivers/staging/media/imx/imx7-media-csi.c
+++ b/drivers/staging/media/imx/imx7-media-csi.c
@@ -1028,6 +1028,18 @@ static int imx7_csi_pad_link_validate(struct v4l2_subdev *sd,
case V4L2_PIX_FMT_SGBRG8:
case V4L2_PIX_FMT_SGRBG8:
case V4L2_PIX_FMT_SRGGB8:
+ case V4L2_PIX_FMT_SBGGR10:
+ case V4L2_PIX_FMT_SGBRG10:
+ case V4L2_PIX_FMT_SGRBG10:
+ case V4L2_PIX_FMT_SRGGB10:
+ case V4L2_PIX_FMT_SBGGR12:
+ case V4L2_PIX_FMT_SGBRG12:
+ case V4L2_PIX_FMT_SGRBG12:
+ case V4L2_PIX_FMT_SRGGB12:
+ case V4L2_PIX_FMT_SBGGR14:
+ case V4L2_PIX_FMT_SGBRG14:
+ case V4L2_PIX_FMT_SGRBG14:
+ case V4L2_PIX_FMT_SRGGB14:
case V4L2_PIX_FMT_SBGGR16:
case V4L2_PIX_FMT_SGBRG16:
case V4L2_PIX_FMT_SGRBG16:
--
2.25.1

2022-03-09 16:16:15

by Alexander Stein

[permalink] [raw]
Subject: [PATCH v3 3/8] media: imx: Use dedicated format handler for i.MX7/8

From: Dorota Czaplejewicz <[email protected]>

This splits out a format handler which takes into account
the capabilities of the i.MX7/8 video device,
as opposed to the default handler compatible with both i.MX5/6 and i.MX7/8.

Signed-off-by: Dorota Czaplejewicz <[email protected]>
Signed-off-by: Alexander Stein <[email protected]>
---
Changes in v3:
* Fixed bbp check (This driver uses bits per pixel, not bytes per pixel)

drivers/staging/media/imx/imx-media-utils.c | 60 +++++++++++++++++++--
1 file changed, 56 insertions(+), 4 deletions(-)

diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
index c42f3da8e3a8..0534f7746cb3 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -516,10 +516,9 @@ void imx_media_try_colorimetry(struct v4l2_mbus_framefmt *tryfmt,
}
EXPORT_SYMBOL_GPL(imx_media_try_colorimetry);

-int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix,
- const struct v4l2_mbus_framefmt *mbus,
- const struct imx_media_pixfmt *cc,
- enum imx_media_device_type type)
+static int imx56_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix,
+ const struct v4l2_mbus_framefmt *mbus,
+ const struct imx_media_pixfmt *cc)
{
u32 width;
u32 stride;
@@ -568,6 +567,59 @@ int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix,

return 0;
}
+
+static int imx78_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix,
+ const struct v4l2_mbus_framefmt *mbus,
+ const struct imx_media_pixfmt *cc)
+{
+ u32 aligned_width;
+ int ret;
+
+ if (!cc)
+ cc = imx_media_find_mbus_format(mbus->code, PIXFMT_SEL_ANY);
+
+ if (!cc)
+ return -EINVAL;
+ /*
+ * The hardware can handle line lengths divisible by 4 pixels
+ * as long as the whole buffer size ends up divisible by 8 bytes.
+ * If not, use the value of 8 pixels recommended in the datasheet.
+ * Only 8bits-per-pixel formats may need to get aligned to 8 pixels,
+ * because both 10-bit and 16-bit pixels occupy 2 bytes.
+ * In those, 4-pixel aligmnent is equal to 8-byte alignment.
+ */
+ if (cc->bpp == 8)
+ aligned_width = round_up(mbus->width, 8);
+ else
+ aligned_width = round_up(mbus->width, 4);
+
+ ret = v4l2_fill_pixfmt(pix, cc->fourcc,
+ aligned_width, mbus->height);
+ if (ret)
+ return ret;
+
+ pix->colorspace = mbus->colorspace;
+ pix->xfer_func = mbus->xfer_func;
+ pix->ycbcr_enc = mbus->ycbcr_enc;
+ pix->quantization = mbus->quantization;
+ pix->field = mbus->field;
+
+ return ret;
+}
+
+int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix,
+ const struct v4l2_mbus_framefmt *mbus,
+ const struct imx_media_pixfmt *cc,
+ enum imx_media_device_type type)
+{
+ switch (type) {
+ case DEVICE_TYPE_IMX56:
+ return imx56_media_mbus_fmt_to_pix_fmt(pix, mbus, cc);
+ case DEVICE_TYPE_IMX78:
+ return imx78_media_mbus_fmt_to_pix_fmt(pix, mbus, cc);
+ }
+ return -EINVAL;
+}
EXPORT_SYMBOL_GPL(imx_media_mbus_fmt_to_pix_fmt);

void imx_media_free_dma_buf(struct device *dev,
--
2.25.1