Section 13.7.6.13 "CSI Image Parameter Register" of the
i.MX 8M Quad Applications Processors Reference Manual
states that the line size should be divisible by 8 bytes.
However, the hardware also accepts sizes divisible by 4 bytes.
This patch accepts line sizes divisible 4-bytes in non-planar mode.
Signed-off-by: Dorota Czaplejewicz <[email protected]>
---
Hello,
my previous patch identified something that was not a problem,
so I'm sending a different one.
This has been tested on the Librem 5.
Cheers,
Dorota
drivers/staging/media/imx/imx-media-utils.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
index 5128915a5d6f..a303003929e3 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -545,13 +545,13 @@ int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix,
}
/* Round up width for minimum burst size */
- width = round_up(mbus->width, 8);
+ width = round_up(mbus->width, 4);
/* Round up stride for IDMAC line start address alignment */
if (cc->planar)
stride = round_up(width, 16);
else
- stride = round_up((width * cc->bpp) >> 3, 8);
+ stride = round_up((width * cc->bpp) >> 3, 4);
pix->width = width;
pix->height = mbus->height;
--
2.31.1
Hi Dorota,
On Wed, 2021-10-06 at 13:05 +0200, Dorota Czaplejewicz wrote:
> Section 13.7.6.13 "CSI Image Parameter Register" of the
> i.MX 8M Quad Applications Processors Reference Manual
> states that the line size should be divisible by 8 bytes.
> However, the hardware also accepts sizes divisible by 4 bytes.
>
> This patch accepts line sizes divisible 4-bytes in non-planar mode.
Thank you, this makes it much clearer. I see two issues with this,
though, one small and one a bit bigger:
First, I'd be wary of disregarding the reference manual - unless we know
better, and then it should be well documented in the code. It might be
that the 8-byte alignment requirement stems from the fact that the FIFO
operates in double-word units, which might cause the CSI to write over
the end of the buffer if the line width is odd (in 32-bit words).
Or maybe it's just that the FBUF_STRIDE conflicts with this, I'm unclear
on whether that is only given in units of dwords (although the driver
currently doesn't support this anyway).
I wonder: if you use 4-byte aligned width and odd height, does the CSI
write over the end of the buffer?
> Signed-off-by: Dorota Czaplejewicz <[email protected]>
> ---
>
> Hello,
>
> my previous patch identified something that was not a problem,
> so I'm sending a different one.
>
> This has been tested on the Librem 5.
>
> Cheers,
> Dorota
>
> drivers/staging/media/imx/imx-media-utils.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
> index 5128915a5d6f..a303003929e3 100644
> --- a/drivers/staging/media/imx/imx-media-utils.c
> +++ b/drivers/staging/media/imx/imx-media-utils.c
> @@ -545,13 +545,13 @@ int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix,
> }
>
> /* Round up width for minimum burst size */
> - width = round_up(mbus->width, 8);
> + width = round_up(mbus->width, 4);
>
> /* Round up stride for IDMAC line start address alignment */
> if (cc->planar)
> stride = round_up(width, 16);
> else
> - stride = round_up((width * cc->bpp) >> 3, 8);
> + stride = round_up((width * cc->bpp) >> 3, 4);
Second, even if this works fine on the i.MX7/8M CSI, the alignment is
still required for the i.MX5/6 IPU, for which this code and the comments
were written. So we need a way to differentiate the two cases here.
regards
Philipp
Hello,
On Fri, 08 Oct 2021 14:19:41 +0200
Philipp Zabel <[email protected]> wrote:
> Hi Dorota,
>
> On Wed, 2021-10-06 at 13:05 +0200, Dorota Czaplejewicz wrote:
> > Section 13.7.6.13 "CSI Image Parameter Register" of the
> > i.MX 8M Quad Applications Processors Reference Manual
> > states that the line size should be divisible by 8 bytes.
> > However, the hardware also accepts sizes divisible by 4 bytes.
> >
> > This patch accepts line sizes divisible 4-bytes in non-planar mode.
>
> Thank you, this makes it much clearer. I see two issues with this,
> though, one small and one a bit bigger:
>
> First, I'd be wary of disregarding the reference manual - unless we know
> better, and then it should be well documented in the code. It might be
> that the 8-byte alignment requirement stems from the fact that the FIFO
> operates in double-word units, which might cause the CSI to write over
> the end of the buffer if the line width is odd (in 32-bit words).
> Or maybe it's just that the FBUF_STRIDE conflicts with this, I'm unclear
> on whether that is only given in units of dwords (although the driver
> currently doesn't support this anyway).
>
> I wonder: if you use 4-byte aligned width and odd height, does the CSI
> write over the end of the buffer?
I tested this case, and found a glitch which suggests the last 4 bytes are ignored:
https://source.puri.sm/Librem5/linux-next/uploads/cfb59e3832431aaa3a69549455502568/image.png
That would be taken care of rounding up towards a number decided at runtime, like:
divisor = 8 >> (mbus->height % 2);
>
> > Signed-off-by: Dorota Czaplejewicz <[email protected]>
> > ---
> >
> > Hello,
> >
> > my previous patch identified something that was not a problem,
> > so I'm sending a different one.
> >
> > This has been tested on the Librem 5.
> >
> > Cheers,
> > Dorota
> >
> > drivers/staging/media/imx/imx-media-utils.c | 4 ++--
> > 1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
> > index 5128915a5d6f..a303003929e3 100644
> > --- a/drivers/staging/media/imx/imx-media-utils.c
> > +++ b/drivers/staging/media/imx/imx-media-utils.c
> > @@ -545,13 +545,13 @@ int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix,
> > }
> >
> > /* Round up width for minimum burst size */
> > - width = round_up(mbus->width, 8);
> > + width = round_up(mbus->width, 4);
> >
> > /* Round up stride for IDMAC line start address alignment */
> > if (cc->planar)
> > stride = round_up(width, 16);
> > else
> > - stride = round_up((width * cc->bpp) >> 3, 8);
> > + stride = round_up((width * cc->bpp) >> 3, 4);
>
> Second, even if this works fine on the i.MX7/8M CSI, the alignment is
> still required for the i.MX5/6 IPU, for which this code and the comments
> were written. So we need a way to differentiate the two cases here.
>
> regards
> Philipp
How best to go about this? I can see in the file imx-media-capture.c that there the video device lives in struct capture_priv.vdev.vfd. Would that be the right place to query about the underlying hardware?
Then the following functions would gain a new "small_divisor" parameter:
- imx_media_mbus_fmt_to_pix_fmt (a GPL symbol)
- imx_media_mbus_fmt_to_ipu_image (a GPL symbol)
- __capture_try_fmt
Those would have to extract the device type from struct capture_priv:
- __capture_legacy_try_fmt
- capture_try_fmt_vid_cap
- capture_s_fmt_vid_cap
- capture_init_format
Regards,
Dorota
Hi Dorota,
On Wed, 2021-10-13 at 11:26 +0200, Dorota Czaplejewicz wrote:
> On Fri, 08 Oct 2021 14:19:41 +0200 Philipp Zabel <[email protected]> wrote:
[...]
> > I wonder: if you use 4-byte aligned width and odd height, does the CSI
> > write over the end of the buffer?
>
> I tested this case, and found a glitch which suggests the last 4 bytes are ignored:
>
> https://source.puri.sm/Librem5/linux-next/uploads/cfb59e3832431aaa3a69549455502568/image.png
Thank you for testing, so it appears that at least without FBUF_STRIDE
the only requirement is that the whole image size must be a multiple of
8 bytes.
> That would be taken care of rounding up towards a number decided at runtime, like:
>
> divisor = 8 >> (mbus->height % 2);
Which would then cause the CSI to write past the end of the buffer?
I'd rather make sure that either the number of lines is even or the
width is a multiple of 8 bytes.
> > > Signed-off-by: Dorota Czaplejewicz <[email protected]>
> > > ---
> > >
> > > Hello,
> > >
> > > my previous patch identified something that was not a problem,
> > > so I'm sending a different one.
> > >
> > > This has been tested on the Librem 5.
> > >
> > > Cheers,
> > > Dorota
> > >
> > > drivers/staging/media/imx/imx-media-utils.c | 4 ++--
> > > 1 file changed, 2 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
> > > index 5128915a5d6f..a303003929e3 100644
> > > --- a/drivers/staging/media/imx/imx-media-utils.c
> > > +++ b/drivers/staging/media/imx/imx-media-utils.c
> > > @@ -545,13 +545,13 @@ int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix,
> > > }
> > >
> > > /* Round up width for minimum burst size */
> > > - width = round_up(mbus->width, 8);
> > > + width = round_up(mbus->width, 4);
> > >
> > > /* Round up stride for IDMAC line start address alignment */
> > > if (cc->planar)
> > > stride = round_up(width, 16);
> > > else
> > > - stride = round_up((width * cc->bpp) >> 3, 8);
> > > + stride = round_up((width * cc->bpp) >> 3, 4);
> >
> > Second, even if this works fine on the i.MX7/8M CSI, the alignment is
> > still required for the i.MX5/6 IPU, for which this code and the comments
> > were written. So we need a way to differentiate the two cases here.
> >
> > regards
> > Philipp
>
> How best to go about this? I can see in the file imx-media-capture.c
> that there the video device lives in struct capture_priv.vdev.vfd.
> Would that be the right place to query about the underlying hardware?
>
> Then the following functions would gain a new "small_divisor" parameter:
> - imx_media_mbus_fmt_to_pix_fmt (a GPL symbol)
> - imx_media_mbus_fmt_to_ipu_image (a GPL symbol)
> - __capture_try_fmt
That sounds like it would work around the current code when it (at least
part of imx_media_mbus_fmt_to_pix_fmt()) should be split between i.MX5/6
and i.MX7/8 implementations. For example rounding up the stride is not
useful on i.MX7/8, it just doesn't currently hurt because imx7-media-csi
is not using bytesperline to set up FBUF_STRIDE. And certainly the
comments don't apply.
imx_media_mbus_fmt_to_ipu_image() is unused and should probably be
dropped, same as imx_media_ipu_image_to_mbus_fmt().
> Those would have to extract the device type from struct capture_priv:
> - __capture_legacy_try_fmt
> - capture_try_fmt_vid_cap
> - capture_s_fmt_vid_cap
> - capture_init_format
Maybe imx_media_mbus_fmt_to_pix_fmt should be moved into imx-media-
capture.c be passed struct capture_priv to avoid duplicating the device
type check?
imx_media_capture_device_init() could gain a new parameter (or maybe
replace legacy_api) to set the device type.
regards
Philipp
Hello,
On Thu, 14 Oct 2021 13:26:26 +0200
Philipp Zabel <[email protected]> wrote:
> Hi Dorota,
>
> On Wed, 2021-10-13 at 11:26 +0200, Dorota Czaplejewicz wrote:
> > On Fri, 08 Oct 2021 14:19:41 +0200 Philipp Zabel <[email protected]> wrote:
> [...]
> > > I wonder: if you use 4-byte aligned width and odd height, does the CSI
> > > write over the end of the buffer?
> >
> > I tested this case, and found a glitch which suggests the last 4 bytes are ignored:
> >
> > https://source.puri.sm/Librem5/linux-next/uploads/cfb59e3832431aaa3a69549455502568/image.png
>
> Thank you for testing, so it appears that at least without FBUF_STRIDE
> the only requirement is that the whole image size must be a multiple of
> 8 bytes.
>
> > That would be taken care of rounding up towards a number decided at runtime, like:
> >
> > divisor = 8 >> (mbus->height % 2);
>
> Which would then cause the CSI to write past the end of the buffer?
>
I'm not sure if you point out the mistake here (should be "4 <<"), or the fact that rounding is happening. If it's the latter, then it's of no concern: the values derived here are used to calculate buffer size.
I'm submitting a new series where this is fixed.
> I'd rather make sure that either the number of lines is even or the
> width is a multiple of 8 bytes.
>
> > > > Signed-off-by: Dorota Czaplejewicz <[email protected]>
> > > > ---
> > > >
> > > > Hello,
> > > >
> > > > my previous patch identified something that was not a problem,
> > > > so I'm sending a different one.
> > > >
> > > > This has been tested on the Librem 5.
> > > >
> > > > Cheers,
> > > > Dorota
> > > >
> > > > drivers/staging/media/imx/imx-media-utils.c | 4 ++--
> > > > 1 file changed, 2 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
> > > > index 5128915a5d6f..a303003929e3 100644
> > > > --- a/drivers/staging/media/imx/imx-media-utils.c
> > > > +++ b/drivers/staging/media/imx/imx-media-utils.c
> > > > @@ -545,13 +545,13 @@ int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix,
> > > > }
> > > >
> > > > /* Round up width for minimum burst size */
> > > > - width = round_up(mbus->width, 8);
> > > > + width = round_up(mbus->width, 4);
> > > >
> > > > /* Round up stride for IDMAC line start address alignment */
> > > > if (cc->planar)
> > > > stride = round_up(width, 16);
> > > > else
> > > > - stride = round_up((width * cc->bpp) >> 3, 8);
> > > > + stride = round_up((width * cc->bpp) >> 3, 4);
> > >
> > > Second, even if this works fine on the i.MX7/8M CSI, the alignment is
> > > still required for the i.MX5/6 IPU, for which this code and the comments
> > > were written. So we need a way to differentiate the two cases here.
> > >
> > > regards
> > > Philipp
> >
> > How best to go about this? I can see in the file imx-media-capture.c
> > that there the video device lives in struct capture_priv.vdev.vfd.
> > Would that be the right place to query about the underlying hardware?
> >
> > Then the following functions would gain a new "small_divisor" parameter:
> > - imx_media_mbus_fmt_to_pix_fmt (a GPL symbol)
> > - imx_media_mbus_fmt_to_ipu_image (a GPL symbol)
> > - __capture_try_fmt
>
> That sounds like it would work around the current code when it (at least
> part of imx_media_mbus_fmt_to_pix_fmt()) should be split between i.MX5/6
> and i.MX7/8 implementations. For example rounding up the stride is not
> useful on i.MX7/8, it just doesn't currently hurt because imx7-media-csi
> is not using bytesperline to set up FBUF_STRIDE. And certainly the
> comments don't apply.
>
> imx_media_mbus_fmt_to_ipu_image() is unused and should probably be
> dropped, same as imx_media_ipu_image_to_mbus_fmt().
Done in next series.
>
> > Those would have to extract the device type from struct capture_priv:
> > - __capture_legacy_try_fmt
> > - capture_try_fmt_vid_cap
> > - capture_s_fmt_vid_cap
> > - capture_init_format
>
> Maybe imx_media_mbus_fmt_to_pix_fmt should be moved into imx-media-
> capture.c be passed struct capture_priv to avoid duplicating the device
> type check?
>
I opted not to, in favor of passing the actual device type. It comes out to the same thing, except a simple value is passed around instead of a device.
> imx_media_capture_device_init() could gain a new parameter (or maybe
> replace legacy_api) to set the device type.
Thanks, this is what I was missing.
Regards,
Dorota
>
> regards
> Philipp
On Sun, 2021-10-17 at 13:07 +0200, Dorota Czaplejewicz wrote:
> Hello,
>
> On Thu, 14 Oct 2021 13:26:26 +0200
> Philipp Zabel <[email protected]> wrote:
>
> > Hi Dorota,
> >
> > On Wed, 2021-10-13 at 11:26 +0200, Dorota Czaplejewicz wrote:
> > > On Fri, 08 Oct 2021 14:19:41 +0200 Philipp Zabel <[email protected]> wrote:
> > [...]
> > > > I wonder: if you use 4-byte aligned width and odd height, does the CSI
> > > > write over the end of the buffer?
> > >
> > > I tested this case, and found a glitch which suggests the last 4 bytes are ignored:
> > >
> > > https://source.puri.sm/Librem5/linux-next/uploads/cfb59e3832431aaa3a69549455502568/image.png
> >
> > Thank you for testing, so it appears that at least without FBUF_STRIDE
> > the only requirement is that the whole image size must be a multiple of
> > 8 bytes.
> >
> > > That would be taken care of rounding up towards a number decided at runtime, like:
> > >
> > > divisor = 8 >> (mbus->height % 2);
> >
> > Which would then cause the CSI to write past the end of the buffer?
> >
> I'm not sure if you point out the mistake here (should be "4 <<"), or
> the fact that rounding is happening. If it's the latter, then it's of
> no concern: the values derived here are used to calculate buffer size.
>
> I'm submitting a new series where this is fixed.
Thanks, I just didn't understand your intention. The name "divisor"
threw me off, see the comment in the new series.
regards
Philipp
