Since v1 Jernej patches to support 10 bits decoding have been merged.
This v2 rebase the previously send patches and add Ezequiel reviewed-by
tag.
This series add 10bits support for newer hardware blocks (i.e without
legacy regs) and for HEVC codec.
It based tested on IMX8MQ with HEVC conformance tests.
The results are that all 10bits bitstreams are OK.
The problematic point raise by this series is the question of where
to put HEVC validation function. Does it have to be only hantro_try_ctrl()
or shall it be done also in .run().
This series doesn't change that behavoir so I will argue in farvor to
merge it and to work on this validation time problem when unstaging
the driver.
Hans, Ezequiel may that sound acceptable for you ?
Regards,
Benjamin
Benjamin Gaignard (7):
media: hantro: Store HEVC bit depth in context
media: hantro: HEVC: Fix auxilary buffer size calculation
media: hantro: HEVC: Fix chroma offset computation
media: hantro: postproc: Configure output regs to support 10bit
media: Hantro: HEVC: Allows 10-bit bitstream
media: hantro: imx8m: Enable 10bit decoding
media: hantro: Allows luma and chroma depth to be different
.../staging/media/hantro/hantro_g2_hevc_dec.c | 4 +--
drivers/staging/media/hantro/hantro_hevc.c | 13 +++++----
.../staging/media/hantro/hantro_postproc.c | 7 ++++-
drivers/staging/media/hantro/imx8m_vpu_hw.c | 27 +++++++++++++++++++
4 files changed, 40 insertions(+), 11 deletions(-)
--
2.32.0
The chroma offset depends of the bitstream depth.
Make sure that ctx->bit_depth is used to compute it.
Signed-off-by: Benjamin Gaignard <[email protected]>
Reviewed-by: Ezequiel Garcia <[email protected]>
---
drivers/staging/media/hantro/hantro_g2_hevc_dec.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
index 233ecd863d5f..a917079a6ed3 100644
--- a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
+++ b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
@@ -12,7 +12,7 @@
static size_t hantro_hevc_chroma_offset(struct hantro_ctx *ctx)
{
- return ctx->dst_fmt.width * ctx->dst_fmt.height;
+ return ctx->dst_fmt.width * ctx->dst_fmt.height * ctx->bit_depth / 8;
}
static size_t hantro_hevc_motion_vectors_offset(struct hantro_ctx *ctx)
--
2.32.0
Expose 10bit pixel formats to enable 10bit decoding in IMX8M SoCs.
Signed-off-by: Benjamin Gaignard <[email protected]>
Reviewed-by: Ezequiel Garcia <[email protected]>
---
drivers/staging/media/hantro/imx8m_vpu_hw.c | 27 +++++++++++++++++++++
1 file changed, 27 insertions(+)
diff --git a/drivers/staging/media/hantro/imx8m_vpu_hw.c b/drivers/staging/media/hantro/imx8m_vpu_hw.c
index 77f574fdfa77..b390228fd3b4 100644
--- a/drivers/staging/media/hantro/imx8m_vpu_hw.c
+++ b/drivers/staging/media/hantro/imx8m_vpu_hw.c
@@ -162,12 +162,39 @@ static const struct hantro_fmt imx8m_vpu_g2_postproc_fmts[] = {
.step_height = MB_DIM,
},
},
+ {
+ .fourcc = V4L2_PIX_FMT_P010,
+ .codec_mode = HANTRO_MODE_NONE,
+ .postprocessed = true,
+ .frmsize = {
+ .min_width = FMT_MIN_WIDTH,
+ .max_width = FMT_UHD_WIDTH,
+ .step_width = MB_DIM,
+ .min_height = FMT_MIN_HEIGHT,
+ .max_height = FMT_UHD_HEIGHT,
+ .step_height = MB_DIM,
+ },
+ },
};
static const struct hantro_fmt imx8m_vpu_g2_dec_fmts[] = {
{
.fourcc = V4L2_PIX_FMT_NV12_4L4,
.codec_mode = HANTRO_MODE_NONE,
+ .match_depth = true,
+ .frmsize = {
+ .min_width = FMT_MIN_WIDTH,
+ .max_width = FMT_UHD_WIDTH,
+ .step_width = TILE_MB_DIM,
+ .min_height = FMT_MIN_HEIGHT,
+ .max_height = FMT_UHD_HEIGHT,
+ .step_height = TILE_MB_DIM,
+ },
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_P010_4L4,
+ .codec_mode = HANTRO_MODE_NONE,
+ .match_depth = true,
.frmsize = {
.min_width = FMT_MIN_WIDTH,
.max_width = FMT_UHD_WIDTH,
--
2.32.0
Luma and chroma depth are set on different hardware registers.
Even if they aren't identical the bitstream can be compliant
to HEVC specifications and decoded by the hardware.
With this patch TSUNEQBD_A_MAIN10_Technicolor_2 conformance test
is successfully decoded.
Signed-off-by: Benjamin Gaignard <[email protected]>
Reviewed-by: Ezequiel Garcia <[email protected]>
---
drivers/staging/media/hantro/hantro_hevc.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro_hevc.c b/drivers/staging/media/hantro/hantro_hevc.c
index 85688a4df166..26b1329893f7 100644
--- a/drivers/staging/media/hantro/hantro_hevc.c
+++ b/drivers/staging/media/hantro/hantro_hevc.c
@@ -156,9 +156,6 @@ static int tile_buffer_reallocate(struct hantro_ctx *ctx)
int hantro_hevc_validate_sps(struct hantro_ctx *ctx, const struct v4l2_ctrl_hevc_sps *sps)
{
- if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
- /* Luma and chroma bit depth mismatch */
- return -EINVAL;
if (sps->bit_depth_luma_minus8 != 0 && sps->bit_depth_luma_minus8 != 2)
/* Only 8-bit and 10-bit is supported */
return -EINVAL;
--
2.32.0
Stop limiting HEVC support to 8-bits bitstreams also
accept 10-bits bitstreams.
Signed-off-by: Benjamin Gaignard <[email protected]>
Reviewed-by: Ezequiel Garcia <[email protected]>
---
drivers/staging/media/hantro/hantro_hevc.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro_hevc.c b/drivers/staging/media/hantro/hantro_hevc.c
index e06837108a09..85688a4df166 100644
--- a/drivers/staging/media/hantro/hantro_hevc.c
+++ b/drivers/staging/media/hantro/hantro_hevc.c
@@ -159,8 +159,8 @@ int hantro_hevc_validate_sps(struct hantro_ctx *ctx, const struct v4l2_ctrl_hevc
if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
/* Luma and chroma bit depth mismatch */
return -EINVAL;
- if (sps->bit_depth_luma_minus8 != 0)
- /* Only 8-bit is supported */
+ if (sps->bit_depth_luma_minus8 != 0 && sps->bit_depth_luma_minus8 != 2)
+ /* Only 8-bit and 10-bit is supported */
return -EINVAL;
ctx->bit_depth = sps->bit_depth_luma_minus8 + 8;
--
2.32.0
SAO and FILTER buffers size depend of the bit depth.
Make sure we have enough space for 10bit bitstreams.
Signed-off-by: Benjamin Gaignard <[email protected]>
Reviewed-by: Ezequiel Garcia <[email protected]>
---
drivers/staging/media/hantro/hantro_hevc.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro_hevc.c b/drivers/staging/media/hantro/hantro_hevc.c
index dcb5c8703b6e..e06837108a09 100644
--- a/drivers/staging/media/hantro/hantro_hevc.c
+++ b/drivers/staging/media/hantro/hantro_hevc.c
@@ -104,7 +104,7 @@ static int tile_buffer_reallocate(struct hantro_ctx *ctx)
hevc_dec->tile_bsd.cpu = NULL;
}
- size = VERT_FILTER_RAM_SIZE * height64 * (num_tile_cols - 1);
+ size = (VERT_FILTER_RAM_SIZE * height64 * (num_tile_cols - 1) * ctx->bit_depth) / 8;
hevc_dec->tile_filter.cpu = dma_alloc_coherent(vpu->dev, size,
&hevc_dec->tile_filter.dma,
GFP_KERNEL);
@@ -112,7 +112,7 @@ static int tile_buffer_reallocate(struct hantro_ctx *ctx)
goto err_free_tile_buffers;
hevc_dec->tile_filter.size = size;
- size = VERT_SAO_RAM_SIZE * height64 * (num_tile_cols - 1);
+ size = (VERT_SAO_RAM_SIZE * height64 * (num_tile_cols - 1) * ctx->bit_depth) / 8;
hevc_dec->tile_sao.cpu = dma_alloc_coherent(vpu->dev, size,
&hevc_dec->tile_sao.dma,
GFP_KERNEL);
--
2.32.0
Move output format setting in postproc and make sure that
8/10bit configuration is correctly set.
Signed-off-by: Benjamin Gaignard <[email protected]>
Reviewed-by: Ezequiel Garcia <[email protected]>
---
drivers/staging/media/hantro/hantro_g2_hevc_dec.c | 2 --
drivers/staging/media/hantro/hantro_postproc.c | 7 ++++++-
2 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
index a917079a6ed3..a9d4ac84a8d8 100644
--- a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
+++ b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
@@ -167,8 +167,6 @@ static void set_params(struct hantro_ctx *ctx)
hantro_reg_write(vpu, &g2_bit_depth_y_minus8, sps->bit_depth_luma_minus8);
hantro_reg_write(vpu, &g2_bit_depth_c_minus8, sps->bit_depth_chroma_minus8);
- hantro_reg_write(vpu, &g2_output_8_bits, 0);
-
hantro_reg_write(vpu, &g2_hdr_skip_length, compute_header_skip_length(ctx));
min_log2_cb_size = sps->log2_min_luma_coding_block_size_minus3 + 3;
diff --git a/drivers/staging/media/hantro/hantro_postproc.c b/drivers/staging/media/hantro/hantro_postproc.c
index a0928c508434..09d8cf942689 100644
--- a/drivers/staging/media/hantro/hantro_postproc.c
+++ b/drivers/staging/media/hantro/hantro_postproc.c
@@ -114,6 +114,7 @@ static void hantro_postproc_g2_enable(struct hantro_ctx *ctx)
struct hantro_dev *vpu = ctx->dev;
struct vb2_v4l2_buffer *dst_buf;
int down_scale = down_scale_factor(ctx);
+ int out_depth;
size_t chroma_offset;
dma_addr_t dst_dma;
@@ -132,8 +133,9 @@ static void hantro_postproc_g2_enable(struct hantro_ctx *ctx)
hantro_write_addr(vpu, G2_RS_OUT_LUMA_ADDR, dst_dma);
hantro_write_addr(vpu, G2_RS_OUT_CHROMA_ADDR, dst_dma + chroma_offset);
}
+
+ out_depth = hantro_get_format_depth(ctx->dst_fmt.pixelformat);
if (ctx->dev->variant->legacy_regs) {
- int out_depth = hantro_get_format_depth(ctx->dst_fmt.pixelformat);
u8 pp_shift = 0;
if (out_depth > 8)
@@ -141,6 +143,9 @@ static void hantro_postproc_g2_enable(struct hantro_ctx *ctx)
hantro_reg_write(ctx->dev, &g2_rs_out_bit_depth, out_depth);
hantro_reg_write(ctx->dev, &g2_pp_pix_shift, pp_shift);
+ } else {
+ hantro_reg_write(vpu, &g2_output_8_bits, out_depth > 8 ? 0 : 1);
+ hantro_reg_write(vpu, &g2_output_format, out_depth > 8 ? 1 : 0);
}
hantro_reg_write(vpu, &g2_out_rs_e, 1);
}
--
2.32.0
Store HEVC bit depth in context.
Bit depth is equal to hevc sps bit_depth_luma_minus8 + 8.
Signed-off-by: Benjamin Gaignard <[email protected]>
---
drivers/staging/media/hantro/hantro_hevc.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/staging/media/hantro/hantro_hevc.c b/drivers/staging/media/hantro/hantro_hevc.c
index 5984c5fa6f83..dcb5c8703b6e 100644
--- a/drivers/staging/media/hantro/hantro_hevc.c
+++ b/drivers/staging/media/hantro/hantro_hevc.c
@@ -163,6 +163,8 @@ int hantro_hevc_validate_sps(struct hantro_ctx *ctx, const struct v4l2_ctrl_hevc
/* Only 8-bit is supported */
return -EINVAL;
+ ctx->bit_depth = sps->bit_depth_luma_minus8 + 8;
+
/*
* for tile pixel format check if the width and height match
* hardware constraints
--
2.32.0
On 7/18/22 10:39, Benjamin Gaignard wrote:
> Since v1 Jernej patches to support 10 bits decoding have been merged.
> This v2 rebase the previously send patches and add Ezequiel reviewed-by
> tag.
> This series add 10bits support for newer hardware blocks (i.e without
> legacy regs) and for HEVC codec.
>
> It based tested on IMX8MQ with HEVC conformance tests.
> The results are that all 10bits bitstreams are OK.
>
> The problematic point raise by this series is the question of where
> to put HEVC validation function. Does it have to be only hantro_try_ctrl()
> or shall it be done also in .run().
> This series doesn't change that behavoir so I will argue in farvor to
> merge it and to work on this validation time problem when unstaging
> the driver.
> Hans, Ezequiel may that sound acceptable for you ?
Hmm, the hantro driver will be unstaged either this week or (hopefully)
very early on in the next cycle. So I think you should no longer consider
hantro as a staging driver but as a regular mainline driver.
This series definitely won't make the 5.20 cut, so I think it makes more
sense to get this right before merging it.
Regards,
Hans
>
> Regards,
> Benjamin
>
> Benjamin Gaignard (7):
> media: hantro: Store HEVC bit depth in context
> media: hantro: HEVC: Fix auxilary buffer size calculation
> media: hantro: HEVC: Fix chroma offset computation
> media: hantro: postproc: Configure output regs to support 10bit
> media: Hantro: HEVC: Allows 10-bit bitstream
> media: hantro: imx8m: Enable 10bit decoding
> media: hantro: Allows luma and chroma depth to be different
>
> .../staging/media/hantro/hantro_g2_hevc_dec.c | 4 +--
> drivers/staging/media/hantro/hantro_hevc.c | 13 +++++----
> .../staging/media/hantro/hantro_postproc.c | 7 ++++-
> drivers/staging/media/hantro/imx8m_vpu_hw.c | 27 +++++++++++++++++++
> 4 files changed, 40 insertions(+), 11 deletions(-)
>