2023-11-05 16:56:30

by Jonas Karlman

[permalink] [raw]
Subject: [PATCH v4 00/11] media: rkvdec: Add H.264 High 10 and 4:2:2 profile support

This is a revival of a 3 year old series [1] now that NV15/NV20/NV30 support
for display driver have landed in mainline tree.

This series adds H.264 High 10 and 4:2:2 profile support to the Rockchip
Video Decoder driver.

Patch 1 adds helpers for calculating plane bytesperline and sizeimage.
Patch 2 adds two new pixelformats for semi-planer 10-bit 4:2:0/4:2:2 YUV.

Patch 3 change to use bytesperline and buffer height to configure strides.
Patch 4 change to use values from SPS/PPS control to configure the HW.
Patch 5 remove an unnecessary call to validate sps at streaming start.

Patch 6-10 refactor code to support filtering of CAPUTRE formats based
on the image format returned from a get_image_fmt ops.

Patch 11 adds final bits to support H.264 High 10 and 4:2:2 profiles.

Tested on a ROCK Pi 4 (RK3399) and Rock64 (RK3328):

v4l2-compliance 1.24.1, 64 bits, 64-bit time_t
...
Total for rkvdec device /dev/video1: 46, Succeeded: 46, Failed: 0, Warnings: 0

Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request
...
Ran 65/69 tests successfully

Running test suite JVT-AVC_V1 with decoder FFmpeg-H.264-V4L2-request
...
Ran 127/135 tests successfully

Before this series:

Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request
...
Ran 44/69 tests successfully

Changes in v4:
- Fix failed v4l2-compliance tests related to CAPTURE queue
- Rework CAPTURE format filter anv validate to use an image format
- Run fluster test suite JVT-FR-EXT [4] and JVT-AVC_V1 [5]
Link to v3: https://lore.kernel.org/linux-media/[email protected]/

Changes in v3:
- Drop merged patches
- Use bpp and bpp_div instead of prior misuse of block_w/block_h
- New patch to use values from SPS/PPS control to configure the HW
- New patch to remove an unnecessary call to validate sps at streaming start
- Reworked pixel format validation
Link to v2: https://lore.kernel.org/linux-media/[email protected]/

Changes in v2:
- Collect r-b tags
- SPS pic width and height in mbs validation moved to rkvdec_try_ctrl
- New patch to not override output buffer sizeimage
- Reworked pixel format validation
- Only align decoded buffer instead of changing frmsize step_width
Link to v1: https://lore.kernel.org/linux-media/[email protected]/

Following commits adds support for NV15/NV20/NV30 to VOP driver:
728c15b4b5f3 ("drm/fourcc: Add NV20 and NV30 YUV formats")
d4b384228562 ("drm/rockchip: vop: Add NV15, NV20 and NV30 support")

To fully runtime test this series you may need above drm commits and ffmpeg
patches from [2], this series and drm patches is also available at [3].

[1] https://lore.kernel.org/linux-media/[email protected]/
[2] https://github.com/Kwiboo/FFmpeg/commits/v4l2-request-n6.1-dev/
[3] https://github.com/Kwiboo/linux-rockchip/commits/linuxtv-rkvdec-high-10-v4/
[4] https://gist.github.com/Kwiboo/f4ac15576b2c72887ae2bc5d58b5c865
[5] https://gist.github.com/Kwiboo/459a1c8f1dcb56e45dc7a7a29cc28adf

Regards,
Jonas

Alex Bee (1):
media: rkvdec: h264: Don't hardcode SPS/PPS parameters

Jonas Karlman (10):
media: v4l2-common: Add helpers to calculate bytesperline and
sizeimage
media: v4l2: Add NV15 and NV20 pixel formats
media: rkvdec: h264: Use bytesperline and buffer height as virstride
media: rkvdec: h264: Remove SPS validation at streaming start
media: rkvdec: Extract rkvdec_fill_decoded_pixfmt into helper
media: rkvdec: Move rkvdec_reset_decoded_fmt helper
media: rkvdec: Extract decoded format enumeration into helper
media: rkvdec: Add image format concept
media: rkvdec: Add get_image_fmt ops
media: rkvdec: h264: Support High 10 and 4:2:2 profiles

.../media/v4l/pixfmt-yuv-planar.rst | 128 +++++++++++
drivers/media/v4l2-core/v4l2-common.c | 80 +++----
drivers/media/v4l2-core/v4l2-ioctl.c | 2 +
drivers/staging/media/rkvdec/rkvdec-h264.c | 83 +++----
drivers/staging/media/rkvdec/rkvdec.c | 217 +++++++++++++-----
drivers/staging/media/rkvdec/rkvdec.h | 18 +-
include/uapi/linux/videodev2.h | 2 +
7 files changed, 396 insertions(+), 134 deletions(-)

--
2.42.0


2023-11-05 16:56:50

by Jonas Karlman

[permalink] [raw]
Subject: [PATCH v4 10/11] media: rkvdec: Add get_image_fmt ops

Add support for a get_image_fmt() ops that return the required image
format.

The CAPTURE format is reset when required image format changes and the
buffer queue is not busy.

Signed-off-by: Jonas Karlman <[email protected]>
---
v4:
- Change fmt_opaque into an image format
- Split patch into two

v3:
- New patch

drivers/staging/media/rkvdec/rkvdec.c | 49 +++++++++++++++++++++++++--
drivers/staging/media/rkvdec/rkvdec.h | 2 ++
2 files changed, 49 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index 5949d59d4cf9..225aa1f0ac48 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -108,15 +108,60 @@ static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl)
{
struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl);
const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
+ struct v4l2_pix_format_mplane *pix_mp = &ctx->decoded_fmt.fmt.pix_mp;
+ enum rkvdec_image_fmt image_fmt;
+ struct vb2_queue *vq;
+ int ret;
+
+ if (desc->ops->try_ctrl) {
+ ret = desc->ops->try_ctrl(ctx, ctrl);
+ if (ret)
+ return ret;
+ }
+
+ if (!desc->ops->get_image_fmt)
+ return 0;

- if (desc->ops->try_ctrl)
- return desc->ops->try_ctrl(ctx, ctrl);
+ image_fmt = desc->ops->get_image_fmt(ctx, ctrl);
+ if (ctx->image_fmt == image_fmt)
+ return 0;
+
+ if (rkvdec_is_valid_fmt(ctx, pix_mp->pixelformat, image_fmt))
+ return 0;
+
+ /* format change not allowed when queue is busy */
+ vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx,
+ V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+ if (vb2_is_busy(vq))
+ return -EINVAL;
+
+ return 0;
+}
+
+static int rkvdec_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl);
+ const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
+ struct v4l2_pix_format_mplane *pix_mp = &ctx->decoded_fmt.fmt.pix_mp;
+ enum rkvdec_image_fmt image_fmt;
+
+ if (!desc->ops->get_image_fmt)
+ return 0;
+
+ image_fmt = desc->ops->get_image_fmt(ctx, ctrl);
+ if (ctx->image_fmt == image_fmt)
+ return 0;
+
+ ctx->image_fmt = image_fmt;
+ if (!rkvdec_is_valid_fmt(ctx, pix_mp->pixelformat, ctx->image_fmt))
+ rkvdec_reset_decoded_fmt(ctx);

return 0;
}

static const struct v4l2_ctrl_ops rkvdec_ctrl_ops = {
.try_ctrl = rkvdec_try_ctrl,
+ .s_ctrl = rkvdec_s_ctrl,
};

static const struct rkvdec_ctrl_desc rkvdec_h264_ctrl_descs[] = {
diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h
index 6f8cf50c5d99..e466a2753ccf 100644
--- a/drivers/staging/media/rkvdec/rkvdec.h
+++ b/drivers/staging/media/rkvdec/rkvdec.h
@@ -73,6 +73,8 @@ struct rkvdec_coded_fmt_ops {
struct vb2_v4l2_buffer *dst_buf,
enum vb2_buffer_state result);
int (*try_ctrl)(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl);
+ enum rkvdec_image_fmt (*get_image_fmt)(struct rkvdec_ctx *ctx,
+ struct v4l2_ctrl *ctrl);
};

enum rkvdec_image_fmt {
--
2.42.0

2023-11-05 17:05:49

by Jonas Karlman

[permalink] [raw]
Subject: [PATCH v4 01/11] media: v4l2-common: Add helpers to calculate bytesperline and sizeimage

Add helper functions to calculate plane bytesperline and sizeimage,
these new helpers consider bpp div, block width and height when
calculating plane bytesperline and sizeimage.

Signed-off-by: Jonas Karlman <[email protected]>
---
v4:
- No change

v3:
- Consider bpp_div in calculation

drivers/media/v4l2-core/v4l2-common.c | 78 +++++++++++++--------------
1 file changed, 39 insertions(+), 39 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
index 3a4b15a98e02..834b426da8b1 100644
--- a/drivers/media/v4l2-core/v4l2-common.c
+++ b/drivers/media/v4l2-core/v4l2-common.c
@@ -350,6 +350,34 @@ static inline unsigned int v4l2_format_block_height(const struct v4l2_format_inf
return info->block_h[plane];
}

+static inline unsigned int v4l2_format_plane_width(const struct v4l2_format_info *info, int plane,
+ unsigned int width)
+{
+ unsigned int hdiv = plane ? info->hdiv : 1;
+ unsigned int aligned_width =
+ ALIGN(width, v4l2_format_block_width(info, plane));
+
+ return DIV_ROUND_UP(aligned_width, hdiv) *
+ info->bpp[plane] / info->bpp_div[plane];
+}
+
+static inline unsigned int v4l2_format_plane_height(const struct v4l2_format_info *info, int plane,
+ unsigned int height)
+{
+ unsigned int vdiv = plane ? info->vdiv : 1;
+ unsigned int aligned_height =
+ ALIGN(height, v4l2_format_block_height(info, plane));
+
+ return DIV_ROUND_UP(aligned_height, vdiv);
+}
+
+static inline unsigned int v4l2_format_plane_size(const struct v4l2_format_info *info, int plane,
+ unsigned int width, unsigned int height)
+{
+ return v4l2_format_plane_width(info, plane, width) *
+ v4l2_format_plane_height(info, plane, height);
+}
+
void v4l2_apply_frmsize_constraints(u32 *width, u32 *height,
const struct v4l2_frmsize_stepwise *frmsize)
{
@@ -385,37 +413,19 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,

if (info->mem_planes == 1) {
plane = &pixfmt->plane_fmt[0];
- plane->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0] / info->bpp_div[0];
+ plane->bytesperline = v4l2_format_plane_width(info, 0, width);
plane->sizeimage = 0;

- for (i = 0; i < info->comp_planes; i++) {
- unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
- unsigned int vdiv = (i == 0) ? 1 : info->vdiv;
- unsigned int aligned_width;
- unsigned int aligned_height;
-
- aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
- aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
-
- plane->sizeimage += info->bpp[i] *
- DIV_ROUND_UP(aligned_width, hdiv) *
- DIV_ROUND_UP(aligned_height, vdiv) / info->bpp_div[i];
- }
+ for (i = 0; i < info->comp_planes; i++)
+ plane->sizeimage +=
+ v4l2_format_plane_size(info, i, width, height);
} else {
for (i = 0; i < info->comp_planes; i++) {
- unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
- unsigned int vdiv = (i == 0) ? 1 : info->vdiv;
- unsigned int aligned_width;
- unsigned int aligned_height;
-
- aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
- aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
-
plane = &pixfmt->plane_fmt[i];
plane->bytesperline =
- info->bpp[i] * DIV_ROUND_UP(aligned_width, hdiv) / info->bpp_div[i];
- plane->sizeimage =
- plane->bytesperline * DIV_ROUND_UP(aligned_height, vdiv);
+ v4l2_format_plane_width(info, i, width);
+ plane->sizeimage = plane->bytesperline *
+ v4l2_format_plane_height(info, i, height);
}
}
return 0;
@@ -439,22 +449,12 @@ int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, u32 pixelformat,
pixfmt->width = width;
pixfmt->height = height;
pixfmt->pixelformat = pixelformat;
- pixfmt->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0] / info->bpp_div[0];
+ pixfmt->bytesperline = v4l2_format_plane_width(info, 0, width);
pixfmt->sizeimage = 0;

- for (i = 0; i < info->comp_planes; i++) {
- unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
- unsigned int vdiv = (i == 0) ? 1 : info->vdiv;
- unsigned int aligned_width;
- unsigned int aligned_height;
-
- aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
- aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
-
- pixfmt->sizeimage += info->bpp[i] *
- DIV_ROUND_UP(aligned_width, hdiv) *
- DIV_ROUND_UP(aligned_height, vdiv) / info->bpp_div[i];
- }
+ for (i = 0; i < info->comp_planes; i++)
+ pixfmt->sizeimage +=
+ v4l2_format_plane_size(info, i, width, height);
return 0;
}
EXPORT_SYMBOL_GPL(v4l2_fill_pixfmt);
--
2.42.0

2023-11-05 17:07:16

by Jonas Karlman

[permalink] [raw]
Subject: [PATCH v4 07/11] media: rkvdec: Move rkvdec_reset_decoded_fmt helper

Move rkvdec_reset_decoded_fmt() and the called rkvdec_reset_fmt() helper
functions in preparation for adding a new caller in an upcoming patch.

Signed-off-by: Jonas Karlman <[email protected]>
---
v4:
- No change

v3:
- New patch

drivers/staging/media/rkvdec/rkvdec.c | 46 +++++++++++++--------------
1 file changed, 23 insertions(+), 23 deletions(-)

diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index 0570c790ad08..7a79840470e1 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -37,6 +37,29 @@ static void rkvdec_fill_decoded_pixfmt(struct rkvdec_ctx *ctx,
DIV_ROUND_UP(pix_mp->height, 16);
}

+static void rkvdec_reset_fmt(struct rkvdec_ctx *ctx, struct v4l2_format *f,
+ u32 fourcc)
+{
+ memset(f, 0, sizeof(*f));
+ f->fmt.pix_mp.pixelformat = fourcc;
+ f->fmt.pix_mp.field = V4L2_FIELD_NONE;
+ f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_REC709;
+ f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+ f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT;
+}
+
+static void rkvdec_reset_decoded_fmt(struct rkvdec_ctx *ctx)
+{
+ struct v4l2_format *f = &ctx->decoded_fmt;
+
+ rkvdec_reset_fmt(ctx, f, ctx->coded_fmt_desc->decoded_fmts[0]);
+ f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+ f->fmt.pix_mp.width = ctx->coded_fmt.fmt.pix_mp.width;
+ f->fmt.pix_mp.height = ctx->coded_fmt.fmt.pix_mp.height;
+ rkvdec_fill_decoded_pixfmt(ctx, &f->fmt.pix_mp);
+}
+
static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl)
{
struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl);
@@ -169,18 +192,6 @@ rkvdec_find_coded_fmt_desc(u32 fourcc)
return NULL;
}

-static void rkvdec_reset_fmt(struct rkvdec_ctx *ctx, struct v4l2_format *f,
- u32 fourcc)
-{
- memset(f, 0, sizeof(*f));
- f->fmt.pix_mp.pixelformat = fourcc;
- f->fmt.pix_mp.field = V4L2_FIELD_NONE;
- f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_REC709;
- f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
- f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
- f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT;
-}
-
static void rkvdec_reset_coded_fmt(struct rkvdec_ctx *ctx)
{
struct v4l2_format *f = &ctx->coded_fmt;
@@ -196,17 +207,6 @@ static void rkvdec_reset_coded_fmt(struct rkvdec_ctx *ctx)
ctx->coded_fmt_desc->ops->adjust_fmt(ctx, f);
}

-static void rkvdec_reset_decoded_fmt(struct rkvdec_ctx *ctx)
-{
- struct v4l2_format *f = &ctx->decoded_fmt;
-
- rkvdec_reset_fmt(ctx, f, ctx->coded_fmt_desc->decoded_fmts[0]);
- f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
- f->fmt.pix_mp.width = ctx->coded_fmt.fmt.pix_mp.width;
- f->fmt.pix_mp.height = ctx->coded_fmt.fmt.pix_mp.height;
- rkvdec_fill_decoded_pixfmt(ctx, &f->fmt.pix_mp);
-}
-
static int rkvdec_enum_framesizes(struct file *file, void *priv,
struct v4l2_frmsizeenum *fsize)
{
--
2.42.0

2023-11-05 17:07:22

by Jonas Karlman

[permalink] [raw]
Subject: [PATCH v4 09/11] media: rkvdec: Add image format concept

Add an enum rkvdec_image_fmt used to signal an image format, e.g.
4:2:0 8-bit, 4:2:0 10-bit or any.

Tag each supported CAPUTRE format with an image format and use this tag
to filter out unsupported CAPTURE formats.

Signed-off-by: Jonas Karlman <[email protected]>
---
v4:
- Change fmt_opaque into an image format
- Split patch into two

v3:
- New patch

drivers/staging/media/rkvdec/rkvdec.c | 45 ++++++++++++++++++++-------
drivers/staging/media/rkvdec/rkvdec.h | 13 +++++++-
2 files changed, 45 insertions(+), 13 deletions(-)

diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index c3aede94c872..5949d59d4cf9 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -27,26 +27,42 @@
#include "rkvdec.h"
#include "rkvdec-regs.h"

-static u32 rkvdec_enum_decoded_fmt(struct rkvdec_ctx *ctx, int index)
+static inline bool rkvdec_image_fmt_match(enum rkvdec_image_fmt fmt1,
+ enum rkvdec_image_fmt fmt2)
+{
+ return fmt1 == fmt2 || fmt2 == RKVDEC_IMG_FMT_ANY ||
+ fmt1 == RKVDEC_IMG_FMT_ANY;
+}
+
+static u32 rkvdec_enum_decoded_fmt(struct rkvdec_ctx *ctx, int index,
+ enum rkvdec_image_fmt image_fmt)
{
const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
+ unsigned int i, j;

if (WARN_ON(!desc))
return 0;

- if (index >= desc->num_decoded_fmts)
- return 0;
+ for (i = 0, j = 0; i < desc->num_decoded_fmts; i++) {
+ if (rkvdec_image_fmt_match(desc->decoded_fmts[i].image_fmt,
+ image_fmt) &&
+ index == j++)
+ return desc->decoded_fmts[i].fourcc;
+ }

- return desc->decoded_fmts[index];
+ return 0;
}

-static bool rkvdec_is_valid_fmt(struct rkvdec_ctx *ctx, u32 fourcc)
+static bool rkvdec_is_valid_fmt(struct rkvdec_ctx *ctx, u32 fourcc,
+ enum rkvdec_image_fmt image_fmt)
{
const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
unsigned int i;

for (i = 0; i < desc->num_decoded_fmts; i++) {
- if (desc->decoded_fmts[i] == fourcc)
+ if (rkvdec_image_fmt_match(desc->decoded_fmts[i].image_fmt,
+ image_fmt) &&
+ desc->decoded_fmts[i].fourcc == fourcc)
return true;
}

@@ -80,7 +96,7 @@ static void rkvdec_reset_decoded_fmt(struct rkvdec_ctx *ctx)
struct v4l2_format *f = &ctx->decoded_fmt;
u32 fourcc;

- fourcc = rkvdec_enum_decoded_fmt(ctx, 0);
+ fourcc = rkvdec_enum_decoded_fmt(ctx, 0, ctx->image_fmt);
rkvdec_reset_fmt(ctx, f, fourcc);
f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
f->fmt.pix_mp.width = ctx->coded_fmt.fmt.pix_mp.width;
@@ -149,8 +165,11 @@ static const struct rkvdec_ctrls rkvdec_h264_ctrls = {
.num_ctrls = ARRAY_SIZE(rkvdec_h264_ctrl_descs),
};

-static const u32 rkvdec_h264_vp9_decoded_fmts[] = {
- V4L2_PIX_FMT_NV12,
+static const struct rkvdec_decoded_fmt_desc rkvdec_h264_vp9_decoded_fmts[] = {
+ {
+ .fourcc = V4L2_PIX_FMT_NV12,
+ .image_fmt = RKVDEC_IMG_FMT_420_8BIT,
+ },
};

static const struct rkvdec_ctrl_desc rkvdec_vp9_ctrl_descs[] = {
@@ -282,8 +301,9 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv,
if (WARN_ON(!coded_desc))
return -EINVAL;

- if (!rkvdec_is_valid_fmt(ctx, pix_mp->pixelformat))
- pix_mp->pixelformat = rkvdec_enum_decoded_fmt(ctx, 0);
+ if (!rkvdec_is_valid_fmt(ctx, pix_mp->pixelformat, ctx->image_fmt))
+ pix_mp->pixelformat = rkvdec_enum_decoded_fmt(ctx, 0,
+ ctx->image_fmt);

/* Always apply the frmsize constraint of the coded end. */
pix_mp->width = max(pix_mp->width, ctx->coded_fmt.fmt.pix_mp.width);
@@ -400,6 +420,7 @@ static int rkvdec_s_output_fmt(struct file *file, void *priv,
*
* Note that this will propagates any size changes to the decoded format.
*/
+ ctx->image_fmt = RKVDEC_IMG_FMT_ANY;
rkvdec_reset_decoded_fmt(ctx);

/* Propagate colorspace information to capture. */
@@ -449,7 +470,7 @@ static int rkvdec_enum_capture_fmt(struct file *file, void *priv,
struct rkvdec_ctx *ctx = fh_to_rkvdec_ctx(priv);
u32 fourcc;

- fourcc = rkvdec_enum_decoded_fmt(ctx, f->index);
+ fourcc = rkvdec_enum_decoded_fmt(ctx, f->index, ctx->image_fmt);
if (!fourcc)
return -EINVAL;

diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h
index 633335ebb9c4..6f8cf50c5d99 100644
--- a/drivers/staging/media/rkvdec/rkvdec.h
+++ b/drivers/staging/media/rkvdec/rkvdec.h
@@ -75,13 +75,23 @@ struct rkvdec_coded_fmt_ops {
int (*try_ctrl)(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl);
};

+enum rkvdec_image_fmt {
+ RKVDEC_IMG_FMT_ANY = 0,
+ RKVDEC_IMG_FMT_420_8BIT,
+};
+
+struct rkvdec_decoded_fmt_desc {
+ u32 fourcc;
+ enum rkvdec_image_fmt image_fmt;
+};
+
struct rkvdec_coded_fmt_desc {
u32 fourcc;
struct v4l2_frmsize_stepwise frmsize;
const struct rkvdec_ctrls *ctrls;
const struct rkvdec_coded_fmt_ops *ops;
unsigned int num_decoded_fmts;
- const u32 *decoded_fmts;
+ const struct rkvdec_decoded_fmt_desc *decoded_fmts;
u32 subsystem_flags;
};

@@ -104,6 +114,7 @@ struct rkvdec_ctx {
const struct rkvdec_coded_fmt_desc *coded_fmt_desc;
struct v4l2_ctrl_handler ctrl_hdl;
struct rkvdec_dev *dev;
+ enum rkvdec_image_fmt image_fmt;
void *priv;
};

--
2.42.0

2023-11-05 17:07:28

by Jonas Karlman

[permalink] [raw]
Subject: [PATCH v4 11/11] media: rkvdec: h264: Support High 10 and 4:2:2 profiles

Add support and enable decoding of H264 High 10 and 4:2:2 profiles.

Decoded CAPTURE buffer width is aligned to 64 pixels to accommodate HW
requirement of 10-bit format buffers, fixes decoding of:

- Hi422FR13_SONY_A
- Hi422FR14_SONY_A
- Hi422FR15_SONY_A
- Hi422FR6_SONY_A
- Hi422FR7_SONY_A
- Hi422FR8_SONY_A
- Hi422FR9_SONY_A
- Hi422FREXT18_SONY_A

The get_image_fmt() ops is implemented to select an image format
required for the provided SPS control.

Signed-off-by: Jonas Karlman <[email protected]>
---
v4:
- Change to use get_image_fmt() ops

v3:
- Add get_fmt_opaque ops, the expected pixelformat is used as opaque
- Add new valid_fmt ops that validate pixelformat matches opaque
- Update H264_PROFILE control max value

drivers/staging/media/rkvdec/rkvdec-h264.c | 37 ++++++++++++++++------
drivers/staging/media/rkvdec/rkvdec.c | 33 +++++++++++++++----
drivers/staging/media/rkvdec/rkvdec.h | 3 ++
3 files changed, 57 insertions(+), 16 deletions(-)

diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index 815d5359ddd5..baac6d012ddd 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -1027,24 +1027,42 @@ static int rkvdec_h264_adjust_fmt(struct rkvdec_ctx *ctx,
return 0;
}

+static enum rkvdec_image_fmt rkvdec_h264_get_image_fmt(struct rkvdec_ctx *ctx,
+ struct v4l2_ctrl *ctrl)
+{
+ const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
+
+ if (ctrl->id != V4L2_CID_STATELESS_H264_SPS)
+ return RKVDEC_IMG_FMT_ANY;
+
+ if (sps->bit_depth_luma_minus8 == 0) {
+ if (sps->chroma_format_idc == 2)
+ return RKVDEC_IMG_FMT_422_8BIT;
+ else
+ return RKVDEC_IMG_FMT_420_8BIT;
+ } else if (sps->bit_depth_luma_minus8 == 2) {
+ if (sps->chroma_format_idc == 2)
+ return RKVDEC_IMG_FMT_422_10BIT;
+ else
+ return RKVDEC_IMG_FMT_420_10BIT;
+ }
+
+ return RKVDEC_IMG_FMT_ANY;
+}
+
static int rkvdec_h264_validate_sps(struct rkvdec_ctx *ctx,
const struct v4l2_ctrl_h264_sps *sps)
{
unsigned int width, height;

- /*
- * TODO: The hardware supports 10-bit and 4:2:2 profiles,
- * but it's currently broken in the driver.
- * Reject them for now, until it's fixed.
- */
- if (sps->chroma_format_idc > 1)
- /* Only 4:0:0 and 4:2:0 are supported */
+ if (sps->chroma_format_idc > 2)
+ /* Only 4:0:0, 4:2:0 and 4:2:2 are supported */
return -EINVAL;
if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
/* Luma and chroma bit depth mismatch */
return -EINVAL;
- if (sps->bit_depth_luma_minus8 != 0)
- /* Only 8-bit is supported */
+ if (sps->bit_depth_luma_minus8 != 0 && sps->bit_depth_luma_minus8 != 2)
+ /* Only 8-bit and 10-bit is supported */
return -EINVAL;

width = (sps->pic_width_in_mbs_minus1 + 1) * 16;
@@ -1175,4 +1193,5 @@ const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops = {
.stop = rkvdec_h264_stop,
.run = rkvdec_h264_run,
.try_ctrl = rkvdec_h264_try_ctrl,
+ .get_image_fmt = rkvdec_h264_get_image_fmt,
};
diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index 225aa1f0ac48..eb59605ccf28 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -73,7 +73,7 @@ static void rkvdec_fill_decoded_pixfmt(struct rkvdec_ctx *ctx,
struct v4l2_pix_format_mplane *pix_mp)
{
v4l2_fill_pixfmt_mp(pix_mp, pix_mp->pixelformat,
- pix_mp->width, pix_mp->height);
+ ALIGN(pix_mp->width, 64), pix_mp->height);
pix_mp->plane_fmt[0].sizeimage += 128 *
DIV_ROUND_UP(pix_mp->width, 16) *
DIV_ROUND_UP(pix_mp->height, 16);
@@ -193,7 +193,7 @@ static const struct rkvdec_ctrl_desc rkvdec_h264_ctrl_descs[] = {
{
.cfg.id = V4L2_CID_MPEG_VIDEO_H264_PROFILE,
.cfg.min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE,
- .cfg.max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
+ .cfg.max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_422,
.cfg.menu_skip_mask =
BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED),
.cfg.def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN,
@@ -210,11 +210,23 @@ static const struct rkvdec_ctrls rkvdec_h264_ctrls = {
.num_ctrls = ARRAY_SIZE(rkvdec_h264_ctrl_descs),
};

-static const struct rkvdec_decoded_fmt_desc rkvdec_h264_vp9_decoded_fmts[] = {
+static const struct rkvdec_decoded_fmt_desc rkvdec_h264_decoded_fmts[] = {
{
.fourcc = V4L2_PIX_FMT_NV12,
.image_fmt = RKVDEC_IMG_FMT_420_8BIT,
},
+ {
+ .fourcc = V4L2_PIX_FMT_NV15,
+ .image_fmt = RKVDEC_IMG_FMT_420_10BIT,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_NV16,
+ .image_fmt = RKVDEC_IMG_FMT_422_8BIT,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_NV20,
+ .image_fmt = RKVDEC_IMG_FMT_422_10BIT,
+ },
};

static const struct rkvdec_ctrl_desc rkvdec_vp9_ctrl_descs[] = {
@@ -237,6 +249,13 @@ static const struct rkvdec_ctrls rkvdec_vp9_ctrls = {
.num_ctrls = ARRAY_SIZE(rkvdec_vp9_ctrl_descs),
};

+static const struct rkvdec_decoded_fmt_desc rkvdec_vp9_decoded_fmts[] = {
+ {
+ .fourcc = V4L2_PIX_FMT_NV12,
+ .image_fmt = RKVDEC_IMG_FMT_420_8BIT,
+ },
+};
+
static const struct rkvdec_coded_fmt_desc rkvdec_coded_fmts[] = {
{
.fourcc = V4L2_PIX_FMT_H264_SLICE,
@@ -250,8 +269,8 @@ static const struct rkvdec_coded_fmt_desc rkvdec_coded_fmts[] = {
},
.ctrls = &rkvdec_h264_ctrls,
.ops = &rkvdec_h264_fmt_ops,
- .num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_vp9_decoded_fmts),
- .decoded_fmts = rkvdec_h264_vp9_decoded_fmts,
+ .num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_decoded_fmts),
+ .decoded_fmts = rkvdec_h264_decoded_fmts,
.subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF,
},
{
@@ -266,8 +285,8 @@ static const struct rkvdec_coded_fmt_desc rkvdec_coded_fmts[] = {
},
.ctrls = &rkvdec_vp9_ctrls,
.ops = &rkvdec_vp9_fmt_ops,
- .num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_vp9_decoded_fmts),
- .decoded_fmts = rkvdec_h264_vp9_decoded_fmts,
+ .num_decoded_fmts = ARRAY_SIZE(rkvdec_vp9_decoded_fmts),
+ .decoded_fmts = rkvdec_vp9_decoded_fmts,
}
};

diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h
index e466a2753ccf..9a9f4fced7a1 100644
--- a/drivers/staging/media/rkvdec/rkvdec.h
+++ b/drivers/staging/media/rkvdec/rkvdec.h
@@ -80,6 +80,9 @@ struct rkvdec_coded_fmt_ops {
enum rkvdec_image_fmt {
RKVDEC_IMG_FMT_ANY = 0,
RKVDEC_IMG_FMT_420_8BIT,
+ RKVDEC_IMG_FMT_420_10BIT,
+ RKVDEC_IMG_FMT_422_8BIT,
+ RKVDEC_IMG_FMT_422_10BIT,
};

struct rkvdec_decoded_fmt_desc {
--
2.42.0

2023-11-05 17:07:42

by Jonas Karlman

[permalink] [raw]
Subject: [PATCH v4 08/11] media: rkvdec: Extract decoded format enumeration into helper

Add a rkvdec_is_valid_fmt() helper that check if a fourcc is a supported
CAPTURE format, and a rkvdec_enum_decoded_fmt() helper that enumerate
valid formats.

This move current code into helper functions in preparation for adding
CAPTURE format filtering and validation in next patch.

Signed-off-by: Jonas Karlman <[email protected]>
---
v4:
- Rename rkvdec_decoded_fmts() to rkvdec_enum_decoded_fmt()
- Rename rkvdec_valid_fmt() to rkvdec_is_valid_fmt()

v3:
- New patch

drivers/staging/media/rkvdec/rkvdec.c | 49 +++++++++++++++++++--------
1 file changed, 35 insertions(+), 14 deletions(-)

diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index 7a79840470e1..c3aede94c872 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -27,6 +27,32 @@
#include "rkvdec.h"
#include "rkvdec-regs.h"

+static u32 rkvdec_enum_decoded_fmt(struct rkvdec_ctx *ctx, int index)
+{
+ const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
+
+ if (WARN_ON(!desc))
+ return 0;
+
+ if (index >= desc->num_decoded_fmts)
+ return 0;
+
+ return desc->decoded_fmts[index];
+}
+
+static bool rkvdec_is_valid_fmt(struct rkvdec_ctx *ctx, u32 fourcc)
+{
+ const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
+ unsigned int i;
+
+ for (i = 0; i < desc->num_decoded_fmts; i++) {
+ if (desc->decoded_fmts[i] == fourcc)
+ return true;
+ }
+
+ return false;
+}
+
static void rkvdec_fill_decoded_pixfmt(struct rkvdec_ctx *ctx,
struct v4l2_pix_format_mplane *pix_mp)
{
@@ -52,8 +78,10 @@ static void rkvdec_reset_fmt(struct rkvdec_ctx *ctx, struct v4l2_format *f,
static void rkvdec_reset_decoded_fmt(struct rkvdec_ctx *ctx)
{
struct v4l2_format *f = &ctx->decoded_fmt;
+ u32 fourcc;

- rkvdec_reset_fmt(ctx, f, ctx->coded_fmt_desc->decoded_fmts[0]);
+ fourcc = rkvdec_enum_decoded_fmt(ctx, 0);
+ rkvdec_reset_fmt(ctx, f, fourcc);
f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
f->fmt.pix_mp.width = ctx->coded_fmt.fmt.pix_mp.width;
f->fmt.pix_mp.height = ctx->coded_fmt.fmt.pix_mp.height;
@@ -244,7 +272,6 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv,
struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
struct rkvdec_ctx *ctx = fh_to_rkvdec_ctx(priv);
const struct rkvdec_coded_fmt_desc *coded_desc;
- unsigned int i;

/*
* The codec context should point to a coded format desc, if the format
@@ -255,13 +282,8 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv,
if (WARN_ON(!coded_desc))
return -EINVAL;

- for (i = 0; i < coded_desc->num_decoded_fmts; i++) {
- if (coded_desc->decoded_fmts[i] == pix_mp->pixelformat)
- break;
- }
-
- if (i == coded_desc->num_decoded_fmts)
- pix_mp->pixelformat = coded_desc->decoded_fmts[0];
+ if (!rkvdec_is_valid_fmt(ctx, pix_mp->pixelformat))
+ pix_mp->pixelformat = rkvdec_enum_decoded_fmt(ctx, 0);

/* Always apply the frmsize constraint of the coded end. */
pix_mp->width = max(pix_mp->width, ctx->coded_fmt.fmt.pix_mp.width);
@@ -425,14 +447,13 @@ static int rkvdec_enum_capture_fmt(struct file *file, void *priv,
struct v4l2_fmtdesc *f)
{
struct rkvdec_ctx *ctx = fh_to_rkvdec_ctx(priv);
+ u32 fourcc;

- if (WARN_ON(!ctx->coded_fmt_desc))
- return -EINVAL;
-
- if (f->index >= ctx->coded_fmt_desc->num_decoded_fmts)
+ fourcc = rkvdec_enum_decoded_fmt(ctx, f->index);
+ if (!fourcc)
return -EINVAL;

- f->pixelformat = ctx->coded_fmt_desc->decoded_fmts[f->index];
+ f->pixelformat = fourcc;
return 0;
}

--
2.42.0

2024-01-08 09:55:29

by Christopher Obbard

[permalink] [raw]
Subject: Re: [PATCH v4 00/11] media: rkvdec: Add H.264 High 10 and 4:2:2 profile support

Hi Jonas,

Happy new year!

On Sun, 2023-11-05 at 16:54 +0000, Jonas Karlman wrote:
> This is a revival of a 3 year old series [1] now that NV15/NV20/NV30 support
> for display driver have landed in mainline tree.
>
> This series adds H.264 High 10 and 4:2:2 profile support to the Rockchip
> Video Decoder driver.
>
> Patch 1 adds helpers for calculating plane bytesperline and sizeimage.
> Patch 2 adds two new pixelformats for semi-planer 10-bit 4:2:0/4:2:2 YUV.
>
> Patch 3 change to use bytesperline and buffer height to configure strides.
> Patch 4 change to use values from SPS/PPS control to configure the HW.
> Patch 5 remove an unnecessary call to validate sps at streaming start.
>
> Patch 6-10 refactor code to support filtering of CAPUTRE formats based
> on the image format returned from a get_image_fmt ops.
>
> Patch 11 adds final bits to support H.264 High 10 and 4:2:2 profiles.

I send my Tested-by to v3 of this series, I also tested with v4 so for future
series please add:

Tested-by: Christopher Obbard <[email protected]>

>
> Tested on a ROCK Pi 4 (RK3399) and Rock64 (RK3328):
>
>   v4l2-compliance 1.24.1, 64 bits, 64-bit time_t
>   ...
>   Total for rkvdec device /dev/video1: 46, Succeeded: 46, Failed: 0,
> Warnings: 0
>
>   Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request
>   ...
>   Ran 65/69 tests successfully
>
>   Running test suite JVT-AVC_V1 with decoder FFmpeg-H.264-V4L2-request
>   ...
>   Ran 127/135 tests successfully
>
> Before this series:
>
>   Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request
>   ...
>   Ran 44/69 tests successfully
>
> Changes in v4:
> - Fix failed v4l2-compliance tests related to CAPTURE queue
> - Rework CAPTURE format filter anv validate to use an image format
> - Run fluster test suite JVT-FR-EXT [4] and JVT-AVC_V1 [5]
> Link to v3:
> https://lore.kernel.org/linux-media/[email protected]/
>
> Changes in v3:
> - Drop merged patches
> - Use bpp and bpp_div instead of prior misuse of block_w/block_h
> - New patch to use values from SPS/PPS control to configure the HW
> - New patch to remove an unnecessary call to validate sps at streaming start
> - Reworked pixel format validation
> Link to v2:
> https://lore.kernel.org/linux-media/[email protected]/
>
> Changes in v2:
> - Collect r-b tags
> - SPS pic width and height in mbs validation moved to rkvdec_try_ctrl
> - New patch to not override output buffer sizeimage
> - Reworked pixel format validation
> - Only align decoded buffer instead of changing frmsize step_width
> Link to v1:
> https://lore.kernel.org/linux-media/[email protected]/
>
> Following commits adds support for NV15/NV20/NV30 to VOP driver:
> 728c15b4b5f3 ("drm/fourcc: Add NV20 and NV30 YUV formats")
> d4b384228562 ("drm/rockchip: vop: Add NV15, NV20 and NV30 support")
>
> To fully runtime test this series you may need above drm commits and ffmpeg
> patches from [2], this series and drm patches is also available at [3].
>
> [1]
> https://lore.kernel.org/linux-media/[email protected]/
> [2] https://github.com/Kwiboo/FFmpeg/commits/v4l2-request-n6.1-dev/
> [3]
> https://github.com/Kwiboo/linux-rockchip/commits/linuxtv-rkvdec-high-10-v4/
> [4] https://gist.github.com/Kwiboo/f4ac15576b2c72887ae2bc5d58b5c865
> [5] https://gist.github.com/Kwiboo/459a1c8f1dcb56e45dc7a7a29cc28adf
>
> Regards,
> Jonas
>
> Alex Bee (1):
>   media: rkvdec: h264: Don't hardcode SPS/PPS parameters
>
> Jonas Karlman (10):
>   media: v4l2-common: Add helpers to calculate bytesperline and
>     sizeimage
>   media: v4l2: Add NV15 and NV20 pixel formats
>   media: rkvdec: h264: Use bytesperline and buffer height as virstride
>   media: rkvdec: h264: Remove SPS validation at streaming start
>   media: rkvdec: Extract rkvdec_fill_decoded_pixfmt into helper
>   media: rkvdec: Move rkvdec_reset_decoded_fmt helper
>   media: rkvdec: Extract decoded format enumeration into helper
>   media: rkvdec: Add image format concept
>   media: rkvdec: Add get_image_fmt ops
>   media: rkvdec: h264: Support High 10 and 4:2:2 profiles
>
>  .../media/v4l/pixfmt-yuv-planar.rst           | 128 +++++++++++
>  drivers/media/v4l2-core/v4l2-common.c         |  80 +++----
>  drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
>  drivers/staging/media/rkvdec/rkvdec-h264.c    |  83 +++----
>  drivers/staging/media/rkvdec/rkvdec.c         | 217 +++++++++++++-----
>  drivers/staging/media/rkvdec/rkvdec.h         |  18 +-
>  include/uapi/linux/videodev2.h                |   2 +
>  7 files changed, 396 insertions(+), 134 deletions(-)
>

2024-06-16 09:48:10

by Diederik de Haas

[permalink] [raw]
Subject: Re: [PATCH v4 00/11] media: rkvdec: Add H.264 High 10 and 4:2:2 profile support

On Sunday, 5 November 2023 17:54:59 CEST Jonas Karlman wrote:
> This is a revival of a 3 year old series [1] now that NV15/NV20/NV30 support
> for display driver have landed in mainline tree.
>
> This series adds H.264 High 10 and 4:2:2 profile support to the Rockchip
> Video Decoder driver.
>
> Patch 1 adds helpers for calculating plane bytesperline and sizeimage.
> Patch 2 adds two new pixelformats for semi-planer 10-bit 4:2:0/4:2:2 YUV.
>
> Patch 3 change to use bytesperline and buffer height to configure strides.
> Patch 4 change to use values from SPS/PPS control to configure the HW.
> Patch 5 remove an unnecessary call to validate sps at streaming start.
>
> Patch 6-10 refactor code to support filtering of CAPUTRE formats based
> on the image format returned from a get_image_fmt ops.
>
> Patch 11 adds final bits to support H.264 High 10 and 4:2:2 profiles.
>
> Tested on a ROCK Pi 4 (RK3399) and Rock64 (RK3328):
>
> v4l2-compliance 1.24.1, 64 bits, 64-bit time_t
> ...
> Total for rkvdec device /dev/video1: 46, Succeeded: 46, Failed: 0,
> Warnings: 0
>
> Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request
> ...
> Ran 65/69 tests successfully
>
> Running test suite JVT-AVC_V1 with decoder FFmpeg-H.264-V4L2-request
> ...
> Ran 127/135 tests successfully
>
> Before this series:
>
> Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request
> ...
> Ran 44/69 tests successfully
>
> ...
>
> Following commits adds support for NV15/NV20/NV30 to VOP driver:
> 728c15b4b5f3 ("drm/fourcc: Add NV20 and NV30 YUV formats")
> d4b384228562 ("drm/rockchip: vop: Add NV15, NV20 and NV30 support")
>
> To fully runtime test this series you may need above drm commits and ffmpeg
> patches from [2], this series and drm patches is also available at [3].
>
> [1]
> https://lore.kernel.org/linux-media/[email protected]/
> [2] https://github.com/Kwiboo/FFmpeg/commits/v4l2-request-n6.1-dev/ [3]
> https://github.com/Kwiboo/linux-rockchip/commits/linuxtv-rkvdec-high-10-v4/
> [4] https://gist.github.com/Kwiboo/f4ac15576b2c72887ae2bc5d58b5c865 [5]
> https://gist.github.com/Kwiboo/459a1c8f1dcb56e45dc7a7a29cc28adf

Reviving this old thread now that rkvdec2 'stuff' emerged.
I have (actually) done quite some tests with this (and "media: rkvdec: Add
HEVC backend" patch set) and they have been part of my kernel builds ever
since.
I _think_, but don't know, that is relevant for Andy's question:

On zondag 16 juni 2024 08:58:20 CEST Andy Yan <[email protected]> wrote:
> How can I test these patches? Do they require any additional userspace
> patches?

I have the same question and I think you'd need this and the HEVC patch set
and then also patch FFmpeg and then it should enable HW acceleration.
So my question boils down to: with the rkvdec2 patch set, should V4L2-requests
now also work with rkvdec, so not just Hantro anymore?

BTW: the libdrm commits have been merged upstream quite some time ago, so if
you have a recent version of that, you don't need to patch that.
If you use FFmpeg 7.0, then Jonas has a branch for that too (haven't tried it
yet though).

FWIW: my test results were a bit mixed. I didn't post them before as I don't
fully/really understand this 'video stuff', and I didn't want you all to suffer
from what was likely a PEBKAC issue.

On my PineTab2 (rk3566) I had some h.264 videos HW accelerated, but not all.
My guess is that it's related to the resolution. 1920x1080 worked, while it
didn't work with a 1280x640 video. The video still played, just not HW
accelerated. IOW: improvements in some and otherwise it was just rendered by
the CPU (I think), just like before.

On my Rock64 I got a pink tint with all videos, like described here:
https://github.com/mpv-player/mpv/issues/12968
IIUC, that's actually a problem in the lima driver?

Cheers,
Diederik


Attachments:
signature.asc (235.00 B)
This is a digitally signed message part.