This is a revival of a 3 year old series [1] now that NV15/NV20/NV30 support
for display driver have landed in mainline tree.
This series adds H.264 High 10 and 4:2:2 profile support to the Rockchip
Video Decoder driver.
Patch 1 adds helpers for calculating plane bytesperline and sizeimage.
Patch 2 adds two new pixelformats for semi-planer 10-bit 4:2:0/4:2:2 YUV.
Patch 3 change to use bytesperline and buffer height to configure strides.
Patch 4 change to use values from SPS/PPS control to configure the HW.
Patch 5 remove an unnecessary call to validate sps at streaming start.
Patch 6-10 refactor code to support filtering of CAPUTRE formats based
on the image format returned from a get_image_fmt ops.
Patch 11 adds final bits to support H.264 High 10 and 4:2:2 profiles.
Tested on a ROCK Pi 4 (RK3399) and Rock64 (RK3328):
v4l2-compliance 1.24.1, 64 bits, 64-bit time_t
...
Total for rkvdec device /dev/video1: 46, Succeeded: 46, Failed: 0, Warnings: 0
Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request
...
Ran 65/69 tests successfully
Running test suite JVT-AVC_V1 with decoder FFmpeg-H.264-V4L2-request
...
Ran 127/135 tests successfully
Before this series:
Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request
...
Ran 44/69 tests successfully
Changes in v4:
- Fix failed v4l2-compliance tests related to CAPTURE queue
- Rework CAPTURE format filter anv validate to use an image format
- Run fluster test suite JVT-FR-EXT [4] and JVT-AVC_V1 [5]
Link to v3: https://lore.kernel.org/linux-media/[email protected]/
Changes in v3:
- Drop merged patches
- Use bpp and bpp_div instead of prior misuse of block_w/block_h
- New patch to use values from SPS/PPS control to configure the HW
- New patch to remove an unnecessary call to validate sps at streaming start
- Reworked pixel format validation
Link to v2: https://lore.kernel.org/linux-media/[email protected]/
Changes in v2:
- Collect r-b tags
- SPS pic width and height in mbs validation moved to rkvdec_try_ctrl
- New patch to not override output buffer sizeimage
- Reworked pixel format validation
- Only align decoded buffer instead of changing frmsize step_width
Link to v1: https://lore.kernel.org/linux-media/[email protected]/
Following commits adds support for NV15/NV20/NV30 to VOP driver:
728c15b4b5f3 ("drm/fourcc: Add NV20 and NV30 YUV formats")
d4b384228562 ("drm/rockchip: vop: Add NV15, NV20 and NV30 support")
To fully runtime test this series you may need above drm commits and ffmpeg
patches from [2], this series and drm patches is also available at [3].
[1] https://lore.kernel.org/linux-media/[email protected]/
[2] https://github.com/Kwiboo/FFmpeg/commits/v4l2-request-n6.1-dev/
[3] https://github.com/Kwiboo/linux-rockchip/commits/linuxtv-rkvdec-high-10-v4/
[4] https://gist.github.com/Kwiboo/f4ac15576b2c72887ae2bc5d58b5c865
[5] https://gist.github.com/Kwiboo/459a1c8f1dcb56e45dc7a7a29cc28adf
Regards,
Jonas
Alex Bee (1):
media: rkvdec: h264: Don't hardcode SPS/PPS parameters
Jonas Karlman (10):
media: v4l2-common: Add helpers to calculate bytesperline and
sizeimage
media: v4l2: Add NV15 and NV20 pixel formats
media: rkvdec: h264: Use bytesperline and buffer height as virstride
media: rkvdec: h264: Remove SPS validation at streaming start
media: rkvdec: Extract rkvdec_fill_decoded_pixfmt into helper
media: rkvdec: Move rkvdec_reset_decoded_fmt helper
media: rkvdec: Extract decoded format enumeration into helper
media: rkvdec: Add image format concept
media: rkvdec: Add get_image_fmt ops
media: rkvdec: h264: Support High 10 and 4:2:2 profiles
.../media/v4l/pixfmt-yuv-planar.rst | 128 +++++++++++
drivers/media/v4l2-core/v4l2-common.c | 80 +++----
drivers/media/v4l2-core/v4l2-ioctl.c | 2 +
drivers/staging/media/rkvdec/rkvdec-h264.c | 83 +++----
drivers/staging/media/rkvdec/rkvdec.c | 217 +++++++++++++-----
drivers/staging/media/rkvdec/rkvdec.h | 18 +-
include/uapi/linux/videodev2.h | 2 +
7 files changed, 396 insertions(+), 134 deletions(-)
--
2.42.0
Add support for a get_image_fmt() ops that return the required image
format.
The CAPTURE format is reset when required image format changes and the
buffer queue is not busy.
Signed-off-by: Jonas Karlman <[email protected]>
---
v4:
- Change fmt_opaque into an image format
- Split patch into two
v3:
- New patch
drivers/staging/media/rkvdec/rkvdec.c | 49 +++++++++++++++++++++++++--
drivers/staging/media/rkvdec/rkvdec.h | 2 ++
2 files changed, 49 insertions(+), 2 deletions(-)
diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index 5949d59d4cf9..225aa1f0ac48 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -108,15 +108,60 @@ static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl)
{
struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl);
const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
+ struct v4l2_pix_format_mplane *pix_mp = &ctx->decoded_fmt.fmt.pix_mp;
+ enum rkvdec_image_fmt image_fmt;
+ struct vb2_queue *vq;
+ int ret;
+
+ if (desc->ops->try_ctrl) {
+ ret = desc->ops->try_ctrl(ctx, ctrl);
+ if (ret)
+ return ret;
+ }
+
+ if (!desc->ops->get_image_fmt)
+ return 0;
- if (desc->ops->try_ctrl)
- return desc->ops->try_ctrl(ctx, ctrl);
+ image_fmt = desc->ops->get_image_fmt(ctx, ctrl);
+ if (ctx->image_fmt == image_fmt)
+ return 0;
+
+ if (rkvdec_is_valid_fmt(ctx, pix_mp->pixelformat, image_fmt))
+ return 0;
+
+ /* format change not allowed when queue is busy */
+ vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx,
+ V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+ if (vb2_is_busy(vq))
+ return -EINVAL;
+
+ return 0;
+}
+
+static int rkvdec_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl);
+ const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
+ struct v4l2_pix_format_mplane *pix_mp = &ctx->decoded_fmt.fmt.pix_mp;
+ enum rkvdec_image_fmt image_fmt;
+
+ if (!desc->ops->get_image_fmt)
+ return 0;
+
+ image_fmt = desc->ops->get_image_fmt(ctx, ctrl);
+ if (ctx->image_fmt == image_fmt)
+ return 0;
+
+ ctx->image_fmt = image_fmt;
+ if (!rkvdec_is_valid_fmt(ctx, pix_mp->pixelformat, ctx->image_fmt))
+ rkvdec_reset_decoded_fmt(ctx);
return 0;
}
static const struct v4l2_ctrl_ops rkvdec_ctrl_ops = {
.try_ctrl = rkvdec_try_ctrl,
+ .s_ctrl = rkvdec_s_ctrl,
};
static const struct rkvdec_ctrl_desc rkvdec_h264_ctrl_descs[] = {
diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h
index 6f8cf50c5d99..e466a2753ccf 100644
--- a/drivers/staging/media/rkvdec/rkvdec.h
+++ b/drivers/staging/media/rkvdec/rkvdec.h
@@ -73,6 +73,8 @@ struct rkvdec_coded_fmt_ops {
struct vb2_v4l2_buffer *dst_buf,
enum vb2_buffer_state result);
int (*try_ctrl)(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl);
+ enum rkvdec_image_fmt (*get_image_fmt)(struct rkvdec_ctx *ctx,
+ struct v4l2_ctrl *ctrl);
};
enum rkvdec_image_fmt {
--
2.42.0
Add helper functions to calculate plane bytesperline and sizeimage,
these new helpers consider bpp div, block width and height when
calculating plane bytesperline and sizeimage.
Signed-off-by: Jonas Karlman <[email protected]>
---
v4:
- No change
v3:
- Consider bpp_div in calculation
drivers/media/v4l2-core/v4l2-common.c | 78 +++++++++++++--------------
1 file changed, 39 insertions(+), 39 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
index 3a4b15a98e02..834b426da8b1 100644
--- a/drivers/media/v4l2-core/v4l2-common.c
+++ b/drivers/media/v4l2-core/v4l2-common.c
@@ -350,6 +350,34 @@ static inline unsigned int v4l2_format_block_height(const struct v4l2_format_inf
return info->block_h[plane];
}
+static inline unsigned int v4l2_format_plane_width(const struct v4l2_format_info *info, int plane,
+ unsigned int width)
+{
+ unsigned int hdiv = plane ? info->hdiv : 1;
+ unsigned int aligned_width =
+ ALIGN(width, v4l2_format_block_width(info, plane));
+
+ return DIV_ROUND_UP(aligned_width, hdiv) *
+ info->bpp[plane] / info->bpp_div[plane];
+}
+
+static inline unsigned int v4l2_format_plane_height(const struct v4l2_format_info *info, int plane,
+ unsigned int height)
+{
+ unsigned int vdiv = plane ? info->vdiv : 1;
+ unsigned int aligned_height =
+ ALIGN(height, v4l2_format_block_height(info, plane));
+
+ return DIV_ROUND_UP(aligned_height, vdiv);
+}
+
+static inline unsigned int v4l2_format_plane_size(const struct v4l2_format_info *info, int plane,
+ unsigned int width, unsigned int height)
+{
+ return v4l2_format_plane_width(info, plane, width) *
+ v4l2_format_plane_height(info, plane, height);
+}
+
void v4l2_apply_frmsize_constraints(u32 *width, u32 *height,
const struct v4l2_frmsize_stepwise *frmsize)
{
@@ -385,37 +413,19 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
if (info->mem_planes == 1) {
plane = &pixfmt->plane_fmt[0];
- plane->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0] / info->bpp_div[0];
+ plane->bytesperline = v4l2_format_plane_width(info, 0, width);
plane->sizeimage = 0;
- for (i = 0; i < info->comp_planes; i++) {
- unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
- unsigned int vdiv = (i == 0) ? 1 : info->vdiv;
- unsigned int aligned_width;
- unsigned int aligned_height;
-
- aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
- aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
-
- plane->sizeimage += info->bpp[i] *
- DIV_ROUND_UP(aligned_width, hdiv) *
- DIV_ROUND_UP(aligned_height, vdiv) / info->bpp_div[i];
- }
+ for (i = 0; i < info->comp_planes; i++)
+ plane->sizeimage +=
+ v4l2_format_plane_size(info, i, width, height);
} else {
for (i = 0; i < info->comp_planes; i++) {
- unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
- unsigned int vdiv = (i == 0) ? 1 : info->vdiv;
- unsigned int aligned_width;
- unsigned int aligned_height;
-
- aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
- aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
-
plane = &pixfmt->plane_fmt[i];
plane->bytesperline =
- info->bpp[i] * DIV_ROUND_UP(aligned_width, hdiv) / info->bpp_div[i];
- plane->sizeimage =
- plane->bytesperline * DIV_ROUND_UP(aligned_height, vdiv);
+ v4l2_format_plane_width(info, i, width);
+ plane->sizeimage = plane->bytesperline *
+ v4l2_format_plane_height(info, i, height);
}
}
return 0;
@@ -439,22 +449,12 @@ int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, u32 pixelformat,
pixfmt->width = width;
pixfmt->height = height;
pixfmt->pixelformat = pixelformat;
- pixfmt->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0] / info->bpp_div[0];
+ pixfmt->bytesperline = v4l2_format_plane_width(info, 0, width);
pixfmt->sizeimage = 0;
- for (i = 0; i < info->comp_planes; i++) {
- unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
- unsigned int vdiv = (i == 0) ? 1 : info->vdiv;
- unsigned int aligned_width;
- unsigned int aligned_height;
-
- aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
- aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
-
- pixfmt->sizeimage += info->bpp[i] *
- DIV_ROUND_UP(aligned_width, hdiv) *
- DIV_ROUND_UP(aligned_height, vdiv) / info->bpp_div[i];
- }
+ for (i = 0; i < info->comp_planes; i++)
+ pixfmt->sizeimage +=
+ v4l2_format_plane_size(info, i, width, height);
return 0;
}
EXPORT_SYMBOL_GPL(v4l2_fill_pixfmt);
--
2.42.0
Move rkvdec_reset_decoded_fmt() and the called rkvdec_reset_fmt() helper
functions in preparation for adding a new caller in an upcoming patch.
Signed-off-by: Jonas Karlman <[email protected]>
---
v4:
- No change
v3:
- New patch
drivers/staging/media/rkvdec/rkvdec.c | 46 +++++++++++++--------------
1 file changed, 23 insertions(+), 23 deletions(-)
diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index 0570c790ad08..7a79840470e1 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -37,6 +37,29 @@ static void rkvdec_fill_decoded_pixfmt(struct rkvdec_ctx *ctx,
DIV_ROUND_UP(pix_mp->height, 16);
}
+static void rkvdec_reset_fmt(struct rkvdec_ctx *ctx, struct v4l2_format *f,
+ u32 fourcc)
+{
+ memset(f, 0, sizeof(*f));
+ f->fmt.pix_mp.pixelformat = fourcc;
+ f->fmt.pix_mp.field = V4L2_FIELD_NONE;
+ f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_REC709;
+ f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+ f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT;
+}
+
+static void rkvdec_reset_decoded_fmt(struct rkvdec_ctx *ctx)
+{
+ struct v4l2_format *f = &ctx->decoded_fmt;
+
+ rkvdec_reset_fmt(ctx, f, ctx->coded_fmt_desc->decoded_fmts[0]);
+ f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+ f->fmt.pix_mp.width = ctx->coded_fmt.fmt.pix_mp.width;
+ f->fmt.pix_mp.height = ctx->coded_fmt.fmt.pix_mp.height;
+ rkvdec_fill_decoded_pixfmt(ctx, &f->fmt.pix_mp);
+}
+
static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl)
{
struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl);
@@ -169,18 +192,6 @@ rkvdec_find_coded_fmt_desc(u32 fourcc)
return NULL;
}
-static void rkvdec_reset_fmt(struct rkvdec_ctx *ctx, struct v4l2_format *f,
- u32 fourcc)
-{
- memset(f, 0, sizeof(*f));
- f->fmt.pix_mp.pixelformat = fourcc;
- f->fmt.pix_mp.field = V4L2_FIELD_NONE;
- f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_REC709;
- f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
- f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
- f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT;
-}
-
static void rkvdec_reset_coded_fmt(struct rkvdec_ctx *ctx)
{
struct v4l2_format *f = &ctx->coded_fmt;
@@ -196,17 +207,6 @@ static void rkvdec_reset_coded_fmt(struct rkvdec_ctx *ctx)
ctx->coded_fmt_desc->ops->adjust_fmt(ctx, f);
}
-static void rkvdec_reset_decoded_fmt(struct rkvdec_ctx *ctx)
-{
- struct v4l2_format *f = &ctx->decoded_fmt;
-
- rkvdec_reset_fmt(ctx, f, ctx->coded_fmt_desc->decoded_fmts[0]);
- f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
- f->fmt.pix_mp.width = ctx->coded_fmt.fmt.pix_mp.width;
- f->fmt.pix_mp.height = ctx->coded_fmt.fmt.pix_mp.height;
- rkvdec_fill_decoded_pixfmt(ctx, &f->fmt.pix_mp);
-}
-
static int rkvdec_enum_framesizes(struct file *file, void *priv,
struct v4l2_frmsizeenum *fsize)
{
--
2.42.0
Add an enum rkvdec_image_fmt used to signal an image format, e.g.
4:2:0 8-bit, 4:2:0 10-bit or any.
Tag each supported CAPUTRE format with an image format and use this tag
to filter out unsupported CAPTURE formats.
Signed-off-by: Jonas Karlman <[email protected]>
---
v4:
- Change fmt_opaque into an image format
- Split patch into two
v3:
- New patch
drivers/staging/media/rkvdec/rkvdec.c | 45 ++++++++++++++++++++-------
drivers/staging/media/rkvdec/rkvdec.h | 13 +++++++-
2 files changed, 45 insertions(+), 13 deletions(-)
diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index c3aede94c872..5949d59d4cf9 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -27,26 +27,42 @@
#include "rkvdec.h"
#include "rkvdec-regs.h"
-static u32 rkvdec_enum_decoded_fmt(struct rkvdec_ctx *ctx, int index)
+static inline bool rkvdec_image_fmt_match(enum rkvdec_image_fmt fmt1,
+ enum rkvdec_image_fmt fmt2)
+{
+ return fmt1 == fmt2 || fmt2 == RKVDEC_IMG_FMT_ANY ||
+ fmt1 == RKVDEC_IMG_FMT_ANY;
+}
+
+static u32 rkvdec_enum_decoded_fmt(struct rkvdec_ctx *ctx, int index,
+ enum rkvdec_image_fmt image_fmt)
{
const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
+ unsigned int i, j;
if (WARN_ON(!desc))
return 0;
- if (index >= desc->num_decoded_fmts)
- return 0;
+ for (i = 0, j = 0; i < desc->num_decoded_fmts; i++) {
+ if (rkvdec_image_fmt_match(desc->decoded_fmts[i].image_fmt,
+ image_fmt) &&
+ index == j++)
+ return desc->decoded_fmts[i].fourcc;
+ }
- return desc->decoded_fmts[index];
+ return 0;
}
-static bool rkvdec_is_valid_fmt(struct rkvdec_ctx *ctx, u32 fourcc)
+static bool rkvdec_is_valid_fmt(struct rkvdec_ctx *ctx, u32 fourcc,
+ enum rkvdec_image_fmt image_fmt)
{
const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
unsigned int i;
for (i = 0; i < desc->num_decoded_fmts; i++) {
- if (desc->decoded_fmts[i] == fourcc)
+ if (rkvdec_image_fmt_match(desc->decoded_fmts[i].image_fmt,
+ image_fmt) &&
+ desc->decoded_fmts[i].fourcc == fourcc)
return true;
}
@@ -80,7 +96,7 @@ static void rkvdec_reset_decoded_fmt(struct rkvdec_ctx *ctx)
struct v4l2_format *f = &ctx->decoded_fmt;
u32 fourcc;
- fourcc = rkvdec_enum_decoded_fmt(ctx, 0);
+ fourcc = rkvdec_enum_decoded_fmt(ctx, 0, ctx->image_fmt);
rkvdec_reset_fmt(ctx, f, fourcc);
f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
f->fmt.pix_mp.width = ctx->coded_fmt.fmt.pix_mp.width;
@@ -149,8 +165,11 @@ static const struct rkvdec_ctrls rkvdec_h264_ctrls = {
.num_ctrls = ARRAY_SIZE(rkvdec_h264_ctrl_descs),
};
-static const u32 rkvdec_h264_vp9_decoded_fmts[] = {
- V4L2_PIX_FMT_NV12,
+static const struct rkvdec_decoded_fmt_desc rkvdec_h264_vp9_decoded_fmts[] = {
+ {
+ .fourcc = V4L2_PIX_FMT_NV12,
+ .image_fmt = RKVDEC_IMG_FMT_420_8BIT,
+ },
};
static const struct rkvdec_ctrl_desc rkvdec_vp9_ctrl_descs[] = {
@@ -282,8 +301,9 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv,
if (WARN_ON(!coded_desc))
return -EINVAL;
- if (!rkvdec_is_valid_fmt(ctx, pix_mp->pixelformat))
- pix_mp->pixelformat = rkvdec_enum_decoded_fmt(ctx, 0);
+ if (!rkvdec_is_valid_fmt(ctx, pix_mp->pixelformat, ctx->image_fmt))
+ pix_mp->pixelformat = rkvdec_enum_decoded_fmt(ctx, 0,
+ ctx->image_fmt);
/* Always apply the frmsize constraint of the coded end. */
pix_mp->width = max(pix_mp->width, ctx->coded_fmt.fmt.pix_mp.width);
@@ -400,6 +420,7 @@ static int rkvdec_s_output_fmt(struct file *file, void *priv,
*
* Note that this will propagates any size changes to the decoded format.
*/
+ ctx->image_fmt = RKVDEC_IMG_FMT_ANY;
rkvdec_reset_decoded_fmt(ctx);
/* Propagate colorspace information to capture. */
@@ -449,7 +470,7 @@ static int rkvdec_enum_capture_fmt(struct file *file, void *priv,
struct rkvdec_ctx *ctx = fh_to_rkvdec_ctx(priv);
u32 fourcc;
- fourcc = rkvdec_enum_decoded_fmt(ctx, f->index);
+ fourcc = rkvdec_enum_decoded_fmt(ctx, f->index, ctx->image_fmt);
if (!fourcc)
return -EINVAL;
diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h
index 633335ebb9c4..6f8cf50c5d99 100644
--- a/drivers/staging/media/rkvdec/rkvdec.h
+++ b/drivers/staging/media/rkvdec/rkvdec.h
@@ -75,13 +75,23 @@ struct rkvdec_coded_fmt_ops {
int (*try_ctrl)(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl);
};
+enum rkvdec_image_fmt {
+ RKVDEC_IMG_FMT_ANY = 0,
+ RKVDEC_IMG_FMT_420_8BIT,
+};
+
+struct rkvdec_decoded_fmt_desc {
+ u32 fourcc;
+ enum rkvdec_image_fmt image_fmt;
+};
+
struct rkvdec_coded_fmt_desc {
u32 fourcc;
struct v4l2_frmsize_stepwise frmsize;
const struct rkvdec_ctrls *ctrls;
const struct rkvdec_coded_fmt_ops *ops;
unsigned int num_decoded_fmts;
- const u32 *decoded_fmts;
+ const struct rkvdec_decoded_fmt_desc *decoded_fmts;
u32 subsystem_flags;
};
@@ -104,6 +114,7 @@ struct rkvdec_ctx {
const struct rkvdec_coded_fmt_desc *coded_fmt_desc;
struct v4l2_ctrl_handler ctrl_hdl;
struct rkvdec_dev *dev;
+ enum rkvdec_image_fmt image_fmt;
void *priv;
};
--
2.42.0
Add support and enable decoding of H264 High 10 and 4:2:2 profiles.
Decoded CAPTURE buffer width is aligned to 64 pixels to accommodate HW
requirement of 10-bit format buffers, fixes decoding of:
- Hi422FR13_SONY_A
- Hi422FR14_SONY_A
- Hi422FR15_SONY_A
- Hi422FR6_SONY_A
- Hi422FR7_SONY_A
- Hi422FR8_SONY_A
- Hi422FR9_SONY_A
- Hi422FREXT18_SONY_A
The get_image_fmt() ops is implemented to select an image format
required for the provided SPS control.
Signed-off-by: Jonas Karlman <[email protected]>
---
v4:
- Change to use get_image_fmt() ops
v3:
- Add get_fmt_opaque ops, the expected pixelformat is used as opaque
- Add new valid_fmt ops that validate pixelformat matches opaque
- Update H264_PROFILE control max value
drivers/staging/media/rkvdec/rkvdec-h264.c | 37 ++++++++++++++++------
drivers/staging/media/rkvdec/rkvdec.c | 33 +++++++++++++++----
drivers/staging/media/rkvdec/rkvdec.h | 3 ++
3 files changed, 57 insertions(+), 16 deletions(-)
diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index 815d5359ddd5..baac6d012ddd 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -1027,24 +1027,42 @@ static int rkvdec_h264_adjust_fmt(struct rkvdec_ctx *ctx,
return 0;
}
+static enum rkvdec_image_fmt rkvdec_h264_get_image_fmt(struct rkvdec_ctx *ctx,
+ struct v4l2_ctrl *ctrl)
+{
+ const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
+
+ if (ctrl->id != V4L2_CID_STATELESS_H264_SPS)
+ return RKVDEC_IMG_FMT_ANY;
+
+ if (sps->bit_depth_luma_minus8 == 0) {
+ if (sps->chroma_format_idc == 2)
+ return RKVDEC_IMG_FMT_422_8BIT;
+ else
+ return RKVDEC_IMG_FMT_420_8BIT;
+ } else if (sps->bit_depth_luma_minus8 == 2) {
+ if (sps->chroma_format_idc == 2)
+ return RKVDEC_IMG_FMT_422_10BIT;
+ else
+ return RKVDEC_IMG_FMT_420_10BIT;
+ }
+
+ return RKVDEC_IMG_FMT_ANY;
+}
+
static int rkvdec_h264_validate_sps(struct rkvdec_ctx *ctx,
const struct v4l2_ctrl_h264_sps *sps)
{
unsigned int width, height;
- /*
- * TODO: The hardware supports 10-bit and 4:2:2 profiles,
- * but it's currently broken in the driver.
- * Reject them for now, until it's fixed.
- */
- if (sps->chroma_format_idc > 1)
- /* Only 4:0:0 and 4:2:0 are supported */
+ if (sps->chroma_format_idc > 2)
+ /* Only 4:0:0, 4:2:0 and 4:2:2 are supported */
return -EINVAL;
if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
/* Luma and chroma bit depth mismatch */
return -EINVAL;
- if (sps->bit_depth_luma_minus8 != 0)
- /* Only 8-bit is supported */
+ if (sps->bit_depth_luma_minus8 != 0 && sps->bit_depth_luma_minus8 != 2)
+ /* Only 8-bit and 10-bit is supported */
return -EINVAL;
width = (sps->pic_width_in_mbs_minus1 + 1) * 16;
@@ -1175,4 +1193,5 @@ const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops = {
.stop = rkvdec_h264_stop,
.run = rkvdec_h264_run,
.try_ctrl = rkvdec_h264_try_ctrl,
+ .get_image_fmt = rkvdec_h264_get_image_fmt,
};
diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index 225aa1f0ac48..eb59605ccf28 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -73,7 +73,7 @@ static void rkvdec_fill_decoded_pixfmt(struct rkvdec_ctx *ctx,
struct v4l2_pix_format_mplane *pix_mp)
{
v4l2_fill_pixfmt_mp(pix_mp, pix_mp->pixelformat,
- pix_mp->width, pix_mp->height);
+ ALIGN(pix_mp->width, 64), pix_mp->height);
pix_mp->plane_fmt[0].sizeimage += 128 *
DIV_ROUND_UP(pix_mp->width, 16) *
DIV_ROUND_UP(pix_mp->height, 16);
@@ -193,7 +193,7 @@ static const struct rkvdec_ctrl_desc rkvdec_h264_ctrl_descs[] = {
{
.cfg.id = V4L2_CID_MPEG_VIDEO_H264_PROFILE,
.cfg.min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE,
- .cfg.max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
+ .cfg.max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_422,
.cfg.menu_skip_mask =
BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED),
.cfg.def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN,
@@ -210,11 +210,23 @@ static const struct rkvdec_ctrls rkvdec_h264_ctrls = {
.num_ctrls = ARRAY_SIZE(rkvdec_h264_ctrl_descs),
};
-static const struct rkvdec_decoded_fmt_desc rkvdec_h264_vp9_decoded_fmts[] = {
+static const struct rkvdec_decoded_fmt_desc rkvdec_h264_decoded_fmts[] = {
{
.fourcc = V4L2_PIX_FMT_NV12,
.image_fmt = RKVDEC_IMG_FMT_420_8BIT,
},
+ {
+ .fourcc = V4L2_PIX_FMT_NV15,
+ .image_fmt = RKVDEC_IMG_FMT_420_10BIT,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_NV16,
+ .image_fmt = RKVDEC_IMG_FMT_422_8BIT,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_NV20,
+ .image_fmt = RKVDEC_IMG_FMT_422_10BIT,
+ },
};
static const struct rkvdec_ctrl_desc rkvdec_vp9_ctrl_descs[] = {
@@ -237,6 +249,13 @@ static const struct rkvdec_ctrls rkvdec_vp9_ctrls = {
.num_ctrls = ARRAY_SIZE(rkvdec_vp9_ctrl_descs),
};
+static const struct rkvdec_decoded_fmt_desc rkvdec_vp9_decoded_fmts[] = {
+ {
+ .fourcc = V4L2_PIX_FMT_NV12,
+ .image_fmt = RKVDEC_IMG_FMT_420_8BIT,
+ },
+};
+
static const struct rkvdec_coded_fmt_desc rkvdec_coded_fmts[] = {
{
.fourcc = V4L2_PIX_FMT_H264_SLICE,
@@ -250,8 +269,8 @@ static const struct rkvdec_coded_fmt_desc rkvdec_coded_fmts[] = {
},
.ctrls = &rkvdec_h264_ctrls,
.ops = &rkvdec_h264_fmt_ops,
- .num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_vp9_decoded_fmts),
- .decoded_fmts = rkvdec_h264_vp9_decoded_fmts,
+ .num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_decoded_fmts),
+ .decoded_fmts = rkvdec_h264_decoded_fmts,
.subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF,
},
{
@@ -266,8 +285,8 @@ static const struct rkvdec_coded_fmt_desc rkvdec_coded_fmts[] = {
},
.ctrls = &rkvdec_vp9_ctrls,
.ops = &rkvdec_vp9_fmt_ops,
- .num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_vp9_decoded_fmts),
- .decoded_fmts = rkvdec_h264_vp9_decoded_fmts,
+ .num_decoded_fmts = ARRAY_SIZE(rkvdec_vp9_decoded_fmts),
+ .decoded_fmts = rkvdec_vp9_decoded_fmts,
}
};
diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h
index e466a2753ccf..9a9f4fced7a1 100644
--- a/drivers/staging/media/rkvdec/rkvdec.h
+++ b/drivers/staging/media/rkvdec/rkvdec.h
@@ -80,6 +80,9 @@ struct rkvdec_coded_fmt_ops {
enum rkvdec_image_fmt {
RKVDEC_IMG_FMT_ANY = 0,
RKVDEC_IMG_FMT_420_8BIT,
+ RKVDEC_IMG_FMT_420_10BIT,
+ RKVDEC_IMG_FMT_422_8BIT,
+ RKVDEC_IMG_FMT_422_10BIT,
};
struct rkvdec_decoded_fmt_desc {
--
2.42.0
Add a rkvdec_is_valid_fmt() helper that check if a fourcc is a supported
CAPTURE format, and a rkvdec_enum_decoded_fmt() helper that enumerate
valid formats.
This move current code into helper functions in preparation for adding
CAPTURE format filtering and validation in next patch.
Signed-off-by: Jonas Karlman <[email protected]>
---
v4:
- Rename rkvdec_decoded_fmts() to rkvdec_enum_decoded_fmt()
- Rename rkvdec_valid_fmt() to rkvdec_is_valid_fmt()
v3:
- New patch
drivers/staging/media/rkvdec/rkvdec.c | 49 +++++++++++++++++++--------
1 file changed, 35 insertions(+), 14 deletions(-)
diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index 7a79840470e1..c3aede94c872 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -27,6 +27,32 @@
#include "rkvdec.h"
#include "rkvdec-regs.h"
+static u32 rkvdec_enum_decoded_fmt(struct rkvdec_ctx *ctx, int index)
+{
+ const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
+
+ if (WARN_ON(!desc))
+ return 0;
+
+ if (index >= desc->num_decoded_fmts)
+ return 0;
+
+ return desc->decoded_fmts[index];
+}
+
+static bool rkvdec_is_valid_fmt(struct rkvdec_ctx *ctx, u32 fourcc)
+{
+ const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
+ unsigned int i;
+
+ for (i = 0; i < desc->num_decoded_fmts; i++) {
+ if (desc->decoded_fmts[i] == fourcc)
+ return true;
+ }
+
+ return false;
+}
+
static void rkvdec_fill_decoded_pixfmt(struct rkvdec_ctx *ctx,
struct v4l2_pix_format_mplane *pix_mp)
{
@@ -52,8 +78,10 @@ static void rkvdec_reset_fmt(struct rkvdec_ctx *ctx, struct v4l2_format *f,
static void rkvdec_reset_decoded_fmt(struct rkvdec_ctx *ctx)
{
struct v4l2_format *f = &ctx->decoded_fmt;
+ u32 fourcc;
- rkvdec_reset_fmt(ctx, f, ctx->coded_fmt_desc->decoded_fmts[0]);
+ fourcc = rkvdec_enum_decoded_fmt(ctx, 0);
+ rkvdec_reset_fmt(ctx, f, fourcc);
f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
f->fmt.pix_mp.width = ctx->coded_fmt.fmt.pix_mp.width;
f->fmt.pix_mp.height = ctx->coded_fmt.fmt.pix_mp.height;
@@ -244,7 +272,6 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv,
struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
struct rkvdec_ctx *ctx = fh_to_rkvdec_ctx(priv);
const struct rkvdec_coded_fmt_desc *coded_desc;
- unsigned int i;
/*
* The codec context should point to a coded format desc, if the format
@@ -255,13 +282,8 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv,
if (WARN_ON(!coded_desc))
return -EINVAL;
- for (i = 0; i < coded_desc->num_decoded_fmts; i++) {
- if (coded_desc->decoded_fmts[i] == pix_mp->pixelformat)
- break;
- }
-
- if (i == coded_desc->num_decoded_fmts)
- pix_mp->pixelformat = coded_desc->decoded_fmts[0];
+ if (!rkvdec_is_valid_fmt(ctx, pix_mp->pixelformat))
+ pix_mp->pixelformat = rkvdec_enum_decoded_fmt(ctx, 0);
/* Always apply the frmsize constraint of the coded end. */
pix_mp->width = max(pix_mp->width, ctx->coded_fmt.fmt.pix_mp.width);
@@ -425,14 +447,13 @@ static int rkvdec_enum_capture_fmt(struct file *file, void *priv,
struct v4l2_fmtdesc *f)
{
struct rkvdec_ctx *ctx = fh_to_rkvdec_ctx(priv);
+ u32 fourcc;
- if (WARN_ON(!ctx->coded_fmt_desc))
- return -EINVAL;
-
- if (f->index >= ctx->coded_fmt_desc->num_decoded_fmts)
+ fourcc = rkvdec_enum_decoded_fmt(ctx, f->index);
+ if (!fourcc)
return -EINVAL;
- f->pixelformat = ctx->coded_fmt_desc->decoded_fmts[f->index];
+ f->pixelformat = fourcc;
return 0;
}
--
2.42.0
Hi Jonas,
Happy new year!
On Sun, 2023-11-05 at 16:54 +0000, Jonas Karlman wrote:
> This is a revival of a 3 year old series [1] now that NV15/NV20/NV30 support
> for display driver have landed in mainline tree.
>
> This series adds H.264 High 10 and 4:2:2 profile support to the Rockchip
> Video Decoder driver.
>
> Patch 1 adds helpers for calculating plane bytesperline and sizeimage.
> Patch 2 adds two new pixelformats for semi-planer 10-bit 4:2:0/4:2:2 YUV.
>
> Patch 3 change to use bytesperline and buffer height to configure strides.
> Patch 4 change to use values from SPS/PPS control to configure the HW.
> Patch 5 remove an unnecessary call to validate sps at streaming start.
>
> Patch 6-10 refactor code to support filtering of CAPUTRE formats based
> on the image format returned from a get_image_fmt ops.
>
> Patch 11 adds final bits to support H.264 High 10 and 4:2:2 profiles.
I send my Tested-by to v3 of this series, I also tested with v4 so for future
series please add:
Tested-by: Christopher Obbard <[email protected]>
>
> Tested on a ROCK Pi 4 (RK3399) and Rock64 (RK3328):
>
> v4l2-compliance 1.24.1, 64 bits, 64-bit time_t
> ...
> Total for rkvdec device /dev/video1: 46, Succeeded: 46, Failed: 0,
> Warnings: 0
>
> Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request
> ...
> Ran 65/69 tests successfully
>
> Running test suite JVT-AVC_V1 with decoder FFmpeg-H.264-V4L2-request
> ...
> Ran 127/135 tests successfully
>
> Before this series:
>
> Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request
> ...
> Ran 44/69 tests successfully
>
> Changes in v4:
> - Fix failed v4l2-compliance tests related to CAPTURE queue
> - Rework CAPTURE format filter anv validate to use an image format
> - Run fluster test suite JVT-FR-EXT [4] and JVT-AVC_V1 [5]
> Link to v3:
> https://lore.kernel.org/linux-media/[email protected]/
>
> Changes in v3:
> - Drop merged patches
> - Use bpp and bpp_div instead of prior misuse of block_w/block_h
> - New patch to use values from SPS/PPS control to configure the HW
> - New patch to remove an unnecessary call to validate sps at streaming start
> - Reworked pixel format validation
> Link to v2:
> https://lore.kernel.org/linux-media/[email protected]/
>
> Changes in v2:
> - Collect r-b tags
> - SPS pic width and height in mbs validation moved to rkvdec_try_ctrl
> - New patch to not override output buffer sizeimage
> - Reworked pixel format validation
> - Only align decoded buffer instead of changing frmsize step_width
> Link to v1:
> https://lore.kernel.org/linux-media/[email protected]/
>
> Following commits adds support for NV15/NV20/NV30 to VOP driver:
> 728c15b4b5f3 ("drm/fourcc: Add NV20 and NV30 YUV formats")
> d4b384228562 ("drm/rockchip: vop: Add NV15, NV20 and NV30 support")
>
> To fully runtime test this series you may need above drm commits and ffmpeg
> patches from [2], this series and drm patches is also available at [3].
>
> [1]
> https://lore.kernel.org/linux-media/[email protected]/
> [2] https://github.com/Kwiboo/FFmpeg/commits/v4l2-request-n6.1-dev/
> [3]
> https://github.com/Kwiboo/linux-rockchip/commits/linuxtv-rkvdec-high-10-v4/
> [4] https://gist.github.com/Kwiboo/f4ac15576b2c72887ae2bc5d58b5c865
> [5] https://gist.github.com/Kwiboo/459a1c8f1dcb56e45dc7a7a29cc28adf
>
> Regards,
> Jonas
>
> Alex Bee (1):
> media: rkvdec: h264: Don't hardcode SPS/PPS parameters
>
> Jonas Karlman (10):
> media: v4l2-common: Add helpers to calculate bytesperline and
> sizeimage
> media: v4l2: Add NV15 and NV20 pixel formats
> media: rkvdec: h264: Use bytesperline and buffer height as virstride
> media: rkvdec: h264: Remove SPS validation at streaming start
> media: rkvdec: Extract rkvdec_fill_decoded_pixfmt into helper
> media: rkvdec: Move rkvdec_reset_decoded_fmt helper
> media: rkvdec: Extract decoded format enumeration into helper
> media: rkvdec: Add image format concept
> media: rkvdec: Add get_image_fmt ops
> media: rkvdec: h264: Support High 10 and 4:2:2 profiles
>
> .../media/v4l/pixfmt-yuv-planar.rst | 128 +++++++++++
> drivers/media/v4l2-core/v4l2-common.c | 80 +++----
> drivers/media/v4l2-core/v4l2-ioctl.c | 2 +
> drivers/staging/media/rkvdec/rkvdec-h264.c | 83 +++----
> drivers/staging/media/rkvdec/rkvdec.c | 217 +++++++++++++-----
> drivers/staging/media/rkvdec/rkvdec.h | 18 +-
> include/uapi/linux/videodev2.h | 2 +
> 7 files changed, 396 insertions(+), 134 deletions(-)
>