Dear all,
This patch series adds VP9 codec V4L2 control interface and two drivers
using the new controls. It is a follow-up of previous v6 series [1].
In this iteration, we've implemented VP9 hardware decoding on two devices:
Rockchip VDEC and Hantro G2, and tested on RK3399, i.MX8MQ and i.MX8MP.
The i.MX8M driver needs proper power domains support, though, which is a
subject of a different effort, but in all 3 cases we were able to run the
drivers.
GStreamer support is also available, the needed changes have been submitted
by Daniel Almeida [2]. This MR is ready to be merged, and just needs the
VP9 V4L2 controls to be merged and released.
Both rkvdec and hantro drivers are passing a significant number of VP9 tests
using Fluster[3]. There are still a few tests that are not passing, due to
dynamic frame resize (not yet supported by V4L2) and small size videos
(due to IP block limitations).
The series adds the VP9 codec V4L2 control API as uAPI, so it aims at being
merged without passing through staging, as agreed[4]. The ABI has been checked
for padding and verified to contain no holes.
[1] https://patchwork.linuxtv.org/project/linux-media/list/?series=6377
[2] https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2144
[3] https://github.com/fluendo/fluster
[4] https://lore.kernel.org/linux-media/[email protected]/
The series depends on the YUV tiled format support prepared by Ezequiel:
https://www.spinics.net/lists/linux-media/msg197047.html
Rebased onto latest media_tree.
Changes related to v6:
- moved setting tile filter and tile bsd auxiliary buffer addresses so
that they are always set, even if no tiles are used (thanks, Jernej)
- added a comment near the place where the 32-bit DMA mask is applied
(thanks, Nicolas)
- improved consistency in register names (thanks, Nicolas)
Changes related to v5:
- improved the doc comments as per Ezequiel's review (thanks, Ezequiel)
- improved pdf output of documentation
- added Benjamin's Reviewed-by (thanks, Benjamin)
Changes related to v4:
- removed unused enum v4l2_vp9_intra_prediction_mode
- converted remaining enums to defines to follow the convention
- improved the documentation, in particular better documented how to use segmentation
features
Changes related to v3:
Apply suggestions from Jernej's review (thanks, Jernej):
- renamed a control and two structs:
V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR_PROBS =>
V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR
v4l2_ctrl_vp9_compressed_hdr_probs =>
v4l2_ctrl_vp9_compressed_hdr
v4l2_vp9_mv_compressed_hdr_probs => v4l2_vp9_mv_probs
- moved tx_mode to v4l2_ctrl_vp9_compressed_hdr
- fixed enum v4l2_vp9_ref_frame_sign_bias values (which are used to test a bitfield)
- explicitly assigned values to all other vp9 enums
Apply suggestion from Nicolas's review (thanks, Nicolas):
- explicitly stated that the v4l2_ctrl_vp9_compressed_hdr control is optional
and implemented only by drivers which need it
Changes related to the RFC v2:
- added another driver including a postprocessor to de-tile
codec-specific tiling
- reworked uAPI structs layout to follow VP8 style
- changed validation of loop filter params
- changed validation of segmentation params
- changed validation of VP9 frame params
- removed level lookup array from loop filter struct
(can be computed by drivers)
- renamed some enum values to match the spec more closely
- V4L2 VP9 library changed the 'eob' member of
'struct v4l2_vp9_frame_symbol_counts' so that it is an array
of pointers instead of an array of pointers to arrays
(IPs such as g2 creatively pass parts of the 'eob' counts in
the 'coeff' counts)
- factored out several repeated portions of code
- minor nitpicks and cleanups
Andrzej Pietrasiewicz (6):
media: uapi: Add VP9 stateless decoder controls
media: Add VP9 v4l2 library
media: hantro: Rename registers
media: hantro: Prepare for other G2 codecs
media: hantro: Support VP9 on the G2 core
media: hantro: Support NV12 on the G2 core
Boris Brezillon (1):
media: rkvdec: Add the VP9 backend
Ezequiel Garcia (4):
hantro: postproc: Fix motion vector space size
hantro: postproc: Introduce struct hantro_postproc_ops
hantro: Simplify postprocessor
hantro: Add quirk for NV12/NV12_4L4 capture format
.../userspace-api/media/v4l/biblio.rst | 10 +
.../media/v4l/ext-ctrls-codec-stateless.rst | 573 +++++
.../media/v4l/pixfmt-compressed.rst | 15 +
.../media/v4l/vidioc-g-ext-ctrls.rst | 8 +
.../media/v4l/vidioc-queryctrl.rst | 12 +
.../media/videodev2.h.rst.exceptions | 2 +
drivers/media/v4l2-core/Kconfig | 4 +
drivers/media/v4l2-core/Makefile | 1 +
drivers/media/v4l2-core/v4l2-ctrls-core.c | 180 ++
drivers/media/v4l2-core/v4l2-ctrls-defs.c | 8 +
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
drivers/media/v4l2-core/v4l2-vp9.c | 1850 +++++++++++++++++
drivers/staging/media/hantro/Kconfig | 1 +
drivers/staging/media/hantro/Makefile | 7 +-
drivers/staging/media/hantro/hantro.h | 40 +-
drivers/staging/media/hantro/hantro_drv.c | 23 +-
drivers/staging/media/hantro/hantro_g2.c | 27 +
.../staging/media/hantro/hantro_g2_hevc_dec.c | 69 +-
drivers/staging/media/hantro/hantro_g2_regs.h | 132 +-
.../staging/media/hantro/hantro_g2_vp9_dec.c | 980 +++++++++
drivers/staging/media/hantro/hantro_hw.h | 83 +-
.../staging/media/hantro/hantro_postproc.c | 79 +-
drivers/staging/media/hantro/hantro_v4l2.c | 20 +
drivers/staging/media/hantro/hantro_vp9.c | 240 +++
drivers/staging/media/hantro/hantro_vp9.h | 103 +
drivers/staging/media/hantro/imx8m_vpu_hw.c | 38 +-
.../staging/media/hantro/rockchip_vpu_hw.c | 7 +-
.../staging/media/hantro/sama5d4_vdec_hw.c | 3 +-
drivers/staging/media/rkvdec/Kconfig | 1 +
drivers/staging/media/rkvdec/Makefile | 2 +-
drivers/staging/media/rkvdec/rkvdec-vp9.c | 1078 ++++++++++
drivers/staging/media/rkvdec/rkvdec.c | 52 +-
drivers/staging/media/rkvdec/rkvdec.h | 12 +-
include/media/v4l2-ctrls.h | 4 +
include/media/v4l2-vp9.h | 182 ++
include/uapi/linux/v4l2-controls.h | 284 +++
include/uapi/linux/videodev2.h | 6 +
37 files changed, 6033 insertions(+), 104 deletions(-)
create mode 100644 drivers/media/v4l2-core/v4l2-vp9.c
create mode 100644 drivers/staging/media/hantro/hantro_g2.c
create mode 100644 drivers/staging/media/hantro/hantro_g2_vp9_dec.c
create mode 100644 drivers/staging/media/hantro/hantro_vp9.c
create mode 100644 drivers/staging/media/hantro/hantro_vp9.h
create mode 100644 drivers/staging/media/rkvdec/rkvdec-vp9.c
create mode 100644 include/media/v4l2-vp9.h
base-commit: e4e737bb5c170df6135a127739a9e6148ee3da82
--
2.17.1
VeriSilicon Hantro G2 core supports other codecs besides hevc.
Factor out some common code in preparation for vp9 support.
Signed-off-by: Andrzej Pietrasiewicz <[email protected]>
Reviewed-by: Benjamin Gaignard <[email protected]>
---
drivers/staging/media/hantro/Makefile | 1 +
drivers/staging/media/hantro/hantro.h | 7 +++++
drivers/staging/media/hantro/hantro_drv.c | 5 +++
drivers/staging/media/hantro/hantro_g2.c | 27 ++++++++++++++++
.../staging/media/hantro/hantro_g2_hevc_dec.c | 31 -------------------
drivers/staging/media/hantro/hantro_g2_regs.h | 7 +++++
drivers/staging/media/hantro/hantro_hw.h | 2 ++
7 files changed, 49 insertions(+), 31 deletions(-)
create mode 100644 drivers/staging/media/hantro/hantro_g2.c
diff --git a/drivers/staging/media/hantro/Makefile b/drivers/staging/media/hantro/Makefile
index 90036831fec4..fe6d84871d07 100644
--- a/drivers/staging/media/hantro/Makefile
+++ b/drivers/staging/media/hantro/Makefile
@@ -12,6 +12,7 @@ hantro-vpu-y += \
hantro_g1_mpeg2_dec.o \
hantro_g2_hevc_dec.o \
hantro_g1_vp8_dec.o \
+ hantro_g2.o \
rockchip_vpu2_hw_jpeg_enc.o \
rockchip_vpu2_hw_h264_dec.o \
rockchip_vpu2_hw_mpeg2_dec.o \
diff --git a/drivers/staging/media/hantro/hantro.h b/drivers/staging/media/hantro/hantro.h
index dd5e56765d4e..d91eb2b1c509 100644
--- a/drivers/staging/media/hantro/hantro.h
+++ b/drivers/staging/media/hantro/hantro.h
@@ -369,6 +369,13 @@ static inline void vdpu_write(struct hantro_dev *vpu, u32 val, u32 reg)
writel(val, vpu->dec_base + reg);
}
+static inline void hantro_write_addr(struct hantro_dev *vpu,
+ unsigned long offset,
+ dma_addr_t addr)
+{
+ vdpu_write(vpu, addr & 0xffffffff, offset);
+}
+
static inline u32 vdpu_read(struct hantro_dev *vpu, u32 reg)
{
u32 val = readl(vpu->dec_base + reg);
diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
index 8a2edd67f2c6..e8eee117d97f 100644
--- a/drivers/staging/media/hantro/hantro_drv.c
+++ b/drivers/staging/media/hantro/hantro_drv.c
@@ -905,6 +905,11 @@ static int hantro_probe(struct platform_device *pdev)
vpu->enc_base = vpu->reg_bases[0] + vpu->variant->enc_offset;
vpu->dec_base = vpu->reg_bases[0] + vpu->variant->dec_offset;
+ /**
+ * TODO: Eventually allow taking advantage of full 64-bit address space.
+ * Until then we assume the MSB portion of buffers' base addresses is
+ * always 0 due to this masking operation.
+ */
ret = dma_set_coherent_mask(vpu->dev, DMA_BIT_MASK(32));
if (ret) {
dev_err(vpu->dev, "Could not set DMA coherent mask.\n");
diff --git a/drivers/staging/media/hantro/hantro_g2.c b/drivers/staging/media/hantro/hantro_g2.c
new file mode 100644
index 000000000000..5f7bb27913de
--- /dev/null
+++ b/drivers/staging/media/hantro/hantro_g2.c
@@ -0,0 +1,27 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Hantro VPU codec driver
+ *
+ * Copyright (C) 2021 Collabora Ltd, Andrzej Pietrasiewicz <[email protected]>
+ */
+
+#include "hantro_hw.h"
+#include "hantro_g2_regs.h"
+
+void hantro_g2_check_idle(struct hantro_dev *vpu)
+{
+ int i;
+
+ for (i = 0; i < 3; i++) {
+ u32 status;
+
+ /* Make sure the VPU is idle */
+ status = vdpu_read(vpu, G2_REG_INTERRUPT);
+ if (status & G2_REG_INTERRUPT_DEC_E) {
+ dev_warn(vpu->dev, "device still running, aborting");
+ status |= G2_REG_INTERRUPT_DEC_ABORT_E | G2_REG_INTERRUPT_DEC_IRQ_DIS;
+ vdpu_write(vpu, status, G2_REG_INTERRUPT);
+ }
+ }
+}
+
diff --git a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
index 97da719a9844..2797825cef47 100644
--- a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
+++ b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
@@ -8,20 +8,6 @@
#include "hantro_hw.h"
#include "hantro_g2_regs.h"
-#define HEVC_DEC_MODE 0xC
-
-#define BUS_WIDTH_32 0
-#define BUS_WIDTH_64 1
-#define BUS_WIDTH_128 2
-#define BUS_WIDTH_256 3
-
-static inline void hantro_write_addr(struct hantro_dev *vpu,
- unsigned long offset,
- dma_addr_t addr)
-{
- vdpu_write(vpu, addr & 0xffffffff, offset);
-}
-
static void prepare_tile_info_buffer(struct hantro_ctx *ctx)
{
struct hantro_dev *vpu = ctx->dev;
@@ -516,23 +502,6 @@ static void set_buffers(struct hantro_ctx *ctx)
hantro_write_addr(vpu, G2_TILE_BSD_ADDR, ctx->hevc_dec.tile_bsd.dma);
}
-static void hantro_g2_check_idle(struct hantro_dev *vpu)
-{
- int i;
-
- for (i = 0; i < 3; i++) {
- u32 status;
-
- /* Make sure the VPU is idle */
- status = vdpu_read(vpu, G2_REG_INTERRUPT);
- if (status & G2_REG_INTERRUPT_DEC_E) {
- dev_warn(vpu->dev, "device still running, aborting");
- status |= G2_REG_INTERRUPT_DEC_ABORT_E | G2_REG_INTERRUPT_DEC_IRQ_DIS;
- vdpu_write(vpu, status, G2_REG_INTERRUPT);
- }
- }
-}
-
int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx)
{
struct hantro_dev *vpu = ctx->dev;
diff --git a/drivers/staging/media/hantro/hantro_g2_regs.h b/drivers/staging/media/hantro/hantro_g2_regs.h
index 24b18f839ff8..136ba6d98a1f 100644
--- a/drivers/staging/media/hantro/hantro_g2_regs.h
+++ b/drivers/staging/media/hantro/hantro_g2_regs.h
@@ -27,6 +27,13 @@
#define G2_REG_INTERRUPT_DEC_IRQ_DIS BIT(4)
#define G2_REG_INTERRUPT_DEC_E BIT(0)
+#define HEVC_DEC_MODE 0xc
+
+#define BUS_WIDTH_32 0
+#define BUS_WIDTH_64 1
+#define BUS_WIDTH_128 2
+#define BUS_WIDTH_256 3
+
#define g2_strm_swap G2_DEC_REG(2, 28, 0xf)
#define g2_dirmv_swap G2_DEC_REG(2, 20, 0xf)
diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
index 4323e63dfbfc..42b3f3961f75 100644
--- a/drivers/staging/media/hantro/hantro_hw.h
+++ b/drivers/staging/media/hantro/hantro_hw.h
@@ -308,4 +308,6 @@ void hantro_vp8_dec_exit(struct hantro_ctx *ctx);
void hantro_vp8_prob_update(struct hantro_ctx *ctx,
const struct v4l2_ctrl_vp8_frame *hdr);
+void hantro_g2_check_idle(struct hantro_dev *vpu);
+
#endif /* HANTRO_HW_H_ */
--
2.17.1
From: Ezequiel Garcia <[email protected]>
When the post-processor hardware block is enabled, the driver
allocates an internal queue of buffers for the decoder enginer,
and uses the vb2 queue for the post-processor engine.
For instance, on a G1 core, the decoder engine produces NV12 buffers
and the post-processor engine can produce YUY2 buffers. The decoder
engine expects motion vectors to be appended to the NV12 buffers,
but this is only required for CODECs that need motion vectors,
such as H.264.
Fix the post-processor logic accordingly.
Signed-off-by: Ezequiel Garcia <[email protected]>
Signed-off-by: Andrzej Pietrasiewicz <[email protected]>
---
drivers/staging/media/hantro/hantro_postproc.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro_postproc.c b/drivers/staging/media/hantro/hantro_postproc.c
index ed8916c950a4..07842152003f 100644
--- a/drivers/staging/media/hantro/hantro_postproc.c
+++ b/drivers/staging/media/hantro/hantro_postproc.c
@@ -132,9 +132,10 @@ int hantro_postproc_alloc(struct hantro_ctx *ctx)
unsigned int num_buffers = cap_queue->num_buffers;
unsigned int i, buf_size;
- buf_size = ctx->dst_fmt.plane_fmt[0].sizeimage +
- hantro_h264_mv_size(ctx->dst_fmt.width,
- ctx->dst_fmt.height);
+ buf_size = ctx->dst_fmt.plane_fmt[0].sizeimage;
+ if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_H264_SLICE)
+ buf_size += hantro_h264_mv_size(ctx->dst_fmt.width,
+ ctx->dst_fmt.height);
for (i = 0; i < num_buffers; ++i) {
struct hantro_aux_buf *priv = &ctx->postproc.dec_q[i];
--
2.17.1
From: Ezequiel Garcia <[email protected]>
Add a 'postprocessed' boolean property to struct hantro_fmt
to signal that a format is produced by the post-processor.
This will allow to introduce the G2 post-processor in a simple way.
Signed-off-by: Ezequiel Garcia <[email protected]>
Signed-off-by: Andrzej Pietrasiewicz <[email protected]>
---
drivers/staging/media/hantro/hantro.h | 2 ++
drivers/staging/media/hantro/hantro_postproc.c | 8 +-------
drivers/staging/media/hantro/imx8m_vpu_hw.c | 1 +
drivers/staging/media/hantro/rockchip_vpu_hw.c | 1 +
drivers/staging/media/hantro/sama5d4_vdec_hw.c | 1 +
5 files changed, 6 insertions(+), 7 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro.h b/drivers/staging/media/hantro/hantro.h
index c2e01959dc00..dd5e56765d4e 100644
--- a/drivers/staging/media/hantro/hantro.h
+++ b/drivers/staging/media/hantro/hantro.h
@@ -263,6 +263,7 @@ struct hantro_ctx {
* @max_depth: Maximum depth, for bitstream formats
* @enc_fmt: Format identifier for encoder registers.
* @frmsize: Supported range of frame sizes (only for bitstream formats).
+ * @postprocessed: Indicates if this format needs the post-processor.
*/
struct hantro_fmt {
char *name;
@@ -272,6 +273,7 @@ struct hantro_fmt {
int max_depth;
enum hantro_enc_fmt enc_fmt;
struct v4l2_frmsize_stepwise frmsize;
+ bool postprocessed;
};
struct hantro_reg {
diff --git a/drivers/staging/media/hantro/hantro_postproc.c b/drivers/staging/media/hantro/hantro_postproc.c
index 882fb8bc5ddd..4549aec08feb 100644
--- a/drivers/staging/media/hantro/hantro_postproc.c
+++ b/drivers/staging/media/hantro/hantro_postproc.c
@@ -53,15 +53,9 @@ const struct hantro_postproc_regs hantro_g1_postproc_regs = {
bool hantro_needs_postproc(const struct hantro_ctx *ctx,
const struct hantro_fmt *fmt)
{
- struct hantro_dev *vpu = ctx->dev;
-
if (ctx->is_encoder)
return false;
-
- if (!vpu->variant->postproc_fmts)
- return false;
-
- return fmt->fourcc != V4L2_PIX_FMT_NV12;
+ return fmt->postprocessed;
}
static void hantro_postproc_g1_enable(struct hantro_ctx *ctx)
diff --git a/drivers/staging/media/hantro/imx8m_vpu_hw.c b/drivers/staging/media/hantro/imx8m_vpu_hw.c
index 22fa7d2f3b64..02e61438220a 100644
--- a/drivers/staging/media/hantro/imx8m_vpu_hw.c
+++ b/drivers/staging/media/hantro/imx8m_vpu_hw.c
@@ -82,6 +82,7 @@ static const struct hantro_fmt imx8m_vpu_postproc_fmts[] = {
{
.fourcc = V4L2_PIX_FMT_YUYV,
.codec_mode = HANTRO_MODE_NONE,
+ .postprocessed = true,
},
};
diff --git a/drivers/staging/media/hantro/rockchip_vpu_hw.c b/drivers/staging/media/hantro/rockchip_vpu_hw.c
index 6c1ad5534ce5..f372f767d4ff 100644
--- a/drivers/staging/media/hantro/rockchip_vpu_hw.c
+++ b/drivers/staging/media/hantro/rockchip_vpu_hw.c
@@ -62,6 +62,7 @@ static const struct hantro_fmt rockchip_vpu1_postproc_fmts[] = {
{
.fourcc = V4L2_PIX_FMT_YUYV,
.codec_mode = HANTRO_MODE_NONE,
+ .postprocessed = true,
},
};
diff --git a/drivers/staging/media/hantro/sama5d4_vdec_hw.c b/drivers/staging/media/hantro/sama5d4_vdec_hw.c
index f3fecc7248c4..b2fc1c5613e1 100644
--- a/drivers/staging/media/hantro/sama5d4_vdec_hw.c
+++ b/drivers/staging/media/hantro/sama5d4_vdec_hw.c
@@ -15,6 +15,7 @@ static const struct hantro_fmt sama5d4_vdec_postproc_fmts[] = {
{
.fourcc = V4L2_PIX_FMT_YUYV,
.codec_mode = HANTRO_MODE_NONE,
+ .postprocessed = true,
},
};
--
2.17.1
From: Ezequiel Garcia <[email protected]>
Turns out the post-processor block on the G2 core is substantially
different from the one on the G1 core. Introduce hantro_postproc_ops
with .enable and .disable methods, which will allow to support
the G2 post-processor cleanly.
Signed-off-by: Ezequiel Garcia <[email protected]>
Signed-off-by: Andrzej Pietrasiewicz <[email protected]>
Reviewed-by: Benjamin Gaignard <[email protected]>
---
drivers/staging/media/hantro/hantro.h | 5 +--
drivers/staging/media/hantro/hantro_hw.h | 13 +++++++-
.../staging/media/hantro/hantro_postproc.c | 33 ++++++++++++++-----
drivers/staging/media/hantro/imx8m_vpu_hw.c | 2 +-
.../staging/media/hantro/rockchip_vpu_hw.c | 6 ++--
.../staging/media/hantro/sama5d4_vdec_hw.c | 2 +-
6 files changed, 44 insertions(+), 17 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro.h b/drivers/staging/media/hantro/hantro.h
index c2e2dca38628..c2e01959dc00 100644
--- a/drivers/staging/media/hantro/hantro.h
+++ b/drivers/staging/media/hantro/hantro.h
@@ -28,6 +28,7 @@
struct hantro_ctx;
struct hantro_codec_ops;
+struct hantro_postproc_ops;
#define HANTRO_JPEG_ENCODER BIT(0)
#define HANTRO_ENCODERS 0x0000ffff
@@ -59,6 +60,7 @@ struct hantro_irq {
* @num_dec_fmts: Number of decoder formats.
* @postproc_fmts: Post-processor formats.
* @num_postproc_fmts: Number of post-processor formats.
+ * @postproc_ops: Post-processor ops.
* @codec: Supported codecs
* @codec_ops: Codec ops.
* @init: Initialize hardware, optional.
@@ -69,7 +71,6 @@ struct hantro_irq {
* @num_clocks: number of clocks in the array
* @reg_names: array of register range names
* @num_regs: number of register range names in the array
- * @postproc_regs: &struct hantro_postproc_regs pointer
*/
struct hantro_variant {
unsigned int enc_offset;
@@ -80,6 +81,7 @@ struct hantro_variant {
unsigned int num_dec_fmts;
const struct hantro_fmt *postproc_fmts;
unsigned int num_postproc_fmts;
+ const struct hantro_postproc_ops *postproc_ops;
unsigned int codec;
const struct hantro_codec_ops *codec_ops;
int (*init)(struct hantro_dev *vpu);
@@ -90,7 +92,6 @@ struct hantro_variant {
int num_clocks;
const char * const *reg_names;
int num_regs;
- const struct hantro_postproc_regs *postproc_regs;
};
/**
diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
index df7b5e3a57b9..4323e63dfbfc 100644
--- a/drivers/staging/media/hantro/hantro_hw.h
+++ b/drivers/staging/media/hantro/hantro_hw.h
@@ -170,6 +170,17 @@ struct hantro_postproc_ctx {
struct hantro_aux_buf dec_q[VB2_MAX_FRAME];
};
+/**
+ * struct hantro_postproc_ops - post-processor operations
+ *
+ * @enable: Enable the post-processor block. Optional.
+ * @disable: Disable the post-processor block. Optional.
+ */
+struct hantro_postproc_ops {
+ void (*enable)(struct hantro_ctx *ctx);
+ void (*disable)(struct hantro_ctx *ctx);
+};
+
/**
* struct hantro_codec_ops - codec mode specific operations
*
@@ -217,7 +228,7 @@ extern const struct hantro_variant rk3328_vpu_variant;
extern const struct hantro_variant rk3399_vpu_variant;
extern const struct hantro_variant sama5d4_vdec_variant;
-extern const struct hantro_postproc_regs hantro_g1_postproc_regs;
+extern const struct hantro_postproc_ops hantro_g1_postproc_ops;
extern const u32 hantro_vp8_dec_mc_filter[8][6];
diff --git a/drivers/staging/media/hantro/hantro_postproc.c b/drivers/staging/media/hantro/hantro_postproc.c
index 07842152003f..882fb8bc5ddd 100644
--- a/drivers/staging/media/hantro/hantro_postproc.c
+++ b/drivers/staging/media/hantro/hantro_postproc.c
@@ -15,14 +15,14 @@
#define HANTRO_PP_REG_WRITE(vpu, reg_name, val) \
{ \
hantro_reg_write(vpu, \
- &(vpu)->variant->postproc_regs->reg_name, \
+ &hantro_g1_postproc_regs.reg_name, \
val); \
}
#define HANTRO_PP_REG_WRITE_S(vpu, reg_name, val) \
{ \
hantro_reg_write_s(vpu, \
- &(vpu)->variant->postproc_regs->reg_name, \
+ &hantro_g1_postproc_regs.reg_name, \
val); \
}
@@ -64,16 +64,13 @@ bool hantro_needs_postproc(const struct hantro_ctx *ctx,
return fmt->fourcc != V4L2_PIX_FMT_NV12;
}
-void hantro_postproc_enable(struct hantro_ctx *ctx)
+static void hantro_postproc_g1_enable(struct hantro_ctx *ctx)
{
struct hantro_dev *vpu = ctx->dev;
struct vb2_v4l2_buffer *dst_buf;
u32 src_pp_fmt, dst_pp_fmt;
dma_addr_t dst_dma;
- if (!vpu->variant->postproc_regs)
- return;
-
/* Turn on pipeline mode. Must be done first. */
HANTRO_PP_REG_WRITE_S(vpu, pipeline_en, 0x1);
@@ -154,12 +151,30 @@ int hantro_postproc_alloc(struct hantro_ctx *ctx)
return 0;
}
+static void hantro_postproc_g1_disable(struct hantro_ctx *ctx)
+{
+ struct hantro_dev *vpu = ctx->dev;
+
+ HANTRO_PP_REG_WRITE_S(vpu, pipeline_en, 0x0);
+}
+
void hantro_postproc_disable(struct hantro_ctx *ctx)
{
struct hantro_dev *vpu = ctx->dev;
- if (!vpu->variant->postproc_regs)
- return;
+ if (vpu->variant->postproc_ops && vpu->variant->postproc_ops->disable)
+ vpu->variant->postproc_ops->disable(ctx);
+}
- HANTRO_PP_REG_WRITE_S(vpu, pipeline_en, 0x0);
+void hantro_postproc_enable(struct hantro_ctx *ctx)
+{
+ struct hantro_dev *vpu = ctx->dev;
+
+ if (vpu->variant->postproc_ops && vpu->variant->postproc_ops->enable)
+ vpu->variant->postproc_ops->enable(ctx);
}
+
+const struct hantro_postproc_ops hantro_g1_postproc_ops = {
+ .enable = hantro_postproc_g1_enable,
+ .disable = hantro_postproc_g1_disable,
+};
diff --git a/drivers/staging/media/hantro/imx8m_vpu_hw.c b/drivers/staging/media/hantro/imx8m_vpu_hw.c
index ea919bfb9891..22fa7d2f3b64 100644
--- a/drivers/staging/media/hantro/imx8m_vpu_hw.c
+++ b/drivers/staging/media/hantro/imx8m_vpu_hw.c
@@ -262,7 +262,7 @@ const struct hantro_variant imx8mq_vpu_variant = {
.num_dec_fmts = ARRAY_SIZE(imx8m_vpu_dec_fmts),
.postproc_fmts = imx8m_vpu_postproc_fmts,
.num_postproc_fmts = ARRAY_SIZE(imx8m_vpu_postproc_fmts),
- .postproc_regs = &hantro_g1_postproc_regs,
+ .postproc_ops = &hantro_g1_postproc_ops,
.codec = HANTRO_MPEG2_DECODER | HANTRO_VP8_DECODER |
HANTRO_H264_DECODER,
.codec_ops = imx8mq_vpu_codec_ops,
diff --git a/drivers/staging/media/hantro/rockchip_vpu_hw.c b/drivers/staging/media/hantro/rockchip_vpu_hw.c
index d4f52957cc53..6c1ad5534ce5 100644
--- a/drivers/staging/media/hantro/rockchip_vpu_hw.c
+++ b/drivers/staging/media/hantro/rockchip_vpu_hw.c
@@ -460,7 +460,7 @@ const struct hantro_variant rk3036_vpu_variant = {
.num_dec_fmts = ARRAY_SIZE(rk3066_vpu_dec_fmts),
.postproc_fmts = rockchip_vpu1_postproc_fmts,
.num_postproc_fmts = ARRAY_SIZE(rockchip_vpu1_postproc_fmts),
- .postproc_regs = &hantro_g1_postproc_regs,
+ .postproc_ops = &hantro_g1_postproc_ops,
.codec = HANTRO_MPEG2_DECODER | HANTRO_VP8_DECODER |
HANTRO_H264_DECODER,
.codec_ops = rk3036_vpu_codec_ops,
@@ -485,7 +485,7 @@ const struct hantro_variant rk3066_vpu_variant = {
.num_dec_fmts = ARRAY_SIZE(rk3066_vpu_dec_fmts),
.postproc_fmts = rockchip_vpu1_postproc_fmts,
.num_postproc_fmts = ARRAY_SIZE(rockchip_vpu1_postproc_fmts),
- .postproc_regs = &hantro_g1_postproc_regs,
+ .postproc_ops = &hantro_g1_postproc_ops,
.codec = HANTRO_JPEG_ENCODER | HANTRO_MPEG2_DECODER |
HANTRO_VP8_DECODER | HANTRO_H264_DECODER,
.codec_ops = rk3066_vpu_codec_ops,
@@ -505,7 +505,7 @@ const struct hantro_variant rk3288_vpu_variant = {
.num_dec_fmts = ARRAY_SIZE(rk3288_vpu_dec_fmts),
.postproc_fmts = rockchip_vpu1_postproc_fmts,
.num_postproc_fmts = ARRAY_SIZE(rockchip_vpu1_postproc_fmts),
- .postproc_regs = &hantro_g1_postproc_regs,
+ .postproc_ops = &hantro_g1_postproc_ops,
.codec = HANTRO_JPEG_ENCODER | HANTRO_MPEG2_DECODER |
HANTRO_VP8_DECODER | HANTRO_H264_DECODER,
.codec_ops = rk3288_vpu_codec_ops,
diff --git a/drivers/staging/media/hantro/sama5d4_vdec_hw.c b/drivers/staging/media/hantro/sama5d4_vdec_hw.c
index 9c3b8cd0b239..f3fecc7248c4 100644
--- a/drivers/staging/media/hantro/sama5d4_vdec_hw.c
+++ b/drivers/staging/media/hantro/sama5d4_vdec_hw.c
@@ -100,7 +100,7 @@ const struct hantro_variant sama5d4_vdec_variant = {
.num_dec_fmts = ARRAY_SIZE(sama5d4_vdec_fmts),
.postproc_fmts = sama5d4_vdec_postproc_fmts,
.num_postproc_fmts = ARRAY_SIZE(sama5d4_vdec_postproc_fmts),
- .postproc_regs = &hantro_g1_postproc_regs,
+ .postproc_ops = &hantro_g1_postproc_ops,
.codec = HANTRO_MPEG2_DECODER | HANTRO_VP8_DECODER |
HANTRO_H264_DECODER,
.codec_ops = sama5d4_vdec_codec_ops,
--
2.17.1
The G2 decoder block produces NV12 4x4 tiled format (NV12_4L4).
Enable the G2 post-processor block, in order to produce regular NV12.
The logic in hantro_postproc.c is leveraged to take care of allocating
the extra buffers and configure the post-processor, which is
significantly simpler than the one on the G1.
Signed-off-by: Ezequiel Garcia <[email protected]>
Signed-off-by: Andrzej Pietrasiewicz <[email protected]>
---
.../staging/media/hantro/hantro_g2_vp9_dec.c | 6 ++--
drivers/staging/media/hantro/hantro_hw.h | 1 +
.../staging/media/hantro/hantro_postproc.c | 31 +++++++++++++++++++
drivers/staging/media/hantro/imx8m_vpu_hw.c | 11 +++++++
4 files changed, 46 insertions(+), 3 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro_g2_vp9_dec.c b/drivers/staging/media/hantro/hantro_g2_vp9_dec.c
index 7f827b9f0133..1a26be72c878 100644
--- a/drivers/staging/media/hantro/hantro_g2_vp9_dec.c
+++ b/drivers/staging/media/hantro/hantro_g2_vp9_dec.c
@@ -152,7 +152,7 @@ static void config_output(struct hantro_ctx *ctx,
hantro_reg_write(ctx->dev, &g2_out_dis, 0);
hantro_reg_write(ctx->dev, &g2_output_format, 0);
- luma_addr = vb2_dma_contig_plane_dma_addr(&dst->base.vb.vb2_buf, 0);
+ luma_addr = hantro_get_dec_buf_addr(ctx, &dst->base.vb.vb2_buf);
hantro_write_addr(ctx->dev, G2_OUT_LUMA_ADDR, luma_addr);
chroma_addr = luma_addr + chroma_offset(ctx, dec_params);
@@ -191,7 +191,7 @@ static void config_ref(struct hantro_ctx *ctx,
hantro_reg_write(ctx->dev, &ref_reg->hor_scale, (refw << 14) / dst->vp9.width);
hantro_reg_write(ctx->dev, &ref_reg->ver_scale, (refh << 14) / dst->vp9.height);
- luma_addr = vb2_dma_contig_plane_dma_addr(&buf->base.vb.vb2_buf, 0);
+ luma_addr = hantro_get_dec_buf_addr(ctx, &buf->base.vb.vb2_buf);
hantro_write_addr(ctx->dev, ref_reg->y_base, luma_addr);
chroma_addr = luma_addr + chroma_offset(ctx, dec_params);
@@ -236,7 +236,7 @@ static void config_ref_registers(struct hantro_ctx *ctx,
config_ref(ctx, dst, &ref_regs[1], dec_params, dec_params->golden_frame_ts);
config_ref(ctx, dst, &ref_regs[2], dec_params, dec_params->alt_frame_ts);
- mv_addr = vb2_dma_contig_plane_dma_addr(&mv_ref->base.vb.vb2_buf, 0) +
+ mv_addr = hantro_get_dec_buf_addr(ctx, &mv_ref->base.vb.vb2_buf) +
mv_offset(ctx, dec_params);
hantro_write_addr(ctx->dev, G2_REF_MV_ADDR(0), mv_addr);
diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
index 2961d399fd60..3d4a5dc1e6d5 100644
--- a/drivers/staging/media/hantro/hantro_hw.h
+++ b/drivers/staging/media/hantro/hantro_hw.h
@@ -274,6 +274,7 @@ extern const struct hantro_variant rk3399_vpu_variant;
extern const struct hantro_variant sama5d4_vdec_variant;
extern const struct hantro_postproc_ops hantro_g1_postproc_ops;
+extern const struct hantro_postproc_ops hantro_g2_postproc_ops;
extern const u32 hantro_vp8_dec_mc_filter[8][6];
diff --git a/drivers/staging/media/hantro/hantro_postproc.c b/drivers/staging/media/hantro/hantro_postproc.c
index 4549aec08feb..79a66d001738 100644
--- a/drivers/staging/media/hantro/hantro_postproc.c
+++ b/drivers/staging/media/hantro/hantro_postproc.c
@@ -11,6 +11,7 @@
#include "hantro.h"
#include "hantro_hw.h"
#include "hantro_g1_regs.h"
+#include "hantro_g2_regs.h"
#define HANTRO_PP_REG_WRITE(vpu, reg_name, val) \
{ \
@@ -99,6 +100,21 @@ static void hantro_postproc_g1_enable(struct hantro_ctx *ctx)
HANTRO_PP_REG_WRITE(vpu, display_width, ctx->dst_fmt.width);
}
+static void hantro_postproc_g2_enable(struct hantro_ctx *ctx)
+{
+ struct hantro_dev *vpu = ctx->dev;
+ struct vb2_v4l2_buffer *dst_buf;
+ size_t chroma_offset = ctx->dst_fmt.width * ctx->dst_fmt.height;
+ dma_addr_t dst_dma;
+
+ dst_buf = hantro_get_dst_buf(ctx);
+ dst_dma = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0);
+
+ hantro_write_addr(vpu, G2_RS_OUT_LUMA_ADDR, dst_dma);
+ hantro_write_addr(vpu, G2_RS_OUT_CHROMA_ADDR, dst_dma + chroma_offset);
+ hantro_reg_write(vpu, &g2_out_rs_e, 1);
+}
+
void hantro_postproc_free(struct hantro_ctx *ctx)
{
struct hantro_dev *vpu = ctx->dev;
@@ -127,6 +143,9 @@ int hantro_postproc_alloc(struct hantro_ctx *ctx)
if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_H264_SLICE)
buf_size += hantro_h264_mv_size(ctx->dst_fmt.width,
ctx->dst_fmt.height);
+ else if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_VP9_FRAME)
+ buf_size += hantro_vp9_mv_size(ctx->dst_fmt.width,
+ ctx->dst_fmt.height);
for (i = 0; i < num_buffers; ++i) {
struct hantro_aux_buf *priv = &ctx->postproc.dec_q[i];
@@ -152,6 +171,13 @@ static void hantro_postproc_g1_disable(struct hantro_ctx *ctx)
HANTRO_PP_REG_WRITE_S(vpu, pipeline_en, 0x0);
}
+static void hantro_postproc_g2_disable(struct hantro_ctx *ctx)
+{
+ struct hantro_dev *vpu = ctx->dev;
+
+ hantro_reg_write(vpu, &g2_out_rs_e, 0);
+}
+
void hantro_postproc_disable(struct hantro_ctx *ctx)
{
struct hantro_dev *vpu = ctx->dev;
@@ -172,3 +198,8 @@ const struct hantro_postproc_ops hantro_g1_postproc_ops = {
.enable = hantro_postproc_g1_enable,
.disable = hantro_postproc_g1_disable,
};
+
+const struct hantro_postproc_ops hantro_g2_postproc_ops = {
+ .enable = hantro_postproc_g2_enable,
+ .disable = hantro_postproc_g2_disable,
+};
diff --git a/drivers/staging/media/hantro/imx8m_vpu_hw.c b/drivers/staging/media/hantro/imx8m_vpu_hw.c
index 455a107ffb02..1a43f6fceef9 100644
--- a/drivers/staging/media/hantro/imx8m_vpu_hw.c
+++ b/drivers/staging/media/hantro/imx8m_vpu_hw.c
@@ -132,6 +132,14 @@ static const struct hantro_fmt imx8m_vpu_dec_fmts[] = {
},
};
+static const struct hantro_fmt imx8m_vpu_g2_postproc_fmts[] = {
+ {
+ .fourcc = V4L2_PIX_FMT_NV12,
+ .codec_mode = HANTRO_MODE_NONE,
+ .postprocessed = true,
+ },
+};
+
static const struct hantro_fmt imx8m_vpu_g2_dec_fmts[] = {
{
.fourcc = V4L2_PIX_FMT_NV12_4L4,
@@ -301,6 +309,9 @@ const struct hantro_variant imx8mq_vpu_g2_variant = {
.dec_offset = 0x0,
.dec_fmts = imx8m_vpu_g2_dec_fmts,
.num_dec_fmts = ARRAY_SIZE(imx8m_vpu_g2_dec_fmts),
+ .postproc_fmts = imx8m_vpu_g2_postproc_fmts,
+ .num_postproc_fmts = ARRAY_SIZE(imx8m_vpu_g2_postproc_fmts),
+ .postproc_ops = &hantro_g2_postproc_ops,
.codec = HANTRO_HEVC_DECODER | HANTRO_VP9_DECODER,
.codec_ops = imx8mq_vpu_g2_codec_ops,
.init = imx8mq_vpu_hw_init,
--
2.17.1
Add the VP9 stateless decoder controls plus the documentation that goes
with it.
Signed-off-by: Boris Brezillon <[email protected]>
Co-developed-by: Ezequiel Garcia <[email protected]>
Signed-off-by: Ezequiel Garcia <[email protected]>
Signed-off-by: Adrian Ratiu <[email protected]>
Signed-off-by: Andrzej Pietrasiewicz <[email protected]>
Co-developed-by: Daniel Almeida <[email protected]>
Signed-off-by: Daniel Almeida <[email protected]>
---
.../userspace-api/media/v4l/biblio.rst | 10 +
.../media/v4l/ext-ctrls-codec-stateless.rst | 573 ++++++++++++++++++
.../media/v4l/pixfmt-compressed.rst | 15 +
.../media/v4l/vidioc-g-ext-ctrls.rst | 8 +
.../media/v4l/vidioc-queryctrl.rst | 12 +
.../media/videodev2.h.rst.exceptions | 2 +
drivers/media/v4l2-core/v4l2-ctrls-core.c | 180 ++++++
drivers/media/v4l2-core/v4l2-ctrls-defs.c | 8 +
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/media/v4l2-ctrls.h | 4 +
include/uapi/linux/v4l2-controls.h | 284 +++++++++
include/uapi/linux/videodev2.h | 6 +
12 files changed, 1103 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/biblio.rst b/Documentation/userspace-api/media/v4l/biblio.rst
index 7b8e6738ff9e..9cd18c153d19 100644
--- a/Documentation/userspace-api/media/v4l/biblio.rst
+++ b/Documentation/userspace-api/media/v4l/biblio.rst
@@ -417,3 +417,13 @@ VP8
:title: RFC 6386: "VP8 Data Format and Decoding Guide"
:author: J. Bankoski et al.
+
+.. _vp9:
+
+VP9
+===
+
+
+:title: VP9 Bitstream & Decoding Process Specification
+
+:author: Adrian Grange (Google), Peter de Rivaz (Argon Design), Jonathan Hunt (Argon Design)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
index 72f5e85b4f34..cc080c4257d0 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
@@ -1458,3 +1458,576 @@ FWHT Flags
.. raw:: latex
\normalsize
+
+.. _v4l2-codec-stateless-vp9:
+
+``V4L2_CID_STATELESS_VP9_COMPRESSED_HDR (struct)``
+ Stores VP9 probabilities updates as parsed from the current compressed frame
+ header. A value of zero in an array element means no update of the relevant
+ probability. Motion vector-related updates contain a new value or zero. All
+ other updates contain values translated with inv_map_table[] (see 6.3.5 in
+ :ref:`vp9`).
+
+.. c:type:: v4l2_ctrl_vp9_compressed_hdr
+
+.. tabularcolumns:: |p{1cm}|p{4.8cm}|p{11.4cm}|
+
+.. cssclass:: longtable
+
+.. flat-table:: struct v4l2_ctrl_vp9_compressed_hdr
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - __u8
+ - ``tx_mode``
+ - Specifies the TX mode. See :ref:`TX Mode <vp9_tx_mode>` for more details.
+ * - __u8
+ - ``tx8[2][1]``
+ - TX 8x8 probabilities delta.
+ * - __u8
+ - ``tx16[2][2]``
+ - TX 16x16 probabilities delta.
+ * - __u8
+ - ``tx32[2][3]``
+ - TX 32x32 probabilities delta.
+ * - __u8
+ - ``coef[4][2][2][6][6][3]``
+ - Coefficient probabilities delta.
+ * - __u8
+ - ``skip[3]``
+ - Skip probabilities delta.
+ * - __u8
+ - ``inter_mode[7][3]``
+ - Inter prediction mode probabilities delta.
+ * - __u8
+ - ``interp_filter[4][2]``
+ - Interpolation filter probabilities delta.
+ * - __u8
+ - ``is_inter[4]``
+ - Is inter-block probabilities delta.
+ * - __u8
+ - ``comp_mode[5]``
+ - Compound prediction mode probabilities delta.
+ * - __u8
+ - ``single_ref[5][2]``
+ - Single reference probabilities delta.
+ * - __u8
+ - ``comp_ref[5]``
+ - Compound reference probabilities delta.
+ * - __u8
+ - ``y_mode[4][9]``
+ - Y prediction mode probabilities delta.
+ * - __u8
+ - ``uv_mode[10][9]``
+ - UV prediction mode probabilities delta.
+ * - __u8
+ - ``partition[16][3]``
+ - Partition probabilities delta.
+ * - __u8
+ - ``mv.joint[3]``
+ - Motion vector joint probabilities delta.
+ * - __u8
+ - ``mv.sign[2]``
+ - Motion vector sign probabilities delta.
+ * - __u8
+ - ``mv.classes[2][10]``
+ - Motion vector class probabilities delta.
+ * - __u8
+ - ``mv.class0_bit[2]``
+ - Motion vector class0 bit probabilities delta.
+ * - __u8
+ - ``mv.bits[2][10]``
+ - Motion vector bits probabilities delta.
+ * - __u8
+ - ``mv.class0_fr[2][2][3]``
+ - Motion vector class0 fractional bit probabilities delta.
+ * - __u8
+ - ``mv.fr[2][3]``
+ - Motion vector fractional bit probabilities delta.
+ * - __u8
+ - ``mv.class0_hp[2]``
+ - Motion vector class0 high precision fractional bit probabilities delta.
+ * - __u8
+ - ``mv.hp[2]``
+ - Motion vector high precision fractional bit probabilities delta.
+
+.. _vp9_tx_mode:
+
+``TX Mode``
+
+.. tabularcolumns:: |p{6.5cm}|p{0.5cm}|p{10.3cm}|
+
+.. flat-table::
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - ``V4L2_VP9_TX_MODE_ONLY_4X4``
+ - 0
+ - Transform size is 4x4.
+ * - ``V4L2_VP9_TX_MODE_ALLOW_8X8``
+ - 1
+ - Transform size can be up to 8x8.
+ * - ``V4L2_VP9_TX_MODE_ALLOW_16X16``
+ - 2
+ - Transform size can be up to 16x16.
+ * - ``V4L2_VP9_TX_MODE_ALLOW_32X32``
+ - 3
+ - transform size can be up to 32x32.
+ * - ``V4L2_VP9_TX_MODE_SELECT``
+ - 4
+ - Bitstream contains the transform size for each block.
+
+See section '7.3.1 Tx mode semantics' of the :ref:`vp9` specification for more details.
+
+``V4L2_CID_STATELESS_VP9_FRAME (struct)``
+ Specifies the frame parameters for the associated VP9 frame decode request.
+ This includes the necessary parameters for configuring a stateless hardware
+ decoding pipeline for VP9. The bitstream parameters are defined according
+ to :ref:`vp9`.
+
+.. c:type:: v4l2_ctrl_vp9_frame
+
+.. raw:: latex
+
+ \small
+
+.. tabularcolumns:: |p{4.7cm}|p{5.5cm}|p{7.1cm}|
+
+.. cssclass:: longtable
+
+.. flat-table:: struct v4l2_ctrl_vp9_frame
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - struct :c:type:`v4l2_vp9_loop_filter`
+ - ``lf``
+ - Loop filter parameters. See struct :c:type:`v4l2_vp9_loop_filter` for more details.
+ * - struct :c:type:`v4l2_vp9_quantization`
+ - ``quant``
+ - Quantization parameters. See :c:type:`v4l2_vp9_quantization` for more details.
+ * - struct :c:type:`v4l2_vp9_segmentation`
+ - ``seg``
+ - Segmentation parameters. See :c:type:`v4l2_vp9_segmentation` for more details.
+ * - __u32
+ - ``flags``
+ - Combination of V4L2_VP9_FRAME_FLAG_* flags. See :ref:`Frame Flags<vp9_frame_flags>`.
+ * - __u16
+ - ``compressed_header_size``
+ - Compressed header size in bytes.
+ * - __u16
+ - ``uncompressed_header_size``
+ - Uncompressed header size in bytes.
+ * - __u16
+ - ``frame_width_minus_1``
+ - Add 1 to get the frame width expressed in pixels. See section 7.2.3 in :ref:`vp9`.
+ * - __u16
+ - ``frame_height_minus_1``
+ - Add 1 to get the frame height expressed in pixels. See section 7.2.3 in :ref:`vp9`.
+ * - __u16
+ - ``render_width_minus_1``
+ - Add 1 to get the expected render width expressed in pixels. This is
+ not used during the decoding process but might be used by HW scalers to
+ prepare a frame that's ready for scanout. See section 7.2.4 in :ref:`vp9`.
+ * - __u16
+ - render_height_minus_1
+ - Add 1 to get the expected render height expressed in pixels. This is
+ not used during the decoding process but might be used by HW scalers to
+ prepare a frame that's ready for scanout. See section 7.2.4 in :ref:`vp9`.
+ * - __u64
+ - ``last_frame_ts``
+ - "last" reference buffer timestamp.
+ The timestamp refers to the ``timestamp`` field in
+ struct :c:type:`v4l2_buffer`. Use the :c:func:`v4l2_timeval_to_ns()`
+ function to convert the struct :c:type:`timeval` in struct
+ :c:type:`v4l2_buffer` to a __u64.
+ * - __u64
+ - ``golden_frame_ts``
+ - "golden" reference buffer timestamp.
+ The timestamp refers to the ``timestamp`` field in
+ struct :c:type:`v4l2_buffer`. Use the :c:func:`v4l2_timeval_to_ns()`
+ function to convert the struct :c:type:`timeval` in struct
+ :c:type:`v4l2_buffer` to a __u64.
+ * - __u64
+ - ``alt_frame_ts``
+ - "alt" reference buffer timestamp.
+ The timestamp refers to the ``timestamp`` field in
+ struct :c:type:`v4l2_buffer`. Use the :c:func:`v4l2_timeval_to_ns()`
+ function to convert the struct :c:type:`timeval` in struct
+ :c:type:`v4l2_buffer` to a __u64.
+ * - __u8
+ - ``ref_frame_sign_bias``
+ - a bitfield specifying whether the sign bias is set for a given
+ reference frame. See :ref:`Reference Frame Sign Bias<vp9_ref_frame_sign_bias>`
+ for more details.
+ * - __u8
+ - ``reset_frame_context``
+ - specifies whether the frame context should be reset to default values. See
+ :ref:`Reset Frame Context<vp9_reset_frame_context>` for more details.
+ * - __u8
+ - ``frame_context_idx``
+ - Frame context that should be used/updated.
+ * - __u8
+ - ``profile``
+ - VP9 profile. Can be 0, 1, 2 or 3.
+ * - __u8
+ - ``bit_depth``
+ - Component depth in bits. Can be 8, 10 or 12. Note that not all profiles
+ support 10 and/or 12 bits depths.
+ * - __u8
+ - ``interpolation_filter``
+ - Specifies the filter selection used for performing inter prediction. See
+ :ref:`Interpolation Filter<vp9_interpolation_filter>` for more details.
+ * - __u8
+ - ``tile_cols_log2``
+ - Specifies the base 2 logarithm of the width of each tile (where the
+ width is measured in units of 8x8 blocks). Shall be less than or equal
+ to 6.
+ * - __u8
+ - ``tile_rows_log2``
+ - Specifies the base 2 logarithm of the height of each tile (where the
+ height is measured in units of 8x8 blocks).
+ * - __u8
+ - ``reference_mode``
+ - Specifies the type of inter prediction to be used. See
+ :ref:`Reference Mode<vp9_reference_mode>` for more details.
+ * - __u8
+ - ``reserved[7]``
+ - Applications and drivers must set this to zero.
+
+.. raw:: latex
+
+ \normalsize
+
+.. _vp9_frame_flags:
+
+``Frame Flags``
+
+.. tabularcolumns:: |p{10.0cm}|p{1.2cm}|p{6.1cm}|
+
+.. flat-table::
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - ``V4L2_VP9_FRAME_FLAG_KEY_FRAME``
+ - 0x001
+ - The frame is a key frame.
+ * - ``V4L2_VP9_FRAME_FLAG_SHOW_FRAME``
+ - 0x002
+ - The frame should be displayed.
+ * - ``V4L2_VP9_FRAME_FLAG_ERROR_RESILIENT``
+ - 0x004
+ - The decoding should be error resilient.
+ * - ``V4L2_VP9_FRAME_FLAG_INTRA_ONLY``
+ - 0x008
+ - The frame does not reference other frames.
+ * - ``V4L2_VP9_FRAME_FLAG_ALLOW_HIGH_PREC_MV``
+ - 0x010
+ - The frame can use high precision motion vectors.
+ * - ``V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX``
+ - 0x020
+ - Frame context should be updated after decoding.
+ * - ``V4L2_VP9_FRAME_FLAG_PARALLEL_DEC_MODE``
+ - 0x040
+ - Parallel decoding is used.
+ * - ``V4L2_VP9_FRAME_FLAG_X_SUBSAMPLING``
+ - 0x080
+ - Vertical subsampling is enabled.
+ * - ``V4L2_VP9_FRAME_FLAG_Y_SUBSAMPLING``
+ - 0x100
+ - Horizontal subsampling is enabled.
+ * - ``V4L2_VP9_FRAME_FLAG_COLOR_RANGE_FULL_SWING``
+ - 0x200
+ - The full UV range is used.
+
+.. _vp9_ref_frame_sign_bias:
+
+``Reference Frame Sign Bias``
+
+.. tabularcolumns:: |p{7.0cm}|p{1.2cm}|p{9.1cm}|
+
+.. flat-table::
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - ``V4L2_VP9_SIGN_BIAS_LAST``
+ - 0x1
+ - Sign bias is set for the last reference frame.
+ * - ``V4L2_VP9_SIGN_BIAS_GOLDEN``
+ - 0x2
+ - Sign bias is set for the golden reference frame.
+ * - ``V4L2_VP9_SIGN_BIAS_ALT``
+ - 0x2
+ - Sign bias is set for the alt reference frame.
+
+.. _vp9_reset_frame_context:
+
+``Reset Frame Context``
+
+.. tabularcolumns:: |p{7.0cm}|p{1.2cm}|p{9.1cm}|
+
+.. flat-table::
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - ``V4L2_VP9_RESET_FRAME_CTX_NONE``
+ - 0
+ - Do not reset any frame context.
+ * - ``V4L2_VP9_RESET_FRAME_CTX_SPEC``
+ - 1
+ - Reset the frame context pointed to by
+ :c:type:`v4l2_ctrl_vp9_frame`.frame_context_idx.
+ * - ``V4L2_VP9_RESET_FRAME_CTX_ALL``
+ - 2
+ - Reset all frame contexts.
+
+See section '7.2 Uncompressed header semantics' of the :ref:`vp9` specification
+for more details.
+
+.. _vp9_interpolation_filter:
+
+``Interpolation Filter``
+
+.. tabularcolumns:: |p{9.0cm}|p{1.2cm}|p{7.1cm}|
+
+.. flat-table::
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - ``V4L2_VP9_INTERP_FILTER_EIGHTTAP``
+ - 0
+ - Eight tap filter.
+ * - ``V4L2_VP9_INTERP_FILTER_EIGHTTAP_SMOOTH``
+ - 1
+ - Eight tap smooth filter.
+ * - ``V4L2_VP9_INTERP_FILTER_EIGHTTAP_SHARP``
+ - 2
+ - Eeight tap sharp filter.
+ * - ``V4L2_VP9_INTERP_FILTER_BILINEAR``
+ - 3
+ - Bilinear filter.
+ * - ``V4L2_VP9_INTERP_FILTER_SWITCHABLE``
+ - 4
+ - Filter selection is signaled at the block level.
+
+See section '7.2.7 Interpolation filter semantics' of the :ref:`vp9` specification
+for more details.
+
+.. _vp9_reference_mode:
+
+``Reference Mode``
+
+.. tabularcolumns:: |p{9.6cm}|p{0.5cm}|p{7.2cm}|
+
+.. flat-table::
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - ``V4L2_VP9_REFERENCE_MODE_SINGLE_REFERENCE``
+ - 0
+ - Indicates that all the inter blocks use only a single reference frame
+ to generate motion compensated prediction.
+ * - ``V4L2_VP9_REFERENCE_MODE_COMPOUND_REFERENCE``
+ - 1
+ - Requires all the inter blocks to use compound mode. Single reference
+ frame prediction is not allowed.
+ * - ``V4L2_VP9_REFERENCE_MODE_SELECT``
+ - 2
+ - Allows each individual inter block to select between single and
+ compound prediction modes.
+
+See section '7.3.6 Frame reference mode semantics' of the :ref:`vp9` specification for more details.
+
+.. c:type:: v4l2_vp9_segmentation
+
+Encodes the quantization parameters. See section '7.2.10 Segmentation
+params syntax' of the :ref:`vp9` specification for more details.
+
+.. tabularcolumns:: |p{0.8cm}|p{5cm}|p{11.4cm}|
+
+.. cssclass:: longtable
+
+.. flat-table:: struct v4l2_vp9_segmentation
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - __u8
+ - ``feature_data[8][4]``
+ - Data attached to each feature. Data entry is only valid if the feature
+ is enabled. The array shall be indexed with segment number as the first dimension
+ (0..7) and one of V4L2_VP9_SEG_* as the second dimension.
+ See :ref:`Segment Feature IDs<vp9_segment_feature>`.
+ * - __u8
+ - ``feature_enabled[8]``
+ - Bitmask defining which features are enabled in each segment. The value for each
+ segment is a combination of V4L2_VP9_SEGMENT_FEATURE_ENABLED(id) values where id is
+ one of V4L2_VP9_SEG_*. See :ref:`Segment Feature IDs<vp9_segment_feature>`.
+ * - __u8
+ - ``tree_probs[7]``
+ - Specifies the probability values to be used when decoding a Segment-ID.
+ See '5.15. Segmentation map' section of :ref:`vp9` for more details.
+ * - __u8
+ - ``pred_probs[3]``
+ - Specifies the probability values to be used when decoding a
+ Predicted-Segment-ID. See '6.4.14. Get segment id syntax'
+ section of :ref:`vp9` for more details.
+ * - __u8
+ - ``flags``
+ - Combination of V4L2_VP9_SEGMENTATION_FLAG_* flags. See
+ :ref:`Segmentation Flags<vp9_segmentation_flags>`.
+ * - __u8
+ - ``reserved[5]``
+ - Applications and drivers must set this to zero.
+
+.. _vp9_segment_feature:
+
+``Segment feature IDs``
+
+.. tabularcolumns:: |p{6.0cm}|p{1cm}|p{10.3cm}|
+
+.. flat-table::
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - ``V4L2_VP9_SEG_LVL_ALT_Q``
+ - 0
+ - Quantizer segment feature.
+ * - ``V4L2_VP9_SEG_LVL_ALT_L``
+ - 1
+ - Loop filter segment feature.
+ * - ``V4L2_VP9_SEG_LVL_REF_FRAME``
+ - 2
+ - Reference frame segment feature.
+ * - ``V4L2_VP9_SEG_LVL_SKIP``
+ - 3
+ - Skip segment feature.
+ * - ``V4L2_VP9_SEG_LVL_MAX``
+ - 4
+ - Number of segment features.
+
+.. _vp9_segmentation_flags:
+
+``Segmentation Flags``
+
+.. tabularcolumns:: |p{10.6cm}|p{0.8cm}|p{5.9cm}|
+
+.. flat-table::
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - ``V4L2_VP9_SEGMENTATION_FLAG_ENABLED``
+ - 0x01
+ - Indicates that this frame makes use of the segmentation tool.
+ * - ``V4L2_VP9_SEGMENTATION_FLAG_UPDATE_MAP``
+ - 0x02
+ - Indicates that the segmentation map should be updated during the
+ decoding of this frame.
+ * - ``V4L2_VP9_SEGMENTATION_FLAG_TEMPORAL_UPDATE``
+ - 0x04
+ - Indicates that the updates to the segmentation map are coded
+ relative to the existing segmentation map.
+ * - ``V4L2_VP9_SEGMENTATION_FLAG_UPDATE_DATA``
+ - 0x08
+ - Indicates that new parameters are about to be specified for each
+ segment.
+ * - ``V4L2_VP9_SEGMENTATION_FLAG_ABS_OR_DELTA_UPDATE``
+ - 0x10
+ - Indicates that the segmentation parameters represent the actual values
+ to be used.
+
+.. c:type:: v4l2_vp9_quantization
+
+Encodes the quantization parameters. See section '7.2.9 Quantization params
+syntax' of the VP9 specification for more details.
+
+.. tabularcolumns:: |p{0.8cm}|p{4cm}|p{12.4cm}|
+
+.. cssclass:: longtable
+
+.. flat-table:: struct v4l2_vp9_quantization
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - __u8
+ - ``base_q_idx``
+ - Indicates the base frame qindex.
+ * - __s8
+ - ``delta_q_y_dc``
+ - Indicates the Y DC quantizer relative to base_q_idx.
+ * - __s8
+ - ``delta_q_uv_dc``
+ - Indicates the UV DC quantizer relative to base_q_idx.
+ * - __s8
+ - ``delta_q_uv_ac``
+ - Indicates the UV AC quantizer relative to base_q_idx.
+ * - __u8
+ - ``reserved[4]``
+ - Applications and drivers must set this to zero.
+
+.. c:type:: v4l2_vp9_loop_filter
+
+This structure contains all loop filter related parameters. See sections
+'7.2.8 Loop filter semantics' of the :ref:`vp9` specification for more details.
+
+.. tabularcolumns:: |p{0.8cm}|p{4cm}|p{12.4cm}|
+
+.. cssclass:: longtable
+
+.. flat-table:: struct v4l2_vp9_loop_filter
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - __s8
+ - ``ref_deltas[4]``
+ - Contains the adjustment needed for the filter level based on the chosen
+ reference frame.
+ * - __s8
+ - ``mode_deltas[2]``
+ - Contains the adjustment needed for the filter level based on the chosen
+ mode.
+ * - __u8
+ - ``level``
+ - Indicates the loop filter strength.
+ * - __u8
+ - ``sharpness``
+ - Indicates the sharpness level.
+ * - __u8
+ - ``flags``
+ - Combination of V4L2_VP9_LOOP_FILTER_FLAG_* flags.
+ See :ref:`Loop Filter Flags <vp9_loop_filter_flags>`.
+ * - __u8
+ - ``reserved[7]``
+ - Applications and drivers must set this to zero.
+
+
+.. _vp9_loop_filter_flags:
+
+``Loop Filter Flags``
+
+.. tabularcolumns:: |p{9.6cm}|p{0.5cm}|p{7.2cm}|
+
+.. flat-table::
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - ``V4L2_VP9_LOOP_FILTER_FLAG_DELTA_ENABLED``
+ - 0x1
+ - When set, the filter level depends on the mode and reference frame used
+ to predict a block.
+ * - ``V4L2_VP9_LOOP_FILTER_FLAG_DELTA_UPDATE``
+ - 0x2
+ - When set, the bitstream contains additional syntax elements that
+ specify which mode and reference frame deltas are to be updated.
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
index 0ede39907ee2..967fc803ef94 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
@@ -172,6 +172,21 @@ Compressed Formats
- VP9 compressed video frame. The encoder generates one
compressed frame per buffer, and the decoder requires one
compressed frame per buffer.
+ * .. _V4L2-PIX-FMT-VP9-FRAME:
+
+ - ``V4L2_PIX_FMT_VP9_FRAME``
+ - 'VP9F'
+ - VP9 parsed frame, including the frame header, as extracted from the container.
+ This format is adapted for stateless video decoders that implement a
+ VP9 pipeline with the :ref:`stateless_decoder`.
+ Metadata associated with the frame to decode is required to be passed
+ through the ``V4L2_CID_STATELESS_VP9_FRAME`` and
+ the ``V4L2_CID_STATELESS_VP9_COMPRESSED_HDR`` controls.
+ See the :ref:`associated Codec Control IDs <v4l2-codec-stateless-vp9>`.
+ Exactly one output and one capture buffer must be provided for use with
+ this pixel format. The output buffer must contain the appropriate number
+ of macroblocks to decode a full corresponding frame to the matching
+ capture buffer.
* .. _V4L2-PIX-FMT-HEVC:
- ``V4L2_PIX_FMT_HEVC``
diff --git a/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst b/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst
index 2d6bc8d94380..d2bdd3db076f 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst
@@ -233,6 +233,14 @@ still cause this situation.
- ``p_mpeg2_quantisation``
- A pointer to a struct :c:type:`v4l2_ctrl_mpeg2_quantisation`. Valid if this control is
of type ``V4L2_CTRL_TYPE_MPEG2_QUANTISATION``.
+ * - struct :c:type:`v4l2_ctrl_vp9_compressed_hdr` *
+ - ``p_vp9_compressed_hdr_probs``
+ - A pointer to a struct :c:type:`v4l2_ctrl_vp9_compressed_hdr`. Valid if this
+ control is of type ``V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR``.
+ * - struct :c:type:`v4l2_ctrl_vp9_frame` *
+ - ``p_vp9_frame``
+ - A pointer to a struct :c:type:`v4l2_ctrl_vp9_frame`. Valid if this
+ control is of type ``V4L2_CTRL_TYPE_VP9_FRAME``.
* - struct :c:type:`v4l2_ctrl_hdr10_cll_info` *
- ``p_hdr10_cll``
- A pointer to a struct :c:type:`v4l2_ctrl_hdr10_cll_info`. Valid if this control is
diff --git a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
index f9ecf6276129..9ad930823960 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
@@ -507,6 +507,18 @@ See also the examples in :ref:`control`.
- n/a
- A struct :c:type:`v4l2_ctrl_hevc_decode_params`, containing HEVC
decoding parameters for stateless video decoders.
+ * - ``V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR``
+ - n/a
+ - n/a
+ - n/a
+ - A struct :c:type:`v4l2_ctrl_vp9_compressed_hdr`, containing VP9
+ probabilities updates for stateless video decoders.
+ * - ``V4L2_CTRL_TYPE_VP9_FRAME``
+ - n/a
+ - n/a
+ - n/a
+ - A struct :c:type:`v4l2_ctrl_vp9_frame`, containing VP9
+ frame decode parameters for stateless video decoders.
.. raw:: latex
diff --git a/Documentation/userspace-api/media/videodev2.h.rst.exceptions b/Documentation/userspace-api/media/videodev2.h.rst.exceptions
index eb0b1cd37abd..9cbb7a0c354a 100644
--- a/Documentation/userspace-api/media/videodev2.h.rst.exceptions
+++ b/Documentation/userspace-api/media/videodev2.h.rst.exceptions
@@ -149,6 +149,8 @@ replace symbol V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS :c:type:`v4l2_ctrl_type`
replace symbol V4L2_CTRL_TYPE_AREA :c:type:`v4l2_ctrl_type`
replace symbol V4L2_CTRL_TYPE_FWHT_PARAMS :c:type:`v4l2_ctrl_type`
replace symbol V4L2_CTRL_TYPE_VP8_FRAME :c:type:`v4l2_ctrl_type`
+replace symbol V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR :c:type:`v4l2_ctrl_type`
+replace symbol V4L2_CTRL_TYPE_VP9_FRAME :c:type:`v4l2_ctrl_type`
replace symbol V4L2_CTRL_TYPE_HDR10_CLL_INFO :c:type:`v4l2_ctrl_type`
replace symbol V4L2_CTRL_TYPE_HDR10_MASTERING_DISPLAY :c:type:`v4l2_ctrl_type`
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c
index c4b5082849b6..52b9ff46ab26 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
@@ -283,6 +283,12 @@ static void std_log(const struct v4l2_ctrl *ctrl)
case V4L2_CTRL_TYPE_MPEG2_PICTURE:
pr_cont("MPEG2_PICTURE");
break;
+ case V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR:
+ pr_cont("VP9_COMPRESSED_HDR");
+ break;
+ case V4L2_CTRL_TYPE_VP9_FRAME:
+ pr_cont("VP9_FRAME");
+ break;
default:
pr_cont("unknown type %d", ctrl->type);
break;
@@ -317,6 +323,168 @@ static void std_log(const struct v4l2_ctrl *ctrl)
#define zero_reserved(s) \
memset(&(s).reserved, 0, sizeof((s).reserved))
+static int
+validate_vp9_lf_params(struct v4l2_vp9_loop_filter *lf)
+{
+ unsigned int i;
+
+ if (lf->flags & ~(V4L2_VP9_LOOP_FILTER_FLAG_DELTA_ENABLED |
+ V4L2_VP9_LOOP_FILTER_FLAG_DELTA_UPDATE))
+ return -EINVAL;
+
+ /* That all values are in the accepted range. */
+ if (lf->level > GENMASK(5, 0))
+ return -EINVAL;
+
+ if (lf->sharpness > GENMASK(2, 0))
+ return -EINVAL;
+
+ for (i = 0; i < ARRAY_SIZE(lf->ref_deltas); i++)
+ if (lf->ref_deltas[i] < -63 || lf->ref_deltas[i] > 63)
+ return -EINVAL;
+
+ for (i = 0; i < ARRAY_SIZE(lf->mode_deltas); i++)
+ if (lf->mode_deltas[i] < -63 || lf->mode_deltas[i] > 63)
+ return -EINVAL;
+
+ zero_reserved(*lf);
+ return 0;
+}
+
+static int
+validate_vp9_quant_params(struct v4l2_vp9_quantization *quant)
+{
+ if (quant->delta_q_y_dc < -15 || quant->delta_q_y_dc > 15 ||
+ quant->delta_q_uv_dc < -15 || quant->delta_q_uv_dc > 15 ||
+ quant->delta_q_uv_ac < -15 || quant->delta_q_uv_ac > 15)
+ return -EINVAL;
+
+ zero_reserved(*quant);
+ return 0;
+}
+
+static int
+validate_vp9_seg_params(struct v4l2_vp9_segmentation *seg)
+{
+ unsigned int i, j;
+
+ if (seg->flags & ~(V4L2_VP9_SEGMENTATION_FLAG_ENABLED |
+ V4L2_VP9_SEGMENTATION_FLAG_UPDATE_MAP |
+ V4L2_VP9_SEGMENTATION_FLAG_TEMPORAL_UPDATE |
+ V4L2_VP9_SEGMENTATION_FLAG_UPDATE_DATA |
+ V4L2_VP9_SEGMENTATION_FLAG_ABS_OR_DELTA_UPDATE))
+ return -EINVAL;
+
+ for (i = 0; i < ARRAY_SIZE(seg->feature_enabled); i++) {
+ if (seg->feature_enabled[i] &
+ ~V4L2_VP9_SEGMENT_FEATURE_ENABLED_MASK)
+ return -EINVAL;
+ }
+
+ for (i = 0; i < ARRAY_SIZE(seg->feature_data); i++) {
+ const int range[] = { 255, 63, 3, 0 };
+
+ for (j = 0; j < ARRAY_SIZE(seg->feature_data[j]); j++) {
+ if (seg->feature_data[i][j] < -range[j] ||
+ seg->feature_data[i][j] > range[j])
+ return -EINVAL;
+ }
+ }
+
+ zero_reserved(*seg);
+ return 0;
+}
+
+static int
+validate_vp9_compressed_hdr(struct v4l2_ctrl_vp9_compressed_hdr *hdr)
+{
+ if (hdr->tx_mode > V4L2_VP9_TX_MODE_SELECT)
+ return -EINVAL;
+
+ return 0;
+}
+
+static int
+validate_vp9_frame(struct v4l2_ctrl_vp9_frame *frame)
+{
+ int ret;
+
+ /* Make sure we're not passed invalid flags. */
+ if (frame->flags & ~(V4L2_VP9_FRAME_FLAG_KEY_FRAME |
+ V4L2_VP9_FRAME_FLAG_SHOW_FRAME |
+ V4L2_VP9_FRAME_FLAG_ERROR_RESILIENT |
+ V4L2_VP9_FRAME_FLAG_INTRA_ONLY |
+ V4L2_VP9_FRAME_FLAG_ALLOW_HIGH_PREC_MV |
+ V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX |
+ V4L2_VP9_FRAME_FLAG_PARALLEL_DEC_MODE |
+ V4L2_VP9_FRAME_FLAG_X_SUBSAMPLING |
+ V4L2_VP9_FRAME_FLAG_Y_SUBSAMPLING |
+ V4L2_VP9_FRAME_FLAG_COLOR_RANGE_FULL_SWING))
+ return -EINVAL;
+
+ if (frame->flags & V4L2_VP9_FRAME_FLAG_ERROR_RESILIENT &&
+ frame->flags & V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX)
+ return -EINVAL;
+
+ if (frame->profile > V4L2_VP9_PROFILE_MAX)
+ return -EINVAL;
+
+ if (frame->reset_frame_context > V4L2_VP9_RESET_FRAME_CTX_ALL)
+ return -EINVAL;
+
+ if (frame->frame_context_idx >= V4L2_VP9_NUM_FRAME_CTX)
+ return -EINVAL;
+
+ /*
+ * Profiles 0 and 1 only support 8-bit depth, profiles 2 and 3 only 10
+ * and 12 bit depths.
+ */
+ if ((frame->profile < 2 && frame->bit_depth != 8) ||
+ (frame->profile >= 2 &&
+ (frame->bit_depth != 10 && frame->bit_depth != 12)))
+ return -EINVAL;
+
+ /* Profile 0 and 2 only accept YUV 4:2:0. */
+ if ((frame->profile == 0 || frame->profile == 2) &&
+ (!(frame->flags & V4L2_VP9_FRAME_FLAG_X_SUBSAMPLING) ||
+ !(frame->flags & V4L2_VP9_FRAME_FLAG_Y_SUBSAMPLING)))
+ return -EINVAL;
+
+ /* Profile 1 and 3 only accept YUV 4:2:2, 4:4:0 and 4:4:4. */
+ if ((frame->profile == 1 || frame->profile == 3) &&
+ ((frame->flags & V4L2_VP9_FRAME_FLAG_X_SUBSAMPLING) &&
+ (frame->flags & V4L2_VP9_FRAME_FLAG_Y_SUBSAMPLING)))
+ return -EINVAL;
+
+ if (frame->interpolation_filter > V4L2_VP9_INTERP_FILTER_SWITCHABLE)
+ return -EINVAL;
+
+ /*
+ * According to the spec, tile_cols_log2 shall be less than or equal
+ * to 6.
+ */
+ if (frame->tile_cols_log2 > 6)
+ return -EINVAL;
+
+ if (frame->reference_mode > V4L2_VP9_REFERENCE_MODE_SELECT)
+ return -EINVAL;
+
+ ret = validate_vp9_lf_params(&frame->lf);
+ if (ret)
+ return ret;
+
+ ret = validate_vp9_quant_params(&frame->quant);
+ if (ret)
+ return ret;
+
+ ret = validate_vp9_seg_params(&frame->seg);
+ if (ret)
+ return ret;
+
+ zero_reserved(*frame);
+ return 0;
+}
+
/*
* Compound controls validation requires setting unused fields/flags to zero
* in order to properly detect unchanged controls with std_equal's memcmp.
@@ -687,6 +855,12 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx,
break;
+ case V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR:
+ return validate_vp9_compressed_hdr(p);
+
+ case V4L2_CTRL_TYPE_VP9_FRAME:
+ return validate_vp9_frame(p);
+
case V4L2_CTRL_TYPE_AREA:
area = p;
if (!area->width || !area->height)
@@ -1249,6 +1423,12 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
case V4L2_CTRL_TYPE_HDR10_MASTERING_DISPLAY:
elem_size = sizeof(struct v4l2_ctrl_hdr10_mastering_display);
break;
+ case V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR:
+ elem_size = sizeof(struct v4l2_ctrl_vp9_compressed_hdr);
+ break;
+ case V4L2_CTRL_TYPE_VP9_FRAME:
+ elem_size = sizeof(struct v4l2_ctrl_vp9_frame);
+ break;
case V4L2_CTRL_TYPE_AREA:
elem_size = sizeof(struct v4l2_area);
break;
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index 421300e13a41..5845c1b6bb2a 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -1175,6 +1175,8 @@ const char *v4l2_ctrl_get_name(u32 id)
case V4L2_CID_STATELESS_MPEG2_SEQUENCE: return "MPEG-2 Sequence Header";
case V4L2_CID_STATELESS_MPEG2_PICTURE: return "MPEG-2 Picture Header";
case V4L2_CID_STATELESS_MPEG2_QUANTISATION: return "MPEG-2 Quantisation Matrices";
+ case V4L2_CID_STATELESS_VP9_COMPRESSED_HDR: return "VP9 Probabilities Updates";
+ case V4L2_CID_STATELESS_VP9_FRAME: return "VP9 Frame Decode Parameters";
/* Colorimetry controls */
/* Keep the order of the 'case's the same as in v4l2-controls.h! */
@@ -1493,6 +1495,12 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS:
*type = V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS;
break;
+ case V4L2_CID_STATELESS_VP9_COMPRESSED_HDR:
+ *type = V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR;
+ break;
+ case V4L2_CID_STATELESS_VP9_FRAME:
+ *type = V4L2_CTRL_TYPE_VP9_FRAME;
+ break;
case V4L2_CID_UNIT_CELL_SIZE:
*type = V4L2_CTRL_TYPE_AREA;
*flags |= V4L2_CTRL_FLAG_READ_ONLY;
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index ec6fc1ef291e..7a5e8120d733 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1394,6 +1394,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_PIX_FMT_VP8: descr = "VP8"; break;
case V4L2_PIX_FMT_VP8_FRAME: descr = "VP8 Frame"; break;
case V4L2_PIX_FMT_VP9: descr = "VP9"; break;
+ case V4L2_PIX_FMT_VP9_FRAME: descr = "VP9 Frame"; break;
case V4L2_PIX_FMT_HEVC: descr = "HEVC"; break; /* aka H.265 */
case V4L2_PIX_FMT_HEVC_SLICE: descr = "HEVC Parsed Slice Data"; break;
case V4L2_PIX_FMT_FWHT: descr = "FWHT"; break; /* used in vicodec */
diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
index 575b59fbac77..b3ce438f1329 100644
--- a/include/media/v4l2-ctrls.h
+++ b/include/media/v4l2-ctrls.h
@@ -50,6 +50,8 @@ struct video_device;
* @p_h264_decode_params: Pointer to a struct v4l2_ctrl_h264_decode_params.
* @p_h264_pred_weights: Pointer to a struct v4l2_ctrl_h264_pred_weights.
* @p_vp8_frame: Pointer to a VP8 frame params structure.
+ * @p_vp9_compressed_hdr_probs: Pointer to a VP9 frame compressed header probs structure.
+ * @p_vp9_frame: Pointer to a VP9 frame params structure.
* @p_hevc_sps: Pointer to an HEVC sequence parameter set structure.
* @p_hevc_pps: Pointer to an HEVC picture parameter set structure.
* @p_hevc_slice_params: Pointer to an HEVC slice parameters structure.
@@ -80,6 +82,8 @@ union v4l2_ctrl_ptr {
struct v4l2_ctrl_hevc_sps *p_hevc_sps;
struct v4l2_ctrl_hevc_pps *p_hevc_pps;
struct v4l2_ctrl_hevc_slice_params *p_hevc_slice_params;
+ struct v4l2_ctrl_vp9_compressed_hdr *p_vp9_compressed_hdr_probs;
+ struct v4l2_ctrl_vp9_frame *p_vp9_frame;
struct v4l2_ctrl_hdr10_cll_info *p_hdr10_cll;
struct v4l2_ctrl_hdr10_mastering_display *p_hdr10_mastering;
struct v4l2_area *p_area;
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 5532b5f68493..36c82ad98030 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -2010,6 +2010,290 @@ struct v4l2_ctrl_hdr10_mastering_display {
__u32 min_display_mastering_luminance;
};
+/* Stateless VP9 controls */
+
+#define V4L2_VP9_LOOP_FILTER_FLAG_DELTA_ENABLED 0x1
+#define V4L2_VP9_LOOP_FILTER_FLAG_DELTA_UPDATE 0x2
+
+/**
+ * struct v4l2_vp9_loop_filter - VP9 loop filter parameters
+ *
+ * @ref_deltas: contains the adjustment needed for the filter level based on the
+ * chosen reference frame. If this syntax element is not present in the bitstream,
+ * users should pass its last value.
+ * @mode_deltas: contains the adjustment needed for the filter level based on the
+ * chosen mode. If this syntax element is not present in the bitstream, users should
+ * pass its last value.
+ * @level: indicates the loop filter strength.
+ * @sharpness: indicates the sharpness level.
+ * @flags: combination of V4L2_VP9_LOOP_FILTER_FLAG_{} flags.
+ * @reserved: padding field. Should be zeroed by applications.
+ *
+ * This structure contains all loop filter related parameters. See sections
+ * '7.2.8 Loop filter semantics' of the VP9 specification for more details.
+ */
+struct v4l2_vp9_loop_filter {
+ __s8 ref_deltas[4];
+ __s8 mode_deltas[2];
+ __u8 level;
+ __u8 sharpness;
+ __u8 flags;
+ __u8 reserved[7];
+};
+
+/**
+ * struct v4l2_vp9_quantization - VP9 quantization parameters
+ *
+ * @base_q_idx: indicates the base frame qindex.
+ * @delta_q_y_dc: indicates the Y DC quantizer relative to base_q_idx.
+ * @delta_q_uv_dc: indicates the UV DC quantizer relative to base_q_idx.
+ * @delta_q_uv_ac: indicates the UV AC quantizer relative to base_q_idx.
+ * @reserved: padding field. Should be zeroed by applications.
+ *
+ * Encodes the quantization parameters. See section '7.2.9 Quantization params
+ * syntax' of the VP9 specification for more details.
+ */
+struct v4l2_vp9_quantization {
+ __u8 base_q_idx;
+ __s8 delta_q_y_dc;
+ __s8 delta_q_uv_dc;
+ __s8 delta_q_uv_ac;
+ __u8 reserved[4];
+};
+
+#define V4L2_VP9_SEGMENTATION_FLAG_ENABLED 0x01
+#define V4L2_VP9_SEGMENTATION_FLAG_UPDATE_MAP 0x02
+#define V4L2_VP9_SEGMENTATION_FLAG_TEMPORAL_UPDATE 0x04
+#define V4L2_VP9_SEGMENTATION_FLAG_UPDATE_DATA 0x08
+#define V4L2_VP9_SEGMENTATION_FLAG_ABS_OR_DELTA_UPDATE 0x10
+
+#define V4L2_VP9_SEG_LVL_ALT_Q 0
+#define V4L2_VP9_SEG_LVL_ALT_L 1
+#define V4L2_VP9_SEG_LVL_REF_FRAME 2
+#define V4L2_VP9_SEG_LVL_SKIP 3
+#define V4L2_VP9_SEG_LVL_MAX 4
+
+#define V4L2_VP9_SEGMENT_FEATURE_ENABLED(id) (1 << (id))
+#define V4L2_VP9_SEGMENT_FEATURE_ENABLED_MASK 0xf
+
+/**
+ * struct v4l2_vp9_segmentation - VP9 segmentation parameters
+ *
+ * @feature_data: data attached to each feature. Data entry is only valid if
+ * the feature is enabled. The array shall be indexed with segment number as
+ * the first dimension (0..7) and one of V4L2_VP9_SEG_{} as the second dimension.
+ * @feature_enabled: bitmask defining which features are enabled in each segment.
+ * The value for each segment is a combination of V4L2_VP9_SEGMENT_FEATURE_ENABLED(id)
+ * values where id is one of V4L2_VP9_SEG_LVL_{}.
+ * @tree_probs: specifies the probability values to be used when decoding a
+ * Segment-ID. See '5.15. Segmentation map' section of the VP9 specification
+ * for more details.
+ * @pred_probs: specifies the probability values to be used when decoding a
+ * Predicted-Segment-ID. See '6.4.14. Get segment id syntax' section of :ref:`vp9`
+ * for more details.
+ * @flags: combination of V4L2_VP9_SEGMENTATION_FLAG_{} flags.
+ * @reserved: padding field. Should be zeroed by applications.
+ *
+ * Encodes the quantization parameters. See section '7.2.10 Segmentation params syntax' of
+ * the VP9 specification for more details.
+ */
+struct v4l2_vp9_segmentation {
+ __s16 feature_data[8][4];
+ __u8 feature_enabled[8];
+ __u8 tree_probs[7];
+ __u8 pred_probs[3];
+ __u8 flags;
+ __u8 reserved[5];
+};
+
+#define V4L2_VP9_FRAME_FLAG_KEY_FRAME 0x001
+#define V4L2_VP9_FRAME_FLAG_SHOW_FRAME 0x002
+#define V4L2_VP9_FRAME_FLAG_ERROR_RESILIENT 0x004
+#define V4L2_VP9_FRAME_FLAG_INTRA_ONLY 0x008
+#define V4L2_VP9_FRAME_FLAG_ALLOW_HIGH_PREC_MV 0x010
+#define V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX 0x020
+#define V4L2_VP9_FRAME_FLAG_PARALLEL_DEC_MODE 0x040
+#define V4L2_VP9_FRAME_FLAG_X_SUBSAMPLING 0x080
+#define V4L2_VP9_FRAME_FLAG_Y_SUBSAMPLING 0x100
+#define V4L2_VP9_FRAME_FLAG_COLOR_RANGE_FULL_SWING 0x200
+
+#define V4L2_VP9_SIGN_BIAS_LAST 0x1
+#define V4L2_VP9_SIGN_BIAS_GOLDEN 0x2
+#define V4L2_VP9_SIGN_BIAS_ALT 0x4
+
+#define V4L2_VP9_RESET_FRAME_CTX_NONE 0
+#define V4L2_VP9_RESET_FRAME_CTX_SPEC 1
+#define V4L2_VP9_RESET_FRAME_CTX_ALL 2
+
+#define V4L2_VP9_INTERP_FILTER_EIGHTTAP 0
+#define V4L2_VP9_INTERP_FILTER_EIGHTTAP_SMOOTH 1
+#define V4L2_VP9_INTERP_FILTER_EIGHTTAP_SHARP 2
+#define V4L2_VP9_INTERP_FILTER_BILINEAR 3
+#define V4L2_VP9_INTERP_FILTER_SWITCHABLE 4
+
+#define V4L2_VP9_REFERENCE_MODE_SINGLE_REFERENCE 0
+#define V4L2_VP9_REFERENCE_MODE_COMPOUND_REFERENCE 1
+#define V4L2_VP9_REFERENCE_MODE_SELECT 2
+
+#define V4L2_VP9_PROFILE_MAX 3
+
+#define V4L2_CID_STATELESS_VP9_FRAME (V4L2_CID_CODEC_STATELESS_BASE + 300)
+/**
+ * struct v4l2_ctrl_vp9_frame - VP9 frame decoding control
+ *
+ * @lf: loop filter parameters. See &v4l2_vp9_loop_filter for more details.
+ * @quant: quantization parameters. See &v4l2_vp9_quantization for more details.
+ * @seg: segmentation parameters. See &v4l2_vp9_segmentation for more details.
+ * @flags: combination of V4L2_VP9_FRAME_FLAG_{} flags.
+ * @compressed_header_size: compressed header size in bytes.
+ * @uncompressed_header_size: uncompressed header size in bytes.
+ * @frame_width_minus_1: add 1 to it and you'll get the frame width expressed in pixels.
+ * @frame_height_minus_1: add 1 to it and you'll get the frame height expressed in pixels.
+ * @render_width_minus_1: add 1 to it and you'll get the expected render width expressed in
+ * pixels. This is not used during the decoding process but might be used by HW scalers
+ * to prepare a frame that's ready for scanout.
+ * @render_height_minus_1: add 1 to it and you'll get the expected render height expressed in
+ * pixels. This is not used during the decoding process but might be used by HW scalers
+ * to prepare a frame that's ready for scanout.
+ * @last_frame_ts: "last" reference buffer timestamp.
+ * The timestamp refers to the timestamp field in struct v4l2_buffer.
+ * Use v4l2_timeval_to_ns() to convert the struct timeval to a __u64.
+ * @golden_frame_ts: "golden" reference buffer timestamp.
+ * The timestamp refers to the timestamp field in struct v4l2_buffer.
+ * Use v4l2_timeval_to_ns() to convert the struct timeval to a __u64.
+ * @alt_frame_ts: "alt" reference buffer timestamp.
+ * The timestamp refers to the timestamp field in struct v4l2_buffer.
+ * Use v4l2_timeval_to_ns() to convert the struct timeval to a __u64.
+ * @ref_frame_sign_bias: a bitfield specifying whether the sign bias is set for a given
+ * reference frame. Either of V4L2_VP9_SIGN_BIAS_{}.
+ * @reset_frame_context: specifies whether the frame context should be reset to default values.
+ * Either of V4L2_VP9_RESET_FRAME_CTX_{}.
+ * @frame_context_idx: frame context that should be used/updated.
+ * @profile: VP9 profile. Can be 0, 1, 2 or 3.
+ * @bit_depth: bits per components. Can be 8, 10 or 12. Note that not all profiles support
+ * 10 and/or 12 bits depths.
+ * @interpolation_filter: specifies the filter selection used for performing inter prediction.
+ * Set to one of V4L2_VP9_INTERP_FILTER_{}.
+ * @tile_cols_log2: specifies the base 2 logarithm of the width of each tile (where the width
+ * is measured in units of 8x8 blocks). Shall be less than or equal to 6.
+ * @tile_rows_log2: specifies the base 2 logarithm of the height of each tile (where the height
+ * is measured in units of 8x8 blocks).
+ * @reference_mode: specifies the type of inter prediction to be used.
+ * Set to one of V4L2_VP9_REFERENCE_MODE_{}.
+ * @reserved: padding field. Should be zeroed by applications.
+ */
+struct v4l2_ctrl_vp9_frame {
+ struct v4l2_vp9_loop_filter lf;
+ struct v4l2_vp9_quantization quant;
+ struct v4l2_vp9_segmentation seg;
+ __u32 flags;
+ __u16 compressed_header_size;
+ __u16 uncompressed_header_size;
+ __u16 frame_width_minus_1;
+ __u16 frame_height_minus_1;
+ __u16 render_width_minus_1;
+ __u16 render_height_minus_1;
+ __u64 last_frame_ts;
+ __u64 golden_frame_ts;
+ __u64 alt_frame_ts;
+ __u8 ref_frame_sign_bias;
+ __u8 reset_frame_context;
+ __u8 frame_context_idx;
+ __u8 profile;
+ __u8 bit_depth;
+ __u8 interpolation_filter;
+ __u8 tile_cols_log2;
+ __u8 tile_rows_log2;
+ __u8 reference_mode;
+ __u8 reserved[7];
+};
+
+#define V4L2_VP9_NUM_FRAME_CTX 4
+
+/**
+ * struct v4l2_vp9_mv_probs - VP9 Motion vector probability updates
+ * @joint: motion vector joint probability updates.
+ * @sign: motion vector sign probability updates.
+ * @classes: motion vector class probability updates.
+ * @class0_bit: motion vector class0 bit probability updates.
+ * @bits: motion vector bits probability updates.
+ * @class0_fr: motion vector class0 fractional bit probability updates.
+ * @fr: motion vector fractional bit probability updates.
+ * @class0_hp: motion vector class0 high precision fractional bit probability updates.
+ * @hp: motion vector high precision fractional bit probability updates.
+ *
+ * This structure contains new values of motion vector probabilities.
+ * A value of zero in an array element means there is no update of the relevant probability.
+ * See `struct v4l2_vp9_prob_updates` for details.
+ */
+struct v4l2_vp9_mv_probs {
+ __u8 joint[3];
+ __u8 sign[2];
+ __u8 classes[2][10];
+ __u8 class0_bit[2];
+ __u8 bits[2][10];
+ __u8 class0_fr[2][2][3];
+ __u8 fr[2][3];
+ __u8 class0_hp[2];
+ __u8 hp[2];
+};
+
+#define V4L2_CID_STATELESS_VP9_COMPRESSED_HDR (V4L2_CID_CODEC_STATELESS_BASE + 301)
+
+#define V4L2_VP9_TX_MODE_ONLY_4X4 0
+#define V4L2_VP9_TX_MODE_ALLOW_8X8 1
+#define V4L2_VP9_TX_MODE_ALLOW_16X16 2
+#define V4L2_VP9_TX_MODE_ALLOW_32X32 3
+#define V4L2_VP9_TX_MODE_SELECT 4
+
+/**
+ * struct v4l2_ctrl_vp9_compressed_hdr - VP9 probability updates control
+ * @tx_mode: specifies the TX mode. Set to one of V4L2_VP9_TX_MODE_{}.
+ * @tx8: TX 8x8 probability updates.
+ * @tx16: TX 16x16 probability updates.
+ * @tx32: TX 32x32 probability updates.
+ * @coef: coefficient probability updates.
+ * @skip: skip probability updates.
+ * @inter_mode: inter mode probability updates.
+ * @interp_filter: interpolation filter probability updates.
+ * @is_inter: is inter-block probability updates.
+ * @comp_mode: compound prediction mode probability updates.
+ * @single_ref: single ref probability updates.
+ * @comp_ref: compound ref probability updates.
+ * @y_mode: Y prediction mode probability updates.
+ * @uv_mode: UV prediction mode probability updates.
+ * @partition: partition probability updates.
+ * @mv: motion vector probability updates.
+ *
+ * This structure holds the probabilities update as parsed in the compressed
+ * header (Spec 6.3). These values represent the value of probability update after
+ * being translated with inv_map_table[] (see 6.3.5). A value of zero in an array element
+ * means that there is no update of the relevant probability.
+ *
+ * This control is optional and needs to be used when dealing with the hardware which is
+ * not capable of parsing the compressed header itself. Only drivers which need it will
+ * implement it.
+ */
+struct v4l2_ctrl_vp9_compressed_hdr {
+ __u8 tx_mode;
+ __u8 tx8[2][1];
+ __u8 tx16[2][2];
+ __u8 tx32[2][3];
+ __u8 coef[4][2][2][6][6][3];
+ __u8 skip[3];
+ __u8 inter_mode[7][3];
+ __u8 interp_filter[4][2];
+ __u8 is_inter[4];
+ __u8 comp_mode[5];
+ __u8 single_ref[5][2];
+ __u8 comp_ref[5];
+ __u8 y_mode[4][9];
+ __u8 uv_mode[10][9];
+ __u8 partition[16][3];
+
+ struct v4l2_vp9_mv_probs mv;
+};
+
/* MPEG-compression definitions kept for backwards compatibility */
#ifndef __KERNEL__
#define V4L2_CTRL_CLASS_MPEG V4L2_CTRL_CLASS_CODEC
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 58392dcd3bf5..2cd8f7e432c5 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -703,6 +703,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_VP8 v4l2_fourcc('V', 'P', '8', '0') /* VP8 */
#define V4L2_PIX_FMT_VP8_FRAME v4l2_fourcc('V', 'P', '8', 'F') /* VP8 parsed frame */
#define V4L2_PIX_FMT_VP9 v4l2_fourcc('V', 'P', '9', '0') /* VP9 */
+#define V4L2_PIX_FMT_VP9_FRAME v4l2_fourcc('V', 'P', '9', 'F') /* VP9 parsed frame */
#define V4L2_PIX_FMT_HEVC v4l2_fourcc('H', 'E', 'V', 'C') /* HEVC aka H.265 */
#define V4L2_PIX_FMT_FWHT v4l2_fourcc('F', 'W', 'H', 'T') /* Fast Walsh Hadamard Transform (vicodec) */
#define V4L2_PIX_FMT_FWHT_STATELESS v4l2_fourcc('S', 'F', 'W', 'H') /* Stateless FWHT (vicodec) */
@@ -1755,6 +1756,8 @@ struct v4l2_ext_control {
struct v4l2_ctrl_mpeg2_sequence __user *p_mpeg2_sequence;
struct v4l2_ctrl_mpeg2_picture __user *p_mpeg2_picture;
struct v4l2_ctrl_mpeg2_quantisation __user *p_mpeg2_quantisation;
+ struct v4l2_ctrl_vp9_compressed_hdr __user *p_vp9_compressed_hdr_probs;
+ struct v4l2_ctrl_vp9_frame __user *p_vp9_frame;
void __user *ptr;
};
} __attribute__ ((packed));
@@ -1819,6 +1822,9 @@ enum v4l2_ctrl_type {
V4L2_CTRL_TYPE_MPEG2_QUANTISATION = 0x0250,
V4L2_CTRL_TYPE_MPEG2_SEQUENCE = 0x0251,
V4L2_CTRL_TYPE_MPEG2_PICTURE = 0x0252,
+
+ V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR = 0x0260,
+ V4L2_CTRL_TYPE_VP9_FRAME = 0x0261,
};
/* Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
--
2.17.1
Hi Andrzej!
Dne sreda, 29. september 2021 ob 18:04:39 CEST je Andrzej Pietrasiewicz
napisal(a):
> The G2 decoder block produces NV12 4x4 tiled format (NV12_4L4).
> Enable the G2 post-processor block, in order to produce regular NV12.
>
> The logic in hantro_postproc.c is leveraged to take care of allocating
> the extra buffers and configure the post-processor, which is
> significantly simpler than the one on the G1.
Quick summary of discussion on LibreELEC Slack:
When using NV12 format on Allwinner H6 variant of G2 (needs some driver
changes), I get frames out of order. If I use native NV12 tiled format, frames
are ordered correctly.
Currently I'm not sure if this is issue with my changes or is this general
issue.
I would be grateful if anyone can test frame order with and without
postprocessing enabled on imx8. Take some dynamic video with a lot of short
scenes. It's pretty obvious when frames are out of order.
However, given that frames themself are correctly decoded and without
postprocessing in right order, that shouldn't block merging previous patches.
I tried few different videos and frames were all decoded correctly.
Best regards,
Jernej
>
> Signed-off-by: Ezequiel Garcia <[email protected]>
> Signed-off-by: Andrzej Pietrasiewicz <[email protected]>
> ---
> .../staging/media/hantro/hantro_g2_vp9_dec.c | 6 ++--
> drivers/staging/media/hantro/hantro_hw.h | 1 +
> .../staging/media/hantro/hantro_postproc.c | 31 +++++++++++++++++++
> drivers/staging/media/hantro/imx8m_vpu_hw.c | 11 +++++++
> 4 files changed, 46 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/staging/media/hantro/hantro_g2_vp9_dec.c b/drivers/
staging/media/hantro/hantro_g2_vp9_dec.c
> index 7f827b9f0133..1a26be72c878 100644
> --- a/drivers/staging/media/hantro/hantro_g2_vp9_dec.c
> +++ b/drivers/staging/media/hantro/hantro_g2_vp9_dec.c
> @@ -152,7 +152,7 @@ static void config_output(struct hantro_ctx *ctx,
> hantro_reg_write(ctx->dev, &g2_out_dis, 0);
> hantro_reg_write(ctx->dev, &g2_output_format, 0);
>
> - luma_addr = vb2_dma_contig_plane_dma_addr(&dst->base.vb.vb2_buf,
0);
> + luma_addr = hantro_get_dec_buf_addr(ctx, &dst->base.vb.vb2_buf);
> hantro_write_addr(ctx->dev, G2_OUT_LUMA_ADDR, luma_addr);
>
> chroma_addr = luma_addr + chroma_offset(ctx, dec_params);
> @@ -191,7 +191,7 @@ static void config_ref(struct hantro_ctx *ctx,
> hantro_reg_write(ctx->dev, &ref_reg->hor_scale, (refw << 14) /
dst->vp9.width);
> hantro_reg_write(ctx->dev, &ref_reg->ver_scale, (refh << 14) /
dst->vp9.height);
>
> - luma_addr = vb2_dma_contig_plane_dma_addr(&buf->base.vb.vb2_buf,
0);
> + luma_addr = hantro_get_dec_buf_addr(ctx, &buf->base.vb.vb2_buf);
> hantro_write_addr(ctx->dev, ref_reg->y_base, luma_addr);
>
> chroma_addr = luma_addr + chroma_offset(ctx, dec_params);
> @@ -236,7 +236,7 @@ static void config_ref_registers(struct hantro_ctx *ctx,
> config_ref(ctx, dst, &ref_regs[1], dec_params, dec_params-
>golden_frame_ts);
> config_ref(ctx, dst, &ref_regs[2], dec_params, dec_params-
>alt_frame_ts);
>
> - mv_addr = vb2_dma_contig_plane_dma_addr(&mv_ref->base.vb.vb2_buf,
0) +
> + mv_addr = hantro_get_dec_buf_addr(ctx, &mv_ref->base.vb.vb2_buf) +
> mv_offset(ctx, dec_params);
> hantro_write_addr(ctx->dev, G2_REF_MV_ADDR(0), mv_addr);
>
> diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/
media/hantro/hantro_hw.h
> index 2961d399fd60..3d4a5dc1e6d5 100644
> --- a/drivers/staging/media/hantro/hantro_hw.h
> +++ b/drivers/staging/media/hantro/hantro_hw.h
> @@ -274,6 +274,7 @@ extern const struct hantro_variant rk3399_vpu_variant;
> extern const struct hantro_variant sama5d4_vdec_variant;
>
> extern const struct hantro_postproc_ops hantro_g1_postproc_ops;
> +extern const struct hantro_postproc_ops hantro_g2_postproc_ops;
>
> extern const u32 hantro_vp8_dec_mc_filter[8][6];
>
> diff --git a/drivers/staging/media/hantro/hantro_postproc.c b/drivers/
staging/media/hantro/hantro_postproc.c
> index 4549aec08feb..79a66d001738 100644
> --- a/drivers/staging/media/hantro/hantro_postproc.c
> +++ b/drivers/staging/media/hantro/hantro_postproc.c
> @@ -11,6 +11,7 @@
> #include "hantro.h"
> #include "hantro_hw.h"
> #include "hantro_g1_regs.h"
> +#include "hantro_g2_regs.h"
>
> #define HANTRO_PP_REG_WRITE(vpu, reg_name, val) \
> { \
> @@ -99,6 +100,21 @@ static void hantro_postproc_g1_enable(struct hantro_ctx
*ctx)
> HANTRO_PP_REG_WRITE(vpu, display_width, ctx->dst_fmt.width);
> }
>
> +static void hantro_postproc_g2_enable(struct hantro_ctx *ctx)
> +{
> + struct hantro_dev *vpu = ctx->dev;
> + struct vb2_v4l2_buffer *dst_buf;
> + size_t chroma_offset = ctx->dst_fmt.width * ctx->dst_fmt.height;
> + dma_addr_t dst_dma;
> +
> + dst_buf = hantro_get_dst_buf(ctx);
> + dst_dma = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0);
> +
> + hantro_write_addr(vpu, G2_RS_OUT_LUMA_ADDR, dst_dma);
> + hantro_write_addr(vpu, G2_RS_OUT_CHROMA_ADDR, dst_dma +
chroma_offset);
> + hantro_reg_write(vpu, &g2_out_rs_e, 1);
> +}
> +
> void hantro_postproc_free(struct hantro_ctx *ctx)
> {
> struct hantro_dev *vpu = ctx->dev;
> @@ -127,6 +143,9 @@ int hantro_postproc_alloc(struct hantro_ctx *ctx)
> if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_H264_SLICE)
> buf_size += hantro_h264_mv_size(ctx->dst_fmt.width,
> ctx-
>dst_fmt.height);
> + else if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_VP9_FRAME)
> + buf_size += hantro_vp9_mv_size(ctx->dst_fmt.width,
> + ctx-
>dst_fmt.height);
>
> for (i = 0; i < num_buffers; ++i) {
> struct hantro_aux_buf *priv = &ctx->postproc.dec_q[i];
> @@ -152,6 +171,13 @@ static void hantro_postproc_g1_disable(struct
hantro_ctx *ctx)
> HANTRO_PP_REG_WRITE_S(vpu, pipeline_en, 0x0);
> }
>
> +static void hantro_postproc_g2_disable(struct hantro_ctx *ctx)
> +{
> + struct hantro_dev *vpu = ctx->dev;
> +
> + hantro_reg_write(vpu, &g2_out_rs_e, 0);
> +}
> +
> void hantro_postproc_disable(struct hantro_ctx *ctx)
> {
> struct hantro_dev *vpu = ctx->dev;
> @@ -172,3 +198,8 @@ const struct hantro_postproc_ops hantro_g1_postproc_ops
= {
> .enable = hantro_postproc_g1_enable,
> .disable = hantro_postproc_g1_disable,
> };
> +
> +const struct hantro_postproc_ops hantro_g2_postproc_ops = {
> + .enable = hantro_postproc_g2_enable,
> + .disable = hantro_postproc_g2_disable,
> +};
> diff --git a/drivers/staging/media/hantro/imx8m_vpu_hw.c b/drivers/staging/
media/hantro/imx8m_vpu_hw.c
> index 455a107ffb02..1a43f6fceef9 100644
> --- a/drivers/staging/media/hantro/imx8m_vpu_hw.c
> +++ b/drivers/staging/media/hantro/imx8m_vpu_hw.c
> @@ -132,6 +132,14 @@ static const struct hantro_fmt imx8m_vpu_dec_fmts[] = {
> },
> };
>
> +static const struct hantro_fmt imx8m_vpu_g2_postproc_fmts[] = {
> + {
> + .fourcc = V4L2_PIX_FMT_NV12,
> + .codec_mode = HANTRO_MODE_NONE,
> + .postprocessed = true,
> + },
> +};
> +
> static const struct hantro_fmt imx8m_vpu_g2_dec_fmts[] = {
> {
> .fourcc = V4L2_PIX_FMT_NV12_4L4,
> @@ -301,6 +309,9 @@ const struct hantro_variant imx8mq_vpu_g2_variant = {
> .dec_offset = 0x0,
> .dec_fmts = imx8m_vpu_g2_dec_fmts,
> .num_dec_fmts = ARRAY_SIZE(imx8m_vpu_g2_dec_fmts),
> + .postproc_fmts = imx8m_vpu_g2_postproc_fmts,
> + .num_postproc_fmts = ARRAY_SIZE(imx8m_vpu_g2_postproc_fmts),
> + .postproc_ops = &hantro_g2_postproc_ops,
> .codec = HANTRO_HEVC_DECODER | HANTRO_VP9_DECODER,
> .codec_ops = imx8mq_vpu_g2_codec_ops,
> .init = imx8mq_vpu_hw_init,
> --
> 2.17.1
>
>
Hi Jernej,
W dniu 14.10.2021 o 19:42, Jernej Škrabec pisze:
> Hi Andrzej!
>
> Dne sreda, 29. september 2021 ob 18:04:39 CEST je Andrzej Pietrasiewicz
> napisal(a):
>> The G2 decoder block produces NV12 4x4 tiled format (NV12_4L4).
>> Enable the G2 post-processor block, in order to produce regular NV12.
>>
>> The logic in hantro_postproc.c is leveraged to take care of allocating
>> the extra buffers and configure the post-processor, which is
>> significantly simpler than the one on the G1.
>
> Quick summary of discussion on LibreELEC Slack:
> When using NV12 format on Allwinner H6 variant of G2 (needs some driver
> changes), I get frames out of order. If I use native NV12 tiled format, frames
> are ordered correctly.
>
> Currently I'm not sure if this is issue with my changes or is this general
> issue.
>
> I would be grateful if anyone can test frame order with and without
> postprocessing enabled on imx8. Take some dynamic video with a lot of short
> scenes. It's pretty obvious when frames are out of order.
>
I checked on imx8 and cannot observe any such artifacts.
Andrzej
> However, given that frames themself are correctly decoded and without
> postprocessing in right order, that shouldn't block merging previous patches.
> I tried few different videos and frames were all decoded correctly.
>
> Best regards,
> Jernej
>
>>
>> Signed-off-by: Ezequiel Garcia <[email protected]>
>> Signed-off-by: Andrzej Pietrasiewicz <[email protected]>
>> ---
>> .../staging/media/hantro/hantro_g2_vp9_dec.c | 6 ++--
>> drivers/staging/media/hantro/hantro_hw.h | 1 +
>> .../staging/media/hantro/hantro_postproc.c | 31 +++++++++++++++++++
>> drivers/staging/media/hantro/imx8m_vpu_hw.c | 11 +++++++
>> 4 files changed, 46 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/staging/media/hantro/hantro_g2_vp9_dec.c b/drivers/
> staging/media/hantro/hantro_g2_vp9_dec.c
>> index 7f827b9f0133..1a26be72c878 100644
>> --- a/drivers/staging/media/hantro/hantro_g2_vp9_dec.c
>> +++ b/drivers/staging/media/hantro/hantro_g2_vp9_dec.c
>> @@ -152,7 +152,7 @@ static void config_output(struct hantro_ctx *ctx,
>> hantro_reg_write(ctx->dev, &g2_out_dis, 0);
>> hantro_reg_write(ctx->dev, &g2_output_format, 0);
>>
>> - luma_addr = vb2_dma_contig_plane_dma_addr(&dst->base.vb.vb2_buf,
> 0);
>> + luma_addr = hantro_get_dec_buf_addr(ctx, &dst->base.vb.vb2_buf);
>> hantro_write_addr(ctx->dev, G2_OUT_LUMA_ADDR, luma_addr);
>>
>> chroma_addr = luma_addr + chroma_offset(ctx, dec_params);
>> @@ -191,7 +191,7 @@ static void config_ref(struct hantro_ctx *ctx,
>> hantro_reg_write(ctx->dev, &ref_reg->hor_scale, (refw << 14) /
> dst->vp9.width);
>> hantro_reg_write(ctx->dev, &ref_reg->ver_scale, (refh << 14) /
> dst->vp9.height);
>>
>> - luma_addr = vb2_dma_contig_plane_dma_addr(&buf->base.vb.vb2_buf,
> 0);
>> + luma_addr = hantro_get_dec_buf_addr(ctx, &buf->base.vb.vb2_buf);
>> hantro_write_addr(ctx->dev, ref_reg->y_base, luma_addr);
>>
>> chroma_addr = luma_addr + chroma_offset(ctx, dec_params);
>> @@ -236,7 +236,7 @@ static void config_ref_registers(struct hantro_ctx *ctx,
>> config_ref(ctx, dst, &ref_regs[1], dec_params, dec_params-
>> golden_frame_ts);
>> config_ref(ctx, dst, &ref_regs[2], dec_params, dec_params-
>> alt_frame_ts);
>>
>> - mv_addr = vb2_dma_contig_plane_dma_addr(&mv_ref->base.vb.vb2_buf,
> 0) +
>> + mv_addr = hantro_get_dec_buf_addr(ctx, &mv_ref->base.vb.vb2_buf) +
>> mv_offset(ctx, dec_params);
>> hantro_write_addr(ctx->dev, G2_REF_MV_ADDR(0), mv_addr);
>>
>> diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/
> media/hantro/hantro_hw.h
>> index 2961d399fd60..3d4a5dc1e6d5 100644
>> --- a/drivers/staging/media/hantro/hantro_hw.h
>> +++ b/drivers/staging/media/hantro/hantro_hw.h
>> @@ -274,6 +274,7 @@ extern const struct hantro_variant rk3399_vpu_variant;
>> extern const struct hantro_variant sama5d4_vdec_variant;
>>
>> extern const struct hantro_postproc_ops hantro_g1_postproc_ops;
>> +extern const struct hantro_postproc_ops hantro_g2_postproc_ops;
>>
>> extern const u32 hantro_vp8_dec_mc_filter[8][6];
>>
>> diff --git a/drivers/staging/media/hantro/hantro_postproc.c b/drivers/
> staging/media/hantro/hantro_postproc.c
>> index 4549aec08feb..79a66d001738 100644
>> --- a/drivers/staging/media/hantro/hantro_postproc.c
>> +++ b/drivers/staging/media/hantro/hantro_postproc.c
>> @@ -11,6 +11,7 @@
>> #include "hantro.h"
>> #include "hantro_hw.h"
>> #include "hantro_g1_regs.h"
>> +#include "hantro_g2_regs.h"
>>
>> #define HANTRO_PP_REG_WRITE(vpu, reg_name, val) \
>> { \
>> @@ -99,6 +100,21 @@ static void hantro_postproc_g1_enable(struct hantro_ctx
> *ctx)
>> HANTRO_PP_REG_WRITE(vpu, display_width, ctx->dst_fmt.width);
>> }
>>
>> +static void hantro_postproc_g2_enable(struct hantro_ctx *ctx)
>> +{
>> + struct hantro_dev *vpu = ctx->dev;
>> + struct vb2_v4l2_buffer *dst_buf;
>> + size_t chroma_offset = ctx->dst_fmt.width * ctx->dst_fmt.height;
>> + dma_addr_t dst_dma;
>> +
>> + dst_buf = hantro_get_dst_buf(ctx);
>> + dst_dma = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0);
>> +
>> + hantro_write_addr(vpu, G2_RS_OUT_LUMA_ADDR, dst_dma);
>> + hantro_write_addr(vpu, G2_RS_OUT_CHROMA_ADDR, dst_dma +
> chroma_offset);
>> + hantro_reg_write(vpu, &g2_out_rs_e, 1);
>> +}
>> +
>> void hantro_postproc_free(struct hantro_ctx *ctx)
>> {
>> struct hantro_dev *vpu = ctx->dev;
>> @@ -127,6 +143,9 @@ int hantro_postproc_alloc(struct hantro_ctx *ctx)
>> if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_H264_SLICE)
>> buf_size += hantro_h264_mv_size(ctx->dst_fmt.width,
>> ctx-
>> dst_fmt.height);
>> + else if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_VP9_FRAME)
>> + buf_size += hantro_vp9_mv_size(ctx->dst_fmt.width,
>> + ctx-
>> dst_fmt.height);
>>
>> for (i = 0; i < num_buffers; ++i) {
>> struct hantro_aux_buf *priv = &ctx->postproc.dec_q[i];
>> @@ -152,6 +171,13 @@ static void hantro_postproc_g1_disable(struct
> hantro_ctx *ctx)
>> HANTRO_PP_REG_WRITE_S(vpu, pipeline_en, 0x0);
>> }
>>
>> +static void hantro_postproc_g2_disable(struct hantro_ctx *ctx)
>> +{
>> + struct hantro_dev *vpu = ctx->dev;
>> +
>> + hantro_reg_write(vpu, &g2_out_rs_e, 0);
>> +}
>> +
>> void hantro_postproc_disable(struct hantro_ctx *ctx)
>> {
>> struct hantro_dev *vpu = ctx->dev;
>> @@ -172,3 +198,8 @@ const struct hantro_postproc_ops hantro_g1_postproc_ops
> = {
>> .enable = hantro_postproc_g1_enable,
>> .disable = hantro_postproc_g1_disable,
>> };
>> +
>> +const struct hantro_postproc_ops hantro_g2_postproc_ops = {
>> + .enable = hantro_postproc_g2_enable,
>> + .disable = hantro_postproc_g2_disable,
>> +};
>> diff --git a/drivers/staging/media/hantro/imx8m_vpu_hw.c b/drivers/staging/
> media/hantro/imx8m_vpu_hw.c
>> index 455a107ffb02..1a43f6fceef9 100644
>> --- a/drivers/staging/media/hantro/imx8m_vpu_hw.c
>> +++ b/drivers/staging/media/hantro/imx8m_vpu_hw.c
>> @@ -132,6 +132,14 @@ static const struct hantro_fmt imx8m_vpu_dec_fmts[] = {
>> },
>> };
>>
>> +static const struct hantro_fmt imx8m_vpu_g2_postproc_fmts[] = {
>> + {
>> + .fourcc = V4L2_PIX_FMT_NV12,
>> + .codec_mode = HANTRO_MODE_NONE,
>> + .postprocessed = true,
>> + },
>> +};
>> +
>> static const struct hantro_fmt imx8m_vpu_g2_dec_fmts[] = {
>> {
>> .fourcc = V4L2_PIX_FMT_NV12_4L4,
>> @@ -301,6 +309,9 @@ const struct hantro_variant imx8mq_vpu_g2_variant = {
>> .dec_offset = 0x0,
>> .dec_fmts = imx8m_vpu_g2_dec_fmts,
>> .num_dec_fmts = ARRAY_SIZE(imx8m_vpu_g2_dec_fmts),
>> + .postproc_fmts = imx8m_vpu_g2_postproc_fmts,
>> + .num_postproc_fmts = ARRAY_SIZE(imx8m_vpu_g2_postproc_fmts),
>> + .postproc_ops = &hantro_g2_postproc_ops,
>> .codec = HANTRO_HEVC_DECODER | HANTRO_VP9_DECODER,
>> .codec_ops = imx8mq_vpu_g2_codec_ops,
>> .init = imx8mq_vpu_hw_init,
>> --
>> 2.17.1
>>
>>
>
>
Hi Andrzej!
Dne petek, 15. oktober 2021 ob 19:19:47 CEST je Andrzej Pietrasiewicz
napisal(a):
> Hi Jernej,
>
> W dniu 14.10.2021 o 19:42, Jernej Škrabec pisze:
> > Hi Andrzej!
> >
> > Dne sreda, 29. september 2021 ob 18:04:39 CEST je Andrzej Pietrasiewicz
> > napisal(a):
> >> The G2 decoder block produces NV12 4x4 tiled format (NV12_4L4).
> >> Enable the G2 post-processor block, in order to produce regular NV12.
> >>
> >> The logic in hantro_postproc.c is leveraged to take care of allocating
> >> the extra buffers and configure the post-processor, which is
> >> significantly simpler than the one on the G1.
> >
> > Quick summary of discussion on LibreELEC Slack:
> > When using NV12 format on Allwinner H6 variant of G2 (needs some driver
> > changes), I get frames out of order. If I use native NV12 tiled format,
frames
> > are ordered correctly.
> >
> > Currently I'm not sure if this is issue with my changes or is this general
> > issue.
> >
> > I would be grateful if anyone can test frame order with and without
> > postprocessing enabled on imx8. Take some dynamic video with a lot of
short
> > scenes. It's pretty obvious when frames are out of order.
> >
>
> I checked on imx8 and cannot observe any such artifacts.
I finally found the issue. As you mentioned on Slack, register write order once
already affected decoding. Well, it's the case again. I made hacky test and
moved postproc enable call after output buffers are set and it worked. So, this
is actually core quirk which is obviously fixed in newer variants.
This makes this series with minor adaptations completely working on H6. I see
no reason not to merge whole series.
Thanks for testing.
Best regards,
Jernej
>
> Andrzej
>
> > However, given that frames themself are correctly decoded and without
> > postprocessing in right order, that shouldn't block merging previous
patches.
> > I tried few different videos and frames were all decoded correctly.
> >
> > Best regards,
> > Jernej
> >
> >>
> >> Signed-off-by: Ezequiel Garcia <[email protected]>
> >> Signed-off-by: Andrzej Pietrasiewicz <[email protected]>
> >> ---
> >> .../staging/media/hantro/hantro_g2_vp9_dec.c | 6 ++--
> >> drivers/staging/media/hantro/hantro_hw.h | 1 +
> >> .../staging/media/hantro/hantro_postproc.c | 31 +++++++++++++++++++
> >> drivers/staging/media/hantro/imx8m_vpu_hw.c | 11 +++++++
> >> 4 files changed, 46 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/drivers/staging/media/hantro/hantro_g2_vp9_dec.c b/drivers/
> > staging/media/hantro/hantro_g2_vp9_dec.c
> >> index 7f827b9f0133..1a26be72c878 100644
> >> --- a/drivers/staging/media/hantro/hantro_g2_vp9_dec.c
> >> +++ b/drivers/staging/media/hantro/hantro_g2_vp9_dec.c
> >> @@ -152,7 +152,7 @@ static void config_output(struct hantro_ctx *ctx,
> >> hantro_reg_write(ctx->dev, &g2_out_dis, 0);
> >> hantro_reg_write(ctx->dev, &g2_output_format, 0);
> >>
> >> - luma_addr = vb2_dma_contig_plane_dma_addr(&dst->base.vb.vb2_buf,
> > 0);
> >> + luma_addr = hantro_get_dec_buf_addr(ctx, &dst->base.vb.vb2_buf);
> >> hantro_write_addr(ctx->dev, G2_OUT_LUMA_ADDR, luma_addr);
> >>
> >> chroma_addr = luma_addr + chroma_offset(ctx, dec_params);
> >> @@ -191,7 +191,7 @@ static void config_ref(struct hantro_ctx *ctx,
> >> hantro_reg_write(ctx->dev, &ref_reg->hor_scale, (refw << 14) /
> > dst->vp9.width);
> >> hantro_reg_write(ctx->dev, &ref_reg->ver_scale, (refh << 14) /
> > dst->vp9.height);
> >>
> >> - luma_addr = vb2_dma_contig_plane_dma_addr(&buf->base.vb.vb2_buf,
> > 0);
> >> + luma_addr = hantro_get_dec_buf_addr(ctx, &buf->base.vb.vb2_buf);
> >> hantro_write_addr(ctx->dev, ref_reg->y_base, luma_addr);
> >>
> >> chroma_addr = luma_addr + chroma_offset(ctx, dec_params);
> >> @@ -236,7 +236,7 @@ static void config_ref_registers(struct hantro_ctx
*ctx,
> >> config_ref(ctx, dst, &ref_regs[1], dec_params, dec_params-
> >> golden_frame_ts);
> >> config_ref(ctx, dst, &ref_regs[2], dec_params, dec_params-
> >> alt_frame_ts);
> >>
> >> - mv_addr = vb2_dma_contig_plane_dma_addr(&mv_ref->base.vb.vb2_buf,
> > 0) +
> >> + mv_addr = hantro_get_dec_buf_addr(ctx, &mv_ref->base.vb.vb2_buf) +
> >> mv_offset(ctx, dec_params);
> >> hantro_write_addr(ctx->dev, G2_REF_MV_ADDR(0), mv_addr);
> >>
> >> diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/
> > media/hantro/hantro_hw.h
> >> index 2961d399fd60..3d4a5dc1e6d5 100644
> >> --- a/drivers/staging/media/hantro/hantro_hw.h
> >> +++ b/drivers/staging/media/hantro/hantro_hw.h
> >> @@ -274,6 +274,7 @@ extern const struct hantro_variant
rk3399_vpu_variant;
> >> extern const struct hantro_variant sama5d4_vdec_variant;
> >>
> >> extern const struct hantro_postproc_ops hantro_g1_postproc_ops;
> >> +extern const struct hantro_postproc_ops hantro_g2_postproc_ops;
> >>
> >> extern const u32 hantro_vp8_dec_mc_filter[8][6];
> >>
> >> diff --git a/drivers/staging/media/hantro/hantro_postproc.c b/drivers/
> > staging/media/hantro/hantro_postproc.c
> >> index 4549aec08feb..79a66d001738 100644
> >> --- a/drivers/staging/media/hantro/hantro_postproc.c
> >> +++ b/drivers/staging/media/hantro/hantro_postproc.c
> >> @@ -11,6 +11,7 @@
> >> #include "hantro.h"
> >> #include "hantro_hw.h"
> >> #include "hantro_g1_regs.h"
> >> +#include "hantro_g2_regs.h"
> >>
> >> #define HANTRO_PP_REG_WRITE(vpu, reg_name, val) \
> >> { \
> >> @@ -99,6 +100,21 @@ static void hantro_postproc_g1_enable(struct
hantro_ctx
> > *ctx)
> >> HANTRO_PP_REG_WRITE(vpu, display_width, ctx->dst_fmt.width);
> >> }
> >>
> >> +static void hantro_postproc_g2_enable(struct hantro_ctx *ctx)
> >> +{
> >> + struct hantro_dev *vpu = ctx->dev;
> >> + struct vb2_v4l2_buffer *dst_buf;
> >> + size_t chroma_offset = ctx->dst_fmt.width * ctx->dst_fmt.height;
> >> + dma_addr_t dst_dma;
> >> +
> >> + dst_buf = hantro_get_dst_buf(ctx);
> >> + dst_dma = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0);
> >> +
> >> + hantro_write_addr(vpu, G2_RS_OUT_LUMA_ADDR, dst_dma);
> >> + hantro_write_addr(vpu, G2_RS_OUT_CHROMA_ADDR, dst_dma +
> > chroma_offset);
> >> + hantro_reg_write(vpu, &g2_out_rs_e, 1);
> >> +}
> >> +
> >> void hantro_postproc_free(struct hantro_ctx *ctx)
> >> {
> >> struct hantro_dev *vpu = ctx->dev;
> >> @@ -127,6 +143,9 @@ int hantro_postproc_alloc(struct hantro_ctx *ctx)
> >> if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_H264_SLICE)
> >> buf_size += hantro_h264_mv_size(ctx->dst_fmt.width,
> >> ctx-
> >> dst_fmt.height);
> >> + else if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_VP9_FRAME)
> >> + buf_size += hantro_vp9_mv_size(ctx->dst_fmt.width,
> >> + ctx-
> >> dst_fmt.height);
> >>
> >> for (i = 0; i < num_buffers; ++i) {
> >> struct hantro_aux_buf *priv = &ctx->postproc.dec_q[i];
> >> @@ -152,6 +171,13 @@ static void hantro_postproc_g1_disable(struct
> > hantro_ctx *ctx)
> >> HANTRO_PP_REG_WRITE_S(vpu, pipeline_en, 0x0);
> >> }
> >>
> >> +static void hantro_postproc_g2_disable(struct hantro_ctx *ctx)
> >> +{
> >> + struct hantro_dev *vpu = ctx->dev;
> >> +
> >> + hantro_reg_write(vpu, &g2_out_rs_e, 0);
> >> +}
> >> +
> >> void hantro_postproc_disable(struct hantro_ctx *ctx)
> >> {
> >> struct hantro_dev *vpu = ctx->dev;
> >> @@ -172,3 +198,8 @@ const struct hantro_postproc_ops
hantro_g1_postproc_ops
> > = {
> >> .enable = hantro_postproc_g1_enable,
> >> .disable = hantro_postproc_g1_disable,
> >> };
> >> +
> >> +const struct hantro_postproc_ops hantro_g2_postproc_ops = {
> >> + .enable = hantro_postproc_g2_enable,
> >> + .disable = hantro_postproc_g2_disable,
> >> +};
> >> diff --git a/drivers/staging/media/hantro/imx8m_vpu_hw.c b/drivers/
staging/
> > media/hantro/imx8m_vpu_hw.c
> >> index 455a107ffb02..1a43f6fceef9 100644
> >> --- a/drivers/staging/media/hantro/imx8m_vpu_hw.c
> >> +++ b/drivers/staging/media/hantro/imx8m_vpu_hw.c
> >> @@ -132,6 +132,14 @@ static const struct hantro_fmt imx8m_vpu_dec_fmts[]
= {
> >> },
> >> };
> >>
> >> +static const struct hantro_fmt imx8m_vpu_g2_postproc_fmts[] = {
> >> + {
> >> + .fourcc = V4L2_PIX_FMT_NV12,
> >> + .codec_mode = HANTRO_MODE_NONE,
> >> + .postprocessed = true,
> >> + },
> >> +};
> >> +
> >> static const struct hantro_fmt imx8m_vpu_g2_dec_fmts[] = {
> >> {
> >> .fourcc = V4L2_PIX_FMT_NV12_4L4,
> >> @@ -301,6 +309,9 @@ const struct hantro_variant imx8mq_vpu_g2_variant = {
> >> .dec_offset = 0x0,
> >> .dec_fmts = imx8m_vpu_g2_dec_fmts,
> >> .num_dec_fmts = ARRAY_SIZE(imx8m_vpu_g2_dec_fmts),
> >> + .postproc_fmts = imx8m_vpu_g2_postproc_fmts,
> >> + .num_postproc_fmts = ARRAY_SIZE(imx8m_vpu_g2_postproc_fmts),
> >> + .postproc_ops = &hantro_g2_postproc_ops,
> >> .codec = HANTRO_HEVC_DECODER | HANTRO_VP9_DECODER,
> >> .codec_ops = imx8mq_vpu_g2_codec_ops,
> >> .init = imx8mq_vpu_hw_init,
> >> --
> >> 2.17.1
> >>
> >>
> >
> >
>
>
Hi everyone,
On Wed, 29 Sept 2021 at 12:04, Andrzej Pietrasiewicz
<[email protected]> wrote:
>
> Dear all,
>
> This patch series adds VP9 codec V4L2 control interface and two drivers
> using the new controls. It is a follow-up of previous v6 series [1].
>
> In this iteration, we've implemented VP9 hardware decoding on two devices:
> Rockchip VDEC and Hantro G2, and tested on RK3399, i.MX8MQ and i.MX8MP.
> The i.MX8M driver needs proper power domains support, though, which is a
> subject of a different effort, but in all 3 cases we were able to run the
> drivers.
>
> GStreamer support is also available, the needed changes have been submitted
> by Daniel Almeida [2]. This MR is ready to be merged, and just needs the
> VP9 V4L2 controls to be merged and released.
>
> Both rkvdec and hantro drivers are passing a significant number of VP9 tests
> using Fluster[3]. There are still a few tests that are not passing, due to
> dynamic frame resize (not yet supported by V4L2) and small size videos
> (due to IP block limitations).
>
> The series adds the VP9 codec V4L2 control API as uAPI, so it aims at being
> merged without passing through staging, as agreed[4]. The ABI has been checked
> for padding and verified to contain no holes.
>
I took another look at this, and I'm fairly happy with it.
I'd just like to have an A-b or R-b from Nicolas Dufresne and
Daniel Almeida, given they've done a lot of work on the client side
of the API.
Another option would be to wait until Jernej finishes the work on
Allwinner H6, so we have another hardware supported.
Thanks,
Ezequiel
> [1] https://patchwork.linuxtv.org/project/linux-media/list/?series=6377
> [2] https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2144
> [3] https://github.com/fluendo/fluster
> [4] https://lore.kernel.org/linux-media/[email protected]/
>
> The series depends on the YUV tiled format support prepared by Ezequiel:
> https://www.spinics.net/lists/linux-media/msg197047.html
>
> Rebased onto latest media_tree.
>
> Changes related to v6:
> - moved setting tile filter and tile bsd auxiliary buffer addresses so
> that they are always set, even if no tiles are used (thanks, Jernej)
> - added a comment near the place where the 32-bit DMA mask is applied
> (thanks, Nicolas)
> - improved consistency in register names (thanks, Nicolas)
>
> Changes related to v5:
> - improved the doc comments as per Ezequiel's review (thanks, Ezequiel)
> - improved pdf output of documentation
> - added Benjamin's Reviewed-by (thanks, Benjamin)
>
> Changes related to v4:
> - removed unused enum v4l2_vp9_intra_prediction_mode
> - converted remaining enums to defines to follow the convention
> - improved the documentation, in particular better documented how to use segmentation
> features
>
> Changes related to v3:
>
> Apply suggestions from Jernej's review (thanks, Jernej):
> - renamed a control and two structs:
> V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR_PROBS =>
> V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR
> v4l2_ctrl_vp9_compressed_hdr_probs =>
> v4l2_ctrl_vp9_compressed_hdr
> v4l2_vp9_mv_compressed_hdr_probs => v4l2_vp9_mv_probs
> - moved tx_mode to v4l2_ctrl_vp9_compressed_hdr
> - fixed enum v4l2_vp9_ref_frame_sign_bias values (which are used to test a bitfield)
> - explicitly assigned values to all other vp9 enums
>
> Apply suggestion from Nicolas's review (thanks, Nicolas):
> - explicitly stated that the v4l2_ctrl_vp9_compressed_hdr control is optional
> and implemented only by drivers which need it
>
> Changes related to the RFC v2:
>
> - added another driver including a postprocessor to de-tile
> codec-specific tiling
> - reworked uAPI structs layout to follow VP8 style
> - changed validation of loop filter params
> - changed validation of segmentation params
> - changed validation of VP9 frame params
> - removed level lookup array from loop filter struct
> (can be computed by drivers)
> - renamed some enum values to match the spec more closely
> - V4L2 VP9 library changed the 'eob' member of
> 'struct v4l2_vp9_frame_symbol_counts' so that it is an array
> of pointers instead of an array of pointers to arrays
> (IPs such as g2 creatively pass parts of the 'eob' counts in
> the 'coeff' counts)
> - factored out several repeated portions of code
> - minor nitpicks and cleanups
>
> Andrzej Pietrasiewicz (6):
> media: uapi: Add VP9 stateless decoder controls
> media: Add VP9 v4l2 library
> media: hantro: Rename registers
> media: hantro: Prepare for other G2 codecs
> media: hantro: Support VP9 on the G2 core
> media: hantro: Support NV12 on the G2 core
>
> Boris Brezillon (1):
> media: rkvdec: Add the VP9 backend
>
> Ezequiel Garcia (4):
> hantro: postproc: Fix motion vector space size
> hantro: postproc: Introduce struct hantro_postproc_ops
> hantro: Simplify postprocessor
> hantro: Add quirk for NV12/NV12_4L4 capture format
>
> .../userspace-api/media/v4l/biblio.rst | 10 +
> .../media/v4l/ext-ctrls-codec-stateless.rst | 573 +++++
> .../media/v4l/pixfmt-compressed.rst | 15 +
> .../media/v4l/vidioc-g-ext-ctrls.rst | 8 +
> .../media/v4l/vidioc-queryctrl.rst | 12 +
> .../media/videodev2.h.rst.exceptions | 2 +
> drivers/media/v4l2-core/Kconfig | 4 +
> drivers/media/v4l2-core/Makefile | 1 +
> drivers/media/v4l2-core/v4l2-ctrls-core.c | 180 ++
> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 8 +
> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
> drivers/media/v4l2-core/v4l2-vp9.c | 1850 +++++++++++++++++
> drivers/staging/media/hantro/Kconfig | 1 +
> drivers/staging/media/hantro/Makefile | 7 +-
> drivers/staging/media/hantro/hantro.h | 40 +-
> drivers/staging/media/hantro/hantro_drv.c | 23 +-
> drivers/staging/media/hantro/hantro_g2.c | 27 +
> .../staging/media/hantro/hantro_g2_hevc_dec.c | 69 +-
> drivers/staging/media/hantro/hantro_g2_regs.h | 132 +-
> .../staging/media/hantro/hantro_g2_vp9_dec.c | 980 +++++++++
> drivers/staging/media/hantro/hantro_hw.h | 83 +-
> .../staging/media/hantro/hantro_postproc.c | 79 +-
> drivers/staging/media/hantro/hantro_v4l2.c | 20 +
> drivers/staging/media/hantro/hantro_vp9.c | 240 +++
> drivers/staging/media/hantro/hantro_vp9.h | 103 +
> drivers/staging/media/hantro/imx8m_vpu_hw.c | 38 +-
> .../staging/media/hantro/rockchip_vpu_hw.c | 7 +-
> .../staging/media/hantro/sama5d4_vdec_hw.c | 3 +-
> drivers/staging/media/rkvdec/Kconfig | 1 +
> drivers/staging/media/rkvdec/Makefile | 2 +-
> drivers/staging/media/rkvdec/rkvdec-vp9.c | 1078 ++++++++++
> drivers/staging/media/rkvdec/rkvdec.c | 52 +-
> drivers/staging/media/rkvdec/rkvdec.h | 12 +-
> include/media/v4l2-ctrls.h | 4 +
> include/media/v4l2-vp9.h | 182 ++
> include/uapi/linux/v4l2-controls.h | 284 +++
> include/uapi/linux/videodev2.h | 6 +
> 37 files changed, 6033 insertions(+), 104 deletions(-)
> create mode 100644 drivers/media/v4l2-core/v4l2-vp9.c
> create mode 100644 drivers/staging/media/hantro/hantro_g2.c
> create mode 100644 drivers/staging/media/hantro/hantro_g2_vp9_dec.c
> create mode 100644 drivers/staging/media/hantro/hantro_vp9.c
> create mode 100644 drivers/staging/media/hantro/hantro_vp9.h
> create mode 100644 drivers/staging/media/rkvdec/rkvdec-vp9.c
> create mode 100644 include/media/v4l2-vp9.h
>
>
> base-commit: e4e737bb5c170df6135a127739a9e6148ee3da82
> --
> 2.17.1
>
Hi Jernej,
On Tue, 19 Oct 2021 at 13:38, Jernej Škrabec <[email protected]> wrote:
>
> Hi Andrzej!
>
> Dne petek, 15. oktober 2021 ob 19:19:47 CEST je Andrzej Pietrasiewicz
> napisal(a):
> > Hi Jernej,
> >
> > W dniu 14.10.2021 o 19:42, Jernej Škrabec pisze:
> > > Hi Andrzej!
> > >
> > > Dne sreda, 29. september 2021 ob 18:04:39 CEST je Andrzej Pietrasiewicz
> > > napisal(a):
> > >> The G2 decoder block produces NV12 4x4 tiled format (NV12_4L4).
> > >> Enable the G2 post-processor block, in order to produce regular NV12.
> > >>
> > >> The logic in hantro_postproc.c is leveraged to take care of allocating
> > >> the extra buffers and configure the post-processor, which is
> > >> significantly simpler than the one on the G1.
> > >
> > > Quick summary of discussion on LibreELEC Slack:
> > > When using NV12 format on Allwinner H6 variant of G2 (needs some driver
> > > changes), I get frames out of order. If I use native NV12 tiled format,
> frames
> > > are ordered correctly.
> > >
> > > Currently I'm not sure if this is issue with my changes or is this general
> > > issue.
> > >
> > > I would be grateful if anyone can test frame order with and without
> > > postprocessing enabled on imx8. Take some dynamic video with a lot of
> short
> > > scenes. It's pretty obvious when frames are out of order.
> > >
> >
> > I checked on imx8 and cannot observe any such artifacts.
>
> I finally found the issue. As you mentioned on Slack, register write order once
> already affected decoding. Well, it's the case again. I made hacky test and
> moved postproc enable call after output buffers are set and it worked. So, this
> is actually core quirk which is obviously fixed in newer variants.
>
Ugh, good catch.
What happens if you move all the calls to HANTRO_PP_REG_WRITE_S
(HANTRO_PP_REG_WRITE does a relaxed write)?
Or what happens if the HANTRO_PP_REG_WRITE(vpu, out_luma_base, dst_dma)
is moved to be done after all the other registers?
> This makes this series with minor adaptations completely working on H6. I see
> no reason not to merge whole series.
>
Do you have plans to submit your H6 work on top of this?
Thanks,
Ezequiel
> Thanks for testing.
>
> Best regards,
> Jernej
>
> >
> > Andrzej
> >
> > > However, given that frames themself are correctly decoded and without
> > > postprocessing in right order, that shouldn't block merging previous
> patches.
> > > I tried few different videos and frames were all decoded correctly.
> > >
> > > Best regards,
> > > Jernej
> > >
> > >>
> > >> Signed-off-by: Ezequiel Garcia <[email protected]>
> > >> Signed-off-by: Andrzej Pietrasiewicz <[email protected]>
> > >> ---
> > >> .../staging/media/hantro/hantro_g2_vp9_dec.c | 6 ++--
> > >> drivers/staging/media/hantro/hantro_hw.h | 1 +
> > >> .../staging/media/hantro/hantro_postproc.c | 31 +++++++++++++++++++
> > >> drivers/staging/media/hantro/imx8m_vpu_hw.c | 11 +++++++
> > >> 4 files changed, 46 insertions(+), 3 deletions(-)
> > >>
> > >> diff --git a/drivers/staging/media/hantro/hantro_g2_vp9_dec.c b/drivers/
> > > staging/media/hantro/hantro_g2_vp9_dec.c
> > >> index 7f827b9f0133..1a26be72c878 100644
> > >> --- a/drivers/staging/media/hantro/hantro_g2_vp9_dec.c
> > >> +++ b/drivers/staging/media/hantro/hantro_g2_vp9_dec.c
> > >> @@ -152,7 +152,7 @@ static void config_output(struct hantro_ctx *ctx,
> > >> hantro_reg_write(ctx->dev, &g2_out_dis, 0);
> > >> hantro_reg_write(ctx->dev, &g2_output_format, 0);
> > >>
> > >> - luma_addr = vb2_dma_contig_plane_dma_addr(&dst->base.vb.vb2_buf,
> > > 0);
> > >> + luma_addr = hantro_get_dec_buf_addr(ctx, &dst->base.vb.vb2_buf);
> > >> hantro_write_addr(ctx->dev, G2_OUT_LUMA_ADDR, luma_addr);
> > >>
> > >> chroma_addr = luma_addr + chroma_offset(ctx, dec_params);
> > >> @@ -191,7 +191,7 @@ static void config_ref(struct hantro_ctx *ctx,
> > >> hantro_reg_write(ctx->dev, &ref_reg->hor_scale, (refw << 14) /
> > > dst->vp9.width);
> > >> hantro_reg_write(ctx->dev, &ref_reg->ver_scale, (refh << 14) /
> > > dst->vp9.height);
> > >>
> > >> - luma_addr = vb2_dma_contig_plane_dma_addr(&buf->base.vb.vb2_buf,
> > > 0);
> > >> + luma_addr = hantro_get_dec_buf_addr(ctx, &buf->base.vb.vb2_buf);
> > >> hantro_write_addr(ctx->dev, ref_reg->y_base, luma_addr);
> > >>
> > >> chroma_addr = luma_addr + chroma_offset(ctx, dec_params);
> > >> @@ -236,7 +236,7 @@ static void config_ref_registers(struct hantro_ctx
> *ctx,
> > >> config_ref(ctx, dst, &ref_regs[1], dec_params, dec_params-
> > >> golden_frame_ts);
> > >> config_ref(ctx, dst, &ref_regs[2], dec_params, dec_params-
> > >> alt_frame_ts);
> > >>
> > >> - mv_addr = vb2_dma_contig_plane_dma_addr(&mv_ref->base.vb.vb2_buf,
> > > 0) +
> > >> + mv_addr = hantro_get_dec_buf_addr(ctx, &mv_ref->base.vb.vb2_buf) +
> > >> mv_offset(ctx, dec_params);
> > >> hantro_write_addr(ctx->dev, G2_REF_MV_ADDR(0), mv_addr);
> > >>
> > >> diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/
> > > media/hantro/hantro_hw.h
> > >> index 2961d399fd60..3d4a5dc1e6d5 100644
> > >> --- a/drivers/staging/media/hantro/hantro_hw.h
> > >> +++ b/drivers/staging/media/hantro/hantro_hw.h
> > >> @@ -274,6 +274,7 @@ extern const struct hantro_variant
> rk3399_vpu_variant;
> > >> extern const struct hantro_variant sama5d4_vdec_variant;
> > >>
> > >> extern const struct hantro_postproc_ops hantro_g1_postproc_ops;
> > >> +extern const struct hantro_postproc_ops hantro_g2_postproc_ops;
> > >>
> > >> extern const u32 hantro_vp8_dec_mc_filter[8][6];
> > >>
> > >> diff --git a/drivers/staging/media/hantro/hantro_postproc.c b/drivers/
> > > staging/media/hantro/hantro_postproc.c
> > >> index 4549aec08feb..79a66d001738 100644
> > >> --- a/drivers/staging/media/hantro/hantro_postproc.c
> > >> +++ b/drivers/staging/media/hantro/hantro_postproc.c
> > >> @@ -11,6 +11,7 @@
> > >> #include "hantro.h"
> > >> #include "hantro_hw.h"
> > >> #include "hantro_g1_regs.h"
> > >> +#include "hantro_g2_regs.h"
> > >>
> > >> #define HANTRO_PP_REG_WRITE(vpu, reg_name, val) \
> > >> { \
> > >> @@ -99,6 +100,21 @@ static void hantro_postproc_g1_enable(struct
> hantro_ctx
> > > *ctx)
> > >> HANTRO_PP_REG_WRITE(vpu, display_width, ctx->dst_fmt.width);
> > >> }
> > >>
> > >> +static void hantro_postproc_g2_enable(struct hantro_ctx *ctx)
> > >> +{
> > >> + struct hantro_dev *vpu = ctx->dev;
> > >> + struct vb2_v4l2_buffer *dst_buf;
> > >> + size_t chroma_offset = ctx->dst_fmt.width * ctx->dst_fmt.height;
> > >> + dma_addr_t dst_dma;
> > >> +
> > >> + dst_buf = hantro_get_dst_buf(ctx);
> > >> + dst_dma = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0);
> > >> +
> > >> + hantro_write_addr(vpu, G2_RS_OUT_LUMA_ADDR, dst_dma);
> > >> + hantro_write_addr(vpu, G2_RS_OUT_CHROMA_ADDR, dst_dma +
> > > chroma_offset);
> > >> + hantro_reg_write(vpu, &g2_out_rs_e, 1);
> > >> +}
> > >> +
> > >> void hantro_postproc_free(struct hantro_ctx *ctx)
> > >> {
> > >> struct hantro_dev *vpu = ctx->dev;
> > >> @@ -127,6 +143,9 @@ int hantro_postproc_alloc(struct hantro_ctx *ctx)
> > >> if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_H264_SLICE)
> > >> buf_size += hantro_h264_mv_size(ctx->dst_fmt.width,
> > >> ctx-
> > >> dst_fmt.height);
> > >> + else if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_VP9_FRAME)
> > >> + buf_size += hantro_vp9_mv_size(ctx->dst_fmt.width,
> > >> + ctx-
> > >> dst_fmt.height);
> > >>
> > >> for (i = 0; i < num_buffers; ++i) {
> > >> struct hantro_aux_buf *priv = &ctx->postproc.dec_q[i];
> > >> @@ -152,6 +171,13 @@ static void hantro_postproc_g1_disable(struct
> > > hantro_ctx *ctx)
> > >> HANTRO_PP_REG_WRITE_S(vpu, pipeline_en, 0x0);
> > >> }
> > >>
> > >> +static void hantro_postproc_g2_disable(struct hantro_ctx *ctx)
> > >> +{
> > >> + struct hantro_dev *vpu = ctx->dev;
> > >> +
> > >> + hantro_reg_write(vpu, &g2_out_rs_e, 0);
> > >> +}
> > >> +
> > >> void hantro_postproc_disable(struct hantro_ctx *ctx)
> > >> {
> > >> struct hantro_dev *vpu = ctx->dev;
> > >> @@ -172,3 +198,8 @@ const struct hantro_postproc_ops
> hantro_g1_postproc_ops
> > > = {
> > >> .enable = hantro_postproc_g1_enable,
> > >> .disable = hantro_postproc_g1_disable,
> > >> };
> > >> +
> > >> +const struct hantro_postproc_ops hantro_g2_postproc_ops = {
> > >> + .enable = hantro_postproc_g2_enable,
> > >> + .disable = hantro_postproc_g2_disable,
> > >> +};
> > >> diff --git a/drivers/staging/media/hantro/imx8m_vpu_hw.c b/drivers/
> staging/
> > > media/hantro/imx8m_vpu_hw.c
> > >> index 455a107ffb02..1a43f6fceef9 100644
> > >> --- a/drivers/staging/media/hantro/imx8m_vpu_hw.c
> > >> +++ b/drivers/staging/media/hantro/imx8m_vpu_hw.c
> > >> @@ -132,6 +132,14 @@ static const struct hantro_fmt imx8m_vpu_dec_fmts[]
> = {
> > >> },
> > >> };
> > >>
> > >> +static const struct hantro_fmt imx8m_vpu_g2_postproc_fmts[] = {
> > >> + {
> > >> + .fourcc = V4L2_PIX_FMT_NV12,
> > >> + .codec_mode = HANTRO_MODE_NONE,
> > >> + .postprocessed = true,
> > >> + },
> > >> +};
> > >> +
> > >> static const struct hantro_fmt imx8m_vpu_g2_dec_fmts[] = {
> > >> {
> > >> .fourcc = V4L2_PIX_FMT_NV12_4L4,
> > >> @@ -301,6 +309,9 @@ const struct hantro_variant imx8mq_vpu_g2_variant = {
> > >> .dec_offset = 0x0,
> > >> .dec_fmts = imx8m_vpu_g2_dec_fmts,
> > >> .num_dec_fmts = ARRAY_SIZE(imx8m_vpu_g2_dec_fmts),
> > >> + .postproc_fmts = imx8m_vpu_g2_postproc_fmts,
> > >> + .num_postproc_fmts = ARRAY_SIZE(imx8m_vpu_g2_postproc_fmts),
> > >> + .postproc_ops = &hantro_g2_postproc_ops,
> > >> .codec = HANTRO_HEVC_DECODER | HANTRO_VP9_DECODER,
> > >> .codec_ops = imx8mq_vpu_g2_codec_ops,
> > >> .init = imx8mq_vpu_hw_init,
> > >> --
> > >> 2.17.1
> > >>
> > >>
> > >
> > >
> >
> >
>
>
Dne sreda, 20. oktober 2021 ob 13:06:59 CEST je Ezequiel Garcia napisal(a):
> Hi Jernej,
>
> On Tue, 19 Oct 2021 at 13:38, Jernej Škrabec <[email protected]>
wrote:
> >
> > Hi Andrzej!
> >
> > Dne petek, 15. oktober 2021 ob 19:19:47 CEST je Andrzej Pietrasiewicz
> > napisal(a):
> > > Hi Jernej,
> > >
> > > W dniu 14.10.2021 o 19:42, Jernej Škrabec pisze:
> > > > Hi Andrzej!
> > > >
> > > > Dne sreda, 29. september 2021 ob 18:04:39 CEST je Andrzej
Pietrasiewicz
> > > > napisal(a):
> > > >> The G2 decoder block produces NV12 4x4 tiled format (NV12_4L4).
> > > >> Enable the G2 post-processor block, in order to produce regular NV12.
> > > >>
> > > >> The logic in hantro_postproc.c is leveraged to take care of
allocating
> > > >> the extra buffers and configure the post-processor, which is
> > > >> significantly simpler than the one on the G1.
> > > >
> > > > Quick summary of discussion on LibreELEC Slack:
> > > > When using NV12 format on Allwinner H6 variant of G2 (needs some
driver
> > > > changes), I get frames out of order. If I use native NV12 tiled
format,
> > frames
> > > > are ordered correctly.
> > > >
> > > > Currently I'm not sure if this is issue with my changes or is this
general
> > > > issue.
> > > >
> > > > I would be grateful if anyone can test frame order with and without
> > > > postprocessing enabled on imx8. Take some dynamic video with a lot of
> > short
> > > > scenes. It's pretty obvious when frames are out of order.
> > > >
> > >
> > > I checked on imx8 and cannot observe any such artifacts.
> >
> > I finally found the issue. As you mentioned on Slack, register write order
once
> > already affected decoding. Well, it's the case again. I made hacky test and
> > moved postproc enable call after output buffers are set and it worked. So,
this
> > is actually core quirk which is obviously fixed in newer variants.
> >
>
> Ugh, good catch.
>
> What happens if you move all the calls to HANTRO_PP_REG_WRITE_S
> (HANTRO_PP_REG_WRITE does a relaxed write)?
>
> Or what happens if the HANTRO_PP_REG_WRITE(vpu, out_luma_base, dst_dma)
> is moved to be done after all the other registers?
Those two macros aren't used on G2. Andrzej introduced new postproc helpers
for G2.
This commit solves issue for H6:
https://github.com/jernejsk/linux-1/commit/
a783a977c0843bb4b555dc9d0b5d64915cd219e7
>
> > This makes this series with minor adaptations completely working on H6. I
see
> > no reason not to merge whole series.
> >
>
> Do you have plans to submit your H6 work on top of this?
Of course, why would I work on this otherwise? :) But before I do that, I have
to clean up and split one commit, which adapts VP9 G2 code for H6 variant.
If you're interested in changes, take a look here:
https://github.com/jernejsk/linux-1/commits/vp9
Best regards,
Jernej
>
> Thanks,
> Ezequiel
>
>
> > Thanks for testing.
> >
> > Best regards,
> > Jernej
> >
> > >
> > > Andrzej
> > >
> > > > However, given that frames themself are correctly decoded and without
> > > > postprocessing in right order, that shouldn't block merging previous
> > patches.
> > > > I tried few different videos and frames were all decoded correctly.
> > > >
> > > > Best regards,
> > > > Jernej
> > > >
> > > >>
> > > >> Signed-off-by: Ezequiel Garcia <[email protected]>
> > > >> Signed-off-by: Andrzej Pietrasiewicz <[email protected]>
> > > >> ---
> > > >> .../staging/media/hantro/hantro_g2_vp9_dec.c | 6 ++--
> > > >> drivers/staging/media/hantro/hantro_hw.h | 1 +
> > > >> .../staging/media/hantro/hantro_postproc.c | 31 ++++++++++++++++
+++
> > > >> drivers/staging/media/hantro/imx8m_vpu_hw.c | 11 +++++++
> > > >> 4 files changed, 46 insertions(+), 3 deletions(-)
> > > >>
> > > >> diff --git a/drivers/staging/media/hantro/hantro_g2_vp9_dec.c b/
drivers/
> > > > staging/media/hantro/hantro_g2_vp9_dec.c
> > > >> index 7f827b9f0133..1a26be72c878 100644
> > > >> --- a/drivers/staging/media/hantro/hantro_g2_vp9_dec.c
> > > >> +++ b/drivers/staging/media/hantro/hantro_g2_vp9_dec.c
> > > >> @@ -152,7 +152,7 @@ static void config_output(struct hantro_ctx *ctx,
> > > >> hantro_reg_write(ctx->dev, &g2_out_dis, 0);
> > > >> hantro_reg_write(ctx->dev, &g2_output_format, 0);
> > > >>
> > > >> - luma_addr = vb2_dma_contig_plane_dma_addr(&dst->base.vb.vb2_buf,
> > > > 0);
> > > >> + luma_addr = hantro_get_dec_buf_addr(ctx, &dst->base.vb.vb2_buf);
> > > >> hantro_write_addr(ctx->dev, G2_OUT_LUMA_ADDR, luma_addr);
> > > >>
> > > >> chroma_addr = luma_addr + chroma_offset(ctx, dec_params);
> > > >> @@ -191,7 +191,7 @@ static void config_ref(struct hantro_ctx *ctx,
> > > >> hantro_reg_write(ctx->dev, &ref_reg->hor_scale, (refw << 14) /
> > > > dst->vp9.width);
> > > >> hantro_reg_write(ctx->dev, &ref_reg->ver_scale, (refh << 14) /
> > > > dst->vp9.height);
> > > >>
> > > >> - luma_addr = vb2_dma_contig_plane_dma_addr(&buf->base.vb.vb2_buf,
> > > > 0);
> > > >> + luma_addr = hantro_get_dec_buf_addr(ctx, &buf->base.vb.vb2_buf);
> > > >> hantro_write_addr(ctx->dev, ref_reg->y_base, luma_addr);
> > > >>
> > > >> chroma_addr = luma_addr + chroma_offset(ctx, dec_params);
> > > >> @@ -236,7 +236,7 @@ static void config_ref_registers(struct hantro_ctx
> > *ctx,
> > > >> config_ref(ctx, dst, &ref_regs[1], dec_params, dec_params-
> > > >> golden_frame_ts);
> > > >> config_ref(ctx, dst, &ref_regs[2], dec_params, dec_params-
> > > >> alt_frame_ts);
> > > >>
> > > >> - mv_addr = vb2_dma_contig_plane_dma_addr(&mv_ref->base.vb.vb2_buf,
> > > > 0) +
> > > >> + mv_addr = hantro_get_dec_buf_addr(ctx, &mv_ref->base.vb.vb2_buf) +
> > > >> mv_offset(ctx, dec_params);
> > > >> hantro_write_addr(ctx->dev, G2_REF_MV_ADDR(0), mv_addr);
> > > >>
> > > >> diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/
staging/
> > > > media/hantro/hantro_hw.h
> > > >> index 2961d399fd60..3d4a5dc1e6d5 100644
> > > >> --- a/drivers/staging/media/hantro/hantro_hw.h
> > > >> +++ b/drivers/staging/media/hantro/hantro_hw.h
> > > >> @@ -274,6 +274,7 @@ extern const struct hantro_variant
> > rk3399_vpu_variant;
> > > >> extern const struct hantro_variant sama5d4_vdec_variant;
> > > >>
> > > >> extern const struct hantro_postproc_ops hantro_g1_postproc_ops;
> > > >> +extern const struct hantro_postproc_ops hantro_g2_postproc_ops;
> > > >>
> > > >> extern const u32 hantro_vp8_dec_mc_filter[8][6];
> > > >>
> > > >> diff --git a/drivers/staging/media/hantro/hantro_postproc.c b/drivers/
> > > > staging/media/hantro/hantro_postproc.c
> > > >> index 4549aec08feb..79a66d001738 100644
> > > >> --- a/drivers/staging/media/hantro/hantro_postproc.c
> > > >> +++ b/drivers/staging/media/hantro/hantro_postproc.c
> > > >> @@ -11,6 +11,7 @@
> > > >> #include "hantro.h"
> > > >> #include "hantro_hw.h"
> > > >> #include "hantro_g1_regs.h"
> > > >> +#include "hantro_g2_regs.h"
> > > >>
> > > >> #define HANTRO_PP_REG_WRITE(vpu, reg_name, val) \
> > > >> { \
> > > >> @@ -99,6 +100,21 @@ static void hantro_postproc_g1_enable(struct
> > hantro_ctx
> > > > *ctx)
> > > >> HANTRO_PP_REG_WRITE(vpu, display_width, ctx->dst_fmt.width);
> > > >> }
> > > >>
> > > >> +static void hantro_postproc_g2_enable(struct hantro_ctx *ctx)
> > > >> +{
> > > >> + struct hantro_dev *vpu = ctx->dev;
> > > >> + struct vb2_v4l2_buffer *dst_buf;
> > > >> + size_t chroma_offset = ctx->dst_fmt.width * ctx->dst_fmt.height;
> > > >> + dma_addr_t dst_dma;
> > > >> +
> > > >> + dst_buf = hantro_get_dst_buf(ctx);
> > > >> + dst_dma = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0);
> > > >> +
> > > >> + hantro_write_addr(vpu, G2_RS_OUT_LUMA_ADDR, dst_dma);
> > > >> + hantro_write_addr(vpu, G2_RS_OUT_CHROMA_ADDR, dst_dma +
> > > > chroma_offset);
> > > >> + hantro_reg_write(vpu, &g2_out_rs_e, 1);
> > > >> +}
> > > >> +
> > > >> void hantro_postproc_free(struct hantro_ctx *ctx)
> > > >> {
> > > >> struct hantro_dev *vpu = ctx->dev;
> > > >> @@ -127,6 +143,9 @@ int hantro_postproc_alloc(struct hantro_ctx *ctx)
> > > >> if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_H264_SLICE)
> > > >> buf_size += hantro_h264_mv_size(ctx->dst_fmt.width,
> > > >> ctx-
> > > >> dst_fmt.height);
> > > >> + else if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_VP9_FRAME)
> > > >> + buf_size += hantro_vp9_mv_size(ctx->dst_fmt.width,
> > > >> + ctx-
> > > >> dst_fmt.height);
> > > >>
> > > >> for (i = 0; i < num_buffers; ++i) {
> > > >> struct hantro_aux_buf *priv = &ctx->postproc.dec_q[i];
> > > >> @@ -152,6 +171,13 @@ static void hantro_postproc_g1_disable(struct
> > > > hantro_ctx *ctx)
> > > >> HANTRO_PP_REG_WRITE_S(vpu, pipeline_en, 0x0);
> > > >> }
> > > >>
> > > >> +static void hantro_postproc_g2_disable(struct hantro_ctx *ctx)
> > > >> +{
> > > >> + struct hantro_dev *vpu = ctx->dev;
> > > >> +
> > > >> + hantro_reg_write(vpu, &g2_out_rs_e, 0);
> > > >> +}
> > > >> +
> > > >> void hantro_postproc_disable(struct hantro_ctx *ctx)
> > > >> {
> > > >> struct hantro_dev *vpu = ctx->dev;
> > > >> @@ -172,3 +198,8 @@ const struct hantro_postproc_ops
> > hantro_g1_postproc_ops
> > > > = {
> > > >> .enable = hantro_postproc_g1_enable,
> > > >> .disable = hantro_postproc_g1_disable,
> > > >> };
> > > >> +
> > > >> +const struct hantro_postproc_ops hantro_g2_postproc_ops = {
> > > >> + .enable = hantro_postproc_g2_enable,
> > > >> + .disable = hantro_postproc_g2_disable,
> > > >> +};
> > > >> diff --git a/drivers/staging/media/hantro/imx8m_vpu_hw.c b/drivers/
> > staging/
> > > > media/hantro/imx8m_vpu_hw.c
> > > >> index 455a107ffb02..1a43f6fceef9 100644
> > > >> --- a/drivers/staging/media/hantro/imx8m_vpu_hw.c
> > > >> +++ b/drivers/staging/media/hantro/imx8m_vpu_hw.c
> > > >> @@ -132,6 +132,14 @@ static const struct hantro_fmt
imx8m_vpu_dec_fmts[]
> > = {
> > > >> },
> > > >> };
> > > >>
> > > >> +static const struct hantro_fmt imx8m_vpu_g2_postproc_fmts[] = {
> > > >> + {
> > > >> + .fourcc = V4L2_PIX_FMT_NV12,
> > > >> + .codec_mode = HANTRO_MODE_NONE,
> > > >> + .postprocessed = true,
> > > >> + },
> > > >> +};
> > > >> +
> > > >> static const struct hantro_fmt imx8m_vpu_g2_dec_fmts[] = {
> > > >> {
> > > >> .fourcc = V4L2_PIX_FMT_NV12_4L4,
> > > >> @@ -301,6 +309,9 @@ const struct hantro_variant imx8mq_vpu_g2_variant
= {
> > > >> .dec_offset = 0x0,
> > > >> .dec_fmts = imx8m_vpu_g2_dec_fmts,
> > > >> .num_dec_fmts = ARRAY_SIZE(imx8m_vpu_g2_dec_fmts),
> > > >> + .postproc_fmts = imx8m_vpu_g2_postproc_fmts,
> > > >> + .num_postproc_fmts = ARRAY_SIZE(imx8m_vpu_g2_postproc_fmts),
> > > >> + .postproc_ops = &hantro_g2_postproc_ops,
> > > >> .codec = HANTRO_HEVC_DECODER | HANTRO_VP9_DECODER,
> > > >> .codec_ops = imx8mq_vpu_g2_codec_ops,
> > > >> .init = imx8mq_vpu_hw_init,
> > > >> --
> > > >> 2.17.1
> > > >>
> > > >>
> > > >
> > > >
> > >
> > >
> >
> >
>
On Wed, 20 Oct 2021 at 12:04, Jernej Škrabec <[email protected]> wrote:
>
> Dne sreda, 20. oktober 2021 ob 13:06:59 CEST je Ezequiel Garcia napisal(a):
> > Hi Jernej,
> >
> > On Tue, 19 Oct 2021 at 13:38, Jernej Škrabec <[email protected]>
> wrote:
> > >
> > > Hi Andrzej!
> > >
> > > Dne petek, 15. oktober 2021 ob 19:19:47 CEST je Andrzej Pietrasiewicz
> > > napisal(a):
> > > > Hi Jernej,
> > > >
> > > > W dniu 14.10.2021 o 19:42, Jernej Škrabec pisze:
> > > > > Hi Andrzej!
> > > > >
> > > > > Dne sreda, 29. september 2021 ob 18:04:39 CEST je Andrzej
> Pietrasiewicz
> > > > > napisal(a):
> > > > >> The G2 decoder block produces NV12 4x4 tiled format (NV12_4L4).
> > > > >> Enable the G2 post-processor block, in order to produce regular NV12.
> > > > >>
> > > > >> The logic in hantro_postproc.c is leveraged to take care of
> allocating
> > > > >> the extra buffers and configure the post-processor, which is
> > > > >> significantly simpler than the one on the G1.
> > > > >
> > > > > Quick summary of discussion on LibreELEC Slack:
> > > > > When using NV12 format on Allwinner H6 variant of G2 (needs some
> driver
> > > > > changes), I get frames out of order. If I use native NV12 tiled
> format,
> > > frames
> > > > > are ordered correctly.
> > > > >
> > > > > Currently I'm not sure if this is issue with my changes or is this
> general
> > > > > issue.
> > > > >
> > > > > I would be grateful if anyone can test frame order with and without
> > > > > postprocessing enabled on imx8. Take some dynamic video with a lot of
> > > short
> > > > > scenes. It's pretty obvious when frames are out of order.
> > > > >
> > > >
> > > > I checked on imx8 and cannot observe any such artifacts.
> > >
> > > I finally found the issue. As you mentioned on Slack, register write order
> once
> > > already affected decoding. Well, it's the case again. I made hacky test and
> > > moved postproc enable call after output buffers are set and it worked. So,
> this
> > > is actually core quirk which is obviously fixed in newer variants.
> > >
> >
> > Ugh, good catch.
> >
> > What happens if you move all the calls to HANTRO_PP_REG_WRITE_S
> > (HANTRO_PP_REG_WRITE does a relaxed write)?
> >
> > Or what happens if the HANTRO_PP_REG_WRITE(vpu, out_luma_base, dst_dma)
> > is moved to be done after all the other registers?
>
> Those two macros aren't used on G2. Andrzej introduced new postproc helpers
> for G2.
>
Ah, so the issue is specific on the G2 post-processor.
> This commit solves issue for H6:
> https://github.com/jernejsk/linux-1/commit/
> a783a977c0843bb4b555dc9d0b5d64915cd219e7
>
Right, but see this comment:
/* Turn on pipeline mode. Must be done first. */
HANTRO_PP_REG_WRITE_S(vpu, pipeline_en, 0x1);
I have vague recollection of why we have that comment,
but I'm reluctant to move post-proc enable to the end.
(or at least not do it on G1?).
> >
> > > This makes this series with minor adaptations completely working on H6. I
> see
> > > no reason not to merge whole series.
> > >
> >
> > Do you have plans to submit your H6 work on top of this?
>
> Of course, why would I work on this otherwise? :) But before I do that, I have
> to clean up and split one commit, which adapts VP9 G2 code for H6 variant.
>
OK, sounds good.
> If you're interested in changes, take a look here:
> https://github.com/jernejsk/linux-1/commits/vp9
>
Will take a look.
Thanks,
Ezequiel
Dne sreda, 20. oktober 2021 ob 17:25:40 CEST je Ezequiel Garcia napisal(a):
> On Wed, 20 Oct 2021 at 12:04, Jernej Škrabec <[email protected]>
wrote:
> >
> > Dne sreda, 20. oktober 2021 ob 13:06:59 CEST je Ezequiel Garcia
napisal(a):
> > > Hi Jernej,
> > >
> > > On Tue, 19 Oct 2021 at 13:38, Jernej Škrabec <[email protected]>
> > wrote:
> > > >
> > > > Hi Andrzej!
> > > >
> > > > Dne petek, 15. oktober 2021 ob 19:19:47 CEST je Andrzej Pietrasiewicz
> > > > napisal(a):
> > > > > Hi Jernej,
> > > > >
> > > > > W dniu 14.10.2021 o 19:42, Jernej Škrabec pisze:
> > > > > > Hi Andrzej!
> > > > > >
> > > > > > Dne sreda, 29. september 2021 ob 18:04:39 CEST je Andrzej
> > Pietrasiewicz
> > > > > > napisal(a):
> > > > > >> The G2 decoder block produces NV12 4x4 tiled format (NV12_4L4).
> > > > > >> Enable the G2 post-processor block, in order to produce regular
NV12.
> > > > > >>
> > > > > >> The logic in hantro_postproc.c is leveraged to take care of
> > allocating
> > > > > >> the extra buffers and configure the post-processor, which is
> > > > > >> significantly simpler than the one on the G1.
> > > > > >
> > > > > > Quick summary of discussion on LibreELEC Slack:
> > > > > > When using NV12 format on Allwinner H6 variant of G2 (needs some
> > driver
> > > > > > changes), I get frames out of order. If I use native NV12 tiled
> > format,
> > > > frames
> > > > > > are ordered correctly.
> > > > > >
> > > > > > Currently I'm not sure if this is issue with my changes or is this
> > general
> > > > > > issue.
> > > > > >
> > > > > > I would be grateful if anyone can test frame order with and
without
> > > > > > postprocessing enabled on imx8. Take some dynamic video with a lot
of
> > > > short
> > > > > > scenes. It's pretty obvious when frames are out of order.
> > > > > >
> > > > >
> > > > > I checked on imx8 and cannot observe any such artifacts.
> > > >
> > > > I finally found the issue. As you mentioned on Slack, register write
order
> > once
> > > > already affected decoding. Well, it's the case again. I made hacky test
and
> > > > moved postproc enable call after output buffers are set and it worked.
So,
> > this
> > > > is actually core quirk which is obviously fixed in newer variants.
> > > >
> > >
> > > Ugh, good catch.
> > >
> > > What happens if you move all the calls to HANTRO_PP_REG_WRITE_S
> > > (HANTRO_PP_REG_WRITE does a relaxed write)?
> > >
> > > Or what happens if the HANTRO_PP_REG_WRITE(vpu, out_luma_base, dst_dma)
> > > is moved to be done after all the other registers?
> >
> > Those two macros aren't used on G2. Andrzej introduced new postproc
helpers
> > for G2.
> >
>
> Ah, so the issue is specific on the G2 post-processor.
To be more precise, issue is specific only to old G2 post-processor, found in
Allwinner H6. Andrzej tested code with newer G2 core and both locations worked
fine.
>
> > This commit solves issue for H6:
> > https://github.com/jernejsk/linux-1/commit/
> > a783a977c0843bb4b555dc9d0b5d64915cd219e7
> >
>
> Right, but see this comment:
>
> /* Turn on pipeline mode. Must be done first. */
> HANTRO_PP_REG_WRITE_S(vpu, pipeline_en, 0x1);
>
> I have vague recollection of why we have that comment,
> but I'm reluctant to move post-proc enable to the end.
> (or at least not do it on G1?).
I missed that. Any idea what would be the cleanest way to move code for G2
only? I can only think of quirk flag in platform specific structure.
Best regards,
Jernej
>
> > >
> > > > This makes this series with minor adaptations completely working on
H6. I
> > see
> > > > no reason not to merge whole series.
> > > >
> > >
> > > Do you have plans to submit your H6 work on top of this?
> >
> > Of course, why would I work on this otherwise? :) But before I do that, I
have
> > to clean up and split one commit, which adapts VP9 G2 code for H6 variant.
> >
>
> OK, sounds good.
>
> > If you're interested in changes, take a look here:
> > https://github.com/jernejsk/linux-1/commits/vp9
> >
>
> Will take a look.
>
> Thanks,
> Ezequiel
>
Hi all,
Andrzej, Jernej, Nicolas, if none of you (or anyone else for that matter)
objects, then I'll make a PR for this early next week.
Regards,
Hans
On 29/09/2021 18:04, Andrzej Pietrasiewicz wrote:
> Dear all,
>
> This patch series adds VP9 codec V4L2 control interface and two drivers
> using the new controls. It is a follow-up of previous v6 series [1].
>
> In this iteration, we've implemented VP9 hardware decoding on two devices:
> Rockchip VDEC and Hantro G2, and tested on RK3399, i.MX8MQ and i.MX8MP.
> The i.MX8M driver needs proper power domains support, though, which is a
> subject of a different effort, but in all 3 cases we were able to run the
> drivers.
>
> GStreamer support is also available, the needed changes have been submitted
> by Daniel Almeida [2]. This MR is ready to be merged, and just needs the
> VP9 V4L2 controls to be merged and released.
>
> Both rkvdec and hantro drivers are passing a significant number of VP9 tests
> using Fluster[3]. There are still a few tests that are not passing, due to
> dynamic frame resize (not yet supported by V4L2) and small size videos
> (due to IP block limitations).
>
> The series adds the VP9 codec V4L2 control API as uAPI, so it aims at being
> merged without passing through staging, as agreed[4]. The ABI has been checked
> for padding and verified to contain no holes.
>
> [1] https://patchwork.linuxtv.org/project/linux-media/list/?series=6377
> [2] https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2144
> [3] https://github.com/fluendo/fluster
> [4] https://lore.kernel.org/linux-media/[email protected]/
>
> The series depends on the YUV tiled format support prepared by Ezequiel:
> https://www.spinics.net/lists/linux-media/msg197047.html
>
> Rebased onto latest media_tree.
>
> Changes related to v6:
> - moved setting tile filter and tile bsd auxiliary buffer addresses so
> that they are always set, even if no tiles are used (thanks, Jernej)
> - added a comment near the place where the 32-bit DMA mask is applied
> (thanks, Nicolas)
> - improved consistency in register names (thanks, Nicolas)
>
> Changes related to v5:
> - improved the doc comments as per Ezequiel's review (thanks, Ezequiel)
> - improved pdf output of documentation
> - added Benjamin's Reviewed-by (thanks, Benjamin)
>
> Changes related to v4:
> - removed unused enum v4l2_vp9_intra_prediction_mode
> - converted remaining enums to defines to follow the convention
> - improved the documentation, in particular better documented how to use segmentation
> features
>
> Changes related to v3:
>
> Apply suggestions from Jernej's review (thanks, Jernej):
> - renamed a control and two structs:
> V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR_PROBS =>
> V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR
> v4l2_ctrl_vp9_compressed_hdr_probs =>
> v4l2_ctrl_vp9_compressed_hdr
> v4l2_vp9_mv_compressed_hdr_probs => v4l2_vp9_mv_probs
> - moved tx_mode to v4l2_ctrl_vp9_compressed_hdr
> - fixed enum v4l2_vp9_ref_frame_sign_bias values (which are used to test a bitfield)
> - explicitly assigned values to all other vp9 enums
>
> Apply suggestion from Nicolas's review (thanks, Nicolas):
> - explicitly stated that the v4l2_ctrl_vp9_compressed_hdr control is optional
> and implemented only by drivers which need it
>
> Changes related to the RFC v2:
>
> - added another driver including a postprocessor to de-tile
> codec-specific tiling
> - reworked uAPI structs layout to follow VP8 style
> - changed validation of loop filter params
> - changed validation of segmentation params
> - changed validation of VP9 frame params
> - removed level lookup array from loop filter struct
> (can be computed by drivers)
> - renamed some enum values to match the spec more closely
> - V4L2 VP9 library changed the 'eob' member of
> 'struct v4l2_vp9_frame_symbol_counts' so that it is an array
> of pointers instead of an array of pointers to arrays
> (IPs such as g2 creatively pass parts of the 'eob' counts in
> the 'coeff' counts)
> - factored out several repeated portions of code
> - minor nitpicks and cleanups
>
> Andrzej Pietrasiewicz (6):
> media: uapi: Add VP9 stateless decoder controls
> media: Add VP9 v4l2 library
> media: hantro: Rename registers
> media: hantro: Prepare for other G2 codecs
> media: hantro: Support VP9 on the G2 core
> media: hantro: Support NV12 on the G2 core
>
> Boris Brezillon (1):
> media: rkvdec: Add the VP9 backend
>
> Ezequiel Garcia (4):
> hantro: postproc: Fix motion vector space size
> hantro: postproc: Introduce struct hantro_postproc_ops
> hantro: Simplify postprocessor
> hantro: Add quirk for NV12/NV12_4L4 capture format
>
> .../userspace-api/media/v4l/biblio.rst | 10 +
> .../media/v4l/ext-ctrls-codec-stateless.rst | 573 +++++
> .../media/v4l/pixfmt-compressed.rst | 15 +
> .../media/v4l/vidioc-g-ext-ctrls.rst | 8 +
> .../media/v4l/vidioc-queryctrl.rst | 12 +
> .../media/videodev2.h.rst.exceptions | 2 +
> drivers/media/v4l2-core/Kconfig | 4 +
> drivers/media/v4l2-core/Makefile | 1 +
> drivers/media/v4l2-core/v4l2-ctrls-core.c | 180 ++
> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 8 +
> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
> drivers/media/v4l2-core/v4l2-vp9.c | 1850 +++++++++++++++++
> drivers/staging/media/hantro/Kconfig | 1 +
> drivers/staging/media/hantro/Makefile | 7 +-
> drivers/staging/media/hantro/hantro.h | 40 +-
> drivers/staging/media/hantro/hantro_drv.c | 23 +-
> drivers/staging/media/hantro/hantro_g2.c | 27 +
> .../staging/media/hantro/hantro_g2_hevc_dec.c | 69 +-
> drivers/staging/media/hantro/hantro_g2_regs.h | 132 +-
> .../staging/media/hantro/hantro_g2_vp9_dec.c | 980 +++++++++
> drivers/staging/media/hantro/hantro_hw.h | 83 +-
> .../staging/media/hantro/hantro_postproc.c | 79 +-
> drivers/staging/media/hantro/hantro_v4l2.c | 20 +
> drivers/staging/media/hantro/hantro_vp9.c | 240 +++
> drivers/staging/media/hantro/hantro_vp9.h | 103 +
> drivers/staging/media/hantro/imx8m_vpu_hw.c | 38 +-
> .../staging/media/hantro/rockchip_vpu_hw.c | 7 +-
> .../staging/media/hantro/sama5d4_vdec_hw.c | 3 +-
> drivers/staging/media/rkvdec/Kconfig | 1 +
> drivers/staging/media/rkvdec/Makefile | 2 +-
> drivers/staging/media/rkvdec/rkvdec-vp9.c | 1078 ++++++++++
> drivers/staging/media/rkvdec/rkvdec.c | 52 +-
> drivers/staging/media/rkvdec/rkvdec.h | 12 +-
> include/media/v4l2-ctrls.h | 4 +
> include/media/v4l2-vp9.h | 182 ++
> include/uapi/linux/v4l2-controls.h | 284 +++
> include/uapi/linux/videodev2.h | 6 +
> 37 files changed, 6033 insertions(+), 104 deletions(-)
> create mode 100644 drivers/media/v4l2-core/v4l2-vp9.c
> create mode 100644 drivers/staging/media/hantro/hantro_g2.c
> create mode 100644 drivers/staging/media/hantro/hantro_g2_vp9_dec.c
> create mode 100644 drivers/staging/media/hantro/hantro_vp9.c
> create mode 100644 drivers/staging/media/hantro/hantro_vp9.h
> create mode 100644 drivers/staging/media/rkvdec/rkvdec-vp9.c
> create mode 100644 include/media/v4l2-vp9.h
>
>
> base-commit: e4e737bb5c170df6135a127739a9e6148ee3da82
>
Hi Hans,
Le jeudi 11 novembre 2021 à 15:44 +0100, Hans Verkuil a écrit :
> Hi all,
>
> Andrzej, Jernej, Nicolas, if none of you (or anyone else for that matter)
> objects, then I'll make a PR for this early next week.
I have no objection. I've myself delayed replying as we have been digging a lot
into our compliance failures, but I believe we have explained most of them by
now and nothing seems to be related to the API.
regards,
Nicolas
>
> Regards,
>
> Hans
>
> On 29/09/2021 18:04, Andrzej Pietrasiewicz wrote:
> > Dear all,
> >
> > This patch series adds VP9 codec V4L2 control interface and two drivers
> > using the new controls. It is a follow-up of previous v6 series [1].
> >
> > In this iteration, we've implemented VP9 hardware decoding on two devices:
> > Rockchip VDEC and Hantro G2, and tested on RK3399, i.MX8MQ and i.MX8MP.
> > The i.MX8M driver needs proper power domains support, though, which is a
> > subject of a different effort, but in all 3 cases we were able to run the
> > drivers.
> >
> > GStreamer support is also available, the needed changes have been submitted
> > by Daniel Almeida [2]. This MR is ready to be merged, and just needs the
> > VP9 V4L2 controls to be merged and released.
> >
> > Both rkvdec and hantro drivers are passing a significant number of VP9 tests
> > using Fluster[3]. There are still a few tests that are not passing, due to
> > dynamic frame resize (not yet supported by V4L2) and small size videos
> > (due to IP block limitations).
> >
> > The series adds the VP9 codec V4L2 control API as uAPI, so it aims at being
> > merged without passing through staging, as agreed[4]. The ABI has been checked
> > for padding and verified to contain no holes.
> >
> > [1] https://patchwork.linuxtv.org/project/linux-media/list/?series=6377
> > [2] https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2144
> > [3] https://github.com/fluendo/fluster
> > [4] https://lore.kernel.org/linux-media/[email protected]/
> >
> > The series depends on the YUV tiled format support prepared by Ezequiel:
> > https://www.spinics.net/lists/linux-media/msg197047.html
> >
> > Rebased onto latest media_tree.
> >
> > Changes related to v6:
> > - moved setting tile filter and tile bsd auxiliary buffer addresses so
> > that they are always set, even if no tiles are used (thanks, Jernej)
> > - added a comment near the place where the 32-bit DMA mask is applied
> > (thanks, Nicolas)
> > - improved consistency in register names (thanks, Nicolas)
> >
> > Changes related to v5:
> > - improved the doc comments as per Ezequiel's review (thanks, Ezequiel)
> > - improved pdf output of documentation
> > - added Benjamin's Reviewed-by (thanks, Benjamin)
> >
> > Changes related to v4:
> > - removed unused enum v4l2_vp9_intra_prediction_mode
> > - converted remaining enums to defines to follow the convention
> > - improved the documentation, in particular better documented how to use segmentation
> > features
> >
> > Changes related to v3:
> >
> > Apply suggestions from Jernej's review (thanks, Jernej):
> > - renamed a control and two structs:
> > V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR_PROBS =>
> > V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR
> > v4l2_ctrl_vp9_compressed_hdr_probs =>
> > v4l2_ctrl_vp9_compressed_hdr
> > v4l2_vp9_mv_compressed_hdr_probs => v4l2_vp9_mv_probs
> > - moved tx_mode to v4l2_ctrl_vp9_compressed_hdr
> > - fixed enum v4l2_vp9_ref_frame_sign_bias values (which are used to test a bitfield)
> > - explicitly assigned values to all other vp9 enums
> >
> > Apply suggestion from Nicolas's review (thanks, Nicolas):
> > - explicitly stated that the v4l2_ctrl_vp9_compressed_hdr control is optional
> > and implemented only by drivers which need it
> >
> > Changes related to the RFC v2:
> >
> > - added another driver including a postprocessor to de-tile
> > codec-specific tiling
> > - reworked uAPI structs layout to follow VP8 style
> > - changed validation of loop filter params
> > - changed validation of segmentation params
> > - changed validation of VP9 frame params
> > - removed level lookup array from loop filter struct
> > (can be computed by drivers)
> > - renamed some enum values to match the spec more closely
> > - V4L2 VP9 library changed the 'eob' member of
> > 'struct v4l2_vp9_frame_symbol_counts' so that it is an array
> > of pointers instead of an array of pointers to arrays
> > (IPs such as g2 creatively pass parts of the 'eob' counts in
> > the 'coeff' counts)
> > - factored out several repeated portions of code
> > - minor nitpicks and cleanups
> >
> > Andrzej Pietrasiewicz (6):
> > media: uapi: Add VP9 stateless decoder controls
> > media: Add VP9 v4l2 library
> > media: hantro: Rename registers
> > media: hantro: Prepare for other G2 codecs
> > media: hantro: Support VP9 on the G2 core
> > media: hantro: Support NV12 on the G2 core
> >
> > Boris Brezillon (1):
> > media: rkvdec: Add the VP9 backend
> >
> > Ezequiel Garcia (4):
> > hantro: postproc: Fix motion vector space size
> > hantro: postproc: Introduce struct hantro_postproc_ops
> > hantro: Simplify postprocessor
> > hantro: Add quirk for NV12/NV12_4L4 capture format
> >
> > .../userspace-api/media/v4l/biblio.rst | 10 +
> > .../media/v4l/ext-ctrls-codec-stateless.rst | 573 +++++
> > .../media/v4l/pixfmt-compressed.rst | 15 +
> > .../media/v4l/vidioc-g-ext-ctrls.rst | 8 +
> > .../media/v4l/vidioc-queryctrl.rst | 12 +
> > .../media/videodev2.h.rst.exceptions | 2 +
> > drivers/media/v4l2-core/Kconfig | 4 +
> > drivers/media/v4l2-core/Makefile | 1 +
> > drivers/media/v4l2-core/v4l2-ctrls-core.c | 180 ++
> > drivers/media/v4l2-core/v4l2-ctrls-defs.c | 8 +
> > drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
> > drivers/media/v4l2-core/v4l2-vp9.c | 1850 +++++++++++++++++
> > drivers/staging/media/hantro/Kconfig | 1 +
> > drivers/staging/media/hantro/Makefile | 7 +-
> > drivers/staging/media/hantro/hantro.h | 40 +-
> > drivers/staging/media/hantro/hantro_drv.c | 23 +-
> > drivers/staging/media/hantro/hantro_g2.c | 27 +
> > .../staging/media/hantro/hantro_g2_hevc_dec.c | 69 +-
> > drivers/staging/media/hantro/hantro_g2_regs.h | 132 +-
> > .../staging/media/hantro/hantro_g2_vp9_dec.c | 980 +++++++++
> > drivers/staging/media/hantro/hantro_hw.h | 83 +-
> > .../staging/media/hantro/hantro_postproc.c | 79 +-
> > drivers/staging/media/hantro/hantro_v4l2.c | 20 +
> > drivers/staging/media/hantro/hantro_vp9.c | 240 +++
> > drivers/staging/media/hantro/hantro_vp9.h | 103 +
> > drivers/staging/media/hantro/imx8m_vpu_hw.c | 38 +-
> > .../staging/media/hantro/rockchip_vpu_hw.c | 7 +-
> > .../staging/media/hantro/sama5d4_vdec_hw.c | 3 +-
> > drivers/staging/media/rkvdec/Kconfig | 1 +
> > drivers/staging/media/rkvdec/Makefile | 2 +-
> > drivers/staging/media/rkvdec/rkvdec-vp9.c | 1078 ++++++++++
> > drivers/staging/media/rkvdec/rkvdec.c | 52 +-
> > drivers/staging/media/rkvdec/rkvdec.h | 12 +-
> > include/media/v4l2-ctrls.h | 4 +
> > include/media/v4l2-vp9.h | 182 ++
> > include/uapi/linux/v4l2-controls.h | 284 +++
> > include/uapi/linux/videodev2.h | 6 +
> > 37 files changed, 6033 insertions(+), 104 deletions(-)
> > create mode 100644 drivers/media/v4l2-core/v4l2-vp9.c
> > create mode 100644 drivers/staging/media/hantro/hantro_g2.c
> > create mode 100644 drivers/staging/media/hantro/hantro_g2_vp9_dec.c
> > create mode 100644 drivers/staging/media/hantro/hantro_vp9.c
> > create mode 100644 drivers/staging/media/hantro/hantro_vp9.h
> > create mode 100644 drivers/staging/media/rkvdec/rkvdec-vp9.c
> > create mode 100644 include/media/v4l2-vp9.h
> >
> >
> > base-commit: e4e737bb5c170df6135a127739a9e6148ee3da82
> >
>
Hi Hans,
W dniu 12.11.2021 o 16:27, Nicolas Dufresne pisze:
> Hi Hans,
>
> Le jeudi 11 novembre 2021 à 15:44 +0100, Hans Verkuil a écrit :
>> Hi all,
>>
>> Andrzej, Jernej, Nicolas, if none of you (or anyone else for that matter)
>> objects, then I'll make a PR for this early next week.
>
> I have no objection. I've myself delayed replying as we have been digging a lot
> into our compliance failures, but I believe we have explained most of them by
> now and nothing seems to be related to the API.
>
> regards,
> Nicolas
I'm fine with making a PR, too.
Andrzej
>
>>
>> Regards,
>>
>> Hans
>>
>> On 29/09/2021 18:04, Andrzej Pietrasiewicz wrote:
>>> Dear all,
>>>
>>> This patch series adds VP9 codec V4L2 control interface and two drivers
>>> using the new controls. It is a follow-up of previous v6 series [1].
>>>
>>> In this iteration, we've implemented VP9 hardware decoding on two devices:
>>> Rockchip VDEC and Hantro G2, and tested on RK3399, i.MX8MQ and i.MX8MP.
>>> The i.MX8M driver needs proper power domains support, though, which is a
>>> subject of a different effort, but in all 3 cases we were able to run the
>>> drivers.
>>>
>>> GStreamer support is also available, the needed changes have been submitted
>>> by Daniel Almeida [2]. This MR is ready to be merged, and just needs the
>>> VP9 V4L2 controls to be merged and released.
>>>
>>> Both rkvdec and hantro drivers are passing a significant number of VP9 tests
>>> using Fluster[3]. There are still a few tests that are not passing, due to
>>> dynamic frame resize (not yet supported by V4L2) and small size videos
>>> (due to IP block limitations).
>>>
>>> The series adds the VP9 codec V4L2 control API as uAPI, so it aims at being
>>> merged without passing through staging, as agreed[4]. The ABI has been checked
>>> for padding and verified to contain no holes.
>>>
>>> [1] https://patchwork.linuxtv.org/project/linux-media/list/?series=6377
>>> [2] https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2144
>>> [3] https://github.com/fluendo/fluster
>>> [4] https://lore.kernel.org/linux-media/[email protected]/
>>>
>>> The series depends on the YUV tiled format support prepared by Ezequiel:
>>> https://www.spinics.net/lists/linux-media/msg197047.html
>>>
>>> Rebased onto latest media_tree.
>>>
>>> Changes related to v6:
>>> - moved setting tile filter and tile bsd auxiliary buffer addresses so
>>> that they are always set, even if no tiles are used (thanks, Jernej)
>>> - added a comment near the place where the 32-bit DMA mask is applied
>>> (thanks, Nicolas)
>>> - improved consistency in register names (thanks, Nicolas)
>>>
>>> Changes related to v5:
>>> - improved the doc comments as per Ezequiel's review (thanks, Ezequiel)
>>> - improved pdf output of documentation
>>> - added Benjamin's Reviewed-by (thanks, Benjamin)
>>>
>>> Changes related to v4:
>>> - removed unused enum v4l2_vp9_intra_prediction_mode
>>> - converted remaining enums to defines to follow the convention
>>> - improved the documentation, in particular better documented how to use segmentation
>>> features
>>>
>>> Changes related to v3:
>>>
>>> Apply suggestions from Jernej's review (thanks, Jernej):
>>> - renamed a control and two structs:
>>> V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR_PROBS =>
>>> V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR
>>> v4l2_ctrl_vp9_compressed_hdr_probs =>
>>> v4l2_ctrl_vp9_compressed_hdr
>>> v4l2_vp9_mv_compressed_hdr_probs => v4l2_vp9_mv_probs
>>> - moved tx_mode to v4l2_ctrl_vp9_compressed_hdr
>>> - fixed enum v4l2_vp9_ref_frame_sign_bias values (which are used to test a bitfield)
>>> - explicitly assigned values to all other vp9 enums
>>>
>>> Apply suggestion from Nicolas's review (thanks, Nicolas):
>>> - explicitly stated that the v4l2_ctrl_vp9_compressed_hdr control is optional
>>> and implemented only by drivers which need it
>>>
>>> Changes related to the RFC v2:
>>>
>>> - added another driver including a postprocessor to de-tile
>>> codec-specific tiling
>>> - reworked uAPI structs layout to follow VP8 style
>>> - changed validation of loop filter params
>>> - changed validation of segmentation params
>>> - changed validation of VP9 frame params
>>> - removed level lookup array from loop filter struct
>>> (can be computed by drivers)
>>> - renamed some enum values to match the spec more closely
>>> - V4L2 VP9 library changed the 'eob' member of
>>> 'struct v4l2_vp9_frame_symbol_counts' so that it is an array
>>> of pointers instead of an array of pointers to arrays
>>> (IPs such as g2 creatively pass parts of the 'eob' counts in
>>> the 'coeff' counts)
>>> - factored out several repeated portions of code
>>> - minor nitpicks and cleanups
>>>
>>> Andrzej Pietrasiewicz (6):
>>> media: uapi: Add VP9 stateless decoder controls
>>> media: Add VP9 v4l2 library
>>> media: hantro: Rename registers
>>> media: hantro: Prepare for other G2 codecs
>>> media: hantro: Support VP9 on the G2 core
>>> media: hantro: Support NV12 on the G2 core
>>>
>>> Boris Brezillon (1):
>>> media: rkvdec: Add the VP9 backend
>>>
>>> Ezequiel Garcia (4):
>>> hantro: postproc: Fix motion vector space size
>>> hantro: postproc: Introduce struct hantro_postproc_ops
>>> hantro: Simplify postprocessor
>>> hantro: Add quirk for NV12/NV12_4L4 capture format
>>>
>>> .../userspace-api/media/v4l/biblio.rst | 10 +
>>> .../media/v4l/ext-ctrls-codec-stateless.rst | 573 +++++
>>> .../media/v4l/pixfmt-compressed.rst | 15 +
>>> .../media/v4l/vidioc-g-ext-ctrls.rst | 8 +
>>> .../media/v4l/vidioc-queryctrl.rst | 12 +
>>> .../media/videodev2.h.rst.exceptions | 2 +
>>> drivers/media/v4l2-core/Kconfig | 4 +
>>> drivers/media/v4l2-core/Makefile | 1 +
>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 180 ++
>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 8 +
>>> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
>>> drivers/media/v4l2-core/v4l2-vp9.c | 1850 +++++++++++++++++
>>> drivers/staging/media/hantro/Kconfig | 1 +
>>> drivers/staging/media/hantro/Makefile | 7 +-
>>> drivers/staging/media/hantro/hantro.h | 40 +-
>>> drivers/staging/media/hantro/hantro_drv.c | 23 +-
>>> drivers/staging/media/hantro/hantro_g2.c | 27 +
>>> .../staging/media/hantro/hantro_g2_hevc_dec.c | 69 +-
>>> drivers/staging/media/hantro/hantro_g2_regs.h | 132 +-
>>> .../staging/media/hantro/hantro_g2_vp9_dec.c | 980 +++++++++
>>> drivers/staging/media/hantro/hantro_hw.h | 83 +-
>>> .../staging/media/hantro/hantro_postproc.c | 79 +-
>>> drivers/staging/media/hantro/hantro_v4l2.c | 20 +
>>> drivers/staging/media/hantro/hantro_vp9.c | 240 +++
>>> drivers/staging/media/hantro/hantro_vp9.h | 103 +
>>> drivers/staging/media/hantro/imx8m_vpu_hw.c | 38 +-
>>> .../staging/media/hantro/rockchip_vpu_hw.c | 7 +-
>>> .../staging/media/hantro/sama5d4_vdec_hw.c | 3 +-
>>> drivers/staging/media/rkvdec/Kconfig | 1 +
>>> drivers/staging/media/rkvdec/Makefile | 2 +-
>>> drivers/staging/media/rkvdec/rkvdec-vp9.c | 1078 ++++++++++
>>> drivers/staging/media/rkvdec/rkvdec.c | 52 +-
>>> drivers/staging/media/rkvdec/rkvdec.h | 12 +-
>>> include/media/v4l2-ctrls.h | 4 +
>>> include/media/v4l2-vp9.h | 182 ++
>>> include/uapi/linux/v4l2-controls.h | 284 +++
>>> include/uapi/linux/videodev2.h | 6 +
>>> 37 files changed, 6033 insertions(+), 104 deletions(-)
>>> create mode 100644 drivers/media/v4l2-core/v4l2-vp9.c
>>> create mode 100644 drivers/staging/media/hantro/hantro_g2.c
>>> create mode 100644 drivers/staging/media/hantro/hantro_g2_vp9_dec.c
>>> create mode 100644 drivers/staging/media/hantro/hantro_vp9.c
>>> create mode 100644 drivers/staging/media/hantro/hantro_vp9.h
>>> create mode 100644 drivers/staging/media/rkvdec/rkvdec-vp9.c
>>> create mode 100644 include/media/v4l2-vp9.h
>>>
>>>
>>> base-commit: e4e737bb5c170df6135a127739a9e6148ee3da82
>>>
>>
>
Hi Hans,
Let me clarify:
W dniu 15.11.2021 o 13:56, Andrzej Pietrasiewicz pisze:
> Hi Hans,
>
> W dniu 12.11.2021 o 16:27, Nicolas Dufresne pisze:
>> Hi Hans,
>>
>> Le jeudi 11 novembre 2021 à 15:44 +0100, Hans Verkuil a écrit :
>>> Hi all,
>>>
>>> Andrzej, Jernej, Nicolas, if none of you (or anyone else for that matter)
>>> objects, then I'll make a PR for this early next week.
>>
>> I have no objection. I've myself delayed replying as we have been digging a lot
>> into our compliance failures, but I believe we have explained most of them by
>> now and nothing seems to be related to the API.
>>
>> regards,
>> Nicolas
> I'm fine with making a PR, too.
What I meant was this: "I'm fine with you making a PR."
>
> Andrzej
>
>>
>>>
>>> Regards,
>>>
>>> Hans
>>>
>>> On 29/09/2021 18:04, Andrzej Pietrasiewicz wrote:
>>>> Dear all,
>>>>
>>>> This patch series adds VP9 codec V4L2 control interface and two drivers
>>>> using the new controls. It is a follow-up of previous v6 series [1].
>>>>
>>>> In this iteration, we've implemented VP9 hardware decoding on two devices:
>>>> Rockchip VDEC and Hantro G2, and tested on RK3399, i.MX8MQ and i.MX8MP.
>>>> The i.MX8M driver needs proper power domains support, though, which is a
>>>> subject of a different effort, but in all 3 cases we were able to run the
>>>> drivers.
>>>>
>>>> GStreamer support is also available, the needed changes have been submitted
>>>> by Daniel Almeida [2]. This MR is ready to be merged, and just needs the
>>>> VP9 V4L2 controls to be merged and released.
>>>>
>>>> Both rkvdec and hantro drivers are passing a significant number of VP9 tests
>>>> using Fluster[3]. There are still a few tests that are not passing, due to
>>>> dynamic frame resize (not yet supported by V4L2) and small size videos
>>>> (due to IP block limitations).
>>>>
>>>> The series adds the VP9 codec V4L2 control API as uAPI, so it aims at being
>>>> merged without passing through staging, as agreed[4]. The ABI has been checked
>>>> for padding and verified to contain no holes.
>>>>
>>>> [1] https://patchwork.linuxtv.org/project/linux-media/list/?series=6377
>>>> [2]
>>>> https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2144
>>>> [3] https://github.com/fluendo/fluster
>>>> [4]
>>>> https://lore.kernel.org/linux-media/[email protected]/
>>>>
>>>>
>>>> The series depends on the YUV tiled format support prepared by Ezequiel:
>>>> https://www.spinics.net/lists/linux-media/msg197047.html
>>>>
>>>> Rebased onto latest media_tree.
>>>>
>>>> Changes related to v6:
>>>> - moved setting tile filter and tile bsd auxiliary buffer addresses so
>>>> that they are always set, even if no tiles are used (thanks, Jernej)
>>>> - added a comment near the place where the 32-bit DMA mask is applied
>>>> (thanks, Nicolas)
>>>> - improved consistency in register names (thanks, Nicolas)
>>>>
>>>> Changes related to v5:
>>>> - improved the doc comments as per Ezequiel's review (thanks, Ezequiel)
>>>> - improved pdf output of documentation
>>>> - added Benjamin's Reviewed-by (thanks, Benjamin)
>>>>
>>>> Changes related to v4:
>>>> - removed unused enum v4l2_vp9_intra_prediction_mode
>>>> - converted remaining enums to defines to follow the convention
>>>> - improved the documentation, in particular better documented how to use
>>>> segmentation
>>>> features
>>>>
>>>> Changes related to v3:
>>>>
>>>> Apply suggestions from Jernej's review (thanks, Jernej):
>>>> - renamed a control and two structs:
>>>> V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR_PROBS =>
>>>> V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR
>>>> v4l2_ctrl_vp9_compressed_hdr_probs =>
>>>> v4l2_ctrl_vp9_compressed_hdr
>>>> v4l2_vp9_mv_compressed_hdr_probs => v4l2_vp9_mv_probs
>>>> - moved tx_mode to v4l2_ctrl_vp9_compressed_hdr
>>>> - fixed enum v4l2_vp9_ref_frame_sign_bias values (which are used to test a
>>>> bitfield)
>>>> - explicitly assigned values to all other vp9 enums
>>>>
>>>> Apply suggestion from Nicolas's review (thanks, Nicolas):
>>>> - explicitly stated that the v4l2_ctrl_vp9_compressed_hdr control is optional
>>>> and implemented only by drivers which need it
>>>>
>>>> Changes related to the RFC v2:
>>>>
>>>> - added another driver including a postprocessor to de-tile
>>>> codec-specific tiling
>>>> - reworked uAPI structs layout to follow VP8 style
>>>> - changed validation of loop filter params
>>>> - changed validation of segmentation params
>>>> - changed validation of VP9 frame params
>>>> - removed level lookup array from loop filter struct
>>>> (can be computed by drivers)
>>>> - renamed some enum values to match the spec more closely
>>>> - V4L2 VP9 library changed the 'eob' member of
>>>> 'struct v4l2_vp9_frame_symbol_counts' so that it is an array
>>>> of pointers instead of an array of pointers to arrays
>>>> (IPs such as g2 creatively pass parts of the 'eob' counts in
>>>> the 'coeff' counts)
>>>> - factored out several repeated portions of code
>>>> - minor nitpicks and cleanups
>>>>
>>>> Andrzej Pietrasiewicz (6):
>>>> media: uapi: Add VP9 stateless decoder controls
>>>> media: Add VP9 v4l2 library
>>>> media: hantro: Rename registers
>>>> media: hantro: Prepare for other G2 codecs
>>>> media: hantro: Support VP9 on the G2 core
>>>> media: hantro: Support NV12 on the G2 core
>>>>
>>>> Boris Brezillon (1):
>>>> media: rkvdec: Add the VP9 backend
>>>>
>>>> Ezequiel Garcia (4):
>>>> hantro: postproc: Fix motion vector space size
>>>> hantro: postproc: Introduce struct hantro_postproc_ops
>>>> hantro: Simplify postprocessor
>>>> hantro: Add quirk for NV12/NV12_4L4 capture format
>>>>
>>>> .../userspace-api/media/v4l/biblio.rst | 10 +
>>>> .../media/v4l/ext-ctrls-codec-stateless.rst | 573 +++++
>>>> .../media/v4l/pixfmt-compressed.rst | 15 +
>>>> .../media/v4l/vidioc-g-ext-ctrls.rst | 8 +
>>>> .../media/v4l/vidioc-queryctrl.rst | 12 +
>>>> .../media/videodev2.h.rst.exceptions | 2 +
>>>> drivers/media/v4l2-core/Kconfig | 4 +
>>>> drivers/media/v4l2-core/Makefile | 1 +
>>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 180 ++
>>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 8 +
>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
>>>> drivers/media/v4l2-core/v4l2-vp9.c | 1850 +++++++++++++++++
>>>> drivers/staging/media/hantro/Kconfig | 1 +
>>>> drivers/staging/media/hantro/Makefile | 7 +-
>>>> drivers/staging/media/hantro/hantro.h | 40 +-
>>>> drivers/staging/media/hantro/hantro_drv.c | 23 +-
>>>> drivers/staging/media/hantro/hantro_g2.c | 27 +
>>>> .../staging/media/hantro/hantro_g2_hevc_dec.c | 69 +-
>>>> drivers/staging/media/hantro/hantro_g2_regs.h | 132 +-
>>>> .../staging/media/hantro/hantro_g2_vp9_dec.c | 980 +++++++++
>>>> drivers/staging/media/hantro/hantro_hw.h | 83 +-
>>>> .../staging/media/hantro/hantro_postproc.c | 79 +-
>>>> drivers/staging/media/hantro/hantro_v4l2.c | 20 +
>>>> drivers/staging/media/hantro/hantro_vp9.c | 240 +++
>>>> drivers/staging/media/hantro/hantro_vp9.h | 103 +
>>>> drivers/staging/media/hantro/imx8m_vpu_hw.c | 38 +-
>>>> .../staging/media/hantro/rockchip_vpu_hw.c | 7 +-
>>>> .../staging/media/hantro/sama5d4_vdec_hw.c | 3 +-
>>>> drivers/staging/media/rkvdec/Kconfig | 1 +
>>>> drivers/staging/media/rkvdec/Makefile | 2 +-
>>>> drivers/staging/media/rkvdec/rkvdec-vp9.c | 1078 ++++++++++
>>>> drivers/staging/media/rkvdec/rkvdec.c | 52 +-
>>>> drivers/staging/media/rkvdec/rkvdec.h | 12 +-
>>>> include/media/v4l2-ctrls.h | 4 +
>>>> include/media/v4l2-vp9.h | 182 ++
>>>> include/uapi/linux/v4l2-controls.h | 284 +++
>>>> include/uapi/linux/videodev2.h | 6 +
>>>> 37 files changed, 6033 insertions(+), 104 deletions(-)
>>>> create mode 100644 drivers/media/v4l2-core/v4l2-vp9.c
>>>> create mode 100644 drivers/staging/media/hantro/hantro_g2.c
>>>> create mode 100644 drivers/staging/media/hantro/hantro_g2_vp9_dec.c
>>>> create mode 100644 drivers/staging/media/hantro/hantro_vp9.c
>>>> create mode 100644 drivers/staging/media/hantro/hantro_vp9.h
>>>> create mode 100644 drivers/staging/media/rkvdec/rkvdec-vp9.c
>>>> create mode 100644 include/media/v4l2-vp9.h
>>>>
>>>>
>>>> base-commit: e4e737bb5c170df6135a127739a9e6148ee3da82
>>>>
>>>
>>
>
Andrzej,
Can you rebase this series on top of the master branch of
https://git.linuxtv.org/media_stage.git/ ? Unfortunately this v7 no longer
applies. Specifically "rkvdec: Add the VP9 backend" failed in a non-trivial
manner.
Regards,
Hans
On 29/09/2021 18:04, Andrzej Pietrasiewicz wrote:
> Dear all,
>
> This patch series adds VP9 codec V4L2 control interface and two drivers
> using the new controls. It is a follow-up of previous v6 series [1].
>
> In this iteration, we've implemented VP9 hardware decoding on two devices:
> Rockchip VDEC and Hantro G2, and tested on RK3399, i.MX8MQ and i.MX8MP.
> The i.MX8M driver needs proper power domains support, though, which is a
> subject of a different effort, but in all 3 cases we were able to run the
> drivers.
>
> GStreamer support is also available, the needed changes have been submitted
> by Daniel Almeida [2]. This MR is ready to be merged, and just needs the
> VP9 V4L2 controls to be merged and released.
>
> Both rkvdec and hantro drivers are passing a significant number of VP9 tests
> using Fluster[3]. There are still a few tests that are not passing, due to
> dynamic frame resize (not yet supported by V4L2) and small size videos
> (due to IP block limitations).
>
> The series adds the VP9 codec V4L2 control API as uAPI, so it aims at being
> merged without passing through staging, as agreed[4]. The ABI has been checked
> for padding and verified to contain no holes.
>
> [1] https://patchwork.linuxtv.org/project/linux-media/list/?series=6377
> [2] https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2144
> [3] https://github.com/fluendo/fluster
> [4] https://lore.kernel.org/linux-media/[email protected]/
>
> The series depends on the YUV tiled format support prepared by Ezequiel:
> https://www.spinics.net/lists/linux-media/msg197047.html
>
> Rebased onto latest media_tree.
>
> Changes related to v6:
> - moved setting tile filter and tile bsd auxiliary buffer addresses so
> that they are always set, even if no tiles are used (thanks, Jernej)
> - added a comment near the place where the 32-bit DMA mask is applied
> (thanks, Nicolas)
> - improved consistency in register names (thanks, Nicolas)
>
> Changes related to v5:
> - improved the doc comments as per Ezequiel's review (thanks, Ezequiel)
> - improved pdf output of documentation
> - added Benjamin's Reviewed-by (thanks, Benjamin)
>
> Changes related to v4:
> - removed unused enum v4l2_vp9_intra_prediction_mode
> - converted remaining enums to defines to follow the convention
> - improved the documentation, in particular better documented how to use segmentation
> features
>
> Changes related to v3:
>
> Apply suggestions from Jernej's review (thanks, Jernej):
> - renamed a control and two structs:
> V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR_PROBS =>
> V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR
> v4l2_ctrl_vp9_compressed_hdr_probs =>
> v4l2_ctrl_vp9_compressed_hdr
> v4l2_vp9_mv_compressed_hdr_probs => v4l2_vp9_mv_probs
> - moved tx_mode to v4l2_ctrl_vp9_compressed_hdr
> - fixed enum v4l2_vp9_ref_frame_sign_bias values (which are used to test a bitfield)
> - explicitly assigned values to all other vp9 enums
>
> Apply suggestion from Nicolas's review (thanks, Nicolas):
> - explicitly stated that the v4l2_ctrl_vp9_compressed_hdr control is optional
> and implemented only by drivers which need it
>
> Changes related to the RFC v2:
>
> - added another driver including a postprocessor to de-tile
> codec-specific tiling
> - reworked uAPI structs layout to follow VP8 style
> - changed validation of loop filter params
> - changed validation of segmentation params
> - changed validation of VP9 frame params
> - removed level lookup array from loop filter struct
> (can be computed by drivers)
> - renamed some enum values to match the spec more closely
> - V4L2 VP9 library changed the 'eob' member of
> 'struct v4l2_vp9_frame_symbol_counts' so that it is an array
> of pointers instead of an array of pointers to arrays
> (IPs such as g2 creatively pass parts of the 'eob' counts in
> the 'coeff' counts)
> - factored out several repeated portions of code
> - minor nitpicks and cleanups
>
> Andrzej Pietrasiewicz (6):
> media: uapi: Add VP9 stateless decoder controls
> media: Add VP9 v4l2 library
> media: hantro: Rename registers
> media: hantro: Prepare for other G2 codecs
> media: hantro: Support VP9 on the G2 core
> media: hantro: Support NV12 on the G2 core
>
> Boris Brezillon (1):
> media: rkvdec: Add the VP9 backend
>
> Ezequiel Garcia (4):
> hantro: postproc: Fix motion vector space size
> hantro: postproc: Introduce struct hantro_postproc_ops
> hantro: Simplify postprocessor
> hantro: Add quirk for NV12/NV12_4L4 capture format
>
> .../userspace-api/media/v4l/biblio.rst | 10 +
> .../media/v4l/ext-ctrls-codec-stateless.rst | 573 +++++
> .../media/v4l/pixfmt-compressed.rst | 15 +
> .../media/v4l/vidioc-g-ext-ctrls.rst | 8 +
> .../media/v4l/vidioc-queryctrl.rst | 12 +
> .../media/videodev2.h.rst.exceptions | 2 +
> drivers/media/v4l2-core/Kconfig | 4 +
> drivers/media/v4l2-core/Makefile | 1 +
> drivers/media/v4l2-core/v4l2-ctrls-core.c | 180 ++
> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 8 +
> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
> drivers/media/v4l2-core/v4l2-vp9.c | 1850 +++++++++++++++++
> drivers/staging/media/hantro/Kconfig | 1 +
> drivers/staging/media/hantro/Makefile | 7 +-
> drivers/staging/media/hantro/hantro.h | 40 +-
> drivers/staging/media/hantro/hantro_drv.c | 23 +-
> drivers/staging/media/hantro/hantro_g2.c | 27 +
> .../staging/media/hantro/hantro_g2_hevc_dec.c | 69 +-
> drivers/staging/media/hantro/hantro_g2_regs.h | 132 +-
> .../staging/media/hantro/hantro_g2_vp9_dec.c | 980 +++++++++
> drivers/staging/media/hantro/hantro_hw.h | 83 +-
> .../staging/media/hantro/hantro_postproc.c | 79 +-
> drivers/staging/media/hantro/hantro_v4l2.c | 20 +
> drivers/staging/media/hantro/hantro_vp9.c | 240 +++
> drivers/staging/media/hantro/hantro_vp9.h | 103 +
> drivers/staging/media/hantro/imx8m_vpu_hw.c | 38 +-
> .../staging/media/hantro/rockchip_vpu_hw.c | 7 +-
> .../staging/media/hantro/sama5d4_vdec_hw.c | 3 +-
> drivers/staging/media/rkvdec/Kconfig | 1 +
> drivers/staging/media/rkvdec/Makefile | 2 +-
> drivers/staging/media/rkvdec/rkvdec-vp9.c | 1078 ++++++++++
> drivers/staging/media/rkvdec/rkvdec.c | 52 +-
> drivers/staging/media/rkvdec/rkvdec.h | 12 +-
> include/media/v4l2-ctrls.h | 4 +
> include/media/v4l2-vp9.h | 182 ++
> include/uapi/linux/v4l2-controls.h | 284 +++
> include/uapi/linux/videodev2.h | 6 +
> 37 files changed, 6033 insertions(+), 104 deletions(-)
> create mode 100644 drivers/media/v4l2-core/v4l2-vp9.c
> create mode 100644 drivers/staging/media/hantro/hantro_g2.c
> create mode 100644 drivers/staging/media/hantro/hantro_g2_vp9_dec.c
> create mode 100644 drivers/staging/media/hantro/hantro_vp9.c
> create mode 100644 drivers/staging/media/hantro/hantro_vp9.h
> create mode 100644 drivers/staging/media/rkvdec/rkvdec-vp9.c
> create mode 100644 include/media/v4l2-vp9.h
>
>
> base-commit: e4e737bb5c170df6135a127739a9e6148ee3da82
>
Hi Hans,
W dniu 15.11.2021 o 16:07, Hans Verkuil pisze:
> Andrzej,
>
> Can you rebase this series on top of the master branch of
> https://git.linuxtv.org/media_stage.git/ ? Unfortunately this v7 no longer
> applies. Specifically "rkvdec: Add the VP9 backend" failed in a non-trivial
> manner.
This is a branch for you:
https://gitlab.collabora.com/linux/for-upstream/-/tree/vp9-uapi
Regards,
Andrzej
>
> Regards,
>
> Hans
>
> On 29/09/2021 18:04, Andrzej Pietrasiewicz wrote:
>> Dear all,
>>
>> This patch series adds VP9 codec V4L2 control interface and two drivers
>> using the new controls. It is a follow-up of previous v6 series [1].
>>
>> In this iteration, we've implemented VP9 hardware decoding on two devices:
>> Rockchip VDEC and Hantro G2, and tested on RK3399, i.MX8MQ and i.MX8MP.
>> The i.MX8M driver needs proper power domains support, though, which is a
>> subject of a different effort, but in all 3 cases we were able to run the
>> drivers.
>>
>> GStreamer support is also available, the needed changes have been submitted
>> by Daniel Almeida [2]. This MR is ready to be merged, and just needs the
>> VP9 V4L2 controls to be merged and released.
>>
>> Both rkvdec and hantro drivers are passing a significant number of VP9 tests
>> using Fluster[3]. There are still a few tests that are not passing, due to
>> dynamic frame resize (not yet supported by V4L2) and small size videos
>> (due to IP block limitations).
>>
>> The series adds the VP9 codec V4L2 control API as uAPI, so it aims at being
>> merged without passing through staging, as agreed[4]. The ABI has been checked
>> for padding and verified to contain no holes.
>>
>> [1] https://patchwork.linuxtv.org/project/linux-media/list/?series=6377
>> [2] https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2144
>> [3] https://github.com/fluendo/fluster
>> [4] https://lore.kernel.org/linux-media/[email protected]/
>>
>> The series depends on the YUV tiled format support prepared by Ezequiel:
>> https://www.spinics.net/lists/linux-media/msg197047.html
>>
>> Rebased onto latest media_tree.
>>
>> Changes related to v6:
>> - moved setting tile filter and tile bsd auxiliary buffer addresses so
>> that they are always set, even if no tiles are used (thanks, Jernej)
>> - added a comment near the place where the 32-bit DMA mask is applied
>> (thanks, Nicolas)
>> - improved consistency in register names (thanks, Nicolas)
>>
>> Changes related to v5:
>> - improved the doc comments as per Ezequiel's review (thanks, Ezequiel)
>> - improved pdf output of documentation
>> - added Benjamin's Reviewed-by (thanks, Benjamin)
>>
>> Changes related to v4:
>> - removed unused enum v4l2_vp9_intra_prediction_mode
>> - converted remaining enums to defines to follow the convention
>> - improved the documentation, in particular better documented how to use segmentation
>> features
>>
>> Changes related to v3:
>>
>> Apply suggestions from Jernej's review (thanks, Jernej):
>> - renamed a control and two structs:
>> V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR_PROBS =>
>> V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR
>> v4l2_ctrl_vp9_compressed_hdr_probs =>
>> v4l2_ctrl_vp9_compressed_hdr
>> v4l2_vp9_mv_compressed_hdr_probs => v4l2_vp9_mv_probs
>> - moved tx_mode to v4l2_ctrl_vp9_compressed_hdr
>> - fixed enum v4l2_vp9_ref_frame_sign_bias values (which are used to test a bitfield)
>> - explicitly assigned values to all other vp9 enums
>>
>> Apply suggestion from Nicolas's review (thanks, Nicolas):
>> - explicitly stated that the v4l2_ctrl_vp9_compressed_hdr control is optional
>> and implemented only by drivers which need it
>>
>> Changes related to the RFC v2:
>>
>> - added another driver including a postprocessor to de-tile
>> codec-specific tiling
>> - reworked uAPI structs layout to follow VP8 style
>> - changed validation of loop filter params
>> - changed validation of segmentation params
>> - changed validation of VP9 frame params
>> - removed level lookup array from loop filter struct
>> (can be computed by drivers)
>> - renamed some enum values to match the spec more closely
>> - V4L2 VP9 library changed the 'eob' member of
>> 'struct v4l2_vp9_frame_symbol_counts' so that it is an array
>> of pointers instead of an array of pointers to arrays
>> (IPs such as g2 creatively pass parts of the 'eob' counts in
>> the 'coeff' counts)
>> - factored out several repeated portions of code
>> - minor nitpicks and cleanups
>>
>> Andrzej Pietrasiewicz (6):
>> media: uapi: Add VP9 stateless decoder controls
>> media: Add VP9 v4l2 library
>> media: hantro: Rename registers
>> media: hantro: Prepare for other G2 codecs
>> media: hantro: Support VP9 on the G2 core
>> media: hantro: Support NV12 on the G2 core
>>
>> Boris Brezillon (1):
>> media: rkvdec: Add the VP9 backend
>>
>> Ezequiel Garcia (4):
>> hantro: postproc: Fix motion vector space size
>> hantro: postproc: Introduce struct hantro_postproc_ops
>> hantro: Simplify postprocessor
>> hantro: Add quirk for NV12/NV12_4L4 capture format
>>
>> .../userspace-api/media/v4l/biblio.rst | 10 +
>> .../media/v4l/ext-ctrls-codec-stateless.rst | 573 +++++
>> .../media/v4l/pixfmt-compressed.rst | 15 +
>> .../media/v4l/vidioc-g-ext-ctrls.rst | 8 +
>> .../media/v4l/vidioc-queryctrl.rst | 12 +
>> .../media/videodev2.h.rst.exceptions | 2 +
>> drivers/media/v4l2-core/Kconfig | 4 +
>> drivers/media/v4l2-core/Makefile | 1 +
>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 180 ++
>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 8 +
>> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
>> drivers/media/v4l2-core/v4l2-vp9.c | 1850 +++++++++++++++++
>> drivers/staging/media/hantro/Kconfig | 1 +
>> drivers/staging/media/hantro/Makefile | 7 +-
>> drivers/staging/media/hantro/hantro.h | 40 +-
>> drivers/staging/media/hantro/hantro_drv.c | 23 +-
>> drivers/staging/media/hantro/hantro_g2.c | 27 +
>> .../staging/media/hantro/hantro_g2_hevc_dec.c | 69 +-
>> drivers/staging/media/hantro/hantro_g2_regs.h | 132 +-
>> .../staging/media/hantro/hantro_g2_vp9_dec.c | 980 +++++++++
>> drivers/staging/media/hantro/hantro_hw.h | 83 +-
>> .../staging/media/hantro/hantro_postproc.c | 79 +-
>> drivers/staging/media/hantro/hantro_v4l2.c | 20 +
>> drivers/staging/media/hantro/hantro_vp9.c | 240 +++
>> drivers/staging/media/hantro/hantro_vp9.h | 103 +
>> drivers/staging/media/hantro/imx8m_vpu_hw.c | 38 +-
>> .../staging/media/hantro/rockchip_vpu_hw.c | 7 +-
>> .../staging/media/hantro/sama5d4_vdec_hw.c | 3 +-
>> drivers/staging/media/rkvdec/Kconfig | 1 +
>> drivers/staging/media/rkvdec/Makefile | 2 +-
>> drivers/staging/media/rkvdec/rkvdec-vp9.c | 1078 ++++++++++
>> drivers/staging/media/rkvdec/rkvdec.c | 52 +-
>> drivers/staging/media/rkvdec/rkvdec.h | 12 +-
>> include/media/v4l2-ctrls.h | 4 +
>> include/media/v4l2-vp9.h | 182 ++
>> include/uapi/linux/v4l2-controls.h | 284 +++
>> include/uapi/linux/videodev2.h | 6 +
>> 37 files changed, 6033 insertions(+), 104 deletions(-)
>> create mode 100644 drivers/media/v4l2-core/v4l2-vp9.c
>> create mode 100644 drivers/staging/media/hantro/hantro_g2.c
>> create mode 100644 drivers/staging/media/hantro/hantro_g2_vp9_dec.c
>> create mode 100644 drivers/staging/media/hantro/hantro_vp9.c
>> create mode 100644 drivers/staging/media/hantro/hantro_vp9.h
>> create mode 100644 drivers/staging/media/rkvdec/rkvdec-vp9.c
>> create mode 100644 include/media/v4l2-vp9.h
>>
>>
>> base-commit: e4e737bb5c170df6135a127739a9e6148ee3da82
>>
>
On 15/11/2021 18:14, Andrzej Pietrasiewicz wrote:
> Hi Hans,
>
> W dniu 15.11.2021 o 16:07, Hans Verkuil pisze:
>> Andrzej,
>>
>> Can you rebase this series on top of the master branch of
>> https://git.linuxtv.org/media_stage.git/ ? Unfortunately this v7 no longer
>> applies. Specifically "rkvdec: Add the VP9 backend" failed in a non-trivial
>> manner.
>
> This is a branch for you:
>
> https://gitlab.collabora.com/linux/for-upstream/-/tree/vp9-uapi
I'm getting a bunch of sparse/smatch warnings:
sparse:
rkvdec/rkvdec-vp9.c:190:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
rkvdec/rkvdec-vp9.c:245:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
SPARSE:hantro/hantro_postproc.c hantro/hantro_postproc.c:37:35: warning: symbol 'hantro_g1_postproc_regs' was not declared. Should it be static?
smatch:
rkvdec/rkvdec-vp9.c:190:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
rkvdec/rkvdec-vp9.c:245:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
rkvdec/rkvdec-vp9.c: rkvdec/rkvdec-vp9.c:236 init_intra_only_probs() error: buffer overflow 'ptr' 90 <= 91
hantro/hantro_g2_vp9_dec.c: hantro/hantro_g2_vp9_dec.c:670 config_probs() error: memcpy() 'adaptive->inter_mode[i]' too small (4 vs 21)
hantro/hantro_g2_vp9_dec.c: hantro/hantro_g2_vp9_dec.c:670 config_probs() error: memcpy() 'probs->inter_mode[i]' too small (3 vs 21
Also a bunch of kerneldoc warnings:
include/media/v4l2-vp9.h:30: warning: Function parameter or member 'joint' not described in 'v4l2_vp9_frame_mv_context'
include/media/v4l2-vp9.h:30: warning: Function parameter or member 'sign' not described in 'v4l2_vp9_frame_mv_context'
include/media/v4l2-vp9.h:30: warning: Function parameter or member 'classes' not described in 'v4l2_vp9_frame_mv_context'
include/media/v4l2-vp9.h:30: warning: Function parameter or member 'class0_bit' not described in 'v4l2_vp9_frame_mv_context'
include/media/v4l2-vp9.h:30: warning: Function parameter or member 'bits' not described in 'v4l2_vp9_frame_mv_context'
include/media/v4l2-vp9.h:30: warning: Function parameter or member 'class0_fr' not described in 'v4l2_vp9_frame_mv_context'
include/media/v4l2-vp9.h:30: warning: Function parameter or member 'fr' not described in 'v4l2_vp9_frame_mv_context'
include/media/v4l2-vp9.h:30: warning: Function parameter or member 'class0_hp' not described in 'v4l2_vp9_frame_mv_context'
include/media/v4l2-vp9.h:30: warning: Function parameter or member 'hp' not described in 'v4l2_vp9_frame_mv_context'
include/media/v4l2-vp9.h:58: warning: Function parameter or member 'tx8' not described in 'v4l2_vp9_frame_context'
include/media/v4l2-vp9.h:58: warning: Function parameter or member 'tx16' not described in 'v4l2_vp9_frame_context'
include/media/v4l2-vp9.h:58: warning: Function parameter or member 'tx32' not described in 'v4l2_vp9_frame_context'
include/media/v4l2-vp9.h:58: warning: Function parameter or member 'coef' not described in 'v4l2_vp9_frame_context'
include/media/v4l2-vp9.h:58: warning: Function parameter or member 'skip' not described in 'v4l2_vp9_frame_context'
include/media/v4l2-vp9.h:58: warning: Function parameter or member 'inter_mode' not described in 'v4l2_vp9_frame_context'
include/media/v4l2-vp9.h:58: warning: Function parameter or member 'interp_filter' not described in 'v4l2_vp9_frame_context'
include/media/v4l2-vp9.h:58: warning: Function parameter or member 'is_inter' not described in 'v4l2_vp9_frame_context'
include/media/v4l2-vp9.h:58: warning: Function parameter or member 'comp_mode' not described in 'v4l2_vp9_frame_context'
include/media/v4l2-vp9.h:58: warning: Function parameter or member 'single_ref' not described in 'v4l2_vp9_frame_context'
include/media/v4l2-vp9.h:58: warning: Function parameter or member 'comp_ref' not described in 'v4l2_vp9_frame_context'
include/media/v4l2-vp9.h:58: warning: Function parameter or member 'y_mode' not described in 'v4l2_vp9_frame_context'
include/media/v4l2-vp9.h:58: warning: Function parameter or member 'uv_mode' not described in 'v4l2_vp9_frame_context'
include/media/v4l2-vp9.h:58: warning: Function parameter or member 'partition' not described in 'v4l2_vp9_frame_context'
include/media/v4l2-vp9.h:58: warning: Function parameter or member 'mv' not described in 'v4l2_vp9_frame_context'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'partition' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'skip' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'intra_inter' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'tx32p' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'tx16p' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'tx8p' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'y_mode' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'uv_mode' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'comp' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'comp_ref' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'single_ref' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'mv_mode' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'filter' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'mv_joint' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'sign' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'classes' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'class0' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'bits' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'class0_fp' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'fp' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'class0_hp' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'hp' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'coeff' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:93: warning: Function parameter or member 'eob' not described in 'v4l2_vp9_frame_symbol_counts'
include/media/v4l2-vp9.h:166: warning: expecting prototype for v4l2_vp9_adapt_coef_probs(). Prototype was for v4l2_vp9_adapt_noncoef_probs()
instead
drivers/media/platform/omap3isp/omap3isp.h:107: warning: Function parameter or member 'vp_clk_pol' not described in 'isp_ccp2_cfg'
drivers/media/platform/omap3isp/omap3isp.h:107: warning: Function parameter or member 'lanecfg' not described in 'isp_ccp2_cfg'
drivers/media/platform/qcom/venus/core.h:202: warning: Function parameter or member 'sys_err_done' not described in 'venus_core'
drivers/media/platform/qcom/venus/core.h:462: warning: Function parameter or member 'fw_min_cnt' not described in 'venus_inst'
drivers/media/platform/qcom/venus/core.h:462: warning: Function parameter or member 'flags' not described in 'venus_inst'
drivers/media/platform/qcom/venus/core.h:462: warning: Function parameter or member 'dpb_ids' not described in 'venus_inst'
drivers/staging/media/hantro/hantro.h:115: warning: Enum value 'HANTRO_MODE_VP9_DEC' not described in enum 'hantro_codec_mode'
drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'tile_edge' not described in 'hantro_vp9_dec_hw_ctx'
drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'segment_map' not described in 'hantro_vp9_dec_hw_ctx'
drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'misc' not described in 'hantro_vp9_dec_hw_ctx'
drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'cnts' not described in 'hantro_vp9_dec_hw_ctx'
drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'probability_tables' not described in
'hantro_vp9_dec_hw_ctx'
drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'frame_context' not described in 'hantro_vp9_dec_hw_ctx'
drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'cur' not described in 'hantro_vp9_dec_hw_ctx'
drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'last' not described in 'hantro_vp9_dec_hw_ctx'
drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'bsd_ctrl_offset' not described in 'hantro_vp9_dec_hw_ctx'
drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'segment_map_size' not described in 'hantro_vp9_dec_hw_ctx'
drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'ctx_counters_offset' not described in
'hantro_vp9_dec_hw_ctx'
drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'tile_info_offset' not described in 'hantro_vp9_dec_hw_ctx'
drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'tile_r_info' not described in 'hantro_vp9_dec_hw_ctx'
drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'tile_c_info' not described in 'hantro_vp9_dec_hw_ctx'
drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'last_tile_r' not described in 'hantro_vp9_dec_hw_ctx'
drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'last_tile_c' not described in 'hantro_vp9_dec_hw_ctx'
drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'last_sbs_r' not described in 'hantro_vp9_dec_hw_ctx'
drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'last_sbs_c' not described in 'hantro_vp9_dec_hw_ctx'
drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'active_segment' not described in 'hantro_vp9_dec_hw_ctx'
drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'feature_enabled' not described in 'hantro_vp9_dec_hw_ctx'
drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'feature_data' not described in 'hantro_vp9_dec_hw_ctx'
You can test kerneldoc yourself with: scripts/kernel-doc -none include/media/v4l2-vp9.h
Regards,
Hans
>
> Regards,
>
> Andrzej
>
>
>>
>> Regards,
>>
>> Hans
>>
>> On 29/09/2021 18:04, Andrzej Pietrasiewicz wrote:
>>> Dear all,
>>>
>>> This patch series adds VP9 codec V4L2 control interface and two drivers
>>> using the new controls. It is a follow-up of previous v6 series [1].
>>>
>>> In this iteration, we've implemented VP9 hardware decoding on two devices:
>>> Rockchip VDEC and Hantro G2, and tested on RK3399, i.MX8MQ and i.MX8MP.
>>> The i.MX8M driver needs proper power domains support, though, which is a
>>> subject of a different effort, but in all 3 cases we were able to run the
>>> drivers.
>>>
>>> GStreamer support is also available, the needed changes have been submitted
>>> by Daniel Almeida [2]. This MR is ready to be merged, and just needs the
>>> VP9 V4L2 controls to be merged and released.
>>>
>>> Both rkvdec and hantro drivers are passing a significant number of VP9 tests
>>> using Fluster[3]. There are still a few tests that are not passing, due to
>>> dynamic frame resize (not yet supported by V4L2) and small size videos
>>> (due to IP block limitations).
>>>
>>> The series adds the VP9 codec V4L2 control API as uAPI, so it aims at being
>>> merged without passing through staging, as agreed[4]. The ABI has been checked
>>> for padding and verified to contain no holes.
>>>
>>> [1] https://patchwork.linuxtv.org/project/linux-media/list/?series=6377
>>> [2] https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2144
>>> [3] https://github.com/fluendo/fluster
>>> [4] https://lore.kernel.org/linux-media/[email protected]/
>>>
>>> The series depends on the YUV tiled format support prepared by Ezequiel:
>>> https://www.spinics.net/lists/linux-media/msg197047.html
>>>
>>> Rebased onto latest media_tree.
>>>
>>> Changes related to v6:
>>> - moved setting tile filter and tile bsd auxiliary buffer addresses so
>>> that they are always set, even if no tiles are used (thanks, Jernej)
>>> - added a comment near the place where the 32-bit DMA mask is applied
>>> (thanks, Nicolas)
>>> - improved consistency in register names (thanks, Nicolas)
>>>
>>> Changes related to v5:
>>> - improved the doc comments as per Ezequiel's review (thanks, Ezequiel)
>>> - improved pdf output of documentation
>>> - added Benjamin's Reviewed-by (thanks, Benjamin)
>>>
>>> Changes related to v4:
>>> - removed unused enum v4l2_vp9_intra_prediction_mode
>>> - converted remaining enums to defines to follow the convention
>>> - improved the documentation, in particular better documented how to use segmentation
>>> features
>>>
>>> Changes related to v3:
>>>
>>> Apply suggestions from Jernej's review (thanks, Jernej):
>>> - renamed a control and two structs:
>>> V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR_PROBS =>
>>> V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR
>>> v4l2_ctrl_vp9_compressed_hdr_probs =>
>>> v4l2_ctrl_vp9_compressed_hdr
>>> v4l2_vp9_mv_compressed_hdr_probs => v4l2_vp9_mv_probs
>>> - moved tx_mode to v4l2_ctrl_vp9_compressed_hdr
>>> - fixed enum v4l2_vp9_ref_frame_sign_bias values (which are used to test a bitfield)
>>> - explicitly assigned values to all other vp9 enums
>>>
>>> Apply suggestion from Nicolas's review (thanks, Nicolas):
>>> - explicitly stated that the v4l2_ctrl_vp9_compressed_hdr control is optional
>>> and implemented only by drivers which need it
>>>
>>> Changes related to the RFC v2:
>>>
>>> - added another driver including a postprocessor to de-tile
>>> codec-specific tiling
>>> - reworked uAPI structs layout to follow VP8 style
>>> - changed validation of loop filter params
>>> - changed validation of segmentation params
>>> - changed validation of VP9 frame params
>>> - removed level lookup array from loop filter struct
>>> (can be computed by drivers)
>>> - renamed some enum values to match the spec more closely
>>> - V4L2 VP9 library changed the 'eob' member of
>>> 'struct v4l2_vp9_frame_symbol_counts' so that it is an array
>>> of pointers instead of an array of pointers to arrays
>>> (IPs such as g2 creatively pass parts of the 'eob' counts in
>>> the 'coeff' counts)
>>> - factored out several repeated portions of code
>>> - minor nitpicks and cleanups
>>>
>>> Andrzej Pietrasiewicz (6):
>>> media: uapi: Add VP9 stateless decoder controls
>>> media: Add VP9 v4l2 library
>>> media: hantro: Rename registers
>>> media: hantro: Prepare for other G2 codecs
>>> media: hantro: Support VP9 on the G2 core
>>> media: hantro: Support NV12 on the G2 core
>>>
>>> Boris Brezillon (1):
>>> media: rkvdec: Add the VP9 backend
>>>
>>> Ezequiel Garcia (4):
>>> hantro: postproc: Fix motion vector space size
>>> hantro: postproc: Introduce struct hantro_postproc_ops
>>> hantro: Simplify postprocessor
>>> hantro: Add quirk for NV12/NV12_4L4 capture format
>>>
>>> .../userspace-api/media/v4l/biblio.rst | 10 +
>>> .../media/v4l/ext-ctrls-codec-stateless.rst | 573 +++++
>>> .../media/v4l/pixfmt-compressed.rst | 15 +
>>> .../media/v4l/vidioc-g-ext-ctrls.rst | 8 +
>>> .../media/v4l/vidioc-queryctrl.rst | 12 +
>>> .../media/videodev2.h.rst.exceptions | 2 +
>>> drivers/media/v4l2-core/Kconfig | 4 +
>>> drivers/media/v4l2-core/Makefile | 1 +
>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 180 ++
>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 8 +
>>> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
>>> drivers/media/v4l2-core/v4l2-vp9.c | 1850 +++++++++++++++++
>>> drivers/staging/media/hantro/Kconfig | 1 +
>>> drivers/staging/media/hantro/Makefile | 7 +-
>>> drivers/staging/media/hantro/hantro.h | 40 +-
>>> drivers/staging/media/hantro/hantro_drv.c | 23 +-
>>> drivers/staging/media/hantro/hantro_g2.c | 27 +
>>> .../staging/media/hantro/hantro_g2_hevc_dec.c | 69 +-
>>> drivers/staging/media/hantro/hantro_g2_regs.h | 132 +-
>>> .../staging/media/hantro/hantro_g2_vp9_dec.c | 980 +++++++++
>>> drivers/staging/media/hantro/hantro_hw.h | 83 +-
>>> .../staging/media/hantro/hantro_postproc.c | 79 +-
>>> drivers/staging/media/hantro/hantro_v4l2.c | 20 +
>>> drivers/staging/media/hantro/hantro_vp9.c | 240 +++
>>> drivers/staging/media/hantro/hantro_vp9.h | 103 +
>>> drivers/staging/media/hantro/imx8m_vpu_hw.c | 38 +-
>>> .../staging/media/hantro/rockchip_vpu_hw.c | 7 +-
>>> .../staging/media/hantro/sama5d4_vdec_hw.c | 3 +-
>>> drivers/staging/media/rkvdec/Kconfig | 1 +
>>> drivers/staging/media/rkvdec/Makefile | 2 +-
>>> drivers/staging/media/rkvdec/rkvdec-vp9.c | 1078 ++++++++++
>>> drivers/staging/media/rkvdec/rkvdec.c | 52 +-
>>> drivers/staging/media/rkvdec/rkvdec.h | 12 +-
>>> include/media/v4l2-ctrls.h | 4 +
>>> include/media/v4l2-vp9.h | 182 ++
>>> include/uapi/linux/v4l2-controls.h | 284 +++
>>> include/uapi/linux/videodev2.h | 6 +
>>> 37 files changed, 6033 insertions(+), 104 deletions(-)
>>> create mode 100644 drivers/media/v4l2-core/v4l2-vp9.c
>>> create mode 100644 drivers/staging/media/hantro/hantro_g2.c
>>> create mode 100644 drivers/staging/media/hantro/hantro_g2_vp9_dec.c
>>> create mode 100644 drivers/staging/media/hantro/hantro_vp9.c
>>> create mode 100644 drivers/staging/media/hantro/hantro_vp9.h
>>> create mode 100644 drivers/staging/media/rkvdec/rkvdec-vp9.c
>>> create mode 100644 include/media/v4l2-vp9.h
>>>
>>>
>>> base-commit: e4e737bb5c170df6135a127739a9e6148ee3da82
>>>
>>
>
Hi Hans,
W dniu 15.11.2021 o 22:16, Hans Verkuil pisze:
> On 15/11/2021 18:14, Andrzej Pietrasiewicz wrote:
>> Hi Hans,
>>
>> W dniu 15.11.2021 o 16:07, Hans Verkuil pisze:
>>> Andrzej,
>>>
>>> Can you rebase this series on top of the master branch of
>>> https://git.linuxtv.org/media_stage.git/ ? Unfortunately this v7 no longer
>>> applies. Specifically "rkvdec: Add the VP9 backend" failed in a non-trivial
>>> manner.
>>
>> This is a branch for you:
>>
>> https://gitlab.collabora.com/linux/for-upstream/-/tree/vp9-uapi
>
> I'm getting a bunch of sparse/smatch warnings:
>
Thanks for finding this, I will re-create the branch and let you know on irc.
Some of the below are "false positives, namely:
drivers/media/platform/omap3isp/omap3isp.h
drivers/media/platform/qcom/venus/core.h
which are not touched by the series.
Regards,
Andrzej
> sparse:
> rkvdec/rkvdec-vp9.c:190:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
> rkvdec/rkvdec-vp9.c:245:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
> SPARSE:hantro/hantro_postproc.c hantro/hantro_postproc.c:37:35: warning: symbol 'hantro_g1_postproc_regs' was not declared. Should it be static?
>
> smatch:
> rkvdec/rkvdec-vp9.c:190:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
> rkvdec/rkvdec-vp9.c:245:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
> rkvdec/rkvdec-vp9.c: rkvdec/rkvdec-vp9.c:236 init_intra_only_probs() error: buffer overflow 'ptr' 90 <= 91
> hantro/hantro_g2_vp9_dec.c: hantro/hantro_g2_vp9_dec.c:670 config_probs() error: memcpy() 'adaptive->inter_mode[i]' too small (4 vs 21)
> hantro/hantro_g2_vp9_dec.c: hantro/hantro_g2_vp9_dec.c:670 config_probs() error: memcpy() 'probs->inter_mode[i]' too small (3 vs 21
>
> Also a bunch of kerneldoc warnings:
>
> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'joint' not described in 'v4l2_vp9_frame_mv_context'
> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'sign' not described in 'v4l2_vp9_frame_mv_context'
> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'classes' not described in 'v4l2_vp9_frame_mv_context'
> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'class0_bit' not described in 'v4l2_vp9_frame_mv_context'
> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'bits' not described in 'v4l2_vp9_frame_mv_context'
> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'class0_fr' not described in 'v4l2_vp9_frame_mv_context'
> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'fr' not described in 'v4l2_vp9_frame_mv_context'
> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'class0_hp' not described in 'v4l2_vp9_frame_mv_context'
> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'hp' not described in 'v4l2_vp9_frame_mv_context'
> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'tx8' not described in 'v4l2_vp9_frame_context'
> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'tx16' not described in 'v4l2_vp9_frame_context'
> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'tx32' not described in 'v4l2_vp9_frame_context'
> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'coef' not described in 'v4l2_vp9_frame_context'
> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'skip' not described in 'v4l2_vp9_frame_context'
> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'inter_mode' not described in 'v4l2_vp9_frame_context'
> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'interp_filter' not described in 'v4l2_vp9_frame_context'
> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'is_inter' not described in 'v4l2_vp9_frame_context'
> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'comp_mode' not described in 'v4l2_vp9_frame_context'
> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'single_ref' not described in 'v4l2_vp9_frame_context'
> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'comp_ref' not described in 'v4l2_vp9_frame_context'
> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'y_mode' not described in 'v4l2_vp9_frame_context'
> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'uv_mode' not described in 'v4l2_vp9_frame_context'
> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'partition' not described in 'v4l2_vp9_frame_context'
> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'mv' not described in 'v4l2_vp9_frame_context'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'partition' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'skip' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'intra_inter' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'tx32p' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'tx16p' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'tx8p' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'y_mode' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'uv_mode' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'comp' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'comp_ref' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'single_ref' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'mv_mode' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'filter' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'mv_joint' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'sign' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'classes' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'class0' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'bits' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'class0_fp' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'fp' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'class0_hp' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'hp' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'coeff' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'eob' not described in 'v4l2_vp9_frame_symbol_counts'
> include/media/v4l2-vp9.h:166: warning: expecting prototype for v4l2_vp9_adapt_coef_probs(). Prototype was for v4l2_vp9_adapt_noncoef_probs()
> instead
> drivers/media/platform/omap3isp/omap3isp.h:107: warning: Function parameter or member 'vp_clk_pol' not described in 'isp_ccp2_cfg'
> drivers/media/platform/omap3isp/omap3isp.h:107: warning: Function parameter or member 'lanecfg' not described in 'isp_ccp2_cfg'
> drivers/media/platform/qcom/venus/core.h:202: warning: Function parameter or member 'sys_err_done' not described in 'venus_core'
> drivers/media/platform/qcom/venus/core.h:462: warning: Function parameter or member 'fw_min_cnt' not described in 'venus_inst'
> drivers/media/platform/qcom/venus/core.h:462: warning: Function parameter or member 'flags' not described in 'venus_inst'
> drivers/media/platform/qcom/venus/core.h:462: warning: Function parameter or member 'dpb_ids' not described in 'venus_inst'
> drivers/staging/media/hantro/hantro.h:115: warning: Enum value 'HANTRO_MODE_VP9_DEC' not described in enum 'hantro_codec_mode'
> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'tile_edge' not described in 'hantro_vp9_dec_hw_ctx'
> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'segment_map' not described in 'hantro_vp9_dec_hw_ctx'
> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'misc' not described in 'hantro_vp9_dec_hw_ctx'
> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'cnts' not described in 'hantro_vp9_dec_hw_ctx'
> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'probability_tables' not described in
> 'hantro_vp9_dec_hw_ctx'
> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'frame_context' not described in 'hantro_vp9_dec_hw_ctx'
> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'cur' not described in 'hantro_vp9_dec_hw_ctx'
> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'last' not described in 'hantro_vp9_dec_hw_ctx'
> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'bsd_ctrl_offset' not described in 'hantro_vp9_dec_hw_ctx'
> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'segment_map_size' not described in 'hantro_vp9_dec_hw_ctx'
> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'ctx_counters_offset' not described in
> 'hantro_vp9_dec_hw_ctx'
> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'tile_info_offset' not described in 'hantro_vp9_dec_hw_ctx'
> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'tile_r_info' not described in 'hantro_vp9_dec_hw_ctx'
> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'tile_c_info' not described in 'hantro_vp9_dec_hw_ctx'
> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'last_tile_r' not described in 'hantro_vp9_dec_hw_ctx'
> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'last_tile_c' not described in 'hantro_vp9_dec_hw_ctx'
> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'last_sbs_r' not described in 'hantro_vp9_dec_hw_ctx'
> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'last_sbs_c' not described in 'hantro_vp9_dec_hw_ctx'
> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'active_segment' not described in 'hantro_vp9_dec_hw_ctx'
> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'feature_enabled' not described in 'hantro_vp9_dec_hw_ctx'
> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'feature_data' not described in 'hantro_vp9_dec_hw_ctx'
>
> You can test kerneldoc yourself with: scripts/kernel-doc -none include/media/v4l2-vp9.h
>
> Regards,
>
> Hans
>
>>
>> Regards,
>>
>> Andrzej
>>
>>
>>>
>>> Regards,
>>>
>>> Hans
>>>
>>> On 29/09/2021 18:04, Andrzej Pietrasiewicz wrote:
>>>> Dear all,
>>>>
>>>> This patch series adds VP9 codec V4L2 control interface and two drivers
>>>> using the new controls. It is a follow-up of previous v6 series [1].
>>>>
>>>> In this iteration, we've implemented VP9 hardware decoding on two devices:
>>>> Rockchip VDEC and Hantro G2, and tested on RK3399, i.MX8MQ and i.MX8MP.
>>>> The i.MX8M driver needs proper power domains support, though, which is a
>>>> subject of a different effort, but in all 3 cases we were able to run the
>>>> drivers.
>>>>
>>>> GStreamer support is also available, the needed changes have been submitted
>>>> by Daniel Almeida [2]. This MR is ready to be merged, and just needs the
>>>> VP9 V4L2 controls to be merged and released.
>>>>
>>>> Both rkvdec and hantro drivers are passing a significant number of VP9 tests
>>>> using Fluster[3]. There are still a few tests that are not passing, due to
>>>> dynamic frame resize (not yet supported by V4L2) and small size videos
>>>> (due to IP block limitations).
>>>>
>>>> The series adds the VP9 codec V4L2 control API as uAPI, so it aims at being
>>>> merged without passing through staging, as agreed[4]. The ABI has been checked
>>>> for padding and verified to contain no holes.
>>>>
>>>> [1] https://patchwork.linuxtv.org/project/linux-media/list/?series=6377
>>>> [2] https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2144
>>>> [3] https://github.com/fluendo/fluster
>>>> [4] https://lore.kernel.org/linux-media/[email protected]/
>>>>
>>>> The series depends on the YUV tiled format support prepared by Ezequiel:
>>>> https://www.spinics.net/lists/linux-media/msg197047.html
>>>>
>>>> Rebased onto latest media_tree.
>>>>
>>>> Changes related to v6:
>>>> - moved setting tile filter and tile bsd auxiliary buffer addresses so
>>>> that they are always set, even if no tiles are used (thanks, Jernej)
>>>> - added a comment near the place where the 32-bit DMA mask is applied
>>>> (thanks, Nicolas)
>>>> - improved consistency in register names (thanks, Nicolas)
>>>>
>>>> Changes related to v5:
>>>> - improved the doc comments as per Ezequiel's review (thanks, Ezequiel)
>>>> - improved pdf output of documentation
>>>> - added Benjamin's Reviewed-by (thanks, Benjamin)
>>>>
>>>> Changes related to v4:
>>>> - removed unused enum v4l2_vp9_intra_prediction_mode
>>>> - converted remaining enums to defines to follow the convention
>>>> - improved the documentation, in particular better documented how to use segmentation
>>>> features
>>>>
>>>> Changes related to v3:
>>>>
>>>> Apply suggestions from Jernej's review (thanks, Jernej):
>>>> - renamed a control and two structs:
>>>> V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR_PROBS =>
>>>> V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR
>>>> v4l2_ctrl_vp9_compressed_hdr_probs =>
>>>> v4l2_ctrl_vp9_compressed_hdr
>>>> v4l2_vp9_mv_compressed_hdr_probs => v4l2_vp9_mv_probs
>>>> - moved tx_mode to v4l2_ctrl_vp9_compressed_hdr
>>>> - fixed enum v4l2_vp9_ref_frame_sign_bias values (which are used to test a bitfield)
>>>> - explicitly assigned values to all other vp9 enums
>>>>
>>>> Apply suggestion from Nicolas's review (thanks, Nicolas):
>>>> - explicitly stated that the v4l2_ctrl_vp9_compressed_hdr control is optional
>>>> and implemented only by drivers which need it
>>>>
>>>> Changes related to the RFC v2:
>>>>
>>>> - added another driver including a postprocessor to de-tile
>>>> codec-specific tiling
>>>> - reworked uAPI structs layout to follow VP8 style
>>>> - changed validation of loop filter params
>>>> - changed validation of segmentation params
>>>> - changed validation of VP9 frame params
>>>> - removed level lookup array from loop filter struct
>>>> (can be computed by drivers)
>>>> - renamed some enum values to match the spec more closely
>>>> - V4L2 VP9 library changed the 'eob' member of
>>>> 'struct v4l2_vp9_frame_symbol_counts' so that it is an array
>>>> of pointers instead of an array of pointers to arrays
>>>> (IPs such as g2 creatively pass parts of the 'eob' counts in
>>>> the 'coeff' counts)
>>>> - factored out several repeated portions of code
>>>> - minor nitpicks and cleanups
>>>>
>>>> Andrzej Pietrasiewicz (6):
>>>> media: uapi: Add VP9 stateless decoder controls
>>>> media: Add VP9 v4l2 library
>>>> media: hantro: Rename registers
>>>> media: hantro: Prepare for other G2 codecs
>>>> media: hantro: Support VP9 on the G2 core
>>>> media: hantro: Support NV12 on the G2 core
>>>>
>>>> Boris Brezillon (1):
>>>> media: rkvdec: Add the VP9 backend
>>>>
>>>> Ezequiel Garcia (4):
>>>> hantro: postproc: Fix motion vector space size
>>>> hantro: postproc: Introduce struct hantro_postproc_ops
>>>> hantro: Simplify postprocessor
>>>> hantro: Add quirk for NV12/NV12_4L4 capture format
>>>>
>>>> .../userspace-api/media/v4l/biblio.rst | 10 +
>>>> .../media/v4l/ext-ctrls-codec-stateless.rst | 573 +++++
>>>> .../media/v4l/pixfmt-compressed.rst | 15 +
>>>> .../media/v4l/vidioc-g-ext-ctrls.rst | 8 +
>>>> .../media/v4l/vidioc-queryctrl.rst | 12 +
>>>> .../media/videodev2.h.rst.exceptions | 2 +
>>>> drivers/media/v4l2-core/Kconfig | 4 +
>>>> drivers/media/v4l2-core/Makefile | 1 +
>>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 180 ++
>>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 8 +
>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
>>>> drivers/media/v4l2-core/v4l2-vp9.c | 1850 +++++++++++++++++
>>>> drivers/staging/media/hantro/Kconfig | 1 +
>>>> drivers/staging/media/hantro/Makefile | 7 +-
>>>> drivers/staging/media/hantro/hantro.h | 40 +-
>>>> drivers/staging/media/hantro/hantro_drv.c | 23 +-
>>>> drivers/staging/media/hantro/hantro_g2.c | 27 +
>>>> .../staging/media/hantro/hantro_g2_hevc_dec.c | 69 +-
>>>> drivers/staging/media/hantro/hantro_g2_regs.h | 132 +-
>>>> .../staging/media/hantro/hantro_g2_vp9_dec.c | 980 +++++++++
>>>> drivers/staging/media/hantro/hantro_hw.h | 83 +-
>>>> .../staging/media/hantro/hantro_postproc.c | 79 +-
>>>> drivers/staging/media/hantro/hantro_v4l2.c | 20 +
>>>> drivers/staging/media/hantro/hantro_vp9.c | 240 +++
>>>> drivers/staging/media/hantro/hantro_vp9.h | 103 +
>>>> drivers/staging/media/hantro/imx8m_vpu_hw.c | 38 +-
>>>> .../staging/media/hantro/rockchip_vpu_hw.c | 7 +-
>>>> .../staging/media/hantro/sama5d4_vdec_hw.c | 3 +-
>>>> drivers/staging/media/rkvdec/Kconfig | 1 +
>>>> drivers/staging/media/rkvdec/Makefile | 2 +-
>>>> drivers/staging/media/rkvdec/rkvdec-vp9.c | 1078 ++++++++++
>>>> drivers/staging/media/rkvdec/rkvdec.c | 52 +-
>>>> drivers/staging/media/rkvdec/rkvdec.h | 12 +-
>>>> include/media/v4l2-ctrls.h | 4 +
>>>> include/media/v4l2-vp9.h | 182 ++
>>>> include/uapi/linux/v4l2-controls.h | 284 +++
>>>> include/uapi/linux/videodev2.h | 6 +
>>>> 37 files changed, 6033 insertions(+), 104 deletions(-)
>>>> create mode 100644 drivers/media/v4l2-core/v4l2-vp9.c
>>>> create mode 100644 drivers/staging/media/hantro/hantro_g2.c
>>>> create mode 100644 drivers/staging/media/hantro/hantro_g2_vp9_dec.c
>>>> create mode 100644 drivers/staging/media/hantro/hantro_vp9.c
>>>> create mode 100644 drivers/staging/media/hantro/hantro_vp9.h
>>>> create mode 100644 drivers/staging/media/rkvdec/rkvdec-vp9.c
>>>> create mode 100644 include/media/v4l2-vp9.h
>>>>
>>>>
>>>> base-commit: e4e737bb5c170df6135a127739a9e6148ee3da82
>>>>
>>>
>>
>
On 16/11/2021 09:09, Andrzej Pietrasiewicz wrote:
> Hi Hans,
>
> W dniu 15.11.2021 o 22:16, Hans Verkuil pisze:
>> On 15/11/2021 18:14, Andrzej Pietrasiewicz wrote:
>>> Hi Hans,
>>>
>>> W dniu 15.11.2021 o 16:07, Hans Verkuil pisze:
>>>> Andrzej,
>>>>
>>>> Can you rebase this series on top of the master branch of
>>>> https://git.linuxtv.org/media_stage.git/ ? Unfortunately this v7 no longer
>>>> applies. Specifically "rkvdec: Add the VP9 backend" failed in a non-trivial
>>>> manner.
>>>
>>> This is a branch for you:
>>>
>>> https://gitlab.collabora.com/linux/for-upstream/-/tree/vp9-uapi
>>
>> I'm getting a bunch of sparse/smatch warnings:
>>
>
> Thanks for finding this, I will re-create the branch and let you know on irc.
> Some of the below are "false positives, namely:
>
> drivers/media/platform/omap3isp/omap3isp.h
> drivers/media/platform/qcom/venus/core.h
Ah, sorry, I though I had filtered those out. Obviously you can ignore those.
Please post a v8. That way the series is archived on lore. And it works better
with patchwork.
Regards,
Hans
>
> which are not touched by the series.
>
> Regards,
>
> Andrzej
>
>> sparse:
>> rkvdec/rkvdec-vp9.c:190:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
>> rkvdec/rkvdec-vp9.c:245:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
>> SPARSE:hantro/hantro_postproc.c hantro/hantro_postproc.c:37:35: warning: symbol 'hantro_g1_postproc_regs' was not declared. Should it be static?
>>
>> smatch:
>> rkvdec/rkvdec-vp9.c:190:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
>> rkvdec/rkvdec-vp9.c:245:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
>> rkvdec/rkvdec-vp9.c: rkvdec/rkvdec-vp9.c:236 init_intra_only_probs() error: buffer overflow 'ptr' 90 <= 91
>> hantro/hantro_g2_vp9_dec.c: hantro/hantro_g2_vp9_dec.c:670 config_probs() error: memcpy() 'adaptive->inter_mode[i]' too small (4 vs 21)
>> hantro/hantro_g2_vp9_dec.c: hantro/hantro_g2_vp9_dec.c:670 config_probs() error: memcpy() 'probs->inter_mode[i]' too small (3 vs 21
>>
>> Also a bunch of kerneldoc warnings:
>>
>> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'joint' not described in 'v4l2_vp9_frame_mv_context'
>> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'sign' not described in 'v4l2_vp9_frame_mv_context'
>> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'classes' not described in 'v4l2_vp9_frame_mv_context'
>> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'class0_bit' not described in 'v4l2_vp9_frame_mv_context'
>> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'bits' not described in 'v4l2_vp9_frame_mv_context'
>> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'class0_fr' not described in 'v4l2_vp9_frame_mv_context'
>> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'fr' not described in 'v4l2_vp9_frame_mv_context'
>> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'class0_hp' not described in 'v4l2_vp9_frame_mv_context'
>> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'hp' not described in 'v4l2_vp9_frame_mv_context'
>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'tx8' not described in 'v4l2_vp9_frame_context'
>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'tx16' not described in 'v4l2_vp9_frame_context'
>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'tx32' not described in 'v4l2_vp9_frame_context'
>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'coef' not described in 'v4l2_vp9_frame_context'
>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'skip' not described in 'v4l2_vp9_frame_context'
>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'inter_mode' not described in 'v4l2_vp9_frame_context'
>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'interp_filter' not described in 'v4l2_vp9_frame_context'
>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'is_inter' not described in 'v4l2_vp9_frame_context'
>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'comp_mode' not described in 'v4l2_vp9_frame_context'
>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'single_ref' not described in 'v4l2_vp9_frame_context'
>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'comp_ref' not described in 'v4l2_vp9_frame_context'
>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'y_mode' not described in 'v4l2_vp9_frame_context'
>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'uv_mode' not described in 'v4l2_vp9_frame_context'
>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'partition' not described in 'v4l2_vp9_frame_context'
>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'mv' not described in 'v4l2_vp9_frame_context'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'partition' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'skip' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'intra_inter' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'tx32p' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'tx16p' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'tx8p' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'y_mode' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'uv_mode' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'comp' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'comp_ref' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'single_ref' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'mv_mode' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'filter' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'mv_joint' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'sign' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'classes' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'class0' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'bits' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'class0_fp' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'fp' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'class0_hp' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'hp' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'coeff' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'eob' not described in 'v4l2_vp9_frame_symbol_counts'
>> include/media/v4l2-vp9.h:166: warning: expecting prototype for v4l2_vp9_adapt_coef_probs(). Prototype was for v4l2_vp9_adapt_noncoef_probs()
>> instead
>> drivers/media/platform/omap3isp/omap3isp.h:107: warning: Function parameter or member 'vp_clk_pol' not described in 'isp_ccp2_cfg'
>> drivers/media/platform/omap3isp/omap3isp.h:107: warning: Function parameter or member 'lanecfg' not described in 'isp_ccp2_cfg'
>> drivers/media/platform/qcom/venus/core.h:202: warning: Function parameter or member 'sys_err_done' not described in 'venus_core'
>> drivers/media/platform/qcom/venus/core.h:462: warning: Function parameter or member 'fw_min_cnt' not described in 'venus_inst'
>> drivers/media/platform/qcom/venus/core.h:462: warning: Function parameter or member 'flags' not described in 'venus_inst'
>> drivers/media/platform/qcom/venus/core.h:462: warning: Function parameter or member 'dpb_ids' not described in 'venus_inst'
>> drivers/staging/media/hantro/hantro.h:115: warning: Enum value 'HANTRO_MODE_VP9_DEC' not described in enum 'hantro_codec_mode'
>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'tile_edge' not described in 'hantro_vp9_dec_hw_ctx'
>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'segment_map' not described in 'hantro_vp9_dec_hw_ctx'
>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'misc' not described in 'hantro_vp9_dec_hw_ctx'
>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'cnts' not described in 'hantro_vp9_dec_hw_ctx'
>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'probability_tables' not described in
>> 'hantro_vp9_dec_hw_ctx'
>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'frame_context' not described in 'hantro_vp9_dec_hw_ctx'
>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'cur' not described in 'hantro_vp9_dec_hw_ctx'
>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'last' not described in 'hantro_vp9_dec_hw_ctx'
>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'bsd_ctrl_offset' not described in 'hantro_vp9_dec_hw_ctx'
>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'segment_map_size' not described in 'hantro_vp9_dec_hw_ctx'
>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'ctx_counters_offset' not described in
>> 'hantro_vp9_dec_hw_ctx'
>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'tile_info_offset' not described in 'hantro_vp9_dec_hw_ctx'
>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'tile_r_info' not described in 'hantro_vp9_dec_hw_ctx'
>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'tile_c_info' not described in 'hantro_vp9_dec_hw_ctx'
>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'last_tile_r' not described in 'hantro_vp9_dec_hw_ctx'
>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'last_tile_c' not described in 'hantro_vp9_dec_hw_ctx'
>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'last_sbs_r' not described in 'hantro_vp9_dec_hw_ctx'
>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'last_sbs_c' not described in 'hantro_vp9_dec_hw_ctx'
>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'active_segment' not described in 'hantro_vp9_dec_hw_ctx'
>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'feature_enabled' not described in 'hantro_vp9_dec_hw_ctx'
>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'feature_data' not described in 'hantro_vp9_dec_hw_ctx'
>>
>> You can test kerneldoc yourself with: scripts/kernel-doc -none include/media/v4l2-vp9.h
>>
>> Regards,
>>
>> Hans
>>
>>>
>>> Regards,
>>>
>>> Andrzej
>>>
>>>
>>>>
>>>> Regards,
>>>>
>>>> Hans
>>>>
>>>> On 29/09/2021 18:04, Andrzej Pietrasiewicz wrote:
>>>>> Dear all,
>>>>>
>>>>> This patch series adds VP9 codec V4L2 control interface and two drivers
>>>>> using the new controls. It is a follow-up of previous v6 series [1].
>>>>>
>>>>> In this iteration, we've implemented VP9 hardware decoding on two devices:
>>>>> Rockchip VDEC and Hantro G2, and tested on RK3399, i.MX8MQ and i.MX8MP.
>>>>> The i.MX8M driver needs proper power domains support, though, which is a
>>>>> subject of a different effort, but in all 3 cases we were able to run the
>>>>> drivers.
>>>>>
>>>>> GStreamer support is also available, the needed changes have been submitted
>>>>> by Daniel Almeida [2]. This MR is ready to be merged, and just needs the
>>>>> VP9 V4L2 controls to be merged and released.
>>>>>
>>>>> Both rkvdec and hantro drivers are passing a significant number of VP9 tests
>>>>> using Fluster[3]. There are still a few tests that are not passing, due to
>>>>> dynamic frame resize (not yet supported by V4L2) and small size videos
>>>>> (due to IP block limitations).
>>>>>
>>>>> The series adds the VP9 codec V4L2 control API as uAPI, so it aims at being
>>>>> merged without passing through staging, as agreed[4]. The ABI has been checked
>>>>> for padding and verified to contain no holes.
>>>>>
>>>>> [1] https://patchwork.linuxtv.org/project/linux-media/list/?series=6377
>>>>> [2] https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2144
>>>>> [3] https://github.com/fluendo/fluster
>>>>> [4] https://lore.kernel.org/linux-media/[email protected]/
>>>>>
>>>>> The series depends on the YUV tiled format support prepared by Ezequiel:
>>>>> https://www.spinics.net/lists/linux-media/msg197047.html
>>>>>
>>>>> Rebased onto latest media_tree.
>>>>>
>>>>> Changes related to v6:
>>>>> - moved setting tile filter and tile bsd auxiliary buffer addresses so
>>>>> that they are always set, even if no tiles are used (thanks, Jernej)
>>>>> - added a comment near the place where the 32-bit DMA mask is applied
>>>>> (thanks, Nicolas)
>>>>> - improved consistency in register names (thanks, Nicolas)
>>>>>
>>>>> Changes related to v5:
>>>>> - improved the doc comments as per Ezequiel's review (thanks, Ezequiel)
>>>>> - improved pdf output of documentation
>>>>> - added Benjamin's Reviewed-by (thanks, Benjamin)
>>>>>
>>>>> Changes related to v4:
>>>>> - removed unused enum v4l2_vp9_intra_prediction_mode
>>>>> - converted remaining enums to defines to follow the convention
>>>>> - improved the documentation, in particular better documented how to use segmentation
>>>>> features
>>>>>
>>>>> Changes related to v3:
>>>>>
>>>>> Apply suggestions from Jernej's review (thanks, Jernej):
>>>>> - renamed a control and two structs:
>>>>> V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR_PROBS =>
>>>>> V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR
>>>>> v4l2_ctrl_vp9_compressed_hdr_probs =>
>>>>> v4l2_ctrl_vp9_compressed_hdr
>>>>> v4l2_vp9_mv_compressed_hdr_probs => v4l2_vp9_mv_probs
>>>>> - moved tx_mode to v4l2_ctrl_vp9_compressed_hdr
>>>>> - fixed enum v4l2_vp9_ref_frame_sign_bias values (which are used to test a bitfield)
>>>>> - explicitly assigned values to all other vp9 enums
>>>>>
>>>>> Apply suggestion from Nicolas's review (thanks, Nicolas):
>>>>> - explicitly stated that the v4l2_ctrl_vp9_compressed_hdr control is optional
>>>>> and implemented only by drivers which need it
>>>>>
>>>>> Changes related to the RFC v2:
>>>>>
>>>>> - added another driver including a postprocessor to de-tile
>>>>> codec-specific tiling
>>>>> - reworked uAPI structs layout to follow VP8 style
>>>>> - changed validation of loop filter params
>>>>> - changed validation of segmentation params
>>>>> - changed validation of VP9 frame params
>>>>> - removed level lookup array from loop filter struct
>>>>> (can be computed by drivers)
>>>>> - renamed some enum values to match the spec more closely
>>>>> - V4L2 VP9 library changed the 'eob' member of
>>>>> 'struct v4l2_vp9_frame_symbol_counts' so that it is an array
>>>>> of pointers instead of an array of pointers to arrays
>>>>> (IPs such as g2 creatively pass parts of the 'eob' counts in
>>>>> the 'coeff' counts)
>>>>> - factored out several repeated portions of code
>>>>> - minor nitpicks and cleanups
>>>>>
>>>>> Andrzej Pietrasiewicz (6):
>>>>> media: uapi: Add VP9 stateless decoder controls
>>>>> media: Add VP9 v4l2 library
>>>>> media: hantro: Rename registers
>>>>> media: hantro: Prepare for other G2 codecs
>>>>> media: hantro: Support VP9 on the G2 core
>>>>> media: hantro: Support NV12 on the G2 core
>>>>>
>>>>> Boris Brezillon (1):
>>>>> media: rkvdec: Add the VP9 backend
>>>>>
>>>>> Ezequiel Garcia (4):
>>>>> hantro: postproc: Fix motion vector space size
>>>>> hantro: postproc: Introduce struct hantro_postproc_ops
>>>>> hantro: Simplify postprocessor
>>>>> hantro: Add quirk for NV12/NV12_4L4 capture format
>>>>>
>>>>> .../userspace-api/media/v4l/biblio.rst | 10 +
>>>>> .../media/v4l/ext-ctrls-codec-stateless.rst | 573 +++++
>>>>> .../media/v4l/pixfmt-compressed.rst | 15 +
>>>>> .../media/v4l/vidioc-g-ext-ctrls.rst | 8 +
>>>>> .../media/v4l/vidioc-queryctrl.rst | 12 +
>>>>> .../media/videodev2.h.rst.exceptions | 2 +
>>>>> drivers/media/v4l2-core/Kconfig | 4 +
>>>>> drivers/media/v4l2-core/Makefile | 1 +
>>>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 180 ++
>>>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 8 +
>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
>>>>> drivers/media/v4l2-core/v4l2-vp9.c | 1850 +++++++++++++++++
>>>>> drivers/staging/media/hantro/Kconfig | 1 +
>>>>> drivers/staging/media/hantro/Makefile | 7 +-
>>>>> drivers/staging/media/hantro/hantro.h | 40 +-
>>>>> drivers/staging/media/hantro/hantro_drv.c | 23 +-
>>>>> drivers/staging/media/hantro/hantro_g2.c | 27 +
>>>>> .../staging/media/hantro/hantro_g2_hevc_dec.c | 69 +-
>>>>> drivers/staging/media/hantro/hantro_g2_regs.h | 132 +-
>>>>> .../staging/media/hantro/hantro_g2_vp9_dec.c | 980 +++++++++
>>>>> drivers/staging/media/hantro/hantro_hw.h | 83 +-
>>>>> .../staging/media/hantro/hantro_postproc.c | 79 +-
>>>>> drivers/staging/media/hantro/hantro_v4l2.c | 20 +
>>>>> drivers/staging/media/hantro/hantro_vp9.c | 240 +++
>>>>> drivers/staging/media/hantro/hantro_vp9.h | 103 +
>>>>> drivers/staging/media/hantro/imx8m_vpu_hw.c | 38 +-
>>>>> .../staging/media/hantro/rockchip_vpu_hw.c | 7 +-
>>>>> .../staging/media/hantro/sama5d4_vdec_hw.c | 3 +-
>>>>> drivers/staging/media/rkvdec/Kconfig | 1 +
>>>>> drivers/staging/media/rkvdec/Makefile | 2 +-
>>>>> drivers/staging/media/rkvdec/rkvdec-vp9.c | 1078 ++++++++++
>>>>> drivers/staging/media/rkvdec/rkvdec.c | 52 +-
>>>>> drivers/staging/media/rkvdec/rkvdec.h | 12 +-
>>>>> include/media/v4l2-ctrls.h | 4 +
>>>>> include/media/v4l2-vp9.h | 182 ++
>>>>> include/uapi/linux/v4l2-controls.h | 284 +++
>>>>> include/uapi/linux/videodev2.h | 6 +
>>>>> 37 files changed, 6033 insertions(+), 104 deletions(-)
>>>>> create mode 100644 drivers/media/v4l2-core/v4l2-vp9.c
>>>>> create mode 100644 drivers/staging/media/hantro/hantro_g2.c
>>>>> create mode 100644 drivers/staging/media/hantro/hantro_g2_vp9_dec.c
>>>>> create mode 100644 drivers/staging/media/hantro/hantro_vp9.c
>>>>> create mode 100644 drivers/staging/media/hantro/hantro_vp9.h
>>>>> create mode 100644 drivers/staging/media/rkvdec/rkvdec-vp9.c
>>>>> create mode 100644 include/media/v4l2-vp9.h
>>>>>
>>>>>
>>>>> base-commit: e4e737bb5c170df6135a127739a9e6148ee3da82
>>>>>
>>>>
>>>
>>
>
Hi,
W dniu 16.11.2021 o 09:21, Hans Verkuil pisze:
> On 16/11/2021 09:09, Andrzej Pietrasiewicz wrote:
>> Hi Hans,
>>
>> W dniu 15.11.2021 o 22:16, Hans Verkuil pisze:
>>> On 15/11/2021 18:14, Andrzej Pietrasiewicz wrote:
>>>> Hi Hans,
>>>>
>>>> W dniu 15.11.2021 o 16:07, Hans Verkuil pisze:
>>>>> Andrzej,
>>>>>
>>>>> Can you rebase this series on top of the master branch of
>>>>> https://git.linuxtv.org/media_stage.git/ ? Unfortunately this v7 no longer
>>>>> applies. Specifically "rkvdec: Add the VP9 backend" failed in a non-trivial
>>>>> manner.
>>>>
>>>> This is a branch for you:
>>>>
>>>> https://gitlab.collabora.com/linux/for-upstream/-/tree/vp9-uapi
>>>
>>> I'm getting a bunch of sparse/smatch warnings:
>>>
>>
>> Thanks for finding this, I will re-create the branch and let you know on irc.
>> Some of the below are "false positives, namely:
>>
>> drivers/media/platform/omap3isp/omap3isp.h
>> drivers/media/platform/qcom/venus/core.h
>
> Ah, sorry, I though I had filtered those out. Obviously you can ignore those.
>
> Please post a v8. That way the series is archived on lore. And it works better
> with patchwork.
Sure, no problem. Also please see below.
>
> Regards,
>
> Hans
>
>>
>> which are not touched by the series.
>>
>> Regards,
>>
>> Andrzej
>>
>>> sparse:
>>> rkvdec/rkvdec-vp9.c:190:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
>>> rkvdec/rkvdec-vp9.c:245:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
>>> SPARSE:hantro/hantro_postproc.c hantro/hantro_postproc.c:37:35: warning: symbol 'hantro_g1_postproc_regs' was not declared. Should it be static?
>>>
>>> smatch:
>>> rkvdec/rkvdec-vp9.c:190:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
>>> rkvdec/rkvdec-vp9.c:245:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
>>> rkvdec/rkvdec-vp9.c: rkvdec/rkvdec-vp9.c:236 init_intra_only_probs() error: buffer overflow 'ptr' 90 <= 91
this looks a false positive.
A portion of memory pointed to by ptr is indexed with i * 23 + m,
where i ranges from 0 to 3, inclusive, and m ranges from 0 to 22,
inclusive if i < 3, otherwise m ranges from 0 to 20, inclusive.
So the largest index value we compute equals 89 (3 * 23 + 20).
Because ptr points to something that is at least 90 bytes large,
89 is a valid index and no greater index will be ever computed.
>>> hantro/hantro_g2_vp9_dec.c: hantro/hantro_g2_vp9_dec.c:670 config_probs() error: memcpy() 'adaptive->inter_mode[i]' too small (4 vs 21)
>>> hantro/hantro_g2_vp9_dec.c: hantro/hantro_g2_vp9_dec.c:670 config_probs() error: memcpy() 'probs->inter_mode[i]' too small (3 vs 21
>>>
>>> Also a bunch of kerneldoc warnings:
>>>
>>> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'joint' not described in 'v4l2_vp9_frame_mv_context'
>>> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'sign' not described in 'v4l2_vp9_frame_mv_context'
>>> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'classes' not described in 'v4l2_vp9_frame_mv_context'
>>> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'class0_bit' not described in 'v4l2_vp9_frame_mv_context'
>>> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'bits' not described in 'v4l2_vp9_frame_mv_context'
>>> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'class0_fr' not described in 'v4l2_vp9_frame_mv_context'
>>> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'fr' not described in 'v4l2_vp9_frame_mv_context'
>>> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'class0_hp' not described in 'v4l2_vp9_frame_mv_context'
>>> include/media/v4l2-vp9.h:30: warning: Function parameter or member 'hp' not described in 'v4l2_vp9_frame_mv_context'
>>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'tx8' not described in 'v4l2_vp9_frame_context'
>>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'tx16' not described in 'v4l2_vp9_frame_context'
>>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'tx32' not described in 'v4l2_vp9_frame_context'
>>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'coef' not described in 'v4l2_vp9_frame_context'
>>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'skip' not described in 'v4l2_vp9_frame_context'
>>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'inter_mode' not described in 'v4l2_vp9_frame_context'
>>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'interp_filter' not described in 'v4l2_vp9_frame_context'
>>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'is_inter' not described in 'v4l2_vp9_frame_context'
>>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'comp_mode' not described in 'v4l2_vp9_frame_context'
>>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'single_ref' not described in 'v4l2_vp9_frame_context'
>>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'comp_ref' not described in 'v4l2_vp9_frame_context'
>>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'y_mode' not described in 'v4l2_vp9_frame_context'
>>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'uv_mode' not described in 'v4l2_vp9_frame_context'
>>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'partition' not described in 'v4l2_vp9_frame_context'
>>> include/media/v4l2-vp9.h:58: warning: Function parameter or member 'mv' not described in 'v4l2_vp9_frame_context'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'partition' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'skip' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'intra_inter' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'tx32p' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'tx16p' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'tx8p' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'y_mode' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'uv_mode' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'comp' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'comp_ref' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'single_ref' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'mv_mode' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'filter' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'mv_joint' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'sign' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'classes' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'class0' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'bits' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'class0_fp' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'fp' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'class0_hp' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'hp' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'coeff' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:93: warning: Function parameter or member 'eob' not described in 'v4l2_vp9_frame_symbol_counts'
>>> include/media/v4l2-vp9.h:166: warning: expecting prototype for v4l2_vp9_adapt_coef_probs(). Prototype was for v4l2_vp9_adapt_noncoef_probs()
>>> instead
>>> drivers/media/platform/omap3isp/omap3isp.h:107: warning: Function parameter or member 'vp_clk_pol' not described in 'isp_ccp2_cfg'
>>> drivers/media/platform/omap3isp/omap3isp.h:107: warning: Function parameter or member 'lanecfg' not described in 'isp_ccp2_cfg'
>>> drivers/media/platform/qcom/venus/core.h:202: warning: Function parameter or member 'sys_err_done' not described in 'venus_core'
>>> drivers/media/platform/qcom/venus/core.h:462: warning: Function parameter or member 'fw_min_cnt' not described in 'venus_inst'
>>> drivers/media/platform/qcom/venus/core.h:462: warning: Function parameter or member 'flags' not described in 'venus_inst'
>>> drivers/media/platform/qcom/venus/core.h:462: warning: Function parameter or member 'dpb_ids' not described in 'venus_inst'
>>> drivers/staging/media/hantro/hantro.h:115: warning: Enum value 'HANTRO_MODE_VP9_DEC' not described in enum 'hantro_codec_mode'
>>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'tile_edge' not described in 'hantro_vp9_dec_hw_ctx'
>>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'segment_map' not described in 'hantro_vp9_dec_hw_ctx'
>>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'misc' not described in 'hantro_vp9_dec_hw_ctx'
>>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'cnts' not described in 'hantro_vp9_dec_hw_ctx'
>>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'probability_tables' not described in
>>> 'hantro_vp9_dec_hw_ctx'
>>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'frame_context' not described in 'hantro_vp9_dec_hw_ctx'
>>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'cur' not described in 'hantro_vp9_dec_hw_ctx'
>>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'last' not described in 'hantro_vp9_dec_hw_ctx'
>>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'bsd_ctrl_offset' not described in 'hantro_vp9_dec_hw_ctx'
>>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'segment_map_size' not described in 'hantro_vp9_dec_hw_ctx'
>>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'ctx_counters_offset' not described in
>>> 'hantro_vp9_dec_hw_ctx'
>>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'tile_info_offset' not described in 'hantro_vp9_dec_hw_ctx'
>>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'tile_r_info' not described in 'hantro_vp9_dec_hw_ctx'
>>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'tile_c_info' not described in 'hantro_vp9_dec_hw_ctx'
>>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'last_tile_r' not described in 'hantro_vp9_dec_hw_ctx'
>>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'last_tile_c' not described in 'hantro_vp9_dec_hw_ctx'
>>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'last_sbs_r' not described in 'hantro_vp9_dec_hw_ctx'
>>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'last_sbs_c' not described in 'hantro_vp9_dec_hw_ctx'
>>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'active_segment' not described in 'hantro_vp9_dec_hw_ctx'
>>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'feature_enabled' not described in 'hantro_vp9_dec_hw_ctx'
>>> drivers/staging/media/hantro/hantro_hw.h:211: warning: Function parameter or member 'feature_data' not described in 'hantro_vp9_dec_hw_ctx'
>>>
>>> You can test kerneldoc yourself with: scripts/kernel-doc -none include/media/v4l2-vp9.h
>>>
>>> Regards,
>>>
>>> Hans
>>>
>>>>
>>>> Regards,
>>>>
>>>> Andrzej
>>>>
>>>>
>>>>>
>>>>> Regards,
>>>>>
>>>>> Hans
>>>>>
>>>>> On 29/09/2021 18:04, Andrzej Pietrasiewicz wrote:
>>>>>> Dear all,
>>>>>>
>>>>>> This patch series adds VP9 codec V4L2 control interface and two drivers
>>>>>> using the new controls. It is a follow-up of previous v6 series [1].
>>>>>>
>>>>>> In this iteration, we've implemented VP9 hardware decoding on two devices:
>>>>>> Rockchip VDEC and Hantro G2, and tested on RK3399, i.MX8MQ and i.MX8MP.
>>>>>> The i.MX8M driver needs proper power domains support, though, which is a
>>>>>> subject of a different effort, but in all 3 cases we were able to run the
>>>>>> drivers.
>>>>>>
>>>>>> GStreamer support is also available, the needed changes have been submitted
>>>>>> by Daniel Almeida [2]. This MR is ready to be merged, and just needs the
>>>>>> VP9 V4L2 controls to be merged and released.
>>>>>>
>>>>>> Both rkvdec and hantro drivers are passing a significant number of VP9 tests
>>>>>> using Fluster[3]. There are still a few tests that are not passing, due to
>>>>>> dynamic frame resize (not yet supported by V4L2) and small size videos
>>>>>> (due to IP block limitations).
>>>>>>
>>>>>> The series adds the VP9 codec V4L2 control API as uAPI, so it aims at being
>>>>>> merged without passing through staging, as agreed[4]. The ABI has been checked
>>>>>> for padding and verified to contain no holes.
>>>>>>
>>>>>> [1] https://patchwork.linuxtv.org/project/linux-media/list/?series=6377
>>>>>> [2] https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2144
>>>>>> [3] https://github.com/fluendo/fluster
>>>>>> [4] https://lore.kernel.org/linux-media/[email protected]/
>>>>>>
>>>>>> The series depends on the YUV tiled format support prepared by Ezequiel:
>>>>>> https://www.spinics.net/lists/linux-media/msg197047.html
>>>>>>
>>>>>> Rebased onto latest media_tree.
>>>>>>
>>>>>> Changes related to v6:
>>>>>> - moved setting tile filter and tile bsd auxiliary buffer addresses so
>>>>>> that they are always set, even if no tiles are used (thanks, Jernej)
>>>>>> - added a comment near the place where the 32-bit DMA mask is applied
>>>>>> (thanks, Nicolas)
>>>>>> - improved consistency in register names (thanks, Nicolas)
>>>>>>
>>>>>> Changes related to v5:
>>>>>> - improved the doc comments as per Ezequiel's review (thanks, Ezequiel)
>>>>>> - improved pdf output of documentation
>>>>>> - added Benjamin's Reviewed-by (thanks, Benjamin)
>>>>>>
>>>>>> Changes related to v4:
>>>>>> - removed unused enum v4l2_vp9_intra_prediction_mode
>>>>>> - converted remaining enums to defines to follow the convention
>>>>>> - improved the documentation, in particular better documented how to use segmentation
>>>>>> features
>>>>>>
>>>>>> Changes related to v3:
>>>>>>
>>>>>> Apply suggestions from Jernej's review (thanks, Jernej):
>>>>>> - renamed a control and two structs:
>>>>>> V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR_PROBS =>
>>>>>> V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR
>>>>>> v4l2_ctrl_vp9_compressed_hdr_probs =>
>>>>>> v4l2_ctrl_vp9_compressed_hdr
>>>>>> v4l2_vp9_mv_compressed_hdr_probs => v4l2_vp9_mv_probs
>>>>>> - moved tx_mode to v4l2_ctrl_vp9_compressed_hdr
>>>>>> - fixed enum v4l2_vp9_ref_frame_sign_bias values (which are used to test a bitfield)
>>>>>> - explicitly assigned values to all other vp9 enums
>>>>>>
>>>>>> Apply suggestion from Nicolas's review (thanks, Nicolas):
>>>>>> - explicitly stated that the v4l2_ctrl_vp9_compressed_hdr control is optional
>>>>>> and implemented only by drivers which need it
>>>>>>
>>>>>> Changes related to the RFC v2:
>>>>>>
>>>>>> - added another driver including a postprocessor to de-tile
>>>>>> codec-specific tiling
>>>>>> - reworked uAPI structs layout to follow VP8 style
>>>>>> - changed validation of loop filter params
>>>>>> - changed validation of segmentation params
>>>>>> - changed validation of VP9 frame params
>>>>>> - removed level lookup array from loop filter struct
>>>>>> (can be computed by drivers)
>>>>>> - renamed some enum values to match the spec more closely
>>>>>> - V4L2 VP9 library changed the 'eob' member of
>>>>>> 'struct v4l2_vp9_frame_symbol_counts' so that it is an array
>>>>>> of pointers instead of an array of pointers to arrays
>>>>>> (IPs such as g2 creatively pass parts of the 'eob' counts in
>>>>>> the 'coeff' counts)
>>>>>> - factored out several repeated portions of code
>>>>>> - minor nitpicks and cleanups
>>>>>>
>>>>>> Andrzej Pietrasiewicz (6):
>>>>>> media: uapi: Add VP9 stateless decoder controls
>>>>>> media: Add VP9 v4l2 library
>>>>>> media: hantro: Rename registers
>>>>>> media: hantro: Prepare for other G2 codecs
>>>>>> media: hantro: Support VP9 on the G2 core
>>>>>> media: hantro: Support NV12 on the G2 core
>>>>>>
>>>>>> Boris Brezillon (1):
>>>>>> media: rkvdec: Add the VP9 backend
>>>>>>
>>>>>> Ezequiel Garcia (4):
>>>>>> hantro: postproc: Fix motion vector space size
>>>>>> hantro: postproc: Introduce struct hantro_postproc_ops
>>>>>> hantro: Simplify postprocessor
>>>>>> hantro: Add quirk for NV12/NV12_4L4 capture format
>>>>>>
>>>>>> .../userspace-api/media/v4l/biblio.rst | 10 +
>>>>>> .../media/v4l/ext-ctrls-codec-stateless.rst | 573 +++++
>>>>>> .../media/v4l/pixfmt-compressed.rst | 15 +
>>>>>> .../media/v4l/vidioc-g-ext-ctrls.rst | 8 +
>>>>>> .../media/v4l/vidioc-queryctrl.rst | 12 +
>>>>>> .../media/videodev2.h.rst.exceptions | 2 +
>>>>>> drivers/media/v4l2-core/Kconfig | 4 +
>>>>>> drivers/media/v4l2-core/Makefile | 1 +
>>>>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 180 ++
>>>>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 8 +
>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
>>>>>> drivers/media/v4l2-core/v4l2-vp9.c | 1850 +++++++++++++++++
>>>>>> drivers/staging/media/hantro/Kconfig | 1 +
>>>>>> drivers/staging/media/hantro/Makefile | 7 +-
>>>>>> drivers/staging/media/hantro/hantro.h | 40 +-
>>>>>> drivers/staging/media/hantro/hantro_drv.c | 23 +-
>>>>>> drivers/staging/media/hantro/hantro_g2.c | 27 +
>>>>>> .../staging/media/hantro/hantro_g2_hevc_dec.c | 69 +-
>>>>>> drivers/staging/media/hantro/hantro_g2_regs.h | 132 +-
>>>>>> .../staging/media/hantro/hantro_g2_vp9_dec.c | 980 +++++++++
>>>>>> drivers/staging/media/hantro/hantro_hw.h | 83 +-
>>>>>> .../staging/media/hantro/hantro_postproc.c | 79 +-
>>>>>> drivers/staging/media/hantro/hantro_v4l2.c | 20 +
>>>>>> drivers/staging/media/hantro/hantro_vp9.c | 240 +++
>>>>>> drivers/staging/media/hantro/hantro_vp9.h | 103 +
>>>>>> drivers/staging/media/hantro/imx8m_vpu_hw.c | 38 +-
>>>>>> .../staging/media/hantro/rockchip_vpu_hw.c | 7 +-
>>>>>> .../staging/media/hantro/sama5d4_vdec_hw.c | 3 +-
>>>>>> drivers/staging/media/rkvdec/Kconfig | 1 +
>>>>>> drivers/staging/media/rkvdec/Makefile | 2 +-
>>>>>> drivers/staging/media/rkvdec/rkvdec-vp9.c | 1078 ++++++++++
>>>>>> drivers/staging/media/rkvdec/rkvdec.c | 52 +-
>>>>>> drivers/staging/media/rkvdec/rkvdec.h | 12 +-
>>>>>> include/media/v4l2-ctrls.h | 4 +
>>>>>> include/media/v4l2-vp9.h | 182 ++
>>>>>> include/uapi/linux/v4l2-controls.h | 284 +++
>>>>>> include/uapi/linux/videodev2.h | 6 +
>>>>>> 37 files changed, 6033 insertions(+), 104 deletions(-)
>>>>>> create mode 100644 drivers/media/v4l2-core/v4l2-vp9.c
>>>>>> create mode 100644 drivers/staging/media/hantro/hantro_g2.c
>>>>>> create mode 100644 drivers/staging/media/hantro/hantro_g2_vp9_dec.c
>>>>>> create mode 100644 drivers/staging/media/hantro/hantro_vp9.c
>>>>>> create mode 100644 drivers/staging/media/hantro/hantro_vp9.h
>>>>>> create mode 100644 drivers/staging/media/rkvdec/rkvdec-vp9.c
>>>>>> create mode 100644 include/media/v4l2-vp9.h
>>>>>>
>>>>>>
>>>>>> base-commit: e4e737bb5c170df6135a127739a9e6148ee3da82
>>>>>>
>>>>>
>>>>
>>>
>>
>
On 16/11/2021 14:14, Andrzej Pietrasiewicz wrote:
> Hi,
>
> W dniu 16.11.2021 o 09:21, Hans Verkuil pisze:
>> On 16/11/2021 09:09, Andrzej Pietrasiewicz wrote:
>>> Hi Hans,
>>>
>>> W dniu 15.11.2021 o 22:16, Hans Verkuil pisze:
>>>> On 15/11/2021 18:14, Andrzej Pietrasiewicz wrote:
>>>>> Hi Hans,
>>>>>
>>>>> W dniu 15.11.2021 o 16:07, Hans Verkuil pisze:
>>>>>> Andrzej,
>>>>>>
>>>>>> Can you rebase this series on top of the master branch of
>>>>>> https://git.linuxtv.org/media_stage.git/ ? Unfortunately this v7 no longer
>>>>>> applies. Specifically "rkvdec: Add the VP9 backend" failed in a non-trivial
>>>>>> manner.
>>>>>
>>>>> This is a branch for you:
>>>>>
>>>>> https://gitlab.collabora.com/linux/for-upstream/-/tree/vp9-uapi
>>>>
>>>> I'm getting a bunch of sparse/smatch warnings:
>>>>
>>>
>>> Thanks for finding this, I will re-create the branch and let you know on irc.
>>> Some of the below are "false positives, namely:
>>>
>>> drivers/media/platform/omap3isp/omap3isp.h
>>> drivers/media/platform/qcom/venus/core.h
>>
>> Ah, sorry, I though I had filtered those out. Obviously you can ignore those.
>>
>> Please post a v8. That way the series is archived on lore. And it works better
>> with patchwork.
>
> Sure, no problem. Also please see below.
>
>>
>> Regards,
>>
>> Hans
>>
>>>
>>> which are not touched by the series.
>>>
>>> Regards,
>>>
>>> Andrzej
>>>
>>>> sparse:
>>>> rkvdec/rkvdec-vp9.c:190:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
>>>> rkvdec/rkvdec-vp9.c:245:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
>>>> SPARSE:hantro/hantro_postproc.c hantro/hantro_postproc.c:37:35: warning: symbol 'hantro_g1_postproc_regs' was not declared. Should it be static?
>>>>
>>>> smatch:
>>>> rkvdec/rkvdec-vp9.c:190:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
>>>> rkvdec/rkvdec-vp9.c:245:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
>>>> rkvdec/rkvdec-vp9.c: rkvdec/rkvdec-vp9.c:236 init_intra_only_probs() error: buffer overflow 'ptr' 90 <= 91
>
> this looks a false positive.
>
> A portion of memory pointed to by ptr is indexed with i * 23 + m,
> where i ranges from 0 to 3, inclusive, and m ranges from 0 to 22,
> inclusive if i < 3, otherwise m ranges from 0 to 20, inclusive.
> So the largest index value we compute equals 89 (3 * 23 + 20).
> Because ptr points to something that is at least 90 bytes large,
> 89 is a valid index and no greater index will be ever computed.
But we do need to get rid of this smatch warning, otherwise it will pollute the
list of smatch warnings.
I was looking at the code and wonder if it wouldn't make more sense to
move writing to rkprobs->intra_mode[i].uv_mode[] into a separate for loop:
for (i = 0; i < ARRAY_SIZE(v4l2_vp9_kf_uv_mode_prob); i++)
rkprobs->intra_mode[i / 23].uv_mode[i % 23] = v4l2_vp9_kf_uv_mode_prob[i];
Wouldn't that do the same as the current code? It looks simpler as well.
Regards,
Hans
Hi,
W dniu 17.11.2021 o 10:59, Hans Verkuil pisze:
> On 16/11/2021 14:14, Andrzej Pietrasiewicz wrote:
>> Hi,
>>
>> W dniu 16.11.2021 o 09:21, Hans Verkuil pisze:
>>> On 16/11/2021 09:09, Andrzej Pietrasiewicz wrote:
>>>> Hi Hans,
>>>>
>>>> W dniu 15.11.2021 o 22:16, Hans Verkuil pisze:
>>>>> On 15/11/2021 18:14, Andrzej Pietrasiewicz wrote:
>>>>>> Hi Hans,
>>>>>>
>>>>>> W dniu 15.11.2021 o 16:07, Hans Verkuil pisze:
>>>>>>> Andrzej,
>>>>>>>
>>>>>>> Can you rebase this series on top of the master branch of
>>>>>>> https://git.linuxtv.org/media_stage.git/ ? Unfortunately this v7 no longer
>>>>>>> applies. Specifically "rkvdec: Add the VP9 backend" failed in a non-trivial
>>>>>>> manner.
>>>>>>
>>>>>> This is a branch for you:
>>>>>>
>>>>>> https://gitlab.collabora.com/linux/for-upstream/-/tree/vp9-uapi
>>>>>
>>>>> I'm getting a bunch of sparse/smatch warnings:
>>>>>
>>>>
>>>> Thanks for finding this, I will re-create the branch and let you know on irc.
>>>> Some of the below are "false positives, namely:
>>>>
>>>> drivers/media/platform/omap3isp/omap3isp.h
>>>> drivers/media/platform/qcom/venus/core.h
>>>
>>> Ah, sorry, I though I had filtered those out. Obviously you can ignore those.
>>>
>>> Please post a v8. That way the series is archived on lore. And it works better
>>> with patchwork.
>>
>> Sure, no problem. Also please see below.
>>
>>>
>>> Regards,
>>>
>>> Hans
>>>
>>>>
>>>> which are not touched by the series.
>>>>
>>>> Regards,
>>>>
>>>> Andrzej
>>>>
>>>>> sparse:
>>>>> rkvdec/rkvdec-vp9.c:190:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
>>>>> rkvdec/rkvdec-vp9.c:245:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
>>>>> SPARSE:hantro/hantro_postproc.c hantro/hantro_postproc.c:37:35: warning: symbol 'hantro_g1_postproc_regs' was not declared. Should it be static?
>>>>>
>>>>> smatch:
>>>>> rkvdec/rkvdec-vp9.c:190:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
>>>>> rkvdec/rkvdec-vp9.c:245:43: warning: variable 'dec_params' set but not used [-Wunused-but-set-variable]
>>>>> rkvdec/rkvdec-vp9.c: rkvdec/rkvdec-vp9.c:236 init_intra_only_probs() error: buffer overflow 'ptr' 90 <= 91
>>
>> this looks a false positive.
>>
>> A portion of memory pointed to by ptr is indexed with i * 23 + m,
>> where i ranges from 0 to 3, inclusive, and m ranges from 0 to 22,
>> inclusive if i < 3, otherwise m ranges from 0 to 20, inclusive.
>> So the largest index value we compute equals 89 (3 * 23 + 20).
>> Because ptr points to something that is at least 90 bytes large,
>> 89 is a valid index and no greater index will be ever computed.
>
> But we do need to get rid of this smatch warning, otherwise it will pollute the
> list of smatch warnings.
>
> I was looking at the code and wonder if it wouldn't make more sense to
> move writing to rkprobs->intra_mode[i].uv_mode[] into a separate for loop:
>
> for (i = 0; i < ARRAY_SIZE(v4l2_vp9_kf_uv_mode_prob); i++)
> rkprobs->intra_mode[i / 23].uv_mode[i % 23] = v4l2_vp9_kf_uv_mode_prob[i];
>
> Wouldn't that do the same as the current code? It looks simpler as well.
>
I think it would, but I would slightly change the loop:
for (i = 0; i < ARRAY_SIZE(v4l2_vp9_kf_uv_mode_prob); i++) {
const u8 *ptr = (const u8 *)v4l2_vp9_kf_uv_mode_prob;
rkprobs->intra_mode[i / 23].uv_mode[i % 23] = ptr[i];
}
because v4l2_vp9_kf_uv_mode_prob is actually a u8[10][9].
I will make such a change locally and test whether it causes regressions.
Once I confirm it works (and I expect I will) would you like me to post a v9,
only reply to the changed patch with its updated version or do you want to make
this change yourself?
Andrzej
Hi again,
W dniu 17.11.2021 o 11:49, Andrzej Pietrasiewicz pisze:
> Hi,
>
> W dniu 17.11.2021 o 10:59, Hans Verkuil pisze:
>> On 16/11/2021 14:14, Andrzej Pietrasiewicz wrote:
>>> Hi,
>>>
>>> W dniu 16.11.2021 o 09:21, Hans Verkuil pisze:
>>>> On 16/11/2021 09:09, Andrzej Pietrasiewicz wrote:
>>>>> Hi Hans,
>>>>>
>>>>> W dniu 15.11.2021 o 22:16, Hans Verkuil pisze:
>>>>>> On 15/11/2021 18:14, Andrzej Pietrasiewicz wrote:
>>>>>>> Hi Hans,
>>>>>>>
>>>>>>> W dniu 15.11.2021 o 16:07, Hans Verkuil pisze:
>>>>>>>> Andrzej,
>>>>>>>>
>>>>>>>> Can you rebase this series on top of the master branch of
>>>>>>>> https://git.linuxtv.org/media_stage.git/ ? Unfortunately this v7 no longer
>>>>>>>> applies. Specifically "rkvdec: Add the VP9 backend" failed in a non-trivial
>>>>>>>> manner.
>>>>>>>
>>>>>>> This is a branch for you:
>>>>>>>
>>>>>>> https://gitlab.collabora.com/linux/for-upstream/-/tree/vp9-uapi
>>>>>>
>>>>>> I'm getting a bunch of sparse/smatch warnings:
>>>>>>
>>>>>
>>>>> Thanks for finding this, I will re-create the branch and let you know on irc.
>>>>> Some of the below are "false positives, namely:
>>>>>
>>>>> drivers/media/platform/omap3isp/omap3isp.h
>>>>> drivers/media/platform/qcom/venus/core.h
>>>>
>>>> Ah, sorry, I though I had filtered those out. Obviously you can ignore those.
>>>>
>>>> Please post a v8. That way the series is archived on lore. And it works better
>>>> with patchwork.
>>>
>>> Sure, no problem. Also please see below.
>>>
>>>>
>>>> Regards,
>>>>
>>>> Hans
>>>>
>>>>>
>>>>> which are not touched by the series.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Andrzej
>>>>>
>>>>>> sparse:
>>>>>> rkvdec/rkvdec-vp9.c:190:43: warning: variable 'dec_params' set but not
>>>>>> used [-Wunused-but-set-variable]
>>>>>> rkvdec/rkvdec-vp9.c:245:43: warning: variable 'dec_params' set but not
>>>>>> used [-Wunused-but-set-variable]
>>>>>> SPARSE:hantro/hantro_postproc.c hantro/hantro_postproc.c:37:35: warning:
>>>>>> symbol 'hantro_g1_postproc_regs' was not declared. Should it be static?
>>>>>>
>>>>>> smatch:
>>>>>> rkvdec/rkvdec-vp9.c:190:43: warning: variable 'dec_params' set but not
>>>>>> used [-Wunused-but-set-variable]
>>>>>> rkvdec/rkvdec-vp9.c:245:43: warning: variable 'dec_params' set but not
>>>>>> used [-Wunused-but-set-variable]
>>>>>> rkvdec/rkvdec-vp9.c: rkvdec/rkvdec-vp9.c:236 init_intra_only_probs()
>>>>>> error: buffer overflow 'ptr' 90 <= 91
>>>
>>> this looks a false positive.
>>>
>>> A portion of memory pointed to by ptr is indexed with i * 23 + m,
>>> where i ranges from 0 to 3, inclusive, and m ranges from 0 to 22,
>>> inclusive if i < 3, otherwise m ranges from 0 to 20, inclusive.
>>> So the largest index value we compute equals 89 (3 * 23 + 20).
>>> Because ptr points to something that is at least 90 bytes large,
>>> 89 is a valid index and no greater index will be ever computed.
>>
>> But we do need to get rid of this smatch warning, otherwise it will pollute the
>> list of smatch warnings.
>>
>> I was looking at the code and wonder if it wouldn't make more sense to
>> move writing to rkprobs->intra_mode[i].uv_mode[] into a separate for loop:
>>
>> for (i = 0; i < ARRAY_SIZE(v4l2_vp9_kf_uv_mode_prob); i++)
>> rkprobs->intra_mode[i / 23].uv_mode[i % 23] =
>> v4l2_vp9_kf_uv_mode_prob[i];
>>
>> Wouldn't that do the same as the current code? It looks simpler as well.
>>
>
> I think it would, but I would slightly change the loop:
>
> for (i = 0; i < ARRAY_SIZE(v4l2_vp9_kf_uv_mode_prob); i++) {
actually, sizeof(v4l2_vp9_kf_uv_mode_prob)
> const u8 *ptr = (const u8 *)v4l2_vp9_kf_uv_mode_prob;
>
> rkprobs->intra_mode[i / 23].uv_mode[i % 23] = ptr[i];
> }
>
> because v4l2_vp9_kf_uv_mode_prob is actually a u8[10][9].
>
> I will make such a change locally and test whether it causes regressions.
>
> Once I confirm it works (and I expect I will) would you like me to post a v9,
> only reply to the changed patch with its updated version or do you want to make
> this change yourself?
>
> Andrzej
Hi Hans,
W dniu 17.11.2021 o 11:51, Andrzej Pietrasiewicz pisze:
> Hi again,
>
> W dniu 17.11.2021 o 11:49, Andrzej Pietrasiewicz pisze:
>> Hi,
>>
>> W dniu 17.11.2021 o 10:59, Hans Verkuil pisze:
>>> On 16/11/2021 14:14, Andrzej Pietrasiewicz wrote:
>>>> Hi,
>>>>
>>>> W dniu 16.11.2021 o 09:21, Hans Verkuil pisze:
>>>>> On 16/11/2021 09:09, Andrzej Pietrasiewicz wrote:
>>>>>> Hi Hans,
>>>>>>
>>>>>> W dniu 15.11.2021 o 22:16, Hans Verkuil pisze:
>>>>>>> On 15/11/2021 18:14, Andrzej Pietrasiewicz wrote:
>>>>>>>> Hi Hans,
>>>>>>>>
>>>>>>>> W dniu 15.11.2021 o 16:07, Hans Verkuil pisze:
>>>>>>>>> Andrzej,
>>>>>>>>>
>>>>>>>>> Can you rebase this series on top of the master branch of
>>>>>>>>> https://git.linuxtv.org/media_stage.git/ ? Unfortunately this v7 no longer
>>>>>>>>> applies. Specifically "rkvdec: Add the VP9 backend" failed in a
>>>>>>>>> non-trivial
>>>>>>>>> manner.
>>>>>>>>
>>>>>>>> This is a branch for you:
>>>>>>>>
>>>>>>>> https://gitlab.collabora.com/linux/for-upstream/-/tree/vp9-uapi
>>>>>>>
>>>>>>> I'm getting a bunch of sparse/smatch warnings:
>>>>>>>
>>>>>>
>>>>>> Thanks for finding this, I will re-create the branch and let you know on irc.
>>>>>> Some of the below are "false positives, namely:
>>>>>>
>>>>>> drivers/media/platform/omap3isp/omap3isp.h
>>>>>> drivers/media/platform/qcom/venus/core.h
>>>>>
>>>>> Ah, sorry, I though I had filtered those out. Obviously you can ignore those.
>>>>>
>>>>> Please post a v8. That way the series is archived on lore. And it works better
>>>>> with patchwork.
>>>>
>>>> Sure, no problem. Also please see below.
>>>>
>>>>>
>>>>> Regards,
>>>>>
>>>>> Hans
>>>>>
>>>>>>
>>>>>> which are not touched by the series.
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Andrzej
>>>>>>
>>>>>>> sparse:
>>>>>>> rkvdec/rkvdec-vp9.c:190:43: warning: variable 'dec_params' set but not
>>>>>>> used [-Wunused-but-set-variable]
>>>>>>> rkvdec/rkvdec-vp9.c:245:43: warning: variable 'dec_params' set but not
>>>>>>> used [-Wunused-but-set-variable]
>>>>>>> SPARSE:hantro/hantro_postproc.c hantro/hantro_postproc.c:37:35: warning:
>>>>>>> symbol 'hantro_g1_postproc_regs' was not declared. Should it be static?
>>>>>>>
>>>>>>> smatch:
>>>>>>> rkvdec/rkvdec-vp9.c:190:43: warning: variable 'dec_params' set but not
>>>>>>> used [-Wunused-but-set-variable]
>>>>>>> rkvdec/rkvdec-vp9.c:245:43: warning: variable 'dec_params' set but not
>>>>>>> used [-Wunused-but-set-variable]
>>>>>>> rkvdec/rkvdec-vp9.c: rkvdec/rkvdec-vp9.c:236 init_intra_only_probs()
>>>>>>> error: buffer overflow 'ptr' 90 <= 91
>>>>
>>>> this looks a false positive.
>>>>
>>>> A portion of memory pointed to by ptr is indexed with i * 23 + m,
>>>> where i ranges from 0 to 3, inclusive, and m ranges from 0 to 22,
>>>> inclusive if i < 3, otherwise m ranges from 0 to 20, inclusive.
>>>> So the largest index value we compute equals 89 (3 * 23 + 20).
>>>> Because ptr points to something that is at least 90 bytes large,
>>>> 89 is a valid index and no greater index will be ever computed.
>>>
>>> But we do need to get rid of this smatch warning, otherwise it will pollute the
>>> list of smatch warnings.
>>>
>>> I was looking at the code and wonder if it wouldn't make more sense to
>>> move writing to rkprobs->intra_mode[i].uv_mode[] into a separate for loop:
>>>
>>> for (i = 0; i < ARRAY_SIZE(v4l2_vp9_kf_uv_mode_prob); i++)
>>> rkprobs->intra_mode[i / 23].uv_mode[i % 23] =
>>> v4l2_vp9_kf_uv_mode_prob[i];
>>>
>>> Wouldn't that do the same as the current code? It looks simpler as well.
>>>
>>
>> I think it would, but I would slightly change the loop:
>>
>> for (i = 0; i < ARRAY_SIZE(v4l2_vp9_kf_uv_mode_prob); i++) {
>
> actually, sizeof(v4l2_vp9_kf_uv_mode_prob)
>
>
>
>> const u8 *ptr = (const u8 *)v4l2_vp9_kf_uv_mode_prob;
>>
>> rkprobs->intra_mode[i / 23].uv_mode[i % 23] = ptr[i];
>> }
>>
>> because v4l2_vp9_kf_uv_mode_prob is actually a u8[10][9].
>>
>> I will make such a change locally and test whether it causes regressions.
This worked, no regressions:
for (i = 0; i < sizeof(v4l2_vp9_kf_uv_mode_prob); ++i) {
const u8 *ptr = (const u8 *)v4l2_vp9_kf_uv_mode_prob;
rkprobs->intra_mode[i / 23].uv_mode[i % 23] = ptr[i];
}
Andrzej