This series aims to make HEVC uapi stable and usable for hardware
decoder. HEVC uapi is used by 2 mainlined drivers (Cedrus and Hantro)
and 2 out of the tree drivers (rkvdec and RPI).
After the remarks done on version 2, I have completely reworked to patches
split so changelogs are meaningless. I have also drop "RFC" from the
titles.
In this v3 I do all the changes (new controls, documentation, etc..)
in the staging directory before moving the HEVC uAPI to stable
steps by steps (unlike the big one patch in v2).
At the end fluster tests results on IMX8MQ is 77/147 for HEVC codec.
Benjamin
Benjamin Gaignard (11):
media: uapi: HEVC: Add missing fields in HEVC controls
media: uapi: HEVC: Rename HEVC stateless controls with STATELESS
prefix
media: uapi: HEVC: Add document uAPI structure
media: uapi: HEVC: Define V4L2_CID_STATELESS_HEVC_SLICE_PARAMS as a
dynamic array
media: uapi: Move parsed HEVC pixel format out of staging
media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control
media: uapi: Move the HEVC stateless control type out of staging
media: controls: Log HEVC stateless control in .std_log
media: uapi: Create a dedicated header for Hantro control
media: uapi: HEVC: fix padding in v4l2 control structures
media: uapi: move HEVC stateless controls out of staging
Hans Verkuil (3):
videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY
v4l2-ctrls: add support for dynamically allocated arrays.
vivid: add dynamic array test control
.../userspace-api/media/drivers/hantro.rst | 5 -
.../media/v4l/ext-ctrls-codec-stateless.rst | 831 ++++++++++++++++++
.../media/v4l/ext-ctrls-codec.rst | 780 ----------------
.../media/v4l/pixfmt-compressed.rst | 7 +-
.../media/v4l/vidioc-g-ext-ctrls.rst | 20 +
.../media/v4l/vidioc-queryctrl.rst | 8 +
.../media/videodev2.h.rst.exceptions | 5 +
.../media/test-drivers/vivid/vivid-ctrls.c | 15 +
drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++-
drivers/media/v4l2-core/v4l2-ctrls-core.c | 198 ++++-
drivers/media/v4l2-core/v4l2-ctrls-defs.c | 32 +-
drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +-
drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +-
drivers/staging/media/hantro/hantro_drv.c | 27 +-
drivers/staging/media/hantro/hantro_hevc.c | 8 +-
drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +-
.../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +-
include/media/hevc-ctrls.h | 250 ------
include/media/v4l2-ctrls.h | 48 +-
include/uapi/linux/hantro-media.h | 19 +
include/uapi/linux/v4l2-controls.h | 436 +++++++++
include/uapi/linux/videodev2.h | 13 +
22 files changed, 1686 insertions(+), 1169 deletions(-)
delete mode 100644 include/media/hevc-ctrls.h
create mode 100644 include/uapi/linux/hantro-media.h
--
2.32.0
Hi Benjamin!
Dne petek, 25. februar 2022 ob 17:45:46 CET je Benjamin Gaignard napisal(a):
> This series aims to make HEVC uapi stable and usable for hardware
> decoder. HEVC uapi is used by 2 mainlined drivers (Cedrus and Hantro)
> and 2 out of the tree drivers (rkvdec and RPI).
>
> After the remarks done on version 2, I have completely reworked to patches
> split so changelogs are meaningless. I have also drop "RFC" from the
> titles.
>
> In this v3 I do all the changes (new controls, documentation, etc..)
> in the staging directory before moving the HEVC uAPI to stable
> steps by steps (unlike the big one patch in v2).
>
> At the end fluster tests results on IMX8MQ is 77/147 for HEVC codec.
I'm working on PoC code for V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET on
Cedrus, which should be finished over weekend.
Note: git am complained about trailing spaces. Run scripts/checkpatch.pl --
strict on patches before sending next round.
Best regards,
Jernej
>
> Benjamin
>
> Benjamin Gaignard (11):
> media: uapi: HEVC: Add missing fields in HEVC controls
> media: uapi: HEVC: Rename HEVC stateless controls with STATELESS
> prefix
> media: uapi: HEVC: Add document uAPI structure
> media: uapi: HEVC: Define V4L2_CID_STATELESS_HEVC_SLICE_PARAMS as a
> dynamic array
> media: uapi: Move parsed HEVC pixel format out of staging
> media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control
> media: uapi: Move the HEVC stateless control type out of staging
> media: controls: Log HEVC stateless control in .std_log
> media: uapi: Create a dedicated header for Hantro control
> media: uapi: HEVC: fix padding in v4l2 control structures
> media: uapi: move HEVC stateless controls out of staging
>
> Hans Verkuil (3):
> videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY
> v4l2-ctrls: add support for dynamically allocated arrays.
> vivid: add dynamic array test control
>
> .../userspace-api/media/drivers/hantro.rst | 5 -
> .../media/v4l/ext-ctrls-codec-stateless.rst | 831 ++++++++++++++++++
> .../media/v4l/ext-ctrls-codec.rst | 780 ----------------
> .../media/v4l/pixfmt-compressed.rst | 7 +-
> .../media/v4l/vidioc-g-ext-ctrls.rst | 20 +
> .../media/v4l/vidioc-queryctrl.rst | 8 +
> .../media/videodev2.h.rst.exceptions | 5 +
> .../media/test-drivers/vivid/vivid-ctrls.c | 15 +
> drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++-
> drivers/media/v4l2-core/v4l2-ctrls-core.c | 198 ++++-
> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 32 +-
> drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +-
> drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +-
> drivers/staging/media/hantro/hantro_drv.c | 27 +-
> drivers/staging/media/hantro/hantro_hevc.c | 8 +-
> drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +-
> .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +-
> include/media/hevc-ctrls.h | 250 ------
> include/media/v4l2-ctrls.h | 48 +-
> include/uapi/linux/hantro-media.h | 19 +
> include/uapi/linux/v4l2-controls.h | 436 +++++++++
> include/uapi/linux/videodev2.h | 13 +
> 22 files changed, 1686 insertions(+), 1169 deletions(-)
> delete mode 100644 include/media/hevc-ctrls.h
> create mode 100644 include/uapi/linux/hantro-media.h
>
> --
> 2.32.0
>
>
From: Hans Verkuil <[email protected]>
Implement support for dynamically allocated arrays.
Most of the changes concern keeping track of the number of elements
of the array and the number of elements allocated for the array and
reallocating memory if needed.
Signed-off-by: Hans Verkuil <[email protected]>
---
drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++++++++---
drivers/media/v4l2-core/v4l2-ctrls-core.c | 182 +++++++++++++++----
drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +-
drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +-
include/media/v4l2-ctrls.h | 42 ++++-
5 files changed, 272 insertions(+), 71 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-api.c b/drivers/media/v4l2-core/v4l2-ctrls-api.c
index db9baa0bd05f..50d012ba3c02 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-api.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-api.c
@@ -97,29 +97,47 @@ static int def_to_user(struct v4l2_ext_control *c, struct v4l2_ctrl *ctrl)
return ptr_to_user(c, ctrl, ctrl->p_new);
}
-/* Helper function: copy the caller-provider value to the given control value */
-static int user_to_ptr(struct v4l2_ext_control *c,
- struct v4l2_ctrl *ctrl,
- union v4l2_ctrl_ptr ptr)
+/* Helper function: copy the caller-provider value as the new control value */
+static int user_to_new(struct v4l2_ext_control *c, struct v4l2_ctrl *ctrl)
{
int ret;
u32 size;
- ctrl->is_new = 1;
+ ctrl->is_new = 0;
+ if (ctrl->is_dyn_array &&
+ c->size > ctrl->p_dyn_alloc_elems * ctrl->elem_size) {
+ void *old = ctrl->p_dyn;
+ void *tmp = kvzalloc(2 * c->size, GFP_KERNEL);
+
+ if (!tmp)
+ return -ENOMEM;
+ memcpy(tmp, ctrl->p_new.p, ctrl->elems * ctrl->elem_size);
+ memcpy(tmp + c->size, ctrl->p_cur.p, ctrl->elems * ctrl->elem_size);
+ ctrl->p_new.p = tmp;
+ ctrl->p_cur.p = tmp + c->size;
+ ctrl->p_dyn = tmp;
+ ctrl->p_dyn_alloc_elems = c->size / ctrl->elem_size;
+ kvfree(old);
+ }
+
if (ctrl->is_ptr && !ctrl->is_string) {
+ unsigned int elems = c->size / ctrl->elem_size;
unsigned int idx;
- ret = copy_from_user(ptr.p, c->ptr, c->size) ? -EFAULT : 0;
- if (ret || !ctrl->is_array)
- return ret;
- for (idx = c->size / ctrl->elem_size; idx < ctrl->elems; idx++)
- ctrl->type_ops->init(ctrl, idx, ptr);
+ if (copy_from_user(ctrl->p_new.p, c->ptr, c->size))
+ return -EFAULT;
+ ctrl->is_new = 1;
+ if (ctrl->is_dyn_array)
+ ctrl->new_elems = elems;
+ else if (ctrl->is_array)
+ for (idx = elems; idx < ctrl->elems; idx++)
+ ctrl->type_ops->init(ctrl, idx, ctrl->p_new);
return 0;
}
switch (ctrl->type) {
case V4L2_CTRL_TYPE_INTEGER64:
- *ptr.p_s64 = c->value64;
+ *ctrl->p_new.p_s64 = c->value64;
break;
case V4L2_CTRL_TYPE_STRING:
size = c->size;
@@ -127,32 +145,27 @@ static int user_to_ptr(struct v4l2_ext_control *c,
return -ERANGE;
if (size > ctrl->maximum + 1)
size = ctrl->maximum + 1;
- ret = copy_from_user(ptr.p_char, c->string, size) ? -EFAULT : 0;
+ ret = copy_from_user(ctrl->p_new.p_char, c->string, size) ? -EFAULT : 0;
if (!ret) {
- char last = ptr.p_char[size - 1];
+ char last = ctrl->p_new.p_char[size - 1];
- ptr.p_char[size - 1] = 0;
+ ctrl->p_new.p_char[size - 1] = 0;
/*
* If the string was longer than ctrl->maximum,
* then return an error.
*/
- if (strlen(ptr.p_char) == ctrl->maximum && last)
+ if (strlen(ctrl->p_new.p_char) == ctrl->maximum && last)
return -ERANGE;
}
return ret;
default:
- *ptr.p_s32 = c->value;
+ *ctrl->p_new.p_s32 = c->value;
break;
}
+ ctrl->is_new = 1;
return 0;
}
-/* Helper function: copy the caller-provider value as the new control value */
-static int user_to_new(struct v4l2_ext_control *c, struct v4l2_ctrl *ctrl)
-{
- return user_to_ptr(c, ctrl, ctrl->p_new);
-}
-
/*
* VIDIOC_G/TRY/S_EXT_CTRLS implementation
*/
@@ -254,7 +267,31 @@ static int prepare_ext_ctrls(struct v4l2_ctrl_handler *hdl,
have_clusters = true;
if (ctrl->cluster[0] != ctrl)
ref = find_ref_lock(hdl, ctrl->cluster[0]->id);
- if (ctrl->is_ptr && !ctrl->is_string) {
+ if (ctrl->is_dyn_array) {
+ unsigned int max_size = ctrl->dims[0] * ctrl->elem_size;
+ unsigned int tot_size = ctrl->elem_size;
+
+ if (cs->which == V4L2_CTRL_WHICH_REQUEST_VAL)
+ tot_size *= ref->p_req_elems;
+ else
+ tot_size *= ctrl->elems;
+
+ c->size = ctrl->elem_size * (c->size / ctrl->elem_size);
+ if (get) {
+ if (c->size < tot_size) {
+ c->size = tot_size;
+ return -ENOSPC;
+ }
+ c->size = tot_size;
+ } else {
+ if (c->size > max_size) {
+ c->size = max_size;
+ return -ENOSPC;
+ }
+ if (!c->size)
+ return -EFAULT;
+ }
+ } else if (ctrl->is_ptr && !ctrl->is_string) {
unsigned int tot_size = ctrl->elems * ctrl->elem_size;
if (c->size < tot_size) {
@@ -346,7 +383,7 @@ static int class_check(struct v4l2_ctrl_handler *hdl, u32 which)
*
* Note that v4l2_g_ext_ctrls_common() with 'which' set to
* V4L2_CTRL_WHICH_REQUEST_VAL is only called if the request was
- * completed, and in that case valid_p_req is true for all controls.
+ * completed, and in that case p_req_valid is true for all controls.
*/
int v4l2_g_ext_ctrls_common(struct v4l2_ctrl_handler *hdl,
struct v4l2_ext_controls *cs,
@@ -430,7 +467,9 @@ int v4l2_g_ext_ctrls_common(struct v4l2_ctrl_handler *hdl,
if (is_default)
ret = def_to_user(cs->controls + idx, ref->ctrl);
- else if (is_request && ref->valid_p_req)
+ else if (is_request && ref->p_req_dyn_enomem)
+ ret = -ENOMEM;
+ else if (is_request && ref->p_req_valid)
ret = req_to_user(cs->controls + idx, ref);
else if (is_volatile)
ret = new_to_user(cs->controls + idx, ref->ctrl);
@@ -457,6 +496,17 @@ int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl, struct video_device *vdev,
}
EXPORT_SYMBOL(v4l2_g_ext_ctrls);
+/* Validate a new control */
+static int validate_new(const struct v4l2_ctrl *ctrl, union v4l2_ctrl_ptr p_new)
+{
+ unsigned int idx;
+ int err = 0;
+
+ for (idx = 0; !err && idx < ctrl->new_elems; idx++)
+ err = ctrl->type_ops->validate(ctrl, idx, p_new);
+ return err;
+}
+
/* Validate controls. */
static int validate_ctrls(struct v4l2_ext_controls *cs,
struct v4l2_ctrl_helper *helpers,
@@ -872,6 +922,9 @@ int __v4l2_ctrl_s_ctrl_compound(struct v4l2_ctrl *ctrl,
/* It's a driver bug if this happens. */
if (WARN_ON(ctrl->type != type))
return -EINVAL;
+ /* Setting dynamic arrays is not (yet?) supported. */
+ if (WARN_ON(ctrl->is_dyn_array))
+ return -EINVAL;
memcpy(ctrl->p_new.p, p, ctrl->elems * ctrl->elem_size);
return set_ctrl(NULL, ctrl, 0);
}
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c
index 85c2d3f39d96..2d24cb8d3926 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
@@ -973,11 +973,12 @@ EXPORT_SYMBOL(v4l2_ctrl_notify);
/* Copy the one value to another. */
static void ptr_to_ptr(struct v4l2_ctrl *ctrl,
- union v4l2_ctrl_ptr from, union v4l2_ctrl_ptr to)
+ union v4l2_ctrl_ptr from, union v4l2_ctrl_ptr to,
+ unsigned int elems)
{
if (ctrl == NULL)
return;
- memcpy(to.p, from.p_const, ctrl->elems * ctrl->elem_size);
+ memcpy(to.p, from.p_const, elems * ctrl->elem_size);
}
/* Copy the new value to the current value. */
@@ -990,8 +991,11 @@ void new_to_cur(struct v4l2_fh *fh, struct v4l2_ctrl *ctrl, u32 ch_flags)
/* has_changed is set by cluster_changed */
changed = ctrl->has_changed;
- if (changed)
- ptr_to_ptr(ctrl, ctrl->p_new, ctrl->p_cur);
+ if (changed) {
+ if (ctrl->is_dyn_array)
+ ctrl->elems = ctrl->new_elems;
+ ptr_to_ptr(ctrl, ctrl->p_new, ctrl->p_cur, ctrl->elems);
+ }
if (ch_flags & V4L2_EVENT_CTRL_CH_FLAGS) {
/* Note: CH_FLAGS is only set for auto clusters. */
@@ -1021,36 +1025,122 @@ void cur_to_new(struct v4l2_ctrl *ctrl)
{
if (ctrl == NULL)
return;
- ptr_to_ptr(ctrl, ctrl->p_cur, ctrl->p_new);
+ if (ctrl->is_dyn_array)
+ ctrl->new_elems = ctrl->elems;
+ ptr_to_ptr(ctrl, ctrl->p_cur, ctrl->p_new, ctrl->new_elems);
+}
+
+static bool req_alloc_dyn_array(struct v4l2_ctrl_ref *ref, u32 elems)
+{
+ void *tmp;
+
+ if (elems < ref->p_req_dyn_alloc_elems)
+ return true;
+
+ tmp = kvmalloc(elems * ref->ctrl->elem_size, GFP_KERNEL);
+
+ if (!tmp) {
+ ref->p_req_dyn_enomem = true;
+ return false;
+ }
+ ref->p_req_dyn_enomem = false;
+ kvfree(ref->p_req.p);
+ ref->p_req.p = tmp;
+ ref->p_req_dyn_alloc_elems = elems;
+ return true;
}
/* Copy the new value to the request value */
void new_to_req(struct v4l2_ctrl_ref *ref)
{
+ struct v4l2_ctrl *ctrl;
+
if (!ref)
return;
- ptr_to_ptr(ref->ctrl, ref->ctrl->p_new, ref->p_req);
- ref->valid_p_req = true;
+
+ ctrl = ref->ctrl;
+ if (ctrl->is_dyn_array && !req_alloc_dyn_array(ref, ctrl->new_elems))
+ return;
+
+ ref->p_req_elems = ctrl->new_elems;
+ ptr_to_ptr(ctrl, ctrl->p_new, ref->p_req, ref->p_req_elems);
+ ref->p_req_valid = true;
}
/* Copy the current value to the request value */
void cur_to_req(struct v4l2_ctrl_ref *ref)
{
+ struct v4l2_ctrl *ctrl;
+
if (!ref)
return;
- ptr_to_ptr(ref->ctrl, ref->ctrl->p_cur, ref->p_req);
- ref->valid_p_req = true;
+
+ ctrl = ref->ctrl;
+ if (ctrl->is_dyn_array && !req_alloc_dyn_array(ref, ctrl->elems))
+ return;
+
+ ref->p_req_elems = ctrl->elems;
+ ptr_to_ptr(ctrl, ctrl->p_cur, ref->p_req, ctrl->elems);
+ ref->p_req_valid = true;
}
/* Copy the request value to the new value */
-void req_to_new(struct v4l2_ctrl_ref *ref)
+int req_to_new(struct v4l2_ctrl_ref *ref)
{
+ struct v4l2_ctrl *ctrl;
+
if (!ref)
- return;
- if (ref->valid_p_req)
- ptr_to_ptr(ref->ctrl, ref->p_req, ref->ctrl->p_new);
- else
- ptr_to_ptr(ref->ctrl, ref->ctrl->p_cur, ref->ctrl->p_new);
+ return 0;
+
+ ctrl = ref->ctrl;
+
+ /*
+ * This control was never set in the request, so just use the current
+ * value.
+ */
+ if (!ref->p_req_valid) {
+ if (ctrl->is_dyn_array)
+ ctrl->new_elems = ctrl->elems;
+ ptr_to_ptr(ctrl, ctrl->p_cur, ctrl->p_new, ctrl->new_elems);
+ return 0;
+ }
+
+ /* Not a dynamic array, so just copy the request value */
+ if (!ctrl->is_dyn_array) {
+ ptr_to_ptr(ctrl, ref->p_req, ctrl->p_new, ctrl->new_elems);
+ return 0;
+ }
+
+ /* Sanity check, should never happen */
+ if (WARN_ON(!ref->p_req_dyn_alloc_elems))
+ return -ENOMEM;
+
+ /*
+ * Check if the number of elements in the request is more than the
+ * elements in ctrl->p_dyn. If so, attempt to realloc ctrl->p_dyn.
+ * Note that p_dyn is allocated with twice the number of elements
+ * in the dynamic array since it has to store both the current and
+ * new value of such a control.
+ */
+ if (ref->p_req_elems > ctrl->p_dyn_alloc_elems) {
+ unsigned int sz = ref->p_req_elems * ctrl->elem_size;
+ void *old = ctrl->p_dyn;
+ void *tmp = kvzalloc(2 * sz, GFP_KERNEL);
+
+ if (!tmp)
+ return -ENOMEM;
+ memcpy(tmp, ctrl->p_new.p, ctrl->elems * ctrl->elem_size);
+ memcpy(tmp + sz, ctrl->p_cur.p, ctrl->elems * ctrl->elem_size);
+ ctrl->p_new.p = tmp;
+ ctrl->p_cur.p = tmp + sz;
+ ctrl->p_dyn = tmp;
+ ctrl->p_dyn_alloc_elems = ref->p_req_elems;
+ kvfree(old);
+ }
+
+ ctrl->new_elems = ref->p_req_elems;
+ ptr_to_ptr(ctrl, ref->p_req, ctrl->p_new, ctrl->new_elems);
+ return 0;
}
/* Control range checking */
@@ -1092,17 +1182,6 @@ int check_range(enum v4l2_ctrl_type type,
}
}
-/* Validate a new control */
-int validate_new(const struct v4l2_ctrl *ctrl, union v4l2_ctrl_ptr p_new)
-{
- unsigned idx;
- int err = 0;
-
- for (idx = 0; !err && idx < ctrl->elems; idx++)
- err = ctrl->type_ops->validate(ctrl, idx, p_new);
- return err;
-}
-
/* Set the handler's error code if it wasn't set earlier already */
static inline int handler_set_err(struct v4l2_ctrl_handler *hdl, int err)
{
@@ -1147,6 +1226,8 @@ void v4l2_ctrl_handler_free(struct v4l2_ctrl_handler *hdl)
/* Free all nodes */
list_for_each_entry_safe(ref, next_ref, &hdl->ctrl_refs, node) {
list_del(&ref->node);
+ if (ref->p_req_dyn_alloc_elems)
+ kvfree(ref->p_req.p);
kfree(ref);
}
/* Free all controls owned by the handler */
@@ -1154,6 +1235,7 @@ void v4l2_ctrl_handler_free(struct v4l2_ctrl_handler *hdl)
list_del(&ctrl->node);
list_for_each_entry_safe(sev, next_sev, &ctrl->ev_subs, node)
list_del(&sev->node);
+ kvfree(ctrl->p_dyn);
kvfree(ctrl);
}
kvfree(hdl->buckets);
@@ -1269,7 +1351,7 @@ int handler_new_ref(struct v4l2_ctrl_handler *hdl,
if (hdl->error)
return hdl->error;
- if (allocate_req)
+ if (allocate_req && !ctrl->is_dyn_array)
size_extra_req = ctrl->elems * ctrl->elem_size;
new_ref = kzalloc(sizeof(*new_ref) + size_extra_req, GFP_KERNEL);
if (!new_ref)
@@ -1443,7 +1525,6 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
elem_size = sizeof(s32);
break;
}
- tot_ctrl_size = elem_size * elems;
/* Sanity checks */
if (id == 0 || name == NULL || !elem_size ||
@@ -1464,17 +1545,33 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
handler_set_err(hdl, -EINVAL);
return NULL;
}
+ if (flags & V4L2_CTRL_FLAG_DYNAMIC_ARRAY) {
+ /*
+ * For now only support this for one-dimensional arrays only.
+ *
+ * This can be relaxed in the future, but this will
+ * require more effort.
+ */
+ if (nr_of_dims != 1) {
+ handler_set_err(hdl, -EINVAL);
+ return NULL;
+ }
+ /* Start with just 1 element */
+ elems = 1;
+ }
+ tot_ctrl_size = elem_size * elems;
sz_extra = 0;
if (type == V4L2_CTRL_TYPE_BUTTON)
flags |= V4L2_CTRL_FLAG_WRITE_ONLY |
V4L2_CTRL_FLAG_EXECUTE_ON_WRITE;
else if (type == V4L2_CTRL_TYPE_CTRL_CLASS)
flags |= V4L2_CTRL_FLAG_READ_ONLY;
- else if (type == V4L2_CTRL_TYPE_INTEGER64 ||
- type == V4L2_CTRL_TYPE_STRING ||
- type >= V4L2_CTRL_COMPOUND_TYPES ||
- is_array)
+ else if (!(flags & V4L2_CTRL_FLAG_DYNAMIC_ARRAY) &&
+ (type == V4L2_CTRL_TYPE_INTEGER64 ||
+ type == V4L2_CTRL_TYPE_STRING ||
+ type >= V4L2_CTRL_COMPOUND_TYPES ||
+ is_array))
sz_extra += 2 * tot_ctrl_size;
if (type >= V4L2_CTRL_COMPOUND_TYPES && p_def.p_const)
@@ -1503,7 +1600,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
ctrl->is_ptr = is_array || type >= V4L2_CTRL_COMPOUND_TYPES || ctrl->is_string;
ctrl->is_int = !ctrl->is_ptr && type != V4L2_CTRL_TYPE_INTEGER64;
ctrl->is_array = is_array;
+ ctrl->is_dyn_array = !!(flags & V4L2_CTRL_FLAG_DYNAMIC_ARRAY);
ctrl->elems = elems;
+ ctrl->new_elems = elems;
ctrl->nr_of_dims = nr_of_dims;
if (nr_of_dims)
memcpy(ctrl->dims, dims, nr_of_dims * sizeof(dims[0]));
@@ -1516,6 +1615,16 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
ctrl->cur.val = ctrl->val = def;
data = &ctrl[1];
+ if (ctrl->is_dyn_array) {
+ ctrl->p_dyn_alloc_elems = elems;
+ ctrl->p_dyn = kvzalloc(2 * elems * elem_size, GFP_KERNEL);
+ if (!ctrl->p_dyn) {
+ kvfree(ctrl);
+ return NULL;
+ }
+ data = ctrl->p_dyn;
+ }
+
if (!ctrl->is_int) {
ctrl->p_new.p = data;
ctrl->p_cur.p = data + tot_ctrl_size;
@@ -1525,7 +1634,10 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
}
if (type >= V4L2_CTRL_COMPOUND_TYPES && p_def.p_const) {
- ctrl->p_def.p = ctrl->p_cur.p + tot_ctrl_size;
+ if (ctrl->is_dyn_array)
+ ctrl->p_def.p = &ctrl[1];
+ else
+ ctrl->p_def.p = ctrl->p_cur.p + tot_ctrl_size;
memcpy(ctrl->p_def.p, p_def.p_const, elem_size);
}
@@ -1535,6 +1647,7 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
}
if (handler_new_ref(hdl, ctrl, NULL, false, false)) {
+ kvfree(ctrl->p_dyn);
kvfree(ctrl);
return NULL;
}
@@ -1872,6 +1985,9 @@ static int cluster_changed(struct v4l2_ctrl *master)
continue;
}
+ if (ctrl->elems != ctrl->new_elems)
+ ctrl_changed = true;
+
for (idx = 0; !ctrl_changed && idx < ctrl->elems; idx++)
ctrl_changed = !ctrl->type_ops->equal(ctrl, idx,
ctrl->p_cur, ctrl->p_new);
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-priv.h b/drivers/media/v4l2-core/v4l2-ctrls-priv.h
index d4bf2c716f97..aba6176fab6c 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-priv.h
+++ b/drivers/media/v4l2-core/v4l2-ctrls-priv.h
@@ -57,10 +57,9 @@ void cur_to_new(struct v4l2_ctrl *ctrl);
void cur_to_req(struct v4l2_ctrl_ref *ref);
void new_to_cur(struct v4l2_fh *fh, struct v4l2_ctrl *ctrl, u32 ch_flags);
void new_to_req(struct v4l2_ctrl_ref *ref);
-void req_to_new(struct v4l2_ctrl_ref *ref);
+int req_to_new(struct v4l2_ctrl_ref *ref);
void send_initial_event(struct v4l2_fh *fh, struct v4l2_ctrl *ctrl);
void send_event(struct v4l2_fh *fh, struct v4l2_ctrl *ctrl, u32 changes);
-int validate_new(const struct v4l2_ctrl *ctrl, union v4l2_ctrl_ptr p_new);
int handler_new_ref(struct v4l2_ctrl_handler *hdl,
struct v4l2_ctrl *ctrl,
struct v4l2_ctrl_ref **ctrl_ref,
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-request.c b/drivers/media/v4l2-core/v4l2-ctrls-request.c
index 7d098f287fd9..c637049d7a2b 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-request.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-request.c
@@ -143,7 +143,7 @@ v4l2_ctrl_request_hdl_ctrl_find(struct v4l2_ctrl_handler *hdl, u32 id)
{
struct v4l2_ctrl_ref *ref = find_ref_lock(hdl, id);
- return (ref && ref->valid_p_req) ? ref->ctrl : NULL;
+ return (ref && ref->p_req_valid) ? ref->ctrl : NULL;
}
EXPORT_SYMBOL_GPL(v4l2_ctrl_request_hdl_ctrl_find);
@@ -373,7 +373,7 @@ void v4l2_ctrl_request_complete(struct media_request *req,
v4l2_ctrl_unlock(master);
continue;
}
- if (ref->valid_p_req)
+ if (ref->p_req_valid)
continue;
/* Copy the current control value into the request */
@@ -442,7 +442,7 @@ int v4l2_ctrl_request_setup(struct media_request *req,
struct v4l2_ctrl_ref *r =
find_ref(hdl, master->cluster[i]->id);
- if (r->valid_p_req) {
+ if (r->p_req_valid) {
have_new_data = true;
break;
}
@@ -458,7 +458,11 @@ int v4l2_ctrl_request_setup(struct media_request *req,
struct v4l2_ctrl_ref *r =
find_ref(hdl, master->cluster[i]->id);
- req_to_new(r);
+ ret = req_to_new(r);
+ if (ret) {
+ v4l2_ctrl_unlock(master);
+ goto error;
+ }
master->cluster[i]->is_new = 1;
r->req_done = true;
}
@@ -490,6 +494,7 @@ int v4l2_ctrl_request_setup(struct media_request *req,
break;
}
+error:
media_request_object_put(obj);
return ret;
}
diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
index b3ce438f1329..f4105de8a8d2 100644
--- a/include/media/v4l2-ctrls.h
+++ b/include/media/v4l2-ctrls.h
@@ -185,6 +185,8 @@ typedef void (*v4l2_ctrl_notify_fnc)(struct v4l2_ctrl *ctrl, void *priv);
* and/or has type %V4L2_CTRL_TYPE_STRING. In other words, &struct
* v4l2_ext_control uses field p to point to the data.
* @is_array: If set, then this control contains an N-dimensional array.
+ * @is_dyn_array: If set, then this control contains a dynamically sized 1-dimensional array.
+ * If this is set, then @is_array is also set.
* @has_volatiles: If set, then one or more members of the cluster are volatile.
* Drivers should never touch this flag.
* @call_notify: If set, then call the handler's notify function whenever the
@@ -205,6 +207,9 @@ typedef void (*v4l2_ctrl_notify_fnc)(struct v4l2_ctrl *ctrl, void *priv);
* @step: The control's step value for non-menu controls.
* @elems: The number of elements in the N-dimensional array.
* @elem_size: The size in bytes of the control.
+ * @new_elems: The number of elements in p_new. This is the same as @elems,
+ * except for dynamic arrays. In that case it is in the range of
+ * 1 to @p_dyn_alloc_elems.
* @dims: The size of each dimension.
* @nr_of_dims:The number of dimensions in @dims.
* @menu_skip_mask: The control's skip mask for menu controls. This makes it
@@ -223,15 +228,21 @@ typedef void (*v4l2_ctrl_notify_fnc)(struct v4l2_ctrl *ctrl, void *priv);
* :math:`ceil(\frac{maximum - minimum}{step}) + 1`.
* Used only if the @type is %V4L2_CTRL_TYPE_INTEGER_MENU.
* @flags: The control's flags.
- * @cur: Structure to store the current value.
- * @cur.val: The control's current value, if the @type is represented via
- * a u32 integer (see &enum v4l2_ctrl_type).
- * @val: The control's new s32 value.
* @priv: The control's private pointer. For use by the driver. It is
* untouched by the control framework. Note that this pointer is
* not freed when the control is deleted. Should this be needed
* then a new internal bitfield can be added to tell the framework
* to free this pointer.
+ * @p_dyn: Pointer to the dynamically allocated array. Only valid if
+ * @is_dyn_array is true.
+ * @p_dyn_alloc_elems: The number of elements in the dynamically allocated
+ * array for both the cur and new values. So @p_dyn is actually
+ * sized for 2 * @p_dyn_alloc_elems * @elem_size. Only valid if
+ * @is_dyn_array is true.
+ * @cur: Structure to store the current value.
+ * @cur.val: The control's current value, if the @type is represented via
+ * a u32 integer (see &enum v4l2_ctrl_type).
+ * @val: The control's new s32 value.
* @p_def: The control's default value represented via a union which
* provides a standard way of accessing control types
* through a pointer (for compound controls only).
@@ -260,6 +271,7 @@ struct v4l2_ctrl {
unsigned int is_string:1;
unsigned int is_ptr:1;
unsigned int is_array:1;
+ unsigned int is_dyn_array:1;
unsigned int has_volatiles:1;
unsigned int call_notify:1;
unsigned int manual_mode_value:8;
@@ -272,6 +284,7 @@ struct v4l2_ctrl {
s64 minimum, maximum, default_value;
u32 elems;
u32 elem_size;
+ u32 new_elems;
u32 dims[V4L2_CTRL_MAX_DIMS];
u32 nr_of_dims;
union {
@@ -284,6 +297,8 @@ struct v4l2_ctrl {
};
unsigned long flags;
void *priv;
+ void *p_dyn;
+ u32 p_dyn_alloc_elems;
s32 val;
struct {
s32 val;
@@ -309,12 +324,22 @@ struct v4l2_ctrl {
* the control has been applied. This prevents applying controls
* from a cluster with multiple controls twice (when the first
* control of a cluster is applied, they all are).
- * @valid_p_req: If set, then p_req contains the control value for the request.
+ * @p_req_valid: If set, then p_req contains the control value for the request.
+ * @p_req_dyn_enomem: If set, then p_req is invalid since allocating space for
+ * a dynamic array failed. Attempting to read this value shall
+ * result in ENOMEM. Only valid if ctrl->is_dyn_array is true.
+ * @p_req_dyn_alloc_elems: The number of elements allocated for the dynamic
+ * array. Only valid if @p_req_valid and ctrl->is_dyn_array are
+ * true.
+ * @p_req_elems: The number of elements in @p_req. This is the same as
+ * ctrl->elems, except for dynamic arrays. In that case it is in
+ * the range of 1 to @p_req_dyn_alloc_elems. Only valid if
+ * @p_req_valid is true.
* @p_req: If the control handler containing this control reference
* is bound to a media request, then this points to the
* value of the control that must be applied when the request
* is executed, or to the value of the control at the time
- * that the request was completed. If @valid_p_req is false,
+ * that the request was completed. If @p_req_valid is false,
* then this control was never set for this request and the
* control will not be updated when this request is applied.
*
@@ -329,7 +354,10 @@ struct v4l2_ctrl_ref {
struct v4l2_ctrl_helper *helper;
bool from_other_dev;
bool req_done;
- bool valid_p_req;
+ bool p_req_valid;
+ bool p_req_dyn_enomem;
+ u32 p_req_dyn_alloc_elems;
+ u32 p_req_elems;
union v4l2_ctrl_ptr p_req;
};
--
2.32.0
The number of 'entry point offset' could be very variable.
Rather than use a large static array define a v4l2 dynamic array
of integer control.
The number of entry point offsets is reported by the elems field.
Signed-off-by: Benjamin Gaignard <[email protected]>
---
.../userspace-api/media/v4l/ext-ctrls-codec.rst | 9 +++++++++
include/media/hevc-ctrls.h | 1 +
2 files changed, 10 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 44a268a948c0..71f7dc1c1ccd 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -3128,6 +3128,15 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
\normalsize
+``V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS (integer)``
+ Specifies the i-th entry point offset in bytes and is represented by
+ offset_len_minus1 plus 1 bits.
+ This control is a dynamically sized array. The number of entry point
+ offsets is reported by the ``elems`` field.
+ This bitstream parameter is defined according to :ref:`hevc`.
+ They are described in section 7.4.7.1 "General slice segment header
+ semantics" of the specification.
+
``V4L2_CID_STATELESS_HEVC_SCALING_MATRIX (struct)``
Specifies the HEVC scaling matrix parameters used for the scaling process
for transform coefficients.
diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
index 3016c1abb1d0..3f8a67924df3 100644
--- a/include/media/hevc-ctrls.h
+++ b/include/media/hevc-ctrls.h
@@ -20,6 +20,7 @@
#define V4L2_CID_STATELESS_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_BASE + 1012)
#define V4L2_CID_STATELESS_HEVC_DECODE_MODE (V4L2_CID_CODEC_BASE + 1015)
#define V4L2_CID_STATELESS_HEVC_START_CODE (V4L2_CID_CODEC_BASE + 1016)
+#define V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET (V4L2_CID_CODEC_BASE + 1017)
/* enum v4l2_ctrl_type type values */
#define V4L2_CTRL_TYPE_HEVC_SPS 0x0120
--
2.32.0
Change HEVC stateless controls names to V4L2_CID_STATELESS_HEVC instead
of V4L2_CID_MPEG_VIDEO_HEVC be coherent with v4l2 naming convention.
Signed-off-by: Benjamin Gaignard <[email protected]>
---
.../media/v4l/ext-ctrls-codec.rst | 26 +++++++--------
drivers/media/v4l2-core/v4l2-ctrls-defs.c | 32 +++++++++----------
drivers/staging/media/hantro/hantro_drv.c | 26 +++++++--------
drivers/staging/media/hantro/hantro_hevc.c | 8 ++---
drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +++++++-------
.../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +++---
include/media/hevc-ctrls.h | 26 +++++++--------
7 files changed, 76 insertions(+), 76 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index b83d45202916..262ad4f3c1f4 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -2639,7 +2639,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
.. _v4l2-mpeg-hevc:
-``V4L2_CID_MPEG_VIDEO_HEVC_SPS (struct)``
+``V4L2_CID_STATELESS_HEVC_SPS (struct)``
Specifies the Sequence Parameter Set fields (as extracted from the
bitstream) for the associated HEVC slice data.
These bitstream parameters are defined according to :ref:`hevc`.
@@ -2792,7 +2792,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
\normalsize
-``V4L2_CID_MPEG_VIDEO_HEVC_PPS (struct)``
+``V4L2_CID_STATELESS_HEVC_PPS (struct)``
Specifies the Picture Parameter Set fields (as extracted from the
bitstream) for the associated HEVC slice data.
These bitstream parameters are defined according to :ref:`hevc`.
@@ -2945,7 +2945,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
\normalsize
-``V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS (struct)``
+``V4L2_CID_STATELESS_HEVC_SLICE_PARAMS (struct)``
Specifies various slice-specific parameters, especially from the NAL unit
header, general slice segment header and weighted prediction parameter
parts of the bitstream.
@@ -3110,7 +3110,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
\normalsize
-``V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX (struct)``
+``V4L2_CID_STATELESS_HEVC_SCALING_MATRIX (struct)``
Specifies the HEVC scaling matrix parameters used for the scaling process
for transform coefficients.
These matrix and parameters are defined according to :ref:`hevc`.
@@ -3260,7 +3260,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
\normalsize
-``V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE (enum)``
+``V4L2_CID_STATELESS_HEVC_DECODE_MODE (enum)``
Specifies the decoding mode to use. Currently exposes slice-based and
frame-based decoding but new modes might be added later on.
This control is used as a modifier for V4L2_PIX_FMT_HEVC_SLICE
@@ -3275,7 +3275,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
This menu control is not yet part of the public kernel API and
it is expected to change.
-.. c:type:: v4l2_mpeg_video_hevc_decode_mode
+.. c:type:: v4l2_stateless_hevc_decode_mode
.. raw:: latex
@@ -3288,11 +3288,11 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
:stub-columns: 0
:widths: 1 1 2
- * - ``V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_SLICE_BASED``
+ * - ``V4L2_STATELESS_HEVC_DECODE_MODE_SLICE_BASED``
- 0
- Decoding is done at the slice granularity.
The OUTPUT buffer must contain a single slice.
- * - ``V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED``
+ * - ``V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED``
- 1
- Decoding is done at the frame granularity.
The OUTPUT buffer must contain all slices needed to decode the
@@ -3302,7 +3302,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
\normalsize
-``V4L2_CID_MPEG_VIDEO_HEVC_START_CODE (enum)``
+``V4L2_CID_STATELESS_HEVC_START_CODE (enum)``
Specifies the HEVC slice start code expected for each slice.
This control is used as a modifier for V4L2_PIX_FMT_HEVC_SLICE
pixel format. Applications that support V4L2_PIX_FMT_HEVC_SLICE
@@ -3316,7 +3316,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
This menu control is not yet part of the public kernel API and
it is expected to change.
-.. c:type:: v4l2_mpeg_video_hevc_start_code
+.. c:type:: v4l2_stateless_hevc_start_code
.. tabularcolumns:: |p{9.2cm}|p{0.6cm}|p{7.5cm}|
@@ -3325,13 +3325,13 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
:stub-columns: 0
:widths: 1 1 2
- * - ``V4L2_MPEG_VIDEO_HEVC_START_CODE_NONE``
+ * - ``V4L2_STATELESS_HEVC_START_CODE_NONE``
- 0
- Selecting this value specifies that HEVC slices are passed
to the driver without any start code. The bitstream data should be
according to :ref:`hevc` 7.3.1.1 General NAL unit syntax, hence
contains emulation prevention bytes when required.
- * - ``V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B``
+ * - ``V4L2_STATELESS_HEVC_START_CODE_ANNEX_B``
- 1
- Selecting this value specifies that HEVC slices are expected
to be prefixed by Annex B start codes. According to :ref:`hevc`
@@ -3364,7 +3364,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
This provides a bitmask which consists of bits [0, LTR_COUNT-1].
This is applicable to the H264 and HEVC encoders.
-``V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS (struct)``
+``V4L2_CID_STATELESS_HEVC_DECODE_PARAMS (struct)``
Specifies various decode parameters, especially the references picture order
count (POC) for all the lists (short, long, before, current, after) and the
number of entries for each of them.
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index 54ca4e6b820b..00cb08083514 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -699,9 +699,9 @@ const char * const *v4l2_ctrl_get_menu(u32 id)
return hevc_tier;
case V4L2_CID_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE:
return hevc_loop_filter_mode;
- case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE:
+ case V4L2_CID_STATELESS_HEVC_DECODE_MODE:
return hevc_decode_mode;
- case V4L2_CID_MPEG_VIDEO_HEVC_START_CODE:
+ case V4L2_CID_STATELESS_HEVC_START_CODE:
return hevc_start_code;
case V4L2_CID_CAMERA_ORIENTATION:
return camera_orientation;
@@ -995,13 +995,13 @@ const char *v4l2_ctrl_get_name(u32 id)
case V4L2_CID_MPEG_VIDEO_HEVC_SIZE_OF_LENGTH_FIELD: return "HEVC Size of Length Field";
case V4L2_CID_MPEG_VIDEO_REF_NUMBER_FOR_PFRAMES: return "Reference Frames for a P-Frame";
case V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR: return "Prepend SPS and PPS to IDR";
- case V4L2_CID_MPEG_VIDEO_HEVC_SPS: return "HEVC Sequence Parameter Set";
- case V4L2_CID_MPEG_VIDEO_HEVC_PPS: return "HEVC Picture Parameter Set";
- case V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS: return "HEVC Slice Parameters";
- case V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX: return "HEVC Scaling Matrix";
- case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS: return "HEVC Decode Parameters";
- case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE: return "HEVC Decode Mode";
- case V4L2_CID_MPEG_VIDEO_HEVC_START_CODE: return "HEVC Start Code";
+ case V4L2_CID_STATELESS_HEVC_SPS: return "HEVC Sequence Parameter Set";
+ case V4L2_CID_STATELESS_HEVC_PPS: return "HEVC Picture Parameter Set";
+ case V4L2_CID_STATELESS_HEVC_SLICE_PARAMS: return "HEVC Slice Parameters";
+ case V4L2_CID_STATELESS_HEVC_SCALING_MATRIX: return "HEVC Scaling Matrix";
+ case V4L2_CID_STATELESS_HEVC_DECODE_PARAMS: return "HEVC Decode Parameters";
+ case V4L2_CID_STATELESS_HEVC_DECODE_MODE: return "HEVC Decode Mode";
+ case V4L2_CID_STATELESS_HEVC_START_CODE: return "HEVC Start Code";
/* CAMERA controls */
/* Keep the order of the 'case's the same as in v4l2-controls.h! */
@@ -1355,8 +1355,8 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
case V4L2_CID_MPEG_VIDEO_HEVC_SIZE_OF_LENGTH_FIELD:
case V4L2_CID_MPEG_VIDEO_HEVC_TIER:
case V4L2_CID_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE:
- case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE:
- case V4L2_CID_MPEG_VIDEO_HEVC_START_CODE:
+ case V4L2_CID_STATELESS_HEVC_DECODE_MODE:
+ case V4L2_CID_STATELESS_HEVC_START_CODE:
case V4L2_CID_STATELESS_H264_DECODE_MODE:
case V4L2_CID_STATELESS_H264_START_CODE:
case V4L2_CID_CAMERA_ORIENTATION:
@@ -1493,19 +1493,19 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
case V4L2_CID_STATELESS_VP8_FRAME:
*type = V4L2_CTRL_TYPE_VP8_FRAME;
break;
- case V4L2_CID_MPEG_VIDEO_HEVC_SPS:
+ case V4L2_CID_STATELESS_HEVC_SPS:
*type = V4L2_CTRL_TYPE_HEVC_SPS;
break;
- case V4L2_CID_MPEG_VIDEO_HEVC_PPS:
+ case V4L2_CID_STATELESS_HEVC_PPS:
*type = V4L2_CTRL_TYPE_HEVC_PPS;
break;
- case V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS:
+ case V4L2_CID_STATELESS_HEVC_SLICE_PARAMS:
*type = V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS;
break;
- case V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX:
+ case V4L2_CID_STATELESS_HEVC_SCALING_MATRIX:
*type = V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX;
break;
- case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS:
+ case V4L2_CID_STATELESS_HEVC_DECODE_PARAMS:
*type = V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS;
break;
case V4L2_CID_STATELESS_VP9_COMPRESSED_HDR:
diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
index dc768884cb79..6f58c259d8fc 100644
--- a/drivers/staging/media/hantro/hantro_drv.c
+++ b/drivers/staging/media/hantro/hantro_drv.c
@@ -255,7 +255,7 @@ static int hantro_try_ctrl(struct v4l2_ctrl *ctrl)
if (sps->bit_depth_luma_minus8 != 0)
/* Only 8-bit is supported */
return -EINVAL;
- } else if (ctrl->id == V4L2_CID_MPEG_VIDEO_HEVC_SPS) {
+ } else if (ctrl->id == V4L2_CID_STATELESS_HEVC_SPS) {
const struct v4l2_ctrl_hevc_sps *sps = ctrl->p_new.p_hevc_sps;
if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
@@ -428,18 +428,18 @@ static const struct hantro_ctrl controls[] = {
}, {
.codec = HANTRO_HEVC_DECODER,
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE,
- .min = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
- .max = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
- .def = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
+ .id = V4L2_CID_STATELESS_HEVC_DECODE_MODE,
+ .min = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
+ .max = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
+ .def = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
},
}, {
.codec = HANTRO_HEVC_DECODER,
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_START_CODE,
- .min = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
- .max = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
- .def = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
+ .id = V4L2_CID_STATELESS_HEVC_START_CODE,
+ .min = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
+ .max = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
+ .def = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
},
}, {
.codec = HANTRO_HEVC_DECODER,
@@ -459,23 +459,23 @@ static const struct hantro_ctrl controls[] = {
}, {
.codec = HANTRO_HEVC_DECODER,
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_SPS,
+ .id = V4L2_CID_STATELESS_HEVC_SPS,
.ops = &hantro_ctrl_ops,
},
}, {
.codec = HANTRO_HEVC_DECODER,
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_PPS,
+ .id = V4L2_CID_STATELESS_HEVC_PPS,
},
}, {
.codec = HANTRO_HEVC_DECODER,
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS,
+ .id = V4L2_CID_STATELESS_HEVC_DECODE_PARAMS,
},
}, {
.codec = HANTRO_HEVC_DECODER,
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX,
+ .id = V4L2_CID_STATELESS_HEVC_SCALING_MATRIX,
},
}, {
.codec = HANTRO_HEVC_DECODER,
diff --git a/drivers/staging/media/hantro/hantro_hevc.c b/drivers/staging/media/hantro/hantro_hevc.c
index b49a41d7ae91..b6ec86d03d91 100644
--- a/drivers/staging/media/hantro/hantro_hevc.c
+++ b/drivers/staging/media/hantro/hantro_hevc.c
@@ -201,22 +201,22 @@ int hantro_hevc_dec_prepare_run(struct hantro_ctx *ctx)
hantro_start_prepare_run(ctx);
ctrls->decode_params =
- hantro_get_ctrl(ctx, V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS);
+ hantro_get_ctrl(ctx, V4L2_CID_STATELESS_HEVC_DECODE_PARAMS);
if (WARN_ON(!ctrls->decode_params))
return -EINVAL;
ctrls->scaling =
- hantro_get_ctrl(ctx, V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX);
+ hantro_get_ctrl(ctx, V4L2_CID_STATELESS_HEVC_SCALING_MATRIX);
if (WARN_ON(!ctrls->scaling))
return -EINVAL;
ctrls->sps =
- hantro_get_ctrl(ctx, V4L2_CID_MPEG_VIDEO_HEVC_SPS);
+ hantro_get_ctrl(ctx, V4L2_CID_STATELESS_HEVC_SPS);
if (WARN_ON(!ctrls->sps))
return -EINVAL;
ctrls->pps =
- hantro_get_ctrl(ctx, V4L2_CID_MPEG_VIDEO_HEVC_PPS);
+ hantro_get_ctrl(ctx, V4L2_CID_STATELESS_HEVC_PPS);
if (WARN_ON(!ctrls->pps))
return -EINVAL;
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c b/drivers/staging/media/sunxi/cedrus/cedrus.c
index 4a4b714b0f26..e0428163f82c 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus.c
@@ -42,7 +42,7 @@ static int cedrus_try_ctrl(struct v4l2_ctrl *ctrl)
if (sps->bit_depth_luma_minus8 != 0)
/* Only 8-bit is supported */
return -EINVAL;
- } else if (ctrl->id == V4L2_CID_MPEG_VIDEO_HEVC_SPS) {
+ } else if (ctrl->id == V4L2_CID_STATELESS_HEVC_SPS) {
const struct v4l2_ctrl_hevc_sps *sps = ctrl->p_new.p_hevc_sps;
struct cedrus_ctx *ctx = container_of(ctrl->handler, struct cedrus_ctx, hdl);
@@ -164,42 +164,42 @@ static const struct cedrus_control cedrus_controls[] = {
},
{
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_SPS,
+ .id = V4L2_CID_STATELESS_HEVC_SPS,
.ops = &cedrus_ctrl_ops,
},
.codec = CEDRUS_CODEC_H265,
},
{
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_PPS,
+ .id = V4L2_CID_STATELESS_HEVC_PPS,
},
.codec = CEDRUS_CODEC_H265,
},
{
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS,
+ .id = V4L2_CID_STATELESS_HEVC_SLICE_PARAMS,
},
.codec = CEDRUS_CODEC_H265,
},
{
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX,
+ .id = V4L2_CID_STATELESS_HEVC_SCALING_MATRIX,
},
.codec = CEDRUS_CODEC_H265,
},
{
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE,
- .max = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_SLICE_BASED,
- .def = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_SLICE_BASED,
+ .id = V4L2_CID_STATELESS_HEVC_DECODE_MODE,
+ .max = V4L2_STATELESS_HEVC_DECODE_MODE_SLICE_BASED,
+ .def = V4L2_STATELESS_HEVC_DECODE_MODE_SLICE_BASED,
},
.codec = CEDRUS_CODEC_H265,
},
{
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_START_CODE,
- .max = V4L2_MPEG_VIDEO_HEVC_START_CODE_NONE,
- .def = V4L2_MPEG_VIDEO_HEVC_START_CODE_NONE,
+ .id = V4L2_CID_STATELESS_HEVC_START_CODE,
+ .max = V4L2_STATELESS_HEVC_START_CODE_NONE,
+ .def = V4L2_STATELESS_HEVC_START_CODE_NONE,
},
.codec = CEDRUS_CODEC_H265,
},
@@ -211,7 +211,7 @@ static const struct cedrus_control cedrus_controls[] = {
},
{
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS,
+ .id = V4L2_CID_STATELESS_HEVC_DECODE_PARAMS,
},
.codec = CEDRUS_CODEC_H265,
},
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
index a16c1422558f..f6be4ae72ee2 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
@@ -65,15 +65,15 @@ void cedrus_device_run(void *priv)
case V4L2_PIX_FMT_HEVC_SLICE:
run.h265.sps = cedrus_find_control_data(ctx,
- V4L2_CID_MPEG_VIDEO_HEVC_SPS);
+ V4L2_CID_STATELESS_HEVC_SPS);
run.h265.pps = cedrus_find_control_data(ctx,
- V4L2_CID_MPEG_VIDEO_HEVC_PPS);
+ V4L2_CID_STATELESS_HEVC_PPS);
run.h265.slice_params = cedrus_find_control_data(ctx,
- V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS);
+ V4L2_CID_STATELESS_HEVC_SLICE_PARAMS);
run.h265.decode_params = cedrus_find_control_data(ctx,
- V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS);
+ V4L2_CID_STATELESS_HEVC_DECODE_PARAMS);
run.h265.scaling_matrix = cedrus_find_control_data(ctx,
- V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX);
+ V4L2_CID_STATELESS_HEVC_SCALING_MATRIX);
break;
case V4L2_PIX_FMT_VP8_FRAME:
diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
index a329e086a89a..b3540167df9e 100644
--- a/include/media/hevc-ctrls.h
+++ b/include/media/hevc-ctrls.h
@@ -16,13 +16,13 @@
/* The pixel format isn't stable at the moment and will likely be renamed. */
#define V4L2_PIX_FMT_HEVC_SLICE v4l2_fourcc('S', '2', '6', '5') /* HEVC parsed slices */
-#define V4L2_CID_MPEG_VIDEO_HEVC_SPS (V4L2_CID_CODEC_BASE + 1008)
-#define V4L2_CID_MPEG_VIDEO_HEVC_PPS (V4L2_CID_CODEC_BASE + 1009)
-#define V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS (V4L2_CID_CODEC_BASE + 1010)
-#define V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX (V4L2_CID_CODEC_BASE + 1011)
-#define V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_BASE + 1012)
-#define V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE (V4L2_CID_CODEC_BASE + 1015)
-#define V4L2_CID_MPEG_VIDEO_HEVC_START_CODE (V4L2_CID_CODEC_BASE + 1016)
+#define V4L2_CID_STATELESS_HEVC_SPS (V4L2_CID_CODEC_BASE + 1008)
+#define V4L2_CID_STATELESS_HEVC_PPS (V4L2_CID_CODEC_BASE + 1009)
+#define V4L2_CID_STATELESS_HEVC_SLICE_PARAMS (V4L2_CID_CODEC_BASE + 1010)
+#define V4L2_CID_STATELESS_HEVC_SCALING_MATRIX (V4L2_CID_CODEC_BASE + 1011)
+#define V4L2_CID_STATELESS_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_BASE + 1012)
+#define V4L2_CID_STATELESS_HEVC_DECODE_MODE (V4L2_CID_CODEC_BASE + 1015)
+#define V4L2_CID_STATELESS_HEVC_START_CODE (V4L2_CID_CODEC_BASE + 1016)
/* enum v4l2_ctrl_type type values */
#define V4L2_CTRL_TYPE_HEVC_SPS 0x0120
@@ -31,14 +31,14 @@
#define V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX 0x0123
#define V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS 0x0124
-enum v4l2_mpeg_video_hevc_decode_mode {
- V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_SLICE_BASED,
- V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
+enum v4l2_stateless_hevc_decode_mode {
+ V4L2_STATELESS_HEVC_DECODE_MODE_SLICE_BASED,
+ V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
};
-enum v4l2_mpeg_video_hevc_start_code {
- V4L2_MPEG_VIDEO_HEVC_START_CODE_NONE,
- V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
+enum v4l2_stateless_hevc_start_code {
+ V4L2_STATELESS_HEVC_START_CODE_NONE,
+ V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
};
#define V4L2_HEVC_SLICE_TYPE_B 0
--
2.32.0
Complete the HEVC controls with missing fields from H.265 specifications.
Even if these fields aren't used by the current mainlined drivers
they will be need for (at least) rkvdec driver.
Signed-off-by: Benjamin Gaignard <[email protected]>
---
.../media/v4l/ext-ctrls-codec.rst | 22 +++++++++++++++++++
include/media/hevc-ctrls.h | 6 ++++-
2 files changed, 27 insertions(+), 1 deletion(-)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 4cd7c541fc30..b83d45202916 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -2661,6 +2661,16 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
:stub-columns: 0
:widths: 1 1 2
+ * - __u8
+ - ``video_parameter_set_id``
+ - Specifies the value of the vps_video_parameter_set_id of the active VPS
+ as descibed in section "7.4.3.2.1 General sequence parameter set RBSP semantics"
+ of H.265 specifications.
+ * - __u8
+ - ``seq_parameter_set_id``
+ - Provides an identifier for the SPS for reference by other syntax elements
+ as descibed in section "7.4.3.2.1 General sequence parameter set RBSP semantics"
+ of H.265 specifications.
* - __u16
- ``pic_width_in_luma_samples``
-
@@ -2800,6 +2810,9 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
:stub-columns: 0
:widths: 1 1 2
+ * - __u8
+ - ``pic_parameter_set_id``
+ - Identifies the PPS for reference by other syntax elements.
* - __u8
- ``num_extra_slice_header_bits``
-
@@ -3026,6 +3039,15 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
* - __u8
- ``ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
- The list of L1 reference elements as indices in the DPB.
+ * - __u16
+ - ``short_term_ref_pic_set_size``
+ - Specifies the number of st_ref_pic_set( ) syntax structures included in the SPS.
+ The value of num_short_term_ref_pic_sets shall be in the range of 0 to 64, inclusive.
+ * - __u16
+ - ``long_term_ref_pic_set_size``
+ - Specifies the number of candidate long-term reference pictures that are specified
+ in the SPS. The value of num_long_term_ref_pics_sps shall be in the range
+ of 0 to 32, inclusive.
* - __u8
- ``padding``
- Applications and drivers must set this to zero.
diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
index 01ccda48d8c5..a329e086a89a 100644
--- a/include/media/hevc-ctrls.h
+++ b/include/media/hevc-ctrls.h
@@ -58,6 +58,8 @@ enum v4l2_mpeg_video_hevc_start_code {
/* The controls are not stable at the moment and will likely be reworked. */
struct v4l2_ctrl_hevc_sps {
/* ISO/IEC 23008-2, ITU-T Rec. H.265: Sequence parameter set */
+ __u8 video_parameter_set_id;
+ __u8 seq_parameter_set_id;
__u16 pic_width_in_luma_samples;
__u16 pic_height_in_luma_samples;
__u8 bit_depth_luma_minus8;
@@ -108,6 +110,7 @@ struct v4l2_ctrl_hevc_sps {
struct v4l2_ctrl_hevc_pps {
/* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture parameter set */
+ __u8 pic_parameter_set_id;
__u8 num_extra_slice_header_bits;
__u8 num_ref_idx_l0_default_active_minus1;
__u8 num_ref_idx_l1_default_active_minus1;
@@ -199,7 +202,8 @@ struct v4l2_ctrl_hevc_slice_params {
__u32 slice_segment_addr;
__u8 ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
__u8 ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
-
+ __u16 short_term_ref_pic_set_size;
+ __u16 long_term_ref_pic_set_size;
__u8 padding;
/* ISO/IEC 23008-2, ITU-T Rec. H.265: Weighted prediction parameter */
--
2.32.0
Hantro hardware needs a dedicated v4l2 control to perform HEVC
decoding. Move it to a dedicated header file to prepare HEVC
uAPI un-staging.
Signed-off-by: Benjamin Gaignard <[email protected]>
---
.../userspace-api/media/drivers/hantro.rst | 5 -----
drivers/staging/media/hantro/hantro_drv.c | 1 +
include/media/hevc-ctrls.h | 13 -------------
include/uapi/linux/hantro-media.h | 19 +++++++++++++++++++
include/uapi/linux/v4l2-controls.h | 5 +++++
5 files changed, 25 insertions(+), 18 deletions(-)
create mode 100644 include/uapi/linux/hantro-media.h
diff --git a/Documentation/userspace-api/media/drivers/hantro.rst b/Documentation/userspace-api/media/drivers/hantro.rst
index cd9754b4e005..78dcd2a44a03 100644
--- a/Documentation/userspace-api/media/drivers/hantro.rst
+++ b/Documentation/userspace-api/media/drivers/hantro.rst
@@ -12,8 +12,3 @@ The Hantro video decoder driver implements the following driver-specific control
to before syntax element "slice_temporal_mvp_enabled_flag".
If IDR, the skipped bits are just "pic_output_flag"
(separate_colour_plane_flag is not supported).
-
-.. note::
-
- This control is not yet part of the public kernel API and
- it is expected to change.
diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
index 6f58c259d8fc..783a92a38c8e 100644
--- a/drivers/staging/media/hantro/hantro_drv.c
+++ b/drivers/staging/media/hantro/hantro_drv.c
@@ -11,6 +11,7 @@
*/
#include <linux/clk.h>
+#include <linux/hantro-media.h>
#include <linux/module.h>
#include <linux/of.h>
#include <linux/platform_device.h>
diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
index cbf41d180fe9..f29298c8c2f8 100644
--- a/include/media/hevc-ctrls.h
+++ b/include/media/hevc-ctrls.h
@@ -443,17 +443,4 @@ struct v4l2_ctrl_hevc_scaling_matrix {
__u8 scaling_list_dc_coef_32x32[2];
};
-/* MPEG-class control IDs specific to the Hantro driver as defined by V4L2 */
-#define V4L2_CID_CODEC_HANTRO_BASE (V4L2_CTRL_CLASS_CODEC | 0x1200)
-/*
- * V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP -
- * the number of data (in bits) to skip in the
- * slice segment header.
- * If non-IDR, the bits to be skipped go from syntax element "pic_output_flag"
- * to before syntax element "slice_temporal_mvp_enabled_flag".
- * If IDR, the skipped bits are just "pic_output_flag"
- * (separate_colour_plane_flag is not supported).
- */
-#define V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP (V4L2_CID_CODEC_HANTRO_BASE + 0)
-
#endif
diff --git a/include/uapi/linux/hantro-media.h b/include/uapi/linux/hantro-media.h
new file mode 100644
index 000000000000..b4e7ab7e6452
--- /dev/null
+++ b/include/uapi/linux/hantro-media.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#ifndef __UAPI_HANTRO_MEDIA_H__
+#define __UAPI_HANTRO_MEDIA_H__
+
+#include <linux/v4l2-controls.h>
+
+/*
+ * V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP -
+ * the number of data (in bits) to skip in the
+ * slice segment header.
+ * If non-IDR, the bits to be skipped go from syntax element "pic_output_flag"
+ * to before syntax element "slice_temporal_mvp_enabled_flag".
+ * If IDR, the skipped bits are just "pic_output_flag"
+ * (separate_colour_plane_flag is not supported).
+ */
+#define V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP (V4L2_CID_USER_HANTRO_BASE + 0)
+
+#endif /* __UAPI_HANTRO_MEDIA_H__ */
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index c8e0f84d204d..d56a1c7fb91d 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -218,6 +218,11 @@ enum v4l2_colorfx {
* We reserve 16 controls for this driver.
*/
#define V4L2_CID_USER_ALLEGRO_BASE (V4L2_CID_USER_BASE + 0x1170)
+/*
+ * The base for Hantro driver controls.
+ * We reserve 128 controls for this driver.
+ */
+#define V4L2_CID_USER_HANTRO_BASE (V4L2_CID_USER_BASE + 0x1180)
/* MPEG-class control IDs */
/* The MPEG controls are applicable to all codec controls
--
2.32.0
Make explicit that V4L2_CID_STATELESS_HEVC_SLICE_PARAMS control is
a dynamic array control type.
Some drivers may be able to receive multiple slices in one control
to improve decoding performance.
Signed-off-by: Benjamin Gaignard <[email protected]>
---
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst | 2 ++
include/media/hevc-ctrls.h | 3 +++
2 files changed, 5 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 18625d18ddf3..44a268a948c0 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -2964,6 +2964,8 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
These bitstream parameters are defined according to :ref:`hevc`.
They are described in section 7.4.7 "General slice segment header
semantics" of the specification.
+ This control is a dynamically sized 1-dimensional array,
+ V4L2_CTRL_FLAG_DYNAMIC_ARRAY flag must be set when using it.
.. c:type:: v4l2_ctrl_hevc_slice_params
diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
index 1c86566cd7cb..057b5032453b 100644
--- a/include/media/hevc-ctrls.h
+++ b/include/media/hevc-ctrls.h
@@ -300,6 +300,9 @@ struct v4l2_hevc_pred_weight_table {
/**
* v4l2_ctrl_hevc_slice_params - HEVC slice parameters
*
+ * This control is a dynamically sized 1-dimensional array,
+ * V4L2_CTRL_FLAG_DYNAMIC_ARRAY flag must be set when using it.
+ *
* @bit_size: size (in bits) of the current slice data
* @data_bit_offset: offset (in bits) to the video data in the current slice data
* @nal_unit_type: specifies the coding type of the slice (B, P or I)
--
2.32.0
Add kernel-doc documentation for all the HEVC structures.
Signed-off-by: Benjamin Gaignard <[email protected]>
---
.../media/v4l/ext-ctrls-codec.rst | 156 +++++++------
include/media/hevc-ctrls.h | 217 +++++++++++++++++-
2 files changed, 304 insertions(+), 69 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 262ad4f3c1f4..18625d18ddf3 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -2673,70 +2673,76 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
of H.265 specifications.
* - __u16
- ``pic_width_in_luma_samples``
- -
+ - Specifies the width of each decoded picture in units of luma samples.
* - __u16
- ``pic_height_in_luma_samples``
- -
+ - Specifies the height of each decoded picture in units of luma samples.
* - __u8
- ``bit_depth_luma_minus8``
- -
+ - Specifies the bit depth of the samples of the luma array.
* - __u8
- ``bit_depth_chroma_minus8``
- -
+ - Specifies the bit depth of the samples of the chroma arrays.
* - __u8
- ``log2_max_pic_order_cnt_lsb_minus4``
- -
+ - Specifies the value of the variable MaxPicOrderCntLsb.
* - __u8
- ``sps_max_dec_pic_buffering_minus1``
- -
+ - Plus 1 specifies the maximum required size of the decoded picture buffer for
+ the CVS
* - __u8
- ``sps_max_num_reorder_pics``
- -
+ - Indicates the maximum allowed number of pictures.
* - __u8
- ``sps_max_latency_increase_plus1``
- -
+ - Not equal to 0 is used to compute the value of SpsMaxLatencyPictures[i].
* - __u8
- ``log2_min_luma_coding_block_size_minus3``
- -
+ - Plus 3 specifies the minimum luma coding block size.
* - __u8
- ``log2_diff_max_min_luma_coding_block_size``
- -
+ - Specifies the difference between the maximum and minimum luma coding block size.
* - __u8
- ``log2_min_luma_transform_block_size_minus2``
- -
+ - Plus 2 specifies the minimum luma transform block size.
* - __u8
- ``log2_diff_max_min_luma_transform_block_size``
- -
+ - Specifies the difference between the maximum and minimum luma transform block size.
* - __u8
- ``max_transform_hierarchy_depth_inter``
- -
+ - Specifies the maximum hierarchy depth for transform units of coding units coded
+ in inter prediction mode.
* - __u8
- ``max_transform_hierarchy_depth_intra``
- -
+ - Specifies the maximum hierarchy depth for transform units of coding units coded in
+ intra prediction mode.
* - __u8
- ``pcm_sample_bit_depth_luma_minus1``
- -
+ - Specifies the number of bits used to represent each of PCM sample values of the
+ luma component.
* - __u8
- ``pcm_sample_bit_depth_chroma_minus1``
- -
+ - Specifies the number of bits used to represent each of PCM sample values of
+ the chroma components.
* - __u8
- ``log2_min_pcm_luma_coding_block_size_minus3``
- -
+ - Plus 3 specifies the minimum size of coding blocks.
* - __u8
- ``log2_diff_max_min_pcm_luma_coding_block_size``
- -
+ - Specifies the difference between the maximum and minimum size of coding blocks.
* - __u8
- ``num_short_term_ref_pic_sets``
- -
+ - Specifies the number of st_ref_pic_set() syntax structures included in the SPS.
* - __u8
- ``num_long_term_ref_pics_sps``
- -
+ - Specifies the number of candidate long-term reference pictures that are
+ specified in the SPS.
* - __u8
- ``chroma_format_idc``
- -
+ - Specifies the chroma sampling.
* - __u8
- ``sps_max_sub_layers_minus1``
- -
+ - plus 1 specifies the maximum number of temporal sub-layers.
* - __u64
- ``flags``
- See :ref:`Sequence Parameter Set Flags <hevc_sps_flags>`
@@ -2815,7 +2821,8 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
- Identifies the PPS for reference by other syntax elements.
* - __u8
- ``num_extra_slice_header_bits``
- -
+ - Specifies the number of extra slice header bits that are present
+ in the slice header RBSP for coded pictures referring to the PPS.
* - __u8
- ``num_ref_idx_l0_default_active_minus1``
- Specifies the inferred value of num_ref_idx_l0_active_minus1
@@ -2824,37 +2831,42 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
- Specifies the inferred value of num_ref_idx_l1_active_minus1
* - __s8
- ``init_qp_minus26``
- -
+ - Plus 26 specifies the initial value of SliceQp Y for each slice
+ referring to the PPS.
* - __u8
- ``diff_cu_qp_delta_depth``
- -
+ - Specifies the difference between the luma coding tree block size
+ and the minimum luma coding block size of coding units that
+ convey cu_qp_delta_abs and cu_qp_delta_sign_flag.
* - __s8
- ``pps_cb_qp_offset``
- -
+ - Specify the offsets to the luma quantization parameter Cb.
* - __s8
- ``pps_cr_qp_offset``
- -
+ - Specify the offsets to the luma quantization parameter Cr.
* - __u8
- ``num_tile_columns_minus1``
- -
+ - Plus 1 specifies the number of tile columns partitioning the picture.
* - __u8
- ``num_tile_rows_minus1``
- -
+ - Plus 1 specifies the number of tile rows partitioning the picture.
* - __u8
- ``column_width_minus1[20]``
- -
+ - Plus 1 specifies the width of the i-th tile column in units of
+ coding tree blocks.
* - __u8
- ``row_height_minus1[22]``
- -
+ - Plus 1 specifies the height of the i-th tile row in units of coding
+ tree blocks.
* - __s8
- ``pps_beta_offset_div2``
- -
+ - Specify the default deblocking parameter offsets for beta divided by 2.
* - __s8
- ``pps_tc_offset_div2``
- -
+ - Specify the default deblocking parameter offsets for tC divided by 2.
* - __u8
- ``log2_parallel_merge_level_minus2``
- -
+ - Plus 2 specifies the value of the variable Log2ParMrgLevel.
* - __u8
- ``padding[4]``
- Applications and drivers must set this to zero.
@@ -2976,10 +2988,10 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
- Offset (in bits) to the video data in the current slice data.
* - __u8
- ``nal_unit_type``
- -
+ - Specifies the coding type of the slice (B, P or I).
* - __u8
- ``nuh_temporal_id_plus1``
- -
+ - Minus 1 specifies a temporal identifier for the NAL unit.
* - __u8
- ``slice_type``
-
@@ -2987,52 +2999,56 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
V4L2_HEVC_SLICE_TYPE_B).
* - __u8
- ``colour_plane_id``
- -
+ - Specifies the colour plane associated with the current slice.
* - __u16
- ``slice_pic_order_cnt``
- -
+ - Specifies the picture order count.
* - __u8
- ``num_ref_idx_l0_active_minus1``
- -
+ - Specifies the maximum reference index for reference picture list 0
+ that may be used to decode the slice.
* - __u8
- ``num_ref_idx_l1_active_minus1``
- -
+ - Specifies the maximum reference index for reference picture list 1
+ that may be used to decode the slice.
* - __u8
- ``collocated_ref_idx``
- -
+ - Specifies the reference index of the collocated picture used for
+ temporal motion vector prediction.
* - __u8
- ``five_minus_max_num_merge_cand``
- -
+ - Specifies the maximum number of merging motion vector prediction
+ candidates supported in the slice subtracted from 5.
* - __s8
- ``slice_qp_delta``
- -
+ - Specifies the initial value of QpY to be used for the coding blocks in the slice.
* - __s8
- ``slice_cb_qp_offset``
- -
+ - Specifies a difference to be added to the value of pps_cb_qp_offset.
* - __s8
- ``slice_cr_qp_offset``
- -
+ - Specifies a difference to be added to the value of pps_cr_qp_offset.
* - __s8
- ``slice_act_y_qp_offset``
- -
+ - screen content extension parameters
* - __s8
- ``slice_act_cb_qp_offset``
- -
+ - screen content extension parameters
* - __s8
- ``slice_act_cr_qp_offset``
- -
+ - screen content extension parameters
* - __s8
- ``slice_beta_offset_div2``
- -
+ - Specify the deblocking parameter offsets for beta divided by 2.
* - __s8
- ``slice_tc_offset_div2``
- -
+ - Specify the deblocking parameter offsets for tC divided by 2.
* - __u8
- ``pic_struct``
- -
+ - Indicates whether a picture should be displayed as a frame or as one or more fields.
* - __u32
- ``slice_segment_addr``
- -
+ - Specifies the address of the first coding tree block in the slice segment.
* - __u8
- ``ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
- The list of L0 reference elements as indices in the DPB.
@@ -3222,36 +3238,44 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
:stub-columns: 0
:widths: 1 1 2
- * - __u8
- - ``luma_log2_weight_denom``
- -
- * - __s8
- - ``delta_chroma_log2_weight_denom``
- -
* - __s8
- ``delta_luma_weight_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
- -
+ - The difference of the weighting factor applied to the luma
+ prediction value for list 0.
* - __s8
- ``luma_offset_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
- -
+ - The additive offset applied to the luma prediction value for list 0.
* - __s8
- ``delta_chroma_weight_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2]``
- -
+ - The difference of the weighting factor applied to the chroma
+ prediction value for list 0.
* - __s8
- ``chroma_offset_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2]``
- -
+ - The difference of the additive offset applied to the chroma
+ prediction values for list 0.
* - __s8
- ``delta_luma_weight_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
- -
+ - The difference of the weighting factor applied to the luma
+ prediction value for list 1.
* - __s8
- ``luma_offset_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
- -
+ - The additive offset applied to the luma prediction value for list 1.
* - __s8
- ``delta_chroma_weight_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2]``
- -
+ - The difference of the weighting factor applied to the chroma
+ prediction value for list 1.
* - __s8
- ``chroma_offset_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2]``
- -
+ - The difference of the additive offset applied to the chroma
+ prediction values for list 1.
+ * - __u8
+ - ``luma_log2_weight_denom``
+ - The base 2 logarithm of the denominator for all luma weighting
+ factors.
+ * - __s8
+ - ``delta_chroma_log2_weight_denom``
+ - The difference of the base 2 logarithm of the denominator for
+ all chroma weighting factors.
* - __u8
- ``padding[6]``
- Applications and drivers must set this to zero.
diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
index b3540167df9e..1c86566cd7cb 100644
--- a/include/media/hevc-ctrls.h
+++ b/include/media/hevc-ctrls.h
@@ -55,9 +55,68 @@ enum v4l2_stateless_hevc_start_code {
#define V4L2_HEVC_SPS_FLAG_SPS_TEMPORAL_MVP_ENABLED (1ULL << 7)
#define V4L2_HEVC_SPS_FLAG_STRONG_INTRA_SMOOTHING_ENABLED (1ULL << 8)
-/* The controls are not stable at the moment and will likely be reworked. */
+/**
+ * struct v4l2_ctrl_hevc_sps - ITU-T Rec. H.265: Sequence parameter set
+ *
+ * @video_parameter_set_id: specifies the value of the
+ * vps_video_parameter_set_id of the active VPS
+ * @seq_parameter_set_id: Provides an identifier for the SPS for
+ * reference by other syntax elements
+ * @pic_width_in_luma_samples: specifies the width of each decoded picture
+ * in units of luma samples
+ * @pic_height_in_luma_samples: specifies the height of each decoded picture
+ * in units of luma samples
+ * @bit_depth_luma_minus8: specifies the bit depth of the samples of the
+ * luma array
+ * @bit_depth_chroma_minus8: specifies the bit depth of the samples of the
+ * chroma arrays
+ * @log2_max_pic_order_cnt_lsb_minus4: specifies the value of the variable
+ * MaxPicOrderCntLsb
+ * @sps_max_dec_pic_buffering_minus1: plus 1 specifies the maximum required
+ * size of the decoded picture buffer for
+ * the CVS
+ * @sps_max_num_reorder_pics: indicates the maximum allowed number of pictures
+ * @sps_max_latency_increase_plus1: not equal to 0 is used to compute the
+ * value of SpsMaxLatencyPictures[i]
+ * @log2_min_luma_coding_block_size_minus3: plus 3 specifies the minimum
+ * luma coding block size
+ * @log2_diff_max_min_luma_coding_block_size: specifies the difference between
+ * the maximum and minimum luma
+ * coding block size
+ * @log2_min_luma_transform_block_size_minus2: plus 2 specifies the minimum luma
+ * transform block size
+ * @log2_diff_max_min_luma_transform_block_size: specifies the difference between
+ * the maximum and minimum luma
+ * transform block size
+ * @max_transform_hierarchy_depth_inter: specifies the maximum hierarchy
+ * depth for transform units of
+ * coding units coded in inter
+ * prediction mode
+ * @max_transform_hierarchy_depth_intra: specifies the maximum hierarchy
+ * depth for transform units of
+ * coding units coded in intra
+ * prediction mode
+ * @pcm_sample_bit_depth_luma_minus1: specifies the number of bits used to
+ * represent each of PCM sample values of the
+ * luma component
+ * @pcm_sample_bit_depth_chroma_minus1: specifies the number of bits used to
+ * represent each of PCM sample values of
+ * the chroma components
+ * @log2_min_pcm_luma_coding_block_size_minus3: plus 3 specifies the minimum size
+ * of coding blocks
+ * @log2_diff_max_min_pcm_luma_coding_block_size: specifies the difference between
+ * the maximum and minimum size of
+ * coding blocks
+ * @num_short_term_ref_pic_sets: speciœfies the number of st_ref_pic_set()
+ * syntax structures included in the SPS
+ * @num_long_term_ref_pics_sps: specifies the number of candidate long-term
+ * reference pictures that are specified in the SPS
+ * @chroma_format_idc: specifies the chroma sampling
+ * @sps_max_sub_layers_minus1: plus 1 specifies the maximum number of temporal
+ * sub-layers
+ * @flags: see V4L2_HEVC_SPS_FLAG_{}
+ */
struct v4l2_ctrl_hevc_sps {
- /* ISO/IEC 23008-2, ITU-T Rec. H.265: Sequence parameter set */
__u8 video_parameter_set_id;
__u8 seq_parameter_set_id;
__u16 pic_width_in_luma_samples;
@@ -108,8 +167,43 @@ struct v4l2_ctrl_hevc_sps {
#define V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT (1ULL << 19)
#define V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING (1ULL << 20)
+/**
+ * struct v4l2_ctrl_hevc_pps - ITU-T Rec. H.265: Picture parameter set
+ *
+ * @pic_parameter_set_id: identifies the PPS for reference by other
+ * syntax elements
+ * @num_extra_slice_header_bits: specifies the number of extra slice header
+ * bits that are present in the slice header RBSP
+ * for coded pictures referring to the PPS.
+ * @num_ref_idx_l0_default_active_minus1: specifies the inferred value of
+ * num_ref_idx_l0_active_minus1
+ * @num_ref_idx_l1_default_active_minus1: specifies the inferred value of
+ * num_ref_idx_l1_active_minus1
+ * @init_qp_minus26: plus 26 specifies the initial value of SliceQp Y for
+ * each slice referring to the PPS
+ * @diff_cu_qp_delta_depth: specifies the difference between the luma coding
+ * tree block size and the minimum luma coding block
+ * size of coding units that convey cu_qp_delta_abs
+ * and cu_qp_delta_sign_flag
+ * @pps_cb_qp_offset: specify the offsets to the luma quantization parameter Cb
+ * @pps_cr_qp_offset: specify the offsets to the luma quantization parameter Cr
+ * @num_tile_columns_minus1: plus 1 specifies the number of tile columns
+ * partitioning the picture
+ * @num_tile_rows_minus1: plus 1 specifies the number of tile rows partitioning
+ * the picture
+ * @column_width_minus1: plus 1 specifies the width of the i-th tile column in
+ * units of coding tree blocks
+ * @row_height_minus1: plus 1 specifies the height of the i-th tile row in
+ * units of coding tree blocks
+ * @pps_beta_offset_div2: specify the default deblocking parameter offsets for
+ * beta divided by 2
+ * @pps_tc_offset_div2: specify the default deblocking parameter offsets for tC
+ * divided by 2
+ * @log2_parallel_merge_level_minus2: plus 2 specifies the value of the variable
+ * Log2ParMrgLevel
+ * @flags: see V4L2_HEVC_PPS_FLAG_{}
+ */
struct v4l2_ctrl_hevc_pps {
- /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture parameter set */
__u8 pic_parameter_set_id;
__u8 num_extra_slice_header_bits;
__u8 num_ref_idx_l0_default_active_minus1;
@@ -134,6 +228,14 @@ struct v4l2_ctrl_hevc_pps {
#define V4L2_HEVC_DPB_ENTRIES_NUM_MAX 16
+/**
+ * struct v4l2_hevc_dpb_entry - HEVC decoded picture buffer entry
+ *
+ * @timestamp: timestamp of the V4L2 capture buffer to use as reference.
+ * @flags: long term flag for the reference frame
+ * @field_pic: whether the reference is a field picture or a frame.
+ * @pic_order_cnt: the picture order count of the reference.
+ */
struct v4l2_hevc_dpb_entry {
__u64 timestamp;
__u8 flags;
@@ -142,6 +244,31 @@ struct v4l2_hevc_dpb_entry {
__u8 padding[2];
};
+/**
+ * struct v4l2_hevc_pred_weight_table - HEVC weighted prediction parameters
+ *
+ * @delta_luma_weight_l0: the difference of the weighting factor applied
+ * to the luma prediction value for list 0
+ * @luma_offset_l0: the additive offset applied to the luma prediction value
+ * for list 0
+ * @delta_chroma_weight_l0: the difference of the weighting factor applied
+ * to the chroma prediction values for list 0
+ * @chroma_offset_l0: the difference of the additive offset applied to
+ * the chroma prediction values for list 0
+ * @delta_luma_weight_l1: the difference of the weighting factor applied
+ * to the luma prediction value for list 1
+ * @luma_offset_l1: the additive offset applied to the luma prediction value
+ * for list 1
+ * @delta_chroma_weight_l1: the difference of the weighting factor applied
+ * to the chroma prediction values for list 1
+ * @chroma_offset_l1:the difference of the additive offset applied to
+ * the chroma prediction values for list 1
+ * @luma_log2_weight_denom: the base 2 logarithm of the denominator for
+ * all luma weighting factors
+ * @delta_chroma_log2_weight_denom: the difference of the base 2 logarithm
+ * of the denominator for all chroma
+ * weighting factors
+ */
struct v4l2_hevc_pred_weight_table {
__s8 delta_luma_weight_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
__s8 luma_offset_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
@@ -170,6 +297,50 @@ struct v4l2_hevc_pred_weight_table {
#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_LOOP_FILTER_ACROSS_SLICES_ENABLED (1ULL << 8)
#define V4L2_HEVC_SLICE_PARAMS_FLAG_DEPENDENT_SLICE_SEGMENT (1ULL << 9)
+/**
+ * v4l2_ctrl_hevc_slice_params - HEVC slice parameters
+ *
+ * @bit_size: size (in bits) of the current slice data
+ * @data_bit_offset: offset (in bits) to the video data in the current slice data
+ * @nal_unit_type: specifies the coding type of the slice (B, P or I)
+ * @nuh_temporal_id_plus1: minus 1 specifies a temporal identifier for the NAL unit
+ * @slice_type: see V4L2_HEVC_SLICE_TYPE_{}
+ * @colour_plane_id: specifies the colour plane associated with the current slice
+ * @slice_pic_order_cnt: specifies the picture order count
+ * @num_ref_idx_l0_active_minus1: specifies the maximum reference index for
+ * reference picture list 0 that may be used to decode
+ * the slice
+ * @num_ref_idx_l1_active_minus1: specifies the maximum reference index for
+ * reference picture list 1 that may be used to decode
+ * the slice
+ * @collocated_ref_idx: specifies the reference index of the collocated picture used
+ * for temporal motion vector prediction
+ * @five_minus_max_num_merge_cand: specifies the maximum number of merging
+ * motion vector prediction candidates supported in
+ * the slice subtracted from 5
+ * @slice_qp_delta: specifies the initial value of QpY to be used for the coding
+ * blocks in the slice
+ * @slice_cb_qp_offset: specifies a difference to be added to the value of pps_cb_qp_offset
+ * @slice_cr_qp_offset: specifies a difference to be added to the value of pps_cr_qp_offset
+ * @slice_act_y_qp_offset: screen content extension parameters
+ * @slice_act_cb_qp_offset: screen content extension parameters
+ * @slice_act_cr_qp_offset: screen content extension parameters
+ * @slice_beta_offset_div2: specify the deblocking parameter offsets for beta divided by 2
+ * @slice_tc_offset_div2: specify the deblocking parameter offsets for tC divided by 2
+ * @pic_struct: indicates whether a picture should be displayed as a frame or as one or
+ * more fields
+ * @slice_segment_addr: specifies the address of the first coding tree block in
+ * the slice segment
+ * @ref_idx_l0: the list of L0 reference elements as indices in the DPB
+ * @ref_idx_l1: the list of L1 reference elements as indices in the DPB
+ * @short_term_ref_pic_set_size: specifies the number of st_ref_pic_set()
+ * syntax structures included in the SPS
+ * @long_term_ref_pic_set_size: specifies the number of candidate long-term
+ * reference pictures that are specified in the SPS
+ * @pred_weight_table: the prediction weight coefficients for inter-picture
+ * prediction
+ * @flags: see V4L2_HEVC_SLICE_PARAMS_FLAG_{}
+ */
struct v4l2_ctrl_hevc_slice_params {
__u32 bit_size;
__u32 data_bit_offset;
@@ -216,6 +387,24 @@ struct v4l2_ctrl_hevc_slice_params {
#define V4L2_HEVC_DECODE_PARAM_FLAG_IDR_PIC 0x2
#define V4L2_HEVC_DECODE_PARAM_FLAG_NO_OUTPUT_OF_PRIOR 0x4
+/**
+ * struct v4l2_ctrl_hevc_decode_params - HEVC decode parameters
+ *
+ * @pic_order_cnt_val: picture order count
+ * @num_active_dpb_entries: the number of entries in dpb
+ * @dpb: the decoded picture buffer, for meta-data about reference frames
+ * @num_poc_st_curr_before: the number of reference pictures in the short-term
+ * set that come before the current frame
+ * @num_poc_st_curr_after: the number of reference pictures in the short-term
+ * set that come after the current frame
+ * @num_poc_lt_curr: the number of reference pictures in the long-term set
+ * @poc_st_curr_before: provides the index of the short term before references
+ * in DPB array
+ * @poc_st_curr_after: provides the index of the short term after references
+ * in DPB array
+ * @poc_lt_curr: provides the index of the long term references in DPB array
+ * @flags: see V4L2_HEVC_DECODE_PARAM_FLAG_{}
+ */
struct v4l2_ctrl_hevc_decode_params {
__s32 pic_order_cnt_val;
__u8 num_active_dpb_entries;
@@ -229,6 +418,28 @@ struct v4l2_ctrl_hevc_decode_params {
__u64 flags;
};
+/**
+ * struct v4l2_ctrl_hevc_scaling_matrix - HEVC scaling lists parameters
+ *
+ * @scaling_list_4x4: scaling list is used for the scaling process for
+ * transform coefficients. The values on each scaling
+ * list are expected in raster scan order
+ * @scaling_list_8x8: scaling list is used for the scaling process for
+ * transform coefficients. The values on each scaling
+ * list are expected in raster scan order
+ * @scaling_list_16x16: scaling list is used for the scaling process for
+ * transform coefficients. The values on each scaling
+ * list are expected in raster scan order
+ * @scaling_list_32x32: scaling list is used for the scaling process for
+ * transform coefficients. The values on each scaling
+ * list are expected in raster scan order
+ * @scaling_list_dc_coef_16x16: scaling list is used for the scaling process
+ * for transform coefficients. The values on each
+ * scaling list are expected in raster scan order.
+ * @scaling_list_dc_coef_32x32: scaling list is used for the scaling process
+ * for transform coefficients. The values on each
+ * scaling list are expected in raster scan order.
+ */
struct v4l2_ctrl_hevc_scaling_matrix {
__u8 scaling_list_4x4[6][16];
__u8 scaling_list_8x8[6][64];
--
2.32.0
Hi!
Dne petek, 25. februar 2022 ob 17:45:55 CET je Benjamin Gaignard napisal(a):
> The number of 'entry point offset' could be very variable.
> Rather than use a large static array define a v4l2 dynamic array
> of integer control.
I suggest we should be more specific and say U32 (V4L2_CTRL_TYPE_U32).
> The number of entry point offsets is reported by the elems field.
>
> Signed-off-by: Benjamin Gaignard <[email protected]>
> ---
> .../userspace-api/media/v4l/ext-ctrls-codec.rst | 9 +++++++++
> include/media/hevc-ctrls.h | 1 +
> 2 files changed, 10 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> index 44a268a948c0..71f7dc1c1ccd 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> @@ -3128,6 +3128,15 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>
> \normalsize
>
> +``V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS (integer)``
Here you have OFFSETS (plural) ...
> + Specifies the i-th entry point offset in bytes and is represented by
> + offset_len_minus1 plus 1 bits.
You probably mean entry_point_offset_minus1? offset_len_minus1 just tells how
much bits need to be read for each element and it's not important for actual
decoding.
> + This control is a dynamically sized array. The number of entry point
> + offsets is reported by the ``elems`` field.
> + This bitstream parameter is defined according to :ref:`hevc`.
> + They are described in section 7.4.7.1 "General slice segment header
> + semantics" of the specification.
> +
> ``V4L2_CID_STATELESS_HEVC_SCALING_MATRIX (struct)``
> Specifies the HEVC scaling matrix parameters used for the scaling
process
> for transform coefficients.
> diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
> index 3016c1abb1d0..3f8a67924df3 100644
> --- a/include/media/hevc-ctrls.h
> +++ b/include/media/hevc-ctrls.h
> @@ -20,6 +20,7 @@
> #define V4L2_CID_STATELESS_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_BASE +
1012)
> #define V4L2_CID_STATELESS_HEVC_DECODE_MODE (V4L2_CID_CODEC_BASE +
1015)
> #define V4L2_CID_STATELESS_HEVC_START_CODE (V4L2_CID_CODEC_BASE + 1016)
> +#define V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET (V4L2_CID_CODEC_BASE +
1017)
... and here you have OFFSET (singlular). I suggest plural form to be used in
all places, including subject line of this commit.
Additionally, it would be nice if control is initialized, like so:
https://github.com/jernejsk/linux-1/commit/
f938e162cd8dd77c9f6f1b248d80144840a37bce
Best regards,
Jernej
>
> /* enum v4l2_ctrl_type type values */
> #define V4L2_CTRL_TYPE_HEVC_SPS 0x0120
> --
> 2.32.0
>
>
Dne petek, 25. februar 2022 ob 17:45:51 CET je Benjamin Gaignard napisal(a):
> Change HEVC stateless controls names to V4L2_CID_STATELESS_HEVC instead
> of V4L2_CID_MPEG_VIDEO_HEVC be coherent with v4l2 naming convention.
>
> Signed-off-by: Benjamin Gaignard <[email protected]>
> ---
> .../media/v4l/ext-ctrls-codec.rst | 26 +++++++--------
> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 32 +++++++++----------
> drivers/staging/media/hantro/hantro_drv.c | 26 +++++++--------
> drivers/staging/media/hantro/hantro_hevc.c | 8 ++---
> drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +++++++-------
> .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +++---
> include/media/hevc-ctrls.h | 26 +++++++--------
> 7 files changed, 76 insertions(+), 76 deletions(-)
>
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> index b83d45202916..262ad4f3c1f4 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> @@ -2639,7 +2639,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>
> .. _v4l2-mpeg-hevc:
>
> -``V4L2_CID_MPEG_VIDEO_HEVC_SPS (struct)``
> +``V4L2_CID_STATELESS_HEVC_SPS (struct)``
> Specifies the Sequence Parameter Set fields (as extracted from the
> bitstream) for the associated HEVC slice data.
> These bitstream parameters are defined according to :ref:`hevc`.
> @@ -2792,7 +2792,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>
> \normalsize
>
> -``V4L2_CID_MPEG_VIDEO_HEVC_PPS (struct)``
> +``V4L2_CID_STATELESS_HEVC_PPS (struct)``
> Specifies the Picture Parameter Set fields (as extracted from the
> bitstream) for the associated HEVC slice data.
> These bitstream parameters are defined according to :ref:`hevc`.
> @@ -2945,7 +2945,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>
> \normalsize
>
> -``V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS (struct)``
> +``V4L2_CID_STATELESS_HEVC_SLICE_PARAMS (struct)``
> Specifies various slice-specific parameters, especially from the NAL unit
> header, general slice segment header and weighted prediction parameter
> parts of the bitstream.
> @@ -3110,7 +3110,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>
> \normalsize
>
> -``V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX (struct)``
> +``V4L2_CID_STATELESS_HEVC_SCALING_MATRIX (struct)``
> Specifies the HEVC scaling matrix parameters used for the scaling
process
> for transform coefficients.
> These matrix and parameters are defined according to :ref:`hevc`.
> @@ -3260,7 +3260,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>
> \normalsize
>
> -``V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE (enum)``
> +``V4L2_CID_STATELESS_HEVC_DECODE_MODE (enum)``
> Specifies the decoding mode to use. Currently exposes slice-based and
> frame-based decoding but new modes might be added later on.
> This control is used as a modifier for V4L2_PIX_FMT_HEVC_SLICE
> @@ -3275,7 +3275,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> This menu control is not yet part of the public kernel API and
> it is expected to change.
>
> -.. c:type:: v4l2_mpeg_video_hevc_decode_mode
> +.. c:type:: v4l2_stateless_hevc_decode_mode
>
> .. raw:: latex
>
> @@ -3288,11 +3288,11 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> :stub-columns: 0
> :widths: 1 1 2
>
> - * - ``V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_SLICE_BASED``
> + * - ``V4L2_STATELESS_HEVC_DECODE_MODE_SLICE_BASED``
> - 0
> - Decoding is done at the slice granularity.
> The OUTPUT buffer must contain a single slice.
> - * - ``V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED``
> + * - ``V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED``
> - 1
> - Decoding is done at the frame granularity.
> The OUTPUT buffer must contain all slices needed to decode the
> @@ -3302,7 +3302,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>
> \normalsize
>
> -``V4L2_CID_MPEG_VIDEO_HEVC_START_CODE (enum)``
> +``V4L2_CID_STATELESS_HEVC_START_CODE (enum)``
> Specifies the HEVC slice start code expected for each slice.
> This control is used as a modifier for V4L2_PIX_FMT_HEVC_SLICE
> pixel format. Applications that support V4L2_PIX_FMT_HEVC_SLICE
> @@ -3316,7 +3316,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> This menu control is not yet part of the public kernel API and
> it is expected to change.
>
> -.. c:type:: v4l2_mpeg_video_hevc_start_code
> +.. c:type:: v4l2_stateless_hevc_start_code
>
> .. tabularcolumns:: |p{9.2cm}|p{0.6cm}|p{7.5cm}|
>
> @@ -3325,13 +3325,13 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> :stub-columns: 0
> :widths: 1 1 2
>
> - * - ``V4L2_MPEG_VIDEO_HEVC_START_CODE_NONE``
> + * - ``V4L2_STATELESS_HEVC_START_CODE_NONE``
> - 0
> - Selecting this value specifies that HEVC slices are passed
> to the driver without any start code. The bitstream data should be
> according to :ref:`hevc` 7.3.1.1 General NAL unit syntax, hence
> contains emulation prevention bytes when required.
> - * - ``V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B``
> + * - ``V4L2_STATELESS_HEVC_START_CODE_ANNEX_B``
> - 1
> - Selecting this value specifies that HEVC slices are expected
> to be prefixed by Annex B start codes. According to :ref:`hevc`
> @@ -3364,7 +3364,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> This provides a bitmask which consists of bits [0, LTR_COUNT-1].
> This is applicable to the H264 and HEVC encoders.
>
> -``V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS (struct)``
> +``V4L2_CID_STATELESS_HEVC_DECODE_PARAMS (struct)``
> Specifies various decode parameters, especially the references picture
order
> count (POC) for all the lists (short, long, before, current, after) and
the
> number of entries for each of them.
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-
core/v4l2-ctrls-defs.c
> index 54ca4e6b820b..00cb08083514 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> @@ -699,9 +699,9 @@ const char * const *v4l2_ctrl_get_menu(u32 id)
> return hevc_tier;
> case V4L2_CID_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE:
> return hevc_loop_filter_mode;
> - case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE:
> + case V4L2_CID_STATELESS_HEVC_DECODE_MODE:
> return hevc_decode_mode;
> - case V4L2_CID_MPEG_VIDEO_HEVC_START_CODE:
> + case V4L2_CID_STATELESS_HEVC_START_CODE:
> return hevc_start_code;
> case V4L2_CID_CAMERA_ORIENTATION:
> return camera_orientation;
> @@ -995,13 +995,13 @@ const char *v4l2_ctrl_get_name(u32 id)
> case V4L2_CID_MPEG_VIDEO_HEVC_SIZE_OF_LENGTH_FIELD: return
"HEVC Size of Length Field";
> case V4L2_CID_MPEG_VIDEO_REF_NUMBER_FOR_PFRAMES: return
"Reference Frames for a P-Frame";
> case V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR:
return "Prepend SPS and PPS to IDR";
> - case V4L2_CID_MPEG_VIDEO_HEVC_SPS:
return "HEVC Sequence Parameter Set";
> - case V4L2_CID_MPEG_VIDEO_HEVC_PPS:
return "HEVC Picture Parameter Set";
> - case V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS: return
"HEVC Slice Parameters";
> - case V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX:
return "HEVC Scaling Matrix";
> - case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS:
return "HEVC Decode Parameters";
> - case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE: return "HEVC
Decode Mode";
> - case V4L2_CID_MPEG_VIDEO_HEVC_START_CODE: return
"HEVC Start Code";
> + case V4L2_CID_STATELESS_HEVC_SPS: return
"HEVC Sequence Parameter Set";
> + case V4L2_CID_STATELESS_HEVC_PPS: return
"HEVC Picture Parameter Set";
> + case V4L2_CID_STATELESS_HEVC_SLICE_PARAMS: return
"HEVC Slice Parameters";
> + case V4L2_CID_STATELESS_HEVC_SCALING_MATRIX: return
"HEVC Scaling Matrix";
> + case V4L2_CID_STATELESS_HEVC_DECODE_PARAMS: return "HEVC
Decode Parameters";
> + case V4L2_CID_STATELESS_HEVC_DECODE_MODE: return "HEVC
Decode Mode";
> + case V4L2_CID_STATELESS_HEVC_START_CODE: return
"HEVC Start Code";
These shouldbe now moved under "/* Stateless Codec controls */" section.
Note, description for V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS is missing,
but that should be a part of another patch (one that introduces that CID).
Best regards,
Jernej
>
> /* CAMERA controls */
> /* Keep the order of the 'case's the same as in v4l2-controls.h!
*/
> @@ -1355,8 +1355,8 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum
v4l2_ctrl_type *type,
> case V4L2_CID_MPEG_VIDEO_HEVC_SIZE_OF_LENGTH_FIELD:
> case V4L2_CID_MPEG_VIDEO_HEVC_TIER:
> case V4L2_CID_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE:
> - case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE:
> - case V4L2_CID_MPEG_VIDEO_HEVC_START_CODE:
> + case V4L2_CID_STATELESS_HEVC_DECODE_MODE:
> + case V4L2_CID_STATELESS_HEVC_START_CODE:
> case V4L2_CID_STATELESS_H264_DECODE_MODE:
> case V4L2_CID_STATELESS_H264_START_CODE:
> case V4L2_CID_CAMERA_ORIENTATION:
> @@ -1493,19 +1493,19 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum
v4l2_ctrl_type *type,
> case V4L2_CID_STATELESS_VP8_FRAME:
> *type = V4L2_CTRL_TYPE_VP8_FRAME;
> break;
> - case V4L2_CID_MPEG_VIDEO_HEVC_SPS:
> + case V4L2_CID_STATELESS_HEVC_SPS:
> *type = V4L2_CTRL_TYPE_HEVC_SPS;
> break;
> - case V4L2_CID_MPEG_VIDEO_HEVC_PPS:
> + case V4L2_CID_STATELESS_HEVC_PPS:
> *type = V4L2_CTRL_TYPE_HEVC_PPS;
> break;
> - case V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS:
> + case V4L2_CID_STATELESS_HEVC_SLICE_PARAMS:
> *type = V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS;
> break;
> - case V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX:
> + case V4L2_CID_STATELESS_HEVC_SCALING_MATRIX:
> *type = V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX;
> break;
> - case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS:
> + case V4L2_CID_STATELESS_HEVC_DECODE_PARAMS:
> *type = V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS;
> break;
> case V4L2_CID_STATELESS_VP9_COMPRESSED_HDR:
> diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/
media/hantro/hantro_drv.c
> index dc768884cb79..6f58c259d8fc 100644
> --- a/drivers/staging/media/hantro/hantro_drv.c
> +++ b/drivers/staging/media/hantro/hantro_drv.c
> @@ -255,7 +255,7 @@ static int hantro_try_ctrl(struct v4l2_ctrl *ctrl)
> if (sps->bit_depth_luma_minus8 != 0)
> /* Only 8-bit is supported */
> return -EINVAL;
> - } else if (ctrl->id == V4L2_CID_MPEG_VIDEO_HEVC_SPS) {
> + } else if (ctrl->id == V4L2_CID_STATELESS_HEVC_SPS) {
> const struct v4l2_ctrl_hevc_sps *sps = ctrl-
>p_new.p_hevc_sps;
>
> if (sps->bit_depth_luma_minus8 != sps-
>bit_depth_chroma_minus8)
> @@ -428,18 +428,18 @@ static const struct hantro_ctrl controls[] = {
> }, {
> .codec = HANTRO_HEVC_DECODER,
> .cfg = {
> - .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE,
> - .min =
V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
> - .max =
V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
> - .def =
V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
> + .id = V4L2_CID_STATELESS_HEVC_DECODE_MODE,
> + .min =
V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
> + .max =
V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
> + .def =
V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
> },
> }, {
> .codec = HANTRO_HEVC_DECODER,
> .cfg = {
> - .id = V4L2_CID_MPEG_VIDEO_HEVC_START_CODE,
> - .min =
V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
> - .max =
V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
> - .def =
V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
> + .id = V4L2_CID_STATELESS_HEVC_START_CODE,
> + .min =
V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
> + .max =
V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
> + .def =
V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
> },
> }, {
> .codec = HANTRO_HEVC_DECODER,
> @@ -459,23 +459,23 @@ static const struct hantro_ctrl controls[] = {
> }, {
> .codec = HANTRO_HEVC_DECODER,
> .cfg = {
> - .id = V4L2_CID_MPEG_VIDEO_HEVC_SPS,
> + .id = V4L2_CID_STATELESS_HEVC_SPS,
> .ops = &hantro_ctrl_ops,
> },
> }, {
> .codec = HANTRO_HEVC_DECODER,
> .cfg = {
> - .id = V4L2_CID_MPEG_VIDEO_HEVC_PPS,
> + .id = V4L2_CID_STATELESS_HEVC_PPS,
> },
> }, {
> .codec = HANTRO_HEVC_DECODER,
> .cfg = {
> - .id =
V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS,
> + .id = V4L2_CID_STATELESS_HEVC_DECODE_PARAMS,
> },
> }, {
> .codec = HANTRO_HEVC_DECODER,
> .cfg = {
> - .id =
V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX,
> + .id = V4L2_CID_STATELESS_HEVC_SCALING_MATRIX,
> },
> }, {
> .codec = HANTRO_HEVC_DECODER,
> diff --git a/drivers/staging/media/hantro/hantro_hevc.c b/drivers/staging/
media/hantro/hantro_hevc.c
> index b49a41d7ae91..b6ec86d03d91 100644
> --- a/drivers/staging/media/hantro/hantro_hevc.c
> +++ b/drivers/staging/media/hantro/hantro_hevc.c
> @@ -201,22 +201,22 @@ int hantro_hevc_dec_prepare_run(struct hantro_ctx
*ctx)
> hantro_start_prepare_run(ctx);
>
> ctrls->decode_params =
> - hantro_get_ctrl(ctx,
V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS);
> + hantro_get_ctrl(ctx,
V4L2_CID_STATELESS_HEVC_DECODE_PARAMS);
> if (WARN_ON(!ctrls->decode_params))
> return -EINVAL;
>
> ctrls->scaling =
> - hantro_get_ctrl(ctx,
V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX);
> + hantro_get_ctrl(ctx,
V4L2_CID_STATELESS_HEVC_SCALING_MATRIX);
> if (WARN_ON(!ctrls->scaling))
> return -EINVAL;
>
> ctrls->sps =
> - hantro_get_ctrl(ctx, V4L2_CID_MPEG_VIDEO_HEVC_SPS);
> + hantro_get_ctrl(ctx, V4L2_CID_STATELESS_HEVC_SPS);
> if (WARN_ON(!ctrls->sps))
> return -EINVAL;
>
> ctrls->pps =
> - hantro_get_ctrl(ctx, V4L2_CID_MPEG_VIDEO_HEVC_PPS);
> + hantro_get_ctrl(ctx, V4L2_CID_STATELESS_HEVC_PPS);
> if (WARN_ON(!ctrls->pps))
> return -EINVAL;
>
> diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c b/drivers/staging/
media/sunxi/cedrus/cedrus.c
> index 4a4b714b0f26..e0428163f82c 100644
> --- a/drivers/staging/media/sunxi/cedrus/cedrus.c
> +++ b/drivers/staging/media/sunxi/cedrus/cedrus.c
> @@ -42,7 +42,7 @@ static int cedrus_try_ctrl(struct v4l2_ctrl *ctrl)
> if (sps->bit_depth_luma_minus8 != 0)
> /* Only 8-bit is supported */
> return -EINVAL;
> - } else if (ctrl->id == V4L2_CID_MPEG_VIDEO_HEVC_SPS) {
> + } else if (ctrl->id == V4L2_CID_STATELESS_HEVC_SPS) {
> const struct v4l2_ctrl_hevc_sps *sps = ctrl-
>p_new.p_hevc_sps;
> struct cedrus_ctx *ctx = container_of(ctrl->handler,
struct cedrus_ctx, hdl);
>
> @@ -164,42 +164,42 @@ static const struct cedrus_control cedrus_controls[] =
{
> },
> {
> .cfg = {
> - .id = V4L2_CID_MPEG_VIDEO_HEVC_SPS,
> + .id = V4L2_CID_STATELESS_HEVC_SPS,
> .ops = &cedrus_ctrl_ops,
> },
> .codec = CEDRUS_CODEC_H265,
> },
> {
> .cfg = {
> - .id = V4L2_CID_MPEG_VIDEO_HEVC_PPS,
> + .id = V4L2_CID_STATELESS_HEVC_PPS,
> },
> .codec = CEDRUS_CODEC_H265,
> },
> {
> .cfg = {
> - .id =
V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS,
> + .id =
V4L2_CID_STATELESS_HEVC_SLICE_PARAMS,
> },
> .codec = CEDRUS_CODEC_H265,
> },
> {
> .cfg = {
> - .id =
V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX,
> + .id =
V4L2_CID_STATELESS_HEVC_SCALING_MATRIX,
> },
> .codec = CEDRUS_CODEC_H265,
> },
> {
> .cfg = {
> - .id =
V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE,
> - .max =
V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_SLICE_BASED,
> - .def =
V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_SLICE_BASED,
> + .id =
V4L2_CID_STATELESS_HEVC_DECODE_MODE,
> + .max =
V4L2_STATELESS_HEVC_DECODE_MODE_SLICE_BASED,
> + .def =
V4L2_STATELESS_HEVC_DECODE_MODE_SLICE_BASED,
> },
> .codec = CEDRUS_CODEC_H265,
> },
> {
> .cfg = {
> - .id =
V4L2_CID_MPEG_VIDEO_HEVC_START_CODE,
> - .max =
V4L2_MPEG_VIDEO_HEVC_START_CODE_NONE,
> - .def =
V4L2_MPEG_VIDEO_HEVC_START_CODE_NONE,
> + .id =
V4L2_CID_STATELESS_HEVC_START_CODE,
> + .max =
V4L2_STATELESS_HEVC_START_CODE_NONE,
> + .def =
V4L2_STATELESS_HEVC_START_CODE_NONE,
> },
> .codec = CEDRUS_CODEC_H265,
> },
> @@ -211,7 +211,7 @@ static const struct cedrus_control cedrus_controls[] = {
> },
> {
> .cfg = {
> - .id =
V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS,
> + .id = V4L2_CID_STATELESS_HEVC_DECODE_PARAMS,
> },
> .codec = CEDRUS_CODEC_H265,
> },
> diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c b/drivers/
staging/media/sunxi/cedrus/cedrus_dec.c
> index a16c1422558f..f6be4ae72ee2 100644
> --- a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
> +++ b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
> @@ -65,15 +65,15 @@ void cedrus_device_run(void *priv)
>
> case V4L2_PIX_FMT_HEVC_SLICE:
> run.h265.sps = cedrus_find_control_data(ctx,
> - V4L2_CID_MPEG_VIDEO_HEVC_SPS);
> + V4L2_CID_STATELESS_HEVC_SPS);
> run.h265.pps = cedrus_find_control_data(ctx,
> - V4L2_CID_MPEG_VIDEO_HEVC_PPS);
> + V4L2_CID_STATELESS_HEVC_PPS);
> run.h265.slice_params = cedrus_find_control_data(ctx,
> - V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS);
> + V4L2_CID_STATELESS_HEVC_SLICE_PARAMS);
> run.h265.decode_params = cedrus_find_control_data(ctx,
> - V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS);
> + V4L2_CID_STATELESS_HEVC_DECODE_PARAMS);
> run.h265.scaling_matrix = cedrus_find_control_data(ctx,
> - V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX);
> + V4L2_CID_STATELESS_HEVC_SCALING_MATRIX);
> break;
>
> case V4L2_PIX_FMT_VP8_FRAME:
> diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
> index a329e086a89a..b3540167df9e 100644
> --- a/include/media/hevc-ctrls.h
> +++ b/include/media/hevc-ctrls.h
> @@ -16,13 +16,13 @@
> /* The pixel format isn't stable at the moment and will likely be renamed.
*/
> #define V4L2_PIX_FMT_HEVC_SLICE v4l2_fourcc('S', '2', '6', '5') /* HEVC
parsed slices */
>
> -#define V4L2_CID_MPEG_VIDEO_HEVC_SPS (V4L2_CID_CODEC_BASE +
1008)
> -#define V4L2_CID_MPEG_VIDEO_HEVC_PPS (V4L2_CID_CODEC_BASE +
1009)
> -#define V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS (V4L2_CID_CODEC_BASE +
1010)
> -#define V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX (V4L2_CID_CODEC_BASE +
1011)
> -#define V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_BASE +
1012)
> -#define V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE (V4L2_CID_CODEC_BASE +
1015)
> -#define V4L2_CID_MPEG_VIDEO_HEVC_START_CODE (V4L2_CID_CODEC_BASE +
1016)
> +#define V4L2_CID_STATELESS_HEVC_SPS (V4L2_CID_CODEC_BASE +
1008)
> +#define V4L2_CID_STATELESS_HEVC_PPS (V4L2_CID_CODEC_BASE +
1009)
> +#define V4L2_CID_STATELESS_HEVC_SLICE_PARAMS (V4L2_CID_CODEC_BASE +
1010)
> +#define V4L2_CID_STATELESS_HEVC_SCALING_MATRIX (V4L2_CID_CODEC_BASE +
1011)
> +#define V4L2_CID_STATELESS_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_BASE +
1012)
> +#define V4L2_CID_STATELESS_HEVC_DECODE_MODE (V4L2_CID_CODEC_BASE +
1015)
> +#define V4L2_CID_STATELESS_HEVC_START_CODE (V4L2_CID_CODEC_BASE + 1016)
>
> /* enum v4l2_ctrl_type type values */
> #define V4L2_CTRL_TYPE_HEVC_SPS 0x0120
> @@ -31,14 +31,14 @@
> #define V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX 0x0123
> #define V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS 0x0124
>
> -enum v4l2_mpeg_video_hevc_decode_mode {
> - V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_SLICE_BASED,
> - V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
> +enum v4l2_stateless_hevc_decode_mode {
> + V4L2_STATELESS_HEVC_DECODE_MODE_SLICE_BASED,
> + V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
> };
>
> -enum v4l2_mpeg_video_hevc_start_code {
> - V4L2_MPEG_VIDEO_HEVC_START_CODE_NONE,
> - V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
> +enum v4l2_stateless_hevc_start_code {
> + V4L2_STATELESS_HEVC_START_CODE_NONE,
> + V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
> };
>
> #define V4L2_HEVC_SLICE_TYPE_B 0
> --
> 2.32.0
>
>
From: Hans Verkuil <[email protected]>
Add a new flag that indicates that this control is a dynamically sized
array. Also document this flag.
Currently dynamically sized arrays are limited to one dimensional arrays,
but that might change in the future if there is a need for it.
The initial use-case of dynamic arrays are stateless codecs. A frame
can be divided in many slices, so you want to provide an array containing
slice information for each slice. Typically the number of slices is small,
but the standard allow for hundreds or thousands of slices. Dynamic arrays
are a good solution since sizing the array for the worst case would waste
substantial amounts of memory.
Signed-off-by: Hans Verkuil <[email protected]>
---
.../userspace-api/media/v4l/vidioc-queryctrl.rst | 8 ++++++++
include/uapi/linux/videodev2.h | 1 +
2 files changed, 9 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
index 88f630252d98..a20dfa2a933b 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
@@ -625,6 +625,14 @@ See also the examples in :ref:`control`.
``V4L2_CTRL_FLAG_GRABBED`` flag when buffers are allocated or
streaming is in progress since most drivers do not support changing
the format in that case.
+ * - ``V4L2_CTRL_FLAG_DYNAMIC_ARRAY``
+ - 0x0800
+ - This control is a dynamically sized 1-dimensional array. It
+ behaves the same as a regular array, except that the number
+ of elements as reported by the ``elems`` field is between 1 and
+ ``dims[0]``. So setting the control with a differently sized
+ array will change the ``elems`` field when the control is
+ queried afterwards.
Return Value
============
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index df8b9c486ba1..e27c8eae78c9 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1884,6 +1884,7 @@ struct v4l2_querymenu {
#define V4L2_CTRL_FLAG_HAS_PAYLOAD 0x0100
#define V4L2_CTRL_FLAG_EXECUTE_ON_WRITE 0x0200
#define V4L2_CTRL_FLAG_MODIFY_LAYOUT 0x0400
+#define V4L2_CTRL_FLAG_DYNAMIC_ARRAY 0x0800
/* Query flags, to be ORed with the control ID */
#define V4L2_CTRL_FLAG_NEXT_CTRL 0x80000000
--
2.32.0
Move HEVC pixel format since we are ready to stabilize the uAPI
Signed-off-by: Benjamin Gaignard <[email protected]>
---
Documentation/userspace-api/media/v4l/pixfmt-compressed.rst | 5 -----
include/media/hevc-ctrls.h | 3 ---
include/uapi/linux/videodev2.h | 1 +
3 files changed, 1 insertion(+), 8 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
index 967fc803ef94..c352d91a73d8 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
@@ -215,11 +215,6 @@ Compressed Formats
See the :ref:`associated Codec Control IDs <v4l2-mpeg-hevc>`.
Buffers associated with this pixel format must contain the appropriate
number of macroblocks to decode a full corresponding frame.
-
- .. note::
-
- This format is not yet part of the public kernel API and it
- is expected to change.
* .. _V4L2-PIX-FMT-FWHT:
- ``V4L2_PIX_FMT_FWHT``
diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
index 057b5032453b..3016c1abb1d0 100644
--- a/include/media/hevc-ctrls.h
+++ b/include/media/hevc-ctrls.h
@@ -13,9 +13,6 @@
#include <linux/videodev2.h>
-/* The pixel format isn't stable at the moment and will likely be renamed. */
-#define V4L2_PIX_FMT_HEVC_SLICE v4l2_fourcc('S', '2', '6', '5') /* HEVC parsed slices */
-
#define V4L2_CID_STATELESS_HEVC_SPS (V4L2_CID_CODEC_BASE + 1008)
#define V4L2_CID_STATELESS_HEVC_PPS (V4L2_CID_CODEC_BASE + 1009)
#define V4L2_CID_STATELESS_HEVC_SLICE_PARAMS (V4L2_CID_CODEC_BASE + 1010)
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index e27c8eae78c9..959a849388f5 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -708,6 +708,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_FWHT v4l2_fourcc('F', 'W', 'H', 'T') /* Fast Walsh Hadamard Transform (vicodec) */
#define V4L2_PIX_FMT_FWHT_STATELESS v4l2_fourcc('S', 'F', 'W', 'H') /* Stateless FWHT (vicodec) */
#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
+#define V4L2_PIX_FMT_HEVC_SLICE v4l2_fourcc('S', '2', '6', '5') /* HEVC parsed slices */
/* Vendor-specific formats */
#define V4L2_PIX_FMT_CPIA1 v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1 YUV */
--
2.32.0
Fix padding where needed to remove holes and stay align on cache boundaries
Signed-off-by: Benjamin Gaignard <[email protected]>
---
.../userspace-api/media/v4l/ext-ctrls-codec.rst | 6 +++---
drivers/media/v4l2-core/v4l2-ctrls-core.c | 1 -
include/media/hevc-ctrls.h | 15 +++++++++------
3 files changed, 12 insertions(+), 10 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 71f7dc1c1ccd..00fa815cb5f8 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -3423,9 +3423,6 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
* - __u8
- ``num_active_dpb_entries``
- The number of entries in ``dpb``.
- * - struct :c:type:`v4l2_hevc_dpb_entry`
- - ``dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
- - The decoded picture buffer, for meta-data about reference frames.
* - __u8
- ``num_poc_st_curr_before``
- The number of reference pictures in the short-term set that come before
@@ -3449,6 +3446,9 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
- ``poc_lt_curr[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
- PocLtCurr as described in section 8.3.2 "Decoding process for reference
picture set": provides the index of the long term references in DPB array.
+ * - struct :c:type:`v4l2_hevc_dpb_entry`
+ - ``dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
+ - The decoded picture buffer, for meta-data about reference frames.
* - __u64
- ``flags``
- See :ref:`Decode Parameters Flags <hevc_decode_params_flags>`
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c
index 694de414f545..eb1de4f699d0 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
@@ -819,7 +819,6 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx,
case V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS:
p_hevc_slice_params = p;
- zero_padding(p_hevc_slice_params->pred_weight_table);
zero_padding(*p_hevc_slice_params);
break;
diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
index f29298c8c2f8..6bffcb2c8a1d 100644
--- a/include/media/hevc-ctrls.h
+++ b/include/media/hevc-ctrls.h
@@ -133,7 +133,9 @@ struct v4l2_ctrl_hevc_sps {
__u8 chroma_format_idc;
__u8 sps_max_sub_layers_minus1;
+ __u8 padding[6];
__u64 flags;
+ __u8 reserved[24];
};
#define V4L2_HEVC_PPS_FLAG_DEPENDENT_SLICE_SEGMENT_ENABLED (1ULL << 0)
@@ -210,9 +212,10 @@ struct v4l2_ctrl_hevc_pps {
__s8 pps_beta_offset_div2;
__s8 pps_tc_offset_div2;
__u8 log2_parallel_merge_level_minus2;
+ __u8 padding[9];
- __u8 padding[4];
__u64 flags;
+ __u8 reserved[56];
};
#define V4L2_HEVC_DPB_ENTRY_LONG_TERM_REFERENCE 0x01
@@ -271,8 +274,6 @@ struct v4l2_hevc_pred_weight_table {
__s8 delta_chroma_weight_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
__s8 chroma_offset_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
- __u8 padding[6];
-
__u8 luma_log2_weight_denom;
__s8 delta_chroma_log2_weight_denom;
};
@@ -363,18 +364,20 @@ struct v4l2_ctrl_hevc_slice_params {
/* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture timing SEI message */
__u8 pic_struct;
+ __u8 reserved;
/* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
__u32 slice_segment_addr;
__u8 ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
__u8 ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
__u16 short_term_ref_pic_set_size;
__u16 long_term_ref_pic_set_size;
- __u8 padding;
/* ISO/IEC 23008-2, ITU-T Rec. H.265: Weighted prediction parameter */
struct v4l2_hevc_pred_weight_table pred_weight_table;
+ __u16 reserved1;
__u64 flags;
+ __u8 padding[48];
};
#define V4L2_HEVC_DECODE_PARAM_FLAG_IRAP_PIC 0x1
@@ -386,7 +389,6 @@ struct v4l2_ctrl_hevc_slice_params {
*
* @pic_order_cnt_val: picture order count
* @num_active_dpb_entries: the number of entries in dpb
- * @dpb: the decoded picture buffer, for meta-data about reference frames
* @num_poc_st_curr_before: the number of reference pictures in the short-term
* set that come before the current frame
* @num_poc_st_curr_after: the number of reference pictures in the short-term
@@ -397,18 +399,19 @@ struct v4l2_ctrl_hevc_slice_params {
* @poc_st_curr_after: provides the index of the short term after references
* in DPB array
* @poc_lt_curr: provides the index of the long term references in DPB array
+ * @dpb: the decoded picture buffer, for meta-data about reference frames
* @flags: see V4L2_HEVC_DECODE_PARAM_FLAG_{}
*/
struct v4l2_ctrl_hevc_decode_params {
__s32 pic_order_cnt_val;
__u8 num_active_dpb_entries;
- struct v4l2_hevc_dpb_entry dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
__u8 num_poc_st_curr_before;
__u8 num_poc_st_curr_after;
__u8 num_poc_lt_curr;
__u8 poc_st_curr_before[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
__u8 poc_st_curr_after[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
__u8 poc_lt_curr[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
+ struct v4l2_hevc_dpb_entry dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
__u64 flags;
};
--
2.32.0
From: Hans Verkuil <[email protected]>
Add a dynamic array test control to help test support for this
feature.
Signed-off-by: Hans Verkuil <[email protected]>
---
drivers/media/test-drivers/vivid/vivid-ctrls.c | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/drivers/media/test-drivers/vivid/vivid-ctrls.c b/drivers/media/test-drivers/vivid/vivid-ctrls.c
index e7516dc1227b..7267892dc18a 100644
--- a/drivers/media/test-drivers/vivid/vivid-ctrls.c
+++ b/drivers/media/test-drivers/vivid/vivid-ctrls.c
@@ -34,6 +34,7 @@
#define VIVID_CID_U8_4D_ARRAY (VIVID_CID_CUSTOM_BASE + 10)
#define VIVID_CID_AREA (VIVID_CID_CUSTOM_BASE + 11)
#define VIVID_CID_RO_INTEGER (VIVID_CID_CUSTOM_BASE + 12)
+#define VIVID_CID_U32_DYN_ARRAY (VIVID_CID_CUSTOM_BASE + 13)
#define VIVID_CID_VIVID_BASE (0x00f00000 | 0xf000)
#define VIVID_CID_VIVID_CLASS (0x00f00000 | 1)
@@ -189,6 +190,19 @@ static const struct v4l2_ctrl_config vivid_ctrl_u32_array = {
.dims = { 1 },
};
+static const struct v4l2_ctrl_config vivid_ctrl_u32_dyn_array = {
+ .ops = &vivid_user_gen_ctrl_ops,
+ .id = VIVID_CID_U32_DYN_ARRAY,
+ .name = "U32 Dynamic Array",
+ .type = V4L2_CTRL_TYPE_U32,
+ .flags = V4L2_CTRL_FLAG_DYNAMIC_ARRAY,
+ .def = 50,
+ .min = 10,
+ .max = 90,
+ .step = 1,
+ .dims = { 100 },
+};
+
static const struct v4l2_ctrl_config vivid_ctrl_u16_matrix = {
.ops = &vivid_user_gen_ctrl_ops,
.id = VIVID_CID_U16_MATRIX,
@@ -1612,6 +1626,7 @@ int vivid_create_controls(struct vivid_dev *dev, bool show_ccs_cap,
dev->ro_int32 = v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_ro_int32, NULL);
v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_area, NULL);
v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_u32_array, NULL);
+ v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_u32_dyn_array, NULL);
v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_u16_matrix, NULL);
v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_u8_4d_array, NULL);
--
2.32.0
Hi!
Dne petek, 25. februar 2022 ob 20:30:20 CET je Jernej Škrabec napisal(a):
> Hi!
>
> Dne petek, 25. februar 2022 ob 17:45:55 CET je Benjamin Gaignard napisal(a):
> > The number of 'entry point offset' could be very variable.
> > Rather than use a large static array define a v4l2 dynamic array
> > of integer control.
>
> I suggest we should be more specific and say U32 (V4L2_CTRL_TYPE_U32).
>
> > The number of entry point offsets is reported by the elems field.
I did few more tests and these are my findings:
1. dynamic array can't be set over size, specified in .dims array
2. entry point offsets are per slice, so if we make slices dynamic array, then
entry points would become two dimensional dynamic array
3. num_entry_point_offsets must be part of slice control, because it can be
zero, but size of array can't be
4. fortunately, not setting entry points doesn't impact decoding correctness.
This is in line what John told me about them.
Hans, can you comment points 1-3? I might misunderstand point 1.
In short, it seems like we don't really need entry points, even if they are
used in BSP library. In both cases, I got fluster score 119/138 (10-bit
excluded), so we can just drop this patch, although I'm a bit uneasy not
setting entry points...
Best regards,
Jernej
> >
> > Signed-off-by: Benjamin Gaignard <[email protected]>
> > ---
> > .../userspace-api/media/v4l/ext-ctrls-codec.rst | 9 +++++++++
> > include/media/hevc-ctrls.h | 1 +
> > 2 files changed, 10 insertions(+)
> >
> > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/
> Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > index 44a268a948c0..71f7dc1c1ccd 100644
> > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > @@ -3128,6 +3128,15 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> >
> > \normalsize
> >
> > +``V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS (integer)``
>
> Here you have OFFSETS (plural) ...
>
> > + Specifies the i-th entry point offset in bytes and is represented by
> > + offset_len_minus1 plus 1 bits.
>
> You probably mean entry_point_offset_minus1? offset_len_minus1 just tells how
> much bits need to be read for each element and it's not important for actual
> decoding.
>
> > + This control is a dynamically sized array. The number of entry point
> > + offsets is reported by the ``elems`` field.
> > + This bitstream parameter is defined according to :ref:`hevc`.
> > + They are described in section 7.4.7.1 "General slice segment header
> > + semantics" of the specification.
> > +
> > ``V4L2_CID_STATELESS_HEVC_SCALING_MATRIX (struct)``
> > Specifies the HEVC scaling matrix parameters used for the scaling
> process
> > for transform coefficients.
> > diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
> > index 3016c1abb1d0..3f8a67924df3 100644
> > --- a/include/media/hevc-ctrls.h
> > +++ b/include/media/hevc-ctrls.h
> > @@ -20,6 +20,7 @@
> > #define V4L2_CID_STATELESS_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_BASE +
> 1012)
> > #define V4L2_CID_STATELESS_HEVC_DECODE_MODE (V4L2_CID_CODEC_BASE +
> 1015)
> > #define V4L2_CID_STATELESS_HEVC_START_CODE (V4L2_CID_CODEC_BASE +
1016)
> > +#define V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET (V4L2_CID_CODEC_BASE +
> 1017)
>
> ... and here you have OFFSET (singlular). I suggest plural form to be used
in
> all places, including subject line of this commit.
>
> Additionally, it would be nice if control is initialized, like so:
> https://github.com/jernejsk/linux-1/commit/
> f938e162cd8dd77c9f6f1b248d80144840a37bce
>
> Best regards,
> Jernej
>
> >
> > /* enum v4l2_ctrl_type type values */
> > #define V4L2_CTRL_TYPE_HEVC_SPS 0x0120
> > --
> > 2.32.0
> >
> >
>
>
>
Dne sobota, 26. februar 2022 ob 18:59:18 CET je Jernej Škrabec napisal(a):
> Hi!
>
> Dne petek, 25. februar 2022 ob 20:30:20 CET je Jernej Škrabec napisal(a):
> > Hi!
> >
> > Dne petek, 25. februar 2022 ob 17:45:55 CET je Benjamin Gaignard
napisal(a):
> > > The number of 'entry point offset' could be very variable.
> > > Rather than use a large static array define a v4l2 dynamic array
> > > of integer control.
> >
> > I suggest we should be more specific and say U32 (V4L2_CTRL_TYPE_U32).
> >
> > > The number of entry point offsets is reported by the elems field.
>
> I did few more tests and these are my findings:
> 1. dynamic array can't be set over size, specified in .dims array
> 2. entry point offsets are per slice, so if we make slices dynamic array,
then
> entry points would become two dimensional dynamic array
> 3. num_entry_point_offsets must be part of slice control, because it can be
> zero, but size of array can't be
Clarification: num_entry_point_offsets is needed in any case, even without entry
point list.
> 4. fortunately, not setting entry points doesn't impact decoding
correctness.
> This is in line what John told me about them.
>
> Hans, can you comment points 1-3? I might misunderstand point 1.
>
> In short, it seems like we don't really need entry points, even if they are
> used in BSP library. In both cases, I got fluster score 119/138 (10-bit
> excluded), so we can just drop this patch, although I'm a bit uneasy not
> setting entry points...
>
> Best regards,
> Jernej
>
> > >
> > > Signed-off-by: Benjamin Gaignard <[email protected]>
> > > ---
> > > .../userspace-api/media/v4l/ext-ctrls-codec.rst | 9 +++++++++
> > > include/media/hevc-ctrls.h | 1 +
> > > 2 files changed, 10 insertions(+)
> > >
> > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/
> > Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > > index 44a268a948c0..71f7dc1c1ccd 100644
> > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > > @@ -3128,6 +3128,15 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> > >
> > > \normalsize
> > >
> > > +``V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS (integer)``
> >
> > Here you have OFFSETS (plural) ...
> >
> > > + Specifies the i-th entry point offset in bytes and is represented by
> > > + offset_len_minus1 plus 1 bits.
> >
> > You probably mean entry_point_offset_minus1? offset_len_minus1 just tells
how
> > much bits need to be read for each element and it's not important for
actual
> > decoding.
> >
> > > + This control is a dynamically sized array. The number of entry
point
> > > + offsets is reported by the ``elems`` field.
> > > + This bitstream parameter is defined according to :ref:`hevc`.
> > > + They are described in section 7.4.7.1 "General slice segment header
> > > + semantics" of the specification.
> > > +
> > > ``V4L2_CID_STATELESS_HEVC_SCALING_MATRIX (struct)``
> > > Specifies the HEVC scaling matrix parameters used for the scaling
> > process
> > > for transform coefficients.
> > > diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
> > > index 3016c1abb1d0..3f8a67924df3 100644
> > > --- a/include/media/hevc-ctrls.h
> > > +++ b/include/media/hevc-ctrls.h
> > > @@ -20,6 +20,7 @@
> > > #define V4L2_CID_STATELESS_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_BASE +
> > 1012)
> > > #define V4L2_CID_STATELESS_HEVC_DECODE_MODE (V4L2_CID_CODEC_BASE +
> > 1015)
> > > #define V4L2_CID_STATELESS_HEVC_START_CODE (V4L2_CID_CODEC_BASE
+
> 1016)
> > > +#define V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET (V4L2_CID_CODEC_BASE
+
> > 1017)
> >
> > ... and here you have OFFSET (singlular). I suggest plural form to be used
> in
> > all places, including subject line of this commit.
> >
> > Additionally, it would be nice if control is initialized, like so:
> > https://github.com/jernejsk/linux-1/commit/
> > f938e162cd8dd77c9f6f1b248d80144840a37bce
> >
> > Best regards,
> > Jernej
> >
> > >
> > > /* enum v4l2_ctrl_type type values */
> > > #define V4L2_CTRL_TYPE_HEVC_SPS 0x0120
> > > --
> > > 2.32.0
> > >
> > >
> >
> >
> >
>
>
>
On Fri, Feb 25, 2022 at 4:41 PM Benjamin Gaignard
<[email protected]> wrote:
>
> This series aims to make HEVC uapi stable and usable for hardware
> decoder. HEVC uapi is used by 2 mainlined drivers (Cedrus and Hantro)
> and 2 out of the tree drivers (rkvdec and RPI).
>
> After the remarks done on version 2, I have completely reworked to patches
> split so changelogs are meaningless. I have also drop "RFC" from the
> titles.
>
> In this v3 I do all the changes (new controls, documentation, etc..)
> in the staging directory before moving the HEVC uAPI to stable
> steps by steps (unlike the big one patch in v2).
>
> At the end fluster tests results on IMX8MQ is 77/147 for HEVC codec.
Benjamin,
I have an imx8mm and imx8mq that I can test. Do you happen to have a
repo that I can clone to test this? The imx8m stuff is spread around
between the media tree and the imx tree since it hasn't been fully
merged yet.
thanks,
adam
>
> Benjamin
>
> Benjamin Gaignard (11):
> media: uapi: HEVC: Add missing fields in HEVC controls
> media: uapi: HEVC: Rename HEVC stateless controls with STATELESS
> prefix
> media: uapi: HEVC: Add document uAPI structure
> media: uapi: HEVC: Define V4L2_CID_STATELESS_HEVC_SLICE_PARAMS as a
> dynamic array
> media: uapi: Move parsed HEVC pixel format out of staging
> media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control
> media: uapi: Move the HEVC stateless control type out of staging
> media: controls: Log HEVC stateless control in .std_log
> media: uapi: Create a dedicated header for Hantro control
> media: uapi: HEVC: fix padding in v4l2 control structures
> media: uapi: move HEVC stateless controls out of staging
>
> Hans Verkuil (3):
> videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY
> v4l2-ctrls: add support for dynamically allocated arrays.
> vivid: add dynamic array test control
>
> .../userspace-api/media/drivers/hantro.rst | 5 -
> .../media/v4l/ext-ctrls-codec-stateless.rst | 831 ++++++++++++++++++
> .../media/v4l/ext-ctrls-codec.rst | 780 ----------------
> .../media/v4l/pixfmt-compressed.rst | 7 +-
> .../media/v4l/vidioc-g-ext-ctrls.rst | 20 +
> .../media/v4l/vidioc-queryctrl.rst | 8 +
> .../media/videodev2.h.rst.exceptions | 5 +
> .../media/test-drivers/vivid/vivid-ctrls.c | 15 +
> drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++-
> drivers/media/v4l2-core/v4l2-ctrls-core.c | 198 ++++-
> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 32 +-
> drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +-
> drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +-
> drivers/staging/media/hantro/hantro_drv.c | 27 +-
> drivers/staging/media/hantro/hantro_hevc.c | 8 +-
> drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +-
> .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +-
> include/media/hevc-ctrls.h | 250 ------
> include/media/v4l2-ctrls.h | 48 +-
> include/uapi/linux/hantro-media.h | 19 +
> include/uapi/linux/v4l2-controls.h | 436 +++++++++
> include/uapi/linux/videodev2.h | 13 +
> 22 files changed, 1686 insertions(+), 1169 deletions(-)
> delete mode 100644 include/media/hevc-ctrls.h
> create mode 100644 include/uapi/linux/hantro-media.h
>
> --
> 2.32.0
>
Le 28/02/2022 à 10:47, John Cox a écrit :
> On Sat, 26 Feb 2022 19:17:46 +0100, you wrote:
>
>> Dne sobota, 26. februar 2022 ob 18:59:18 CET je Jernej Škrabec napisal(a):
>>> Hi!
>>>
>>> Dne petek, 25. februar 2022 ob 20:30:20 CET je Jernej Škrabec napisal(a):
>>>> Hi!
>>>>
>>>> Dne petek, 25. februar 2022 ob 17:45:55 CET je Benjamin Gaignard
>> napisal(a):
>>>>> The number of 'entry point offset' could be very variable.
>>>>> Rather than use a large static array define a v4l2 dynamic array
>>>>> of integer control.
>>>> I suggest we should be more specific and say U32 (V4L2_CTRL_TYPE_U32).
>>>>
>>>>> The number of entry point offsets is reported by the elems field.
>>> I did few more tests and these are my findings:
>>> 1. dynamic array can't be set over size, specified in .dims array
>>> 2. entry point offsets are per slice, so if we make slices dynamic array,
>> then
>>> entry points would become two dimensional dynamic array
>>> 3. num_entry_point_offsets must be part of slice control, because it can be
>>> zero, but size of array can't be
>> Clarification: num_entry_point_offsets is needed in any case, even without entry
>> point list.
> I thought that at least one decoder wanted the entry points or have we
> decided that we don't? Whilst they aren't required for decode, what they
> give you, if your h/w supports it, is the ability to decode multiple
> lines of CTBs in parallel (with a 2 CTB offset between lines), so they
> aren't useless.
>
> If we do decided we want the entry point offset control, now or later,
> then num_entry_points in the slice_param allows you to determine which
> offsets apply to which slices without the complexity of a variable array
> of variable arrays.
>
> But whether or not we end up supporting entry point offsets, the Pi h/w
> requires me to generate 1 table entry for each CTB line and
> num_entry_points is the parameter I need for that, so I really want it
> in the slice_params.
I'm adding a control (dynamic array) and num_entry_points field in slice
parameters in the next version.
See here:
https://gitlab.collabora.com/benjamin.gaignard/for-upstream/-/commit/6b4efa4dc54de069893f5203d885fea2c1347916
Regards,
Benjamin
> Regards
>
> John Cox
>
>>> 4. fortunately, not setting entry points doesn't impact decoding
>> correctness.
>>> This is in line what John told me about them.
>>>
>>> Hans, can you comment points 1-3? I might misunderstand point 1.
>>>
>>> In short, it seems like we don't really need entry points, even if they are
>>> used in BSP library. In both cases, I got fluster score 119/138 (10-bit
>>> excluded), so we can just drop this patch, although I'm a bit uneasy not
>>> setting entry points...
>>>
>>> Best regards,
>>> Jernej
>>>
>>>>> Signed-off-by: Benjamin Gaignard <[email protected]>
>>>>> ---
>>>>> .../userspace-api/media/v4l/ext-ctrls-codec.rst | 9 +++++++++
>>>>> include/media/hevc-ctrls.h | 1 +
>>>>> 2 files changed, 10 insertions(+)
>>>>>
>>>>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/
>>>> Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>>>> index 44a268a948c0..71f7dc1c1ccd 100644
>>>>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>>>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>>>> @@ -3128,6 +3128,15 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>>>>>
>>>>> \normalsize
>>>>>
>>>>> +``V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS (integer)``
>>>> Here you have OFFSETS (plural) ...
>>>>
>>>>> + Specifies the i-th entry point offset in bytes and is represented by
>>>>> + offset_len_minus1 plus 1 bits.
>>>> You probably mean entry_point_offset_minus1? offset_len_minus1 just tells
>> how
>>>> much bits need to be read for each element and it's not important for
>> actual
>>>> decoding.
>>>>
>>>>> + This control is a dynamically sized array. The number of entry
>> point
>>>>> + offsets is reported by the ``elems`` field.
>>>>> + This bitstream parameter is defined according to :ref:`hevc`.
>>>>> + They are described in section 7.4.7.1 "General slice segment header
>>>>> + semantics" of the specification.
>>>>> +
>>>>> ``V4L2_CID_STATELESS_HEVC_SCALING_MATRIX (struct)``
>>>>> Specifies the HEVC scaling matrix parameters used for the scaling
>>>> process
>>>>> for transform coefficients.
>>>>> diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
>>>>> index 3016c1abb1d0..3f8a67924df3 100644
>>>>> --- a/include/media/hevc-ctrls.h
>>>>> +++ b/include/media/hevc-ctrls.h
>>>>> @@ -20,6 +20,7 @@
>>>>> #define V4L2_CID_STATELESS_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_BASE +
>>>> 1012)
>>>>> #define V4L2_CID_STATELESS_HEVC_DECODE_MODE (V4L2_CID_CODEC_BASE +
>>>> 1015)
>>>>> #define V4L2_CID_STATELESS_HEVC_START_CODE (V4L2_CID_CODEC_BASE
>> +
>>> 1016)
>>>>> +#define V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET (V4L2_CID_CODEC_BASE
>> +
>>>> 1017)
>>>>
>>>> ... and here you have OFFSET (singlular). I suggest plural form to be used
>>> in
>>>> all places, including subject line of this commit.
>>>>
>>>> Additionally, it would be nice if control is initialized, like so:
>>>> https://github.com/jernejsk/linux-1/commit/
>>>> f938e162cd8dd77c9f6f1b248d80144840a37bce
>>>>
>>>> Best regards,
>>>> Jernej
>>>>
>>>>>
>>>>> /* enum v4l2_ctrl_type type values */
>>>>> #define V4L2_CTRL_TYPE_HEVC_SPS 0x0120
>>>>> --
>>>>> 2.32.0
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
On Sat, 26 Feb 2022 19:17:46 +0100, you wrote:
>Dne sobota, 26. februar 2022 ob 18:59:18 CET je Jernej Škrabec napisal(a):
>> Hi!
>>
>> Dne petek, 25. februar 2022 ob 20:30:20 CET je Jernej Škrabec napisal(a):
>> > Hi!
>> >
>> > Dne petek, 25. februar 2022 ob 17:45:55 CET je Benjamin Gaignard
>napisal(a):
>> > > The number of 'entry point offset' could be very variable.
>> > > Rather than use a large static array define a v4l2 dynamic array
>> > > of integer control.
>> >
>> > I suggest we should be more specific and say U32 (V4L2_CTRL_TYPE_U32).
>> >
>> > > The number of entry point offsets is reported by the elems field.
>>
>> I did few more tests and these are my findings:
>> 1. dynamic array can't be set over size, specified in .dims array
>> 2. entry point offsets are per slice, so if we make slices dynamic array,
>then
>> entry points would become two dimensional dynamic array
>> 3. num_entry_point_offsets must be part of slice control, because it can be
>> zero, but size of array can't be
>
>Clarification: num_entry_point_offsets is needed in any case, even without entry
>point list.
I thought that at least one decoder wanted the entry points or have we
decided that we don't? Whilst they aren't required for decode, what they
give you, if your h/w supports it, is the ability to decode multiple
lines of CTBs in parallel (with a 2 CTB offset between lines), so they
aren't useless.
If we do decided we want the entry point offset control, now or later,
then num_entry_points in the slice_param allows you to determine which
offsets apply to which slices without the complexity of a variable array
of variable arrays.
But whether or not we end up supporting entry point offsets, the Pi h/w
requires me to generate 1 table entry for each CTB line and
num_entry_points is the parameter I need for that, so I really want it
in the slice_params.
Regards
John Cox
>> 4. fortunately, not setting entry points doesn't impact decoding
>correctness.
>> This is in line what John told me about them.
>>
>> Hans, can you comment points 1-3? I might misunderstand point 1.
>>
>> In short, it seems like we don't really need entry points, even if they are
>> used in BSP library. In both cases, I got fluster score 119/138 (10-bit
>> excluded), so we can just drop this patch, although I'm a bit uneasy not
>> setting entry points...
>>
>> Best regards,
>> Jernej
>>
>> > >
>> > > Signed-off-by: Benjamin Gaignard <[email protected]>
>> > > ---
>> > > .../userspace-api/media/v4l/ext-ctrls-codec.rst | 9 +++++++++
>> > > include/media/hevc-ctrls.h | 1 +
>> > > 2 files changed, 10 insertions(+)
>> > >
>> > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/
>> > Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> > > index 44a268a948c0..71f7dc1c1ccd 100644
>> > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> > > @@ -3128,6 +3128,15 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>> > >
>> > > \normalsize
>> > >
>> > > +``V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS (integer)``
>> >
>> > Here you have OFFSETS (plural) ...
>> >
>> > > + Specifies the i-th entry point offset in bytes and is represented by
>> > > + offset_len_minus1 plus 1 bits.
>> >
>> > You probably mean entry_point_offset_minus1? offset_len_minus1 just tells
>how
>> > much bits need to be read for each element and it's not important for
>actual
>> > decoding.
>> >
>> > > + This control is a dynamically sized array. The number of entry
>point
>> > > + offsets is reported by the ``elems`` field.
>> > > + This bitstream parameter is defined according to :ref:`hevc`.
>> > > + They are described in section 7.4.7.1 "General slice segment header
>> > > + semantics" of the specification.
>> > > +
>> > > ``V4L2_CID_STATELESS_HEVC_SCALING_MATRIX (struct)``
>> > > Specifies the HEVC scaling matrix parameters used for the scaling
>> > process
>> > > for transform coefficients.
>> > > diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
>> > > index 3016c1abb1d0..3f8a67924df3 100644
>> > > --- a/include/media/hevc-ctrls.h
>> > > +++ b/include/media/hevc-ctrls.h
>> > > @@ -20,6 +20,7 @@
>> > > #define V4L2_CID_STATELESS_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_BASE +
>> > 1012)
>> > > #define V4L2_CID_STATELESS_HEVC_DECODE_MODE (V4L2_CID_CODEC_BASE +
>> > 1015)
>> > > #define V4L2_CID_STATELESS_HEVC_START_CODE (V4L2_CID_CODEC_BASE
>+
>> 1016)
>> > > +#define V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET (V4L2_CID_CODEC_BASE
>+
>> > 1017)
>> >
>> > ... and here you have OFFSET (singlular). I suggest plural form to be used
>> in
>> > all places, including subject line of this commit.
>> >
>> > Additionally, it would be nice if control is initialized, like so:
>> > https://github.com/jernejsk/linux-1/commit/
>> > f938e162cd8dd77c9f6f1b248d80144840a37bce
>> >
>> > Best regards,
>> > Jernej
>> >
>> > >
>> > > /* enum v4l2_ctrl_type type values */
>> > > #define V4L2_CTRL_TYPE_HEVC_SPS 0x0120
>> > > --
>> > > 2.32.0
>> > >
>> > >
>> >
>> >
>> >
>>
>>
>>
>
Le 26/02/2022 à 23:25, Adam Ford a écrit :
> On Fri, Feb 25, 2022 at 4:41 PM Benjamin Gaignard
> <[email protected]> wrote:
>> This series aims to make HEVC uapi stable and usable for hardware
>> decoder. HEVC uapi is used by 2 mainlined drivers (Cedrus and Hantro)
>> and 2 out of the tree drivers (rkvdec and RPI).
>>
>> After the remarks done on version 2, I have completely reworked to patches
>> split so changelogs are meaningless. I have also drop "RFC" from the
>> titles.
>>
>> In this v3 I do all the changes (new controls, documentation, etc..)
>> in the staging directory before moving the HEVC uAPI to stable
>> steps by steps (unlike the big one patch in v2).
>>
>> At the end fluster tests results on IMX8MQ is 77/147 for HEVC codec.
I have push a branch here:
https://gitlab.collabora.com/benjamin.gaignard/for-upstream/-/commits/HEVC_UAPI_V4
it is the incoming version 4 of this series + patches to enable G2 on my IMX8MQ
Regards,
Benjamin
> Benjamin,
>
> I have an imx8mm and imx8mq that I can test. Do you happen to have a
> repo that I can clone to test this? The imx8m stuff is spread around
> between the media tree and the imx tree since it hasn't been fully
> merged yet.
>
> thanks,
>
> adam
>
>> Benjamin
>>
>> Benjamin Gaignard (11):
>> media: uapi: HEVC: Add missing fields in HEVC controls
>> media: uapi: HEVC: Rename HEVC stateless controls with STATELESS
>> prefix
>> media: uapi: HEVC: Add document uAPI structure
>> media: uapi: HEVC: Define V4L2_CID_STATELESS_HEVC_SLICE_PARAMS as a
>> dynamic array
>> media: uapi: Move parsed HEVC pixel format out of staging
>> media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control
>> media: uapi: Move the HEVC stateless control type out of staging
>> media: controls: Log HEVC stateless control in .std_log
>> media: uapi: Create a dedicated header for Hantro control
>> media: uapi: HEVC: fix padding in v4l2 control structures
>> media: uapi: move HEVC stateless controls out of staging
>>
>> Hans Verkuil (3):
>> videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY
>> v4l2-ctrls: add support for dynamically allocated arrays.
>> vivid: add dynamic array test control
>>
>> .../userspace-api/media/drivers/hantro.rst | 5 -
>> .../media/v4l/ext-ctrls-codec-stateless.rst | 831 ++++++++++++++++++
>> .../media/v4l/ext-ctrls-codec.rst | 780 ----------------
>> .../media/v4l/pixfmt-compressed.rst | 7 +-
>> .../media/v4l/vidioc-g-ext-ctrls.rst | 20 +
>> .../media/v4l/vidioc-queryctrl.rst | 8 +
>> .../media/videodev2.h.rst.exceptions | 5 +
>> .../media/test-drivers/vivid/vivid-ctrls.c | 15 +
>> drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++-
>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 198 ++++-
>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 32 +-
>> drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +-
>> drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +-
>> drivers/staging/media/hantro/hantro_drv.c | 27 +-
>> drivers/staging/media/hantro/hantro_hevc.c | 8 +-
>> drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +-
>> .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +-
>> include/media/hevc-ctrls.h | 250 ------
>> include/media/v4l2-ctrls.h | 48 +-
>> include/uapi/linux/hantro-media.h | 19 +
>> include/uapi/linux/v4l2-controls.h | 436 +++++++++
>> include/uapi/linux/videodev2.h | 13 +
>> 22 files changed, 1686 insertions(+), 1169 deletions(-)
>> delete mode 100644 include/media/hevc-ctrls.h
>> create mode 100644 include/uapi/linux/hantro-media.h
>>
>> --
>> 2.32.0
>>
On Mon, Feb 28, 2022 at 4:13 AM Benjamin Gaignard
<[email protected]> wrote:
>
>
> Le 26/02/2022 à 23:25, Adam Ford a écrit :
> > On Fri, Feb 25, 2022 at 4:41 PM Benjamin Gaignard
> > <[email protected]> wrote:
> >> This series aims to make HEVC uapi stable and usable for hardware
> >> decoder. HEVC uapi is used by 2 mainlined drivers (Cedrus and Hantro)
> >> and 2 out of the tree drivers (rkvdec and RPI).
> >>
> >> After the remarks done on version 2, I have completely reworked to patches
> >> split so changelogs are meaningless. I have also drop "RFC" from the
> >> titles.
> >>
> >> In this v3 I do all the changes (new controls, documentation, etc..)
> >> in the staging directory before moving the HEVC uAPI to stable
> >> steps by steps (unlike the big one patch in v2).
> >>
> >> At the end fluster tests results on IMX8MQ is 77/147 for HEVC codec.
>
> I have push a branch here:
> https://gitlab.collabora.com/benjamin.gaignard/for-upstream/-/commits/HEVC_UAPI_V4
>
> it is the incoming version 4 of this series + patches to enable G2 on my IMX8MQ
Benjamin,
I checked this repo out, and built it along with pulling the latest
versions of G-streamer and fluster.
When I check for v4l2 compatibility, I get the following:
GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for
GStreamer 1.0... ❌
GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for
GStreamer 1.0... ✔️
GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for
GStreamer 1.0... ❌
GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for
GStreamer 1.0... ❌
GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for
GStreamer 1.0... ❌
GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌
GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for
GStreamer 1.0... ✔️
GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for
GStreamer 1.0... ✔️
I see H,264, VP8 and VP9, but I would have also expected
GStreamer-H.265-V4L2SL-Gst1.0 to return with a check box.
When I checked to see if both decoders were being enumerated, I found they were.
[gst-main] root@localhost:~/gstreamer/fluster# dmesg |grep -i hantro
[ 16.044243] hantro_vpu: module is from the staging directory, the
quality is unknown, you have been warned.
[ 16.044243] hantro_vpu: module is from the staging directory, the
quality is unknown, you have been warned.
[ 16.095661] hantro-vpu 38300000.video-codec: registered
nxp,imx8mq-vpu-g1-dec as /dev/video0
[ 16.096782] hantro-vpu 38310000.video-codec: registered
nxp,imx8mq-vpu-g2-dec as /dev/video1
Did I do something wrong, or did I miss something?
adam
>
> Regards,
> Benjamin
>
> > Benjamin,
> >
> > I have an imx8mm and imx8mq that I can test. Do you happen to have a
> > repo that I can clone to test this? The imx8m stuff is spread around
> > between the media tree and the imx tree since it hasn't been fully
> > merged yet.
> >
> > thanks,
> >
> > adam
> >
> >> Benjamin
> >>
> >> Benjamin Gaignard (11):
> >> media: uapi: HEVC: Add missing fields in HEVC controls
> >> media: uapi: HEVC: Rename HEVC stateless controls with STATELESS
> >> prefix
> >> media: uapi: HEVC: Add document uAPI structure
> >> media: uapi: HEVC: Define V4L2_CID_STATELESS_HEVC_SLICE_PARAMS as a
> >> dynamic array
> >> media: uapi: Move parsed HEVC pixel format out of staging
> >> media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control
> >> media: uapi: Move the HEVC stateless control type out of staging
> >> media: controls: Log HEVC stateless control in .std_log
> >> media: uapi: Create a dedicated header for Hantro control
> >> media: uapi: HEVC: fix padding in v4l2 control structures
> >> media: uapi: move HEVC stateless controls out of staging
> >>
> >> Hans Verkuil (3):
> >> videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY
> >> v4l2-ctrls: add support for dynamically allocated arrays.
> >> vivid: add dynamic array test control
> >>
> >> .../userspace-api/media/drivers/hantro.rst | 5 -
> >> .../media/v4l/ext-ctrls-codec-stateless.rst | 831 ++++++++++++++++++
> >> .../media/v4l/ext-ctrls-codec.rst | 780 ----------------
> >> .../media/v4l/pixfmt-compressed.rst | 7 +-
> >> .../media/v4l/vidioc-g-ext-ctrls.rst | 20 +
> >> .../media/v4l/vidioc-queryctrl.rst | 8 +
> >> .../media/videodev2.h.rst.exceptions | 5 +
> >> .../media/test-drivers/vivid/vivid-ctrls.c | 15 +
> >> drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++-
> >> drivers/media/v4l2-core/v4l2-ctrls-core.c | 198 ++++-
> >> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 32 +-
> >> drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +-
> >> drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +-
> >> drivers/staging/media/hantro/hantro_drv.c | 27 +-
> >> drivers/staging/media/hantro/hantro_hevc.c | 8 +-
> >> drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +-
> >> .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +-
> >> include/media/hevc-ctrls.h | 250 ------
> >> include/media/v4l2-ctrls.h | 48 +-
> >> include/uapi/linux/hantro-media.h | 19 +
> >> include/uapi/linux/v4l2-controls.h | 436 +++++++++
> >> include/uapi/linux/videodev2.h | 13 +
> >> 22 files changed, 1686 insertions(+), 1169 deletions(-)
> >> delete mode 100644 include/media/hevc-ctrls.h
> >> create mode 100644 include/uapi/linux/hantro-media.h
> >>
> >> --
> >> 2.32.0
> >>
Le 03/03/2022 à 02:23, Adam Ford a écrit :
> On Mon, Feb 28, 2022 at 4:13 AM Benjamin Gaignard
> <[email protected]> wrote:
>>
>> Le 26/02/2022 à 23:25, Adam Ford a écrit :
>>> On Fri, Feb 25, 2022 at 4:41 PM Benjamin Gaignard
>>> <[email protected]> wrote:
>>>> This series aims to make HEVC uapi stable and usable for hardware
>>>> decoder. HEVC uapi is used by 2 mainlined drivers (Cedrus and Hantro)
>>>> and 2 out of the tree drivers (rkvdec and RPI).
>>>>
>>>> After the remarks done on version 2, I have completely reworked to patches
>>>> split so changelogs are meaningless. I have also drop "RFC" from the
>>>> titles.
>>>>
>>>> In this v3 I do all the changes (new controls, documentation, etc..)
>>>> in the staging directory before moving the HEVC uAPI to stable
>>>> steps by steps (unlike the big one patch in v2).
>>>>
>>>> At the end fluster tests results on IMX8MQ is 77/147 for HEVC codec.
>> I have push a branch here:
>> https://gitlab.collabora.com/benjamin.gaignard/for-upstream/-/commits/HEVC_UAPI_V4
>>
>> it is the incoming version 4 of this series + patches to enable G2 on my IMX8MQ
> Benjamin,
>
> I checked this repo out, and built it along with pulling the latest
> versions of G-streamer and fluster.
> When I check for v4l2 compatibility, I get the following:
>
> GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for
> GStreamer 1.0... ❌
> GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for
> GStreamer 1.0... ✔️
> GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for
> GStreamer 1.0... ❌
> GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for
> GStreamer 1.0... ❌
> GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for
> GStreamer 1.0... ❌
> GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌
> GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for
> GStreamer 1.0... ✔️
> GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for
> GStreamer 1.0... ✔️
>
> I see H,264, VP8 and VP9, but I would have also expected
> GStreamer-H.265-V4L2SL-Gst1.0 to return with a check box.
>
> When I checked to see if both decoders were being enumerated, I found they were.
> [gst-main] root@localhost:~/gstreamer/fluster# dmesg |grep -i hantro
> [ 16.044243] hantro_vpu: module is from the staging directory, the
> quality is unknown, you have been warned.
> [ 16.044243] hantro_vpu: module is from the staging directory, the
> quality is unknown, you have been warned.
> [ 16.095661] hantro-vpu 38300000.video-codec: registered
> nxp,imx8mq-vpu-g1-dec as /dev/video0
> [ 16.096782] hantro-vpu 38310000.video-codec: registered
> nxp,imx8mq-vpu-g2-dec as /dev/video1
>
> Did I do something wrong, or did I miss something?
Hi Adam,
I guess it could be a misalignment between v4l2 kernel headers
and GStreamer v4l2 headers.
I have push a new version of the GST merge request which is aligned with
HEVC uAPI v4 proposal.
When you inspect v4l2codecs gst plugin (gst-inspect-1.0 v4l2codecs) do
you see v4l2slh265dec plugin ?
I have fluster happy with it:
./fluster.py list -c
H265
...
GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for GStreamer 1.0... ❌
GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for GStreamer 1.0... ✔️
GStreamer-H.265-VA-Gst1.0: GStreamer H.265 VA decoder for GStreamer 1.0... ❌
GStreamer-H.265-VAAPI-Gst1.0: GStreamer H.265 VAAPI decoder for GStreamer 1.0... ❌
JCT-VT-H.265: JCT-VT H.265/HEVC reference decoder... ❌
I hope that will help you,
Regards,
Benjamin
>
> adam
>
>> Regards,
>> Benjamin
>>
>>> Benjamin,
>>>
>>> I have an imx8mm and imx8mq that I can test. Do you happen to have a
>>> repo that I can clone to test this? The imx8m stuff is spread around
>>> between the media tree and the imx tree since it hasn't been fully
>>> merged yet.
>>>
>>> thanks,
>>>
>>> adam
>>>
>>>> Benjamin
>>>>
>>>> Benjamin Gaignard (11):
>>>> media: uapi: HEVC: Add missing fields in HEVC controls
>>>> media: uapi: HEVC: Rename HEVC stateless controls with STATELESS
>>>> prefix
>>>> media: uapi: HEVC: Add document uAPI structure
>>>> media: uapi: HEVC: Define V4L2_CID_STATELESS_HEVC_SLICE_PARAMS as a
>>>> dynamic array
>>>> media: uapi: Move parsed HEVC pixel format out of staging
>>>> media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control
>>>> media: uapi: Move the HEVC stateless control type out of staging
>>>> media: controls: Log HEVC stateless control in .std_log
>>>> media: uapi: Create a dedicated header for Hantro control
>>>> media: uapi: HEVC: fix padding in v4l2 control structures
>>>> media: uapi: move HEVC stateless controls out of staging
>>>>
>>>> Hans Verkuil (3):
>>>> videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY
>>>> v4l2-ctrls: add support for dynamically allocated arrays.
>>>> vivid: add dynamic array test control
>>>>
>>>> .../userspace-api/media/drivers/hantro.rst | 5 -
>>>> .../media/v4l/ext-ctrls-codec-stateless.rst | 831 ++++++++++++++++++
>>>> .../media/v4l/ext-ctrls-codec.rst | 780 ----------------
>>>> .../media/v4l/pixfmt-compressed.rst | 7 +-
>>>> .../media/v4l/vidioc-g-ext-ctrls.rst | 20 +
>>>> .../media/v4l/vidioc-queryctrl.rst | 8 +
>>>> .../media/videodev2.h.rst.exceptions | 5 +
>>>> .../media/test-drivers/vivid/vivid-ctrls.c | 15 +
>>>> drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++-
>>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 198 ++++-
>>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 32 +-
>>>> drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +-
>>>> drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +-
>>>> drivers/staging/media/hantro/hantro_drv.c | 27 +-
>>>> drivers/staging/media/hantro/hantro_hevc.c | 8 +-
>>>> drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +-
>>>> .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +-
>>>> include/media/hevc-ctrls.h | 250 ------
>>>> include/media/v4l2-ctrls.h | 48 +-
>>>> include/uapi/linux/hantro-media.h | 19 +
>>>> include/uapi/linux/v4l2-controls.h | 436 +++++++++
>>>> include/uapi/linux/videodev2.h | 13 +
>>>> 22 files changed, 1686 insertions(+), 1169 deletions(-)
>>>> delete mode 100644 include/media/hevc-ctrls.h
>>>> create mode 100644 include/uapi/linux/hantro-media.h
>>>>
>>>> --
>>>> 2.32.0
>>>>
On Thu, Mar 3, 2022 at 4:13 AM Benjamin Gaignard
<[email protected]> wrote:
>
>
> Le 03/03/2022 à 02:23, Adam Ford a écrit :
> > On Mon, Feb 28, 2022 at 4:13 AM Benjamin Gaignard
> > <[email protected]> wrote:
> >>
> >> Le 26/02/2022 à 23:25, Adam Ford a écrit :
> >>> On Fri, Feb 25, 2022 at 4:41 PM Benjamin Gaignard
> >>> <[email protected]> wrote:
> >>>> This series aims to make HEVC uapi stable and usable for hardware
> >>>> decoder. HEVC uapi is used by 2 mainlined drivers (Cedrus and Hantro)
> >>>> and 2 out of the tree drivers (rkvdec and RPI).
> >>>>
> >>>> After the remarks done on version 2, I have completely reworked to patches
> >>>> split so changelogs are meaningless. I have also drop "RFC" from the
> >>>> titles.
> >>>>
> >>>> In this v3 I do all the changes (new controls, documentation, etc..)
> >>>> in the staging directory before moving the HEVC uAPI to stable
> >>>> steps by steps (unlike the big one patch in v2).
> >>>>
> >>>> At the end fluster tests results on IMX8MQ is 77/147 for HEVC codec.
> >> I have push a branch here:
> >> https://gitlab.collabora.com/benjamin.gaignard/for-upstream/-/commits/HEVC_UAPI_V4
> >>
> >> it is the incoming version 4 of this series + patches to enable G2 on my IMX8MQ
> > Benjamin,
> >
> > I checked this repo out, and built it along with pulling the latest
> > versions of G-streamer and fluster.
> > When I check for v4l2 compatibility, I get the following:
> >
> > GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for
> > GStreamer 1.0... ❌
> > GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for
> > GStreamer 1.0... ✔️
> > GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for
> > GStreamer 1.0... ❌
> > GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for
> > GStreamer 1.0... ❌
> > GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for
> > GStreamer 1.0... ❌
> > GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌
> > GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for
> > GStreamer 1.0... ✔️
> > GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for
> > GStreamer 1.0... ✔️
> >
> > I see H,264, VP8 and VP9, but I would have also expected
> > GStreamer-H.265-V4L2SL-Gst1.0 to return with a check box.
> >
> > When I checked to see if both decoders were being enumerated, I found they were.
> > [gst-main] root@localhost:~/gstreamer/fluster# dmesg |grep -i hantro
> > [ 16.044243] hantro_vpu: module is from the staging directory, the
> > quality is unknown, you have been warned.
> > [ 16.044243] hantro_vpu: module is from the staging directory, the
> > quality is unknown, you have been warned.
> > [ 16.095661] hantro-vpu 38300000.video-codec: registered
> > nxp,imx8mq-vpu-g1-dec as /dev/video0
> > [ 16.096782] hantro-vpu 38310000.video-codec: registered
> > nxp,imx8mq-vpu-g2-dec as /dev/video1
> >
> > Did I do something wrong, or did I miss something?
>
> Hi Adam,
>
> I guess it could be a misalignment between v4l2 kernel headers
> and GStreamer v4l2 headers.
> I have push a new version of the GST merge request which is aligned with
> HEVC uAPI v4 proposal.
>
> When you inspect v4l2codecs gst plugin (gst-inspect-1.0 v4l2codecs) do
> you see v4l2slh265dec plugin ?
> I have fluster happy with it:
> ./fluster.py list -c
I pulled the latest from g-streamer's git repo again, and re-built.
gst-instpect does not appear to show the h265 codec anymore.
# gst-inspect-1.0 |grep v4l2codecs
v4l2codecs: v4l2slh264dec: V4L2 Stateless H.264 Video Decoder
v4l2codecs: v4l2slmpeg2dec: V4L2 Stateless Mpeg2 Video Decoder
v4l2codecs: v4l2slvp8alphadecodebin: VP8 Alpha Decoder
v4l2codecs: v4l2slvp8dec: V4L2 Stateless VP8 Video Decoder
v4l2codecs: v4l2slvp9alphadecodebin: VP9 Alpha Decoder
v4l2codecs: v4l2slvp9dec: V4L2 Stateless VP9 Video Decoder
Is there a specific branch of g-streamer I should be using that can
handle the newer API?
adam
>
>
> H265
> ...
> GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for GStreamer 1.0... ❌
> GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for GStreamer 1.0... ✔️
> GStreamer-H.265-VA-Gst1.0: GStreamer H.265 VA decoder for GStreamer 1.0... ❌
> GStreamer-H.265-VAAPI-Gst1.0: GStreamer H.265 VAAPI decoder for GStreamer 1.0... ❌
> JCT-VT-H.265: JCT-VT H.265/HEVC reference decoder... ❌
>
> I hope that will help you,
>
> Regards,
> Benjamin
>
> >
> > adam
> >
> >> Regards,
> >> Benjamin
> >>
> >>> Benjamin,
> >>>
> >>> I have an imx8mm and imx8mq that I can test. Do you happen to have a
> >>> repo that I can clone to test this? The imx8m stuff is spread around
> >>> between the media tree and the imx tree since it hasn't been fully
> >>> merged yet.
> >>>
> >>> thanks,
> >>>
> >>> adam
> >>>
> >>>> Benjamin
> >>>>
> >>>> Benjamin Gaignard (11):
> >>>> media: uapi: HEVC: Add missing fields in HEVC controls
> >>>> media: uapi: HEVC: Rename HEVC stateless controls with STATELESS
> >>>> prefix
> >>>> media: uapi: HEVC: Add document uAPI structure
> >>>> media: uapi: HEVC: Define V4L2_CID_STATELESS_HEVC_SLICE_PARAMS as a
> >>>> dynamic array
> >>>> media: uapi: Move parsed HEVC pixel format out of staging
> >>>> media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control
> >>>> media: uapi: Move the HEVC stateless control type out of staging
> >>>> media: controls: Log HEVC stateless control in .std_log
> >>>> media: uapi: Create a dedicated header for Hantro control
> >>>> media: uapi: HEVC: fix padding in v4l2 control structures
> >>>> media: uapi: move HEVC stateless controls out of staging
> >>>>
> >>>> Hans Verkuil (3):
> >>>> videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY
> >>>> v4l2-ctrls: add support for dynamically allocated arrays.
> >>>> vivid: add dynamic array test control
> >>>>
> >>>> .../userspace-api/media/drivers/hantro.rst | 5 -
> >>>> .../media/v4l/ext-ctrls-codec-stateless.rst | 831 ++++++++++++++++++
> >>>> .../media/v4l/ext-ctrls-codec.rst | 780 ----------------
> >>>> .../media/v4l/pixfmt-compressed.rst | 7 +-
> >>>> .../media/v4l/vidioc-g-ext-ctrls.rst | 20 +
> >>>> .../media/v4l/vidioc-queryctrl.rst | 8 +
> >>>> .../media/videodev2.h.rst.exceptions | 5 +
> >>>> .../media/test-drivers/vivid/vivid-ctrls.c | 15 +
> >>>> drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++-
> >>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 198 ++++-
> >>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 32 +-
> >>>> drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +-
> >>>> drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +-
> >>>> drivers/staging/media/hantro/hantro_drv.c | 27 +-
> >>>> drivers/staging/media/hantro/hantro_hevc.c | 8 +-
> >>>> drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +-
> >>>> .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +-
> >>>> include/media/hevc-ctrls.h | 250 ------
> >>>> include/media/v4l2-ctrls.h | 48 +-
> >>>> include/uapi/linux/hantro-media.h | 19 +
> >>>> include/uapi/linux/v4l2-controls.h | 436 +++++++++
> >>>> include/uapi/linux/videodev2.h | 13 +
> >>>> 22 files changed, 1686 insertions(+), 1169 deletions(-)
> >>>> delete mode 100644 include/media/hevc-ctrls.h
> >>>> create mode 100644 include/uapi/linux/hantro-media.h
> >>>>
> >>>> --
> >>>> 2.32.0
> >>>>
Le 04/03/2022 à 13:59, Adam Ford a écrit :
> On Thu, Mar 3, 2022 at 4:13 AM Benjamin Gaignard
> <[email protected]> wrote:
>>
>> Le 03/03/2022 à 02:23, Adam Ford a écrit :
>>> On Mon, Feb 28, 2022 at 4:13 AM Benjamin Gaignard
>>> <[email protected]> wrote:
>>>> Le 26/02/2022 à 23:25, Adam Ford a écrit :
>>>>> On Fri, Feb 25, 2022 at 4:41 PM Benjamin Gaignard
>>>>> <[email protected]> wrote:
>>>>>> This series aims to make HEVC uapi stable and usable for hardware
>>>>>> decoder. HEVC uapi is used by 2 mainlined drivers (Cedrus and Hantro)
>>>>>> and 2 out of the tree drivers (rkvdec and RPI).
>>>>>>
>>>>>> After the remarks done on version 2, I have completely reworked to patches
>>>>>> split so changelogs are meaningless. I have also drop "RFC" from the
>>>>>> titles.
>>>>>>
>>>>>> In this v3 I do all the changes (new controls, documentation, etc..)
>>>>>> in the staging directory before moving the HEVC uAPI to stable
>>>>>> steps by steps (unlike the big one patch in v2).
>>>>>>
>>>>>> At the end fluster tests results on IMX8MQ is 77/147 for HEVC codec.
>>>> I have push a branch here:
>>>> https://gitlab.collabora.com/benjamin.gaignard/for-upstream/-/commits/HEVC_UAPI_V4
>>>>
>>>> it is the incoming version 4 of this series + patches to enable G2 on my IMX8MQ
>>> Benjamin,
>>>
>>> I checked this repo out, and built it along with pulling the latest
>>> versions of G-streamer and fluster.
>>> When I check for v4l2 compatibility, I get the following:
>>>
>>> GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for
>>> GStreamer 1.0... ❌
>>> GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for
>>> GStreamer 1.0... ✔️
>>> GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for
>>> GStreamer 1.0... ❌
>>> GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for
>>> GStreamer 1.0... ❌
>>> GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for
>>> GStreamer 1.0... ❌
>>> GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌
>>> GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for
>>> GStreamer 1.0... ✔️
>>> GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for
>>> GStreamer 1.0... ✔️
>>>
>>> I see H,264, VP8 and VP9, but I would have also expected
>>> GStreamer-H.265-V4L2SL-Gst1.0 to return with a check box.
>>>
>>> When I checked to see if both decoders were being enumerated, I found they were.
>>> [gst-main] root@localhost:~/gstreamer/fluster# dmesg |grep -i hantro
>>> [ 16.044243] hantro_vpu: module is from the staging directory, the
>>> quality is unknown, you have been warned.
>>> [ 16.044243] hantro_vpu: module is from the staging directory, the
>>> quality is unknown, you have been warned.
>>> [ 16.095661] hantro-vpu 38300000.video-codec: registered
>>> nxp,imx8mq-vpu-g1-dec as /dev/video0
>>> [ 16.096782] hantro-vpu 38310000.video-codec: registered
>>> nxp,imx8mq-vpu-g2-dec as /dev/video1
>>>
>>> Did I do something wrong, or did I miss something?
>> Hi Adam,
>>
>> I guess it could be a misalignment between v4l2 kernel headers
>> and GStreamer v4l2 headers.
>> I have push a new version of the GST merge request which is aligned with
>> HEVC uAPI v4 proposal.
>>
>> When you inspect v4l2codecs gst plugin (gst-inspect-1.0 v4l2codecs) do
>> you see v4l2slh265dec plugin ?
>> I have fluster happy with it:
>> ./fluster.py list -c
> I pulled the latest from g-streamer's git repo again, and re-built.
> gst-instpect does not appear to show the h265 codec anymore.
>
> # gst-inspect-1.0 |grep v4l2codecs
> v4l2codecs: v4l2slh264dec: V4L2 Stateless H.264 Video Decoder
> v4l2codecs: v4l2slmpeg2dec: V4L2 Stateless Mpeg2 Video Decoder
> v4l2codecs: v4l2slvp8alphadecodebin: VP8 Alpha Decoder
> v4l2codecs: v4l2slvp8dec: V4L2 Stateless VP8 Video Decoder
> v4l2codecs: v4l2slvp9alphadecodebin: VP9 Alpha Decoder
> v4l2codecs: v4l2slvp9dec: V4L2 Stateless VP9 Video Decoder
>
> Is there a specific branch of g-streamer I should be using that can
> handle the newer API?
You can use this branch:
https://gitlab.freedesktop.org/benjamin.gaignard1/gstreamer/-/tree/HEVC_aligned_with_kernel_5.15
It is the one used to create the merge request for h265 plugin.
Benjamin
adam
>>
>> H265
>> ...
>> GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for GStreamer 1.0... ❌
>> GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for GStreamer 1.0... ✔️
>> GStreamer-H.265-VA-Gst1.0: GStreamer H.265 VA decoder for GStreamer 1.0... ❌
>> GStreamer-H.265-VAAPI-Gst1.0: GStreamer H.265 VAAPI decoder for GStreamer 1.0... ❌
>> JCT-VT-H.265: JCT-VT H.265/HEVC reference decoder... ❌
>>
>> I hope that will help you,
>>
>> Regards,
>> Benjamin
>>
>>> adam
>>>
>>>> Regards,
>>>> Benjamin
>>>>
>>>>> Benjamin,
>>>>>
>>>>> I have an imx8mm and imx8mq that I can test. Do you happen to have a
>>>>> repo that I can clone to test this? The imx8m stuff is spread around
>>>>> between the media tree and the imx tree since it hasn't been fully
>>>>> merged yet.
>>>>>
>>>>> thanks,
>>>>>
>>>>> adam
>>>>>
>>>>>> Benjamin
>>>>>>
>>>>>> Benjamin Gaignard (11):
>>>>>> media: uapi: HEVC: Add missing fields in HEVC controls
>>>>>> media: uapi: HEVC: Rename HEVC stateless controls with STATELESS
>>>>>> prefix
>>>>>> media: uapi: HEVC: Add document uAPI structure
>>>>>> media: uapi: HEVC: Define V4L2_CID_STATELESS_HEVC_SLICE_PARAMS as a
>>>>>> dynamic array
>>>>>> media: uapi: Move parsed HEVC pixel format out of staging
>>>>>> media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control
>>>>>> media: uapi: Move the HEVC stateless control type out of staging
>>>>>> media: controls: Log HEVC stateless control in .std_log
>>>>>> media: uapi: Create a dedicated header for Hantro control
>>>>>> media: uapi: HEVC: fix padding in v4l2 control structures
>>>>>> media: uapi: move HEVC stateless controls out of staging
>>>>>>
>>>>>> Hans Verkuil (3):
>>>>>> videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY
>>>>>> v4l2-ctrls: add support for dynamically allocated arrays.
>>>>>> vivid: add dynamic array test control
>>>>>>
>>>>>> .../userspace-api/media/drivers/hantro.rst | 5 -
>>>>>> .../media/v4l/ext-ctrls-codec-stateless.rst | 831 ++++++++++++++++++
>>>>>> .../media/v4l/ext-ctrls-codec.rst | 780 ----------------
>>>>>> .../media/v4l/pixfmt-compressed.rst | 7 +-
>>>>>> .../media/v4l/vidioc-g-ext-ctrls.rst | 20 +
>>>>>> .../media/v4l/vidioc-queryctrl.rst | 8 +
>>>>>> .../media/videodev2.h.rst.exceptions | 5 +
>>>>>> .../media/test-drivers/vivid/vivid-ctrls.c | 15 +
>>>>>> drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++-
>>>>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 198 ++++-
>>>>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 32 +-
>>>>>> drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +-
>>>>>> drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +-
>>>>>> drivers/staging/media/hantro/hantro_drv.c | 27 +-
>>>>>> drivers/staging/media/hantro/hantro_hevc.c | 8 +-
>>>>>> drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +-
>>>>>> .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +-
>>>>>> include/media/hevc-ctrls.h | 250 ------
>>>>>> include/media/v4l2-ctrls.h | 48 +-
>>>>>> include/uapi/linux/hantro-media.h | 19 +
>>>>>> include/uapi/linux/v4l2-controls.h | 436 +++++++++
>>>>>> include/uapi/linux/videodev2.h | 13 +
>>>>>> 22 files changed, 1686 insertions(+), 1169 deletions(-)
>>>>>> delete mode 100644 include/media/hevc-ctrls.h
>>>>>> create mode 100644 include/uapi/linux/hantro-media.h
>>>>>>
>>>>>> --
>>>>>> 2.32.0
>>>>>>
On Fri, Mar 4, 2022 at 7:03 AM Benjamin Gaignard
<[email protected]> wrote:
>
>
> Le 04/03/2022 à 13:59, Adam Ford a écrit :
> > On Thu, Mar 3, 2022 at 4:13 AM Benjamin Gaignard
> > <[email protected]> wrote:
> >>
> >> Le 03/03/2022 à 02:23, Adam Ford a écrit :
> >>> On Mon, Feb 28, 2022 at 4:13 AM Benjamin Gaignard
> >>> <[email protected]> wrote:
> >>>> Le 26/02/2022 à 23:25, Adam Ford a écrit :
> >>>>> On Fri, Feb 25, 2022 at 4:41 PM Benjamin Gaignard
> >>>>> <[email protected]> wrote:
> >>>>>> This series aims to make HEVC uapi stable and usable for hardware
> >>>>>> decoder. HEVC uapi is used by 2 mainlined drivers (Cedrus and Hantro)
> >>>>>> and 2 out of the tree drivers (rkvdec and RPI).
> >>>>>>
> >>>>>> After the remarks done on version 2, I have completely reworked to patches
> >>>>>> split so changelogs are meaningless. I have also drop "RFC" from the
> >>>>>> titles.
> >>>>>>
> >>>>>> In this v3 I do all the changes (new controls, documentation, etc..)
> >>>>>> in the staging directory before moving the HEVC uAPI to stable
> >>>>>> steps by steps (unlike the big one patch in v2).
> >>>>>>
> >>>>>> At the end fluster tests results on IMX8MQ is 77/147 for HEVC codec.
> >>>> I have push a branch here:
> >>>> https://gitlab.collabora.com/benjamin.gaignard/for-upstream/-/commits/HEVC_UAPI_V4
> >>>>
> >>>> it is the incoming version 4 of this series + patches to enable G2 on my IMX8MQ
> >>> Benjamin,
> >>>
> >>> I checked this repo out, and built it along with pulling the latest
> >>> versions of G-streamer and fluster.
> >>> When I check for v4l2 compatibility, I get the following:
> >>>
> >>> GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for
> >>> GStreamer 1.0... ❌
> >>> GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for
> >>> GStreamer 1.0... ✔️
> >>> GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for
> >>> GStreamer 1.0... ❌
> >>> GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for
> >>> GStreamer 1.0... ❌
> >>> GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for
> >>> GStreamer 1.0... ❌
> >>> GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌
> >>> GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for
> >>> GStreamer 1.0... ✔️
> >>> GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for
> >>> GStreamer 1.0... ✔️
> >>>
> >>> I see H,264, VP8 and VP9, but I would have also expected
> >>> GStreamer-H.265-V4L2SL-Gst1.0 to return with a check box.
> >>>
> >>> When I checked to see if both decoders were being enumerated, I found they were.
> >>> [gst-main] root@localhost:~/gstreamer/fluster# dmesg |grep -i hantro
> >>> [ 16.044243] hantro_vpu: module is from the staging directory, the
> >>> quality is unknown, you have been warned.
> >>> [ 16.044243] hantro_vpu: module is from the staging directory, the
> >>> quality is unknown, you have been warned.
> >>> [ 16.095661] hantro-vpu 38300000.video-codec: registered
> >>> nxp,imx8mq-vpu-g1-dec as /dev/video0
> >>> [ 16.096782] hantro-vpu 38310000.video-codec: registered
> >>> nxp,imx8mq-vpu-g2-dec as /dev/video1
> >>>
> >>> Did I do something wrong, or did I miss something?
> >> Hi Adam,
> >>
> >> I guess it could be a misalignment between v4l2 kernel headers
> >> and GStreamer v4l2 headers.
> >> I have push a new version of the GST merge request which is aligned with
> >> HEVC uAPI v4 proposal.
> >>
> >> When you inspect v4l2codecs gst plugin (gst-inspect-1.0 v4l2codecs) do
> >> you see v4l2slh265dec plugin ?
> >> I have fluster happy with it:
> >> ./fluster.py list -c
> > I pulled the latest from g-streamer's git repo again, and re-built.
> > gst-instpect does not appear to show the h265 codec anymore.
> >
> > # gst-inspect-1.0 |grep v4l2codecs
> > v4l2codecs: v4l2slh264dec: V4L2 Stateless H.264 Video Decoder
> > v4l2codecs: v4l2slmpeg2dec: V4L2 Stateless Mpeg2 Video Decoder
> > v4l2codecs: v4l2slvp8alphadecodebin: VP8 Alpha Decoder
> > v4l2codecs: v4l2slvp8dec: V4L2 Stateless VP8 Video Decoder
> > v4l2codecs: v4l2slvp9alphadecodebin: VP9 Alpha Decoder
> > v4l2codecs: v4l2slvp9dec: V4L2 Stateless VP9 Video Decoder
> >
> > Is there a specific branch of g-streamer I should be using that can
> > handle the newer API?
>
> You can use this branch:
>
> https://gitlab.freedesktop.org/benjamin.gaignard1/gstreamer/-/tree/HEVC_aligned_with_kernel_5.15
>
> It is the one used to create the merge request for h265 plugin.
I used the above gstreamer branch with
https://gitlab.collabora.com/benjamin.gaignard/for-upstream/-/commits/HEVC_UAPI_V4
for the kernel.
I was able to confirm the gst-inspect showed the H265:
[gst-HEAD] root@localhost:~/fluster# gst-inspect-1.0 |grep 265
libav: avdec_h265: libav HEVC (High Efficiency Video Coding) decoder
rtp: rtph265depay: RTP H265 depayloader
rtp: rtph265pay: RTP H265 payloader
typefindfunctions: video/x-h265: h265, x265, 265
v4l2codecs: v4l2slh265dec: V4L2 Stateless H.265 Video Decoder
videoparsersbad: h265parse: H.265 parser
Unfortunately, fluster doesn't appear to recognize it:
GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for
GStreamer 1.0... ❌
Attempting to run fluster with that decoder fail:
# ./fluster.py run -d GStreamer-H.265-V4L2SL-Gst1.0 -j1
Skipping decoder GStreamer-H.265-V4L2SL-Gst1.0 because it cannot be run
[gst-HEAD] root@localhost:~/fluster#
>
> Benjamin
>
> adam
>
> >>
> >> H265
> >> ...
> >> GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for GStreamer 1.0... ❌
> >> GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for GStreamer 1.0... ✔️
> >> GStreamer-H.265-VA-Gst1.0: GStreamer H.265 VA decoder for GStreamer 1.0... ❌
> >> GStreamer-H.265-VAAPI-Gst1.0: GStreamer H.265 VAAPI decoder for GStreamer 1.0... ❌
> >> JCT-VT-H.265: JCT-VT H.265/HEVC reference decoder... ❌
> >>
> >> I hope that will help you,
> >>
> >> Regards,
> >> Benjamin
> >>
> >>> adam
> >>>
> >>>> Regards,
> >>>> Benjamin
> >>>>
> >>>>> Benjamin,
> >>>>>
> >>>>> I have an imx8mm and imx8mq that I can test. Do you happen to have a
> >>>>> repo that I can clone to test this? The imx8m stuff is spread around
> >>>>> between the media tree and the imx tree since it hasn't been fully
> >>>>> merged yet.
> >>>>>
> >>>>> thanks,
> >>>>>
> >>>>> adam
> >>>>>
> >>>>>> Benjamin
> >>>>>>
> >>>>>> Benjamin Gaignard (11):
> >>>>>> media: uapi: HEVC: Add missing fields in HEVC controls
> >>>>>> media: uapi: HEVC: Rename HEVC stateless controls with STATELESS
> >>>>>> prefix
> >>>>>> media: uapi: HEVC: Add document uAPI structure
> >>>>>> media: uapi: HEVC: Define V4L2_CID_STATELESS_HEVC_SLICE_PARAMS as a
> >>>>>> dynamic array
> >>>>>> media: uapi: Move parsed HEVC pixel format out of staging
> >>>>>> media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control
> >>>>>> media: uapi: Move the HEVC stateless control type out of staging
> >>>>>> media: controls: Log HEVC stateless control in .std_log
> >>>>>> media: uapi: Create a dedicated header for Hantro control
> >>>>>> media: uapi: HEVC: fix padding in v4l2 control structures
> >>>>>> media: uapi: move HEVC stateless controls out of staging
> >>>>>>
> >>>>>> Hans Verkuil (3):
> >>>>>> videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY
> >>>>>> v4l2-ctrls: add support for dynamically allocated arrays.
> >>>>>> vivid: add dynamic array test control
> >>>>>>
> >>>>>> .../userspace-api/media/drivers/hantro.rst | 5 -
> >>>>>> .../media/v4l/ext-ctrls-codec-stateless.rst | 831 ++++++++++++++++++
> >>>>>> .../media/v4l/ext-ctrls-codec.rst | 780 ----------------
> >>>>>> .../media/v4l/pixfmt-compressed.rst | 7 +-
> >>>>>> .../media/v4l/vidioc-g-ext-ctrls.rst | 20 +
> >>>>>> .../media/v4l/vidioc-queryctrl.rst | 8 +
> >>>>>> .../media/videodev2.h.rst.exceptions | 5 +
> >>>>>> .../media/test-drivers/vivid/vivid-ctrls.c | 15 +
> >>>>>> drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++-
> >>>>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 198 ++++-
> >>>>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 32 +-
> >>>>>> drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +-
> >>>>>> drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +-
> >>>>>> drivers/staging/media/hantro/hantro_drv.c | 27 +-
> >>>>>> drivers/staging/media/hantro/hantro_hevc.c | 8 +-
> >>>>>> drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +-
> >>>>>> .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +-
> >>>>>> include/media/hevc-ctrls.h | 250 ------
> >>>>>> include/media/v4l2-ctrls.h | 48 +-
> >>>>>> include/uapi/linux/hantro-media.h | 19 +
> >>>>>> include/uapi/linux/v4l2-controls.h | 436 +++++++++
> >>>>>> include/uapi/linux/videodev2.h | 13 +
> >>>>>> 22 files changed, 1686 insertions(+), 1169 deletions(-)
> >>>>>> delete mode 100644 include/media/hevc-ctrls.h
> >>>>>> create mode 100644 include/uapi/linux/hantro-media.h
> >>>>>>
> >>>>>> --
> >>>>>> 2.32.0
> >>>>>>
Le 04/03/2022 à 23:46, Adam Ford a écrit :
> On Fri, Mar 4, 2022 at 7:03 AM Benjamin Gaignard
> <[email protected]> wrote:
>>
>> Le 04/03/2022 à 13:59, Adam Ford a écrit :
>>> On Thu, Mar 3, 2022 at 4:13 AM Benjamin Gaignard
>>> <[email protected]> wrote:
>>>> Le 03/03/2022 à 02:23, Adam Ford a écrit :
>>>>> On Mon, Feb 28, 2022 at 4:13 AM Benjamin Gaignard
>>>>> <[email protected]> wrote:
>>>>>> Le 26/02/2022 à 23:25, Adam Ford a écrit :
>>>>>>> On Fri, Feb 25, 2022 at 4:41 PM Benjamin Gaignard
>>>>>>> <[email protected]> wrote:
>>>>>>>> This series aims to make HEVC uapi stable and usable for hardware
>>>>>>>> decoder. HEVC uapi is used by 2 mainlined drivers (Cedrus and Hantro)
>>>>>>>> and 2 out of the tree drivers (rkvdec and RPI).
>>>>>>>>
>>>>>>>> After the remarks done on version 2, I have completely reworked to patches
>>>>>>>> split so changelogs are meaningless. I have also drop "RFC" from the
>>>>>>>> titles.
>>>>>>>>
>>>>>>>> In this v3 I do all the changes (new controls, documentation, etc..)
>>>>>>>> in the staging directory before moving the HEVC uAPI to stable
>>>>>>>> steps by steps (unlike the big one patch in v2).
>>>>>>>>
>>>>>>>> At the end fluster tests results on IMX8MQ is 77/147 for HEVC codec.
>>>>>> I have push a branch here:
>>>>>> https://gitlab.collabora.com/benjamin.gaignard/for-upstream/-/commits/HEVC_UAPI_V4
>>>>>>
>>>>>> it is the incoming version 4 of this series + patches to enable G2 on my IMX8MQ
>>>>> Benjamin,
>>>>>
>>>>> I checked this repo out, and built it along with pulling the latest
>>>>> versions of G-streamer and fluster.
>>>>> When I check for v4l2 compatibility, I get the following:
>>>>>
>>>>> GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for
>>>>> GStreamer 1.0... ❌
>>>>> GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for
>>>>> GStreamer 1.0... ✔️
>>>>> GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for
>>>>> GStreamer 1.0... ❌
>>>>> GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for
>>>>> GStreamer 1.0... ❌
>>>>> GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for
>>>>> GStreamer 1.0... ❌
>>>>> GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌
>>>>> GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for
>>>>> GStreamer 1.0... ✔️
>>>>> GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for
>>>>> GStreamer 1.0... ✔️
>>>>>
>>>>> I see H,264, VP8 and VP9, but I would have also expected
>>>>> GStreamer-H.265-V4L2SL-Gst1.0 to return with a check box.
>>>>>
>>>>> When I checked to see if both decoders were being enumerated, I found they were.
>>>>> [gst-main] root@localhost:~/gstreamer/fluster# dmesg |grep -i hantro
>>>>> [ 16.044243] hantro_vpu: module is from the staging directory, the
>>>>> quality is unknown, you have been warned.
>>>>> [ 16.044243] hantro_vpu: module is from the staging directory, the
>>>>> quality is unknown, you have been warned.
>>>>> [ 16.095661] hantro-vpu 38300000.video-codec: registered
>>>>> nxp,imx8mq-vpu-g1-dec as /dev/video0
>>>>> [ 16.096782] hantro-vpu 38310000.video-codec: registered
>>>>> nxp,imx8mq-vpu-g2-dec as /dev/video1
>>>>>
>>>>> Did I do something wrong, or did I miss something?
>>>> Hi Adam,
>>>>
>>>> I guess it could be a misalignment between v4l2 kernel headers
>>>> and GStreamer v4l2 headers.
>>>> I have push a new version of the GST merge request which is aligned with
>>>> HEVC uAPI v4 proposal.
>>>>
>>>> When you inspect v4l2codecs gst plugin (gst-inspect-1.0 v4l2codecs) do
>>>> you see v4l2slh265dec plugin ?
>>>> I have fluster happy with it:
>>>> ./fluster.py list -c
>>> I pulled the latest from g-streamer's git repo again, and re-built.
>>> gst-instpect does not appear to show the h265 codec anymore.
>>>
>>> # gst-inspect-1.0 |grep v4l2codecs
>>> v4l2codecs: v4l2slh264dec: V4L2 Stateless H.264 Video Decoder
>>> v4l2codecs: v4l2slmpeg2dec: V4L2 Stateless Mpeg2 Video Decoder
>>> v4l2codecs: v4l2slvp8alphadecodebin: VP8 Alpha Decoder
>>> v4l2codecs: v4l2slvp8dec: V4L2 Stateless VP8 Video Decoder
>>> v4l2codecs: v4l2slvp9alphadecodebin: VP9 Alpha Decoder
>>> v4l2codecs: v4l2slvp9dec: V4L2 Stateless VP9 Video Decoder
>>>
>>> Is there a specific branch of g-streamer I should be using that can
>>> handle the newer API?
>> You can use this branch:
>>
>> https://gitlab.freedesktop.org/benjamin.gaignard1/gstreamer/-/tree/HEVC_aligned_with_kernel_5.15
>>
>> It is the one used to create the merge request for h265 plugin.
> I used the above gstreamer branch with
> https://gitlab.collabora.com/benjamin.gaignard/for-upstream/-/commits/HEVC_UAPI_V4
> for the kernel.
>
> I was able to confirm the gst-inspect showed the H265:
>
> [gst-HEAD] root@localhost:~/fluster# gst-inspect-1.0 |grep 265
> libav: avdec_h265: libav HEVC (High Efficiency Video Coding) decoder
> rtp: rtph265depay: RTP H265 depayloader
>
> rtp: rtph265pay: RTP H265 payloader
> typefindfunctions: video/x-h265: h265, x265, 265
> v4l2codecs: v4l2slh265dec: V4L2 Stateless H.265 Video Decoder
> videoparsersbad: h265parse: H.265 parser
>
> Unfortunately, fluster doesn't appear to recognize it:
>
> GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for
> GStreamer 1.0... ❌
>
> Attempting to run fluster with that decoder fail:
>
> # ./fluster.py run -d GStreamer-H.265-V4L2SL-Gst1.0 -j1
> Skipping decoder GStreamer-H.265-V4L2SL-Gst1.0 because it cannot be run
> [gst-HEAD] root@localhost:~/fluster#
Can you try with -vvv option in fluster command line ?
Does a manual gst pipeline with v4l2slh265dec works ?
Regards,
Benjamin
>
>
>
>
>
>
>
>> Benjamin
>>
>> adam
>>
>>>> H265
>>>> ...
>>>> GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for GStreamer 1.0... ❌
>>>> GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for GStreamer 1.0... ✔️
>>>> GStreamer-H.265-VA-Gst1.0: GStreamer H.265 VA decoder for GStreamer 1.0... ❌
>>>> GStreamer-H.265-VAAPI-Gst1.0: GStreamer H.265 VAAPI decoder for GStreamer 1.0... ❌
>>>> JCT-VT-H.265: JCT-VT H.265/HEVC reference decoder... ❌
>>>>
>>>> I hope that will help you,
>>>>
>>>> Regards,
>>>> Benjamin
>>>>
>>>>> adam
>>>>>
>>>>>> Regards,
>>>>>> Benjamin
>>>>>>
>>>>>>> Benjamin,
>>>>>>>
>>>>>>> I have an imx8mm and imx8mq that I can test. Do you happen to have a
>>>>>>> repo that I can clone to test this? The imx8m stuff is spread around
>>>>>>> between the media tree and the imx tree since it hasn't been fully
>>>>>>> merged yet.
>>>>>>>
>>>>>>> thanks,
>>>>>>>
>>>>>>> adam
>>>>>>>
>>>>>>>> Benjamin
>>>>>>>>
>>>>>>>> Benjamin Gaignard (11):
>>>>>>>> media: uapi: HEVC: Add missing fields in HEVC controls
>>>>>>>> media: uapi: HEVC: Rename HEVC stateless controls with STATELESS
>>>>>>>> prefix
>>>>>>>> media: uapi: HEVC: Add document uAPI structure
>>>>>>>> media: uapi: HEVC: Define V4L2_CID_STATELESS_HEVC_SLICE_PARAMS as a
>>>>>>>> dynamic array
>>>>>>>> media: uapi: Move parsed HEVC pixel format out of staging
>>>>>>>> media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control
>>>>>>>> media: uapi: Move the HEVC stateless control type out of staging
>>>>>>>> media: controls: Log HEVC stateless control in .std_log
>>>>>>>> media: uapi: Create a dedicated header for Hantro control
>>>>>>>> media: uapi: HEVC: fix padding in v4l2 control structures
>>>>>>>> media: uapi: move HEVC stateless controls out of staging
>>>>>>>>
>>>>>>>> Hans Verkuil (3):
>>>>>>>> videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY
>>>>>>>> v4l2-ctrls: add support for dynamically allocated arrays.
>>>>>>>> vivid: add dynamic array test control
>>>>>>>>
>>>>>>>> .../userspace-api/media/drivers/hantro.rst | 5 -
>>>>>>>> .../media/v4l/ext-ctrls-codec-stateless.rst | 831 ++++++++++++++++++
>>>>>>>> .../media/v4l/ext-ctrls-codec.rst | 780 ----------------
>>>>>>>> .../media/v4l/pixfmt-compressed.rst | 7 +-
>>>>>>>> .../media/v4l/vidioc-g-ext-ctrls.rst | 20 +
>>>>>>>> .../media/v4l/vidioc-queryctrl.rst | 8 +
>>>>>>>> .../media/videodev2.h.rst.exceptions | 5 +
>>>>>>>> .../media/test-drivers/vivid/vivid-ctrls.c | 15 +
>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++-
>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 198 ++++-
>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 32 +-
>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +-
>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +-
>>>>>>>> drivers/staging/media/hantro/hantro_drv.c | 27 +-
>>>>>>>> drivers/staging/media/hantro/hantro_hevc.c | 8 +-
>>>>>>>> drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +-
>>>>>>>> .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +-
>>>>>>>> include/media/hevc-ctrls.h | 250 ------
>>>>>>>> include/media/v4l2-ctrls.h | 48 +-
>>>>>>>> include/uapi/linux/hantro-media.h | 19 +
>>>>>>>> include/uapi/linux/v4l2-controls.h | 436 +++++++++
>>>>>>>> include/uapi/linux/videodev2.h | 13 +
>>>>>>>> 22 files changed, 1686 insertions(+), 1169 deletions(-)
>>>>>>>> delete mode 100644 include/media/hevc-ctrls.h
>>>>>>>> create mode 100644 include/uapi/linux/hantro-media.h
>>>>>>>>
>>>>>>>> --
>>>>>>>> 2.32.0
>>>>>>>>