Hello,
This patch series aims to bring H.264 support as well as compliance update
to the amlogic stateful video decoder driver.
The issue in the V1 patchset at [1] is solved by patch #1 following comments
and requirements from hans. It moves the full draining & stopped state tracking
and handling from vicodec to core v4l2-mem2mem.
The vicodec changes still passes the v4l2-utils "media-test" tests, log at [5]:
[...]
vicodec media controller compliance tests
Thu Jan 16 13:00:56 UTC 2020
v4l2-compliance SHA: 7ead0e1856b89f2e19369af452bb03fd0cd16793, 64 bits
[...]
Summary:
Total for vicodec device /dev/media3: 7, Succeeded: 7, Failed: 0, Warnings: 0
Total for vicodec device /dev/video13: 51, Succeeded: 51, Failed: 0, Warnings: 0
Total for vicodec device /dev/video14: 51, Succeeded: 51, Failed: 0, Warnings: 0
Total for vicodec device /dev/video14: 51, Succeeded: 51, Failed: 0, Warnings: 0
Total for vicodec device /dev/video14: 51, Succeeded: 51, Failed: 0, Warnings: 0
Total for vicodec device /dev/video14: 51, Succeeded: 51, Failed: 0, Warnings: 0
Total for vicodec device /dev/video14: 51, Succeeded: 51, Failed: 0, Warnings: 0
Total for vicodec device /dev/video14: 51, Succeeded: 51, Failed: 0, Warnings: 0
Final Summary: 364, Succeeded: 364, Failed: 0, Warnings: 0
Thu Jan 16 13:02:59 UTC 2020
With this, it also passes vdec v4l2-compliance with H264 streaming on Amlogic G12A
and Amlogic SM1 SoCs successfully.
The compliance log is:
# v4l2-compliance --stream-from-hdr test-25fps.h264.hdr -s
v4l2-compliance SHA: 7ead0e1856b89f2e19369af452bb03fd0cd16793, 64 bits
Compliance test for meson-vdec device /dev/video0:
Driver Info:
Driver name : meson-vdec
Card type : Amlogic Video Decoder
Bus info : platform:meson-vdec
Driver version : 5.5.0
Capabilities : 0x84204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Detected Stateful Decoder
Required ioctls:
test VIDIOC_QUERYCAP: OK
Allow for multiple opens:
test second /dev/video19 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 2 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
Test input 0:
Streaming ioctls:
test read/write: OK (Not Supported)
test blocking wait: OK
Video Capture Multiplanar: Captured 60 buffers
test MMAP (select): OK
Video Capture Multiplanar: Captured 60 buffers
test MMAP (epoll): OK
test USERPTR (select): OK (Not Supported)
test DMABUF: Cannot test, specify --expbuf-device
Total for meson-vdec device /dev/video0: 49, Succeeded: 49, Failed: 0, Warnings: 0
Changes since v4 at [5]:
- Squashed "don't resume instantly if not streaming capture" and "fix OUTPUT buffer size configuration" fixes from maxime
Changes since v3 at [3]:
- Fixed vicodec compliance
- Fixed vdec compliance with v4l2 state management
- fixed doc errors for v4l2-mem2mem.h
Changes since v2 at [2]:
- Move full draining & stopped state tracking into core v4l2-mem2mem
- Adapt vicodec to use the core v4l2-mem2mem draining & stopped state tracking
Changes since v1 at [1]:
- fixed output_size is never used reported by hans
- rebased on G12A and SM1 patches
- added handling of qbuf after STREAMON and STOP before enought buffer queued
[1] https://lore.kernel.org/linux-media/[email protected]
[2] https://lore.kernel.org/linux-media/[email protected]
[3] https://lore.kernel.org/linux-media/[email protected]
[4] https://people.freedesktop.org/~narmstrong/vicodec-compliance-7ead0e1856b89f2e19369af452bb03fd0cd16793-20200116.log
[5] https://lore.kernel.org/linux-media/[email protected]
Maxime Jourdan (2):
media: meson: vdec: bring up to compliance
media: meson: vdec: add H.264 decoding support
Neil Armstrong (2):
media: v4l2-mem2mem: handle draining, stopped and next-buf-is-last
states
media: vicodec: use v4l2-mem2mem draining, stopped and
next-buf-is-last states handling
drivers/media/platform/vicodec/vicodec-core.c | 162 ++----
drivers/media/v4l2-core/v4l2-mem2mem.c | 172 ++++++-
drivers/staging/media/meson/vdec/Makefile | 2 +-
drivers/staging/media/meson/vdec/codec_h264.c | 482 ++++++++++++++++++
drivers/staging/media/meson/vdec/codec_h264.h | 14 +
drivers/staging/media/meson/vdec/esparser.c | 58 +--
drivers/staging/media/meson/vdec/vdec.c | 90 ++--
drivers/staging/media/meson/vdec/vdec.h | 14 +-
.../staging/media/meson/vdec/vdec_helpers.c | 88 ++--
.../staging/media/meson/vdec/vdec_helpers.h | 6 +-
.../staging/media/meson/vdec/vdec_platform.c | 71 +++
include/media/v4l2-mem2mem.h | 95 ++++
12 files changed, 1024 insertions(+), 230 deletions(-)
create mode 100644 drivers/staging/media/meson/vdec/codec_h264.c
create mode 100644 drivers/staging/media/meson/vdec/codec_h264.h
--
2.22.0
Since the draining and stop phase of the HW decoder mem2mem bahaviour is
now clearly defined, we can move handling of the following states to the
common v4l2-mem2mem core code:
- draining
- stopped
- next-buf-is-last
By introducing the following v4l2-mem2mem APIS:
- v4l2_m2m_encoder_cmd/v4l2_m2m_ioctl_encoder_cmd to handle start/stop command
- v4l2_m2m_decoder_cmd/v4l2_m2m_ioctl_decoder_cmd to handle start/stop command
- v4l2_m2m_start_streaming to handle start of streaming of the de/encoder queue
- v4l2_m2m_stop_streaming to handle stop of streaming of the de/encoder queue
- v4l2_m2m_last_buffer_done to maek the current dest buffer as the last one
And inline helpers:
- v4l2_m2m_mark_stopped to mark the de/encoding process as stopped
- v4l2_m2m_clear_state to clear the de/encoding state
- v4l2_m2m_dst_buf_is_last to detect the current dequeud dst_buf is the last
- v4l2_m2m_has_stopped to detect the de/encoding stopped state
- v4l2_m2m_is_last_draining_src_buf to detect the currect source buffer should
be the last processing before stopping the de/encoding process
The special next-buf-is-last when min_buffers != 1 case is also handled
in v4l2_m2m_qbuf() by reusing the other introduced APIs.
This state management has been stolen from the vicodec implementation,
and is no-op for drivers not calling the v4l2_m2m_encoder_cmd or
v4l2_m2m_decoder_cmd and v4l2_m2m_start_streaming/v4l2_m2m_stop_streaming.
The vicodec will be the first one to be converted as an example.
Signed-off-by: Neil Armstrong <[email protected]>
---
drivers/media/v4l2-core/v4l2-mem2mem.c | 172 ++++++++++++++++++++++++-
include/media/v4l2-mem2mem.h | 95 ++++++++++++++
2 files changed, 265 insertions(+), 2 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-core/v4l2-mem2mem.c
index 1afd9c6ad908..f221d6c7a137 100644
--- a/drivers/media/v4l2-core/v4l2-mem2mem.c
+++ b/drivers/media/v4l2-core/v4l2-mem2mem.c
@@ -340,6 +340,11 @@ static void __v4l2_m2m_try_queue(struct v4l2_m2m_dev *m2m_dev,
m2m_ctx->new_frame = !dst->vb2_buf.copied_timestamp ||
dst->vb2_buf.timestamp != src->vb2_buf.timestamp;
+ if (m2m_ctx->has_stopped) {
+ dprintk("Device has stopped\n");
+ goto job_unlock;
+ }
+
if (m2m_dev->m2m_ops->job_ready
&& (!m2m_dev->m2m_ops->job_ready(m2m_ctx->priv))) {
dprintk("Driver not ready\n");
@@ -556,6 +561,99 @@ int v4l2_m2m_querybuf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
}
EXPORT_SYMBOL_GPL(v4l2_m2m_querybuf);
+void v4l2_m2m_last_buffer_done(struct v4l2_m2m_ctx *m2m_ctx,
+ struct vb2_v4l2_buffer *vbuf)
+{
+ vbuf->flags |= V4L2_BUF_FLAG_LAST;
+ vb2_buffer_done(&vbuf->vb2_buf, VB2_BUF_STATE_DONE);
+
+ v4l2_m2m_mark_stopped(m2m_ctx);
+}
+EXPORT_SYMBOL_GPL(v4l2_m2m_last_buffer_done);
+
+static int v4l2_mark_last_buf(struct v4l2_m2m_ctx *m2m_ctx)
+{
+ struct vb2_v4l2_buffer *next_dst_buf;
+
+ if (m2m_ctx->is_draining)
+ return -EBUSY;
+
+ if (m2m_ctx->has_stopped)
+ return 0;
+
+ m2m_ctx->last_src_buf = v4l2_m2m_last_src_buf(m2m_ctx);
+ m2m_ctx->is_draining = true;
+
+ if (m2m_ctx->last_src_buf)
+ return 0;
+
+ next_dst_buf = v4l2_m2m_dst_buf_remove(m2m_ctx);
+ if (!next_dst_buf) {
+ m2m_ctx->next_buf_last = true;
+ return 0;
+ }
+
+ v4l2_m2m_last_buffer_done(m2m_ctx, next_dst_buf);
+
+ return 0;
+}
+
+void v4l2_m2m_start_streaming(struct v4l2_m2m_ctx *m2m_ctx, struct vb2_queue *q)
+{
+ if (V4L2_TYPE_IS_OUTPUT(q->type))
+ m2m_ctx->last_src_buf = NULL;
+}
+EXPORT_SYMBOL_GPL(v4l2_m2m_start_streaming);
+
+void v4l2_m2m_stop_streaming(struct v4l2_m2m_ctx *m2m_ctx, struct vb2_queue *q)
+{
+ if (V4L2_TYPE_IS_OUTPUT(q->type)) {
+ if (m2m_ctx->is_draining) {
+ struct vb2_v4l2_buffer *next_dst_buf;
+
+ m2m_ctx->last_src_buf = NULL;
+ next_dst_buf = v4l2_m2m_dst_buf_remove(m2m_ctx);
+ if (!next_dst_buf)
+ m2m_ctx->next_buf_last = true;
+ else
+ v4l2_m2m_last_buffer_done(m2m_ctx,
+ next_dst_buf);
+ }
+ } else {
+ v4l2_m2m_clear_state(m2m_ctx);
+ }
+}
+EXPORT_SYMBOL_GPL(v4l2_m2m_stop_streaming);
+
+static void v4l2_m2m_force_last_buf_done(struct v4l2_m2m_ctx *m2m_ctx,
+ struct vb2_queue *q)
+{
+ struct vb2_buffer *vb;
+ struct vb2_v4l2_buffer *vbuf;
+ unsigned int i;
+
+ if (WARN_ON(q->is_output))
+ return;
+ if (list_empty(&q->queued_list))
+ return;
+
+ vb = list_first_entry(&q->queued_list, struct vb2_buffer, queued_entry);
+ for (i = 0; i < vb->num_planes; i++)
+ vb2_set_plane_payload(vb, i, 0);
+
+ /*
+ * Since the buffer hasn't been queued to the ready queue,
+ * mark is active and owned before marking it LAST and DONE
+ */
+ vb->state = VB2_BUF_STATE_ACTIVE;
+ atomic_inc(&q->owned_by_drv_count);
+
+ vbuf = to_vb2_v4l2_buffer(vb);
+ vbuf->field = V4L2_FIELD_NONE;
+
+ v4l2_m2m_last_buffer_done(m2m_ctx, vbuf);
+}
+
int v4l2_m2m_qbuf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
struct v4l2_buffer *buf)
{
@@ -570,11 +668,25 @@ int v4l2_m2m_qbuf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
__func__);
return -EPERM;
}
+
ret = vb2_qbuf(vq, vdev->v4l2_dev->mdev, buf);
- if (!ret && !(buf->flags & V4L2_BUF_FLAG_IN_REQUEST))
+ if (ret)
+ return ret;
+
+ /*
+ * If the capture queue is streaming, but streaming hasn't started
+ * on the device, but was asked to stop, mark the previously queued
+ * buffer as DONE with LAST flag since it won't be queued on the
+ * device.
+ */
+ if (!V4L2_TYPE_IS_OUTPUT(vq->type) &&
+ vb2_is_streaming(vq) && !vb2_start_streaming_called(vq) &&
+ (v4l2_m2m_has_stopped(m2m_ctx) || v4l2_m2m_dst_buf_is_last(m2m_ctx)))
+ v4l2_m2m_force_last_buf_done(m2m_ctx, vq);
+ else if (!(buf->flags & V4L2_BUF_FLAG_IN_REQUEST))
v4l2_m2m_try_schedule(m2m_ctx);
- return ret;
+ return 0;
}
EXPORT_SYMBOL_GPL(v4l2_m2m_qbuf);
@@ -1225,6 +1337,62 @@ int v4l2_m2m_ioctl_try_decoder_cmd(struct file *file, void *fh,
}
EXPORT_SYMBOL_GPL(v4l2_m2m_ioctl_try_decoder_cmd);
+int v4l2_m2m_encoder_cmd(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
+ struct v4l2_encoder_cmd *ec)
+{
+ if (ec->cmd != V4L2_ENC_CMD_STOP && ec->cmd != V4L2_ENC_CMD_START)
+ return -EINVAL;
+
+ if (ec->cmd == V4L2_ENC_CMD_STOP)
+ return v4l2_mark_last_buf(m2m_ctx);
+
+ if (m2m_ctx->is_draining)
+ return -EBUSY;
+
+ if (m2m_ctx->has_stopped)
+ m2m_ctx->has_stopped = false;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_m2m_encoder_cmd);
+
+int v4l2_m2m_decoder_cmd(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
+ struct v4l2_decoder_cmd *dc)
+{
+ if (dc->cmd != V4L2_DEC_CMD_STOP && dc->cmd != V4L2_DEC_CMD_START)
+ return -EINVAL;
+
+ if (dc->cmd == V4L2_DEC_CMD_STOP)
+ return v4l2_mark_last_buf(m2m_ctx);
+
+ if (m2m_ctx->is_draining)
+ return -EBUSY;
+
+ if (m2m_ctx->has_stopped)
+ m2m_ctx->has_stopped = false;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_m2m_decoder_cmd);
+
+int v4l2_m2m_ioctl_encoder_cmd(struct file *file, void *priv,
+ struct v4l2_encoder_cmd *ec)
+{
+ struct v4l2_fh *fh = file->private_data;
+
+ return v4l2_m2m_encoder_cmd(file, fh->m2m_ctx, ec);
+}
+EXPORT_SYMBOL_GPL(v4l2_m2m_ioctl_encoder_cmd);
+
+int v4l2_m2m_ioctl_decoder_cmd(struct file *file, void *priv,
+ struct v4l2_decoder_cmd *dc)
+{
+ struct v4l2_fh *fh = file->private_data;
+
+ return v4l2_m2m_decoder_cmd(file, fh->m2m_ctx, dc);
+}
+EXPORT_SYMBOL_GPL(v4l2_m2m_ioctl_decoder_cmd);
+
int v4l2_m2m_ioctl_stateless_try_decoder_cmd(struct file *file, void *fh,
struct v4l2_decoder_cmd *dc)
{
diff --git a/include/media/v4l2-mem2mem.h b/include/media/v4l2-mem2mem.h
index 1d85e24791e4..3476889af46c 100644
--- a/include/media/v4l2-mem2mem.h
+++ b/include/media/v4l2-mem2mem.h
@@ -80,6 +80,10 @@ struct v4l2_m2m_queue_ctx {
* for an existing frame. This is always true unless
* V4L2_BUF_CAP_SUPPORTS_M2M_HOLD_CAPTURE_BUF is set, which
* indicates slicing support.
+ * @is_draining: indicates device is in draining phase
+ * @last_src_buf: indicate the last source buffer for draining
+ * @next_buf_last: next capture queud buffer will be tagged as last
+ * @has_stopped: indicate the device has been stopped
* @m2m_dev: opaque pointer to the internal data to handle M2M context
* @cap_q_ctx: Capture (output to memory) queue context
* @out_q_ctx: Output (input from memory) queue context
@@ -98,6 +102,11 @@ struct v4l2_m2m_ctx {
bool new_frame;
+ bool is_draining;
+ struct vb2_v4l2_buffer *last_src_buf;
+ bool next_buf_last;
+ bool has_stopped;
+
/* internal use only */
struct v4l2_m2m_dev *m2m_dev;
@@ -215,6 +224,50 @@ v4l2_m2m_buf_done(struct vb2_v4l2_buffer *buf, enum vb2_buffer_state state)
vb2_buffer_done(&buf->vb2_buf, state);
}
+static inline void
+v4l2_m2m_clear_state(struct v4l2_m2m_ctx *m2m_ctx)
+{
+ m2m_ctx->next_buf_last = false;
+ m2m_ctx->is_draining = false;
+ m2m_ctx->has_stopped = false;
+}
+
+static inline void
+v4l2_m2m_mark_stopped(struct v4l2_m2m_ctx *m2m_ctx)
+{
+ m2m_ctx->next_buf_last = false;
+ m2m_ctx->is_draining = false;
+ m2m_ctx->has_stopped = true;
+}
+
+static inline bool
+v4l2_m2m_dst_buf_is_last(struct v4l2_m2m_ctx *m2m_ctx)
+{
+ return m2m_ctx->is_draining && m2m_ctx->next_buf_last;
+}
+
+static inline bool
+v4l2_m2m_has_stopped(struct v4l2_m2m_ctx *m2m_ctx)
+{
+ return m2m_ctx->has_stopped;
+}
+
+static inline bool
+v4l2_m2m_is_last_draining_src_buf(struct v4l2_m2m_ctx *m2m_ctx,
+ struct vb2_v4l2_buffer *buf)
+{
+ return m2m_ctx->is_draining && buf == m2m_ctx->last_src_buf;
+}
+
+/**
+ * v4l2_m2m_last_buffer_done() - marks the buffer with LAST flag and DONE
+ *
+ * @m2m_ctx: m2m context assigned to the instance given by struct &v4l2_m2m_ctx
+ * @vbuf: pointer to struct &v4l2_buffer
+ */
+void v4l2_m2m_last_buffer_done(struct v4l2_m2m_ctx *m2m_ctx,
+ struct vb2_v4l2_buffer *vbuf);
+
/**
* v4l2_m2m_reqbufs() - multi-queue-aware REQBUFS multiplexer
*
@@ -312,6 +365,44 @@ int v4l2_m2m_streamon(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
int v4l2_m2m_streamoff(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
enum v4l2_buf_type type);
+/**
+ * v4l2_m2m_start_streaming() - handle start of streaming of a video queue
+ *
+ * @m2m_ctx: m2m context assigned to the instance given by struct &v4l2_m2m_ctx
+ * @q: queue
+ */
+void v4l2_m2m_start_streaming(struct v4l2_m2m_ctx *m2m_ctx,
+ struct vb2_queue *q);
+
+/**
+ * v4l2_m2m_stop_streaming() - handle stop of streaming of a video queue
+ *
+ * @m2m_ctx: m2m context assigned to the instance given by struct &v4l2_m2m_ctx
+ * @q: queue
+ */
+void v4l2_m2m_stop_streaming(struct v4l2_m2m_ctx *m2m_ctx,
+ struct vb2_queue *q);
+
+/**
+ * v4l2_m2m_encoder_cmd() - execute an encoder command
+ *
+ * @file: pointer to struct &file
+ * @m2m_ctx: m2m context assigned to the instance given by struct &v4l2_m2m_ctx
+ * @ec: pointer to the encoder command
+ */
+int v4l2_m2m_encoder_cmd(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
+ struct v4l2_encoder_cmd *ec);
+
+/**
+ * v4l2_m2m_decoder_cmd() - execute a decoder command
+ *
+ * @file: pointer to struct &file
+ * @m2m_ctx: m2m context assigned to the instance given by struct &v4l2_m2m_ctx
+ * @dc: pointer to the decoder command
+ */
+int v4l2_m2m_decoder_cmd(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
+ struct v4l2_decoder_cmd *dc);
+
/**
* v4l2_m2m_poll() - poll replacement, for destination buffers only
*
@@ -704,6 +795,10 @@ int v4l2_m2m_ioctl_streamon(struct file *file, void *fh,
enum v4l2_buf_type type);
int v4l2_m2m_ioctl_streamoff(struct file *file, void *fh,
enum v4l2_buf_type type);
+int v4l2_m2m_ioctl_encoder_cmd(struct file *file, void *fh,
+ struct v4l2_encoder_cmd *ec);
+int v4l2_m2m_ioctl_decoder_cmd(struct file *file, void *fh,
+ struct v4l2_decoder_cmd *dc);
int v4l2_m2m_ioctl_try_encoder_cmd(struct file *file, void *fh,
struct v4l2_encoder_cmd *ec);
int v4l2_m2m_ioctl_try_decoder_cmd(struct file *file, void *fh,
--
2.22.0
From: Maxime Jourdan <[email protected]>
Add support for the H264 compressed format (V4L2_PIX_FMT_H264).
Signed-off-by: Maxime Jourdan <[email protected]>
Signed-off-by: Neil Armstrong <[email protected]>
---
drivers/staging/media/meson/vdec/Makefile | 2 +-
drivers/staging/media/meson/vdec/codec_h264.c | 482 ++++++++++++++++++
drivers/staging/media/meson/vdec/codec_h264.h | 14 +
.../staging/media/meson/vdec/vdec_platform.c | 61 +++
4 files changed, 558 insertions(+), 1 deletion(-)
create mode 100644 drivers/staging/media/meson/vdec/codec_h264.c
create mode 100644 drivers/staging/media/meson/vdec/codec_h264.h
diff --git a/drivers/staging/media/meson/vdec/Makefile b/drivers/staging/media/meson/vdec/Makefile
index 6bea129084b7..711d990c760e 100644
--- a/drivers/staging/media/meson/vdec/Makefile
+++ b/drivers/staging/media/meson/vdec/Makefile
@@ -3,6 +3,6 @@
meson-vdec-objs = esparser.o vdec.o vdec_helpers.o vdec_platform.o
meson-vdec-objs += vdec_1.o
-meson-vdec-objs += codec_mpeg12.o
+meson-vdec-objs += codec_mpeg12.o codec_h264.o
obj-$(CONFIG_VIDEO_MESON_VDEC) += meson-vdec.o
diff --git a/drivers/staging/media/meson/vdec/codec_h264.c b/drivers/staging/media/meson/vdec/codec_h264.c
new file mode 100644
index 000000000000..4528a6a01c3d
--- /dev/null
+++ b/drivers/staging/media/meson/vdec/codec_h264.c
@@ -0,0 +1,482 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright (C) 2019 BayLibre, SAS
+ * Author: Maxime Jourdan <[email protected]>
+ */
+
+#include <media/v4l2-mem2mem.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "vdec_helpers.h"
+#include "dos_regs.h"
+
+#define SIZE_EXT_FW (20 * SZ_1K)
+#define SIZE_WORKSPACE 0x1ee000
+#define SIZE_SEI (8 * SZ_1K)
+
+/*
+ * Offset added by the firmware which must be substracted
+ * from the workspace phyaddr
+ */
+#define WORKSPACE_BUF_OFFSET 0x1000000
+
+/* ISR status */
+#define CMD_MASK GENMASK(7, 0)
+#define CMD_SRC_CHANGE 1
+#define CMD_FRAMES_READY 2
+#define CMD_FATAL_ERROR 6
+#define CMD_BAD_WIDTH 7
+#define CMD_BAD_HEIGHT 8
+
+#define SEI_DATA_READY BIT(15)
+
+/* Picture type */
+#define PIC_TOP_BOT 5
+#define PIC_BOT_TOP 6
+
+/* Size of Motion Vector per macroblock */
+#define MB_MV_SIZE 96
+
+/* Frame status data */
+#define PIC_STRUCT_BIT 5
+#define PIC_STRUCT_MASK GENMASK(2, 0)
+#define BUF_IDX_MASK GENMASK(4, 0)
+#define ERROR_FLAG BIT(9)
+#define OFFSET_BIT 16
+#define OFFSET_MASK GENMASK(15, 0)
+
+/* Bitstream parsed data */
+#define MB_TOTAL_BIT 8
+#define MB_TOTAL_MASK GENMASK(15, 0)
+#define MB_WIDTH_MASK GENMASK(7, 0)
+#define MAX_REF_BIT 24
+#define MAX_REF_MASK GENMASK(6, 0)
+#define AR_IDC_BIT 16
+#define AR_IDC_MASK GENMASK(7, 0)
+#define AR_PRESENT_FLAG BIT(0)
+#define AR_EXTEND 0xff
+
+/*
+ * Buffer to send to the ESPARSER to signal End Of Stream for H.264.
+ * This is a 16x16 encoded picture that will trigger drain firmware-side.
+ * There is no known alternative.
+ */
+static const u8 eos_sequence[SZ_4K] = {
+ 0x00, 0x00, 0x00, 0x01, 0x06, 0x05, 0xff, 0xe4, 0xdc, 0x45, 0xe9, 0xbd,
+ 0xe6, 0xd9, 0x48, 0xb7, 0x96, 0x2c, 0xd8, 0x20, 0xd9, 0x23, 0xee, 0xef,
+ 0x78, 0x32, 0x36, 0x34, 0x20, 0x2d, 0x20, 0x63, 0x6f, 0x72, 0x65, 0x20,
+ 0x36, 0x37, 0x20, 0x72, 0x31, 0x31, 0x33, 0x30, 0x20, 0x38, 0x34, 0x37,
+ 0x35, 0x39, 0x37, 0x37, 0x20, 0x2d, 0x20, 0x48, 0x2e, 0x32, 0x36, 0x34,
+ 0x2f, 0x4d, 0x50, 0x45, 0x47, 0x2d, 0x34, 0x20, 0x41, 0x56, 0x43, 0x20,
+ 0x63, 0x6f, 0x64, 0x65, 0x63, 0x20, 0x2d, 0x20, 0x43, 0x6f, 0x70, 0x79,
+ 0x6c, 0x65, 0x66, 0x74, 0x20, 0x32, 0x30, 0x30, 0x33, 0x2d, 0x32, 0x30,
+ 0x30, 0x39, 0x20, 0x2d, 0x20, 0x68, 0x74, 0x74, 0x70, 0x3a, 0x2f, 0x2f,
+ 0x77, 0x77, 0x77, 0x2e, 0x76, 0x69, 0x64, 0x65, 0x6f, 0x6c, 0x61, 0x6e,
+ 0x2e, 0x6f, 0x72, 0x67, 0x2f, 0x78, 0x32, 0x36, 0x34, 0x2e, 0x68, 0x74,
+ 0x6d, 0x6c, 0x20, 0x2d, 0x20, 0x6f, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x73,
+ 0x3a, 0x20, 0x63, 0x61, 0x62, 0x61, 0x63, 0x3d, 0x31, 0x20, 0x72, 0x65,
+ 0x66, 0x3d, 0x31, 0x20, 0x64, 0x65, 0x62, 0x6c, 0x6f, 0x63, 0x6b, 0x3d,
+ 0x31, 0x3a, 0x30, 0x3a, 0x30, 0x20, 0x61, 0x6e, 0x61, 0x6c, 0x79, 0x73,
+ 0x65, 0x3d, 0x30, 0x78, 0x31, 0x3a, 0x30, 0x78, 0x31, 0x31, 0x31, 0x20,
+ 0x6d, 0x65, 0x3d, 0x68, 0x65, 0x78, 0x20, 0x73, 0x75, 0x62, 0x6d, 0x65,
+ 0x3d, 0x36, 0x20, 0x70, 0x73, 0x79, 0x5f, 0x72, 0x64, 0x3d, 0x31, 0x2e,
+ 0x30, 0x3a, 0x30, 0x2e, 0x30, 0x20, 0x6d, 0x69, 0x78, 0x65, 0x64, 0x5f,
+ 0x72, 0x65, 0x66, 0x3d, 0x30, 0x20, 0x6d, 0x65, 0x5f, 0x72, 0x61, 0x6e,
+ 0x67, 0x65, 0x3d, 0x31, 0x36, 0x20, 0x63, 0x68, 0x72, 0x6f, 0x6d, 0x61,
+ 0x5f, 0x6d, 0x65, 0x3d, 0x31, 0x20, 0x74, 0x72, 0x65, 0x6c, 0x6c, 0x69,
+ 0x73, 0x3d, 0x30, 0x20, 0x38, 0x78, 0x38, 0x64, 0x63, 0x74, 0x3d, 0x30,
+ 0x20, 0x63, 0x71, 0x6d, 0x3d, 0x30, 0x20, 0x64, 0x65, 0x61, 0x64, 0x7a,
+ 0x6f, 0x6e, 0x65, 0x3d, 0x32, 0x31, 0x2c, 0x31, 0x31, 0x20, 0x63, 0x68,
+ 0x72, 0x6f, 0x6d, 0x61, 0x5f, 0x71, 0x70, 0x5f, 0x6f, 0x66, 0x66, 0x73,
+ 0x65, 0x74, 0x3d, 0x2d, 0x32, 0x20, 0x74, 0x68, 0x72, 0x65, 0x61, 0x64,
+ 0x73, 0x3d, 0x31, 0x20, 0x6e, 0x72, 0x3d, 0x30, 0x20, 0x64, 0x65, 0x63,
+ 0x69, 0x6d, 0x61, 0x74, 0x65, 0x3d, 0x31, 0x20, 0x6d, 0x62, 0x61, 0x66,
+ 0x66, 0x3d, 0x30, 0x20, 0x62, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x73, 0x3d,
+ 0x30, 0x20, 0x6b, 0x65, 0x79, 0x69, 0x6e, 0x74, 0x3d, 0x32, 0x35, 0x30,
+ 0x20, 0x6b, 0x65, 0x79, 0x69, 0x6e, 0x74, 0x5f, 0x6d, 0x69, 0x6e, 0x3d,
+ 0x32, 0x35, 0x20, 0x73, 0x63, 0x65, 0x6e, 0x65, 0x63, 0x75, 0x74, 0x3d,
+ 0x34, 0x30, 0x20, 0x72, 0x63, 0x3d, 0x61, 0x62, 0x72, 0x20, 0x62, 0x69,
+ 0x74, 0x72, 0x61, 0x74, 0x65, 0x3d, 0x31, 0x30, 0x20, 0x72, 0x61, 0x74,
+ 0x65, 0x74, 0x6f, 0x6c, 0x3d, 0x31, 0x2e, 0x30, 0x20, 0x71, 0x63, 0x6f,
+ 0x6d, 0x70, 0x3d, 0x30, 0x2e, 0x36, 0x30, 0x20, 0x71, 0x70, 0x6d, 0x69,
+ 0x6e, 0x3d, 0x31, 0x30, 0x20, 0x71, 0x70, 0x6d, 0x61, 0x78, 0x3d, 0x35,
+ 0x31, 0x20, 0x71, 0x70, 0x73, 0x74, 0x65, 0x70, 0x3d, 0x34, 0x20, 0x69,
+ 0x70, 0x5f, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x3d, 0x31, 0x2e, 0x34, 0x30,
+ 0x20, 0x61, 0x71, 0x3d, 0x31, 0x3a, 0x31, 0x2e, 0x30, 0x30, 0x00, 0x80,
+ 0x00, 0x00, 0x00, 0x01, 0x67, 0x4d, 0x40, 0x0a, 0x9a, 0x74, 0xf4, 0x20,
+ 0x00, 0x00, 0x03, 0x00, 0x20, 0x00, 0x00, 0x06, 0x51, 0xe2, 0x44, 0xd4,
+ 0x00, 0x00, 0x00, 0x01, 0x68, 0xee, 0x32, 0xc8, 0x00, 0x00, 0x00, 0x01,
+ 0x65, 0x88, 0x80, 0x20, 0x00, 0x08, 0x7f, 0xea, 0x6a, 0xe2, 0x99, 0xb6,
+ 0x57, 0xae, 0x49, 0x30, 0xf5, 0xfe, 0x5e, 0x46, 0x0b, 0x72, 0x44, 0xc4,
+ 0xe1, 0xfc, 0x62, 0xda, 0xf1, 0xfb, 0xa2, 0xdb, 0xd6, 0xbe, 0x5c, 0xd7,
+ 0x24, 0xa3, 0xf5, 0xb9, 0x2f, 0x57, 0x16, 0x49, 0x75, 0x47, 0x77, 0x09,
+ 0x5c, 0xa1, 0xb4, 0xc3, 0x4f, 0x60, 0x2b, 0xb0, 0x0c, 0xc8, 0xd6, 0x66,
+ 0xba, 0x9b, 0x82, 0x29, 0x33, 0x92, 0x26, 0x99, 0x31, 0x1c, 0x7f, 0x9b,
+ 0x00, 0x00, 0x01, 0x0ff,
+};
+
+static const u8 *codec_h264_eos_sequence(u32 *len)
+{
+ *len = ARRAY_SIZE(eos_sequence);
+ return eos_sequence;
+}
+
+struct codec_h264 {
+ /* H.264 decoder requires an extended firmware */
+ void *ext_fw_vaddr;
+ dma_addr_t ext_fw_paddr;
+
+ /* Buffer for the H.264 Workspace */
+ void *workspace_vaddr;
+ dma_addr_t workspace_paddr;
+
+ /* Buffer for the H.264 references MV */
+ void *ref_vaddr;
+ dma_addr_t ref_paddr;
+ u32 ref_size;
+
+ /* Buffer for parsed SEI data */
+ void *sei_vaddr;
+ dma_addr_t sei_paddr;
+
+ u32 mb_width;
+ u32 mb_height;
+ u32 max_refs;
+};
+
+static int codec_h264_can_recycle(struct amvdec_core *core)
+{
+ return !amvdec_read_dos(core, AV_SCRATCH_7) ||
+ !amvdec_read_dos(core, AV_SCRATCH_8);
+}
+
+static void codec_h264_recycle(struct amvdec_core *core, u32 buf_idx)
+{
+ /*
+ * Tell the firmware it can recycle this buffer.
+ * AV_SCRATCH_8 serves the same purpose.
+ */
+ if (!amvdec_read_dos(core, AV_SCRATCH_7))
+ amvdec_write_dos(core, AV_SCRATCH_7, buf_idx + 1);
+ else
+ amvdec_write_dos(core, AV_SCRATCH_8, buf_idx + 1);
+}
+
+static int codec_h264_start(struct amvdec_session *sess)
+{
+ u32 workspace_offset;
+ struct amvdec_core *core = sess->core;
+ struct codec_h264 *h264 = sess->priv;
+
+ /* Allocate some memory for the H.264 decoder's state */
+ h264->workspace_vaddr =
+ dma_alloc_coherent(core->dev, SIZE_WORKSPACE,
+ &h264->workspace_paddr, GFP_KERNEL);
+ if (!h264->workspace_vaddr)
+ return -ENOMEM;
+
+ /* Allocate some memory for the H.264 SEI dump */
+ h264->sei_vaddr = dma_alloc_coherent(core->dev, SIZE_SEI,
+ &h264->sei_paddr, GFP_KERNEL);
+ if (!h264->sei_vaddr)
+ return -ENOMEM;
+
+ amvdec_write_dos_bits(core, POWER_CTL_VLD, BIT(9) | BIT(6));
+
+ workspace_offset = h264->workspace_paddr - WORKSPACE_BUF_OFFSET;
+ amvdec_write_dos(core, AV_SCRATCH_1, workspace_offset);
+ amvdec_write_dos(core, AV_SCRATCH_G, h264->ext_fw_paddr);
+ amvdec_write_dos(core, AV_SCRATCH_I, h264->sei_paddr -
+ workspace_offset);
+
+ /* Enable "error correction" */
+ amvdec_write_dos(core, AV_SCRATCH_F,
+ (amvdec_read_dos(core, AV_SCRATCH_F) & 0xffffffc3) |
+ BIT(4) | BIT(7));
+
+ amvdec_write_dos(core, MDEC_PIC_DC_THRESH, 0x404038aa);
+
+ return 0;
+}
+
+static int codec_h264_stop(struct amvdec_session *sess)
+{
+ struct codec_h264 *h264 = sess->priv;
+ struct amvdec_core *core = sess->core;
+
+ if (h264->ext_fw_vaddr)
+ dma_free_coherent(core->dev, SIZE_EXT_FW,
+ h264->ext_fw_vaddr, h264->ext_fw_paddr);
+
+ if (h264->workspace_vaddr)
+ dma_free_coherent(core->dev, SIZE_WORKSPACE,
+ h264->workspace_vaddr, h264->workspace_paddr);
+
+ if (h264->ref_vaddr)
+ dma_free_coherent(core->dev, h264->ref_size,
+ h264->ref_vaddr, h264->ref_paddr);
+
+ if (h264->sei_vaddr)
+ dma_free_coherent(core->dev, SIZE_SEI,
+ h264->sei_vaddr, h264->sei_paddr);
+
+ return 0;
+}
+
+static int codec_h264_load_extended_firmware(struct amvdec_session *sess,
+ const u8 *data, u32 len)
+{
+ struct codec_h264 *h264;
+ struct amvdec_core *core = sess->core;
+
+ if (len < SIZE_EXT_FW)
+ return -EINVAL;
+
+ h264 = kzalloc(sizeof(*h264), GFP_KERNEL);
+ if (!h264)
+ return -ENOMEM;
+
+ h264->ext_fw_vaddr = dma_alloc_coherent(core->dev, SIZE_EXT_FW,
+ &h264->ext_fw_paddr, GFP_KERNEL);
+ if (!h264->ext_fw_vaddr) {
+ kfree(h264);
+ return -ENOMEM;
+ }
+
+ memcpy(h264->ext_fw_vaddr, data, SIZE_EXT_FW);
+ sess->priv = h264;
+
+ return 0;
+}
+
+static const struct v4l2_fract par_table[] = {
+ { 1, 1 }, { 1, 1 }, { 12, 11 }, { 10, 11 },
+ { 16, 11 }, { 40, 33 }, { 24, 11 }, { 20, 11 },
+ { 32, 11 }, { 80, 33 }, { 18, 11 }, { 15, 11 },
+ { 64, 33 }, { 160, 99 }, { 4, 3 }, { 3, 2 },
+ { 2, 1 }
+};
+
+static void codec_h264_set_par(struct amvdec_session *sess)
+{
+ struct amvdec_core *core = sess->core;
+ u32 seq_info = amvdec_read_dos(core, AV_SCRATCH_2);
+ u32 ar_idc = (seq_info >> AR_IDC_BIT) & AR_IDC_MASK;
+
+ if (!(seq_info & AR_PRESENT_FLAG))
+ return;
+
+ if (ar_idc == AR_EXTEND) {
+ u32 ar_info = amvdec_read_dos(core, AV_SCRATCH_3);
+
+ sess->pixelaspect.numerator = ar_info & 0xffff;
+ sess->pixelaspect.denominator = (ar_info >> 16) & 0xffff;
+ return;
+ }
+
+ if (ar_idc >= ARRAY_SIZE(par_table))
+ return;
+
+ sess->pixelaspect = par_table[ar_idc];
+}
+
+static void codec_h264_resume(struct amvdec_session *sess)
+{
+ struct amvdec_core *core = sess->core;
+ struct codec_h264 *h264 = sess->priv;
+ u32 mb_width, mb_height, mb_total;
+
+ amvdec_set_canvases(sess, (u32[]){ ANC0_CANVAS_ADDR, 0 },
+ (u32[]){ 24, 0 });
+
+ dev_dbg(core->dev, "max_refs = %u; actual_dpb_size = %u\n",
+ h264->max_refs, sess->num_dst_bufs);
+
+ /* Align to a multiple of 4 macroblocks */
+ mb_width = ALIGN(h264->mb_width, 4);
+ mb_height = ALIGN(h264->mb_height, 4);
+ mb_total = mb_width * mb_height;
+
+ h264->ref_size = mb_total * MB_MV_SIZE * h264->max_refs;
+ h264->ref_vaddr = dma_alloc_coherent(core->dev, h264->ref_size,
+ &h264->ref_paddr, GFP_KERNEL);
+ if (!h264->ref_vaddr) {
+ amvdec_abort(sess);
+ return;
+ }
+
+ /* Address to store the references' MVs */
+ amvdec_write_dos(core, AV_SCRATCH_1, h264->ref_paddr);
+ /* End of ref MV */
+ amvdec_write_dos(core, AV_SCRATCH_4, h264->ref_paddr + h264->ref_size);
+
+ amvdec_write_dos(core, AV_SCRATCH_0, (h264->max_refs << 24) |
+ (sess->num_dst_bufs << 16) |
+ ((h264->max_refs - 1) << 8));
+}
+
+/**
+ * Configure the H.264 decoder when the parser detected a parameter set change
+ */
+static void codec_h264_src_change(struct amvdec_session *sess)
+{
+ struct amvdec_core *core = sess->core;
+ struct codec_h264 *h264 = sess->priv;
+ u32 parsed_info, mb_total;
+ u32 crop_infor, crop_bottom, crop_right;
+ u32 frame_width, frame_height;
+
+ sess->keyframe_found = 1;
+
+ parsed_info = amvdec_read_dos(core, AV_SCRATCH_1);
+
+ /* Total number of 16x16 macroblocks */
+ mb_total = (parsed_info >> MB_TOTAL_BIT) & MB_TOTAL_MASK;
+ /* Number of macroblocks per line */
+ h264->mb_width = parsed_info & MB_WIDTH_MASK;
+ /* Number of macroblock lines */
+ h264->mb_height = mb_total / h264->mb_width;
+
+ h264->max_refs = ((parsed_info >> MAX_REF_BIT) & MAX_REF_MASK) + 1;
+
+ crop_infor = amvdec_read_dos(core, AV_SCRATCH_6);
+ crop_bottom = (crop_infor & 0xff);
+ crop_right = (crop_infor >> 16) & 0xff;
+
+ frame_width = h264->mb_width * 16 - crop_right;
+ frame_height = h264->mb_height * 16 - crop_bottom;
+
+ dev_dbg(core->dev, "frame: %ux%u; crop: %u %u\n",
+ frame_width, frame_height, crop_right, crop_bottom);
+
+ codec_h264_set_par(sess);
+ amvdec_src_change(sess, frame_width, frame_height, h264->max_refs + 5);
+}
+
+/**
+ * The bitstream offset is split in half in 2 different registers.
+ * Fetch its MSB here, which location depends on the frame number.
+ */
+static u32 get_offset_msb(struct amvdec_core *core, int frame_num)
+{
+ int take_msb = frame_num % 2;
+ int reg_offset = (frame_num / 2) * 4;
+ u32 offset_msb = amvdec_read_dos(core, AV_SCRATCH_A + reg_offset);
+
+ if (take_msb)
+ return offset_msb & 0xffff0000;
+
+ return (offset_msb & 0x0000ffff) << 16;
+}
+
+static void codec_h264_frames_ready(struct amvdec_session *sess, u32 status)
+{
+ struct amvdec_core *core = sess->core;
+ int error_count;
+ int num_frames;
+ int i;
+
+ error_count = amvdec_read_dos(core, AV_SCRATCH_D);
+ num_frames = (status >> 8) & 0xff;
+ if (error_count) {
+ dev_warn(core->dev,
+ "decoder error(s) happened, count %d\n", error_count);
+ amvdec_write_dos(core, AV_SCRATCH_D, 0);
+ }
+
+ for (i = 0; i < num_frames; i++) {
+ u32 frame_status = amvdec_read_dos(core, AV_SCRATCH_1 + i * 4);
+ u32 buffer_index = frame_status & BUF_IDX_MASK;
+ u32 pic_struct = (frame_status >> PIC_STRUCT_BIT) &
+ PIC_STRUCT_MASK;
+ u32 offset = (frame_status >> OFFSET_BIT) & OFFSET_MASK;
+ u32 field = V4L2_FIELD_NONE;
+
+ /*
+ * A buffer decode error means it was decoded,
+ * but part of the picture will have artifacts.
+ * Typical reason is a temporarily corrupted bitstream
+ */
+ if (frame_status & ERROR_FLAG)
+ dev_dbg(core->dev, "Buffer %d decode error\n",
+ buffer_index);
+
+ if (pic_struct == PIC_TOP_BOT)
+ field = V4L2_FIELD_INTERLACED_TB;
+ else if (pic_struct == PIC_BOT_TOP)
+ field = V4L2_FIELD_INTERLACED_BT;
+
+ offset |= get_offset_msb(core, i);
+ amvdec_dst_buf_done_idx(sess, buffer_index, offset, field);
+ }
+}
+
+static irqreturn_t codec_h264_threaded_isr(struct amvdec_session *sess)
+{
+ struct amvdec_core *core = sess->core;
+ u32 status;
+ u32 size;
+ u8 cmd;
+
+ status = amvdec_read_dos(core, AV_SCRATCH_0);
+ cmd = status & CMD_MASK;
+
+ switch (cmd) {
+ case CMD_SRC_CHANGE:
+ codec_h264_src_change(sess);
+ break;
+ case CMD_FRAMES_READY:
+ codec_h264_frames_ready(sess, status);
+ break;
+ case CMD_FATAL_ERROR:
+ dev_err(core->dev, "H.264 decoder fatal error\n");
+ goto abort;
+ case CMD_BAD_WIDTH:
+ size = (amvdec_read_dos(core, AV_SCRATCH_1) + 1) * 16;
+ dev_err(core->dev, "Unsupported video width: %u\n", size);
+ goto abort;
+ case CMD_BAD_HEIGHT:
+ size = (amvdec_read_dos(core, AV_SCRATCH_1) + 1) * 16;
+ dev_err(core->dev, "Unsupported video height: %u\n", size);
+ goto abort;
+ case 0: /* Unused but not worth printing for */
+ case 9:
+ break;
+ default:
+ dev_info(core->dev, "Unexpected H264 ISR: %08X\n", cmd);
+ break;
+ }
+
+ if (cmd && cmd != CMD_SRC_CHANGE)
+ amvdec_write_dos(core, AV_SCRATCH_0, 0);
+
+ /* Decoder has some SEI data for us ; ignore */
+ if (amvdec_read_dos(core, AV_SCRATCH_J) & SEI_DATA_READY)
+ amvdec_write_dos(core, AV_SCRATCH_J, 0);
+
+ return IRQ_HANDLED;
+abort:
+ amvdec_abort(sess);
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t codec_h264_isr(struct amvdec_session *sess)
+{
+ struct amvdec_core *core = sess->core;
+
+ amvdec_write_dos(core, ASSIST_MBOX1_CLR_REG, 1);
+
+ return IRQ_WAKE_THREAD;
+}
+
+struct amvdec_codec_ops codec_h264_ops = {
+ .start = codec_h264_start,
+ .stop = codec_h264_stop,
+ .load_extended_firmware = codec_h264_load_extended_firmware,
+ .isr = codec_h264_isr,
+ .threaded_isr = codec_h264_threaded_isr,
+ .can_recycle = codec_h264_can_recycle,
+ .recycle = codec_h264_recycle,
+ .eos_sequence = codec_h264_eos_sequence,
+ .resume = codec_h264_resume,
+};
diff --git a/drivers/staging/media/meson/vdec/codec_h264.h b/drivers/staging/media/meson/vdec/codec_h264.h
new file mode 100644
index 000000000000..7cb4fb86ff36
--- /dev/null
+++ b/drivers/staging/media/meson/vdec/codec_h264.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright (C) 2019 BayLibre, SAS
+ * Author: Maxime Jourdan <[email protected]>
+ */
+
+#ifndef __MESON_VDEC_CODEC_H264_H_
+#define __MESON_VDEC_CODEC_H264_H_
+
+#include "vdec.h"
+
+extern struct amvdec_codec_ops codec_h264_ops;
+
+#endif
diff --git a/drivers/staging/media/meson/vdec/vdec_platform.c b/drivers/staging/media/meson/vdec/vdec_platform.c
index 066d4a055894..e9356a46828f 100644
--- a/drivers/staging/media/meson/vdec/vdec_platform.c
+++ b/drivers/staging/media/meson/vdec/vdec_platform.c
@@ -9,9 +9,22 @@
#include "vdec_1.h"
#include "codec_mpeg12.h"
+#include "codec_h264.h"
static const struct amvdec_format vdec_formats_gxbb[] = {
{
+ .pixfmt = V4L2_PIX_FMT_H264,
+ .min_buffers = 2,
+ .max_buffers = 24,
+ .max_width = 1920,
+ .max_height = 1080,
+ .vdec_ops = &vdec_1_ops,
+ .codec_ops = &codec_h264_ops,
+ .firmware_path = "meson/vdec/gxbb_h264.bin",
+ .pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
+ .flags = V4L2_FMT_FLAG_COMPRESSED |
+ V4L2_FMT_FLAG_DYN_RESOLUTION,
+ }, {
.pixfmt = V4L2_PIX_FMT_MPEG1,
.min_buffers = 8,
.max_buffers = 8,
@@ -38,6 +51,18 @@ static const struct amvdec_format vdec_formats_gxbb[] = {
static const struct amvdec_format vdec_formats_gxl[] = {
{
+ .pixfmt = V4L2_PIX_FMT_H264,
+ .min_buffers = 2,
+ .max_buffers = 24,
+ .max_width = 3840,
+ .max_height = 2160,
+ .vdec_ops = &vdec_1_ops,
+ .codec_ops = &codec_h264_ops,
+ .firmware_path = "meson/vdec/gxl_h264.bin",
+ .pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
+ .flags = V4L2_FMT_FLAG_COMPRESSED |
+ V4L2_FMT_FLAG_DYN_RESOLUTION,
+ }, {
.pixfmt = V4L2_PIX_FMT_MPEG1,
.min_buffers = 8,
.max_buffers = 8,
@@ -64,6 +89,18 @@ static const struct amvdec_format vdec_formats_gxl[] = {
static const struct amvdec_format vdec_formats_gxm[] = {
{
+ .pixfmt = V4L2_PIX_FMT_H264,
+ .min_buffers = 2,
+ .max_buffers = 24,
+ .max_width = 3840,
+ .max_height = 2160,
+ .vdec_ops = &vdec_1_ops,
+ .codec_ops = &codec_h264_ops,
+ .firmware_path = "meson/vdec/gxm_h264.bin",
+ .pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
+ .flags = V4L2_FMT_FLAG_COMPRESSED |
+ V4L2_FMT_FLAG_DYN_RESOLUTION,
+ }, {
.pixfmt = V4L2_PIX_FMT_MPEG1,
.min_buffers = 8,
.max_buffers = 8,
@@ -90,6 +127,18 @@ static const struct amvdec_format vdec_formats_gxm[] = {
static const struct amvdec_format vdec_formats_g12a[] = {
{
+ .pixfmt = V4L2_PIX_FMT_H264,
+ .min_buffers = 2,
+ .max_buffers = 24,
+ .max_width = 3840,
+ .max_height = 2160,
+ .vdec_ops = &vdec_1_ops,
+ .codec_ops = &codec_h264_ops,
+ .firmware_path = "meson/vdec/g12a_h264.bin",
+ .pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
+ .flags = V4L2_FMT_FLAG_COMPRESSED |
+ V4L2_FMT_FLAG_DYN_RESOLUTION,
+ }, {
.pixfmt = V4L2_PIX_FMT_MPEG1,
.min_buffers = 8,
.max_buffers = 8,
@@ -116,6 +165,18 @@ static const struct amvdec_format vdec_formats_g12a[] = {
static const struct amvdec_format vdec_formats_sm1[] = {
{
+ .pixfmt = V4L2_PIX_FMT_H264,
+ .min_buffers = 2,
+ .max_buffers = 24,
+ .max_width = 3840,
+ .max_height = 2160,
+ .vdec_ops = &vdec_1_ops,
+ .codec_ops = &codec_h264_ops,
+ .firmware_path = "meson/vdec/g12a_h264.bin",
+ .pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
+ .flags = V4L2_FMT_FLAG_COMPRESSED |
+ V4L2_FMT_FLAG_DYN_RESOLUTION,
+ }, {
.pixfmt = V4L2_PIX_FMT_MPEG1,
.min_buffers = 8,
.max_buffers = 8,
--
2.22.0
Use the previously introduced v4l2-mem2mem core APIs to handle the drainig,
stopped and next-buf-is-last states.
With these changes, the v4l2-compliance still passes with the following
commands :
# v4l2-ctl --stream-mmap --stream-out-mmap --stream-to-hdr out.comp --stream-from in.yuv
>>>><><><><><><><><><><><><><><><><>< 15.53 fps
15.53 fps
><><><><><><><><><><><><>< 13.99 fps
13.99 fps
><><><><><><><><><><><>< 13.52 fps
13.52 fps
><><><><><><><><><><><><>< 13.41 fps
13.41 fps
><><><><><><><><><><><><>< 13.21 fps
13.21 fps
><><><><><><><><><><><>< 13.09 fps
13.09 fps
><><><><><><><
STOP ENCODER
<<<
EOS EVENT
# v4l2-compliance --stream-from in.yuv -s
v4l2-compliance SHA: 7ead0e1856b89f2e19369af452bb03fd0cd16793, 64 bits
[...]
Total for vicodec device /dev/video0: 50, Succeeded: 50, Failed: 0, Warnings: 0
The full output is available at [1]
# v4l2-compliance -d1 --stream-from-hdr out.comp -s
v4l2-compliance SHA: 7ead0e1856b89f2e19369af452bb03fd0cd16793, 64 bits
[...]
Total for vicodec device /dev/video1: 50, Succeeded: 50, Failed: 0, Warnings: 0
The full output is available at [2]
No functional changes should be noticed.
[1] https://termbin.com/25nn
[2] https://termbin.com/dza4
Suggested-by: Hans Verkuil <[email protected]>
Suggested-by: Maxime Jourdan <[email protected]>
Signed-off-by: Neil Armstrong <[email protected]>
---
drivers/media/platform/vicodec/vicodec-core.c | 162 ++++++------------
1 file changed, 52 insertions(+), 110 deletions(-)
diff --git a/drivers/media/platform/vicodec/vicodec-core.c b/drivers/media/platform/vicodec/vicodec-core.c
index 82350097503e..425c47216dae 100644
--- a/drivers/media/platform/vicodec/vicodec-core.c
+++ b/drivers/media/platform/vicodec/vicodec-core.c
@@ -117,15 +117,10 @@ struct vicodec_ctx {
struct vicodec_dev *dev;
bool is_enc;
bool is_stateless;
- bool is_draining;
- bool next_is_last;
- bool has_stopped;
spinlock_t *lock;
struct v4l2_ctrl_handler hdl;
- struct vb2_v4l2_buffer *last_src_buf;
-
/* Source and destination queue data */
struct vicodec_q_data q_data[2];
struct v4l2_fwht_state state;
@@ -431,11 +426,11 @@ static void device_run(void *priv)
v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, false);
spin_lock(ctx->lock);
- if (!ctx->comp_has_next_frame && src_buf == ctx->last_src_buf) {
+ if (!ctx->comp_has_next_frame &&
+ v4l2_m2m_is_last_draining_src_buf(ctx->fh.m2m_ctx, src_buf)) {
dst_buf->flags |= V4L2_BUF_FLAG_LAST;
v4l2_event_queue_fh(&ctx->fh, &vicodec_eos_event);
- ctx->is_draining = false;
- ctx->has_stopped = true;
+ v4l2_m2m_mark_stopped(ctx->fh.m2m_ctx);
}
if (ctx->is_enc || ctx->is_stateless) {
src_buf->sequence = q_src->sequence++;
@@ -586,8 +581,6 @@ static int job_ready(void *priv)
unsigned int max_to_copy;
unsigned int comp_frame_size;
- if (ctx->has_stopped)
- return 0;
if (ctx->source_changed)
return 0;
if (ctx->is_stateless || ctx->is_enc || ctx->comp_has_frame)
@@ -607,7 +600,8 @@ static int job_ready(void *priv)
if (ctx->header_size < sizeof(struct fwht_cframe_hdr)) {
state = get_next_header(ctx, &p, p_src + sz - p);
if (ctx->header_size < sizeof(struct fwht_cframe_hdr)) {
- if (ctx->is_draining && src_buf == ctx->last_src_buf)
+ if (v4l2_m2m_is_last_draining_src_buf(ctx->fh.m2m_ctx,
+ src_buf))
return 1;
job_remove_src_buf(ctx, state);
goto restart;
@@ -636,7 +630,8 @@ static int job_ready(void *priv)
p += copy;
ctx->comp_size += copy;
if (ctx->comp_size < max_to_copy) {
- if (ctx->is_draining && src_buf == ctx->last_src_buf)
+ if (v4l2_m2m_is_last_draining_src_buf(ctx->fh.m2m_ctx,
+ src_buf))
return 1;
job_remove_src_buf(ctx, state);
goto restart;
@@ -1219,41 +1214,6 @@ static int vidioc_s_selection(struct file *file, void *priv,
return 0;
}
-static int vicodec_mark_last_buf(struct vicodec_ctx *ctx)
-{
- struct vb2_v4l2_buffer *next_dst_buf;
- int ret = 0;
-
- spin_lock(ctx->lock);
- if (ctx->is_draining) {
- ret = -EBUSY;
- goto unlock;
- }
- if (ctx->has_stopped)
- goto unlock;
-
- ctx->last_src_buf = v4l2_m2m_last_src_buf(ctx->fh.m2m_ctx);
- ctx->is_draining = true;
- if (ctx->last_src_buf)
- goto unlock;
-
- next_dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
- if (!next_dst_buf) {
- ctx->next_is_last = true;
- goto unlock;
- }
-
- next_dst_buf->flags |= V4L2_BUF_FLAG_LAST;
- vb2_buffer_done(&next_dst_buf->vb2_buf, VB2_BUF_STATE_DONE);
- ctx->is_draining = false;
- ctx->has_stopped = true;
- v4l2_event_queue_fh(&ctx->fh, &vicodec_eos_event);
-
-unlock:
- spin_unlock(ctx->lock);
- return ret;
-}
-
static int vicodec_encoder_cmd(struct file *file, void *fh,
struct v4l2_encoder_cmd *ec)
{
@@ -1268,18 +1228,19 @@ static int vicodec_encoder_cmd(struct file *file, void *fh,
!vb2_is_streaming(&ctx->fh.m2m_ctx->out_q_ctx.q))
return 0;
- if (ec->cmd == V4L2_ENC_CMD_STOP)
- return vicodec_mark_last_buf(ctx);
- ret = 0;
- spin_lock(ctx->lock);
- if (ctx->is_draining) {
- ret = -EBUSY;
- } else if (ctx->has_stopped) {
- ctx->has_stopped = false;
+ ret = v4l2_m2m_ioctl_encoder_cmd(file, fh, ec);
+ if (ret < 0)
+ return ret;
+
+ if (ec->cmd == V4L2_ENC_CMD_STOP &&
+ v4l2_m2m_has_stopped(ctx->fh.m2m_ctx))
+ v4l2_event_queue_fh(&ctx->fh, &vicodec_eos_event);
+
+ if (ec->cmd == V4L2_ENC_CMD_START &&
+ v4l2_m2m_has_stopped(ctx->fh.m2m_ctx))
vb2_clear_last_buffer_dequeued(&ctx->fh.m2m_ctx->cap_q_ctx.q);
- }
- spin_unlock(ctx->lock);
- return ret;
+
+ return 0;
}
static int vicodec_decoder_cmd(struct file *file, void *fh,
@@ -1296,18 +1257,19 @@ static int vicodec_decoder_cmd(struct file *file, void *fh,
!vb2_is_streaming(&ctx->fh.m2m_ctx->out_q_ctx.q))
return 0;
- if (dc->cmd == V4L2_DEC_CMD_STOP)
- return vicodec_mark_last_buf(ctx);
- ret = 0;
- spin_lock(ctx->lock);
- if (ctx->is_draining) {
- ret = -EBUSY;
- } else if (ctx->has_stopped) {
- ctx->has_stopped = false;
+ ret = v4l2_m2m_ioctl_decoder_cmd(file, fh, dc);
+ if (ret < 0)
+ return ret;
+
+ if (dc->cmd == V4L2_DEC_CMD_STOP &&
+ v4l2_m2m_has_stopped(ctx->fh.m2m_ctx))
+ v4l2_event_queue_fh(&ctx->fh, &vicodec_eos_event);
+
+ if (dc->cmd == V4L2_DEC_CMD_START &&
+ v4l2_m2m_has_stopped(ctx->fh.m2m_ctx))
vb2_clear_last_buffer_dequeued(&ctx->fh.m2m_ctx->cap_q_ctx.q);
- }
- spin_unlock(ctx->lock);
- return ret;
+
+ return 0;
}
static int vicodec_enum_framesizes(struct file *file, void *fh,
@@ -1480,23 +1442,21 @@ static void vicodec_buf_queue(struct vb2_buffer *vb)
.u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION,
};
- if (vb2_is_streaming(vq_cap)) {
- if (!V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type) &&
- ctx->next_is_last) {
- unsigned int i;
+ if (!V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type) &&
+ vb2_is_streaming(vb->vb2_queue) &&
+ v4l2_m2m_dst_buf_is_last(ctx->fh.m2m_ctx)) {
+ unsigned int i;
- for (i = 0; i < vb->num_planes; i++)
- vb->planes[i].bytesused = 0;
- vbuf->flags = V4L2_BUF_FLAG_LAST;
- vbuf->field = V4L2_FIELD_NONE;
- vbuf->sequence = get_q_data(ctx, vb->vb2_queue->type)->sequence++;
- vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
- ctx->is_draining = false;
- ctx->has_stopped = true;
- ctx->next_is_last = false;
- v4l2_event_queue_fh(&ctx->fh, &vicodec_eos_event);
- return;
- }
+ for (i = 0; i < vb->num_planes; i++)
+ vb->planes[i].bytesused = 0;
+
+ vbuf->field = V4L2_FIELD_NONE;
+ vbuf->sequence =
+ get_q_data(ctx, vb->vb2_queue->type)->sequence++;
+
+ v4l2_m2m_last_buffer_done(ctx->fh.m2m_ctx, vbuf);
+ v4l2_event_queue_fh(&ctx->fh, &vicodec_eos_event);
+ return;
}
/* buf_queue handles only the first source change event */
@@ -1609,8 +1569,7 @@ static int vicodec_start_streaming(struct vb2_queue *q,
chroma_div = info->width_div * info->height_div;
q_data->sequence = 0;
- if (V4L2_TYPE_IS_OUTPUT(q->type))
- ctx->last_src_buf = NULL;
+ v4l2_m2m_start_streaming(ctx->fh.m2m_ctx, q);
state->gop_cnt = 0;
@@ -1689,29 +1648,12 @@ static void vicodec_stop_streaming(struct vb2_queue *q)
vicodec_return_bufs(q, VB2_BUF_STATE_ERROR);
- if (V4L2_TYPE_IS_OUTPUT(q->type)) {
- if (ctx->is_draining) {
- struct vb2_v4l2_buffer *next_dst_buf;
-
- spin_lock(ctx->lock);
- ctx->last_src_buf = NULL;
- next_dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
- if (!next_dst_buf) {
- ctx->next_is_last = true;
- } else {
- next_dst_buf->flags |= V4L2_BUF_FLAG_LAST;
- vb2_buffer_done(&next_dst_buf->vb2_buf, VB2_BUF_STATE_DONE);
- ctx->is_draining = false;
- ctx->has_stopped = true;
- v4l2_event_queue_fh(&ctx->fh, &vicodec_eos_event);
- }
- spin_unlock(ctx->lock);
- }
- } else {
- ctx->is_draining = false;
- ctx->has_stopped = false;
- ctx->next_is_last = false;
- }
+ v4l2_m2m_stop_streaming(ctx->fh.m2m_ctx, q);
+
+ if (V4L2_TYPE_IS_OUTPUT(q->type) &&
+ v4l2_m2m_has_stopped(ctx->fh.m2m_ctx))
+ v4l2_event_queue_fh(&ctx->fh, &vicodec_eos_event);
+
if (!ctx->is_enc && V4L2_TYPE_IS_OUTPUT(q->type))
ctx->first_source_change_sent = false;
--
2.22.0
From: Maxime Jourdan <[email protected]>
Add all the necessary bits to pass v4l2-compliance in stateful decoding
mode.
Mostly includes tracking the state of the decoder, allowing the OUTPUT
queue to stream while the CAPTURE queue is inactive, handling resolution
change events, draining with V4L2_DEC_CMD_STOP, copying more metadata
from the src buffers to the dst buffers, etc.
Signed-off-by: Maxime Jourdan <[email protected]>
Signed-off-by: Neil Armstrong <[email protected]>
---
drivers/staging/media/meson/vdec/esparser.c | 58 +++++-------
drivers/staging/media/meson/vdec/vdec.c | 90 ++++++++++++-------
drivers/staging/media/meson/vdec/vdec.h | 14 ++-
.../staging/media/meson/vdec/vdec_helpers.c | 88 +++++++++---------
.../staging/media/meson/vdec/vdec_helpers.h | 6 +-
.../staging/media/meson/vdec/vdec_platform.c | 10 +++
6 files changed, 149 insertions(+), 117 deletions(-)
diff --git a/drivers/staging/media/meson/vdec/esparser.c b/drivers/staging/media/meson/vdec/esparser.c
index 95102a4bdc62..adc5c1e81a4c 100644
--- a/drivers/staging/media/meson/vdec/esparser.c
+++ b/drivers/staging/media/meson/vdec/esparser.c
@@ -79,22 +79,28 @@ static irqreturn_t esparser_isr(int irq, void *dev)
* Also append a start code 000001ff at the end to trigger
* the ESPARSER interrupt.
*/
-static u32 esparser_pad_start_code(struct vb2_buffer *vb)
+static u32 esparser_pad_start_code(struct amvdec_core *core, struct vb2_buffer *vb)
{
u32 payload_size = vb2_get_plane_payload(vb, 0);
u32 pad_size = 0;
- u8 *vaddr = vb2_plane_vaddr(vb, 0) + payload_size;
+ u8 *vaddr = vb2_plane_vaddr(vb, 0);
if (payload_size < ESPARSER_MIN_PACKET_SIZE) {
pad_size = ESPARSER_MIN_PACKET_SIZE - payload_size;
- memset(vaddr, 0, pad_size);
+ memset(vaddr + payload_size, 0, pad_size);
}
- memset(vaddr + pad_size, 0, SEARCH_PATTERN_LEN);
- vaddr[pad_size] = 0x00;
- vaddr[pad_size + 1] = 0x00;
- vaddr[pad_size + 2] = 0x01;
- vaddr[pad_size + 3] = 0xff;
+ if ((payload_size + pad_size + SEARCH_PATTERN_LEN) >
+ vb2_plane_size(vb, 0)) {
+ dev_warn(core->dev, "%s: unable to pad start code\n", __func__);
+ return pad_size;
+ }
+
+ memset(vaddr + payload_size + pad_size, 0, SEARCH_PATTERN_LEN);
+ vaddr[payload_size + pad_size] = 0x00;
+ vaddr[payload_size + pad_size + 1] = 0x00;
+ vaddr[payload_size + pad_size + 2] = 0x01;
+ vaddr[payload_size + pad_size + 3] = 0xff;
return pad_size;
}
@@ -180,31 +186,27 @@ esparser_queue(struct amvdec_session *sess, struct vb2_v4l2_buffer *vbuf)
int ret;
struct vb2_buffer *vb = &vbuf->vb2_buf;
struct amvdec_core *core = sess->core;
- struct amvdec_codec_ops *codec_ops = sess->fmt_out->codec_ops;
- u32 num_dst_bufs = 0;
u32 payload_size = vb2_get_plane_payload(vb, 0);
dma_addr_t phy = vb2_dma_contig_plane_dma_addr(vb, 0);
u32 offset;
u32 pad_size;
- if (codec_ops->num_pending_bufs)
- num_dst_bufs = codec_ops->num_pending_bufs(sess);
-
- num_dst_bufs += v4l2_m2m_num_dst_bufs_ready(sess->m2m_ctx);
-
- if (esparser_vififo_get_free_space(sess) < payload_size ||
- atomic_read(&sess->esparser_queued_bufs) >= num_dst_bufs)
+ if (esparser_vififo_get_free_space(sess) < payload_size)
return -EAGAIN;
v4l2_m2m_src_buf_remove_by_buf(sess->m2m_ctx, vbuf);
offset = esparser_get_offset(sess);
- amvdec_add_ts_reorder(sess, vb->timestamp, offset);
- dev_dbg(core->dev, "esparser: ts = %llu pld_size = %u offset = %08X\n",
- vb->timestamp, payload_size, offset);
+ amvdec_add_ts(sess, vb->timestamp, vbuf->timecode, offset, vbuf->flags);
+ dev_dbg(core->dev, "esparser: ts = %llu pld_size = %u offset = %08X flags = %08X\n",
+ vb->timestamp, payload_size, offset, vbuf->flags);
+
+ vbuf->flags = 0;
+ vbuf->field = V4L2_FIELD_NONE;
+ vbuf->sequence = sess->sequence_out++;
- pad_size = esparser_pad_start_code(vb);
+ pad_size = esparser_pad_start_code(core, vb);
ret = esparser_write_data(core, phy, payload_size + pad_size);
if (ret <= 0) {
@@ -216,19 +218,7 @@ esparser_queue(struct amvdec_session *sess, struct vb2_v4l2_buffer *vbuf)
return 0;
}
- /* We need to wait until we parse the first keyframe.
- * All buffers prior to the first keyframe must be dropped.
- */
- if (!sess->keyframe_found)
- usleep_range(1000, 2000);
-
- if (sess->keyframe_found)
- atomic_inc(&sess->esparser_queued_bufs);
- else
- amvdec_remove_ts(sess, vb->timestamp);
-
- vbuf->flags = 0;
- vbuf->field = V4L2_FIELD_NONE;
+ atomic_inc(&sess->esparser_queued_bufs);
v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_DONE);
return 0;
diff --git a/drivers/staging/media/meson/vdec/vdec.c b/drivers/staging/media/meson/vdec/vdec.c
index 5c5dabed2f09..92f0258868b1 100644
--- a/drivers/staging/media/meson/vdec/vdec.c
+++ b/drivers/staging/media/meson/vdec/vdec.c
@@ -168,7 +168,10 @@ static void process_num_buffers(struct vb2_queue *q,
{
const struct amvdec_format *fmt_out = sess->fmt_out;
unsigned int buffers_total = q->num_buffers + *num_buffers;
+ u32 min_buf_capture = v4l2_ctrl_g_ctrl(sess->ctrl_min_buf_capture);
+ if (q->num_buffers + *num_buffers < min_buf_capture)
+ *num_buffers = min_buf_capture - q->num_buffers;
if (is_reqbufs && buffers_total < fmt_out->min_buffers)
*num_buffers = fmt_out->min_buffers - q->num_buffers;
if (buffers_total > fmt_out->max_buffers)
@@ -193,7 +196,8 @@ static int vdec_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
if (*num_planes) {
switch (q->type) {
case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
- if (*num_planes != 1 || sizes[0] < output_size)
+ if (*num_planes != 1 ||
+ sizes[0] < sess->src_buffer_size)
return -EINVAL;
break;
case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
@@ -224,7 +228,7 @@ static int vdec_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
switch (q->type) {
case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
- sizes[0] = amvdec_get_output_size(sess);
+ sizes[0] = sess->src_buffer_size;
*num_planes = 1;
break;
case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
@@ -250,6 +254,7 @@ static int vdec_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
return -EINVAL;
}
+ sess->changed_format = 1;
return 0;
}
@@ -261,10 +266,11 @@ static void vdec_vb2_buf_queue(struct vb2_buffer *vb)
v4l2_m2m_buf_queue(m2m_ctx, vbuf);
- if (!sess->streamon_out || !sess->streamon_cap)
+ if (!sess->streamon_out)
return;
- if (vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+ if (sess->streamon_cap &&
+ vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
vdec_codec_needs_recycle(sess))
vdec_queue_recycle(sess, vb);
@@ -289,16 +295,22 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
else
sess->streamon_cap = 1;
- if (!sess->streamon_out || !sess->streamon_cap)
+ if (!sess->streamon_out)
return 0;
if (sess->status == STATUS_NEEDS_RESUME &&
- q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+ q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+ sess->changed_format) {
codec_ops->resume(sess);
sess->status = STATUS_RUNNING;
return 0;
}
+ if (sess->status == STATUS_RUNNING ||
+ sess->status == STATUS_NEEDS_RESUME ||
+ sess->status == STATUS_INIT)
+ return 0;
+
sess->vififo_size = SIZE_VIFIFO;
sess->vififo_vaddr =
dma_alloc_coherent(sess->core->dev, sess->vififo_size,
@@ -323,13 +335,14 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
goto vififo_free;
sess->sequence_cap = 0;
+ sess->sequence_out = 0;
if (vdec_codec_needs_recycle(sess))
sess->recycle_thread = kthread_run(vdec_recycle_thread, sess,
"vdec_recycle");
- sess->status = STATUS_RUNNING;
+ sess->status = STATUS_INIT;
core->cur_sess = sess;
-
+ schedule_work(&sess->esparser_queue_work);
return 0;
vififo_free:
@@ -386,6 +399,7 @@ static void vdec_stop_streaming(struct vb2_queue *q)
struct vb2_v4l2_buffer *buf;
if (sess->status == STATUS_RUNNING ||
+ sess->status == STATUS_INIT ||
(sess->status == STATUS_NEEDS_RESUME &&
(!sess->streamon_out || !sess->streamon_cap))) {
if (vdec_codec_needs_recycle(sess))
@@ -476,20 +490,34 @@ vdec_try_fmt_common(struct amvdec_session *sess, u32 size,
struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
struct v4l2_plane_pix_format *pfmt = pixmp->plane_fmt;
const struct amvdec_format *fmts = sess->core->platform->formats;
- const struct amvdec_format *fmt_out;
+ const struct amvdec_format *fmt_out = NULL;
+ u32 output_size = 0;
memset(pfmt[0].reserved, 0, sizeof(pfmt[0].reserved));
memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
- if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ switch (f->type) {
+ case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
fmt_out = find_format(fmts, size, pixmp->pixelformat);
if (!fmt_out) {
pixmp->pixelformat = V4L2_PIX_FMT_MPEG2;
fmt_out = find_format(fmts, size, pixmp->pixelformat);
}
+ break;
+ case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+ fmt_out = sess->fmt_out;
+ break;
+ default:
+ return NULL;
+ }
- pfmt[0].sizeimage =
- get_output_size(pixmp->width, pixmp->height);
+ pixmp->width = clamp(pixmp->width, (u32)256, fmt_out->max_width);
+ pixmp->height = clamp(pixmp->height, (u32)144, fmt_out->max_height);
+ output_size = get_output_size(pixmp->width, pixmp->height);
+
+ if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ if (!pfmt[0].sizeimage)
+ pfmt[0].sizeimage = sess->src_buffer_size;
pfmt[0].bytesperline = 0;
pixmp->num_planes = 1;
} else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
@@ -499,35 +527,25 @@ vdec_try_fmt_common(struct amvdec_session *sess, u32 size,
memset(pfmt[1].reserved, 0, sizeof(pfmt[1].reserved));
if (pixmp->pixelformat == V4L2_PIX_FMT_NV12M) {
- pfmt[0].sizeimage =
- get_output_size(pixmp->width, pixmp->height);
+ pfmt[0].sizeimage = output_size;
pfmt[0].bytesperline = ALIGN(pixmp->width, 64);
- pfmt[1].sizeimage =
- get_output_size(pixmp->width, pixmp->height) / 2;
+ pfmt[1].sizeimage = output_size / 2;
pfmt[1].bytesperline = ALIGN(pixmp->width, 64);
pixmp->num_planes = 2;
} else if (pixmp->pixelformat == V4L2_PIX_FMT_YUV420M) {
- pfmt[0].sizeimage =
- get_output_size(pixmp->width, pixmp->height);
+ pfmt[0].sizeimage = output_size;
pfmt[0].bytesperline = ALIGN(pixmp->width, 64);
- pfmt[1].sizeimage =
- get_output_size(pixmp->width, pixmp->height) / 4;
+ pfmt[1].sizeimage = output_size / 4;
pfmt[1].bytesperline = ALIGN(pixmp->width, 64) / 2;
- pfmt[2].sizeimage =
- get_output_size(pixmp->width, pixmp->height) / 4;
+ pfmt[2].sizeimage = output_size / 2;
pfmt[2].bytesperline = ALIGN(pixmp->width, 64) / 2;
pixmp->num_planes = 3;
}
- } else {
- return NULL;
}
- pixmp->width = clamp(pixmp->width, (u32)256, fmt_out->max_width);
- pixmp->height = clamp(pixmp->height, (u32)144, fmt_out->max_height);
-
if (pixmp->field == V4L2_FIELD_ANY)
pixmp->field = V4L2_FIELD_NONE;
@@ -586,6 +604,8 @@ static int vdec_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
orig_pixmp = *pixmp;
fmt_out = vdec_try_fmt_common(sess, num_formats, f);
+ if (!fmt_out)
+ return -EINVAL;
if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
pixfmt_out = pixmp->pixelformat;
@@ -610,6 +630,7 @@ static int vdec_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
sess->ycbcr_enc = pixmp->ycbcr_enc;
sess->quantization = pixmp->quantization;
sess->xfer_func = pixmp->xfer_func;
+ sess->src_buffer_size = pixmp->plane_fmt[0].sizeimage;
}
memset(&format, 0, sizeof(format));
@@ -701,29 +722,31 @@ vdec_decoder_cmd(struct file *file, void *fh, struct v4l2_decoder_cmd *cmd)
if (!(sess->streamon_out & sess->streamon_cap))
return 0;
- /* Currently not handled since we do not support dynamic resolution
- * for MPEG2. We consider both queues streaming to mean that the
- * decoding session is started
- */
- if (cmd->cmd == V4L2_DEC_CMD_START)
+ if (cmd->cmd == V4L2_DEC_CMD_START) {
+ v4l2_m2m_clear_state(sess->m2m_ctx);
+ sess->should_stop = 0;
return 0;
+ }
/* Should not happen */
if (cmd->cmd != V4L2_DEC_CMD_STOP)
return -EINVAL;
dev_dbg(dev, "Received V4L2_DEC_CMD_STOP\n");
+
sess->should_stop = 1;
- vdec_wait_inactive(sess);
+ v4l2_m2m_mark_stopped(sess->m2m_ctx);
if (codec_ops->drain) {
+ vdec_wait_inactive(sess);
codec_ops->drain(sess);
} else if (codec_ops->eos_sequence) {
u32 len;
const u8 *data = codec_ops->eos_sequence(&len);
esparser_queue_eos(sess->core, data, len);
+ vdec_wait_inactive(sess);
}
return ret;
@@ -883,6 +906,7 @@ static int vdec_open(struct file *file)
sess->height = 720;
sess->pixelaspect.numerator = 1;
sess->pixelaspect.denominator = 1;
+ sess->src_buffer_size = SZ_1M;
INIT_LIST_HEAD(&sess->timestamps);
INIT_LIST_HEAD(&sess->bufs_recycle);
diff --git a/drivers/staging/media/meson/vdec/vdec.h b/drivers/staging/media/meson/vdec/vdec.h
index 0faa1ec4858e..f95445ac0658 100644
--- a/drivers/staging/media/meson/vdec/vdec.h
+++ b/drivers/staging/media/meson/vdec/vdec.h
@@ -29,13 +29,19 @@ struct amvdec_buffer {
* struct amvdec_timestamp - stores a src timestamp along with a VIFIFO offset
*
* @list: used to make lists out of this struct
- * @ts: timestamp
+ * @tc: timecode from the v4l2 buffer
+ * @ts: timestamp from the VB2 buffer
* @offset: offset in the VIFIFO where the associated packet was written
+ * @flags: flags from the v4l2 buffer
+ * @used_count: times this timestamp was checked for a match with a dst buffer
*/
struct amvdec_timestamp {
struct list_head list;
+ struct v4l2_timecode tc;
u64 ts;
u32 offset;
+ u32 flags;
+ u32 used_count;
};
struct amvdec_session;
@@ -165,6 +171,7 @@ struct amvdec_format {
enum amvdec_status {
STATUS_STOPPED,
+ STATUS_INIT,
STATUS_RUNNING,
STATUS_NEEDS_RESUME,
};
@@ -180,6 +187,7 @@ enum amvdec_status {
* @ctrl_min_buf_capture: V4L2 control V4L2_CID_MIN_BUFFERS_FOR_CAPTURE
* @fmt_out: vdec pixel format for the OUTPUT queue
* @pixfmt_cap: V4L2 pixel format for the CAPTURE queue
+ * @src_buffer_size: size in bytes of the OUTPUT buffers' only plane
* @width: current picture width
* @height: current picture height
* @colorspace: current colorspace
@@ -221,6 +229,7 @@ struct amvdec_session {
const struct amvdec_format *fmt_out;
u32 pixfmt_cap;
+ u32 src_buffer_size;
u32 width;
u32 height;
@@ -235,10 +244,11 @@ struct amvdec_session {
struct work_struct esparser_queue_work;
unsigned int streamon_cap, streamon_out;
- unsigned int sequence_cap;
+ unsigned int sequence_cap, sequence_out;
unsigned int should_stop;
unsigned int keyframe_found;
unsigned int num_dst_bufs;
+ unsigned int changed_format;
u8 canvas_alloc[MAX_CANVAS];
u32 canvas_num;
diff --git a/drivers/staging/media/meson/vdec/vdec_helpers.c b/drivers/staging/media/meson/vdec/vdec_helpers.c
index f16948bdbf2f..a4970ec1bf2e 100644
--- a/drivers/staging/media/meson/vdec/vdec_helpers.c
+++ b/drivers/staging/media/meson/vdec/vdec_helpers.c
@@ -200,33 +200,23 @@ int amvdec_set_canvases(struct amvdec_session *sess,
}
EXPORT_SYMBOL_GPL(amvdec_set_canvases);
-void amvdec_add_ts_reorder(struct amvdec_session *sess, u64 ts, u32 offset)
+void amvdec_add_ts(struct amvdec_session *sess, u64 ts,
+ struct v4l2_timecode tc, u32 offset, u32 vbuf_flags)
{
- struct amvdec_timestamp *new_ts, *tmp;
+ struct amvdec_timestamp *new_ts;
unsigned long flags;
- new_ts = kmalloc(sizeof(*new_ts), GFP_KERNEL);
+ new_ts = kzalloc(sizeof(*new_ts), GFP_KERNEL);
new_ts->ts = ts;
+ new_ts->tc = tc;
new_ts->offset = offset;
+ new_ts->flags = vbuf_flags;
spin_lock_irqsave(&sess->ts_spinlock, flags);
-
- if (list_empty(&sess->timestamps))
- goto add_tail;
-
- list_for_each_entry(tmp, &sess->timestamps, list) {
- if (ts <= tmp->ts) {
- list_add_tail(&new_ts->list, &tmp->list);
- goto unlock;
- }
- }
-
-add_tail:
list_add_tail(&new_ts->list, &sess->timestamps);
-unlock:
spin_unlock_irqrestore(&sess->ts_spinlock, flags);
}
-EXPORT_SYMBOL_GPL(amvdec_add_ts_reorder);
+EXPORT_SYMBOL_GPL(amvdec_add_ts);
void amvdec_remove_ts(struct amvdec_session *sess, u64 ts)
{
@@ -251,8 +241,8 @@ EXPORT_SYMBOL_GPL(amvdec_remove_ts);
static void dst_buf_done(struct amvdec_session *sess,
struct vb2_v4l2_buffer *vbuf,
- u32 field,
- u64 timestamp)
+ u32 field, u64 timestamp,
+ struct v4l2_timecode timecode, u32 flags)
{
struct device *dev = sess->core->dev_dec;
u32 output_size = amvdec_get_output_size(sess);
@@ -271,19 +261,23 @@ static void dst_buf_done(struct amvdec_session *sess,
vbuf->vb2_buf.timestamp = timestamp;
vbuf->sequence = sess->sequence_cap++;
+ vbuf->flags = flags;
+ vbuf->timecode = timecode;
if (sess->should_stop &&
- atomic_read(&sess->esparser_queued_bufs) <= 2) {
+ atomic_read(&sess->esparser_queued_bufs) <= 1) {
const struct v4l2_event ev = { .type = V4L2_EVENT_EOS };
- dev_dbg(dev, "Signaling EOS\n");
+ dev_dbg(dev, "Signaling EOS, sequence_cap = %u\n",
+ sess->sequence_cap - 1);
v4l2_event_queue_fh(&sess->fh, &ev);
vbuf->flags |= V4L2_BUF_FLAG_LAST;
} else if (sess->should_stop)
dev_dbg(dev, "should_stop, %u bufs remain\n",
atomic_read(&sess->esparser_queued_bufs));
- dev_dbg(dev, "Buffer %u done\n", vbuf->vb2_buf.index);
+ dev_dbg(dev, "Buffer %u done, ts = %llu, flags = %08X\n",
+ vbuf->vb2_buf.index, timestamp, flags);
vbuf->field = field;
v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_DONE);
@@ -297,7 +291,9 @@ void amvdec_dst_buf_done(struct amvdec_session *sess,
struct device *dev = sess->core->dev_dec;
struct amvdec_timestamp *tmp;
struct list_head *timestamps = &sess->timestamps;
+ struct v4l2_timecode timecode;
u64 timestamp;
+ u32 vbuf_flags;
unsigned long flags;
spin_lock_irqsave(&sess->ts_spinlock, flags);
@@ -312,11 +308,13 @@ void amvdec_dst_buf_done(struct amvdec_session *sess,
tmp = list_first_entry(timestamps, struct amvdec_timestamp, list);
timestamp = tmp->ts;
+ timecode = tmp->tc;
+ vbuf_flags = tmp->flags;
list_del(&tmp->list);
kfree(tmp);
spin_unlock_irqrestore(&sess->ts_spinlock, flags);
- dst_buf_done(sess, vbuf, field, timestamp);
+ dst_buf_done(sess, vbuf, field, timestamp, timecode, vbuf_flags);
atomic_dec(&sess->esparser_queued_bufs);
}
EXPORT_SYMBOL_GPL(amvdec_dst_buf_done);
@@ -328,48 +326,43 @@ void amvdec_dst_buf_done_offset(struct amvdec_session *sess,
struct device *dev = sess->core->dev_dec;
struct amvdec_timestamp *match = NULL;
struct amvdec_timestamp *tmp, *n;
+ struct v4l2_timecode timecode = { 0 };
u64 timestamp = 0;
+ u32 vbuf_flags = 0;
unsigned long flags;
spin_lock_irqsave(&sess->ts_spinlock, flags);
/* Look for our vififo offset to get the corresponding timestamp. */
list_for_each_entry_safe(tmp, n, &sess->timestamps, list) {
- s64 delta = (s64)offset - tmp->offset;
-
- /* Offsets reported by codecs usually differ slightly,
- * so we need some wiggle room.
- * 4KiB being the minimum packet size, there is no risk here.
- */
- if (delta > (-1 * (s32)SZ_4K) && delta < SZ_4K) {
- match = tmp;
+ if (tmp->offset > offset) {
+ /*
+ * Delete any record that remained unused for 32 match
+ * checks
+ */
+ if (tmp->used_count++ >= 32) {
+ list_del(&tmp->list);
+ kfree(tmp);
+ }
break;
}
- if (!allow_drop)
- continue;
-
- /* Delete any timestamp entry that appears before our target
- * (not all src packets/timestamps lead to a frame)
- */
- if (delta > 0 || delta < -1 * (s32)sess->vififo_size) {
- atomic_dec(&sess->esparser_queued_bufs);
- list_del(&tmp->list);
- kfree(tmp);
- }
+ match = tmp;
}
if (!match) {
- dev_dbg(dev, "Buffer %u done but can't match offset (%08X)\n",
+ dev_err(dev, "Buffer %u done but can't match offset (%08X)\n",
vbuf->vb2_buf.index, offset);
} else {
timestamp = match->ts;
+ timecode = match->tc;
+ vbuf_flags = match->flags;
list_del(&match->list);
kfree(match);
}
spin_unlock_irqrestore(&sess->ts_spinlock, flags);
- dst_buf_done(sess, vbuf, field, timestamp);
+ dst_buf_done(sess, vbuf, field, timestamp, timecode, vbuf_flags);
if (match)
atomic_dec(&sess->esparser_queued_bufs);
}
@@ -420,16 +413,19 @@ void amvdec_src_change(struct amvdec_session *sess, u32 width,
v4l2_ctrl_s_ctrl(sess->ctrl_min_buf_capture, dpb_size);
- /* Check if the capture queue is already configured well for our
+ /*
+ * Check if the capture queue is already configured well for our
* usecase. If so, keep decoding with it and do not send the event
*/
- if (sess->width == width &&
+ if (sess->streamon_cap &&
+ sess->width == width &&
sess->height == height &&
dpb_size <= sess->num_dst_bufs) {
sess->fmt_out->codec_ops->resume(sess);
return;
}
+ sess->changed_format = 0;
sess->width = width;
sess->height = height;
sess->status = STATUS_NEEDS_RESUME;
diff --git a/drivers/staging/media/meson/vdec/vdec_helpers.h b/drivers/staging/media/meson/vdec/vdec_helpers.h
index a455a9ee1cc2..165e6293ffba 100644
--- a/drivers/staging/media/meson/vdec/vdec_helpers.h
+++ b/drivers/staging/media/meson/vdec/vdec_helpers.h
@@ -44,13 +44,15 @@ void amvdec_dst_buf_done_offset(struct amvdec_session *sess,
u32 offset, u32 field, bool allow_drop);
/**
- * amvdec_add_ts_reorder() - Add a timestamp to the list in chronological order
+ * amvdec_add_ts() - Add a timestamp to the list
*
* @sess: current session
* @ts: timestamp to add
* @offset: offset in the VIFIFO where the associated packet was written
+ * @flags the vb2_v4l2_buffer flags
*/
-void amvdec_add_ts_reorder(struct amvdec_session *sess, u64 ts, u32 offset);
+void amvdec_add_ts(struct amvdec_session *sess, u64 ts,
+ struct v4l2_timecode tc, u32 offset, u32 flags);
void amvdec_remove_ts(struct amvdec_session *sess, u64 ts);
/**
diff --git a/drivers/staging/media/meson/vdec/vdec_platform.c b/drivers/staging/media/meson/vdec/vdec_platform.c
index ea39f8209ec7..066d4a055894 100644
--- a/drivers/staging/media/meson/vdec/vdec_platform.c
+++ b/drivers/staging/media/meson/vdec/vdec_platform.c
@@ -21,6 +21,7 @@ static const struct amvdec_format vdec_formats_gxbb[] = {
.codec_ops = &codec_mpeg12_ops,
.firmware_path = "meson/vdec/gxl_mpeg12.bin",
.pixfmts_cap = { V4L2_PIX_FMT_NV12M, V4L2_PIX_FMT_YUV420M, 0 },
+ .flags = V4L2_FMT_FLAG_COMPRESSED,
}, {
.pixfmt = V4L2_PIX_FMT_MPEG2,
.min_buffers = 8,
@@ -31,6 +32,7 @@ static const struct amvdec_format vdec_formats_gxbb[] = {
.codec_ops = &codec_mpeg12_ops,
.firmware_path = "meson/vdec/gxl_mpeg12.bin",
.pixfmts_cap = { V4L2_PIX_FMT_NV12M, V4L2_PIX_FMT_YUV420M, 0 },
+ .flags = V4L2_FMT_FLAG_COMPRESSED,
},
};
@@ -45,6 +47,7 @@ static const struct amvdec_format vdec_formats_gxl[] = {
.codec_ops = &codec_mpeg12_ops,
.firmware_path = "meson/vdec/gxl_mpeg12.bin",
.pixfmts_cap = { V4L2_PIX_FMT_NV12M, V4L2_PIX_FMT_YUV420M, 0 },
+ .flags = V4L2_FMT_FLAG_COMPRESSED,
}, {
.pixfmt = V4L2_PIX_FMT_MPEG2,
.min_buffers = 8,
@@ -55,6 +58,7 @@ static const struct amvdec_format vdec_formats_gxl[] = {
.codec_ops = &codec_mpeg12_ops,
.firmware_path = "meson/vdec/gxl_mpeg12.bin",
.pixfmts_cap = { V4L2_PIX_FMT_NV12M, V4L2_PIX_FMT_YUV420M, 0 },
+ .flags = V4L2_FMT_FLAG_COMPRESSED,
},
};
@@ -69,6 +73,7 @@ static const struct amvdec_format vdec_formats_gxm[] = {
.codec_ops = &codec_mpeg12_ops,
.firmware_path = "meson/vdec/gxl_mpeg12.bin",
.pixfmts_cap = { V4L2_PIX_FMT_NV12M, V4L2_PIX_FMT_YUV420M, 0 },
+ .flags = V4L2_FMT_FLAG_COMPRESSED,
}, {
.pixfmt = V4L2_PIX_FMT_MPEG2,
.min_buffers = 8,
@@ -79,6 +84,7 @@ static const struct amvdec_format vdec_formats_gxm[] = {
.codec_ops = &codec_mpeg12_ops,
.firmware_path = "meson/vdec/gxl_mpeg12.bin",
.pixfmts_cap = { V4L2_PIX_FMT_NV12M, V4L2_PIX_FMT_YUV420M, 0 },
+ .flags = V4L2_FMT_FLAG_COMPRESSED,
},
};
@@ -93,6 +99,7 @@ static const struct amvdec_format vdec_formats_g12a[] = {
.codec_ops = &codec_mpeg12_ops,
.firmware_path = "meson/vdec/gxl_mpeg12.bin",
.pixfmts_cap = { V4L2_PIX_FMT_NV12M, V4L2_PIX_FMT_YUV420M, 0 },
+ .flags = V4L2_FMT_FLAG_COMPRESSED,
}, {
.pixfmt = V4L2_PIX_FMT_MPEG2,
.min_buffers = 8,
@@ -103,6 +110,7 @@ static const struct amvdec_format vdec_formats_g12a[] = {
.codec_ops = &codec_mpeg12_ops,
.firmware_path = "meson/vdec/gxl_mpeg12.bin",
.pixfmts_cap = { V4L2_PIX_FMT_NV12M, V4L2_PIX_FMT_YUV420M, 0 },
+ .flags = V4L2_FMT_FLAG_COMPRESSED,
},
};
@@ -117,6 +125,7 @@ static const struct amvdec_format vdec_formats_sm1[] = {
.codec_ops = &codec_mpeg12_ops,
.firmware_path = "meson/vdec/gxl_mpeg12.bin",
.pixfmts_cap = { V4L2_PIX_FMT_NV12M, V4L2_PIX_FMT_YUV420M, 0 },
+ .flags = V4L2_FMT_FLAG_COMPRESSED,
}, {
.pixfmt = V4L2_PIX_FMT_MPEG2,
.min_buffers = 8,
@@ -127,6 +136,7 @@ static const struct amvdec_format vdec_formats_sm1[] = {
.codec_ops = &codec_mpeg12_ops,
.firmware_path = "meson/vdec/gxl_mpeg12.bin",
.pixfmts_cap = { V4L2_PIX_FMT_NV12M, V4L2_PIX_FMT_YUV420M, 0 },
+ .flags = V4L2_FMT_FLAG_COMPRESSED,
},
};
--
2.22.0
On 2/6/20 9:26 AM, Neil Armstrong wrote:
> From: Maxime Jourdan <[email protected]>
>
> Add support for the H264 compressed format (V4L2_PIX_FMT_H264).
>
> Signed-off-by: Maxime Jourdan <[email protected]>
> Signed-off-by: Neil Armstrong <[email protected]>
> ---
> drivers/staging/media/meson/vdec/Makefile | 2 +-
> drivers/staging/media/meson/vdec/codec_h264.c | 482 ++++++++++++++++++
> drivers/staging/media/meson/vdec/codec_h264.h | 14 +
> .../staging/media/meson/vdec/vdec_platform.c | 61 +++
> 4 files changed, 558 insertions(+), 1 deletion(-)
> create mode 100644 drivers/staging/media/meson/vdec/codec_h264.c
> create mode 100644 drivers/staging/media/meson/vdec/codec_h264.h
I get warnings when compiling this:
drivers/staging/media/meson/vdec/codec_h264.c:322: warning: Function parameter or member 'sess' not described in 'codec_h264_src_change'
drivers/staging/media/meson/vdec/codec_h264.c:361: warning: Function parameter or member 'core' not described in 'get_offset_msb'
drivers/staging/media/meson/vdec/codec_h264.c:361: warning: Function parameter or member 'frame_num' not described in 'get_offset_msb'
Regards,
Hans
>
> diff --git a/drivers/staging/media/meson/vdec/Makefile b/drivers/staging/media/meson/vdec/Makefile
> index 6bea129084b7..711d990c760e 100644
> --- a/drivers/staging/media/meson/vdec/Makefile
> +++ b/drivers/staging/media/meson/vdec/Makefile
> @@ -3,6 +3,6 @@
>
> meson-vdec-objs = esparser.o vdec.o vdec_helpers.o vdec_platform.o
> meson-vdec-objs += vdec_1.o
> -meson-vdec-objs += codec_mpeg12.o
> +meson-vdec-objs += codec_mpeg12.o codec_h264.o
>
> obj-$(CONFIG_VIDEO_MESON_VDEC) += meson-vdec.o
> diff --git a/drivers/staging/media/meson/vdec/codec_h264.c b/drivers/staging/media/meson/vdec/codec_h264.c
> new file mode 100644
> index 000000000000..4528a6a01c3d
> --- /dev/null
> +++ b/drivers/staging/media/meson/vdec/codec_h264.c
> @@ -0,0 +1,482 @@
> +// SPDX-License-Identifier: GPL-2.0+
> +/*
> + * Copyright (C) 2019 BayLibre, SAS
> + * Author: Maxime Jourdan <[email protected]>
> + */
> +
> +#include <media/v4l2-mem2mem.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "vdec_helpers.h"
> +#include "dos_regs.h"
> +
> +#define SIZE_EXT_FW (20 * SZ_1K)
> +#define SIZE_WORKSPACE 0x1ee000
> +#define SIZE_SEI (8 * SZ_1K)
> +
> +/*
> + * Offset added by the firmware which must be substracted
> + * from the workspace phyaddr
> + */
> +#define WORKSPACE_BUF_OFFSET 0x1000000
> +
> +/* ISR status */
> +#define CMD_MASK GENMASK(7, 0)
> +#define CMD_SRC_CHANGE 1
> +#define CMD_FRAMES_READY 2
> +#define CMD_FATAL_ERROR 6
> +#define CMD_BAD_WIDTH 7
> +#define CMD_BAD_HEIGHT 8
> +
> +#define SEI_DATA_READY BIT(15)
> +
> +/* Picture type */
> +#define PIC_TOP_BOT 5
> +#define PIC_BOT_TOP 6
> +
> +/* Size of Motion Vector per macroblock */
> +#define MB_MV_SIZE 96
> +
> +/* Frame status data */
> +#define PIC_STRUCT_BIT 5
> +#define PIC_STRUCT_MASK GENMASK(2, 0)
> +#define BUF_IDX_MASK GENMASK(4, 0)
> +#define ERROR_FLAG BIT(9)
> +#define OFFSET_BIT 16
> +#define OFFSET_MASK GENMASK(15, 0)
> +
> +/* Bitstream parsed data */
> +#define MB_TOTAL_BIT 8
> +#define MB_TOTAL_MASK GENMASK(15, 0)
> +#define MB_WIDTH_MASK GENMASK(7, 0)
> +#define MAX_REF_BIT 24
> +#define MAX_REF_MASK GENMASK(6, 0)
> +#define AR_IDC_BIT 16
> +#define AR_IDC_MASK GENMASK(7, 0)
> +#define AR_PRESENT_FLAG BIT(0)
> +#define AR_EXTEND 0xff
> +
> +/*
> + * Buffer to send to the ESPARSER to signal End Of Stream for H.264.
> + * This is a 16x16 encoded picture that will trigger drain firmware-side.
> + * There is no known alternative.
> + */
> +static const u8 eos_sequence[SZ_4K] = {
> + 0x00, 0x00, 0x00, 0x01, 0x06, 0x05, 0xff, 0xe4, 0xdc, 0x45, 0xe9, 0xbd,
> + 0xe6, 0xd9, 0x48, 0xb7, 0x96, 0x2c, 0xd8, 0x20, 0xd9, 0x23, 0xee, 0xef,
> + 0x78, 0x32, 0x36, 0x34, 0x20, 0x2d, 0x20, 0x63, 0x6f, 0x72, 0x65, 0x20,
> + 0x36, 0x37, 0x20, 0x72, 0x31, 0x31, 0x33, 0x30, 0x20, 0x38, 0x34, 0x37,
> + 0x35, 0x39, 0x37, 0x37, 0x20, 0x2d, 0x20, 0x48, 0x2e, 0x32, 0x36, 0x34,
> + 0x2f, 0x4d, 0x50, 0x45, 0x47, 0x2d, 0x34, 0x20, 0x41, 0x56, 0x43, 0x20,
> + 0x63, 0x6f, 0x64, 0x65, 0x63, 0x20, 0x2d, 0x20, 0x43, 0x6f, 0x70, 0x79,
> + 0x6c, 0x65, 0x66, 0x74, 0x20, 0x32, 0x30, 0x30, 0x33, 0x2d, 0x32, 0x30,
> + 0x30, 0x39, 0x20, 0x2d, 0x20, 0x68, 0x74, 0x74, 0x70, 0x3a, 0x2f, 0x2f,
> + 0x77, 0x77, 0x77, 0x2e, 0x76, 0x69, 0x64, 0x65, 0x6f, 0x6c, 0x61, 0x6e,
> + 0x2e, 0x6f, 0x72, 0x67, 0x2f, 0x78, 0x32, 0x36, 0x34, 0x2e, 0x68, 0x74,
> + 0x6d, 0x6c, 0x20, 0x2d, 0x20, 0x6f, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x73,
> + 0x3a, 0x20, 0x63, 0x61, 0x62, 0x61, 0x63, 0x3d, 0x31, 0x20, 0x72, 0x65,
> + 0x66, 0x3d, 0x31, 0x20, 0x64, 0x65, 0x62, 0x6c, 0x6f, 0x63, 0x6b, 0x3d,
> + 0x31, 0x3a, 0x30, 0x3a, 0x30, 0x20, 0x61, 0x6e, 0x61, 0x6c, 0x79, 0x73,
> + 0x65, 0x3d, 0x30, 0x78, 0x31, 0x3a, 0x30, 0x78, 0x31, 0x31, 0x31, 0x20,
> + 0x6d, 0x65, 0x3d, 0x68, 0x65, 0x78, 0x20, 0x73, 0x75, 0x62, 0x6d, 0x65,
> + 0x3d, 0x36, 0x20, 0x70, 0x73, 0x79, 0x5f, 0x72, 0x64, 0x3d, 0x31, 0x2e,
> + 0x30, 0x3a, 0x30, 0x2e, 0x30, 0x20, 0x6d, 0x69, 0x78, 0x65, 0x64, 0x5f,
> + 0x72, 0x65, 0x66, 0x3d, 0x30, 0x20, 0x6d, 0x65, 0x5f, 0x72, 0x61, 0x6e,
> + 0x67, 0x65, 0x3d, 0x31, 0x36, 0x20, 0x63, 0x68, 0x72, 0x6f, 0x6d, 0x61,
> + 0x5f, 0x6d, 0x65, 0x3d, 0x31, 0x20, 0x74, 0x72, 0x65, 0x6c, 0x6c, 0x69,
> + 0x73, 0x3d, 0x30, 0x20, 0x38, 0x78, 0x38, 0x64, 0x63, 0x74, 0x3d, 0x30,
> + 0x20, 0x63, 0x71, 0x6d, 0x3d, 0x30, 0x20, 0x64, 0x65, 0x61, 0x64, 0x7a,
> + 0x6f, 0x6e, 0x65, 0x3d, 0x32, 0x31, 0x2c, 0x31, 0x31, 0x20, 0x63, 0x68,
> + 0x72, 0x6f, 0x6d, 0x61, 0x5f, 0x71, 0x70, 0x5f, 0x6f, 0x66, 0x66, 0x73,
> + 0x65, 0x74, 0x3d, 0x2d, 0x32, 0x20, 0x74, 0x68, 0x72, 0x65, 0x61, 0x64,
> + 0x73, 0x3d, 0x31, 0x20, 0x6e, 0x72, 0x3d, 0x30, 0x20, 0x64, 0x65, 0x63,
> + 0x69, 0x6d, 0x61, 0x74, 0x65, 0x3d, 0x31, 0x20, 0x6d, 0x62, 0x61, 0x66,
> + 0x66, 0x3d, 0x30, 0x20, 0x62, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x73, 0x3d,
> + 0x30, 0x20, 0x6b, 0x65, 0x79, 0x69, 0x6e, 0x74, 0x3d, 0x32, 0x35, 0x30,
> + 0x20, 0x6b, 0x65, 0x79, 0x69, 0x6e, 0x74, 0x5f, 0x6d, 0x69, 0x6e, 0x3d,
> + 0x32, 0x35, 0x20, 0x73, 0x63, 0x65, 0x6e, 0x65, 0x63, 0x75, 0x74, 0x3d,
> + 0x34, 0x30, 0x20, 0x72, 0x63, 0x3d, 0x61, 0x62, 0x72, 0x20, 0x62, 0x69,
> + 0x74, 0x72, 0x61, 0x74, 0x65, 0x3d, 0x31, 0x30, 0x20, 0x72, 0x61, 0x74,
> + 0x65, 0x74, 0x6f, 0x6c, 0x3d, 0x31, 0x2e, 0x30, 0x20, 0x71, 0x63, 0x6f,
> + 0x6d, 0x70, 0x3d, 0x30, 0x2e, 0x36, 0x30, 0x20, 0x71, 0x70, 0x6d, 0x69,
> + 0x6e, 0x3d, 0x31, 0x30, 0x20, 0x71, 0x70, 0x6d, 0x61, 0x78, 0x3d, 0x35,
> + 0x31, 0x20, 0x71, 0x70, 0x73, 0x74, 0x65, 0x70, 0x3d, 0x34, 0x20, 0x69,
> + 0x70, 0x5f, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x3d, 0x31, 0x2e, 0x34, 0x30,
> + 0x20, 0x61, 0x71, 0x3d, 0x31, 0x3a, 0x31, 0x2e, 0x30, 0x30, 0x00, 0x80,
> + 0x00, 0x00, 0x00, 0x01, 0x67, 0x4d, 0x40, 0x0a, 0x9a, 0x74, 0xf4, 0x20,
> + 0x00, 0x00, 0x03, 0x00, 0x20, 0x00, 0x00, 0x06, 0x51, 0xe2, 0x44, 0xd4,
> + 0x00, 0x00, 0x00, 0x01, 0x68, 0xee, 0x32, 0xc8, 0x00, 0x00, 0x00, 0x01,
> + 0x65, 0x88, 0x80, 0x20, 0x00, 0x08, 0x7f, 0xea, 0x6a, 0xe2, 0x99, 0xb6,
> + 0x57, 0xae, 0x49, 0x30, 0xf5, 0xfe, 0x5e, 0x46, 0x0b, 0x72, 0x44, 0xc4,
> + 0xe1, 0xfc, 0x62, 0xda, 0xf1, 0xfb, 0xa2, 0xdb, 0xd6, 0xbe, 0x5c, 0xd7,
> + 0x24, 0xa3, 0xf5, 0xb9, 0x2f, 0x57, 0x16, 0x49, 0x75, 0x47, 0x77, 0x09,
> + 0x5c, 0xa1, 0xb4, 0xc3, 0x4f, 0x60, 0x2b, 0xb0, 0x0c, 0xc8, 0xd6, 0x66,
> + 0xba, 0x9b, 0x82, 0x29, 0x33, 0x92, 0x26, 0x99, 0x31, 0x1c, 0x7f, 0x9b,
> + 0x00, 0x00, 0x01, 0x0ff,
> +};
> +
> +static const u8 *codec_h264_eos_sequence(u32 *len)
> +{
> + *len = ARRAY_SIZE(eos_sequence);
> + return eos_sequence;
> +}
> +
> +struct codec_h264 {
> + /* H.264 decoder requires an extended firmware */
> + void *ext_fw_vaddr;
> + dma_addr_t ext_fw_paddr;
> +
> + /* Buffer for the H.264 Workspace */
> + void *workspace_vaddr;
> + dma_addr_t workspace_paddr;
> +
> + /* Buffer for the H.264 references MV */
> + void *ref_vaddr;
> + dma_addr_t ref_paddr;
> + u32 ref_size;
> +
> + /* Buffer for parsed SEI data */
> + void *sei_vaddr;
> + dma_addr_t sei_paddr;
> +
> + u32 mb_width;
> + u32 mb_height;
> + u32 max_refs;
> +};
> +
> +static int codec_h264_can_recycle(struct amvdec_core *core)
> +{
> + return !amvdec_read_dos(core, AV_SCRATCH_7) ||
> + !amvdec_read_dos(core, AV_SCRATCH_8);
> +}
> +
> +static void codec_h264_recycle(struct amvdec_core *core, u32 buf_idx)
> +{
> + /*
> + * Tell the firmware it can recycle this buffer.
> + * AV_SCRATCH_8 serves the same purpose.
> + */
> + if (!amvdec_read_dos(core, AV_SCRATCH_7))
> + amvdec_write_dos(core, AV_SCRATCH_7, buf_idx + 1);
> + else
> + amvdec_write_dos(core, AV_SCRATCH_8, buf_idx + 1);
> +}
> +
> +static int codec_h264_start(struct amvdec_session *sess)
> +{
> + u32 workspace_offset;
> + struct amvdec_core *core = sess->core;
> + struct codec_h264 *h264 = sess->priv;
> +
> + /* Allocate some memory for the H.264 decoder's state */
> + h264->workspace_vaddr =
> + dma_alloc_coherent(core->dev, SIZE_WORKSPACE,
> + &h264->workspace_paddr, GFP_KERNEL);
> + if (!h264->workspace_vaddr)
> + return -ENOMEM;
> +
> + /* Allocate some memory for the H.264 SEI dump */
> + h264->sei_vaddr = dma_alloc_coherent(core->dev, SIZE_SEI,
> + &h264->sei_paddr, GFP_KERNEL);
> + if (!h264->sei_vaddr)
> + return -ENOMEM;
> +
> + amvdec_write_dos_bits(core, POWER_CTL_VLD, BIT(9) | BIT(6));
> +
> + workspace_offset = h264->workspace_paddr - WORKSPACE_BUF_OFFSET;
> + amvdec_write_dos(core, AV_SCRATCH_1, workspace_offset);
> + amvdec_write_dos(core, AV_SCRATCH_G, h264->ext_fw_paddr);
> + amvdec_write_dos(core, AV_SCRATCH_I, h264->sei_paddr -
> + workspace_offset);
> +
> + /* Enable "error correction" */
> + amvdec_write_dos(core, AV_SCRATCH_F,
> + (amvdec_read_dos(core, AV_SCRATCH_F) & 0xffffffc3) |
> + BIT(4) | BIT(7));
> +
> + amvdec_write_dos(core, MDEC_PIC_DC_THRESH, 0x404038aa);
> +
> + return 0;
> +}
> +
> +static int codec_h264_stop(struct amvdec_session *sess)
> +{
> + struct codec_h264 *h264 = sess->priv;
> + struct amvdec_core *core = sess->core;
> +
> + if (h264->ext_fw_vaddr)
> + dma_free_coherent(core->dev, SIZE_EXT_FW,
> + h264->ext_fw_vaddr, h264->ext_fw_paddr);
> +
> + if (h264->workspace_vaddr)
> + dma_free_coherent(core->dev, SIZE_WORKSPACE,
> + h264->workspace_vaddr, h264->workspace_paddr);
> +
> + if (h264->ref_vaddr)
> + dma_free_coherent(core->dev, h264->ref_size,
> + h264->ref_vaddr, h264->ref_paddr);
> +
> + if (h264->sei_vaddr)
> + dma_free_coherent(core->dev, SIZE_SEI,
> + h264->sei_vaddr, h264->sei_paddr);
> +
> + return 0;
> +}
> +
> +static int codec_h264_load_extended_firmware(struct amvdec_session *sess,
> + const u8 *data, u32 len)
> +{
> + struct codec_h264 *h264;
> + struct amvdec_core *core = sess->core;
> +
> + if (len < SIZE_EXT_FW)
> + return -EINVAL;
> +
> + h264 = kzalloc(sizeof(*h264), GFP_KERNEL);
> + if (!h264)
> + return -ENOMEM;
> +
> + h264->ext_fw_vaddr = dma_alloc_coherent(core->dev, SIZE_EXT_FW,
> + &h264->ext_fw_paddr, GFP_KERNEL);
> + if (!h264->ext_fw_vaddr) {
> + kfree(h264);
> + return -ENOMEM;
> + }
> +
> + memcpy(h264->ext_fw_vaddr, data, SIZE_EXT_FW);
> + sess->priv = h264;
> +
> + return 0;
> +}
> +
> +static const struct v4l2_fract par_table[] = {
> + { 1, 1 }, { 1, 1 }, { 12, 11 }, { 10, 11 },
> + { 16, 11 }, { 40, 33 }, { 24, 11 }, { 20, 11 },
> + { 32, 11 }, { 80, 33 }, { 18, 11 }, { 15, 11 },
> + { 64, 33 }, { 160, 99 }, { 4, 3 }, { 3, 2 },
> + { 2, 1 }
> +};
> +
> +static void codec_h264_set_par(struct amvdec_session *sess)
> +{
> + struct amvdec_core *core = sess->core;
> + u32 seq_info = amvdec_read_dos(core, AV_SCRATCH_2);
> + u32 ar_idc = (seq_info >> AR_IDC_BIT) & AR_IDC_MASK;
> +
> + if (!(seq_info & AR_PRESENT_FLAG))
> + return;
> +
> + if (ar_idc == AR_EXTEND) {
> + u32 ar_info = amvdec_read_dos(core, AV_SCRATCH_3);
> +
> + sess->pixelaspect.numerator = ar_info & 0xffff;
> + sess->pixelaspect.denominator = (ar_info >> 16) & 0xffff;
> + return;
> + }
> +
> + if (ar_idc >= ARRAY_SIZE(par_table))
> + return;
> +
> + sess->pixelaspect = par_table[ar_idc];
> +}
> +
> +static void codec_h264_resume(struct amvdec_session *sess)
> +{
> + struct amvdec_core *core = sess->core;
> + struct codec_h264 *h264 = sess->priv;
> + u32 mb_width, mb_height, mb_total;
> +
> + amvdec_set_canvases(sess, (u32[]){ ANC0_CANVAS_ADDR, 0 },
> + (u32[]){ 24, 0 });
> +
> + dev_dbg(core->dev, "max_refs = %u; actual_dpb_size = %u\n",
> + h264->max_refs, sess->num_dst_bufs);
> +
> + /* Align to a multiple of 4 macroblocks */
> + mb_width = ALIGN(h264->mb_width, 4);
> + mb_height = ALIGN(h264->mb_height, 4);
> + mb_total = mb_width * mb_height;
> +
> + h264->ref_size = mb_total * MB_MV_SIZE * h264->max_refs;
> + h264->ref_vaddr = dma_alloc_coherent(core->dev, h264->ref_size,
> + &h264->ref_paddr, GFP_KERNEL);
> + if (!h264->ref_vaddr) {
> + amvdec_abort(sess);
> + return;
> + }
> +
> + /* Address to store the references' MVs */
> + amvdec_write_dos(core, AV_SCRATCH_1, h264->ref_paddr);
> + /* End of ref MV */
> + amvdec_write_dos(core, AV_SCRATCH_4, h264->ref_paddr + h264->ref_size);
> +
> + amvdec_write_dos(core, AV_SCRATCH_0, (h264->max_refs << 24) |
> + (sess->num_dst_bufs << 16) |
> + ((h264->max_refs - 1) << 8));
> +}
> +
> +/**
> + * Configure the H.264 decoder when the parser detected a parameter set change
> + */
> +static void codec_h264_src_change(struct amvdec_session *sess)
> +{
> + struct amvdec_core *core = sess->core;
> + struct codec_h264 *h264 = sess->priv;
> + u32 parsed_info, mb_total;
> + u32 crop_infor, crop_bottom, crop_right;
> + u32 frame_width, frame_height;
> +
> + sess->keyframe_found = 1;
> +
> + parsed_info = amvdec_read_dos(core, AV_SCRATCH_1);
> +
> + /* Total number of 16x16 macroblocks */
> + mb_total = (parsed_info >> MB_TOTAL_BIT) & MB_TOTAL_MASK;
> + /* Number of macroblocks per line */
> + h264->mb_width = parsed_info & MB_WIDTH_MASK;
> + /* Number of macroblock lines */
> + h264->mb_height = mb_total / h264->mb_width;
> +
> + h264->max_refs = ((parsed_info >> MAX_REF_BIT) & MAX_REF_MASK) + 1;
> +
> + crop_infor = amvdec_read_dos(core, AV_SCRATCH_6);
> + crop_bottom = (crop_infor & 0xff);
> + crop_right = (crop_infor >> 16) & 0xff;
> +
> + frame_width = h264->mb_width * 16 - crop_right;
> + frame_height = h264->mb_height * 16 - crop_bottom;
> +
> + dev_dbg(core->dev, "frame: %ux%u; crop: %u %u\n",
> + frame_width, frame_height, crop_right, crop_bottom);
> +
> + codec_h264_set_par(sess);
> + amvdec_src_change(sess, frame_width, frame_height, h264->max_refs + 5);
> +}
> +
> +/**
> + * The bitstream offset is split in half in 2 different registers.
> + * Fetch its MSB here, which location depends on the frame number.
> + */
> +static u32 get_offset_msb(struct amvdec_core *core, int frame_num)
> +{
> + int take_msb = frame_num % 2;
> + int reg_offset = (frame_num / 2) * 4;
> + u32 offset_msb = amvdec_read_dos(core, AV_SCRATCH_A + reg_offset);
> +
> + if (take_msb)
> + return offset_msb & 0xffff0000;
> +
> + return (offset_msb & 0x0000ffff) << 16;
> +}
> +
> +static void codec_h264_frames_ready(struct amvdec_session *sess, u32 status)
> +{
> + struct amvdec_core *core = sess->core;
> + int error_count;
> + int num_frames;
> + int i;
> +
> + error_count = amvdec_read_dos(core, AV_SCRATCH_D);
> + num_frames = (status >> 8) & 0xff;
> + if (error_count) {
> + dev_warn(core->dev,
> + "decoder error(s) happened, count %d\n", error_count);
> + amvdec_write_dos(core, AV_SCRATCH_D, 0);
> + }
> +
> + for (i = 0; i < num_frames; i++) {
> + u32 frame_status = amvdec_read_dos(core, AV_SCRATCH_1 + i * 4);
> + u32 buffer_index = frame_status & BUF_IDX_MASK;
> + u32 pic_struct = (frame_status >> PIC_STRUCT_BIT) &
> + PIC_STRUCT_MASK;
> + u32 offset = (frame_status >> OFFSET_BIT) & OFFSET_MASK;
> + u32 field = V4L2_FIELD_NONE;
> +
> + /*
> + * A buffer decode error means it was decoded,
> + * but part of the picture will have artifacts.
> + * Typical reason is a temporarily corrupted bitstream
> + */
> + if (frame_status & ERROR_FLAG)
> + dev_dbg(core->dev, "Buffer %d decode error\n",
> + buffer_index);
> +
> + if (pic_struct == PIC_TOP_BOT)
> + field = V4L2_FIELD_INTERLACED_TB;
> + else if (pic_struct == PIC_BOT_TOP)
> + field = V4L2_FIELD_INTERLACED_BT;
> +
> + offset |= get_offset_msb(core, i);
> + amvdec_dst_buf_done_idx(sess, buffer_index, offset, field);
> + }
> +}
> +
> +static irqreturn_t codec_h264_threaded_isr(struct amvdec_session *sess)
> +{
> + struct amvdec_core *core = sess->core;
> + u32 status;
> + u32 size;
> + u8 cmd;
> +
> + status = amvdec_read_dos(core, AV_SCRATCH_0);
> + cmd = status & CMD_MASK;
> +
> + switch (cmd) {
> + case CMD_SRC_CHANGE:
> + codec_h264_src_change(sess);
> + break;
> + case CMD_FRAMES_READY:
> + codec_h264_frames_ready(sess, status);
> + break;
> + case CMD_FATAL_ERROR:
> + dev_err(core->dev, "H.264 decoder fatal error\n");
> + goto abort;
> + case CMD_BAD_WIDTH:
> + size = (amvdec_read_dos(core, AV_SCRATCH_1) + 1) * 16;
> + dev_err(core->dev, "Unsupported video width: %u\n", size);
> + goto abort;
> + case CMD_BAD_HEIGHT:
> + size = (amvdec_read_dos(core, AV_SCRATCH_1) + 1) * 16;
> + dev_err(core->dev, "Unsupported video height: %u\n", size);
> + goto abort;
> + case 0: /* Unused but not worth printing for */
> + case 9:
> + break;
> + default:
> + dev_info(core->dev, "Unexpected H264 ISR: %08X\n", cmd);
> + break;
> + }
> +
> + if (cmd && cmd != CMD_SRC_CHANGE)
> + amvdec_write_dos(core, AV_SCRATCH_0, 0);
> +
> + /* Decoder has some SEI data for us ; ignore */
> + if (amvdec_read_dos(core, AV_SCRATCH_J) & SEI_DATA_READY)
> + amvdec_write_dos(core, AV_SCRATCH_J, 0);
> +
> + return IRQ_HANDLED;
> +abort:
> + amvdec_abort(sess);
> + return IRQ_HANDLED;
> +}
> +
> +static irqreturn_t codec_h264_isr(struct amvdec_session *sess)
> +{
> + struct amvdec_core *core = sess->core;
> +
> + amvdec_write_dos(core, ASSIST_MBOX1_CLR_REG, 1);
> +
> + return IRQ_WAKE_THREAD;
> +}
> +
> +struct amvdec_codec_ops codec_h264_ops = {
> + .start = codec_h264_start,
> + .stop = codec_h264_stop,
> + .load_extended_firmware = codec_h264_load_extended_firmware,
> + .isr = codec_h264_isr,
> + .threaded_isr = codec_h264_threaded_isr,
> + .can_recycle = codec_h264_can_recycle,
> + .recycle = codec_h264_recycle,
> + .eos_sequence = codec_h264_eos_sequence,
> + .resume = codec_h264_resume,
> +};
> diff --git a/drivers/staging/media/meson/vdec/codec_h264.h b/drivers/staging/media/meson/vdec/codec_h264.h
> new file mode 100644
> index 000000000000..7cb4fb86ff36
> --- /dev/null
> +++ b/drivers/staging/media/meson/vdec/codec_h264.h
> @@ -0,0 +1,14 @@
> +/* SPDX-License-Identifier: GPL-2.0+ */
> +/*
> + * Copyright (C) 2019 BayLibre, SAS
> + * Author: Maxime Jourdan <[email protected]>
> + */
> +
> +#ifndef __MESON_VDEC_CODEC_H264_H_
> +#define __MESON_VDEC_CODEC_H264_H_
> +
> +#include "vdec.h"
> +
> +extern struct amvdec_codec_ops codec_h264_ops;
> +
> +#endif
> diff --git a/drivers/staging/media/meson/vdec/vdec_platform.c b/drivers/staging/media/meson/vdec/vdec_platform.c
> index 066d4a055894..e9356a46828f 100644
> --- a/drivers/staging/media/meson/vdec/vdec_platform.c
> +++ b/drivers/staging/media/meson/vdec/vdec_platform.c
> @@ -9,9 +9,22 @@
>
> #include "vdec_1.h"
> #include "codec_mpeg12.h"
> +#include "codec_h264.h"
>
> static const struct amvdec_format vdec_formats_gxbb[] = {
> {
> + .pixfmt = V4L2_PIX_FMT_H264,
> + .min_buffers = 2,
> + .max_buffers = 24,
> + .max_width = 1920,
> + .max_height = 1080,
> + .vdec_ops = &vdec_1_ops,
> + .codec_ops = &codec_h264_ops,
> + .firmware_path = "meson/vdec/gxbb_h264.bin",
> + .pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
> + .flags = V4L2_FMT_FLAG_COMPRESSED |
> + V4L2_FMT_FLAG_DYN_RESOLUTION,
> + }, {
> .pixfmt = V4L2_PIX_FMT_MPEG1,
> .min_buffers = 8,
> .max_buffers = 8,
> @@ -38,6 +51,18 @@ static const struct amvdec_format vdec_formats_gxbb[] = {
>
> static const struct amvdec_format vdec_formats_gxl[] = {
> {
> + .pixfmt = V4L2_PIX_FMT_H264,
> + .min_buffers = 2,
> + .max_buffers = 24,
> + .max_width = 3840,
> + .max_height = 2160,
> + .vdec_ops = &vdec_1_ops,
> + .codec_ops = &codec_h264_ops,
> + .firmware_path = "meson/vdec/gxl_h264.bin",
> + .pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
> + .flags = V4L2_FMT_FLAG_COMPRESSED |
> + V4L2_FMT_FLAG_DYN_RESOLUTION,
> + }, {
> .pixfmt = V4L2_PIX_FMT_MPEG1,
> .min_buffers = 8,
> .max_buffers = 8,
> @@ -64,6 +89,18 @@ static const struct amvdec_format vdec_formats_gxl[] = {
>
> static const struct amvdec_format vdec_formats_gxm[] = {
> {
> + .pixfmt = V4L2_PIX_FMT_H264,
> + .min_buffers = 2,
> + .max_buffers = 24,
> + .max_width = 3840,
> + .max_height = 2160,
> + .vdec_ops = &vdec_1_ops,
> + .codec_ops = &codec_h264_ops,
> + .firmware_path = "meson/vdec/gxm_h264.bin",
> + .pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
> + .flags = V4L2_FMT_FLAG_COMPRESSED |
> + V4L2_FMT_FLAG_DYN_RESOLUTION,
> + }, {
> .pixfmt = V4L2_PIX_FMT_MPEG1,
> .min_buffers = 8,
> .max_buffers = 8,
> @@ -90,6 +127,18 @@ static const struct amvdec_format vdec_formats_gxm[] = {
>
> static const struct amvdec_format vdec_formats_g12a[] = {
> {
> + .pixfmt = V4L2_PIX_FMT_H264,
> + .min_buffers = 2,
> + .max_buffers = 24,
> + .max_width = 3840,
> + .max_height = 2160,
> + .vdec_ops = &vdec_1_ops,
> + .codec_ops = &codec_h264_ops,
> + .firmware_path = "meson/vdec/g12a_h264.bin",
> + .pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
> + .flags = V4L2_FMT_FLAG_COMPRESSED |
> + V4L2_FMT_FLAG_DYN_RESOLUTION,
> + }, {
> .pixfmt = V4L2_PIX_FMT_MPEG1,
> .min_buffers = 8,
> .max_buffers = 8,
> @@ -116,6 +165,18 @@ static const struct amvdec_format vdec_formats_g12a[] = {
>
> static const struct amvdec_format vdec_formats_sm1[] = {
> {
> + .pixfmt = V4L2_PIX_FMT_H264,
> + .min_buffers = 2,
> + .max_buffers = 24,
> + .max_width = 3840,
> + .max_height = 2160,
> + .vdec_ops = &vdec_1_ops,
> + .codec_ops = &codec_h264_ops,
> + .firmware_path = "meson/vdec/g12a_h264.bin",
> + .pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
> + .flags = V4L2_FMT_FLAG_COMPRESSED |
> + V4L2_FMT_FLAG_DYN_RESOLUTION,
> + }, {
> .pixfmt = V4L2_PIX_FMT_MPEG1,
> .min_buffers = 8,
> .max_buffers = 8,
>
On 14/02/2020 16:19, Hans Verkuil wrote:
> On 2/6/20 9:26 AM, Neil Armstrong wrote:
>> From: Maxime Jourdan <[email protected]>
>>
>> Add support for the H264 compressed format (V4L2_PIX_FMT_H264).
>>
>> Signed-off-by: Maxime Jourdan <[email protected]>
>> Signed-off-by: Neil Armstrong <[email protected]>
>> ---
>> drivers/staging/media/meson/vdec/Makefile | 2 +-
>> drivers/staging/media/meson/vdec/codec_h264.c | 482 ++++++++++++++++++
>> drivers/staging/media/meson/vdec/codec_h264.h | 14 +
>> .../staging/media/meson/vdec/vdec_platform.c | 61 +++
>> 4 files changed, 558 insertions(+), 1 deletion(-)
>> create mode 100644 drivers/staging/media/meson/vdec/codec_h264.c
>> create mode 100644 drivers/staging/media/meson/vdec/codec_h264.h
>
> I get warnings when compiling this:
>
> drivers/staging/media/meson/vdec/codec_h264.c:322: warning: Function parameter or member 'sess' not described in 'codec_h264_src_change'
> drivers/staging/media/meson/vdec/codec_h264.c:361: warning: Function parameter or member 'core' not described in 'get_offset_msb'
> drivers/staging/media/meson/vdec/codec_h264.c:361: warning: Function parameter or member 'frame_num' not described in 'get_offset_msb'
I'll fix these.
Neil
>
> Regards,
>
> Hans
>
>>
>> diff --git a/drivers/staging/media/meson/vdec/Makefile b/drivers/staging/media/meson/vdec/Makefile
>> index 6bea129084b7..711d990c760e 100644
>> --- a/drivers/staging/media/meson/vdec/Makefile
>> +++ b/drivers/staging/media/meson/vdec/Makefile
>> @@ -3,6 +3,6 @@
>>
>> meson-vdec-objs = esparser.o vdec.o vdec_helpers.o vdec_platform.o
>> meson-vdec-objs += vdec_1.o
>> -meson-vdec-objs += codec_mpeg12.o
>> +meson-vdec-objs += codec_mpeg12.o codec_h264.o
>>
>> obj-$(CONFIG_VIDEO_MESON_VDEC) += meson-vdec.o
>> diff --git a/drivers/staging/media/meson/vdec/codec_h264.c b/drivers/staging/media/meson/vdec/codec_h264.c
>> new file mode 100644
>> index 000000000000..4528a6a01c3d
>> --- /dev/null
>> +++ b/drivers/staging/media/meson/vdec/codec_h264.c
>> @@ -0,0 +1,482 @@
>> +// SPDX-License-Identifier: GPL-2.0+
>> +/*
>> + * Copyright (C) 2019 BayLibre, SAS
>> + * Author: Maxime Jourdan <[email protected]>
>> + */
>> +
>> +#include <media/v4l2-mem2mem.h>
>> +#include <media/videobuf2-dma-contig.h>
>> +
>> +#include "vdec_helpers.h"
>> +#include "dos_regs.h"
>> +
>> +#define SIZE_EXT_FW (20 * SZ_1K)
>> +#define SIZE_WORKSPACE 0x1ee000
>> +#define SIZE_SEI (8 * SZ_1K)
>> +
>> +/*
>> + * Offset added by the firmware which must be substracted
>> + * from the workspace phyaddr
>> + */
>> +#define WORKSPACE_BUF_OFFSET 0x1000000
>> +
>> +/* ISR status */
>> +#define CMD_MASK GENMASK(7, 0)
>> +#define CMD_SRC_CHANGE 1
>> +#define CMD_FRAMES_READY 2
>> +#define CMD_FATAL_ERROR 6
>> +#define CMD_BAD_WIDTH 7
>> +#define CMD_BAD_HEIGHT 8
>> +
>> +#define SEI_DATA_READY BIT(15)
>> +
>> +/* Picture type */
>> +#define PIC_TOP_BOT 5
>> +#define PIC_BOT_TOP 6
>> +
>> +/* Size of Motion Vector per macroblock */
>> +#define MB_MV_SIZE 96
>> +
>> +/* Frame status data */
>> +#define PIC_STRUCT_BIT 5
>> +#define PIC_STRUCT_MASK GENMASK(2, 0)
>> +#define BUF_IDX_MASK GENMASK(4, 0)
>> +#define ERROR_FLAG BIT(9)
>> +#define OFFSET_BIT 16
>> +#define OFFSET_MASK GENMASK(15, 0)
>> +
>> +/* Bitstream parsed data */
>> +#define MB_TOTAL_BIT 8
>> +#define MB_TOTAL_MASK GENMASK(15, 0)
>> +#define MB_WIDTH_MASK GENMASK(7, 0)
>> +#define MAX_REF_BIT 24
>> +#define MAX_REF_MASK GENMASK(6, 0)
>> +#define AR_IDC_BIT 16
>> +#define AR_IDC_MASK GENMASK(7, 0)
>> +#define AR_PRESENT_FLAG BIT(0)
>> +#define AR_EXTEND 0xff
>> +
>> +/*
>> + * Buffer to send to the ESPARSER to signal End Of Stream for H.264.
>> + * This is a 16x16 encoded picture that will trigger drain firmware-side.
>> + * There is no known alternative.
>> + */
>> +static const u8 eos_sequence[SZ_4K] = {
>> + 0x00, 0x00, 0x00, 0x01, 0x06, 0x05, 0xff, 0xe4, 0xdc, 0x45, 0xe9, 0xbd,
>> + 0xe6, 0xd9, 0x48, 0xb7, 0x96, 0x2c, 0xd8, 0x20, 0xd9, 0x23, 0xee, 0xef,
>> + 0x78, 0x32, 0x36, 0x34, 0x20, 0x2d, 0x20, 0x63, 0x6f, 0x72, 0x65, 0x20,
>> + 0x36, 0x37, 0x20, 0x72, 0x31, 0x31, 0x33, 0x30, 0x20, 0x38, 0x34, 0x37,
>> + 0x35, 0x39, 0x37, 0x37, 0x20, 0x2d, 0x20, 0x48, 0x2e, 0x32, 0x36, 0x34,
>> + 0x2f, 0x4d, 0x50, 0x45, 0x47, 0x2d, 0x34, 0x20, 0x41, 0x56, 0x43, 0x20,
>> + 0x63, 0x6f, 0x64, 0x65, 0x63, 0x20, 0x2d, 0x20, 0x43, 0x6f, 0x70, 0x79,
>> + 0x6c, 0x65, 0x66, 0x74, 0x20, 0x32, 0x30, 0x30, 0x33, 0x2d, 0x32, 0x30,
>> + 0x30, 0x39, 0x20, 0x2d, 0x20, 0x68, 0x74, 0x74, 0x70, 0x3a, 0x2f, 0x2f,
>> + 0x77, 0x77, 0x77, 0x2e, 0x76, 0x69, 0x64, 0x65, 0x6f, 0x6c, 0x61, 0x6e,
>> + 0x2e, 0x6f, 0x72, 0x67, 0x2f, 0x78, 0x32, 0x36, 0x34, 0x2e, 0x68, 0x74,
>> + 0x6d, 0x6c, 0x20, 0x2d, 0x20, 0x6f, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x73,
>> + 0x3a, 0x20, 0x63, 0x61, 0x62, 0x61, 0x63, 0x3d, 0x31, 0x20, 0x72, 0x65,
>> + 0x66, 0x3d, 0x31, 0x20, 0x64, 0x65, 0x62, 0x6c, 0x6f, 0x63, 0x6b, 0x3d,
>> + 0x31, 0x3a, 0x30, 0x3a, 0x30, 0x20, 0x61, 0x6e, 0x61, 0x6c, 0x79, 0x73,
>> + 0x65, 0x3d, 0x30, 0x78, 0x31, 0x3a, 0x30, 0x78, 0x31, 0x31, 0x31, 0x20,
>> + 0x6d, 0x65, 0x3d, 0x68, 0x65, 0x78, 0x20, 0x73, 0x75, 0x62, 0x6d, 0x65,
>> + 0x3d, 0x36, 0x20, 0x70, 0x73, 0x79, 0x5f, 0x72, 0x64, 0x3d, 0x31, 0x2e,
>> + 0x30, 0x3a, 0x30, 0x2e, 0x30, 0x20, 0x6d, 0x69, 0x78, 0x65, 0x64, 0x5f,
>> + 0x72, 0x65, 0x66, 0x3d, 0x30, 0x20, 0x6d, 0x65, 0x5f, 0x72, 0x61, 0x6e,
>> + 0x67, 0x65, 0x3d, 0x31, 0x36, 0x20, 0x63, 0x68, 0x72, 0x6f, 0x6d, 0x61,
>> + 0x5f, 0x6d, 0x65, 0x3d, 0x31, 0x20, 0x74, 0x72, 0x65, 0x6c, 0x6c, 0x69,
>> + 0x73, 0x3d, 0x30, 0x20, 0x38, 0x78, 0x38, 0x64, 0x63, 0x74, 0x3d, 0x30,
>> + 0x20, 0x63, 0x71, 0x6d, 0x3d, 0x30, 0x20, 0x64, 0x65, 0x61, 0x64, 0x7a,
>> + 0x6f, 0x6e, 0x65, 0x3d, 0x32, 0x31, 0x2c, 0x31, 0x31, 0x20, 0x63, 0x68,
>> + 0x72, 0x6f, 0x6d, 0x61, 0x5f, 0x71, 0x70, 0x5f, 0x6f, 0x66, 0x66, 0x73,
>> + 0x65, 0x74, 0x3d, 0x2d, 0x32, 0x20, 0x74, 0x68, 0x72, 0x65, 0x61, 0x64,
>> + 0x73, 0x3d, 0x31, 0x20, 0x6e, 0x72, 0x3d, 0x30, 0x20, 0x64, 0x65, 0x63,
>> + 0x69, 0x6d, 0x61, 0x74, 0x65, 0x3d, 0x31, 0x20, 0x6d, 0x62, 0x61, 0x66,
>> + 0x66, 0x3d, 0x30, 0x20, 0x62, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x73, 0x3d,
>> + 0x30, 0x20, 0x6b, 0x65, 0x79, 0x69, 0x6e, 0x74, 0x3d, 0x32, 0x35, 0x30,
>> + 0x20, 0x6b, 0x65, 0x79, 0x69, 0x6e, 0x74, 0x5f, 0x6d, 0x69, 0x6e, 0x3d,
>> + 0x32, 0x35, 0x20, 0x73, 0x63, 0x65, 0x6e, 0x65, 0x63, 0x75, 0x74, 0x3d,
>> + 0x34, 0x30, 0x20, 0x72, 0x63, 0x3d, 0x61, 0x62, 0x72, 0x20, 0x62, 0x69,
>> + 0x74, 0x72, 0x61, 0x74, 0x65, 0x3d, 0x31, 0x30, 0x20, 0x72, 0x61, 0x74,
>> + 0x65, 0x74, 0x6f, 0x6c, 0x3d, 0x31, 0x2e, 0x30, 0x20, 0x71, 0x63, 0x6f,
>> + 0x6d, 0x70, 0x3d, 0x30, 0x2e, 0x36, 0x30, 0x20, 0x71, 0x70, 0x6d, 0x69,
>> + 0x6e, 0x3d, 0x31, 0x30, 0x20, 0x71, 0x70, 0x6d, 0x61, 0x78, 0x3d, 0x35,
>> + 0x31, 0x20, 0x71, 0x70, 0x73, 0x74, 0x65, 0x70, 0x3d, 0x34, 0x20, 0x69,
>> + 0x70, 0x5f, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x3d, 0x31, 0x2e, 0x34, 0x30,
>> + 0x20, 0x61, 0x71, 0x3d, 0x31, 0x3a, 0x31, 0x2e, 0x30, 0x30, 0x00, 0x80,
>> + 0x00, 0x00, 0x00, 0x01, 0x67, 0x4d, 0x40, 0x0a, 0x9a, 0x74, 0xf4, 0x20,
>> + 0x00, 0x00, 0x03, 0x00, 0x20, 0x00, 0x00, 0x06, 0x51, 0xe2, 0x44, 0xd4,
>> + 0x00, 0x00, 0x00, 0x01, 0x68, 0xee, 0x32, 0xc8, 0x00, 0x00, 0x00, 0x01,
>> + 0x65, 0x88, 0x80, 0x20, 0x00, 0x08, 0x7f, 0xea, 0x6a, 0xe2, 0x99, 0xb6,
>> + 0x57, 0xae, 0x49, 0x30, 0xf5, 0xfe, 0x5e, 0x46, 0x0b, 0x72, 0x44, 0xc4,
>> + 0xe1, 0xfc, 0x62, 0xda, 0xf1, 0xfb, 0xa2, 0xdb, 0xd6, 0xbe, 0x5c, 0xd7,
>> + 0x24, 0xa3, 0xf5, 0xb9, 0x2f, 0x57, 0x16, 0x49, 0x75, 0x47, 0x77, 0x09,
>> + 0x5c, 0xa1, 0xb4, 0xc3, 0x4f, 0x60, 0x2b, 0xb0, 0x0c, 0xc8, 0xd6, 0x66,
>> + 0xba, 0x9b, 0x82, 0x29, 0x33, 0x92, 0x26, 0x99, 0x31, 0x1c, 0x7f, 0x9b,
>> + 0x00, 0x00, 0x01, 0x0ff,
>> +};
>> +
>> +static const u8 *codec_h264_eos_sequence(u32 *len)
>> +{
>> + *len = ARRAY_SIZE(eos_sequence);
>> + return eos_sequence;
>> +}
>> +
>> +struct codec_h264 {
>> + /* H.264 decoder requires an extended firmware */
>> + void *ext_fw_vaddr;
>> + dma_addr_t ext_fw_paddr;
>> +
>> + /* Buffer for the H.264 Workspace */
>> + void *workspace_vaddr;
>> + dma_addr_t workspace_paddr;
>> +
>> + /* Buffer for the H.264 references MV */
>> + void *ref_vaddr;
>> + dma_addr_t ref_paddr;
>> + u32 ref_size;
>> +
>> + /* Buffer for parsed SEI data */
>> + void *sei_vaddr;
>> + dma_addr_t sei_paddr;
>> +
>> + u32 mb_width;
>> + u32 mb_height;
>> + u32 max_refs;
>> +};
>> +
>> +static int codec_h264_can_recycle(struct amvdec_core *core)
>> +{
>> + return !amvdec_read_dos(core, AV_SCRATCH_7) ||
>> + !amvdec_read_dos(core, AV_SCRATCH_8);
>> +}
>> +
>> +static void codec_h264_recycle(struct amvdec_core *core, u32 buf_idx)
>> +{
>> + /*
>> + * Tell the firmware it can recycle this buffer.
>> + * AV_SCRATCH_8 serves the same purpose.
>> + */
>> + if (!amvdec_read_dos(core, AV_SCRATCH_7))
>> + amvdec_write_dos(core, AV_SCRATCH_7, buf_idx + 1);
>> + else
>> + amvdec_write_dos(core, AV_SCRATCH_8, buf_idx + 1);
>> +}
>> +
>> +static int codec_h264_start(struct amvdec_session *sess)
>> +{
>> + u32 workspace_offset;
>> + struct amvdec_core *core = sess->core;
>> + struct codec_h264 *h264 = sess->priv;
>> +
>> + /* Allocate some memory for the H.264 decoder's state */
>> + h264->workspace_vaddr =
>> + dma_alloc_coherent(core->dev, SIZE_WORKSPACE,
>> + &h264->workspace_paddr, GFP_KERNEL);
>> + if (!h264->workspace_vaddr)
>> + return -ENOMEM;
>> +
>> + /* Allocate some memory for the H.264 SEI dump */
>> + h264->sei_vaddr = dma_alloc_coherent(core->dev, SIZE_SEI,
>> + &h264->sei_paddr, GFP_KERNEL);
>> + if (!h264->sei_vaddr)
>> + return -ENOMEM;
>> +
>> + amvdec_write_dos_bits(core, POWER_CTL_VLD, BIT(9) | BIT(6));
>> +
>> + workspace_offset = h264->workspace_paddr - WORKSPACE_BUF_OFFSET;
>> + amvdec_write_dos(core, AV_SCRATCH_1, workspace_offset);
>> + amvdec_write_dos(core, AV_SCRATCH_G, h264->ext_fw_paddr);
>> + amvdec_write_dos(core, AV_SCRATCH_I, h264->sei_paddr -
>> + workspace_offset);
>> +
>> + /* Enable "error correction" */
>> + amvdec_write_dos(core, AV_SCRATCH_F,
>> + (amvdec_read_dos(core, AV_SCRATCH_F) & 0xffffffc3) |
>> + BIT(4) | BIT(7));
>> +
>> + amvdec_write_dos(core, MDEC_PIC_DC_THRESH, 0x404038aa);
>> +
>> + return 0;
>> +}
>> +
>> +static int codec_h264_stop(struct amvdec_session *sess)
>> +{
>> + struct codec_h264 *h264 = sess->priv;
>> + struct amvdec_core *core = sess->core;
>> +
>> + if (h264->ext_fw_vaddr)
>> + dma_free_coherent(core->dev, SIZE_EXT_FW,
>> + h264->ext_fw_vaddr, h264->ext_fw_paddr);
>> +
>> + if (h264->workspace_vaddr)
>> + dma_free_coherent(core->dev, SIZE_WORKSPACE,
>> + h264->workspace_vaddr, h264->workspace_paddr);
>> +
>> + if (h264->ref_vaddr)
>> + dma_free_coherent(core->dev, h264->ref_size,
>> + h264->ref_vaddr, h264->ref_paddr);
>> +
>> + if (h264->sei_vaddr)
>> + dma_free_coherent(core->dev, SIZE_SEI,
>> + h264->sei_vaddr, h264->sei_paddr);
>> +
>> + return 0;
>> +}
>> +
>> +static int codec_h264_load_extended_firmware(struct amvdec_session *sess,
>> + const u8 *data, u32 len)
>> +{
>> + struct codec_h264 *h264;
>> + struct amvdec_core *core = sess->core;
>> +
>> + if (len < SIZE_EXT_FW)
>> + return -EINVAL;
>> +
>> + h264 = kzalloc(sizeof(*h264), GFP_KERNEL);
>> + if (!h264)
>> + return -ENOMEM;
>> +
>> + h264->ext_fw_vaddr = dma_alloc_coherent(core->dev, SIZE_EXT_FW,
>> + &h264->ext_fw_paddr, GFP_KERNEL);
>> + if (!h264->ext_fw_vaddr) {
>> + kfree(h264);
>> + return -ENOMEM;
>> + }
>> +
>> + memcpy(h264->ext_fw_vaddr, data, SIZE_EXT_FW);
>> + sess->priv = h264;
>> +
>> + return 0;
>> +}
>> +
>> +static const struct v4l2_fract par_table[] = {
>> + { 1, 1 }, { 1, 1 }, { 12, 11 }, { 10, 11 },
>> + { 16, 11 }, { 40, 33 }, { 24, 11 }, { 20, 11 },
>> + { 32, 11 }, { 80, 33 }, { 18, 11 }, { 15, 11 },
>> + { 64, 33 }, { 160, 99 }, { 4, 3 }, { 3, 2 },
>> + { 2, 1 }
>> +};
>> +
>> +static void codec_h264_set_par(struct amvdec_session *sess)
>> +{
>> + struct amvdec_core *core = sess->core;
>> + u32 seq_info = amvdec_read_dos(core, AV_SCRATCH_2);
>> + u32 ar_idc = (seq_info >> AR_IDC_BIT) & AR_IDC_MASK;
>> +
>> + if (!(seq_info & AR_PRESENT_FLAG))
>> + return;
>> +
>> + if (ar_idc == AR_EXTEND) {
>> + u32 ar_info = amvdec_read_dos(core, AV_SCRATCH_3);
>> +
>> + sess->pixelaspect.numerator = ar_info & 0xffff;
>> + sess->pixelaspect.denominator = (ar_info >> 16) & 0xffff;
>> + return;
>> + }
>> +
>> + if (ar_idc >= ARRAY_SIZE(par_table))
>> + return;
>> +
>> + sess->pixelaspect = par_table[ar_idc];
>> +}
>> +
>> +static void codec_h264_resume(struct amvdec_session *sess)
>> +{
>> + struct amvdec_core *core = sess->core;
>> + struct codec_h264 *h264 = sess->priv;
>> + u32 mb_width, mb_height, mb_total;
>> +
>> + amvdec_set_canvases(sess, (u32[]){ ANC0_CANVAS_ADDR, 0 },
>> + (u32[]){ 24, 0 });
>> +
>> + dev_dbg(core->dev, "max_refs = %u; actual_dpb_size = %u\n",
>> + h264->max_refs, sess->num_dst_bufs);
>> +
>> + /* Align to a multiple of 4 macroblocks */
>> + mb_width = ALIGN(h264->mb_width, 4);
>> + mb_height = ALIGN(h264->mb_height, 4);
>> + mb_total = mb_width * mb_height;
>> +
>> + h264->ref_size = mb_total * MB_MV_SIZE * h264->max_refs;
>> + h264->ref_vaddr = dma_alloc_coherent(core->dev, h264->ref_size,
>> + &h264->ref_paddr, GFP_KERNEL);
>> + if (!h264->ref_vaddr) {
>> + amvdec_abort(sess);
>> + return;
>> + }
>> +
>> + /* Address to store the references' MVs */
>> + amvdec_write_dos(core, AV_SCRATCH_1, h264->ref_paddr);
>> + /* End of ref MV */
>> + amvdec_write_dos(core, AV_SCRATCH_4, h264->ref_paddr + h264->ref_size);
>> +
>> + amvdec_write_dos(core, AV_SCRATCH_0, (h264->max_refs << 24) |
>> + (sess->num_dst_bufs << 16) |
>> + ((h264->max_refs - 1) << 8));
>> +}
>> +
>> +/**
>> + * Configure the H.264 decoder when the parser detected a parameter set change
>> + */
>> +static void codec_h264_src_change(struct amvdec_session *sess)
>> +{
>> + struct amvdec_core *core = sess->core;
>> + struct codec_h264 *h264 = sess->priv;
>> + u32 parsed_info, mb_total;
>> + u32 crop_infor, crop_bottom, crop_right;
>> + u32 frame_width, frame_height;
>> +
>> + sess->keyframe_found = 1;
>> +
>> + parsed_info = amvdec_read_dos(core, AV_SCRATCH_1);
>> +
>> + /* Total number of 16x16 macroblocks */
>> + mb_total = (parsed_info >> MB_TOTAL_BIT) & MB_TOTAL_MASK;
>> + /* Number of macroblocks per line */
>> + h264->mb_width = parsed_info & MB_WIDTH_MASK;
>> + /* Number of macroblock lines */
>> + h264->mb_height = mb_total / h264->mb_width;
>> +
>> + h264->max_refs = ((parsed_info >> MAX_REF_BIT) & MAX_REF_MASK) + 1;
>> +
>> + crop_infor = amvdec_read_dos(core, AV_SCRATCH_6);
>> + crop_bottom = (crop_infor & 0xff);
>> + crop_right = (crop_infor >> 16) & 0xff;
>> +
>> + frame_width = h264->mb_width * 16 - crop_right;
>> + frame_height = h264->mb_height * 16 - crop_bottom;
>> +
>> + dev_dbg(core->dev, "frame: %ux%u; crop: %u %u\n",
>> + frame_width, frame_height, crop_right, crop_bottom);
>> +
>> + codec_h264_set_par(sess);
>> + amvdec_src_change(sess, frame_width, frame_height, h264->max_refs + 5);
>> +}
>> +
>> +/**
>> + * The bitstream offset is split in half in 2 different registers.
>> + * Fetch its MSB here, which location depends on the frame number.
>> + */
>> +static u32 get_offset_msb(struct amvdec_core *core, int frame_num)
>> +{
>> + int take_msb = frame_num % 2;
>> + int reg_offset = (frame_num / 2) * 4;
>> + u32 offset_msb = amvdec_read_dos(core, AV_SCRATCH_A + reg_offset);
>> +
>> + if (take_msb)
>> + return offset_msb & 0xffff0000;
>> +
>> + return (offset_msb & 0x0000ffff) << 16;
>> +}
>> +
>> +static void codec_h264_frames_ready(struct amvdec_session *sess, u32 status)
>> +{
>> + struct amvdec_core *core = sess->core;
>> + int error_count;
>> + int num_frames;
>> + int i;
>> +
>> + error_count = amvdec_read_dos(core, AV_SCRATCH_D);
>> + num_frames = (status >> 8) & 0xff;
>> + if (error_count) {
>> + dev_warn(core->dev,
>> + "decoder error(s) happened, count %d\n", error_count);
>> + amvdec_write_dos(core, AV_SCRATCH_D, 0);
>> + }
>> +
>> + for (i = 0; i < num_frames; i++) {
>> + u32 frame_status = amvdec_read_dos(core, AV_SCRATCH_1 + i * 4);
>> + u32 buffer_index = frame_status & BUF_IDX_MASK;
>> + u32 pic_struct = (frame_status >> PIC_STRUCT_BIT) &
>> + PIC_STRUCT_MASK;
>> + u32 offset = (frame_status >> OFFSET_BIT) & OFFSET_MASK;
>> + u32 field = V4L2_FIELD_NONE;
>> +
>> + /*
>> + * A buffer decode error means it was decoded,
>> + * but part of the picture will have artifacts.
>> + * Typical reason is a temporarily corrupted bitstream
>> + */
>> + if (frame_status & ERROR_FLAG)
>> + dev_dbg(core->dev, "Buffer %d decode error\n",
>> + buffer_index);
>> +
>> + if (pic_struct == PIC_TOP_BOT)
>> + field = V4L2_FIELD_INTERLACED_TB;
>> + else if (pic_struct == PIC_BOT_TOP)
>> + field = V4L2_FIELD_INTERLACED_BT;
>> +
>> + offset |= get_offset_msb(core, i);
>> + amvdec_dst_buf_done_idx(sess, buffer_index, offset, field);
>> + }
>> +}
>> +
>> +static irqreturn_t codec_h264_threaded_isr(struct amvdec_session *sess)
>> +{
>> + struct amvdec_core *core = sess->core;
>> + u32 status;
>> + u32 size;
>> + u8 cmd;
>> +
>> + status = amvdec_read_dos(core, AV_SCRATCH_0);
>> + cmd = status & CMD_MASK;
>> +
>> + switch (cmd) {
>> + case CMD_SRC_CHANGE:
>> + codec_h264_src_change(sess);
>> + break;
>> + case CMD_FRAMES_READY:
>> + codec_h264_frames_ready(sess, status);
>> + break;
>> + case CMD_FATAL_ERROR:
>> + dev_err(core->dev, "H.264 decoder fatal error\n");
>> + goto abort;
>> + case CMD_BAD_WIDTH:
>> + size = (amvdec_read_dos(core, AV_SCRATCH_1) + 1) * 16;
>> + dev_err(core->dev, "Unsupported video width: %u\n", size);
>> + goto abort;
>> + case CMD_BAD_HEIGHT:
>> + size = (amvdec_read_dos(core, AV_SCRATCH_1) + 1) * 16;
>> + dev_err(core->dev, "Unsupported video height: %u\n", size);
>> + goto abort;
>> + case 0: /* Unused but not worth printing for */
>> + case 9:
>> + break;
>> + default:
>> + dev_info(core->dev, "Unexpected H264 ISR: %08X\n", cmd);
>> + break;
>> + }
>> +
>> + if (cmd && cmd != CMD_SRC_CHANGE)
>> + amvdec_write_dos(core, AV_SCRATCH_0, 0);
>> +
>> + /* Decoder has some SEI data for us ; ignore */
>> + if (amvdec_read_dos(core, AV_SCRATCH_J) & SEI_DATA_READY)
>> + amvdec_write_dos(core, AV_SCRATCH_J, 0);
>> +
>> + return IRQ_HANDLED;
>> +abort:
>> + amvdec_abort(sess);
>> + return IRQ_HANDLED;
>> +}
>> +
>> +static irqreturn_t codec_h264_isr(struct amvdec_session *sess)
>> +{
>> + struct amvdec_core *core = sess->core;
>> +
>> + amvdec_write_dos(core, ASSIST_MBOX1_CLR_REG, 1);
>> +
>> + return IRQ_WAKE_THREAD;
>> +}
>> +
>> +struct amvdec_codec_ops codec_h264_ops = {
>> + .start = codec_h264_start,
>> + .stop = codec_h264_stop,
>> + .load_extended_firmware = codec_h264_load_extended_firmware,
>> + .isr = codec_h264_isr,
>> + .threaded_isr = codec_h264_threaded_isr,
>> + .can_recycle = codec_h264_can_recycle,
>> + .recycle = codec_h264_recycle,
>> + .eos_sequence = codec_h264_eos_sequence,
>> + .resume = codec_h264_resume,
>> +};
>> diff --git a/drivers/staging/media/meson/vdec/codec_h264.h b/drivers/staging/media/meson/vdec/codec_h264.h
>> new file mode 100644
>> index 000000000000..7cb4fb86ff36
>> --- /dev/null
>> +++ b/drivers/staging/media/meson/vdec/codec_h264.h
>> @@ -0,0 +1,14 @@
>> +/* SPDX-License-Identifier: GPL-2.0+ */
>> +/*
>> + * Copyright (C) 2019 BayLibre, SAS
>> + * Author: Maxime Jourdan <[email protected]>
>> + */
>> +
>> +#ifndef __MESON_VDEC_CODEC_H264_H_
>> +#define __MESON_VDEC_CODEC_H264_H_
>> +
>> +#include "vdec.h"
>> +
>> +extern struct amvdec_codec_ops codec_h264_ops;
>> +
>> +#endif
>> diff --git a/drivers/staging/media/meson/vdec/vdec_platform.c b/drivers/staging/media/meson/vdec/vdec_platform.c
>> index 066d4a055894..e9356a46828f 100644
>> --- a/drivers/staging/media/meson/vdec/vdec_platform.c
>> +++ b/drivers/staging/media/meson/vdec/vdec_platform.c
>> @@ -9,9 +9,22 @@
>>
>> #include "vdec_1.h"
>> #include "codec_mpeg12.h"
>> +#include "codec_h264.h"
>>
>> static const struct amvdec_format vdec_formats_gxbb[] = {
>> {
>> + .pixfmt = V4L2_PIX_FMT_H264,
>> + .min_buffers = 2,
>> + .max_buffers = 24,
>> + .max_width = 1920,
>> + .max_height = 1080,
>> + .vdec_ops = &vdec_1_ops,
>> + .codec_ops = &codec_h264_ops,
>> + .firmware_path = "meson/vdec/gxbb_h264.bin",
>> + .pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
>> + .flags = V4L2_FMT_FLAG_COMPRESSED |
>> + V4L2_FMT_FLAG_DYN_RESOLUTION,
>> + }, {
>> .pixfmt = V4L2_PIX_FMT_MPEG1,
>> .min_buffers = 8,
>> .max_buffers = 8,
>> @@ -38,6 +51,18 @@ static const struct amvdec_format vdec_formats_gxbb[] = {
>>
>> static const struct amvdec_format vdec_formats_gxl[] = {
>> {
>> + .pixfmt = V4L2_PIX_FMT_H264,
>> + .min_buffers = 2,
>> + .max_buffers = 24,
>> + .max_width = 3840,
>> + .max_height = 2160,
>> + .vdec_ops = &vdec_1_ops,
>> + .codec_ops = &codec_h264_ops,
>> + .firmware_path = "meson/vdec/gxl_h264.bin",
>> + .pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
>> + .flags = V4L2_FMT_FLAG_COMPRESSED |
>> + V4L2_FMT_FLAG_DYN_RESOLUTION,
>> + }, {
>> .pixfmt = V4L2_PIX_FMT_MPEG1,
>> .min_buffers = 8,
>> .max_buffers = 8,
>> @@ -64,6 +89,18 @@ static const struct amvdec_format vdec_formats_gxl[] = {
>>
>> static const struct amvdec_format vdec_formats_gxm[] = {
>> {
>> + .pixfmt = V4L2_PIX_FMT_H264,
>> + .min_buffers = 2,
>> + .max_buffers = 24,
>> + .max_width = 3840,
>> + .max_height = 2160,
>> + .vdec_ops = &vdec_1_ops,
>> + .codec_ops = &codec_h264_ops,
>> + .firmware_path = "meson/vdec/gxm_h264.bin",
>> + .pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
>> + .flags = V4L2_FMT_FLAG_COMPRESSED |
>> + V4L2_FMT_FLAG_DYN_RESOLUTION,
>> + }, {
>> .pixfmt = V4L2_PIX_FMT_MPEG1,
>> .min_buffers = 8,
>> .max_buffers = 8,
>> @@ -90,6 +127,18 @@ static const struct amvdec_format vdec_formats_gxm[] = {
>>
>> static const struct amvdec_format vdec_formats_g12a[] = {
>> {
>> + .pixfmt = V4L2_PIX_FMT_H264,
>> + .min_buffers = 2,
>> + .max_buffers = 24,
>> + .max_width = 3840,
>> + .max_height = 2160,
>> + .vdec_ops = &vdec_1_ops,
>> + .codec_ops = &codec_h264_ops,
>> + .firmware_path = "meson/vdec/g12a_h264.bin",
>> + .pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
>> + .flags = V4L2_FMT_FLAG_COMPRESSED |
>> + V4L2_FMT_FLAG_DYN_RESOLUTION,
>> + }, {
>> .pixfmt = V4L2_PIX_FMT_MPEG1,
>> .min_buffers = 8,
>> .max_buffers = 8,
>> @@ -116,6 +165,18 @@ static const struct amvdec_format vdec_formats_g12a[] = {
>>
>> static const struct amvdec_format vdec_formats_sm1[] = {
>> {
>> + .pixfmt = V4L2_PIX_FMT_H264,
>> + .min_buffers = 2,
>> + .max_buffers = 24,
>> + .max_width = 3840,
>> + .max_height = 2160,
>> + .vdec_ops = &vdec_1_ops,
>> + .codec_ops = &codec_h264_ops,
>> + .firmware_path = "meson/vdec/g12a_h264.bin",
>> + .pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
>> + .flags = V4L2_FMT_FLAG_COMPRESSED |
>> + V4L2_FMT_FLAG_DYN_RESOLUTION,
>> + }, {
>> .pixfmt = V4L2_PIX_FMT_MPEG1,
>> .min_buffers = 8,
>> .max_buffers = 8,
>>
>
On 2/6/20 9:26 AM, Neil Armstrong wrote:
> Since the draining and stop phase of the HW decoder mem2mem bahaviour is
behavior
> now clearly defined, we can move handling of the following states to the
> common v4l2-mem2mem core code:
> - draining
> - stopped
> - next-buf-is-last
>
> By introducing the following v4l2-mem2mem APIS:
APIs
> - v4l2_m2m_encoder_cmd/v4l2_m2m_ioctl_encoder_cmd to handle start/stop command
> - v4l2_m2m_decoder_cmd/v4l2_m2m_ioctl_decoder_cmd to handle start/stop command
> - v4l2_m2m_start_streaming to handle start of streaming of the de/encoder queue
> - v4l2_m2m_stop_streaming to handle stop of streaming of the de/encoder queue
> - v4l2_m2m_last_buffer_done to maek the current dest buffer as the last one
make
>
> And inline helpers:
> - v4l2_m2m_mark_stopped to mark the de/encoding process as stopped
> - v4l2_m2m_clear_state to clear the de/encoding state
> - v4l2_m2m_dst_buf_is_last to detect the current dequeud dst_buf is the last
dequeued
> - v4l2_m2m_has_stopped to detect the de/encoding stopped state
> - v4l2_m2m_is_last_draining_src_buf to detect the currect source buffer should
current
> be the last processing before stopping the de/encoding process
>
> The special next-buf-is-last when min_buffers != 1 case is also handled
> in v4l2_m2m_qbuf() by reusing the other introduced APIs.
>
> This state management has been stolen from the vicodec implementation,
> and is no-op for drivers not calling the v4l2_m2m_encoder_cmd or
> v4l2_m2m_decoder_cmd and v4l2_m2m_start_streaming/v4l2_m2m_stop_streaming.
Documenting these new helpers in the commit log is not very useful, they should
be documented in v4l2-mem2mem.h.
Since this is all fairly complex, I would like to see more comments in the
source as well, explaining what is happening and how/when to use it.
>
> The vicodec will be the first one to be converted as an example.
>
> Signed-off-by: Neil Armstrong <[email protected]>
> ---
> drivers/media/v4l2-core/v4l2-mem2mem.c | 172 ++++++++++++++++++++++++-
> include/media/v4l2-mem2mem.h | 95 ++++++++++++++
> 2 files changed, 265 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-core/v4l2-mem2mem.c
> index 1afd9c6ad908..f221d6c7a137 100644
> --- a/drivers/media/v4l2-core/v4l2-mem2mem.c
> +++ b/drivers/media/v4l2-core/v4l2-mem2mem.c
> @@ -340,6 +340,11 @@ static void __v4l2_m2m_try_queue(struct v4l2_m2m_dev *m2m_dev,
> m2m_ctx->new_frame = !dst->vb2_buf.copied_timestamp ||
> dst->vb2_buf.timestamp != src->vb2_buf.timestamp;
>
> + if (m2m_ctx->has_stopped) {
> + dprintk("Device has stopped\n");
> + goto job_unlock;
> + }
> +
> if (m2m_dev->m2m_ops->job_ready
> && (!m2m_dev->m2m_ops->job_ready(m2m_ctx->priv))) {
> dprintk("Driver not ready\n");
> @@ -556,6 +561,99 @@ int v4l2_m2m_querybuf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
> }
> EXPORT_SYMBOL_GPL(v4l2_m2m_querybuf);
>
> +void v4l2_m2m_last_buffer_done(struct v4l2_m2m_ctx *m2m_ctx,
> + struct vb2_v4l2_buffer *vbuf)
> +{
> + vbuf->flags |= V4L2_BUF_FLAG_LAST;
> + vb2_buffer_done(&vbuf->vb2_buf, VB2_BUF_STATE_DONE);
> +
> + v4l2_m2m_mark_stopped(m2m_ctx);
> +}
> +EXPORT_SYMBOL_GPL(v4l2_m2m_last_buffer_done);
> +
> +static int v4l2_mark_last_buf(struct v4l2_m2m_ctx *m2m_ctx)
The name of this function is not very clear and actually confusing given
the name of the previous function v4l2_m2m_last_buffer_done(). Hopefully
you can come up with something a bit more descriptive.
> +{
> + struct vb2_v4l2_buffer *next_dst_buf;
> +
> + if (m2m_ctx->is_draining)
> + return -EBUSY;
> +
> + if (m2m_ctx->has_stopped)
> + return 0;
> +
> + m2m_ctx->last_src_buf = v4l2_m2m_last_src_buf(m2m_ctx);
> + m2m_ctx->is_draining = true;
> +
> + if (m2m_ctx->last_src_buf)
> + return 0;
> +
> + next_dst_buf = v4l2_m2m_dst_buf_remove(m2m_ctx);
> + if (!next_dst_buf) {
> + m2m_ctx->next_buf_last = true;
> + return 0;
> + }
> +
> + v4l2_m2m_last_buffer_done(m2m_ctx, next_dst_buf);
> +
> + return 0;
> +}
> +
> +void v4l2_m2m_start_streaming(struct v4l2_m2m_ctx *m2m_ctx, struct vb2_queue *q)
> +{
> + if (V4L2_TYPE_IS_OUTPUT(q->type))
> + m2m_ctx->last_src_buf = NULL;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_m2m_start_streaming);
> +
> +void v4l2_m2m_stop_streaming(struct v4l2_m2m_ctx *m2m_ctx, struct vb2_queue *q)
> +{
> + if (V4L2_TYPE_IS_OUTPUT(q->type)) {
> + if (m2m_ctx->is_draining) {
> + struct vb2_v4l2_buffer *next_dst_buf;
> +
> + m2m_ctx->last_src_buf = NULL;
> + next_dst_buf = v4l2_m2m_dst_buf_remove(m2m_ctx);
> + if (!next_dst_buf)
> + m2m_ctx->next_buf_last = true;
> + else
> + v4l2_m2m_last_buffer_done(m2m_ctx,
> + next_dst_buf);
> + }
> + } else {
> + v4l2_m2m_clear_state(m2m_ctx);
> + }
> +}
> +EXPORT_SYMBOL_GPL(v4l2_m2m_stop_streaming);
Same for these two functions: the names suggest that these implement start and stop
streaming, but instead they are called from start and stop streaming to update the
state.
> +
> +static void v4l2_m2m_force_last_buf_done(struct v4l2_m2m_ctx *m2m_ctx,
> + struct vb2_queue *q)
> +{
> + struct vb2_buffer *vb;
> + struct vb2_v4l2_buffer *vbuf;
> + unsigned int i;
> +
> + if (WARN_ON(q->is_output))
> + return;
> + if (list_empty(&q->queued_list))
> + return;
> +
> + vb = list_first_entry(&q->queued_list, struct vb2_buffer, queued_entry);
> + for (i = 0; i < vb->num_planes; i++)
> + vb2_set_plane_payload(vb, i, 0);
> +
> + /*
> + * Since the buffer hasn't been queued to the ready queue,
> + * mark is active and owned before marking it LAST and DONE
> + */
> + vb->state = VB2_BUF_STATE_ACTIVE;
> + atomic_inc(&q->owned_by_drv_count);
> +
> + vbuf = to_vb2_v4l2_buffer(vb);
> + vbuf->field = V4L2_FIELD_NONE;
> +
> + v4l2_m2m_last_buffer_done(m2m_ctx, vbuf);
> +}
> +
> int v4l2_m2m_qbuf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
> struct v4l2_buffer *buf)
> {
> @@ -570,11 +668,25 @@ int v4l2_m2m_qbuf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
> __func__);
> return -EPERM;
> }
> +
> ret = vb2_qbuf(vq, vdev->v4l2_dev->mdev, buf);
> - if (!ret && !(buf->flags & V4L2_BUF_FLAG_IN_REQUEST))
> + if (ret)
> + return ret;
> +
> + /*
> + * If the capture queue is streaming, but streaming hasn't started
> + * on the device, but was asked to stop, mark the previously queued
> + * buffer as DONE with LAST flag since it won't be queued on the
> + * device.
> + */
> + if (!V4L2_TYPE_IS_OUTPUT(vq->type) &&
> + vb2_is_streaming(vq) && !vb2_start_streaming_called(vq) &&
> + (v4l2_m2m_has_stopped(m2m_ctx) || v4l2_m2m_dst_buf_is_last(m2m_ctx)))
> + v4l2_m2m_force_last_buf_done(m2m_ctx, vq);
> + else if (!(buf->flags & V4L2_BUF_FLAG_IN_REQUEST))
> v4l2_m2m_try_schedule(m2m_ctx);
>
> - return ret;
> + return 0;
> }
> EXPORT_SYMBOL_GPL(v4l2_m2m_qbuf);
>
> @@ -1225,6 +1337,62 @@ int v4l2_m2m_ioctl_try_decoder_cmd(struct file *file, void *fh,
> }
> EXPORT_SYMBOL_GPL(v4l2_m2m_ioctl_try_decoder_cmd);
>
> +int v4l2_m2m_encoder_cmd(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
> + struct v4l2_encoder_cmd *ec)
> +{
> + if (ec->cmd != V4L2_ENC_CMD_STOP && ec->cmd != V4L2_ENC_CMD_START)
> + return -EINVAL;
> +
> + if (ec->cmd == V4L2_ENC_CMD_STOP)
> + return v4l2_mark_last_buf(m2m_ctx);
> +
> + if (m2m_ctx->is_draining)
> + return -EBUSY;
> +
> + if (m2m_ctx->has_stopped)
> + m2m_ctx->has_stopped = false;
> +
> + return 0;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_m2m_encoder_cmd);
The next patch uses this function as follows in vicodec:
+ ret = v4l2_m2m_ioctl_encoder_cmd(file, fh, ec);
+ if (ret < 0)
+ return ret;
+
+ if (ec->cmd == V4L2_ENC_CMD_STOP &&
+ v4l2_m2m_has_stopped(ctx->fh.m2m_ctx))
+ v4l2_event_queue_fh(&ctx->fh, &vicodec_eos_event);
+
+ if (ec->cmd == V4L2_ENC_CMD_START &&
+ v4l2_m2m_has_stopped(ctx->fh.m2m_ctx))
vb2_clear_last_buffer_dequeued(&ctx->fh.m2m_ctx->cap_q_ctx.q);
I was wondering if that would be standard behavior for codecs and should
be added to v4l2_m2m_encoder_cmd. Ditto for decoder_cmd below.
> +
> +int v4l2_m2m_decoder_cmd(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
> + struct v4l2_decoder_cmd *dc)
> +{
> + if (dc->cmd != V4L2_DEC_CMD_STOP && dc->cmd != V4L2_DEC_CMD_START)
> + return -EINVAL;
> +
> + if (dc->cmd == V4L2_DEC_CMD_STOP)
> + return v4l2_mark_last_buf(m2m_ctx);
> +
> + if (m2m_ctx->is_draining)
> + return -EBUSY;
> +
> + if (m2m_ctx->has_stopped)
> + m2m_ctx->has_stopped = false;
> +
> + return 0;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_m2m_decoder_cmd);
> +
> +int v4l2_m2m_ioctl_encoder_cmd(struct file *file, void *priv,
> + struct v4l2_encoder_cmd *ec)
> +{
> + struct v4l2_fh *fh = file->private_data;
> +
> + return v4l2_m2m_encoder_cmd(file, fh->m2m_ctx, ec);
> +}
> +EXPORT_SYMBOL_GPL(v4l2_m2m_ioctl_encoder_cmd);
> +
> +int v4l2_m2m_ioctl_decoder_cmd(struct file *file, void *priv,
> + struct v4l2_decoder_cmd *dc)
> +{
> + struct v4l2_fh *fh = file->private_data;
> +
> + return v4l2_m2m_decoder_cmd(file, fh->m2m_ctx, dc);
> +}
> +EXPORT_SYMBOL_GPL(v4l2_m2m_ioctl_decoder_cmd);
> +
> int v4l2_m2m_ioctl_stateless_try_decoder_cmd(struct file *file, void *fh,
> struct v4l2_decoder_cmd *dc)
> {
> diff --git a/include/media/v4l2-mem2mem.h b/include/media/v4l2-mem2mem.h
> index 1d85e24791e4..3476889af46c 100644
> --- a/include/media/v4l2-mem2mem.h
> +++ b/include/media/v4l2-mem2mem.h
> @@ -80,6 +80,10 @@ struct v4l2_m2m_queue_ctx {
> * for an existing frame. This is always true unless
> * V4L2_BUF_CAP_SUPPORTS_M2M_HOLD_CAPTURE_BUF is set, which
> * indicates slicing support.
> + * @is_draining: indicates device is in draining phase
> + * @last_src_buf: indicate the last source buffer for draining
> + * @next_buf_last: next capture queud buffer will be tagged as last
> + * @has_stopped: indicate the device has been stopped
> * @m2m_dev: opaque pointer to the internal data to handle M2M context
> * @cap_q_ctx: Capture (output to memory) queue context
> * @out_q_ctx: Output (input from memory) queue context
> @@ -98,6 +102,11 @@ struct v4l2_m2m_ctx {
>
> bool new_frame;
>
> + bool is_draining;
> + struct vb2_v4l2_buffer *last_src_buf;
> + bool next_buf_last;
> + bool has_stopped;
> +
> /* internal use only */
> struct v4l2_m2m_dev *m2m_dev;
>
> @@ -215,6 +224,50 @@ v4l2_m2m_buf_done(struct vb2_v4l2_buffer *buf, enum vb2_buffer_state state)
> vb2_buffer_done(&buf->vb2_buf, state);
> }
>
> +static inline void
> +v4l2_m2m_clear_state(struct v4l2_m2m_ctx *m2m_ctx)
> +{
> + m2m_ctx->next_buf_last = false;
> + m2m_ctx->is_draining = false;
> + m2m_ctx->has_stopped = false;
> +}
> +
> +static inline void
> +v4l2_m2m_mark_stopped(struct v4l2_m2m_ctx *m2m_ctx)
> +{
> + m2m_ctx->next_buf_last = false;
> + m2m_ctx->is_draining = false;
> + m2m_ctx->has_stopped = true;
> +}
> +
> +static inline bool
> +v4l2_m2m_dst_buf_is_last(struct v4l2_m2m_ctx *m2m_ctx)
> +{
> + return m2m_ctx->is_draining && m2m_ctx->next_buf_last;
> +}
> +
> +static inline bool
> +v4l2_m2m_has_stopped(struct v4l2_m2m_ctx *m2m_ctx)
> +{
> + return m2m_ctx->has_stopped;
> +}
> +
> +static inline bool
> +v4l2_m2m_is_last_draining_src_buf(struct v4l2_m2m_ctx *m2m_ctx,
> + struct vb2_v4l2_buffer *buf)
> +{
> + return m2m_ctx->is_draining && buf == m2m_ctx->last_src_buf;
> +}
Comments are needed for all 5 functions above.
> +
> +/**
> + * v4l2_m2m_last_buffer_done() - marks the buffer with LAST flag and DONE
> + *
> + * @m2m_ctx: m2m context assigned to the instance given by struct &v4l2_m2m_ctx
> + * @vbuf: pointer to struct &v4l2_buffer
> + */
> +void v4l2_m2m_last_buffer_done(struct v4l2_m2m_ctx *m2m_ctx,
> + struct vb2_v4l2_buffer *vbuf);
> +
> /**
> * v4l2_m2m_reqbufs() - multi-queue-aware REQBUFS multiplexer
> *
> @@ -312,6 +365,44 @@ int v4l2_m2m_streamon(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
> int v4l2_m2m_streamoff(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
> enum v4l2_buf_type type);
>
> +/**
> + * v4l2_m2m_start_streaming() - handle start of streaming of a video queue
> + *
> + * @m2m_ctx: m2m context assigned to the instance given by struct &v4l2_m2m_ctx
> + * @q: queue
> + */
> +void v4l2_m2m_start_streaming(struct v4l2_m2m_ctx *m2m_ctx,
> + struct vb2_queue *q);
> +
> +/**
> + * v4l2_m2m_stop_streaming() - handle stop of streaming of a video queue
> + *
> + * @m2m_ctx: m2m context assigned to the instance given by struct &v4l2_m2m_ctx
> + * @q: queue
> + */
> +void v4l2_m2m_stop_streaming(struct v4l2_m2m_ctx *m2m_ctx,
> + struct vb2_queue *q);
> +
> +/**
> + * v4l2_m2m_encoder_cmd() - execute an encoder command
> + *
> + * @file: pointer to struct &file
> + * @m2m_ctx: m2m context assigned to the instance given by struct &v4l2_m2m_ctx
> + * @ec: pointer to the encoder command
> + */
> +int v4l2_m2m_encoder_cmd(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
> + struct v4l2_encoder_cmd *ec);
> +
> +/**
> + * v4l2_m2m_decoder_cmd() - execute a decoder command
> + *
> + * @file: pointer to struct &file
> + * @m2m_ctx: m2m context assigned to the instance given by struct &v4l2_m2m_ctx
> + * @dc: pointer to the decoder command
> + */
> +int v4l2_m2m_decoder_cmd(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
> + struct v4l2_decoder_cmd *dc);
> +
> /**
> * v4l2_m2m_poll() - poll replacement, for destination buffers only
> *
> @@ -704,6 +795,10 @@ int v4l2_m2m_ioctl_streamon(struct file *file, void *fh,
> enum v4l2_buf_type type);
> int v4l2_m2m_ioctl_streamoff(struct file *file, void *fh,
> enum v4l2_buf_type type);
> +int v4l2_m2m_ioctl_encoder_cmd(struct file *file, void *fh,
> + struct v4l2_encoder_cmd *ec);
> +int v4l2_m2m_ioctl_decoder_cmd(struct file *file, void *fh,
> + struct v4l2_decoder_cmd *dc);
> int v4l2_m2m_ioctl_try_encoder_cmd(struct file *file, void *fh,
> struct v4l2_encoder_cmd *ec);
> int v4l2_m2m_ioctl_try_decoder_cmd(struct file *file, void *fh,
>
Regards,
Hans
Hi Hans,
On 14/02/2020 16:54, Hans Verkuil wrote:
> On 2/6/20 9:26 AM, Neil Armstrong wrote:
>> Since the draining and stop phase of the HW decoder mem2mem bahaviour is
>
> behavior
>
>> now clearly defined, we can move handling of the following states to the
>> common v4l2-mem2mem core code:
>> - draining
>> - stopped
>> - next-buf-is-last
>>
>> By introducing the following v4l2-mem2mem APIS:
>
> APIs
>
>> - v4l2_m2m_encoder_cmd/v4l2_m2m_ioctl_encoder_cmd to handle start/stop command
>> - v4l2_m2m_decoder_cmd/v4l2_m2m_ioctl_decoder_cmd to handle start/stop command
>> - v4l2_m2m_start_streaming to handle start of streaming of the de/encoder queue
>> - v4l2_m2m_stop_streaming to handle stop of streaming of the de/encoder queue
>> - v4l2_m2m_last_buffer_done to maek the current dest buffer as the last one
>
> make
>
>>
>> And inline helpers:
>> - v4l2_m2m_mark_stopped to mark the de/encoding process as stopped
>> - v4l2_m2m_clear_state to clear the de/encoding state
>> - v4l2_m2m_dst_buf_is_last to detect the current dequeud dst_buf is the last
>
> dequeued
>
>> - v4l2_m2m_has_stopped to detect the de/encoding stopped state
>> - v4l2_m2m_is_last_draining_src_buf to detect the currect source buffer should
>
> current
>
>> be the last processing before stopping the de/encoding process
>>
>> The special next-buf-is-last when min_buffers != 1 case is also handled
>> in v4l2_m2m_qbuf() by reusing the other introduced APIs.
>>
>> This state management has been stolen from the vicodec implementation,
>> and is no-op for drivers not calling the v4l2_m2m_encoder_cmd or
>> v4l2_m2m_decoder_cmd and v4l2_m2m_start_streaming/v4l2_m2m_stop_streaming.
>
> Documenting these new helpers in the commit log is not very useful, they should
> be documented in v4l2-mem2mem.h.
I added an explicit documentation for these helpers.
>
> Since this is all fairly complex, I would like to see more comments in the
> source as well, explaining what is happening and how/when to use it.
Ok
>
>>
>> The vicodec will be the first one to be converted as an example.
>>
>> Signed-off-by: Neil Armstrong <[email protected]>
>> ---
>> drivers/media/v4l2-core/v4l2-mem2mem.c | 172 ++++++++++++++++++++++++-
>> include/media/v4l2-mem2mem.h | 95 ++++++++++++++
>> 2 files changed, 265 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-core/v4l2-mem2mem.c
>> index 1afd9c6ad908..f221d6c7a137 100644
>> --- a/drivers/media/v4l2-core/v4l2-mem2mem.c
>> +++ b/drivers/media/v4l2-core/v4l2-mem2mem.c
>> @@ -340,6 +340,11 @@ static void __v4l2_m2m_try_queue(struct v4l2_m2m_dev *m2m_dev,
>> m2m_ctx->new_frame = !dst->vb2_buf.copied_timestamp ||
>> dst->vb2_buf.timestamp != src->vb2_buf.timestamp;
>>
>> + if (m2m_ctx->has_stopped) {
>> + dprintk("Device has stopped\n");
>> + goto job_unlock;
>> + }
>> +
>> if (m2m_dev->m2m_ops->job_ready
>> && (!m2m_dev->m2m_ops->job_ready(m2m_ctx->priv))) {
>> dprintk("Driver not ready\n");
>> @@ -556,6 +561,99 @@ int v4l2_m2m_querybuf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
>> }
>> EXPORT_SYMBOL_GPL(v4l2_m2m_querybuf);
>>
>> +void v4l2_m2m_last_buffer_done(struct v4l2_m2m_ctx *m2m_ctx,
>> + struct vb2_v4l2_buffer *vbuf)
>> +{
>> + vbuf->flags |= V4L2_BUF_FLAG_LAST;
>> + vb2_buffer_done(&vbuf->vb2_buf, VB2_BUF_STATE_DONE);
>> +
>> + v4l2_m2m_mark_stopped(m2m_ctx);
>> +}
>> +EXPORT_SYMBOL_GPL(v4l2_m2m_last_buffer_done);
>> +
>> +static int v4l2_mark_last_buf(struct v4l2_m2m_ctx *m2m_ctx)
>
> The name of this function is not very clear and actually confusing given
> the name of the previous function v4l2_m2m_last_buffer_done(). Hopefully
> you can come up with something a bit more descriptive.
OK
>
>> +{
>> + struct vb2_v4l2_buffer *next_dst_buf;
>> +
>> + if (m2m_ctx->is_draining)
>> + return -EBUSY;
>> +
>> + if (m2m_ctx->has_stopped)
>> + return 0;
>> +
>> + m2m_ctx->last_src_buf = v4l2_m2m_last_src_buf(m2m_ctx);
>> + m2m_ctx->is_draining = true;
>> +
>> + if (m2m_ctx->last_src_buf)
>> + return 0;
>> +
>> + next_dst_buf = v4l2_m2m_dst_buf_remove(m2m_ctx);
>> + if (!next_dst_buf) {
>> + m2m_ctx->next_buf_last = true;
>> + return 0;
>> + }
>> +
>> + v4l2_m2m_last_buffer_done(m2m_ctx, next_dst_buf);
>> +
>> + return 0;
>> +}
>> +
>> +void v4l2_m2m_start_streaming(struct v4l2_m2m_ctx *m2m_ctx, struct vb2_queue *q)
>> +{
>> + if (V4L2_TYPE_IS_OUTPUT(q->type))
>> + m2m_ctx->last_src_buf = NULL;
>> +}
>> +EXPORT_SYMBOL_GPL(v4l2_m2m_start_streaming);
>> +
>> +void v4l2_m2m_stop_streaming(struct v4l2_m2m_ctx *m2m_ctx, struct vb2_queue *q)
>> +{
>> + if (V4L2_TYPE_IS_OUTPUT(q->type)) {
>> + if (m2m_ctx->is_draining) {
>> + struct vb2_v4l2_buffer *next_dst_buf;
>> +
>> + m2m_ctx->last_src_buf = NULL;
>> + next_dst_buf = v4l2_m2m_dst_buf_remove(m2m_ctx);
>> + if (!next_dst_buf)
>> + m2m_ctx->next_buf_last = true;
>> + else
>> + v4l2_m2m_last_buffer_done(m2m_ctx,
>> + next_dst_buf);
>> + }
>> + } else {
>> + v4l2_m2m_clear_state(m2m_ctx);
>> + }
>> +}
>> +EXPORT_SYMBOL_GPL(v4l2_m2m_stop_streaming);
>
> Same for these two functions: the names suggest that these implement start and stop
> streaming, but instead they are called from start and stop streaming to update the
> state.
Indeed, renaming
>
>> +
>> +static void v4l2_m2m_force_last_buf_done(struct v4l2_m2m_ctx *m2m_ctx,
>> + struct vb2_queue *q)
>> +{
>> + struct vb2_buffer *vb;
>> + struct vb2_v4l2_buffer *vbuf;
>> + unsigned int i;
>> +
>> + if (WARN_ON(q->is_output))
>> + return;
>> + if (list_empty(&q->queued_list))
>> + return;
>> +
>> + vb = list_first_entry(&q->queued_list, struct vb2_buffer, queued_entry);
>> + for (i = 0; i < vb->num_planes; i++)
>> + vb2_set_plane_payload(vb, i, 0);
>> +
>> + /*
>> + * Since the buffer hasn't been queued to the ready queue,
>> + * mark is active and owned before marking it LAST and DONE
>> + */
>> + vb->state = VB2_BUF_STATE_ACTIVE;
>> + atomic_inc(&q->owned_by_drv_count);
>> +
>> + vbuf = to_vb2_v4l2_buffer(vb);
>> + vbuf->field = V4L2_FIELD_NONE;
>> +
>> + v4l2_m2m_last_buffer_done(m2m_ctx, vbuf);
>> +}
>> +
>> int v4l2_m2m_qbuf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
>> struct v4l2_buffer *buf)
>> {
>> @@ -570,11 +668,25 @@ int v4l2_m2m_qbuf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
>> __func__);
>> return -EPERM;
>> }
>> +
>> ret = vb2_qbuf(vq, vdev->v4l2_dev->mdev, buf);
>> - if (!ret && !(buf->flags & V4L2_BUF_FLAG_IN_REQUEST))
>> + if (ret)
>> + return ret;
>> +
>> + /*
>> + * If the capture queue is streaming, but streaming hasn't started
>> + * on the device, but was asked to stop, mark the previously queued
>> + * buffer as DONE with LAST flag since it won't be queued on the
>> + * device.
>> + */
>> + if (!V4L2_TYPE_IS_OUTPUT(vq->type) &&
>> + vb2_is_streaming(vq) && !vb2_start_streaming_called(vq) &&
>> + (v4l2_m2m_has_stopped(m2m_ctx) || v4l2_m2m_dst_buf_is_last(m2m_ctx)))
>> + v4l2_m2m_force_last_buf_done(m2m_ctx, vq);
>> + else if (!(buf->flags & V4L2_BUF_FLAG_IN_REQUEST))
>> v4l2_m2m_try_schedule(m2m_ctx);
>>
>> - return ret;
>> + return 0;
>> }
>> EXPORT_SYMBOL_GPL(v4l2_m2m_qbuf);
>>
>> @@ -1225,6 +1337,62 @@ int v4l2_m2m_ioctl_try_decoder_cmd(struct file *file, void *fh,
>> }
>> EXPORT_SYMBOL_GPL(v4l2_m2m_ioctl_try_decoder_cmd);
>>
>> +int v4l2_m2m_encoder_cmd(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
>> + struct v4l2_encoder_cmd *ec)
>> +{
>> + if (ec->cmd != V4L2_ENC_CMD_STOP && ec->cmd != V4L2_ENC_CMD_START)
>> + return -EINVAL;
>> +
>> + if (ec->cmd == V4L2_ENC_CMD_STOP)
>> + return v4l2_mark_last_buf(m2m_ctx);
>> +
>> + if (m2m_ctx->is_draining)
>> + return -EBUSY;
>> +
>> + if (m2m_ctx->has_stopped)
>> + m2m_ctx->has_stopped = false;
>> +
>> + return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(v4l2_m2m_encoder_cmd);
>
> The next patch uses this function as follows in vicodec:
>
> + ret = v4l2_m2m_ioctl_encoder_cmd(file, fh, ec);
> + if (ret < 0)
> + return ret;
> +
> + if (ec->cmd == V4L2_ENC_CMD_STOP &&
> + v4l2_m2m_has_stopped(ctx->fh.m2m_ctx))
> + v4l2_event_queue_fh(&ctx->fh, &vicodec_eos_event);
> +
> + if (ec->cmd == V4L2_ENC_CMD_START &&
> + v4l2_m2m_has_stopped(ctx->fh.m2m_ctx))
> vb2_clear_last_buffer_dequeued(&ctx->fh.m2m_ctx->cap_q_ctx.q);
>
> I was wondering if that would be standard behavior for codecs and should
> be added to v4l2_m2m_encoder_cmd. Ditto for decoder_cmd below.
Unfortunately, no.
The issue comes from the synchronous behaviour of the decoder, in the Amlogic case,
the H264 decoder behavior with src and dst buf is asynchronous, where we need to dequeue
and push to hardware a unknown variable number of buffers before getting capture buffers.
This is not true for VP9 and HEVC for example, but since we need to manage all the
different codecs in a single codebase, we can't assume/predict a direct
src_buf->dst_buf relationship in all cases.
This is why I left this in the vicodec code.
>
>> +
>> +int v4l2_m2m_decoder_cmd(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
>> + struct v4l2_decoder_cmd *dc)
>> +{
>> + if (dc->cmd != V4L2_DEC_CMD_STOP && dc->cmd != V4L2_DEC_CMD_START)
>> + return -EINVAL;
>> +
>> + if (dc->cmd == V4L2_DEC_CMD_STOP)
>> + return v4l2_mark_last_buf(m2m_ctx);
>> +
>> + if (m2m_ctx->is_draining)
>> + return -EBUSY;
>> +
>> + if (m2m_ctx->has_stopped)
>> + m2m_ctx->has_stopped = false;
>> +
>> + return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(v4l2_m2m_decoder_cmd);
>> +
>> +int v4l2_m2m_ioctl_encoder_cmd(struct file *file, void *priv,
>> + struct v4l2_encoder_cmd *ec)
>> +{
>> + struct v4l2_fh *fh = file->private_data;
>> +
>> + return v4l2_m2m_encoder_cmd(file, fh->m2m_ctx, ec);
>> +}
>> +EXPORT_SYMBOL_GPL(v4l2_m2m_ioctl_encoder_cmd);
>> +
>> +int v4l2_m2m_ioctl_decoder_cmd(struct file *file, void *priv,
>> + struct v4l2_decoder_cmd *dc)
>> +{
>> + struct v4l2_fh *fh = file->private_data;
>> +
>> + return v4l2_m2m_decoder_cmd(file, fh->m2m_ctx, dc);
>> +}
>> +EXPORT_SYMBOL_GPL(v4l2_m2m_ioctl_decoder_cmd);
>> +
>> int v4l2_m2m_ioctl_stateless_try_decoder_cmd(struct file *file, void *fh,
>> struct v4l2_decoder_cmd *dc)
>> {
>> diff --git a/include/media/v4l2-mem2mem.h b/include/media/v4l2-mem2mem.h
>> index 1d85e24791e4..3476889af46c 100644
>> --- a/include/media/v4l2-mem2mem.h
>> +++ b/include/media/v4l2-mem2mem.h
>> @@ -80,6 +80,10 @@ struct v4l2_m2m_queue_ctx {
>> * for an existing frame. This is always true unless
>> * V4L2_BUF_CAP_SUPPORTS_M2M_HOLD_CAPTURE_BUF is set, which
>> * indicates slicing support.
>> + * @is_draining: indicates device is in draining phase
>> + * @last_src_buf: indicate the last source buffer for draining
>> + * @next_buf_last: next capture queud buffer will be tagged as last
>> + * @has_stopped: indicate the device has been stopped
>> * @m2m_dev: opaque pointer to the internal data to handle M2M context
>> * @cap_q_ctx: Capture (output to memory) queue context
>> * @out_q_ctx: Output (input from memory) queue context
>> @@ -98,6 +102,11 @@ struct v4l2_m2m_ctx {
>>
>> bool new_frame;
>>
>> + bool is_draining;
>> + struct vb2_v4l2_buffer *last_src_buf;
>> + bool next_buf_last;
>> + bool has_stopped;
>> +
>> /* internal use only */
>> struct v4l2_m2m_dev *m2m_dev;
>>
>> @@ -215,6 +224,50 @@ v4l2_m2m_buf_done(struct vb2_v4l2_buffer *buf, enum vb2_buffer_state state)
>> vb2_buffer_done(&buf->vb2_buf, state);
>> }
>>
>> +static inline void
>> +v4l2_m2m_clear_state(struct v4l2_m2m_ctx *m2m_ctx)
>> +{
>> + m2m_ctx->next_buf_last = false;
>> + m2m_ctx->is_draining = false;
>> + m2m_ctx->has_stopped = false;
>> +}
>> +
>> +static inline void
>> +v4l2_m2m_mark_stopped(struct v4l2_m2m_ctx *m2m_ctx)
>> +{
>> + m2m_ctx->next_buf_last = false;
>> + m2m_ctx->is_draining = false;
>> + m2m_ctx->has_stopped = true;
>> +}
>> +
>> +static inline bool
>> +v4l2_m2m_dst_buf_is_last(struct v4l2_m2m_ctx *m2m_ctx)
>> +{
>> + return m2m_ctx->is_draining && m2m_ctx->next_buf_last;
>> +}
>> +
>> +static inline bool
>> +v4l2_m2m_has_stopped(struct v4l2_m2m_ctx *m2m_ctx)
>> +{
>> + return m2m_ctx->has_stopped;
>> +}
>> +
>> +static inline bool
>> +v4l2_m2m_is_last_draining_src_buf(struct v4l2_m2m_ctx *m2m_ctx,
>> + struct vb2_v4l2_buffer *buf)
>> +{
>> + return m2m_ctx->is_draining && buf == m2m_ctx->last_src_buf;
>> +}
>
> Comments are needed for all 5 functions above.
Ok
>
>> +
>> +/**
>> + * v4l2_m2m_last_buffer_done() - marks the buffer with LAST flag and DONE
>> + *
>> + * @m2m_ctx: m2m context assigned to the instance given by struct &v4l2_m2m_ctx
>> + * @vbuf: pointer to struct &v4l2_buffer
>> + */
>> +void v4l2_m2m_last_buffer_done(struct v4l2_m2m_ctx *m2m_ctx,
>> + struct vb2_v4l2_buffer *vbuf);
>> +
>> /**
>> * v4l2_m2m_reqbufs() - multi-queue-aware REQBUFS multiplexer
>> *
>> @@ -312,6 +365,44 @@ int v4l2_m2m_streamon(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
>> int v4l2_m2m_streamoff(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
>> enum v4l2_buf_type type);
>>
>> +/**
>> + * v4l2_m2m_start_streaming() - handle start of streaming of a video queue
>> + *
>> + * @m2m_ctx: m2m context assigned to the instance given by struct &v4l2_m2m_ctx
>> + * @q: queue
>> + */
>> +void v4l2_m2m_start_streaming(struct v4l2_m2m_ctx *m2m_ctx,
>> + struct vb2_queue *q);
>> +
>> +/**
>> + * v4l2_m2m_stop_streaming() - handle stop of streaming of a video queue
>> + *
>> + * @m2m_ctx: m2m context assigned to the instance given by struct &v4l2_m2m_ctx
>> + * @q: queue
>> + */
>> +void v4l2_m2m_stop_streaming(struct v4l2_m2m_ctx *m2m_ctx,
>> + struct vb2_queue *q);
>> +
>> +/**
>> + * v4l2_m2m_encoder_cmd() - execute an encoder command
>> + *
>> + * @file: pointer to struct &file
>> + * @m2m_ctx: m2m context assigned to the instance given by struct &v4l2_m2m_ctx
>> + * @ec: pointer to the encoder command
>> + */
>> +int v4l2_m2m_encoder_cmd(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
>> + struct v4l2_encoder_cmd *ec);
>> +
>> +/**
>> + * v4l2_m2m_decoder_cmd() - execute a decoder command
>> + *
>> + * @file: pointer to struct &file
>> + * @m2m_ctx: m2m context assigned to the instance given by struct &v4l2_m2m_ctx
>> + * @dc: pointer to the decoder command
>> + */
>> +int v4l2_m2m_decoder_cmd(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
>> + struct v4l2_decoder_cmd *dc);
>> +
>> /**
>> * v4l2_m2m_poll() - poll replacement, for destination buffers only
>> *
>> @@ -704,6 +795,10 @@ int v4l2_m2m_ioctl_streamon(struct file *file, void *fh,
>> enum v4l2_buf_type type);
>> int v4l2_m2m_ioctl_streamoff(struct file *file, void *fh,
>> enum v4l2_buf_type type);
>> +int v4l2_m2m_ioctl_encoder_cmd(struct file *file, void *fh,
>> + struct v4l2_encoder_cmd *ec);
>> +int v4l2_m2m_ioctl_decoder_cmd(struct file *file, void *fh,
>> + struct v4l2_decoder_cmd *dc);
>> int v4l2_m2m_ioctl_try_encoder_cmd(struct file *file, void *fh,
>> struct v4l2_encoder_cmd *ec);
>> int v4l2_m2m_ioctl_try_decoder_cmd(struct file *file, void *fh,
>>
>
> Regards,
>
> Hans
>
Thanks for the review,
Neil