MT8195 has two H264 encoder cores, they have their own power-domains,
clocks, interrupts, register base. The two H264 encoder cores can work
together to achieve higher performance.
This series of patches is to use enable two h264 encoder cores.
path[1..2]: use linux component framework to manage encoder hardware,
user call "mt8195-vcodec-enc" driver can get the encoder master device,
the encoding work is done by the two encoder core device. The hw_mode
variable is added to distinguish from old platform, two encoder cores
called "FRAME_RACING_MODE".
The hardware mode of two encoder cores work together(overlap, another
word called) on MT8195 called "frame_racing_mode", the two encoder
power-domains should be power on together while encoding, the encoding
process look like this:
VENC Core0 frm#0....frm#2....frm#4
VENC Core1 .frm#1....frm#3....frm#5
patch[3..5]: due to the component device, the master device has no
power-domains/clocks properties in dtsi, so the power/clock init function
can't use for "frame_racing_mode" device in master device probe process,
it should be called in component device probe process. Power on the
hardware power and clock on demand.
patch[6]: "frame_racing_mode" encoding need a new set of memory buffer
for two encoder cores. For compatibility, we should new a encoder driver
interface.
patch[7..9]: add "frame_racing_mode" encoding process:
As-Is: Synchronous
VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder IRQ -->
encode done with result --> job_finish
VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder IRQ -->
encode done with result --> job_finish
...
To-Be: Asynchronous
VIDIOC_QBUF#0 --> device_run(triger encoder core0) --> job_finish
..VIDIOC_QBUF#1 --> device_run(triger encoder core1) --> job_finish
(core0 may encode done here, return encode result to client)
VIDIOC_QBUF#2 --> device_run(triger encoder core0) --> job_finish
Thers is no "wait encoder IRQ" synchronous call during "frame_racing_mode"
encoding process, which can full use the two encoder cores to achieve
higher performance.
Irui Wang (9):
dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores
media: mtk-vcodec: Use component framework to manage encoder hardware
media: mtk-vcodec: Rewrite venc power manage interface
media: mtk-vcodec: Add venc power on/off interface
media: mtk-vcodec: Rewrite venc clock interface
media: mtk-vcodec: Add new venc drv interface for frame_racing mode
media: mtk-vcodec: Add frame racing mode encode process
media: mtk-vcodec: Return encode result to client
media: mtk-vcodec: Add delayed worker for encode timeout
.../bindings/media/mediatek-vcodec.txt | 2 +
drivers/media/platform/mtk-vcodec/Makefile | 2 +
.../platform/mtk-vcodec/mtk_vcodec_drv.h | 34 +-
.../platform/mtk-vcodec/mtk_vcodec_enc.c | 120 +++-
.../platform/mtk-vcodec/mtk_vcodec_enc.h | 10 +-
.../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 204 +++++-
.../platform/mtk-vcodec/mtk_vcodec_enc_hw.c | 253 +++++++
.../platform/mtk-vcodec/mtk_vcodec_enc_hw.h | 38 +
.../platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 213 ++++--
.../platform/mtk-vcodec/mtk_vcodec_enc_pm.h | 13 +-
.../platform/mtk-vcodec/mtk_vcodec_util.c | 19 +
.../platform/mtk-vcodec/mtk_vcodec_util.h | 5 +
.../platform/mtk-vcodec/venc/venc_common_if.c | 675 ++++++++++++++++++
.../platform/mtk-vcodec/venc/venc_h264_if.c | 6 +-
.../platform/mtk-vcodec/venc/venc_vp8_if.c | 2 +-
.../media/platform/mtk-vcodec/venc_drv_if.c | 96 ++-
.../media/platform/mtk-vcodec/venc_drv_if.h | 7 +
.../media/platform/mtk-vcodec/venc_vpu_if.c | 11 +-
.../media/platform/mtk-vcodec/venc_vpu_if.h | 3 +-
19 files changed, 1564 insertions(+), 149 deletions(-)
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
create mode 100644 drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
--
2.25.1
When encoding timeout, a delayed timeout worker is needed because
of asynchronous encoding process, restore encoding process in
timeout worker.
Signed-off-by: Irui Wang <[email protected]>
---
.../platform/mtk-vcodec/mtk_vcodec_enc.c | 12 ++++--
.../platform/mtk-vcodec/mtk_vcodec_enc_hw.c | 26 +++++++++++
.../platform/mtk-vcodec/mtk_vcodec_enc_hw.h | 2 +
.../platform/mtk-vcodec/venc/venc_common_if.c | 43 +++++++++++++++++--
4 files changed, 77 insertions(+), 6 deletions(-)
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index de8e012f1b0e..bc635436aa90 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -1340,9 +1340,15 @@ void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int core_id,
if (key_frame)
dst_vb2_v4l2->flags |= V4L2_BUF_FLAG_KEYFRAME;
- v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_DONE);
- dst_vb2_v4l2->vb2_buf.planes[0].bytesused = bs_size;
- v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_DONE);
+ if (time_out) {
+ v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_ERROR);
+ dst_vb2_v4l2->vb2_buf.planes[0].bytesused = 0;
+ v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_ERROR);
+ } else {
+ v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_DONE);
+ dst_vb2_v4l2->vb2_buf.planes[0].bytesused = bs_size;
+ v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_DONE);
+ }
}
}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
index 687b255c58e8..f6a5dfa6929f 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
@@ -16,6 +16,28 @@
#define VENC_PIC_FRM_TYPE 0x0010
#define VENC_PIC_KEY_FRM 0x2
+static void mtk_venc_timeout_worker(struct work_struct *work)
+{
+ struct mtk_venc_comp_dev *dev = container_of(work,
+ struct mtk_venc_comp_dev,
+ enc_timeout_worker.work);
+
+ struct mtk_vcodec_ctx *ctx;
+ unsigned long flags;
+
+ spin_lock_irqsave(&dev->master_dev->irqlock, flags);
+ ctx = dev->curr_ctx;
+
+ dev_err(&dev->plat_dev->dev, "core: %d timeout\n", dev->core_id);
+
+ mtk_venc_buf_done(ctx, dev->core_id, 0, 1, 0);
+
+ mtk_vcodec_enc_clock_off(dev->master_dev, dev->core_id);
+ dev->curr_ctx = NULL;
+ spin_unlock_irqrestore(&dev->master_dev->irqlock, flags);
+
+ mtk_venc_unlock(ctx, dev->core_id);
+}
/* Wake up core context wait_queue */
static void mtk_venc_comp_wake_up_ctx(struct mtk_vcodec_ctx *ctx,
unsigned int hw_id)
@@ -67,6 +89,7 @@ static irqreturn_t mtk_enc_comp_irq_handler(int irq, void *priv)
frm_type = readl(dev->reg_base + VENC_PIC_FRM_TYPE);
clean_irq_status(ctx->irq_status, addr);
if (ctx->irq_status & MTK_VENC_IRQ_STATUS_FRM) {
+ cancel_delayed_work(&dev->enc_timeout_worker);
if (ctx->hdr_size != 0) {
bs_size += ctx->hdr_size;
ctx->hdr_size = 0;
@@ -182,6 +205,9 @@ static int mtk_venc_comp_probe(struct platform_device *pdev)
platform_set_drvdata(pdev, comp_dev);
+ INIT_DELAYED_WORK(&comp_dev->enc_timeout_worker,
+ mtk_venc_timeout_worker);
+
ret = component_add(&pdev->dev, &mtk_venc_component_ops);
if (ret) {
dev_err(&pdev->dev, "Failed to add component: %d\n", ret);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
index f9ae97e252dc..e69344e38ca6 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
@@ -19,6 +19,7 @@
* @irq_status: venc hw irq status
* @enc_irq: venc device irq
* @core id: for venc core id: core#0, core#1...
+ * @enc_timeout_worker: venc timeout worker
*/
struct mtk_venc_comp_dev {
struct platform_device *plat_dev;
@@ -31,6 +32,7 @@ struct mtk_venc_comp_dev {
unsigned int irq_status;
int enc_irq;
int core_id;
+ struct delayed_work enc_timeout_worker;
};
#endif /* _MTK_VCODEC_ENC_HW_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
index 75b4e90232ae..1aab84faadf0 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
@@ -10,11 +10,13 @@
#include "../mtk_vcodec_drv.h"
#include "../mtk_vcodec_util.h"
#include "../mtk_vcodec_enc_hw.h"
+#include "../mtk_vcodec_enc_pm.h"
static const char h264_filler_marker[] = {0x0, 0x0, 0x0, 0x1, 0xc};
#define H264_FILLER_MARKER_SIZE ARRAY_SIZE(h264_filler_marker)
#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+#define MTK_VENC_HW_TIMEOUT_MSEC 2000
enum venc_vpu_work_buf {
VENC_VPU_WORK_BUF_RC_INFO_CORE0,
@@ -413,6 +415,34 @@ static void h264_encode_filler(struct venc_common_inst *inst, void *buf,
memset(p, 0xff, size);
}
+static void init_schedule_work(struct mtk_vcodec_ctx *ctx)
+{
+ unsigned long timeout;
+ struct mtk_venc_comp_dev *venc;
+
+ venc = ctx->dev->enc_comp_dev[ctx->hw_id];
+ timeout = msecs_to_jiffies(MTK_VENC_HW_TIMEOUT_MSEC);
+ schedule_delayed_work(&venc->enc_timeout_worker, timeout);
+}
+
+static void cancel_schedule_work(struct mtk_vcodec_ctx *ctx)
+{
+ struct mtk_venc_comp_dev *venc;
+ unsigned long flags;
+
+ venc = ctx->dev->enc_comp_dev[ctx->hw_id];
+ cancel_delayed_work(&venc->enc_timeout_worker);
+
+ mtk_venc_buf_done(ctx, ctx->hw_id, 0, 1, 0);
+ mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id);
+
+ spin_lock_irqsave(&ctx->dev->irqlock, flags);
+ venc->curr_ctx = NULL;
+ spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+
+ mtk_venc_unlock(ctx, ctx->hw_id);
+}
+
static int venc_init(struct mtk_vcodec_ctx *ctx)
{
int i;
@@ -489,12 +519,13 @@ static int venc_encode(void *handle,
ctx->pbs_buf[ctx->hw_id] = bs_buf->buf;
if (!inst->prepend_hdr) {
+ init_schedule_work(ctx);
ret = venc_encode_frame(inst, frm_buf, bs_buf,
&result->bs_size, ctx->hw_id);
if (ret) {
mtk_vcodec_err(inst, "encode frame failed: %d",
ret);
- return ret;
+ goto encode_err;
}
result->is_key_frm = inst->vpu_inst.is_key_frm;
@@ -503,11 +534,13 @@ static int venc_encode(void *handle,
mtk_vcodec_debug(inst, "venc_encode_frame prepend SPS/PPS");
+ init_schedule_work(ctx);
+
ret = venc_encode_header(inst, bs_buf, &bs_size_hdr);
if (ret) {
mtk_vcodec_err(inst, "encode prepend hdr failed: %d",
ret);
- return ret;
+ goto encode_err;
}
hdr_sz = bs_size_hdr;
@@ -530,7 +563,7 @@ static int venc_encode(void *handle,
if (ret) {
mtk_vcodec_err(inst, "encode hdr frame failed: %d",
ret);
- return ret;
+ goto encode_err;
}
result->bs_size = hdr_sz + filler_sz + bs_size_frm;
@@ -552,6 +585,10 @@ static int venc_encode(void *handle,
}
return ret;
+
+encode_err:
+ cancel_schedule_work(ctx);
+ return ret;
}
static int venc_set_param(void *handle,
--
2.25.1
For frame-racing mode encoding, there will no sync wait IRQ done
function, so the encode result can't return to client in
device_run. After device_run called to trigger encoding process,
finish it, so another scheduled job can be ran. When encoding
done, return the encode result to client in IRQ handler.
Signed-off-by: Irui Wang <[email protected]>
---
.../platform/mtk-vcodec/mtk_vcodec_drv.h | 6 ++
.../platform/mtk-vcodec/mtk_vcodec_enc.c | 61 ++++++++++++++++++-
.../platform/mtk-vcodec/mtk_vcodec_enc.h | 4 +-
.../platform/mtk-vcodec/mtk_vcodec_enc_hw.c | 35 +++++++++++
.../platform/mtk-vcodec/mtk_vcodec_util.h | 1 +
.../platform/mtk-vcodec/venc/venc_common_if.c | 9 +++
.../media/platform/mtk-vcodec/venc_drv_if.h | 2 +
7 files changed, 115 insertions(+), 3 deletions(-)
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index 7a9c2dd52b91..c43321362896 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -274,6 +274,9 @@ struct vdec_pic_info {
* @lock: protect variables accessed by V4L2 threads and worker thread such as
* mtk_video_dec_buf.
* @enc_idx: used to record encoded frame count
+ * @pfrm_buf: used to store current ctx's frame buffer information
+ * @pbs_buf: used to store current ctx's bitstream buffer information
+ * @hdr_size: used to store prepend header size
*/
struct mtk_vcodec_ctx {
enum mtk_instance_type type;
@@ -316,6 +319,9 @@ struct mtk_vcodec_ctx {
int hw_id;
int enc_idx;
+ struct vb2_v4l2_buffer *pfrm_buf[MTK_VENC_HW_MAX];
+ struct vb2_v4l2_buffer *pbs_buf[MTK_VENC_HW_MAX];
+ unsigned int hdr_size;
};
enum mtk_chip {
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index 6b37bd092fdf..de8e012f1b0e 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -867,6 +867,8 @@ static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
mtk_v4l2_debug(2, "[%d]-> type=%d", ctx->id, q->type);
+ mtk_venc_lock_all(ctx);
+
if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
while ((dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) {
dst_buf->vb2_buf.planes[0].bytesused = 0;
@@ -1014,10 +1016,13 @@ static int mtk_venc_param_change(struct mtk_vcodec_ctx *ctx)
ctx->id,
vb2_v4l2->vb2_buf.index,
mtk_buf->enc_params.force_intra);
- if (mtk_buf->enc_params.force_intra)
+ if (mtk_buf->enc_params.force_intra) {
ret |= venc_if_set_param(ctx,
VENC_SET_PARAM_FORCE_INTRA,
NULL);
+
+ ctx->enc_idx = 0;
+ }
}
mtk_buf->param_change = MTK_ENCODE_PARAM_NONE;
@@ -1068,9 +1073,12 @@ static void mtk_venc_worker(struct work_struct *work)
frm_buf.fb_addr[i].size =
(size_t)src_buf->vb2_buf.planes[i].length;
}
+ frm_buf.src_addr = src_buf;
+
bs_buf.va = vb2_plane_vaddr(&dst_buf->vb2_buf, 0);
bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0);
bs_buf.size = (size_t)dst_buf->vb2_buf.planes[0].length;
+ bs_buf.buf = dst_buf;
mtk_v4l2_debug(2,
"Framebuf PA=%llx Size=0x%zx;PA=0x%llx Size=0x%zx;PA=0x%llx Size=%zu",
@@ -1119,6 +1127,8 @@ static void m2mops_venc_device_run(void *priv)
{
struct mtk_vcodec_ctx *ctx = priv;
+ mtk_venc_param_change(ctx);
+
if ((ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc == V4L2_PIX_FMT_H264) &&
(ctx->state != MTK_STATE_HEADER)) {
/* encode h264 sps/pps header */
@@ -1127,7 +1137,6 @@ static void m2mops_venc_device_run(void *priv)
return;
}
- mtk_venc_param_change(ctx);
queue_work(ctx->dev->encode_workqueue, &ctx->encode_work);
}
@@ -1310,6 +1319,33 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
return vb2_queue_init(dst_vq);
}
+void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int core_id,
+ unsigned int bs_size, bool time_out, bool key_frame)
+{
+ struct vb2_v4l2_buffer *src_vb2_v4l2 = NULL;
+ struct vb2_v4l2_buffer *dst_vb2_v4l2 = NULL;
+
+ /*
+ * the frm_buf(src_buf) and bs_buf(dst_buf) can be obtained from ctx,
+ * then put them to done list, user can get them by dqbuf call
+ */
+ src_vb2_v4l2 = ctx->pfrm_buf[core_id];
+ dst_vb2_v4l2 = ctx->pbs_buf[core_id];
+
+ if (src_vb2_v4l2 && dst_vb2_v4l2) {
+ dst_vb2_v4l2->vb2_buf.timestamp =
+ src_vb2_v4l2->vb2_buf.timestamp;
+ dst_vb2_v4l2->timecode = src_vb2_v4l2->timecode;
+
+ if (key_frame)
+ dst_vb2_v4l2->flags |= V4L2_BUF_FLAG_KEYFRAME;
+
+ v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_DONE);
+ dst_vb2_v4l2->vb2_buf.planes[0].bytesused = bs_size;
+ v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_DONE);
+ }
+}
+
int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int core_id)
{
struct mtk_vcodec_dev *dev = ctx->dev;
@@ -1335,3 +1371,24 @@ void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx)
ctx->state = MTK_STATE_FREE;
}
+
+void mtk_venc_lock_all(struct mtk_vcodec_ctx *ctx)
+{
+ unsigned int i;
+ struct mtk_vcodec_dev *dev = ctx->dev;
+
+ /*
+ * For frame-racing mode encoding, there are may be bufs being encoded
+ * when get the empty flush buffer or stop streaming, for example, the
+ * buffer with LAST flag will return to client before the encoding
+ * buffers, which will cause frame lost.
+ * The encoder device mutex will be locked during encoding process,
+ * when encode done, the mutex unlocked. So if all encoder device mutex
+ * can be locked, which means there are no bufs being encoded at this
+ * time, then the buffer with LAST flag can return to client properly.
+ */
+ for (i = 0; i < MTK_VENC_HW_MAX; i++) {
+ mutex_lock(&dev->enc_mutex[i]);
+ mutex_unlock(&dev->enc_mutex[i]);
+ }
+}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
index 8d486f532025..1d603ccb2f3d 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
@@ -48,5 +48,7 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx);
int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx);
void mtk_vcodec_enc_set_default_params(struct mtk_vcodec_ctx *ctx);
-
+void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int core_id,
+ unsigned int bs_size, bool time_out, bool key_frame);
+void mtk_venc_lock_all(struct mtk_vcodec_ctx *ctx);
#endif /* _MTK_VCODEC_ENC_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
index 2a9112b0561d..687b255c58e8 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
@@ -12,6 +12,17 @@
#include "mtk_vcodec_enc_hw.h"
#include "mtk_vcodec_enc_pm.h"
#include "mtk_vcodec_enc.h"
+#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+#define VENC_PIC_FRM_TYPE 0x0010
+#define VENC_PIC_KEY_FRM 0x2
+
+/* Wake up core context wait_queue */
+static void mtk_venc_comp_wake_up_ctx(struct mtk_vcodec_ctx *ctx,
+ unsigned int hw_id)
+{
+ ctx->int_cond = 1;
+ wake_up_interruptible(&ctx->queue);
+}
static void clean_irq_status(unsigned int irq_status, void __iomem *addr)
{
@@ -40,6 +51,9 @@ static irqreturn_t mtk_enc_comp_irq_handler(int irq, void *priv)
struct mtk_vcodec_ctx *ctx;
unsigned long flags;
void __iomem *addr;
+ unsigned int bs_size;
+ unsigned int frm_type;
+ bool is_key_frame = 0;
spin_lock_irqsave(&dev->master_dev->irqlock, flags);
ctx = dev->curr_ctx;
@@ -49,7 +63,28 @@ static irqreturn_t mtk_enc_comp_irq_handler(int irq, void *priv)
addr = dev->reg_base + MTK_VENC_IRQ_ACK_OFFSET;
ctx->irq_status = readl(dev->reg_base + MTK_VENC_IRQ_STATUS_OFFSET);
+ bs_size = readl(dev->reg_base + VENC_PIC_BITSTREAM_BYTE_CNT);
+ frm_type = readl(dev->reg_base + VENC_PIC_FRM_TYPE);
clean_irq_status(ctx->irq_status, addr);
+ if (ctx->irq_status & MTK_VENC_IRQ_STATUS_FRM) {
+ if (ctx->hdr_size != 0) {
+ bs_size += ctx->hdr_size;
+ ctx->hdr_size = 0;
+ }
+ if (frm_type & VENC_PIC_KEY_FRM)
+ is_key_frame = 1;
+
+ mtk_venc_buf_done(ctx, dev->core_id, bs_size, 0, is_key_frame);
+ mtk_vcodec_enc_clock_off(dev->master_dev, dev->core_id);
+
+ spin_lock_irqsave(&dev->master_dev->irqlock, flags);
+ dev->curr_ctx = NULL;
+ spin_unlock_irqrestore(&dev->master_dev->irqlock, flags);
+
+ mtk_venc_unlock(ctx, dev->core_id);
+ } else {
+ mtk_venc_comp_wake_up_ctx(ctx, dev->core_id);
+ }
return IRQ_HANDLED;
}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
index 0027747c0a27..7321b7537f20 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h
@@ -15,6 +15,7 @@ struct mtk_vcodec_mem {
size_t size;
void *va;
dma_addr_t dma_addr;
+ void *buf;
};
struct mtk_vcodec_fb {
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
index 1580bd1beaac..75b4e90232ae 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
@@ -481,6 +481,13 @@ static int venc_encode(void *handle,
unsigned int bs_size_hdr;
unsigned int bs_size_frm;
+ /*
+ * the frm_buf and bs_buf need to be recorded into ctx,
+ * when encoding done, the target buffer can be obtained
+ */
+ ctx->pfrm_buf[ctx->hw_id] = frm_buf->src_addr;
+ ctx->pbs_buf[ctx->hw_id] = bs_buf->buf;
+
if (!inst->prepend_hdr) {
ret = venc_encode_frame(inst, frm_buf, bs_buf,
&result->bs_size, ctx->hw_id);
@@ -528,6 +535,8 @@ static int venc_encode(void *handle,
result->bs_size = hdr_sz + filler_sz + bs_size_frm;
+ ctx->hdr_size = hdr_sz + filler_sz;
+
mtk_vcodec_debug(inst, "hdr %d filler %d frame %d bs %d",
hdr_sz, filler_sz, bs_size_frm,
result->bs_size);
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
index 7bd1f437d8e3..50824e718c91 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.h
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
@@ -108,9 +108,11 @@ struct venc_frame_info {
/*
* struct venc_frm_buf - frame buffer information used in venc_if_encode()
* @fb_addr: plane frame buffer addresses
+ * @src_addr: vb2_v4l2_buffer src buffer address
*/
struct venc_frm_buf {
struct mtk_vcodec_fb fb_addr[MTK_VCODEC_MAX_PLANES];
+ void *src_addr;
};
/*
--
2.25.1
The frame_racing mode encoding is try to use the two venc cores:
frame#0 use core#0, frame#1 use core#1, frame#2 use core#0...,
Lock the device and enabe the clock by used core, for sequence
header encoding, it always used core#0.
Signed-off-by: Irui Wang <[email protected]>
---
.../platform/mtk-vcodec/mtk_vcodec_drv.h | 5 +-
.../platform/mtk-vcodec/mtk_vcodec_enc.c | 35 ++++----
.../platform/mtk-vcodec/mtk_vcodec_enc.h | 4 +-
.../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 6 +-
.../platform/mtk-vcodec/venc/venc_common_if.c | 6 +-
.../platform/mtk-vcodec/venc/venc_h264_if.c | 6 +-
.../platform/mtk-vcodec/venc/venc_vp8_if.c | 2 +-
.../media/platform/mtk-vcodec/venc_drv_if.c | 85 ++++++++++++++-----
.../media/platform/mtk-vcodec/venc_drv_if.h | 4 +
.../media/platform/mtk-vcodec/venc_vpu_if.c | 11 ++-
.../media/platform/mtk-vcodec/venc_vpu_if.h | 3 +-
11 files changed, 118 insertions(+), 49 deletions(-)
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index 1c6c1eca649f..7a9c2dd52b91 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -273,6 +273,7 @@ struct vdec_pic_info {
* @decoded_frame_cnt: number of decoded frames
* @lock: protect variables accessed by V4L2 threads and worker thread such as
* mtk_video_dec_buf.
+ * @enc_idx: used to record encoded frame count
*/
struct mtk_vcodec_ctx {
enum mtk_instance_type type;
@@ -313,6 +314,8 @@ struct mtk_vcodec_ctx {
int decoded_frame_cnt;
struct mutex lock;
+ int hw_id;
+ int enc_idx;
};
enum mtk_chip {
@@ -422,7 +425,7 @@ struct mtk_vcodec_dev {
int enc_irq;
struct mutex dec_mutex;
- struct mutex enc_mutex;
+ struct mutex enc_mutex[MTK_VENC_HW_MAX];
struct mtk_vcodec_pm pm;
unsigned int dec_capability;
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index d22f9c8fd063..6b37bd092fdf 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -1090,21 +1090,26 @@ static void mtk_venc_worker(struct work_struct *work)
if (enc_result.is_key_frm)
dst_buf->flags |= V4L2_BUF_FLAG_KEYFRAME;
- if (ret) {
- v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
- dst_buf->vb2_buf.planes[0].bytesused = 0;
- v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
- mtk_v4l2_err("venc_if_encode failed=%d", ret);
- } else {
- v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE);
- dst_buf->vb2_buf.planes[0].bytesused = enc_result.bs_size;
- v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE);
- mtk_v4l2_debug(2, "venc_if_encode bs size=%d",
- enc_result.bs_size);
+ if (ctx->dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE) {
+ if (ret) {
+ v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
+ dst_buf->vb2_buf.planes[0].bytesused = 0;
+ v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
+ mtk_v4l2_err("venc_if_encode failed=%d", ret);
+ } else {
+ v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE);
+ dst_buf->vb2_buf.planes[0].bytesused =
+ enc_result.bs_size;
+ v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE);
+ mtk_v4l2_debug(2, "venc_if_encode bs size=%d",
+ enc_result.bs_size);
+ }
}
v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
+ ctx->enc_idx++;
+
mtk_v4l2_debug(1, "<=== src_buf[%d] dst_buf[%d] venc_if_encode ret=%d Size=%u===>",
src_buf->vb2_buf.index, dst_buf->vb2_buf.index, ret,
enc_result.bs_size);
@@ -1305,19 +1310,19 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
return vb2_queue_init(dst_vq);
}
-int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx)
+int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int core_id)
{
struct mtk_vcodec_dev *dev = ctx->dev;
- mutex_unlock(&dev->enc_mutex);
+ mutex_unlock(&dev->enc_mutex[core_id]);
return 0;
}
-int mtk_venc_lock(struct mtk_vcodec_ctx *ctx)
+int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int core_id)
{
struct mtk_vcodec_dev *dev = ctx->dev;
- mutex_lock(&dev->enc_mutex);
+ mutex_lock(&dev->enc_mutex[core_id]);
return 0;
}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
index 30b6d7f22ed7..8d486f532025 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h
@@ -41,8 +41,8 @@ struct mtk_video_enc_buf {
extern const struct v4l2_ioctl_ops mtk_venc_ioctl_ops;
extern const struct v4l2_m2m_ops mtk_venc_m2m_ops;
-int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx);
-int mtk_venc_lock(struct mtk_vcodec_ctx *ctx);
+int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int core_id);
+int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int core_id);
int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
struct vb2_queue *dst_vq);
void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index b78435e8223a..ff1f05a5ca28 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -320,7 +320,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
struct resource *res;
phandle rproc_phandle;
enum mtk_vcodec_fw_type fw_type;
- int ret;
+ int ret, i;
struct component_match *match = NULL;
dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
@@ -384,7 +384,9 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
}
}
- mutex_init(&dev->enc_mutex);
+ for (i = 0; i < MTK_VENC_HW_MAX; i++)
+ mutex_init(&dev->enc_mutex[i]);
+
mutex_init(&dev->dev_mutex);
spin_lock_init(&dev->irqlock);
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
index 01b7d93241af..1580bd1beaac 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
@@ -300,7 +300,7 @@ static int venc_encode_sps(struct venc_common_inst *inst,
mtk_vcodec_debug_enter(inst);
ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_SPS, NULL,
- bs_buf, bs_size, NULL);
+ bs_buf, bs_size, NULL, MTK_VENC_CORE0);
if (ret)
return ret;
@@ -328,7 +328,7 @@ static int venc_encode_pps(struct venc_common_inst *inst,
mtk_vcodec_debug_enter(inst);
ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_PPS, NULL,
- bs_buf, bs_size, NULL);
+ bs_buf, bs_size, NULL, MTK_VENC_CORE0);
if (ret)
return ret;
@@ -386,7 +386,7 @@ static int venc_encode_frame(struct venc_common_inst *inst,
frame_info.frm_count, frame_info.skip_frm_count,
frame_info.frm_type);
ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_FRAME, frm_buf,
- bs_buf, bs_size, &frame_info);
+ bs_buf, bs_size, &frame_info, hw_id);
if (ret)
return ret;
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
index b6a4f2074fa5..a90b06a3fe5a 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
@@ -368,7 +368,7 @@ static int h264_encode_sps(struct venc_h264_inst *inst,
mtk_vcodec_debug_enter(inst);
ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_SPS, NULL,
- bs_buf, bs_size, NULL);
+ bs_buf, bs_size, NULL, MTK_VENC_CORE0);
if (ret)
return ret;
@@ -395,7 +395,7 @@ static int h264_encode_pps(struct venc_h264_inst *inst,
mtk_vcodec_debug_enter(inst);
ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_PPS, NULL,
- bs_buf, bs_size, NULL);
+ bs_buf, bs_size, NULL, MTK_VENC_CORE0);
if (ret)
return ret;
@@ -452,7 +452,7 @@ static int h264_encode_frame(struct venc_h264_inst *inst,
frame_info.frm_count, frame_info.skip_frm_count,
frame_info.frm_type);
ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_FRAME, frm_buf,
- bs_buf, bs_size, &frame_info);
+ bs_buf, bs_size, &frame_info, MTK_VENC_CORE0);
if (ret)
return ret;
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
index 8267a9c4fd25..a68bf53d7d81 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
@@ -303,7 +303,7 @@ static int vp8_enc_encode_frame(struct venc_vp8_inst *inst,
mtk_vcodec_debug(inst, "->frm_cnt=%d", inst->frm_cnt);
ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf, bs_size,
- NULL);
+ NULL, MTK_VENC_CORE0);
if (ret)
return ret;
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
index 93936d002bed..09fcec05a655 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
@@ -15,6 +15,7 @@
#include "mtk_vcodec_enc.h"
#include "mtk_vcodec_enc_pm.h"
+#include "mtk_vcodec_enc_hw.h"
int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
{
@@ -34,9 +35,9 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
return -EINVAL;
}
- mtk_venc_lock(ctx);
+ mtk_venc_lock(ctx, 0);
ret = ctx->enc_if->init(ctx);
- mtk_venc_unlock(ctx);
+ mtk_venc_unlock(ctx, 0);
return ret;
}
@@ -46,9 +47,9 @@ int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
{
int ret = 0;
- mtk_venc_lock(ctx);
+ mtk_venc_lock(ctx, 0);
ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
- mtk_venc_unlock(ctx);
+ mtk_venc_unlock(ctx, 0);
return ret;
}
@@ -59,24 +60,12 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx,
struct venc_done_result *result)
{
int ret = 0;
- unsigned long flags;
-
- mtk_venc_lock(ctx);
- spin_lock_irqsave(&ctx->dev->irqlock, flags);
- ctx->dev->curr_ctx = ctx;
- spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
-
- mtk_vcodec_enc_clock_on(ctx->dev, 0);
+ venc_encode_prepare(ctx, opt);
ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf,
bs_buf, result);
- mtk_vcodec_enc_clock_off(ctx->dev, 0);
-
- spin_lock_irqsave(&ctx->dev->irqlock, flags);
- ctx->dev->curr_ctx = NULL;
- spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+ venc_encode_unprepare(ctx, opt);
- mtk_venc_unlock(ctx);
return ret;
}
@@ -87,11 +76,67 @@ int venc_if_deinit(struct mtk_vcodec_ctx *ctx)
if (!ctx->drv_handle)
return 0;
- mtk_venc_lock(ctx);
+ mtk_venc_lock(ctx, 0);
ret = ctx->enc_if->deinit(ctx->drv_handle);
- mtk_venc_unlock(ctx);
+ mtk_venc_unlock(ctx, 0);
ctx->drv_handle = NULL;
return ret;
}
+
+void venc_encode_prepare(struct mtk_vcodec_ctx *ctx,
+ enum venc_start_opt opt)
+{
+ unsigned long flags;
+ struct mtk_venc_comp_dev *venc;
+
+ if (ctx->dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+ if (ctx->enc_idx & 0x01)
+ ctx->hw_id = MTK_VENC_CORE1;
+ else
+ ctx->hw_id = MTK_VENC_CORE0;
+ } else {
+ ctx->hw_id = MTK_VENC_CORE0;
+ }
+ mtk_venc_lock(ctx, ctx->hw_id);
+
+ spin_lock_irqsave(&ctx->dev->irqlock, flags);
+
+ if (ctx->dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+ venc = ctx->dev->enc_comp_dev[ctx->hw_id];
+
+ venc->curr_ctx = ctx;
+ } else {
+ ctx->dev->curr_ctx = ctx;
+ }
+
+ spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+
+ mtk_vcodec_enc_clock_on(ctx->dev, ctx->hw_id);
+}
+
+void venc_encode_unprepare(struct mtk_vcodec_ctx *ctx,
+ enum venc_start_opt opt)
+{
+ unsigned long flags;
+ struct mtk_venc_comp_dev *venc;
+
+ /*clock off and unlock after irq done*/
+ if (ctx->dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
+ if (opt == VENC_START_OPT_ENCODE_SEQUENCE_HEADER) {
+ mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id);
+ spin_lock_irqsave(&ctx->dev->irqlock, flags);
+ venc = ctx->dev->enc_comp_dev[ctx->hw_id];
+ venc->curr_ctx = NULL;
+ spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+ mtk_venc_unlock(ctx, ctx->hw_id);
+ }
+ } else {
+ mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id);
+ spin_lock_irqsave(&ctx->dev->irqlock, flags);
+ ctx->dev->curr_ctx = NULL;
+ spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
+ mtk_venc_unlock(ctx, ctx->hw_id);
+ }
+}
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
index 8f384ba24ddf..7bd1f437d8e3 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.h
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
@@ -168,4 +168,8 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx,
struct mtk_vcodec_mem *bs_buf,
struct venc_done_result *result);
+void venc_encode_prepare(struct mtk_vcodec_ctx *ctx,
+ enum venc_start_opt opt);
+void venc_encode_unprepare(struct mtk_vcodec_ctx *ctx,
+ enum venc_start_opt opt);
#endif /* _VENC_DRV_IF_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
index be6d8790a41e..5bee5f1828f1 100644
--- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
@@ -226,9 +226,12 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
struct venc_frm_buf *frm_buf,
struct mtk_vcodec_mem *bs_buf,
unsigned int *bs_size,
- struct venc_frame_info *frame_info)
+ struct venc_frame_info *frame_info,
+ int hw_id)
{
const bool is_ext = MTK_ENC_CTX_IS_EXT(vpu->ctx);
+ const int hw_mode = MTK_ENC_HW_MODE(vpu->ctx);
+
size_t msg_size = is_ext ?
sizeof(struct venc_ap_ipi_msg_enc_ext) :
sizeof(struct venc_ap_ipi_msg_enc);
@@ -262,6 +265,12 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
out.data[1] = frame_info->skip_frm_count;
out.data[2] = frame_info->frm_type;
}
+
+ if (hw_mode == VENC_FRAME_RACING_MODE) {
+ out.data_item = 4;
+ out.data[3] = hw_id;
+ }
+
if (vpu_enc_send_msg(vpu, &out, msg_size)) {
mtk_vcodec_err(vpu, "AP_IPIMSG_ENC_ENCODE %d fail",
bs_mode);
diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.h b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
index f9be9cab7ff7..e55656781f6e 100644
--- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
+++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
@@ -46,7 +46,8 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
struct venc_frm_buf *frm_buf,
struct mtk_vcodec_mem *bs_buf,
unsigned int *bs_size,
- struct venc_frame_info *frame_info);
+ struct venc_frame_info *frame_info,
+ int hw_id);
int vpu_enc_deinit(struct venc_vpu_inst *vpu);
#endif
--
2.25.1
On Mon, Aug 16, 2021 at 06:59:32PM +0800, Irui Wang wrote:
> The frame_racing mode encoding is try to use the two venc cores:
s/is try/tries/
> frame#0 use core#0, frame#1 use core#1, frame#2 use core#0...,
s/use/uses/g
> Lock the device and enabe the clock by used core, for sequence
s/enabe/enable/
> header encoding, it always used core#0.
s/used/uses/
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> @@ -273,6 +273,7 @@ struct vdec_pic_info {
> * @decoded_frame_cnt: number of decoded frames
> * @lock: protect variables accessed by V4L2 threads and worker thread such as
> * mtk_video_dec_buf.
> + * @enc_idx: used to record encoded frame count
> */
> struct mtk_vcodec_ctx {
> enum mtk_instance_type type;
> @@ -313,6 +314,8 @@ struct mtk_vcodec_ctx {
> int decoded_frame_cnt;
> struct mutex lock;
>
> + int hw_id;
> + int enc_idx;
hw_id lacks of kerneldoc which could introduce smatch warning.
> --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> @@ -15,6 +15,7 @@
>
> #include "mtk_vcodec_enc.h"
> #include "mtk_vcodec_enc_pm.h"
> +#include "mtk_vcodec_enc_hw.h"
Please try to maintain the order.
> @@ -34,9 +35,9 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
> return -EINVAL;
> }
>
> - mtk_venc_lock(ctx);
> + mtk_venc_lock(ctx, 0);
Does it make more sense to use ctx->hw_id instead 0 (even if it is always 0 in the path)?
> ret = ctx->enc_if->init(ctx);
> - mtk_venc_unlock(ctx);
> + mtk_venc_unlock(ctx, 0);
Same.
> @@ -46,9 +47,9 @@ int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
> {
> int ret = 0;
>
> - mtk_venc_lock(ctx);
> + mtk_venc_lock(ctx, 0);
Same.
> ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
> - mtk_venc_unlock(ctx);
> + mtk_venc_unlock(ctx, 0);
Same.
> @@ -87,11 +76,67 @@ int venc_if_deinit(struct mtk_vcodec_ctx *ctx)
> if (!ctx->drv_handle)
> return 0;
>
> - mtk_venc_lock(ctx);
> + mtk_venc_lock(ctx, 0);
Same.
> ret = ctx->enc_if->deinit(ctx->drv_handle);
> - mtk_venc_unlock(ctx);
> + mtk_venc_unlock(ctx, 0);
Same.
> +void venc_encode_unprepare(struct mtk_vcodec_ctx *ctx,
> + enum venc_start_opt opt)
> +{
> + unsigned long flags;
> + struct mtk_venc_comp_dev *venc;
> +
> + /*clock off and unlock after irq done*/
> + if (ctx->dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) {
> + if (opt == VENC_START_OPT_ENCODE_SEQUENCE_HEADER) {
> + mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id);
> + spin_lock_irqsave(&ctx->dev->irqlock, flags);
> + venc = ctx->dev->enc_comp_dev[ctx->hw_id];
> + venc->curr_ctx = NULL;
> + spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
> + mtk_venc_unlock(ctx, ctx->hw_id);
> + }
> + } else {
> + mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id);
> + spin_lock_irqsave(&ctx->dev->irqlock, flags);
> + ctx->dev->curr_ctx = NULL;
> + spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
> + mtk_venc_unlock(ctx, ctx->hw_id);
The few statements are identical. Should try to reuse them.
Il 16/08/21 12:59, Irui Wang ha scritto:
> MT8195 has two H264 encoder cores, they have their own power-domains,
> clocks, interrupts, register base. The two H264 encoder cores can work
> together to achieve higher performance.
>
> This series of patches is to use enable two h264 encoder cores.
> path[1..2]: use linux component framework to manage encoder hardware,
> user call "mt8195-vcodec-enc" driver can get the encoder master device,
> the encoding work is done by the two encoder core device. The hw_mode
> variable is added to distinguish from old platform, two encoder cores
> called "FRAME_RACING_MODE".
>
> The hardware mode of two encoder cores work together(overlap, another
> word called) on MT8195 called "frame_racing_mode", the two encoder
> power-domains should be power on together while encoding, the encoding
> process look like this:
>
> VENC Core0 frm#0....frm#2....frm#4
> VENC Core1 .frm#1....frm#3....frm#5
>
> patch[3..5]: due to the component device, the master device has no
> power-domains/clocks properties in dtsi, so the power/clock init function
> can't use for "frame_racing_mode" device in master device probe process,
> it should be called in component device probe process. Power on the
> hardware power and clock on demand.
>
> patch[6]: "frame_racing_mode" encoding need a new set of memory buffer
> for two encoder cores. For compatibility, we should new a encoder driver
> interface.
>
> patch[7..9]: add "frame_racing_mode" encoding process:
> As-Is: Synchronous
> VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder IRQ -->
> encode done with result --> job_finish
>
> VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder IRQ -->
> encode done with result --> job_finish
> ...
>
> To-Be: Asynchronous
> VIDIOC_QBUF#0 --> device_run(triger encoder core0) --> job_finish
> ..VIDIOC_QBUF#1 --> device_run(triger encoder core1) --> job_finish
> (core0 may encode done here, return encode result to client)
> VIDIOC_QBUF#2 --> device_run(triger encoder core0) --> job_finish
>
> Thers is no "wait encoder IRQ" synchronous call during "frame_racing_mode"
> encoding process, which can full use the two encoder cores to achieve
> higher performance.
>
> Irui Wang (9):
> dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores
> media: mtk-vcodec: Use component framework to manage encoder hardware
> media: mtk-vcodec: Rewrite venc power manage interface
> media: mtk-vcodec: Add venc power on/off interface
> media: mtk-vcodec: Rewrite venc clock interface
> media: mtk-vcodec: Add new venc drv interface for frame_racing mode
> media: mtk-vcodec: Add frame racing mode encode process
> media: mtk-vcodec: Return encode result to client
> media: mtk-vcodec: Add delayed worker for encode timeout
>
> .../bindings/media/mediatek-vcodec.txt | 2 +
> drivers/media/platform/mtk-vcodec/Makefile | 2 +
> .../platform/mtk-vcodec/mtk_vcodec_drv.h | 34 +-
> .../platform/mtk-vcodec/mtk_vcodec_enc.c | 120 +++-
> .../platform/mtk-vcodec/mtk_vcodec_enc.h | 10 +-
> .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 204 +++++-
> .../platform/mtk-vcodec/mtk_vcodec_enc_hw.c | 253 +++++++
> .../platform/mtk-vcodec/mtk_vcodec_enc_hw.h | 38 +
> .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 213 ++++--
> .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h | 13 +-
> .../platform/mtk-vcodec/mtk_vcodec_util.c | 19 +
> .../platform/mtk-vcodec/mtk_vcodec_util.h | 5 +
> .../platform/mtk-vcodec/venc/venc_common_if.c | 675 ++++++++++++++++++
> .../platform/mtk-vcodec/venc/venc_h264_if.c | 6 +-
> .../platform/mtk-vcodec/venc/venc_vp8_if.c | 2 +-
> .../media/platform/mtk-vcodec/venc_drv_if.c | 96 ++-
> .../media/platform/mtk-vcodec/venc_drv_if.h | 7 +
> .../media/platform/mtk-vcodec/venc_vpu_if.c | 11 +-
> .../media/platform/mtk-vcodec/venc_vpu_if.h | 3 +-
> 19 files changed, 1564 insertions(+), 149 deletions(-)
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
> create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
> create mode 100644 drivers/media/platform/mtk-vcodec/venc/venc_common_if.c
>
Hello Irui,
we have some interest in this series; can you please rebase it over the latest
changes that were sent in series [1] [2]?
Also, please follow [2], where of_platform_populate was used in place of the
component framework (where applicable and where possible) on this series, as well.
Thanks!
- Angelo
[1]: https://patchwork.kernel.org/project/linux-mediatek/list/?series=579201
[2]: https://patchwork.kernel.org/project/linux-mediatek/list/?series=578467