LinuxLists.cc - [PATCH v2 00/34] Qualcomm video encoder and decoder driver

[permalink] [raw]

Subject: [PATCH v2 09/34] media: iris: initialize shared queues for host and firmware communication

Shared queues are used for communication between driver and firmware.
There are 3 types of queues:
Command queue - driver to write any command to firmware.
Message queue - firmware to send any response to driver.
Debug queue - firmware to write debug message.

Above queues are initialized and configured to firmware during probe.

Signed-off-by: Dikshita Agarwal <[email protected]>
---
drivers/media/platform/qcom/vcodec/iris/Makefile | 2 ++
.../media/platform/qcom/vcodec/iris/iris_core.h | 11 ++++++++
.../media/platform/qcom/vcodec/iris/iris_probe.c | 31 ++++++++++++++++++++++
3 files changed, 44 insertions(+)

diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
index 12a74de..59798e5d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/Makefile
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -1,3 +1,5 @@
+iris-objs += ../hfi_queue.o
+
iris-objs += iris_probe.o \
resources.o \
iris_state.o
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index 56a5b7d..77124f9 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -9,6 +9,7 @@
#include <linux/types.h>
#include <media/v4l2-device.h>

+#include "../hfi_queue.h"
#include "iris_state.h"

/**
@@ -30,6 +31,11 @@
* @reset_tbl: table of iris reset clocks
* @reset_count: count of iris reset clocks
* @state: current state of core
+ * @iface_q_table: Interface queue table memory
+ * @command_queue: shared interface queue to send commands to firmware
+ * @message_queue: shared interface queue to receive responses from firmware
+ * @debug_queue: shared interface queue to receive debug info from firmware
+ * @sfr: SFR register memory
*/

struct iris_core {
@@ -49,6 +55,11 @@ struct iris_core {
struct reset_info *reset_tbl;
u32 reset_count;
enum iris_core_state state;
+ struct mem_desc iface_q_table;
+ struct iface_q_info command_queue;
+ struct iface_q_info message_queue;
+ struct iface_q_info debug_queue;
+ struct mem_desc sfr;
};

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
index 7bb9c92..fd349a3 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -7,6 +7,7 @@
#include <linux/of_device.h>
#include <linux/platform_device.h>

+#include "../hfi_queue.h"
#include "iris_core.h"
#include "resources.h"

@@ -50,6 +51,10 @@ static void iris_remove(struct platform_device *pdev)
if (!core)
return;

+ hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr,
+ &core->command_queue, &core->message_queue,
+ &core->debug_queue);
+
video_unregister_device(core->vdev_dec);

v4l2_device_unregister(&core->v4l2_dev);
@@ -59,6 +64,7 @@ static int iris_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct iris_core *core;
+ u64 dma_mask;
int ret;

core = devm_kzalloc(&pdev->dev, sizeof(*core), GFP_KERNEL);
@@ -91,8 +97,33 @@ static int iris_probe(struct platform_device *pdev)

platform_set_drvdata(pdev, core);

+ /*
+ * Specify the max value of address space, which can be used
+ * for buffer transactions.
+ */
+ dma_mask = DMA_BIT_MASK(32);
+ dma_mask &= ~BIT(29);
+
+ ret = dma_set_mask_and_coherent(dev, dma_mask);
+ if (ret)
+ goto err_vdev_unreg;
+
+ dma_set_max_seg_size(&pdev->dev, (unsigned int)DMA_BIT_MASK(32));
+ dma_set_seg_boundary(&pdev->dev, (unsigned long)DMA_BIT_MASK(64));
+
+ ret = hfi_queue_init(core->dev, &core->iface_q_table, &core->sfr,
+ &core->command_queue, &core->message_queue,
+ &core->debug_queue, core);
+ if (ret) {
+ dev_err_probe(core->dev, ret,
+ "%s: interface queues init failed\n", __func__);
+ goto err_vdev_unreg;
+ }
+
return ret;

+err_vdev_unreg:
+ iris_unregister_video_device(core);
err_v4l2_unreg:
v4l2_device_unregister(&core->v4l2_dev);

--
2.7.4

2023-12-18 11:36:44

[permalink] [raw]

Subject: [PATCH v2 13/34] media: iris: introduce platform specific capabilities for core and instance

2023-12-18 11:37:13

[permalink] [raw]

Subject: [PATCH v2 11/34] media: iris: introduce packetization layer for creating HFI packets

Host firmware interface (HFI) is well defined set of interfaces
for communication between host driver and firmware.
The command and responses are exchanged in form of packets.
One or multiple packets are grouped under packet header.
Each packet has packet type which describes the specific HFI
and payload which holds the corresponding value for that HFI.

Sys_init is the first packets sent to firmware, which initializes
the firmware. Sys_image_version packet is to get the firmware
version string.

Signed-off-by: Dikshita Agarwal <[email protected]>
---
drivers/media/platform/qcom/vcodec/iris/Makefile | 1 +
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 15 +++
.../media/platform/qcom/vcodec/iris/iris_core.h | 10 ++
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 69 +++++++++++
.../platform/qcom/vcodec/iris/iris_hfi_packet.c | 129 +++++++++++++++++++++
.../platform/qcom/vcodec/iris/iris_hfi_packet.h | 71 ++++++++++++
.../media/platform/qcom/vcodec/iris/iris_probe.c | 5 +
7 files changed, 300 insertions(+)
create mode 100644 drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h

diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
index 74bd344..a2d5d74 100644
--- a/drivers/media/platform/qcom/vcodec/iris/Makefile
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -6,6 +6,7 @@ iris-objs += iris_probe.o \
iris_state.o \
iris_helpers.o \
iris_hfi.o \
+ iris_hfi_packet.o \
resources.o

obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o
diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
new file mode 100644
index 0000000..4c48c90
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _HFI_DEFINES_H_
+#define _HFI_DEFINES_H_
+
+#define HFI_VIDEO_ARCH_LX 0x1
+
+#define HFI_CMD_INIT 0x01000001
+
+#define HFI_PROP_IMAGE_VERSION 0x03000001
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index 2740ff1..de0cfef 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -39,6 +39,11 @@
* @sfr: SFR register memory
* @lock: a lock for this strucure
* @use_tz: a flag that suggests presence of trustzone
+ * @packet: pointer to packet from driver to fw
+ * @packet_size: size of packet
+ * @sys_init_id: id of sys init packet
+ * @header_id: id of packet header
+ * @packet_id: id of packet
*/

struct iris_core {
@@ -65,6 +70,11 @@ struct iris_core {
struct mem_desc sfr;
struct mutex lock; /* lock for core structure */
unsigned int use_tz;
+ u8 *packet;
+ u32 packet_size;
+ u32 sys_init_id;
+ u32 header_id;
+ u32 packet_id;
};

int iris_core_init(struct iris_core *core);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index 4f51a8c..fe16448 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -4,8 +4,67 @@
*/

#include "../firmware.h"
+#include "../hfi_queue.h"
#include "iris_helpers.h"
#include "iris_hfi.h"
+#include "iris_hfi_packet.h"
+
+static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt)
+{
+ struct iface_q_info *q_info;
+ struct hfi_header *header;
+ u32 packet_size, rx_req;
+ int ret;
+
+ ret = check_core_lock(core);
+ if (ret)
+ return ret;
+
+ if (!core_in_valid_state(core))
+ return -EINVAL;
+
+ q_info = &core->command_queue;
+ if (!q_info || !q_info->q_array.kernel_vaddr || !pkt) {
+ dev_err(core->dev, "cannot write to shared CMD Q's\n");
+ return -ENODATA;
+ }
+
+ header = pkt;
+ packet_size = header->size;
+
+ if (!write_queue(q_info, pkt, packet_size, &rx_req)) {
+ dev_err(core->dev, "queue full\n");
+ return -ENODATA;
+ }
+
+ return ret;
+}
+
+static int sys_init(struct iris_core *core)
+{
+ int ret;
+
+ ret = hfi_packet_sys_init(core, core->packet, core->packet_size);
+ if (ret)
+ return ret;
+
+ ret = iris_hfi_queue_cmd_write(core, core->packet);
+
+ return ret;
+}
+
+static int sys_image_version(struct iris_core *core)
+{
+ int ret;
+
+ ret = hfi_packet_image_version(core, core->packet, core->packet_size);
+ if (ret)
+ return ret;
+
+ ret = iris_hfi_queue_cmd_write(core, core->packet);
+
+ return ret;
+}

#define CP_START 0
#define CP_SIZE 0x25800000
@@ -46,6 +105,16 @@ int iris_hfi_core_init(struct iris_core *core)

ret = protect_secure_region(CP_START, CP_SIZE, CP_NONPIXEL_START,
CP_NONPIXEL_SIZE, IRIS_PAS_ID);
+ if (ret)
+ goto error;
+
+ ret = sys_init(core);
+ if (ret)
+ goto error;
+
+ ret = sys_image_version(core);
+ if (ret)
+ goto error;

return ret;

diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
new file mode 100644
index 0000000..73bba07
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
@@ -0,0 +1,129 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "iris_core.h"
+#include "iris_hfi_packet.h"
+#include "hfi_defines.h"
+
+static int hfi_create_header(u8 *packet, u32 packet_size, u32 session_id,
+ u32 header_id)
+{
+ struct hfi_header *hdr = (struct hfi_header *)packet;
+
+ if (!packet || packet_size < sizeof(*hdr))
+ return -EINVAL;
+
+ memset(hdr, 0, sizeof(*hdr));
+
+ hdr->size = sizeof(*hdr);
+ hdr->session_id = session_id;
+ hdr->header_id = header_id;
+ hdr->num_packets = 0;
+
+ return 0;
+}
+
+static int hfi_create_packet(u8 *packet, u32 packet_size, u32 pkt_type,
+ u32 pkt_flags, u32 payload_type, u32 port,
+ u32 packet_id, void *payload, u32 payload_size)
+{
+ struct hfi_header *hdr;
+ struct hfi_packet *pkt;
+ u32 pkt_size;
+
+ if (!packet)
+ return -EINVAL;
+
+ hdr = (struct hfi_header *)packet;
+ if (hdr->size < sizeof(*hdr))
+ return -EINVAL;
+
+ pkt = (struct hfi_packet *)(packet + hdr->size);
+ pkt_size = sizeof(*pkt) + payload_size;
+ if (packet_size < hdr->size + pkt_size)
+ return -EINVAL;
+
+ memset(pkt, 0, pkt_size);
+ pkt->size = pkt_size;
+ pkt->type = pkt_type;
+ pkt->flags = pkt_flags;
+ pkt->payload_info = payload_type;
+ pkt->port = port;
+ pkt->packet_id = packet_id;
+ if (payload_size)
+ memcpy((u8 *)pkt + sizeof(*pkt),
+ payload, payload_size);
+
+ hdr->num_packets++;
+ hdr->size += pkt->size;
+
+ return 0;
+}
+
+int hfi_packet_sys_init(struct iris_core *core,
+ u8 *pkt, u32 pkt_size)
+{
+ u32 payload = 0;
+ int ret;
+
+ ret = hfi_create_header(pkt, pkt_size,
+ 0,
+ core->header_id++);
+ if (ret)
+ goto error;
+
+ payload = HFI_VIDEO_ARCH_LX;
+ core->sys_init_id = core->packet_id++;
+ ret = hfi_create_packet(pkt, pkt_size,
+ HFI_CMD_INIT,
+ (HFI_HOST_FLAGS_RESPONSE_REQUIRED |
+ HFI_HOST_FLAGS_INTR_REQUIRED |
+ HFI_HOST_FLAGS_NON_DISCARDABLE),
+ HFI_PAYLOAD_U32,
+ HFI_PORT_NONE,
+ core->sys_init_id,
+ &payload,
+ sizeof(u32));
+ if (ret)
+ goto error;
+
+ return ret;
+
+error:
+ dev_err(core->dev, "%s: create sys init packet failed\n", __func__);
+
+ return ret;
+}
+
+int hfi_packet_image_version(struct iris_core *core,
+ u8 *pkt, u32 pkt_size)
+{
+ int ret;
+
+ ret = hfi_create_header(pkt, pkt_size,
+ 0,
+ core->header_id++);
+ if (ret)
+ goto error;
+
+ ret = hfi_create_packet(pkt, pkt_size,
+ HFI_PROP_IMAGE_VERSION,
+ (HFI_HOST_FLAGS_RESPONSE_REQUIRED |
+ HFI_HOST_FLAGS_INTR_REQUIRED |
+ HFI_HOST_FLAGS_GET_PROPERTY),
+ HFI_PAYLOAD_NONE,
+ HFI_PORT_NONE,
+ core->packet_id++,
+ NULL, 0);
+ if (ret)
+ goto error;
+
+ return ret;
+
+error:
+ dev_err(core->dev, "%s: create image version packet failed\n", __func__);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
new file mode 100644
index 0000000..e36612c
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
@@ -0,0 +1,71 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_HFI_PACKET_H_
+#define _IRIS_HFI_PACKET_H_
+
+struct hfi_header {
+ u32 size;
+ u32 session_id;
+ u32 header_id;
+ u32 reserved[4];
+ u32 num_packets;
+};
+
+struct hfi_packet {
+ u32 size;
+ u32 type;
+ u32 flags;
+ u32 payload_info;
+ u32 port;
+ u32 packet_id;
+ u32 reserved[2];
+};
+
+enum hfi_packet_host_flags {
+ HFI_HOST_FLAGS_NONE = 0x00000000,
+ HFI_HOST_FLAGS_INTR_REQUIRED = 0x00000001,
+ HFI_HOST_FLAGS_RESPONSE_REQUIRED = 0x00000002,
+ HFI_HOST_FLAGS_NON_DISCARDABLE = 0x00000004,
+ HFI_HOST_FLAGS_GET_PROPERTY = 0x00000008,
+};
+
+enum hfi_packet_firmware_flags {
+ HFI_FW_FLAGS_NONE = 0x00000000,
+ HFI_FW_FLAGS_SUCCESS = 0x00000001,
+ HFI_FW_FLAGS_INFORMATION = 0x00000002,
+ HFI_FW_FLAGS_SESSION_ERROR = 0x00000004,
+ HFI_FW_FLAGS_SYSTEM_ERROR = 0x00000008,
+};
+
+enum hfi_packet_payload_info {
+ HFI_PAYLOAD_NONE = 0x00000000,
+ HFI_PAYLOAD_U32 = 0x00000001,
+ HFI_PAYLOAD_S32 = 0x00000002,
+ HFI_PAYLOAD_U64 = 0x00000003,
+ HFI_PAYLOAD_S64 = 0x00000004,
+ HFI_PAYLOAD_STRUCTURE = 0x00000005,
+ HFI_PAYLOAD_BLOB = 0x00000006,
+ HFI_PAYLOAD_STRING = 0x00000007,
+ HFI_PAYLOAD_Q16 = 0x00000008,
+ HFI_PAYLOAD_U32_ENUM = 0x00000009,
+ HFI_PAYLOAD_32_PACKED = 0x0000000a,
+ HFI_PAYLOAD_U32_ARRAY = 0x0000000b,
+ HFI_PAYLOAD_S32_ARRAY = 0x0000000c,
+ HFI_PAYLOAD_64_PACKED = 0x0000000d,
+};
+
+enum hfi_packet_port_type {
+ HFI_PORT_NONE = 0x00000000,
+ HFI_PORT_BITSTREAM = 0x00000001,
+ HFI_PORT_RAW = 0x00000002,
+};
+
+int hfi_packet_sys_init(struct iris_core *core,
+ u8 *pkt, u32 pkt_size);
+int hfi_packet_image_version(struct iris_core *core,
+ u8 *pkt, u32 pkt_size);
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
index f39b4aa..570c64e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -79,6 +79,11 @@ static int iris_probe(struct platform_device *pdev)
core->state = IRIS_CORE_DEINIT;
mutex_init(&core->lock);

+ core->packet_size = IFACEQ_CORE_PKT_SIZE;
+ core->packet = devm_kzalloc(core->dev, core->packet_size, GFP_KERNEL);
+ if (!core->packet)
+ return -ENOMEM;
+
core->reg_base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(core->reg_base))
return PTR_ERR(core->reg_base);
--
2.7.4

2023-12-18 11:37:20

[permalink] [raw]

Subject: [PATCH v2 01/34] media: introduce common helpers for video firmware handling

Re-organize the video driver code by introducing a new folder
'vcodec' and placing 'venus' driver code inside that.

Introduce common helpers for trustzone based firmware
load/unload etc. which are placed in common folder
i.e. 'vcodec'.
Use these helpers in 'venus' driver. These helpers will be
used by 'iris' driver as well which is introduced later
in this patch series.

Signed-off-by: Dikshita Agarwal <[email protected]>
---
drivers/media/platform/qcom/Kconfig | 2 +-
drivers/media/platform/qcom/Makefile | 2 +-
drivers/media/platform/qcom/vcodec/firmware.c | 147 +++++++++
drivers/media/platform/qcom/vcodec/firmware.h | 21 ++
.../media/platform/qcom/{ => vcodec}/venus/Kconfig | 0
.../platform/qcom/{ => vcodec}/venus/Makefile | 4 +-
.../media/platform/qcom/{ => vcodec}/venus/core.c | 102 +++++-
.../media/platform/qcom/{ => vcodec}/venus/core.h | 0
.../media/platform/qcom/{ => vcodec}/venus/dbgfs.c | 0
.../media/platform/qcom/{ => vcodec}/venus/dbgfs.h | 0
.../platform/qcom/vcodec/venus/firmware_no_tz.c | 194 +++++++++++
.../platform/qcom/vcodec/venus/firmware_no_tz.h | 19 ++
.../platform/qcom/{ => vcodec}/venus/helpers.c | 0
.../platform/qcom/{ => vcodec}/venus/helpers.h | 0
.../media/platform/qcom/{ => vcodec}/venus/hfi.c | 0
.../media/platform/qcom/{ => vcodec}/venus/hfi.h | 0
.../platform/qcom/{ => vcodec}/venus/hfi_cmds.c | 0
.../platform/qcom/{ => vcodec}/venus/hfi_cmds.h | 0
.../platform/qcom/{ => vcodec}/venus/hfi_helper.h | 0
.../platform/qcom/{ => vcodec}/venus/hfi_msgs.c | 0
.../platform/qcom/{ => vcodec}/venus/hfi_msgs.h | 0
.../platform/qcom/{ => vcodec}/venus/hfi_parser.c | 0
.../platform/qcom/{ => vcodec}/venus/hfi_parser.h | 0
.../qcom/{ => vcodec}/venus/hfi_plat_bufs.h | 0
.../qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c | 0
.../qcom/{ => vcodec}/venus/hfi_platform.c | 0
.../qcom/{ => vcodec}/venus/hfi_platform.h | 0
.../qcom/{ => vcodec}/venus/hfi_platform_v4.c | 0
.../qcom/{ => vcodec}/venus/hfi_platform_v6.c | 0
.../platform/qcom/{ => vcodec}/venus/hfi_venus.c | 21 +-
.../platform/qcom/{ => vcodec}/venus/hfi_venus.h | 0
.../qcom/{ => vcodec}/venus/hfi_venus_io.h | 0
.../platform/qcom/{ => vcodec}/venus/pm_helpers.c | 0
.../platform/qcom/{ => vcodec}/venus/pm_helpers.h | 0
.../media/platform/qcom/{ => vcodec}/venus/vdec.c | 0
.../media/platform/qcom/{ => vcodec}/venus/vdec.h | 0
.../platform/qcom/{ => vcodec}/venus/vdec_ctrls.c | 0
.../media/platform/qcom/{ => vcodec}/venus/venc.c | 0
.../media/platform/qcom/{ => vcodec}/venus/venc.h | 0
.../platform/qcom/{ => vcodec}/venus/venc_ctrls.c | 0
drivers/media/platform/qcom/venus/firmware.c | 363 ---------------------
drivers/media/platform/qcom/venus/firmware.h | 26 --
42 files changed, 492 insertions(+), 409 deletions(-)
create mode 100644 drivers/media/platform/qcom/vcodec/firmware.c
create mode 100644 drivers/media/platform/qcom/vcodec/firmware.h
rename drivers/media/platform/qcom/{ => vcodec}/venus/Kconfig (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/Makefile (83%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/core.c (91%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/core.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.h (100%)
create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_helper.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v4.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v6.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.c (99%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus_io.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec_ctrls.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.c (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.h (100%)
rename drivers/media/platform/qcom/{ => vcodec}/venus/venc_ctrls.c (100%)
delete mode 100644 drivers/media/platform/qcom/venus/firmware.c
delete mode 100644 drivers/media/platform/qcom/venus/firmware.h

diff --git a/drivers/media/platform/qcom/Kconfig b/drivers/media/platform/qcom/Kconfig
index cc5799b..e94142f 100644
--- a/drivers/media/platform/qcom/Kconfig
+++ b/drivers/media/platform/qcom/Kconfig
@@ -3,4 +3,4 @@
comment "Qualcomm media platform drivers"

source "drivers/media/platform/qcom/camss/Kconfig"
-source "drivers/media/platform/qcom/venus/Kconfig"
+source "drivers/media/platform/qcom/vcodec/venus/Kconfig"
diff --git a/drivers/media/platform/qcom/Makefile b/drivers/media/platform/qcom/Makefile
index 4f055c3..3d2d82b 100644
--- a/drivers/media/platform/qcom/Makefile
+++ b/drivers/media/platform/qcom/Makefile
@@ -1,3 +1,3 @@
# SPDX-License-Identifier: GPL-2.0-only
obj-y += camss/
-obj-y += venus/
+obj-y += vcodec/venus/
diff --git a/drivers/media/platform/qcom/vcodec/firmware.c b/drivers/media/platform/qcom/vcodec/firmware.c
new file mode 100644
index 0000000..dbc220a
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/firmware.c
@@ -0,0 +1,147 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/firmware.h>
+#include <linux/kernel.h>
+#include <linux/iommu.h>
+#include <linux/of_device.h>
+#include <linux/firmware/qcom/qcom_scm.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/platform_device.h>
+#include <linux/soc/qcom/mdt_loader.h>
+
+#include "firmware.h"
+
+bool use_tz(struct device *core_dev)
+{
+ struct device_node *np;
+
+ np = of_get_child_by_name(core_dev->of_node, "video-firmware");
+ if (!np)
+ return true;
+
+ return false;
+}
+
+int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
+ u32 cp_nonpixel_size, u32 pas_id)
+{
+ int ret;
+ /*
+ * Clues for porting using downstream data:
+ * cp_start = 0
+ * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and not size!
+ * This works, as the non-secure context bank is placed
+ * contiguously right after the Content Protection region.
+ *
+ * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0]
+ * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1]
+ */
+ ret = qcom_scm_mem_protect_video_var(cp_start,
+ cp_size,
+ cp_nonpixel_start,
+ cp_nonpixel_size);
+ if (ret)
+ qcom_scm_pas_shutdown(pas_id);
+
+ return ret;
+}
+
+int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
+ size_t *mem_size, u32 pas_id, bool use_tz)
+{
+ const struct firmware *firmware = NULL;
+ struct reserved_mem *rmem;
+ struct device_node *node;
+ void *mem_virt = NULL;
+ ssize_t fw_size = 0;
+ int ret;
+
+ if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
+ (use_tz && !qcom_scm_is_available()))
+ return -EPROBE_DEFER;
+
+ if (!fw_name || !(*fw_name))
+ return -EINVAL;
+
+ *mem_phys = 0;
+ *mem_size = 0;
+
+ node = of_parse_phandle(dev->of_node, "memory-region", 0);
+ if (!node) {
+ dev_err(dev, "no memory-region specified\n");
+ return -EINVAL;
+ }
+
+ rmem = of_reserved_mem_lookup(node);
+ of_node_put(node);
+ if (!rmem) {
+ dev_err(dev, "failed to lookup reserved memory-region\n");
+ return -EINVAL;
+ }
+
+ ret = request_firmware(&firmware, fw_name, dev);
+ if (ret) {
+ dev_err(dev, "%s: failed to request fw \"%s\", error %d\n",
+ __func__, fw_name, ret);
+ return ret;
+ }
+
+ fw_size = qcom_mdt_get_size(firmware);
+ if (fw_size < 0) {
+ ret = fw_size;
+ dev_err(dev, "%s: out of bound fw image fw size: %ld\n",
+ __func__, fw_size);
+ goto err_release_fw;
+ }
+
+ *mem_phys = rmem->base;
+ *mem_size = rmem->size;
+
+ if (*mem_size < fw_size) {
+ ret = -EINVAL;
+ goto err_release_fw;
+ }
+
+ mem_virt = memremap(*mem_phys, *mem_size, MEMREMAP_WC);
+ if (!mem_virt) {
+ dev_err(dev, "unable to remap fw memory region %pa size %#zx\n",
+ mem_phys, *mem_size);
+ goto err_release_fw;
+ }
+
+ if (use_tz)
+ ret = qcom_mdt_load(dev, firmware, fw_name, pas_id, mem_virt,
+ *mem_phys, *mem_size, NULL);
+ else
+ ret = qcom_mdt_load_no_init(dev, firmware, fw_name, pas_id, mem_virt,
+ *mem_phys, *mem_size, NULL);
+ if (ret) {
+ dev_err(dev, "%s: error %d loading fw \"%s\"\n",
+ __func__, ret, fw_name);
+ }
+
+ memunmap(mem_virt);
+err_release_fw:
+ release_firmware(firmware);
+ return ret;
+}
+
+int auth_reset_fw(u32 pas_id)
+{
+ return qcom_scm_pas_auth_and_reset(pas_id);
+}
+
+void unload_fw(u32 pas_id)
+{
+ qcom_scm_pas_shutdown(pas_id);
+}
+
+int set_hw_state(bool resume)
+{
+ return qcom_scm_set_remote_state(resume, 0);
+}
diff --git a/drivers/media/platform/qcom/vcodec/firmware.h b/drivers/media/platform/qcom/vcodec/firmware.h
new file mode 100644
index 0000000..7d410a8
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/firmware.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _FIRMWARE_H_
+#define _FIRMWARE_H_
+
+#include <linux/device.h>
+#include <linux/types.h>
+
+bool use_tz(struct device *core_dev);
+int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
+ size_t *mem_size, u32 pas_id, bool use_tz);
+int auth_reset_fw(u32 pas_id);
+int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
+ u32 cp_nonpixel_size, u32 pas_id);
+void unload_fw(u32 pas_id);
+int set_hw_state(bool resume);
+
+#endif
diff --git a/drivers/media/platform/qcom/venus/Kconfig b/drivers/media/platform/qcom/vcodec/venus/Kconfig
similarity index 100%
rename from drivers/media/platform/qcom/venus/Kconfig
rename to drivers/media/platform/qcom/vcodec/venus/Kconfig
diff --git a/drivers/media/platform/qcom/venus/Makefile b/drivers/media/platform/qcom/vcodec/venus/Makefile
similarity index 83%
rename from drivers/media/platform/qcom/venus/Makefile
rename to drivers/media/platform/qcom/vcodec/venus/Makefile
index 91ee6be..f6f3a88 100644
--- a/drivers/media/platform/qcom/venus/Makefile
+++ b/drivers/media/platform/qcom/vcodec/venus/Makefile
@@ -1,7 +1,9 @@
# SPDX-License-Identifier: GPL-2.0
# Makefile for Qualcomm Venus driver

-venus-core-objs += core.o helpers.o firmware.o \
+venus-core-objs += ../firmware.o
+
+venus-core-objs += core.o helpers.o firmware_no_tz.o \
hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \
hfi_parser.o pm_helpers.o dbgfs.o \
hfi_platform.o hfi_platform_v4.o \
diff --git a/drivers/media/platform/qcom/venus/core.c b/drivers/media/platform/qcom/vcodec/venus/core.c
similarity index 91%
rename from drivers/media/platform/qcom/venus/core.c
rename to drivers/media/platform/qcom/vcodec/venus/core.c
index 9cffe97..56d9a53 100644
--- a/drivers/media/platform/qcom/venus/core.c
+++ b/drivers/media/platform/qcom/vcodec/venus/core.c
@@ -22,7 +22,8 @@
#include <media/v4l2-ioctl.h>

#include "core.h"
-#include "firmware.h"
+#include "../firmware.h"
+#include "firmware_no_tz.h"
#include "pm_helpers.h"
#include "hfi_venus_io.h"

@@ -86,6 +87,8 @@ static void venus_sys_error_handler(struct work_struct *work)
struct venus_core *core =
container_of(work, struct venus_core, work.work);
int ret, i, max_attempts = RPM_WAIT_FOR_IDLE_MAX_ATTEMPTS;
+ const struct venus_resources *res = core->res;
+ const char *fwpath = NULL;
const char *err_msg = "";
bool failed = false;

@@ -107,7 +110,10 @@ static void venus_sys_error_handler(struct work_struct *work)

mutex_lock(&core->lock);

- venus_shutdown(core);
+ if (core->use_tz)
+ unload_fw(VENUS_PAS_ID);
+ else
+ unload_fw_no_tz(core);

venus_coredump(core);

@@ -127,12 +133,39 @@ static void venus_sys_error_handler(struct work_struct *work)
failed = true;
}

- ret = venus_boot(core);
+ ret = of_property_read_string_index(core->dev->of_node, "firmware-name", 0,
+ &fwpath);
+ if (ret)
+ fwpath = core->res->fwname;
+
+ ret = load_fw(core->dev, fwpath, &core->fw.mem_phys, &core->fw.mem_size,
+ VENUS_PAS_ID, core->use_tz);
if (ret && !failed) {
- err_msg = "boot Venus";
+ err_msg = "load FW";
failed = true;
}

+ if (core->use_tz)
+ ret = auth_reset_fw(VENUS_PAS_ID);
+ else
+ ret = auth_reset_fw_no_tz(core, core->fw.mem_phys, core->fw.mem_size);
+ if (ret && !failed) {
+ err_msg = "Auth and Reset";
+ failed = true;
+ }
+
+ if (core->use_tz && res->cp_size) {
+ ret = protect_secure_region(res->cp_start,
+ res->cp_size,
+ res->cp_nonpixel_start,
+ res->cp_nonpixel_size,
+ VENUS_PAS_ID);
+ if (ret && !failed) {
+ err_msg = "Protect CP Mem";
+ failed = true;
+ }
+ }
+
ret = hfi_core_resume(core, true);
if (ret && !failed) {
err_msg = "resume HFI";
@@ -281,7 +314,9 @@ static irqreturn_t venus_isr_thread(int irq, void *dev_id)

static int venus_probe(struct platform_device *pdev)
{
+ const struct venus_resources *res;
struct device *dev = &pdev->dev;
+ const char *fwpath = NULL;
struct venus_core *core;
int ret;

@@ -362,14 +397,42 @@ static int venus_probe(struct platform_device *pdev)
if (ret)
goto err_runtime_disable;

- ret = venus_firmware_init(core);
+ core->use_tz = use_tz(core->dev);
+
+ if (!core->use_tz) {
+ ret = init_fw_no_tz(core);
+ if (ret)
+ goto err_of_depopulate;
+ }
+
+ ret = of_property_read_string_index(dev->of_node, "firmware-name", 0,
+ &fwpath);
if (ret)
- goto err_of_depopulate;
+ fwpath = core->res->fwname;

- ret = venus_boot(core);
+ ret = load_fw(core->dev, fwpath, &core->fw.mem_phys, &core->fw.mem_size,
+ VENUS_PAS_ID, core->use_tz);
if (ret)
goto err_firmware_deinit;

+ if (core->use_tz)
+ ret = auth_reset_fw(VENUS_PAS_ID);
+ else
+ ret = auth_reset_fw_no_tz(core, core->fw.mem_phys, core->fw.mem_size);
+ if (ret)
+ goto err_firmware_deinit;
+
+ res = core->res;
+ if (core->use_tz && res->cp_size) {
+ ret = protect_secure_region(res->cp_start,
+ res->cp_size,
+ res->cp_nonpixel_start,
+ res->cp_nonpixel_size,
+ VENUS_PAS_ID);
+ if (ret)
+ goto err_firmware_deinit;
+ }
+
ret = hfi_core_resume(core, true);
if (ret)
goto err_venus_shutdown;
@@ -399,9 +462,13 @@ static int venus_probe(struct platform_device *pdev)
err_dev_unregister:
v4l2_device_unregister(&core->v4l2_dev);
err_venus_shutdown:
- venus_shutdown(core);
+ if (core->use_tz)
+ unload_fw(VENUS_PAS_ID);
+ else
+ unload_fw_no_tz(core);
err_firmware_deinit:
- venus_firmware_deinit(core);
+ if (!core->use_tz)
+ deinit_fw_no_tz(core);
err_of_depopulate:
of_platform_depopulate(dev);
err_runtime_disable:
@@ -430,10 +497,15 @@ static void venus_remove(struct platform_device *pdev)
ret = hfi_core_deinit(core, true);
WARN_ON(ret);

- venus_shutdown(core);
+ if (core->use_tz)
+ unload_fw(VENUS_PAS_ID);
+ else
+ unload_fw_no_tz(core);
+
of_platform_depopulate(dev);

- venus_firmware_deinit(core);
+ if (!core->use_tz)
+ deinit_fw_no_tz(core);

pm_runtime_put_sync(dev);
pm_runtime_disable(dev);
@@ -455,8 +527,12 @@ static void venus_core_shutdown(struct platform_device *pdev)
struct venus_core *core = platform_get_drvdata(pdev);

pm_runtime_get_sync(core->dev);
- venus_shutdown(core);
- venus_firmware_deinit(core);
+ if (core->use_tz) {
+ unload_fw(VENUS_PAS_ID);
+ } else {
+ unload_fw_no_tz(core);
+ deinit_fw_no_tz(core);
+ }
pm_runtime_put_sync(core->dev);
}

diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/vcodec/venus/core.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/core.h
rename to drivers/media/platform/qcom/vcodec/venus/core.h
diff --git a/drivers/media/platform/qcom/venus/dbgfs.c b/drivers/media/platform/qcom/vcodec/venus/dbgfs.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/dbgfs.c
rename to drivers/media/platform/qcom/vcodec/venus/dbgfs.c
diff --git a/drivers/media/platform/qcom/venus/dbgfs.h b/drivers/media/platform/qcom/vcodec/venus/dbgfs.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/dbgfs.h
rename to drivers/media/platform/qcom/vcodec/venus/dbgfs.h
diff --git a/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
new file mode 100644
index 0000000..9dca6e23
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
@@ -0,0 +1,194 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2017 Linaro Ltd.
+ */
+
+#include <linux/iommu.h>
+#include <linux/of_device.h>
+#include "core.h"
+#include "firmware_no_tz.h"
+#include "hfi_venus_io.h"
+
+#define VENUS_FW_MEM_SIZE (6 * SZ_1M)
+#define VENUS_FW_START_ADDR 0x0
+
+int init_fw_no_tz(struct venus_core *core)
+{
+ struct platform_device_info info;
+ struct iommu_domain *iommu_dom;
+ struct platform_device *pdev;
+ struct device_node *np;
+ int ret;
+
+ np = of_get_child_by_name(core->dev->of_node, "video-firmware");
+
+ memset(&info, 0, sizeof(info));
+ info.fwnode = &np->fwnode;
+ info.parent = core->dev;
+ info.name = np->name;
+ info.dma_mask = DMA_BIT_MASK(32);
+
+ pdev = platform_device_register_full(&info);
+ if (IS_ERR(pdev)) {
+ of_node_put(np);
+ return PTR_ERR(pdev);
+ }
+
+ pdev->dev.of_node = np;
+
+ ret = of_dma_configure(&pdev->dev, np, true);
+ if (ret) {
+ dev_err(core->dev, "dma configure fail\n");
+ goto err_unregister;
+ }
+
+ core->fw.dev = &pdev->dev;
+
+ iommu_dom = iommu_domain_alloc(&platform_bus_type);
+ if (!iommu_dom) {
+ dev_err(core->fw.dev, "Failed to allocate iommu domain\n");
+ ret = -ENOMEM;
+ goto err_unregister;
+ }
+
+ ret = iommu_attach_device(iommu_dom, core->fw.dev);
+ if (ret) {
+ dev_err(core->fw.dev, "could not attach device\n");
+ goto err_iommu_free;
+ }
+
+ core->fw.iommu_domain = iommu_dom;
+
+ of_node_put(np);
+
+ return 0;
+
+err_iommu_free:
+ iommu_domain_free(iommu_dom);
+err_unregister:
+ platform_device_unregister(pdev);
+ of_node_put(np);
+ return ret;
+}
+
+void deinit_fw_no_tz(struct venus_core *core)
+{
+ struct iommu_domain *iommu;
+
+ if (!core->fw.dev)
+ return;
+
+ iommu = core->fw.iommu_domain;
+
+ iommu_detach_device(iommu, core->fw.dev);
+
+ if (iommu) {
+ iommu_domain_free(iommu);
+ iommu = NULL;
+ }
+
+ platform_device_unregister(to_platform_device(core->fw.dev));
+}
+
+static void reset_cpu_no_tz(struct venus_core *core)
+{
+ u32 fw_size = core->fw.mapped_mem_size;
+ void __iomem *wrapper_base;
+
+ if (IS_IRIS2_1(core))
+ wrapper_base = core->wrapper_tz_base;
+ else
+ wrapper_base = core->wrapper_base;
+
+ writel(0, wrapper_base + WRAPPER_FW_START_ADDR);
+ writel(fw_size, wrapper_base + WRAPPER_FW_END_ADDR);
+ writel(0, wrapper_base + WRAPPER_CPA_START_ADDR);
+ writel(fw_size, wrapper_base + WRAPPER_CPA_END_ADDR);
+ writel(fw_size, wrapper_base + WRAPPER_NONPIX_START_ADDR);
+ writel(fw_size, wrapper_base + WRAPPER_NONPIX_END_ADDR);
+
+ if (IS_IRIS2_1(core)) {
+ /* Bring XTSS out of reset */
+ writel(0, wrapper_base + WRAPPER_TZ_XTSS_SW_RESET);
+ } else {
+ writel(0x0, wrapper_base + WRAPPER_CPU_CGC_DIS);
+ writel(0x0, wrapper_base + WRAPPER_CPU_CLOCK_CONFIG);
+
+ /* Bring ARM9 out of reset */
+ writel(0, wrapper_base + WRAPPER_A9SS_SW_RESET);
+ }
+}
+
+void set_hw_state_no_tz(struct venus_core *core, bool resume)
+{
+ if (resume) {
+ reset_cpu_no_tz(core);
+ } else {
+ if (IS_IRIS2_1(core))
+ writel(WRAPPER_XTSS_SW_RESET_BIT,
+ core->wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
+ else
+ writel(WRAPPER_A9SS_SW_RESET_BIT,
+ core->wrapper_base + WRAPPER_A9SS_SW_RESET);
+ }
+}
+
+int auth_reset_fw_no_tz(struct venus_core *core, phys_addr_t mem_phys,
+ size_t mem_size)
+{
+ struct iommu_domain *iommu;
+ struct device *dev;
+ int ret;
+
+ dev = core->fw.dev;
+ if (!dev)
+ return -EPROBE_DEFER;
+
+ iommu = core->fw.iommu_domain;
+ core->fw.mapped_mem_size = mem_size;
+
+ ret = iommu_map(iommu, VENUS_FW_START_ADDR, mem_phys, mem_size,
+ IOMMU_READ | IOMMU_WRITE | IOMMU_PRIV, GFP_KERNEL);
+ if (ret) {
+ dev_err(dev, "could not map video firmware region\n");
+ return ret;
+ }
+
+ reset_cpu_no_tz(core);
+
+ return 0;
+}
+
+void unload_fw_no_tz(struct venus_core *core)
+{
+ const size_t mapped = core->fw.mapped_mem_size;
+ struct iommu_domain *iommu;
+ size_t unmapped;
+ u32 reg;
+ struct device *dev = core->fw.dev;
+ void __iomem *wrapper_base = core->wrapper_base;
+ void __iomem *wrapper_tz_base = core->wrapper_tz_base;
+
+ if (IS_IRIS2_1(core)) {
+ /* Assert the reset to XTSS */
+ reg = readl(wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
+ reg |= WRAPPER_XTSS_SW_RESET_BIT;
+ writel(reg, wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
+ } else {
+ /* Assert the reset to ARM9 */
+ reg = readl(wrapper_base + WRAPPER_A9SS_SW_RESET);
+ reg |= WRAPPER_A9SS_SW_RESET_BIT;
+ writel(reg, wrapper_base + WRAPPER_A9SS_SW_RESET);
+ }
+
+ iommu = core->fw.iommu_domain;
+
+ if (core->fw.mapped_mem_size && iommu) {
+ unmapped = iommu_unmap(iommu, VENUS_FW_START_ADDR, mapped);
+
+ if (unmapped != mapped)
+ dev_err(dev, "failed to unmap firmware\n");
+ else
+ core->fw.mapped_mem_size = 0;
+ }
+}
diff --git a/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
new file mode 100644
index 0000000..5f008ef
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2017 Linaro Ltd.
+ */
+#ifndef __FIRMWARE_NO_TZ_H__
+#define __FIRMWARE_NO_TZ_H__
+
+struct device;
+
+#define VENUS_PAS_ID 9
+
+int init_fw_no_tz(struct venus_core *core);
+void deinit_fw_no_tz(struct venus_core *core);
+int auth_reset_fw_no_tz(struct venus_core *core, phys_addr_t mem_phys,
+ size_t mem_size);
+void unload_fw_no_tz(struct venus_core *core);
+void set_hw_state_no_tz(struct venus_core *core, bool resume);
+
+#endif
diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/vcodec/venus/helpers.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/helpers.c
rename to drivers/media/platform/qcom/vcodec/venus/helpers.c
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/vcodec/venus/helpers.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/helpers.h
rename to drivers/media/platform/qcom/vcodec/venus/helpers.h
diff --git a/drivers/media/platform/qcom/venus/hfi.c b/drivers/media/platform/qcom/vcodec/venus/hfi.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi.c
rename to drivers/media/platform/qcom/vcodec/venus/hfi.c
diff --git a/drivers/media/platform/qcom/venus/hfi.h b/drivers/media/platform/qcom/vcodec/venus/hfi.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi.h
rename to drivers/media/platform/qcom/vcodec/venus/hfi.h
diff --git a/drivers/media/platform/qcom/venus/hfi_cmds.c b/drivers/media/platform/qcom/vcodec/venus/hfi_cmds.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_cmds.c
rename to drivers/media/platform/qcom/vcodec/venus/hfi_cmds.c
diff --git a/drivers/media/platform/qcom/venus/hfi_cmds.h b/drivers/media/platform/qcom/vcodec/venus/hfi_cmds.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_cmds.h
rename to drivers/media/platform/qcom/vcodec/venus/hfi_cmds.h
diff --git a/drivers/media/platform/qcom/venus/hfi_helper.h b/drivers/media/platform/qcom/vcodec/venus/hfi_helper.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_helper.h
rename to drivers/media/platform/qcom/vcodec/venus/hfi_helper.h
diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.c b/drivers/media/platform/qcom/vcodec/venus/hfi_msgs.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_msgs.c
rename to drivers/media/platform/qcom/vcodec/venus/hfi_msgs.c
diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.h b/drivers/media/platform/qcom/vcodec/venus/hfi_msgs.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_msgs.h
rename to drivers/media/platform/qcom/vcodec/venus/hfi_msgs.h
diff --git a/drivers/media/platform/qcom/venus/hfi_parser.c b/drivers/media/platform/qcom/vcodec/venus/hfi_parser.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_parser.c
rename to drivers/media/platform/qcom/vcodec/venus/hfi_parser.c
diff --git a/drivers/media/platform/qcom/venus/hfi_parser.h b/drivers/media/platform/qcom/vcodec/venus/hfi_parser.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_parser.h
rename to drivers/media/platform/qcom/vcodec/venus/hfi_parser.h
diff --git a/drivers/media/platform/qcom/venus/hfi_plat_bufs.h b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_plat_bufs.h
rename to drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
diff --git a/drivers/media/platform/qcom/venus/hfi_plat_bufs_v6.c b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_plat_bufs_v6.c
rename to drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
diff --git a/drivers/media/platform/qcom/venus/hfi_platform.c b/drivers/media/platform/qcom/vcodec/venus/hfi_platform.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_platform.c
rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform.c
diff --git a/drivers/media/platform/qcom/venus/hfi_platform.h b/drivers/media/platform/qcom/vcodec/venus/hfi_platform.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_platform.h
rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform.h
diff --git a/drivers/media/platform/qcom/venus/hfi_platform_v4.c b/drivers/media/platform/qcom/vcodec/venus/hfi_platform_v4.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_platform_v4.c
rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform_v4.c
diff --git a/drivers/media/platform/qcom/venus/hfi_platform_v6.c b/drivers/media/platform/qcom/vcodec/venus/hfi_platform_v6.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_platform_v6.c
rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform_v6.c
diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
similarity index 99%
rename from drivers/media/platform/qcom/venus/hfi_venus.c
rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
index f9437b6..5a68db9 100644
--- a/drivers/media/platform/qcom/venus/hfi_venus.c
+++ b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
@@ -13,11 +13,12 @@
#include <linux/slab.h>

#include "core.h"
+#include "../firmware.h"
#include "hfi_cmds.h"
#include "hfi_msgs.h"
#include "hfi_venus.h"
#include "hfi_venus_io.h"
-#include "firmware.h"
+#include "firmware_no_tz.h"

#define HFI_MASK_QHDR_TX_TYPE 0xff000000
#define HFI_MASK_QHDR_RX_TYPE 0x00ff0000
@@ -635,7 +636,10 @@ static int venus_power_off(struct venus_hfi_device *hdev)
if (!hdev->power_enabled)
return 0;

- ret = venus_set_hw_state_suspend(hdev->core);
+ if (hdev->core->use_tz)
+ ret = set_hw_state(false);
+ else
+ set_hw_state_no_tz(hdev->core, false);
if (ret)
return ret;

@@ -655,7 +659,13 @@ static int venus_power_on(struct venus_hfi_device *hdev)
if (hdev->power_enabled)
return 0;

- ret = venus_set_hw_state_resume(hdev->core);
+ if (hdev->core->use_tz) {
+ ret = set_hw_state(true);
+ if (ret == -EINVAL)
+ ret = 0;
+ } else {
+ set_hw_state_no_tz(hdev->core, true);
+ }
if (ret)
goto err;

@@ -668,7 +678,10 @@ static int venus_power_on(struct venus_hfi_device *hdev)
return 0;

err_suspend:
- venus_set_hw_state_suspend(hdev->core);
+ if (hdev->core->use_tz)
+ set_hw_state(false);
+ else
+ set_hw_state_no_tz(hdev->core, false);
err:
hdev->power_enabled = false;
return ret;
diff --git a/drivers/media/platform/qcom/venus/hfi_venus.h b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_venus.h
rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus.h
diff --git a/drivers/media/platform/qcom/venus/hfi_venus_io.h b/drivers/media/platform/qcom/vcodec/venus/hfi_venus_io.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/hfi_venus_io.h
rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus_io.h
diff --git a/drivers/media/platform/qcom/venus/pm_helpers.c b/drivers/media/platform/qcom/vcodec/venus/pm_helpers.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/pm_helpers.c
rename to drivers/media/platform/qcom/vcodec/venus/pm_helpers.c
diff --git a/drivers/media/platform/qcom/venus/pm_helpers.h b/drivers/media/platform/qcom/vcodec/venus/pm_helpers.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/pm_helpers.h
rename to drivers/media/platform/qcom/vcodec/venus/pm_helpers.h
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/vcodec/venus/vdec.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/vdec.c
rename to drivers/media/platform/qcom/vcodec/venus/vdec.c
diff --git a/drivers/media/platform/qcom/venus/vdec.h b/drivers/media/platform/qcom/vcodec/venus/vdec.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/vdec.h
rename to drivers/media/platform/qcom/vcodec/venus/vdec.h
diff --git a/drivers/media/platform/qcom/venus/vdec_ctrls.c b/drivers/media/platform/qcom/vcodec/venus/vdec_ctrls.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/vdec_ctrls.c
rename to drivers/media/platform/qcom/vcodec/venus/vdec_ctrls.c
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/vcodec/venus/venc.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/venc.c
rename to drivers/media/platform/qcom/vcodec/venus/venc.c
diff --git a/drivers/media/platform/qcom/venus/venc.h b/drivers/media/platform/qcom/vcodec/venus/venc.h
similarity index 100%
rename from drivers/media/platform/qcom/venus/venc.h
rename to drivers/media/platform/qcom/vcodec/venus/venc.h
diff --git a/drivers/media/platform/qcom/venus/venc_ctrls.c b/drivers/media/platform/qcom/vcodec/venus/venc_ctrls.c
similarity index 100%
rename from drivers/media/platform/qcom/venus/venc_ctrls.c
rename to drivers/media/platform/qcom/vcodec/venus/venc_ctrls.c
diff --git a/drivers/media/platform/qcom/venus/firmware.c b/drivers/media/platform/qcom/venus/firmware.c
deleted file mode 100644
index fe7da2b..0000000
--- a/drivers/media/platform/qcom/venus/firmware.c
+++ /dev/null
@@ -1,363 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * Copyright (C) 2017 Linaro Ltd.
- */
-
-#include <linux/device.h>
-#include <linux/firmware.h>
-#include <linux/kernel.h>
-#include <linux/iommu.h>
-#include <linux/io.h>
-#include <linux/of.h>
-#include <linux/of_address.h>
-#include <linux/of_reserved_mem.h>
-#include <linux/platform_device.h>
-#include <linux/of_device.h>
-#include <linux/firmware/qcom/qcom_scm.h>
-#include <linux/sizes.h>
-#include <linux/soc/qcom/mdt_loader.h>
-
-#include "core.h"
-#include "firmware.h"
-#include "hfi_venus_io.h"
-
-#define VENUS_PAS_ID 9
-#define VENUS_FW_MEM_SIZE (6 * SZ_1M)
-#define VENUS_FW_START_ADDR 0x0
-
-static void venus_reset_cpu(struct venus_core *core)
-{
- u32 fw_size = core->fw.mapped_mem_size;
- void __iomem *wrapper_base;
-
- if (IS_IRIS2_1(core))
- wrapper_base = core->wrapper_tz_base;
- else
- wrapper_base = core->wrapper_base;
-
- writel(0, wrapper_base + WRAPPER_FW_START_ADDR);
- writel(fw_size, wrapper_base + WRAPPER_FW_END_ADDR);
- writel(0, wrapper_base + WRAPPER_CPA_START_ADDR);
- writel(fw_size, wrapper_base + WRAPPER_CPA_END_ADDR);
- writel(fw_size, wrapper_base + WRAPPER_NONPIX_START_ADDR);
- writel(fw_size, wrapper_base + WRAPPER_NONPIX_END_ADDR);
-
- if (IS_IRIS2_1(core)) {
- /* Bring XTSS out of reset */
- writel(0, wrapper_base + WRAPPER_TZ_XTSS_SW_RESET);
- } else {
- writel(0x0, wrapper_base + WRAPPER_CPU_CGC_DIS);
- writel(0x0, wrapper_base + WRAPPER_CPU_CLOCK_CONFIG);
-
- /* Bring ARM9 out of reset */
- writel(0, wrapper_base + WRAPPER_A9SS_SW_RESET);
- }
-}
-
-int venus_set_hw_state(struct venus_core *core, bool resume)
-{
- int ret;
-
- if (core->use_tz) {
- ret = qcom_scm_set_remote_state(resume, 0);
- if (resume && ret == -EINVAL)
- ret = 0;
- return ret;
- }
-
- if (resume) {
- venus_reset_cpu(core);
- } else {
- if (IS_IRIS2_1(core))
- writel(WRAPPER_XTSS_SW_RESET_BIT,
- core->wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
- else
- writel(WRAPPER_A9SS_SW_RESET_BIT,
- core->wrapper_base + WRAPPER_A9SS_SW_RESET);
- }
-
- return 0;
-}
-
-static int venus_load_fw(struct venus_core *core, const char *fwname,
- phys_addr_t *mem_phys, size_t *mem_size)
-{
- const struct firmware *mdt;
- struct reserved_mem *rmem;
- struct device_node *node;
- struct device *dev;
- ssize_t fw_size;
- void *mem_va;
- int ret;
-
- *mem_phys = 0;
- *mem_size = 0;
-
- dev = core->dev;
- node = of_parse_phandle(dev->of_node, "memory-region", 0);
- if (!node) {
- dev_err(dev, "no memory-region specified\n");
- return -EINVAL;
- }
-
- rmem = of_reserved_mem_lookup(node);
- of_node_put(node);
- if (!rmem) {
- dev_err(dev, "failed to lookup reserved memory-region\n");
- return -EINVAL;
- }
-
- ret = request_firmware(&mdt, fwname, dev);
- if (ret < 0)
- return ret;
-
- fw_size = qcom_mdt_get_size(mdt);
- if (fw_size < 0) {
- ret = fw_size;
- goto err_release_fw;
- }
-
- *mem_phys = rmem->base;
- *mem_size = rmem->size;
-
- if (*mem_size < fw_size || fw_size > VENUS_FW_MEM_SIZE) {
- ret = -EINVAL;
- goto err_release_fw;
- }
-
- mem_va = memremap(*mem_phys, *mem_size, MEMREMAP_WC);
- if (!mem_va) {
- dev_err(dev, "unable to map memory region %pa size %#zx\n", mem_phys, *mem_size);
- ret = -ENOMEM;
- goto err_release_fw;
- }
-
- if (core->use_tz)
- ret = qcom_mdt_load(dev, mdt, fwname, VENUS_PAS_ID,
- mem_va, *mem_phys, *mem_size, NULL);
- else
- ret = qcom_mdt_load_no_init(dev, mdt, fwname, VENUS_PAS_ID,
- mem_va, *mem_phys, *mem_size, NULL);
-
- memunmap(mem_va);
-err_release_fw:
- release_firmware(mdt);
- return ret;
-}
-
-static int venus_boot_no_tz(struct venus_core *core, phys_addr_t mem_phys,
- size_t mem_size)
-{
- struct iommu_domain *iommu;
- struct device *dev;
- int ret;
-
- dev = core->fw.dev;
- if (!dev)
- return -EPROBE_DEFER;
-
- iommu = core->fw.iommu_domain;
- core->fw.mapped_mem_size = mem_size;
-
- ret = iommu_map(iommu, VENUS_FW_START_ADDR, mem_phys, mem_size,
- IOMMU_READ | IOMMU_WRITE | IOMMU_PRIV, GFP_KERNEL);
- if (ret) {
- dev_err(dev, "could not map video firmware region\n");
- return ret;
- }
-
- venus_reset_cpu(core);
-
- return 0;
-}
-
-static int venus_shutdown_no_tz(struct venus_core *core)
-{
- const size_t mapped = core->fw.mapped_mem_size;
- struct iommu_domain *iommu;
- size_t unmapped;
- u32 reg;
- struct device *dev = core->fw.dev;
- void __iomem *wrapper_base = core->wrapper_base;
- void __iomem *wrapper_tz_base = core->wrapper_tz_base;
-
- if (IS_IRIS2_1(core)) {
- /* Assert the reset to XTSS */
- reg = readl(wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
- reg |= WRAPPER_XTSS_SW_RESET_BIT;
- writel(reg, wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
- } else {
- /* Assert the reset to ARM9 */
- reg = readl(wrapper_base + WRAPPER_A9SS_SW_RESET);
- reg |= WRAPPER_A9SS_SW_RESET_BIT;
- writel(reg, wrapper_base + WRAPPER_A9SS_SW_RESET);
- }
-
- iommu = core->fw.iommu_domain;
-
- if (core->fw.mapped_mem_size && iommu) {
- unmapped = iommu_unmap(iommu, VENUS_FW_START_ADDR, mapped);
-
- if (unmapped != mapped)
- dev_err(dev, "failed to unmap firmware\n");
- else
- core->fw.mapped_mem_size = 0;
- }
-
- return 0;
-}
-
-int venus_boot(struct venus_core *core)
-{
- struct device *dev = core->dev;
- const struct venus_resources *res = core->res;
- const char *fwpath = NULL;
- phys_addr_t mem_phys;
- size_t mem_size;
- int ret;
-
- if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
- (core->use_tz && !qcom_scm_is_available()))
- return -EPROBE_DEFER;
-
- ret = of_property_read_string_index(dev->of_node, "firmware-name", 0,
- &fwpath);
- if (ret)
- fwpath = core->res->fwname;
-
- ret = venus_load_fw(core, fwpath, &mem_phys, &mem_size);
- if (ret) {
- dev_err(dev, "fail to load video firmware\n");
- return -EINVAL;
- }
-
- core->fw.mem_size = mem_size;
- core->fw.mem_phys = mem_phys;
-
- if (core->use_tz)
- ret = qcom_scm_pas_auth_and_reset(VENUS_PAS_ID);
- else
- ret = venus_boot_no_tz(core, mem_phys, mem_size);
-
- if (ret)
- return ret;
-
- if (core->use_tz && res->cp_size) {
- /*
- * Clues for porting using downstream data:
- * cp_start = 0
- * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and not size!
- * This works, as the non-secure context bank is placed
- * contiguously right after the Content Protection region.
- *
- * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0]
- * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1]
- */
- ret = qcom_scm_mem_protect_video_var(res->cp_start,
- res->cp_size,
- res->cp_nonpixel_start,
- res->cp_nonpixel_size);
- if (ret) {
- qcom_scm_pas_shutdown(VENUS_PAS_ID);
- dev_err(dev, "set virtual address ranges fail (%d)\n",
- ret);
- return ret;
- }
- }
-
- return 0;
-}
-
-int venus_shutdown(struct venus_core *core)
-{
- int ret;
-
- if (core->use_tz)
- ret = qcom_scm_pas_shutdown(VENUS_PAS_ID);
- else
- ret = venus_shutdown_no_tz(core);
-
- return ret;
-}
-
-int venus_firmware_init(struct venus_core *core)
-{
- struct platform_device_info info;
- struct iommu_domain *iommu_dom;
- struct platform_device *pdev;
- struct device_node *np;
- int ret;
-
- np = of_get_child_by_name(core->dev->of_node, "video-firmware");
- if (!np) {
- core->use_tz = true;
- return 0;
- }
-
- memset(&info, 0, sizeof(info));
- info.fwnode = &np->fwnode;
- info.parent = core->dev;
- info.name = np->name;
- info.dma_mask = DMA_BIT_MASK(32);
-
- pdev = platform_device_register_full(&info);
- if (IS_ERR(pdev)) {
- of_node_put(np);
- return PTR_ERR(pdev);
- }
-
- pdev->dev.of_node = np;
-
- ret = of_dma_configure(&pdev->dev, np, true);
- if (ret) {
- dev_err(core->dev, "dma configure fail\n");
- goto err_unregister;
- }
-
- core->fw.dev = &pdev->dev;
-
- iommu_dom = iommu_domain_alloc(&platform_bus_type);
- if (!iommu_dom) {
- dev_err(core->fw.dev, "Failed to allocate iommu domain\n");
- ret = -ENOMEM;
- goto err_unregister;
- }
-
- ret = iommu_attach_device(iommu_dom, core->fw.dev);
- if (ret) {
- dev_err(core->fw.dev, "could not attach device\n");
- goto err_iommu_free;
- }
-
- core->fw.iommu_domain = iommu_dom;
-
- of_node_put(np);
-
- return 0;
-
-err_iommu_free:
- iommu_domain_free(iommu_dom);
-err_unregister:
- platform_device_unregister(pdev);
- of_node_put(np);
- return ret;
-}
-
-void venus_firmware_deinit(struct venus_core *core)
-{
- struct iommu_domain *iommu;
-
- if (!core->fw.dev)
- return;
-
- iommu = core->fw.iommu_domain;
-
- iommu_detach_device(iommu, core->fw.dev);
-
- if (core->fw.iommu_domain) {
- iommu_domain_free(iommu);
- core->fw.iommu_domain = NULL;
- }
-
- platform_device_unregister(to_platform_device(core->fw.dev));
-}
diff --git a/drivers/media/platform/qcom/venus/firmware.h b/drivers/media/platform/qcom/venus/firmware.h
deleted file mode 100644
index aaccd84..0000000
--- a/drivers/media/platform/qcom/venus/firmware.h
+++ /dev/null
@@ -1,26 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-only */
-/*
- * Copyright (C) 2017 Linaro Ltd.
- */
-#ifndef __VENUS_FIRMWARE_H__
-#define __VENUS_FIRMWARE_H__
-
-struct device;
-
-int venus_firmware_init(struct venus_core *core);
-void venus_firmware_deinit(struct venus_core *core);
-int venus_boot(struct venus_core *core);
-int venus_shutdown(struct venus_core *core);
-int venus_set_hw_state(struct venus_core *core, bool suspend);
-
-static inline int venus_set_hw_state_suspend(struct venus_core *core)
-{
- return venus_set_hw_state(core, false);
-}
-
-static inline int venus_set_hw_state_resume(struct venus_core *core)
-{
- return venus_set_hw_state(core, true);
-}
-
-#endif
--
2.7.4

2023-12-18 11:38:38

[permalink] [raw]

Subject: [PATCH v2 06/34] media: iris: register video device to platform driver

Iris is a multi pipe based video acceleration hardware
block that offloads video stream encoding and decoding
from the application processor (AP). It supports H.264
and H.265 encoding and decoding, as well as VP9 decoding.
The AP communicates with hardware through a well defined
protocol which provides fine-grained and asynchronous
control over individual hardware features.

This patch introduces basic probe and remove functions.
It handles setting up a video device as well as registering
it with the V4L2 subsystem.

Signed-off-by: Dikshita Agarwal <[email protected]>
---
drivers/media/platform/qcom/Kconfig | 1 +
drivers/media/platform/qcom/Makefile | 1 +
drivers/media/platform/qcom/vcodec/iris/Kconfig | 13 +++
drivers/media/platform/qcom/vcodec/iris/Makefile | 3 +
.../media/platform/qcom/vcodec/iris/iris_core.h | 34 +++++++
.../media/platform/qcom/vcodec/iris/iris_probe.c | 111 +++++++++++++++++++++
6 files changed, 163 insertions(+)
create mode 100644 drivers/media/platform/qcom/vcodec/iris/Kconfig
create mode 100644 drivers/media/platform/qcom/vcodec/iris/Makefile
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_core.h
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_probe.c

diff --git a/drivers/media/platform/qcom/Kconfig b/drivers/media/platform/qcom/Kconfig
index e94142f..7c88837 100644
--- a/drivers/media/platform/qcom/Kconfig
+++ b/drivers/media/platform/qcom/Kconfig
@@ -4,3 +4,4 @@ comment "Qualcomm media platform drivers"

source "drivers/media/platform/qcom/camss/Kconfig"
source "drivers/media/platform/qcom/vcodec/venus/Kconfig"
+source "drivers/media/platform/qcom/vcodec/iris/Kconfig"
diff --git a/drivers/media/platform/qcom/Makefile b/drivers/media/platform/qcom/Makefile
index 3d2d82b..3c76969 100644
--- a/drivers/media/platform/qcom/Makefile
+++ b/drivers/media/platform/qcom/Makefile
@@ -1,3 +1,4 @@
# SPDX-License-Identifier: GPL-2.0-only
obj-y += camss/
obj-y += vcodec/venus/
+obj-y += vcodec/iris/
diff --git a/drivers/media/platform/qcom/vcodec/iris/Kconfig b/drivers/media/platform/qcom/vcodec/iris/Kconfig
new file mode 100644
index 0000000..850a5b4
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/Kconfig
@@ -0,0 +1,13 @@
+config VIDEO_QCOM_IRIS
+ tristate "Qualcomm Iris V4L2 encoder/decoder driver"
+ depends on VIDEO_DEV
+ depends on ARCH_QCOM
+ select QCOM_MDT_LOADER if ARCH_QCOM
+ select QCOM_SCM
+ select DMABUF_HEAPS
+ select DMABUF_HEAPS_SYSTEM
+ help
+ This is a V4L2 driver for Qualcomm Iris video accelerator
+ hardware. It accelerates encoding and decoding operations
+ on various Qualcomm SoCs.
+ To compile this driver as a module choose m here.
diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
new file mode 100644
index 0000000..5536ae0
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -0,0 +1,3 @@
+iris-objs += iris_probe.o
+
+obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
new file mode 100644
index 0000000..ab7fcee
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_CORE_H_
+#define _IRIS_CORE_H_
+
+#include <linux/types.h>
+#include <media/v4l2-device.h>
+
+/**
+ * struct iris_core - holds core parameters valid for all instances
+ *
+ * @dev: reference to device structure
+ * @reg_base: IO memory base address
+ * @irq: iris irq
+ * @v4l2_dev: a holder for v4l2 device structure
+ * @vdev_dec: iris video device structure for decoder
+ * @v4l2_file_ops: iris v4l2 file ops
+ * @v4l2_ioctl_ops: iris v4l2 ioctl ops
+ */
+
+struct iris_core {
+ struct device *dev;
+ void __iomem *reg_base;
+ int irq;
+ struct v4l2_device v4l2_dev;
+ struct video_device *vdev_dec;
+ const struct v4l2_file_operations *v4l2_file_ops;
+ const struct v4l2_ioctl_ops *v4l2_ioctl_ops;
+};
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
new file mode 100644
index 0000000..2e93118
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -0,0 +1,111 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+
+#include "iris_core.h"
+
+static int iris_register_video_device(struct iris_core *core)
+{
+ struct video_device *vdev;
+ int ret;
+
+ vdev = video_device_alloc();
+ if (!vdev)
+ return -ENOMEM;
+
+ strscpy(vdev->name, "qcom-iris-decoder", sizeof(vdev->name));
+ vdev->release = video_device_release;
+ vdev->fops = core->v4l2_file_ops;
+ vdev->ioctl_ops = core->v4l2_ioctl_ops;
+ vdev->vfl_dir = VFL_DIR_M2M;
+ vdev->v4l2_dev = &core->v4l2_dev;
+ vdev->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
+
+ ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
+ if (ret)
+ goto err_vdev_release;
+
+ core->vdev_dec = vdev;
+ video_set_drvdata(vdev, core);
+
+ return ret;
+
+err_vdev_release:
+ video_device_release(vdev);
+
+ return ret;
+}
+
+static void iris_remove(struct platform_device *pdev)
+{
+ struct iris_core *core;
+
+ core = platform_get_drvdata(pdev);
+ if (!core)
+ return;
+
+ video_unregister_device(core->vdev_dec);
+
+ v4l2_device_unregister(&core->v4l2_dev);
+}
+
+static int iris_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct iris_core *core;
+ int ret;
+
+ core = devm_kzalloc(&pdev->dev, sizeof(*core), GFP_KERNEL);
+ if (!core)
+ return -ENOMEM;
+ core->dev = dev;
+
+ core->reg_base = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(core->reg_base))
+ return PTR_ERR(core->reg_base);
+
+ core->irq = platform_get_irq(pdev, 0);
+ if (core->irq < 0)
+ return core->irq;
+
+ ret = v4l2_device_register(dev, &core->v4l2_dev);
+ if (ret)
+ return ret;
+
+ ret = iris_register_video_device(core);
+ if (ret)
+ goto err_v4l2_unreg;
+
+ platform_set_drvdata(pdev, core);
+
+ return ret;
+
+err_v4l2_unreg:
+ v4l2_device_unregister(&core->v4l2_dev);
+
+ return ret;
+}
+
+static const struct of_device_id iris_dt_match[] = {
+ { .compatible = "qcom,sm8550-iris", },
+ { },
+};
+MODULE_DEVICE_TABLE(of, iris_dt_match);
+
+static struct platform_driver qcom_iris_driver = {
+ .probe = iris_probe,
+ .remove_new = iris_remove,
+ .driver = {
+ .name = "qcom-iris",
+ .of_match_table = iris_dt_match,
+ },
+};
+
+module_platform_driver(qcom_iris_driver);
+MODULE_DESCRIPTION("Qualcomm Iris video driver");
+MODULE_LICENSE("GPL");
--
2.7.4

2023-12-18 11:38:54

[permalink] [raw]

Subject: [PATCH v2 26/34] media: iris: implement vb2 streaming ops on capture and output planes

During stream on, set some mandatory properties
to firmware to start a session. Subscribe for different
bitstream parameters to get notified for change in any of
the subscribed parameters. Set all v4l2 properties set
by client, to firmware prepared with the dependency graph.
Also, configure the hardware internal buffers required
for frame processing. Send HFI_CMD_START on capture and
output planes to start the processing on respective planes.
The instance state is changed accordingly.

During stream off, send HFI_CMD_STOP to firmware which is
a synchronous command. After the response is received from
firmware, the session is closed on firmware and
instance state is changed accordingly.

Signed-off-by: Dikshita Agarwal <[email protected]>
---
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 12 ++
.../media/platform/qcom/vcodec/iris/iris_buffer.c | 21 +++
.../media/platform/qcom/vcodec/iris/iris_buffer.h | 3 +
.../media/platform/qcom/vcodec/iris/iris_ctrls.c | 28 ++++
.../media/platform/qcom/vcodec/iris/iris_ctrls.h | 1 +
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 51 ++++++
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 1 +
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 173 +++++++++++++++++++--
drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 7 +
.../platform/qcom/vcodec/iris/iris_hfi_packet.c | 14 ++
.../platform/qcom/vcodec/iris/iris_hfi_packet.h | 2 +-
.../platform/qcom/vcodec/iris/iris_instance.h | 4 +
.../media/platform/qcom/vcodec/iris/iris_state.c | 38 +++++
.../media/platform/qcom/vcodec/iris/iris_state.h | 3 +
drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 78 ++++++++++
drivers/media/platform/qcom/vcodec/iris/iris_vb2.h | 2 +
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 138 +++++++++++++++-
.../media/platform/qcom/vcodec/iris/iris_vdec.h | 6 +-
.../media/platform/qcom/vcodec/iris/iris_vidc.c | 4 +
19 files changed, 566 insertions(+), 20 deletions(-)

diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index 0ef6bad..dfd1a4c 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -14,6 +14,8 @@
#define HFI_CMD_INIT 0x01000001
#define HFI_CMD_OPEN 0x01000003
#define HFI_CMD_CLOSE 0x01000004
+#define HFI_CMD_START 0x01000005
+#define HFI_CMD_STOP 0x01000006
#define HFI_CMD_BUFFER 0x01000009

#define HFI_PROP_IMAGE_VERSION 0x03000001
@@ -45,6 +47,16 @@ enum hfi_property_mode_type {

#define HFI_PROP_IMAGE_VERSION 0x03000001

+enum hfi_codec_type {
+ HFI_CODEC_DECODE_AVC = 1,
+ HFI_CODEC_ENCODE_AVC = 2,
+ HFI_CODEC_DECODE_HEVC = 3,
+ HFI_CODEC_ENCODE_HEVC = 4,
+ HFI_CODEC_DECODE_VP9 = 5,
+};
+
+#define HFI_PROP_CODEC 0x03000100
+
enum hfi_color_format {
HFI_COLOR_FMT_OPAQUE = 0,
HFI_COLOR_FMT_NV12 = 1,
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
index 1ee840e..ab25026 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
@@ -560,3 +560,24 @@ int iris_release_input_internal_buffers(struct iris_inst *inst)

return ret;
}
+
+int iris_alloc_and_queue_session_int_bufs(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type)
+{
+ int ret;
+
+ if (buffer_type != BUF_PERSIST)
+ return -EINVAL;
+
+ ret = iris_get_internal_buf_info(inst, buffer_type);
+ if (ret)
+ return ret;
+
+ ret = iris_create_internal_buffers(inst, buffer_type);
+ if (ret)
+ return ret;
+
+ ret = iris_queue_internal_buffers(inst, buffer_type);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
index bdef15f..8769c3d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
@@ -59,4 +59,7 @@ int iris_destroy_internal_buffers(struct iris_inst *inst,
u32 plane);
int iris_release_input_internal_buffers(struct iris_inst *inst);

+int iris_alloc_and_queue_session_int_bufs(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type);
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
index 28977e8..94fff74 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
@@ -113,6 +113,20 @@ static int adjust_cap(struct iris_inst *inst,
return cap->adjust(inst, ctrl);
}

+static int set_cap(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ struct plat_inst_cap *cap;
+
+ cap = &inst->cap[cap_id];
+ if (!inst->cap[cap_id].cap_id)
+ return 0;
+
+ if (!cap->set)
+ return 0;
+
+ return cap->set(inst, cap_id);
+}
+
static int adjust_dynamic_property(struct iris_inst *inst,
enum plat_inst_cap_type cap_id,
struct v4l2_ctrl *ctrl,
@@ -657,6 +671,20 @@ int set_pipe(struct iris_inst *inst,
&work_route, sizeof(u32));
}

+int set_v4l2_properties(struct iris_inst *inst)
+{
+ struct cap_entry *entry = NULL, *temp = NULL;
+ int ret = 0;
+
+ list_for_each_entry_safe(entry, temp, &inst->caps_list, list) {
+ ret = set_cap(inst, entry->cap_id);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
int adjust_v4l2_properties(struct iris_inst *inst)
{
struct cap_entry *entry = NULL, *temp = NULL;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
index 22ee6c4b..28ce767 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
@@ -28,6 +28,7 @@ int prepare_dependency_list(struct iris_inst *inst);
int iris_init_instance_caps(struct iris_core *core);
int iris_init_core_caps(struct iris_core *core);
int get_inst_capability(struct iris_inst *inst);
+int set_v4l2_properties(struct iris_inst *inst);
int adjust_v4l2_properties(struct iris_inst *inst);
int ctrls_init(struct iris_inst *inst, bool init);

diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index 4cad673..7868566 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -9,6 +9,7 @@
#include "iris_hfi.h"
#include "iris_hfi_packet.h"
#include "iris_instance.h"
+#include "iris_vidc.h"

int check_core_lock(struct iris_core *core)
{
@@ -377,3 +378,53 @@ struct iris_inst *to_instance(struct iris_core *core, u32 session_id)

return NULL;
}
+
+static int kill_session(struct iris_inst *inst)
+{
+ if (!inst->session_id)
+ return 0;
+
+ close_session(inst);
+ iris_inst_change_state(inst, IRIS_INST_ERROR);
+
+ return 0;
+}
+
+int session_streamoff(struct iris_inst *inst, u32 plane)
+{
+ enum signal_session_response signal_type;
+ u32 hw_response_timeout_val;
+ struct iris_core *core;
+ int ret;
+
+ ret = iris_hfi_stop(inst, plane);
+ if (ret)
+ goto error;
+
+ core = inst->core;
+ hw_response_timeout_val = core->cap[HW_RESPONSE_TIMEOUT].value;
+ mutex_unlock(&inst->lock);
+ ret = wait_for_completion_timeout(&inst->completions[signal_type],
+ msecs_to_jiffies(hw_response_timeout_val));
+ if (!ret) {
+ ret = -ETIMEDOUT;
+ process_inst_timeout(inst);
+ } else {
+ ret = 0;
+ }
+ mutex_lock(&inst->lock);
+
+ if (ret)
+ goto error;
+
+ ret = iris_inst_state_change_streamoff(inst, plane);
+ if (ret)
+ goto error;
+
+ return 0;
+
+error:
+ kill_session(inst);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
index cb22adf..47a017d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -34,6 +34,7 @@ bool is_split_mode_enabled(struct iris_inst *inst);
int signal_session_msg_receipt(struct iris_inst *inst,
enum signal_session_response cmd);
struct iris_inst *to_instance(struct iris_core *core, u32 session_id);
+int session_streamoff(struct iris_inst *inst, u32 plane);

u32 v4l2_codec_from_driver(struct iris_inst *inst, enum codec_type codec);
enum codec_type v4l2_codec_to_driver(struct iris_inst *inst, u32 v4l2_codec);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index d15ce5a..e0da3be 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -219,9 +219,7 @@ int iris_hfi_session_open(struct iris_inst *inst)
return ret;
}

-int iris_hfi_session_subscribe_mode(struct iris_inst *inst,
- u32 cmd, u32 plane, u32 payload_type,
- void *payload, u32 payload_size)
+int iris_hfi_session_close(struct iris_inst *inst)
{
struct iris_core *core;
int ret;
@@ -238,14 +236,126 @@ int iris_hfi_session_subscribe_mode(struct iris_inst *inst,
}

ret = hfi_packet_session_command(inst,
- cmd,
+ HFI_CMD_CLOSE,
+ (HFI_HOST_FLAGS_RESPONSE_REQUIRED |
+ HFI_HOST_FLAGS_INTR_REQUIRED |
+ HFI_HOST_FLAGS_NON_DISCARDABLE),
+ HFI_PORT_NONE,
+ inst->session_id,
+ HFI_PAYLOAD_NONE,
+ NULL,
+ 0);
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+int iris_hfi_session_set_codec(struct iris_inst *inst)
+{
+ struct iris_core *core;
+ int ret;
+ u32 codec;
+
+ if (!inst->packet)
+ return -EINVAL;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ codec = get_hfi_codec(inst);
+ ret = hfi_packet_session_property(inst,
+ HFI_PROP_CODEC,
+ HFI_HOST_FLAGS_NONE,
+ HFI_PORT_NONE,
+ HFI_PAYLOAD_U32_ENUM,
+ &codec,
+ sizeof(u32));
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+int iris_hfi_session_set_default_header(struct iris_inst *inst)
+{
+ struct iris_core *core;
+ u32 default_header = false;
+ int ret;
+
+ if (!inst->packet)
+ return -EINVAL;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ default_header = inst->cap[DEFAULT_HEADER].value;
+ ret = hfi_packet_session_property(inst,
+ HFI_PROP_DEC_DEFAULT_HEADER,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(INPUT_MPLANE),
+ HFI_PAYLOAD_U32,
+ &default_header,
+ sizeof(u32));
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+int iris_hfi_start(struct iris_inst *inst, u32 plane)
+{
+ struct iris_core *core;
+ int ret = 0;
+
+ if (!inst->packet)
+ return -EINVAL;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (plane != INPUT_MPLANE && plane != OUTPUT_MPLANE)
+ goto unlock;
+
+ ret = hfi_packet_session_command(inst,
+ HFI_CMD_START,
(HFI_HOST_FLAGS_RESPONSE_REQUIRED |
HFI_HOST_FLAGS_INTR_REQUIRED),
get_hfi_port(plane),
inst->session_id,
- payload_type,
- payload,
- payload_size);
+ HFI_PAYLOAD_NONE,
+ NULL,
+ 0);
if (ret)
goto unlock;

@@ -257,10 +367,10 @@ int iris_hfi_session_subscribe_mode(struct iris_inst *inst,
return ret;
}

-int iris_hfi_session_close(struct iris_inst *inst)
+int iris_hfi_stop(struct iris_inst *inst, u32 plane)
{
struct iris_core *core;
- int ret;
+ int ret = 0;

if (!inst->packet)
return -EINVAL;
@@ -273,12 +383,15 @@ int iris_hfi_session_close(struct iris_inst *inst)
goto unlock;
}

+ if (plane != INPUT_MPLANE && plane != OUTPUT_MPLANE)
+ goto unlock;
+
ret = hfi_packet_session_command(inst,
- HFI_CMD_CLOSE,
+ HFI_CMD_STOP,
(HFI_HOST_FLAGS_RESPONSE_REQUIRED |
HFI_HOST_FLAGS_INTR_REQUIRED |
HFI_HOST_FLAGS_NON_DISCARDABLE),
- HFI_PORT_NONE,
+ get_hfi_port(plane),
inst->session_id,
HFI_PAYLOAD_NONE,
NULL,
@@ -294,6 +407,44 @@ int iris_hfi_session_close(struct iris_inst *inst)
return ret;
}

+int iris_hfi_session_subscribe_mode(struct iris_inst *inst,
+ u32 cmd, u32 plane, u32 payload_type,
+ void *payload, u32 payload_size)
+{
+ struct iris_core *core;
+ int ret;
+
+ if (!inst->packet)
+ return -EINVAL;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ ret = hfi_packet_session_command(inst,
+ cmd,
+ (HFI_HOST_FLAGS_RESPONSE_REQUIRED |
+ HFI_HOST_FLAGS_INTR_REQUIRED),
+ get_hfi_port(plane),
+ inst->session_id,
+ payload_type,
+ payload,
+ payload_size);
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
irqreturn_t iris_hfi_isr(int irq, void *data)
{
disable_irq_nosync(irq);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
index d6b3fca..f054c2d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
@@ -19,6 +19,13 @@ int iris_hfi_session_subscribe_mode(struct iris_inst *inst,
int iris_hfi_set_property(struct iris_inst *inst,
u32 packet_type, u32 flag, u32 plane, u32 payload_type,
void *payload, u32 payload_size);
+
+int iris_hfi_session_set_codec(struct iris_inst *inst);
+int iris_hfi_session_set_default_header(struct iris_inst *inst);
+
+int iris_hfi_start(struct iris_inst *inst, u32 plane);
+int iris_hfi_stop(struct iris_inst *inst, u32 plane);
+
int iris_hfi_queue_buffer(struct iris_inst *inst,
struct iris_buffer *buffer);
int iris_hfi_release_buffer(struct iris_inst *inst,
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
index dc7157d..b8785a9 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
@@ -76,6 +76,20 @@ static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type)
}
}

+u32 get_hfi_codec(struct iris_inst *inst)
+{
+ switch (inst->codec) {
+ case H264:
+ return HFI_CODEC_DECODE_AVC;
+ case HEVC:
+ return HFI_CODEC_DECODE_HEVC;
+ case VP9:
+ return HFI_CODEC_DECODE_VP9;
+ default:
+ return 0;
+ }
+}
+
u32 get_hfi_colorformat(u32 colorformat)
{
u32 hfi_colorformat = HFI_COLOR_FMT_NV12_UBWC;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
index f813116..cf0960b6 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
@@ -81,8 +81,8 @@ enum hfi_packet_port_type {

u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type);
u32 get_hfi_port(u32 plane);
-
u32 get_hfi_colorformat(u32 colorformat);
+u32 get_hfi_codec(struct iris_inst *inst);
u32 get_hfi_color_primaries(u32 primaries);
u32 get_hfi_transfer_char(u32 characterstics);
u32 get_hfi_matrix_coefficients(u32 coefficients);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
index 4f51d68..cddf143 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -40,10 +40,12 @@
* @fw_min_count: minimnum count of buffers needed by fw
* @state: instance state
* @ipsc_properties_set: boolean to set ipsc properties to fw
+ * @opsc_properties_set: boolean to set opsc properties to fw
* @hfi_frame_info: structure of frame info
* @src_subcr_params: subscription params to fw on input port
* @dst_subcr_params: subscription params to fw on output port
* @dpb_list_payload: array of dpb buffers
+ * @once_per_session_set: boolean to set once per session property
*/

struct iris_inst {
@@ -70,10 +72,12 @@ struct iris_inst {
u32 fw_min_count;
enum iris_inst_state state;
bool ipsc_properties_set;
+ bool opsc_properties_set;
struct iris_hfi_frame_info hfi_frame_info;
struct subscription_params src_subcr_params;
struct subscription_params dst_subcr_params;
u32 dpb_list_payload[MAX_DPB_LIST_ARRAY_SIZE];
+ bool once_per_session_set;
};

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
index 4cf6b69..6553029 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_state.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
@@ -193,3 +193,41 @@ bool allow_s_ctrl(struct iris_inst *inst, u32 cap_id)
(inst->state == IRIS_INST_INPUT_STREAMING ||
inst->state == IRIS_INST_STREAMING)));
}
+
+int iris_inst_state_change_streamon(struct iris_inst *inst, u32 plane)
+{
+ enum iris_inst_state new_state = IRIS_INST_ERROR;
+
+ if (plane == INPUT_MPLANE) {
+ if (inst->state == IRIS_INST_OPEN)
+ new_state = IRIS_INST_INPUT_STREAMING;
+ else if (inst->state == IRIS_INST_OUTPUT_STREAMING)
+ new_state = IRIS_INST_STREAMING;
+ } else if (plane == OUTPUT_MPLANE) {
+ if (inst->state == IRIS_INST_OPEN)
+ new_state = IRIS_INST_OUTPUT_STREAMING;
+ else if (inst->state == IRIS_INST_INPUT_STREAMING)
+ new_state = IRIS_INST_STREAMING;
+ }
+
+ return iris_inst_change_state(inst, new_state);
+}
+
+int iris_inst_state_change_streamoff(struct iris_inst *inst, u32 plane)
+{
+ enum iris_inst_state new_state = IRIS_INST_ERROR;
+
+ if (plane == INPUT_MPLANE) {
+ if (inst->state == IRIS_INST_INPUT_STREAMING)
+ new_state = IRIS_INST_OPEN;
+ else if (inst->state == IRIS_INST_STREAMING)
+ new_state = IRIS_INST_OUTPUT_STREAMING;
+ } else if (plane == OUTPUT_MPLANE) {
+ if (inst->state == IRIS_INST_OUTPUT_STREAMING)
+ new_state = IRIS_INST_OPEN;
+ else if (inst->state == IRIS_INST_STREAMING)
+ new_state = IRIS_INST_INPUT_STREAMING;
+ }
+
+ return iris_inst_change_state(inst, new_state);
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.h b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
index 35263e8..28d5380 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_state.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
@@ -47,4 +47,7 @@ bool allow_streamon(struct iris_inst *inst, u32 type);
bool allow_streamoff(struct iris_inst *inst, u32 type);
bool allow_s_ctrl(struct iris_inst *inst, u32 cap_id);

+int iris_inst_state_change_streamon(struct iris_inst *inst, u32 plane);
+int iris_inst_state_change_streamoff(struct iris_inst *inst, u32 plane);
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
index 66b5295..d599366 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
@@ -6,9 +6,12 @@
#include "iris_buffer.h"
#include "iris_ctrls.h"
#include "iris_core.h"
+#include "iris_ctrls.h"
#include "iris_helpers.h"
+#include "iris_hfi.h"
#include "iris_instance.h"
#include "iris_vb2.h"
+#include "iris_vdec.h"

int iris_vb2_queue_setup(struct vb2_queue *q,
unsigned int *num_buffers, unsigned int *num_planes,
@@ -90,6 +93,81 @@ int iris_vb2_queue_setup(struct vb2_queue *q,
return ret;
}

+int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ if (!q)
+ return -EINVAL;
+
+ inst = vb2_get_drv_priv(q);
+ if (!inst || !inst->core)
+ return -EINVAL;
+
+ if (q->type != INPUT_MPLANE && q->type != OUTPUT_MPLANE) {
+ ret = -EINVAL;
+ goto error;
+ }
+
+ if (!inst->once_per_session_set) {
+ inst->once_per_session_set = true;
+ ret = iris_hfi_session_set_codec(inst);
+ if (ret)
+ goto error;
+
+ ret = iris_hfi_session_set_default_header(inst);
+ if (ret)
+ goto error;
+
+ ret = iris_alloc_and_queue_session_int_bufs(inst, BUF_PERSIST);
+ if (ret)
+ goto error;
+ }
+
+ if (q->type == INPUT_MPLANE)
+ ret = vdec_streamon_input(inst);
+ else if (q->type == OUTPUT_MPLANE)
+ ret = vdec_streamon_output(inst);
+ if (ret)
+ goto error;
+
+ return ret;
+error:
+ iris_inst_change_state(inst, IRIS_INST_ERROR);
+
+ return ret;
+}
+
+void iris_vb2_stop_streaming(struct vb2_queue *q)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ if (!q)
+ return;
+
+ inst = vb2_get_drv_priv(q);
+ if (!inst)
+ return;
+
+ if (q->type != INPUT_MPLANE && q->type != OUTPUT_MPLANE)
+ goto error;
+
+ if (q->type == INPUT_MPLANE)
+ ret = session_streamoff(inst, INPUT_MPLANE);
+ else if (q->type == OUTPUT_MPLANE)
+ ret = session_streamoff(inst, OUTPUT_MPLANE);
+
+ if (ret)
+ goto error;
+
+ return;
+
+error:
+ iris_inst_change_state(inst, IRIS_INST_ERROR);
+}
+
void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev,
struct dma_buf *dbuf, unsigned long size)
{
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
index 4342034..fc0e804 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
@@ -11,6 +11,8 @@
int iris_vb2_queue_setup(struct vb2_queue *q,
unsigned int *num_buffers, unsigned int *num_planes,
unsigned int sizes[], struct device *alloc_devs[]);
+int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count);
+void iris_vb2_stop_streaming(struct vb2_queue *q);

/* vb2_mem_ops */
void *iris_vb2_alloc(struct vb2_buffer *vb, struct device *dev, unsigned long size);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index ac47fc0..0d8ca4b 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -340,7 +340,7 @@ int vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscri
return ret;
}

-int vdec_subscribe_property(struct iris_inst *inst, u32 plane)
+static int vdec_subscribe_property(struct iris_inst *inst, u32 plane)
{
const u32 *subcribe_prop = NULL;
u32 subscribe_prop_size = 0;
@@ -597,7 +597,7 @@ static int vdec_set_tier(struct iris_inst *inst)
sizeof(u32));
}

-int vdec_subscribe_src_change_param(struct iris_inst *inst)
+static int vdec_subscribe_src_change_param(struct iris_inst *inst)
{
const u32 *src_change_param;
u32 src_change_param_size;
@@ -925,7 +925,7 @@ static int vdec_set_ubwc_stride_scanline(struct iris_inst *inst)
sizeof(u32) * 4);
}

-int vdec_set_output_property(struct iris_inst *inst)
+static int vdec_set_output_property(struct iris_inst *inst)
{
int ret;

@@ -940,7 +940,7 @@ int vdec_set_output_property(struct iris_inst *inst)
return vdec_set_ubwc_stride_scanline(inst);
}

-int vdec_subscribe_dst_change_param(struct iris_inst *inst)
+static int vdec_subscribe_dst_change_param(struct iris_inst *inst)
{
u32 prop_type, payload_size, payload_type;
struct subscription_params subsc_params;
@@ -1063,3 +1063,133 @@ int vdec_subscribe_dst_change_param(struct iris_inst *inst)

return ret;
}
+
+static int process_streamon_input(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = iris_hfi_start(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = iris_inst_state_change_streamon(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ return ret;
+}
+
+int vdec_streamon_input(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = check_session_supported(inst);
+ if (ret)
+ return ret;
+
+ ret = set_v4l2_properties(inst);
+ if (ret)
+ return ret;
+
+ ret = iris_get_internal_buffers(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = iris_destroy_internal_buffers(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = iris_create_input_internal_buffers(inst);
+ if (ret)
+ return ret;
+
+ ret = iris_queue_input_internal_buffers(inst);
+ if (ret)
+ return ret;
+
+ if (!inst->ipsc_properties_set) {
+ ret = vdec_subscribe_src_change_param(inst);
+ if (ret)
+ return ret;
+ inst->ipsc_properties_set = true;
+ }
+
+ ret = vdec_subscribe_property(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = process_streamon_input(inst);
+ if (ret)
+ return ret;
+
+ return ret;
+}
+
+static int process_streamon_output(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = iris_hfi_start(inst, OUTPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = iris_inst_state_change_streamon(inst, OUTPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ return ret;
+}
+
+int vdec_streamon_output(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = check_session_supported(inst);
+ if (ret)
+ return ret;
+
+ ret = vdec_set_output_property(inst);
+ if (ret)
+ goto error;
+
+ if (!inst->opsc_properties_set) {
+ memcpy(&inst->dst_subcr_params,
+ &inst->src_subcr_params,
+ sizeof(inst->src_subcr_params));
+ ret = vdec_subscribe_dst_change_param(inst);
+ if (ret)
+ goto error;
+ inst->opsc_properties_set = true;
+ }
+
+ ret = vdec_subscribe_property(inst, OUTPUT_MPLANE);
+ if (ret)
+ goto error;
+
+ ret = iris_get_internal_buffers(inst, OUTPUT_MPLANE);
+ if (ret)
+ goto error;
+
+ ret = iris_destroy_internal_buffers(inst, OUTPUT_MPLANE);
+ if (ret)
+ goto error;
+
+ ret = iris_create_output_internal_buffers(inst);
+ if (ret)
+ goto error;
+
+ ret = process_streamon_output(inst);
+ if (ret)
+ goto error;
+
+ ret = iris_queue_output_internal_buffers(inst);
+ if (ret)
+ goto error;
+
+ return ret;
+
+error:
+ session_streamoff(inst, OUTPUT_MPLANE);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
index e0db653..0722da1 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
@@ -14,11 +14,9 @@ int vdec_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f);
int vdec_try_fmt(struct iris_inst *inst, struct v4l2_format *f);
int vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f);
int vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscription *sub);
-int vdec_subscribe_property(struct iris_inst *inst, u32 plane);
-int vdec_set_output_property(struct iris_inst *inst);
int vdec_init_src_change_param(struct iris_inst *inst);
int vdec_src_change(struct iris_inst *inst);
-int vdec_subscribe_src_change_param(struct iris_inst *inst);
-int vdec_subscribe_dst_change_param(struct iris_inst *inst);
+int vdec_streamon_input(struct iris_inst *inst);
+int vdec_streamon_output(struct iris_inst *inst);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
index 5c76821..14d0077 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
@@ -176,6 +176,8 @@ int vidc_open(struct file *filp)

inst->core = core;
inst->session_id = hash32_ptr(inst);
+ inst->ipsc_properties_set = false;
+ inst->opsc_properties_set = false;
iris_inst_change_state(inst, IRIS_INST_OPEN);
mutex_init(&inst->lock);
mutex_init(&inst->ctx_q_lock);
@@ -908,6 +910,8 @@ static const struct v4l2_file_operations v4l2_file_ops = {

static const struct vb2_ops iris_vb2_ops = {
.queue_setup = iris_vb2_queue_setup,
+ .start_streaming = iris_vb2_start_streaming,
+ .stop_streaming = iris_vb2_stop_streaming,
};

static struct vb2_mem_ops iris_vb2_mem_ops = {
--
2.7.4

2023-12-18 11:39:25

[permalink] [raw]

Subject: [PATCH v2 17/34] media: iris: implement vb2_ops queue setup

Implement queue_setup vb2_ops.
Calculate the buffer count and buffer size as par video
hardware requirement and updates to client.
Also, allocate the video driver buffers for output and
capture plane.

Signed-off-by: Dikshita Agarwal <[email protected]>
---
drivers/media/platform/qcom/vcodec/iris/Makefile | 4 +-
.../media/platform/qcom/vcodec/iris/iris_buffer.c | 179 +++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_buffer.h | 49 ++++++
.../media/platform/qcom/vcodec/iris/iris_common.h | 71 +++++++-
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 24 +++
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 4 +-
.../platform/qcom/vcodec/iris/iris_instance.h | 6 +
drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 71 ++++++++
drivers/media/platform/qcom/vcodec/iris/iris_vb2.h | 15 ++
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 5 +-
.../media/platform/qcom/vcodec/iris/iris_vidc.c | 19 ++-
.../platform/qcom/vcodec/iris/platform_common.h | 16 +-
12 files changed, 442 insertions(+), 21 deletions(-)
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vb2.h

diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
index 7fdee5b..a94e36b 100644
--- a/drivers/media/platform/qcom/vcodec/iris/Makefile
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -1,9 +1,10 @@
-iris-objs += ../hfi_queue.o ../firmware.o
+iris-objs += ../hfi_queue.o ../firmware.o ../buffers.o

iris-objs += iris_probe.o \
iris_state.o \
iris_core.o \
iris_vidc.o \
+ iris_vb2.o \
iris_vdec.o \
iris_state.o \
iris_ctrls.o \
@@ -11,6 +12,7 @@ iris-objs += iris_probe.o \
iris_hfi.o \
iris_hfi_response.o \
iris_hfi_packet.o \
+ iris_buffer.o \
resources.o \
vpu_common.o \
vpu_iris3.o \
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
new file mode 100644
index 0000000..b9cffbf
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
@@ -0,0 +1,179 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "../buffers.h"
+#include "iris_buffer.h"
+#include "iris_helpers.h"
+#include "iris_instance.h"
+
+static int input_min_count(struct iris_inst *inst)
+{
+ return MIN_BUFFERS;
+}
+
+static int output_min_count(struct iris_inst *inst)
+{
+ int output_min_count;
+
+ switch (inst->codec) {
+ case H264:
+ case HEVC:
+ output_min_count = 4;
+ break;
+ case VP9:
+ output_min_count = 9;
+ break;
+ default:
+ output_min_count = 4;
+ break;
+ }
+
+ return output_min_count;
+}
+
+int iris_get_buf_min_count(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type)
+{
+ switch (buffer_type) {
+ case BUF_INPUT:
+ return input_min_count(inst);
+ case BUF_OUTPUT:
+ return output_min_count(inst);
+ default:
+ return 0;
+ }
+}
+
+static u32 input_buffer_size(struct iris_inst *inst)
+{
+ u32 base_res_mbs = NUM_MBS_4k;
+ u32 frame_size, num_mbs;
+ struct v4l2_format *f;
+ u32 div_factor = 1;
+ u32 codec;
+
+ f = inst->fmt_src;
+ codec = f->fmt.pix_mp.pixelformat;
+
+ num_mbs = get_mbpf(inst);
+ if (num_mbs > NUM_MBS_4k) {
+ div_factor = 4;
+ base_res_mbs = inst->cap[MBPF].value;
+ } else {
+ base_res_mbs = NUM_MBS_4k;
+ if (codec == V4L2_PIX_FMT_VP9)
+ div_factor = 1;
+ else
+ div_factor = 2;
+ }
+
+ frame_size = base_res_mbs * MB_IN_PIXEL * 3 / 2 / div_factor;
+
+ /* multiply by 10/8 (1.25) to get size for 10 bit case */
+ if (codec == V4L2_PIX_FMT_VP9 || codec == V4L2_PIX_FMT_HEVC)
+ frame_size = frame_size + (frame_size >> 2);
+
+ return ALIGN(frame_size, SZ_4K);
+}
+
+static u32 output_buffer_size(struct iris_inst *inst)
+{
+ struct v4l2_format *f;
+ u32 size;
+
+ f = inst->fmt_dst;
+
+ size = video_raw_buffer_size(f->fmt.pix_mp.pixelformat, f->fmt.pix_mp.width,
+ f->fmt.pix_mp.height);
+ return size;
+}
+
+int iris_get_buffer_size(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type)
+{
+ switch (buffer_type) {
+ case BUF_INPUT:
+ return input_buffer_size(inst);
+ case BUF_OUTPUT:
+ return output_buffer_size(inst);
+ default:
+ return 0;
+ }
+}
+
+struct iris_buffers *iris_get_buffer_list(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type)
+{
+ switch (buffer_type) {
+ case BUF_INPUT:
+ return &inst->buffers.input;
+ case BUF_OUTPUT:
+ return &inst->buffers.output;
+ case BUF_READ_ONLY:
+ return &inst->buffers.read_only;
+ case BUF_BIN:
+ return &inst->buffers.bin;
+ case BUF_ARP:
+ return &inst->buffers.arp;
+ case BUF_COMV:
+ return &inst->buffers.comv;
+ case BUF_NON_COMV:
+ return &inst->buffers.non_comv;
+ case BUF_LINE:
+ return &inst->buffers.line;
+ case BUF_DPB:
+ return &inst->buffers.dpb;
+ case BUF_PERSIST:
+ return &inst->buffers.persist;
+ case BUF_VPSS:
+ return &inst->buffers.vpss;
+ default:
+ return NULL;
+ }
+}
+
+int iris_allocate_buffers(struct iris_inst *inst,
+ enum iris_buffer_type buf_type,
+ u32 num_buffers)
+{
+ struct iris_buffer *buf = NULL;
+ struct iris_buffers *buffers;
+ int idx = 0;
+
+ buffers = iris_get_buffer_list(inst, buf_type);
+ if (!buffers)
+ return -EINVAL;
+
+ for (idx = 0; idx < num_buffers; idx++) {
+ buf = kzalloc(sizeof(*buf), GFP_KERNEL);
+ if (!buf)
+ return -EINVAL;
+
+ INIT_LIST_HEAD(&buf->list);
+ list_add_tail(&buf->list, &buffers->list);
+ buf->type = buf_type;
+ buf->index = idx;
+ }
+
+ return 0;
+}
+
+int iris_free_buffers(struct iris_inst *inst,
+ enum iris_buffer_type buf_type)
+{
+ struct iris_buffer *buf, *dummy;
+ struct iris_buffers *buffers;
+
+ buffers = iris_get_buffer_list(inst, buf_type);
+ if (!buffers)
+ return -EINVAL;
+
+ list_for_each_entry_safe(buf, dummy, &buffers->list, list) {
+ list_del_init(&buf->list);
+ kfree(buf);
+ }
+
+ return 0;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
new file mode 100644
index 0000000..1cd76a9
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
@@ -0,0 +1,49 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_BUFFER_H_
+#define _IRIS_BUFFER_H_
+
+#define MIN_BUFFERS 4
+
+#include "iris_common.h"
+
+struct iris_inst;
+
+struct iris_buffers {
+ struct list_head list; // list of "struct iris_buffer"
+ u32 min_count;
+ u32 actual_count;
+ u32 size;
+ bool reuse;
+};
+
+struct iris_buffers_info {
+ struct iris_buffers input;
+ struct iris_buffers output;
+ struct iris_buffers read_only;
+ struct iris_buffers bin;
+ struct iris_buffers arp;
+ struct iris_buffers comv;
+ struct iris_buffers non_comv;
+ struct iris_buffers line;
+ struct iris_buffers dpb;
+ struct iris_buffers persist;
+ struct iris_buffers vpss;
+};
+
+int iris_get_buf_min_count(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type);
+int iris_get_buffer_size(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type);
+struct iris_buffers *iris_get_buffer_list(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type);
+int iris_allocate_buffers(struct iris_inst *inst,
+ enum iris_buffer_type buf_type,
+ u32 num_buffers);
+int iris_free_buffers(struct iris_inst *inst,
+ enum iris_buffer_type buf_type);
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
index 3e4dd71..4edadc3 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
@@ -5,6 +5,7 @@
#ifndef _IRIS_COMMON_H_
#define _IRIS_COMMON_H_

+#include <media/v4l2-dev.h>
#include <media/v4l2-device.h>
#include <media/v4l2-event.h>
#include <media/v4l2-ioctl.h>
@@ -14,7 +15,31 @@
#define OUTPUT_MPLANE V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE
#define DEFAULT_WIDTH 320
#define DEFAULT_HEIGHT 240
-#define DEFAULT_BUF_SIZE 115200
+
+#define MB_IN_PIXEL (16 * 16)
+
+#define NUM_MBS_4k (((4096 + 15) >> 4) * ((2304 + 15) >> 4))
+
+enum codec_type {
+ H264 = BIT(0),
+ HEVC = BIT(1),
+ VP9 = BIT(2),
+};
+
+enum colorformat_type {
+ FMT_NONE = 0,
+ FMT_NV12C = BIT(0),
+ FMT_NV12 = BIT(1),
+ FMT_NV21 = BIT(2),
+ FMT_TP10C = BIT(3),
+};
+
+struct rect_desc {
+ u32 left;
+ u32 top;
+ u32 width;
+ u32 height;
+};

enum signal_session_response {
SIGNAL_CMD_STOP_INPUT = 0,
@@ -23,4 +48,48 @@ enum signal_session_response {
MAX_SIGNAL,
};

+enum iris_buffer_type {
+ BUF_NONE,
+ BUF_INPUT,
+ BUF_OUTPUT,
+ BUF_READ_ONLY,
+ BUF_BIN,
+ BUF_ARP,
+ BUF_COMV,
+ BUF_NON_COMV,
+ BUF_LINE,
+ BUF_DPB,
+ BUF_PERSIST,
+ BUF_VPSS,
+};
+
+enum iris_buffer_attributes {
+ BUF_ATTR_DEFERRED = BIT(0),
+ BUF_ATTR_READ_ONLY = BIT(1),
+ BUF_ATTR_PENDING_RELEASE = BIT(2),
+ BUF_ATTR_QUEUED = BIT(3),
+ BUF_ATTR_DEQUEUED = BIT(4),
+ BUF_ATTR_BUFFER_DONE = BIT(5),
+};
+
+struct iris_buffer {
+ struct list_head list;
+ struct iris_inst *inst;
+ enum iris_buffer_type type;
+ u32 index;
+ int fd;
+ u32 buffer_size;
+ u32 data_offset;
+ u32 data_size;
+ u64 device_addr;
+ void *kvaddr;
+ unsigned long dma_attrs;
+ u32 flags;
+ u64 timestamp;
+ enum iris_buffer_attributes attr;
+ void *dmabuf;
+ struct sg_table *sg_table;
+ struct dma_buf_attachment *attach;
+};
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index 8d8bc3a..54a7851 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -42,6 +42,30 @@ u32 get_port_info(struct iris_inst *inst,
return HFI_PORT_NONE;
}

+enum iris_buffer_type v4l2_type_to_driver(u32 type)
+{
+ switch (type) {
+ case INPUT_MPLANE:
+ return BUF_INPUT;
+ case OUTPUT_MPLANE:
+ return BUF_OUTPUT;
+ default:
+ return 0;
+ }
+}
+
+int get_mbpf(struct iris_inst *inst)
+{
+ int height = 0, width = 0;
+ struct v4l2_format *inp_f;
+
+ inp_f = inst->fmt_src;
+ width = max(inp_f->fmt.pix_mp.width, inst->crop.width);
+ height = max(inp_f->fmt.pix_mp.height, inst->crop.height);
+
+ return NUM_MBS_PER_FRAME(height, width);
+}
+
static int process_inst_timeout(struct iris_inst *inst)
{
struct iris_inst *instance;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
index 60c79124..3bae969 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -11,6 +11,7 @@

#include "iris_core.h"
#include "iris_instance.h"
+#include "iris_buffer.h"
#include "platform_common.h"

#define NUM_MBS_PER_FRAME(__height, __width) \
@@ -21,7 +22,8 @@ bool res_is_less_than(u32 width, u32 height,
u32 ref_width, u32 ref_height);
u32 get_port_info(struct iris_inst *inst,
enum plat_inst_cap_type cap_id);
-
+enum iris_buffer_type v4l2_type_to_driver(u32 type);
+int get_mbpf(struct iris_inst *inst);
int close_session(struct iris_inst *inst);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
index f6a3066..a5c6cb48 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -8,6 +8,8 @@

#include <media/v4l2-ctrls.h>

+#include "iris_buffer.h"
+#include "iris_common.h"
#include "iris_core.h"
#include "iris_common.h"
#include "platform_common.h"
@@ -25,6 +27,7 @@
* @fmt_src: structure of v4l2_format for source
* @fmt_dst: structure of v4l2_format for destination
* @ctrl_handler: reference of v4l2 ctrl handler
+ * @crop: structure of crop info
* @packet: HFI packet
* @packet_size: HFI packet size
* @completions: structure of signal completions
@@ -32,6 +35,7 @@
* @num_ctrls: supported number of controls
* @caps_list: list head of capability
* @codec: codec type
+ * @buffers: structure of buffer info
*/

struct iris_inst {
@@ -45,6 +49,7 @@ struct iris_inst {
struct v4l2_format *fmt_src;
struct v4l2_format *fmt_dst;
struct v4l2_ctrl_handler ctrl_handler;
+ struct rect_desc crop;
void *packet;
u32 packet_size;
struct completion completions[MAX_SIGNAL];
@@ -52,6 +57,7 @@ struct iris_inst {
u32 num_ctrls;
struct list_head caps_list;
enum codec_type codec;
+ struct iris_buffers_info buffers;
};

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
new file mode 100644
index 0000000..b040d27
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
@@ -0,0 +1,71 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "iris_buffer.h"
+#include "iris_core.h"
+#include "iris_helpers.h"
+#include "iris_instance.h"
+#include "iris_vb2.h"
+
+int iris_vb2_queue_setup(struct vb2_queue *q,
+ unsigned int *num_buffers, unsigned int *num_planes,
+ unsigned int sizes[], struct device *alloc_devs[])
+{
+ enum iris_buffer_type buffer_type = 0;
+ struct iris_buffers *buffers;
+ struct iris_inst *inst;
+ struct iris_core *core;
+ struct v4l2_format *f;
+ int ret;
+
+ if (!q || !num_buffers || !num_planes || !sizes)
+ return -EINVAL;
+
+ inst = vb2_get_drv_priv(q);
+ if (!inst || !inst->core)
+ return -EINVAL;
+
+ core = inst->core;
+
+ if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+ f = inst->fmt_src;
+ else
+ f = inst->fmt_dst;
+
+ if (*num_planes) {
+ if (*num_planes != f->fmt.pix_mp.num_planes ||
+ sizes[0] < f->fmt.pix_mp.plane_fmt[0].sizeimage)
+ return -EINVAL;
+ }
+
+ buffer_type = v4l2_type_to_driver(q->type);
+ if (!buffer_type)
+ return -EINVAL;
+
+ ret = iris_free_buffers(inst, buffer_type);
+ if (ret)
+ return ret;
+
+ buffers = iris_get_buffer_list(inst, buffer_type);
+ if (!buffers)
+ return -EINVAL;
+
+ buffers->min_count = iris_get_buf_min_count(inst, buffer_type);
+ if (*num_buffers < buffers->min_count)
+ *num_buffers = buffers->min_count;
+ buffers->actual_count = *num_buffers;
+ *num_planes = 1;
+
+ buffers->size = iris_get_buffer_size(inst, buffer_type);
+
+ f->fmt.pix_mp.plane_fmt[0].sizeimage = buffers->size;
+ sizes[0] = f->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+ ret = iris_allocate_buffers(inst, buffer_type, *num_buffers);
+
+ q->dev = core->dev;
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
new file mode 100644
index 0000000..8a8e1039
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_VB2_H_
+#define _IRIS_VB2_H_
+
+#include <media/videobuf2-v4l2.h>
+
+int iris_vb2_queue_setup(struct vb2_queue *q,
+ unsigned int *num_buffers, unsigned int *num_planes,
+ unsigned int sizes[], struct device *alloc_devs[]);
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index 984be34..1b957a13 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -3,6 +3,7 @@
* Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
*/

+#include "iris_buffer.h"
#include "iris_common.h"
#include "iris_vdec.h"

@@ -22,7 +23,7 @@ void vdec_inst_init(struct iris_inst *inst)
f->fmt.pix_mp.pixelformat = V4L2_PIX_FMT_H264;
f->fmt.pix_mp.num_planes = 1;
f->fmt.pix_mp.plane_fmt[0].bytesperline = 0;
- f->fmt.pix_mp.plane_fmt[0].sizeimage = DEFAULT_BUF_SIZE;
+ f->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_INPUT);
f->fmt.pix_mp.field = V4L2_FIELD_NONE;

f = inst->fmt_dst;
@@ -32,7 +33,7 @@ void vdec_inst_init(struct iris_inst *inst)
f->fmt.pix_mp.height = ALIGN(DEFAULT_HEIGHT, 32);
f->fmt.pix_mp.num_planes = 1;
f->fmt.pix_mp.plane_fmt[0].bytesperline = ALIGN(DEFAULT_WIDTH, 128);
- f->fmt.pix_mp.plane_fmt[0].sizeimage = DEFAULT_BUF_SIZE;
+ f->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_OUTPUT);
f->fmt.pix_mp.field = V4L2_FIELD_NONE;
f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT;
f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
index 3a26edb..410de720 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
@@ -10,6 +10,7 @@
#include "iris_vdec.h"
#include "iris_vidc.h"
#include "iris_ctrls.h"
+#include "iris_vb2.h"

static int vidc_v4l2_fh_init(struct iris_inst *inst)
{
@@ -168,13 +169,22 @@ int vidc_open(struct file *filp)
inst->core = core;
inst->session_id = hash32_ptr(inst);
mutex_init(&inst->ctx_q_lock);
- for (i = 0; i < MAX_SIGNAL; i++)
- init_completion(&inst->completions[i]);

ret = vidc_add_session(inst);
if (ret)
goto fail_free_inst;

+ INIT_LIST_HEAD(&inst->buffers.input.list);
+ INIT_LIST_HEAD(&inst->buffers.output.list);
+ INIT_LIST_HEAD(&inst->buffers.read_only.list);
+ INIT_LIST_HEAD(&inst->buffers.bin.list);
+ INIT_LIST_HEAD(&inst->buffers.arp.list);
+ INIT_LIST_HEAD(&inst->buffers.comv.list);
+ INIT_LIST_HEAD(&inst->buffers.non_comv.list);
+ INIT_LIST_HEAD(&inst->buffers.line.list);
+ INIT_LIST_HEAD(&inst->buffers.dpb.list);
+ INIT_LIST_HEAD(&inst->buffers.persist.list);
+ INIT_LIST_HEAD(&inst->buffers.vpss.list);
INIT_LIST_HEAD(&inst->caps_list);
for (i = 0; i < MAX_SIGNAL; i++)
init_completion(&inst->completions[i]);
@@ -306,9 +316,14 @@ static const struct v4l2_file_operations v4l2_file_ops = {
.poll = vidc_poll,
};

+static const struct vb2_ops iris_vb2_ops = {
+ .queue_setup = iris_vb2_queue_setup,
+};
+
int init_ops(struct iris_core *core)
{
core->v4l2_file_ops = &v4l2_file_ops;
+ core->vb2_ops = &iris_vb2_ops;

return 0;
}
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
index 8305c65..e242614 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
@@ -9,6 +9,8 @@
#include <linux/bits.h>
#include <media/v4l2-ctrls.h>

+#include "iris_common.h"
+
struct iris_core;
struct iris_inst;

@@ -35,20 +37,6 @@ struct iris_inst;
.bank_spreading = bsp, \
}

-enum codec_type {
- H264 = BIT(0),
- HEVC = BIT(1),
- VP9 = BIT(2),
-};
-
-enum colorformat_type {
- FMT_NONE = 0,
- FMT_NV12C = BIT(0),
- FMT_NV12 = BIT(1),
- FMT_NV21 = BIT(2),
- FMT_TP10C = BIT(3),
-};
-
enum stage_type {
STAGE_NONE = 0,
STAGE_1 = 1,
--
2.7.4

2023-12-18 11:39:36

[permalink] [raw]

Subject: [PATCH v2 15/34] media: iris: add handling for interrupt service routine(ISR) invoked by hardware

Allocate interrupt resources, enable the interrupt line and IRQ handling.
Register the IRQ handler to be called when interrupt occurs and
the function to be called from IRQ handler thread.
The threads invoke the driver's response handler which handles
all different responses from firmware.

Signed-off-by: Dikshita Agarwal <[email protected]>
---
drivers/media/platform/qcom/vcodec/iris/Makefile | 1 +
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 10 ++
.../media/platform/qcom/vcodec/iris/iris_core.h | 4 +
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 29 +++-
drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 3 +
.../platform/qcom/vcodec/iris/iris_hfi_response.c | 184 +++++++++++++++++++++
.../platform/qcom/vcodec/iris/iris_hfi_response.h | 20 +++
.../media/platform/qcom/vcodec/iris/iris_probe.c | 27 +++
.../media/platform/qcom/vcodec/iris/vpu_common.h | 2 +
.../media/platform/qcom/vcodec/iris/vpu_iris3.c | 44 +++++
10 files changed, 323 insertions(+), 1 deletion(-)
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.h

diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
index 3c076d0..85fdf63 100644
--- a/drivers/media/platform/qcom/vcodec/iris/Makefile
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -8,6 +8,7 @@ iris-objs += iris_probe.o \
iris_state.o \
iris_helpers.o \
iris_hfi.o \
+ iris_hfi_response.o \
iris_hfi_packet.o \
resources.o \
vpu_common.o \
diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index 423ba1a..9dd5f11 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -6,6 +6,8 @@
#ifndef _HFI_DEFINES_H_
#define _HFI_DEFINES_H_

+#include <linux/types.h>
+
#define HFI_VIDEO_ARCH_LX 0x1

#define HFI_CMD_INIT 0x01000001
@@ -50,4 +52,12 @@

#define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169

+#define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001
+
+struct hfi_debug_header {
+ u32 size;
+ u32 debug_level;
+ u32 reserved[2];
+};
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index b47520a..c125bce 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -45,6 +45,7 @@
* @use_tz: a flag that suggests presence of trustzone
* @packet: pointer to packet from driver to fw
* @packet_size: size of packet
+ * @response_packet: a pointer to response packet from fw to driver
* @sys_init_id: id of sys init packet
* @header_id: id of packet header
* @packet_id: id of packet
@@ -52,6 +53,7 @@
* @platform_data: a structure for platform data
* @cap: an array for supported core capabilities
* @instances: a list_head of all instances
+ * @intr_status: interrupt status
*/

struct iris_core {
@@ -82,6 +84,7 @@ struct iris_core {
unsigned int use_tz;
u8 *packet;
u32 packet_size;
+ u8 *response_packet;
u32 sys_init_id;
u32 header_id;
u32 packet_id;
@@ -89,6 +92,7 @@ struct iris_core {
struct platform_data *platform_data;
struct plat_core_cap cap[CORE_CAP_MAX + 1];
struct list_head instances;
+ u32 intr_status;
};

int iris_core_init(struct iris_core *core);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index d144ae5..4b8d8c7 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -5,10 +5,11 @@

#include "../firmware.h"
#include "../hfi_queue.h"
+#include "hfi_defines.h"
#include "iris_helpers.h"
#include "iris_hfi.h"
#include "iris_hfi_packet.h"
-#include "hfi_defines.h"
+#include "iris_hfi_response.h"
#include "vpu_common.h"

static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt)
@@ -253,3 +254,29 @@ int iris_hfi_session_close(struct iris_inst *inst)

return ret;
}
+
+irqreturn_t iris_hfi_isr(int irq, void *data)
+{
+ disable_irq_nosync(irq);
+
+ return IRQ_WAKE_THREAD;
+}
+
+irqreturn_t iris_hfi_isr_handler(int irq, void *data)
+{
+ struct iris_core *core = data;
+
+ if (!core)
+ return IRQ_NONE;
+
+ mutex_lock(&core->lock);
+ call_vpu_op(core, clear_interrupt, core);
+ mutex_unlock(&core->lock);
+
+ __response_handler(core);
+
+ if (!call_vpu_op(core, watchdog, core, core->intr_status))
+ enable_irq(irq);
+
+ return IRQ_HANDLED;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
index 8a057cc..8a62986 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
@@ -14,4 +14,7 @@ int iris_hfi_core_deinit(struct iris_core *core);
int iris_hfi_session_open(struct iris_inst *inst);
int iris_hfi_session_close(struct iris_inst *inst);

+irqreturn_t iris_hfi_isr(int irq, void *data);
+irqreturn_t iris_hfi_isr_handler(int irq, void *data);
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
new file mode 100644
index 0000000..829f3f6
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
@@ -0,0 +1,184 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "hfi_defines.h"
+#include "iris_hfi_packet.h"
+#include "iris_hfi_response.h"
+
+static void print_sfr_message(struct iris_core *core)
+{
+ struct sfr_buffer *vsfr = NULL;
+ u32 vsfr_size = 0;
+ void *p = NULL;
+
+ vsfr = (struct sfr_buffer *)core->sfr.kernel_vaddr;
+ if (vsfr) {
+ if (vsfr->bufsize != core->sfr.size)
+ return;
+ vsfr_size = vsfr->bufsize - sizeof(u32);
+ p = memchr(vsfr->rg_data, '\0', vsfr_size);
+ /* SFR isn't guaranteed to be NULL terminated */
+ if (!p)
+ vsfr->rg_data[vsfr_size - 1] = '\0';
+ }
+}
+
+static int validate_packet(u8 *response_pkt, u8 *core_resp_pkt,
+ u32 core_resp_pkt_size)
+{
+ u32 response_pkt_size = 0;
+ u8 *response_limit;
+
+ if (!response_pkt || !core_resp_pkt || !core_resp_pkt_size)
+ return -EINVAL;
+
+ response_limit = core_resp_pkt + core_resp_pkt_size;
+
+ if (response_pkt < core_resp_pkt || response_pkt > response_limit)
+ return -EINVAL;
+
+ response_pkt_size = *(u32 *)response_pkt;
+ if (!response_pkt_size)
+ return -EINVAL;
+
+ if (response_pkt_size < sizeof(struct hfi_packet))
+ return -EINVAL;
+
+ if (response_pkt + response_pkt_size > response_limit)
+ return -EINVAL;
+
+ return 0;
+}
+
+static int validate_hdr_packet(struct iris_core *core,
+ struct hfi_header *hdr)
+{
+ struct hfi_packet *packet;
+ int i, ret = 0;
+ u8 *pkt;
+
+ if (hdr->size < sizeof(*hdr) + sizeof(*packet))
+ return -EINVAL;
+
+ pkt = (u8 *)((u8 *)hdr + sizeof(*hdr));
+
+ for (i = 0; i < hdr->num_packets; i++) {
+ packet = (struct hfi_packet *)pkt;
+ ret = validate_packet(pkt, core->response_packet, core->packet_size);
+ if (ret)
+ return ret;
+
+ pkt += packet->size;
+ }
+
+ return ret;
+}
+
+static int handle_system_error(struct iris_core *core,
+ struct hfi_packet *pkt)
+{
+ print_sfr_message(core);
+
+ iris_core_deinit(core);
+
+ return 0;
+}
+
+static int handle_response(struct iris_core *core, void *response)
+{
+ struct hfi_header *hdr;
+ int ret;
+
+ hdr = (struct hfi_header *)response;
+ ret = validate_hdr_packet(core, hdr);
+ if (ret)
+ return handle_system_error(core, NULL);
+
+ return ret;
+}
+
+static int iris_hfi_queue_read(struct iris_core *core, void *pkt,
+ struct iface_q_info *q_info)
+{
+ u32 tx_req;
+
+ if (!core_in_valid_state(core))
+ return -EINVAL;
+
+ if (!q_info || !q_info->q_array.kernel_vaddr || !pkt) {
+ dev_err(core->dev, "cannot read from shared Q's\n");
+ return -ENODATA;
+ }
+
+ if (read_queue(q_info, pkt, &tx_req))
+ return -ENODATA;
+
+ return 0;
+}
+
+int __response_handler(struct iris_core *core)
+{
+ int ret = 0;
+
+ if (call_vpu_op(core, watchdog, core, core->intr_status)) {
+ struct hfi_packet pkt = {.type = HFI_SYS_ERROR_WD_TIMEOUT};
+
+ mutex_lock(&core->lock);
+ iris_change_core_state(core, IRIS_CORE_ERROR);
+ dev_err(core->dev, "%s: CPU WD error received\n", __func__);
+ mutex_unlock(&core->lock);
+
+ return handle_system_error(core, &pkt);
+ }
+
+ memset(core->response_packet, 0, core->packet_size);
+ while (!iris_hfi_queue_read(core, core->response_packet, &core->message_queue)) {
+ ret = handle_response(core, core->response_packet);
+ if (ret)
+ continue;
+ if (core->state != IRIS_CORE_INIT)
+ break;
+ memset(core->response_packet, 0, core->packet_size);
+ }
+
+ iris_flush_debug_queue(core, core->response_packet, core->packet_size);
+
+ return ret;
+}
+
+void iris_flush_debug_queue(struct iris_core *core,
+ u8 *packet, u32 packet_size)
+{
+ struct hfi_debug_header *pkt;
+ bool local_packet = false;
+ u8 *log;
+
+ if (!packet || !packet_size) {
+ packet = kzalloc(IFACEQ_CORE_PKT_SIZE, GFP_KERNEL);
+ if (!packet)
+ return;
+
+ packet_size = IFACEQ_CORE_PKT_SIZE;
+
+ local_packet = true;
+ }
+
+ while (!iris_hfi_queue_read(core, packet, &core->debug_queue)) {
+ pkt = (struct hfi_debug_header *)packet;
+
+ if (pkt->size < sizeof(*pkt))
+ continue;
+
+ if (pkt->size >= packet_size)
+ continue;
+
+ packet[pkt->size] = '\0';
+ log = (u8 *)packet + sizeof(*pkt) + 1;
+ dev_dbg(core->dev, "%s", log);
+ }
+
+ if (local_packet)
+ kfree(packet);
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.h
new file mode 100644
index 0000000..c1d2d27
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_HFI_RESPONSE_H_
+#define _IRIS_HFI_RESPONSE_H_
+
+#include "iris_core.h"
+
+struct sfr_buffer {
+ u32 bufsize;
+ u8 rg_data[];
+};
+
+int __response_handler(struct iris_core *core);
+void iris_flush_debug_queue(struct iris_core *core,
+ u8 *packet, u32 packet_size);
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
index 4f20da8..8c591da 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -10,9 +10,25 @@
#include "../hfi_queue.h"
#include "iris_core.h"
#include "iris_helpers.h"
+#include "iris_hfi.h"
#include "resources.h"
#include "iris_vidc.h"

+static int init_iris_isr(struct iris_core *core)
+{
+ int ret;
+
+ ret = devm_request_threaded_irq(core->dev, core->irq, iris_hfi_isr,
+ iris_hfi_isr_handler, IRQF_TRIGGER_HIGH, "iris", core);
+ if (ret) {
+ dev_err(core->dev, "%s: Failed to allocate iris IRQ\n", __func__);
+ return ret;
+ }
+ disable_irq_nosync(core->irq);
+
+ return ret;
+}
+
static int iris_register_video_device(struct iris_core *core)
{
struct video_device *vdev;
@@ -86,6 +102,10 @@ static int iris_probe(struct platform_device *pdev)
if (!core->packet)
return -ENOMEM;

+ core->response_packet = devm_kzalloc(core->dev, core->packet_size, GFP_KERNEL);
+ if (!core->response_packet)
+ return -ENOMEM;
+
INIT_LIST_HEAD(&core->instances);

core->reg_base = devm_platform_ioremap_resource(pdev, 0);
@@ -96,6 +116,13 @@ static int iris_probe(struct platform_device *pdev)
if (core->irq < 0)
return core->irq;

+ ret = init_iris_isr(core);
+ if (ret) {
+ dev_err_probe(core->dev, ret,
+ "%s: Failed to init isr with %d\n", __func__, ret);
+ return ret;
+ }
+
ret = init_platform(core);
if (ret) {
dev_err_probe(core->dev, ret,
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h
index 790496a..6512039 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h
@@ -22,6 +22,8 @@ struct compat_handle {
struct vpu_ops {
int (*boot_firmware)(struct iris_core *core);
int (*raise_interrupt)(struct iris_core *core);
+ int (*clear_interrupt)(struct iris_core *core);
+ int (*watchdog)(struct iris_core *core, u32 intr_status);
};

int init_vpu(struct iris_core *core);
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
index 95bf223..a34d0ed 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
@@ -17,6 +17,8 @@
#define CPU_CS_VCICMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x24)
#define CPU_CS_VCICMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x28)

+#define CPU_CS_A2HSOFTINTCLR_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x1C)
+
/* HFI_CTRL_INIT */
#define CPU_CS_SCIACMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x48)

@@ -57,10 +59,19 @@
#define CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3 0xfe
#define CTRL_ERROR_STATUS__M_IRIS3 \
CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3
+#define CTRL_INIT_IDLE_MSG_BMSK_IRIS3 \
+ CPU_CS_SCIACMDARG0_HFI_CTRL_INIT_IDLE_MSG_BMSK_IRIS3
+
+#define WRAPPER_BASE_OFFS_IRIS3 0x000B0000
+#define WRAPPER_INTR_STATUS_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x0C)
+#define WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3 0x8
+#define WRAPPER_INTR_STATUS_A2H_BMSK_IRIS3 0x4

#define CPU_IC_SOFTINT_IRIS3 (CPU_IC_BASE_OFFS_IRIS3 + 0x150)
#define CPU_IC_SOFTINT_H2A_SHFT_IRIS3 0x0

+#define WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3 0x8
+
static int setup_ucregion_memory_map_iris3(struct iris_core *core)
{
int ret;
@@ -153,9 +164,42 @@ static int raise_interrupt_iris3(struct iris_core *core)
return write_register(core, CPU_IC_SOFTINT_IRIS3, 1 << CPU_IC_SOFTINT_H2A_SHFT_IRIS3);
}

+static int clear_interrupt_iris3(struct iris_core *core)
+{
+ u32 intr_status = 0, mask = 0;
+ int ret;
+
+ ret = read_register(core, WRAPPER_INTR_STATUS_IRIS3, &intr_status);
+ if (ret)
+ return ret;
+
+ mask = (WRAPPER_INTR_STATUS_A2H_BMSK_IRIS3 |
+ WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3 |
+ CTRL_INIT_IDLE_MSG_BMSK_IRIS3);
+
+ if (intr_status & mask)
+ core->intr_status |= intr_status;
+
+ ret = write_register(core, CPU_CS_A2HSOFTINTCLR_IRIS3, 1);
+
+ return ret;
+}
+
+static int watchdog_iris3(struct iris_core *core, u32 intr_status)
+{
+ if (intr_status & WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3) {
+ dev_err(core->dev, "%s: received watchdog interrupt\n", __func__);
+ return -ETIME;
+ }
+
+ return 0;
+}
+
static const struct vpu_ops iris3_ops = {
.boot_firmware = boot_firmware_iris3,
.raise_interrupt = raise_interrupt_iris3,
+ .clear_interrupt = clear_interrupt_iris3,
+ .watchdog = watchdog_iris3,
};

int init_iris3(struct iris_core *core)
--
2.7.4

2023-12-18 11:39:53

[permalink] [raw]

Subject: [PATCH v2 24/34] media: iris: subscribe input and output properties to firmware

From: Vikash Garodia <[email protected]>

Driver can subscribe to different properties for which
it expects a response from firmware. Different SOCs
firmware can support different properties which can be
subscribed by driver.
HFI_CMD_SUBSCRIBE_MODE with HFI_MODE_PROPERTY - to subscribe
any property to firmware.

Also, set below mandatory properties on capture plane:
HFI_PROP_COLOR_FORMAT
HFI_PROP_LINEAR_STRIDE_SCANLINE
HFI_PROP_UBWC_STRIDE_SCANLINE.

Signed-off-by: Vikash Garodia <[email protected]>
Signed-off-by: Dikshita Agarwal <[email protected]>
---
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 31 +++++
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 38 +++++
drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 3 +
.../platform/qcom/vcodec/iris/iris_hfi_packet.c | 42 ++++++
.../platform/qcom/vcodec/iris/iris_hfi_packet.h | 3 +
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 154 +++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_vdec.h | 2 +
.../platform/qcom/vcodec/iris/platform_common.h | 8 ++
.../platform/qcom/vcodec/iris/platform_sm8550.c | 29 ++++
9 files changed, 310 insertions(+)

diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index ce4eaff..a6078a5 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -25,6 +25,29 @@
#define HFI_PROP_UBWC_BANK_SWZL_LEVEL3 0x03000008
#define HFI_PROP_UBWC_BANK_SPREADING 0x03000009

+enum hfi_property_mode_type {
+ HFI_MODE_NONE = 0x00000000,
+ HFI_MODE_PORT_SETTINGS_CHANGE = 0x00000001,
+ HFI_MODE_PROPERTY = 0x00000002,
+};
+
+#define HFI_CMD_SUBSCRIBE_MODE 0x0100000B
+
+enum hfi_color_format {
+ HFI_COLOR_FMT_OPAQUE = 0,
+ HFI_COLOR_FMT_NV12 = 1,
+ HFI_COLOR_FMT_NV12_UBWC = 2,
+ HFI_COLOR_FMT_P010 = 3,
+ HFI_COLOR_FMT_TP10_UBWC = 4,
+ HFI_COLOR_FMT_RGBA8888 = 5,
+ HFI_COLOR_FMT_RGBA8888_UBWC = 6,
+ HFI_COLOR_FMT_NV21 = 7,
+};
+
+#define HFI_PROP_COLOR_FORMAT 0x03000101
+
+#define HFI_PROP_LINEAR_STRIDE_SCANLINE 0x03000104
+
#define HFI_PROP_PROFILE 0x03000107

#define HFI_PROP_LEVEL 0x03000108
@@ -49,10 +72,18 @@

#define HFI_PROP_DECODE_ORDER_OUTPUT 0x0300015b

+#define HFI_PROP_PICTURE_TYPE 0x03000162
+
#define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168

#define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169

+#define HFI_PROP_NO_OUTPUT 0x0300016a
+
+#define HFI_PROP_DPB_LIST 0x0300017A
+
+#define HFI_PROP_UBWC_STRIDE_SCANLINE 0x03000190
+
#define HFI_PROP_COMV_BUFFER_COUNT 0x03000193

#define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index 8f1e456..d15ce5a 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -219,6 +219,44 @@ int iris_hfi_session_open(struct iris_inst *inst)
return ret;
}

+int iris_hfi_session_subscribe_mode(struct iris_inst *inst,
+ u32 cmd, u32 plane, u32 payload_type,
+ void *payload, u32 payload_size)
+{
+ struct iris_core *core;
+ int ret;
+
+ if (!inst->packet)
+ return -EINVAL;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ ret = hfi_packet_session_command(inst,
+ cmd,
+ (HFI_HOST_FLAGS_RESPONSE_REQUIRED |
+ HFI_HOST_FLAGS_INTR_REQUIRED),
+ get_hfi_port(plane),
+ inst->session_id,
+ payload_type,
+ payload,
+ payload_size);
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
int iris_hfi_session_close(struct iris_inst *inst)
{
struct iris_core *core;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
index ca2339e..d6b3fca 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
@@ -13,6 +13,9 @@ int iris_hfi_core_init(struct iris_core *core);
int iris_hfi_core_deinit(struct iris_core *core);
int iris_hfi_session_open(struct iris_inst *inst);
int iris_hfi_session_close(struct iris_inst *inst);
+int iris_hfi_session_subscribe_mode(struct iris_inst *inst,
+ u32 cmd, u32 plane, u32 payload_type,
+ void *payload, u32 payload_size);
int iris_hfi_set_property(struct iris_inst *inst,
u32 packet_type, u32 flag, u32 plane, u32 payload_type,
void *payload, u32 payload_size);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
index a3544d8..d4cdfcf 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
@@ -34,6 +34,24 @@ u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type)
return hfi_port;
}

+u32 get_hfi_port(u32 plane)
+{
+ u32 hfi_port = HFI_PORT_NONE;
+
+ switch (plane) {
+ case INPUT_MPLANE:
+ hfi_port = HFI_PORT_BITSTREAM;
+ break;
+ case OUTPUT_MPLANE:
+ hfi_port = HFI_PORT_RAW;
+ break;
+ default:
+ break;
+ }
+
+ return hfi_port;
+}
+
static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type)
{
switch (buffer_type) {
@@ -58,6 +76,30 @@ static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type)
}
}

+u32 get_hfi_colorformat(u32 colorformat)
+{
+ u32 hfi_colorformat = HFI_COLOR_FMT_NV12_UBWC;
+
+ switch (colorformat) {
+ case V4L2_PIX_FMT_NV12:
+ hfi_colorformat = HFI_COLOR_FMT_NV12;
+ break;
+ case V4L2_PIX_FMT_QC08C:
+ hfi_colorformat = HFI_COLOR_FMT_NV12_UBWC;
+ break;
+ case V4L2_PIX_FMT_QC10C:
+ hfi_colorformat = HFI_COLOR_FMT_TP10_UBWC;
+ break;
+ case V4L2_PIX_FMT_NV21:
+ hfi_colorformat = HFI_COLOR_FMT_NV21;
+ break;
+ default:
+ break;
+ }
+
+ return hfi_colorformat;
+}
+
int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf)
{
memset(buf, 0, sizeof(*buf));
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
index bf18553..4276d6d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
@@ -81,6 +81,9 @@ enum hfi_packet_port_type {

u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type);
int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf);
+u32 get_hfi_colorformat(u32 colorformat);
+u32 get_hfi_port(u32 plane);
+
int hfi_packet_sys_init(struct iris_core *core,
u8 *pkt, u32 pkt_size);
int hfi_packet_image_version(struct iris_core *core,
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index 64067d5..7d16c96 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -5,10 +5,13 @@

#include <media/v4l2-event.h>

+#include "hfi_defines.h"
#include "iris_buffer.h"
#include "iris_common.h"
#include "iris_ctrls.h"
#include "iris_helpers.h"
+#include "iris_hfi.h"
+#include "iris_hfi_packet.h"
#include "iris_vdec.h"

static int vdec_codec_change(struct iris_inst *inst, u32 v4l2_codec)
@@ -329,3 +332,154 @@ int vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscri

return ret;
}
+
+int vdec_subscribe_property(struct iris_inst *inst, u32 plane)
+{
+ const u32 *subcribe_prop = NULL;
+ u32 subscribe_prop_size = 0;
+ struct iris_core *core;
+ u32 payload[32] = {0};
+ u32 i;
+
+ core = inst->core;
+
+ payload[0] = HFI_MODE_PROPERTY;
+
+ if (plane == INPUT_MPLANE) {
+ subscribe_prop_size = core->platform_data->dec_input_prop_size;
+ subcribe_prop = core->platform_data->dec_input_prop;
+ } else if (plane == OUTPUT_MPLANE) {
+ if (inst->codec == H264) {
+ subscribe_prop_size = core->platform_data->dec_output_prop_size_avc;
+ subcribe_prop = core->platform_data->dec_output_prop_avc;
+ } else if (inst->codec == HEVC) {
+ subscribe_prop_size = core->platform_data->dec_output_prop_size_hevc;
+ subcribe_prop = core->platform_data->dec_output_prop_hevc;
+ } else if (inst->codec == VP9) {
+ subscribe_prop_size = core->platform_data->dec_output_prop_size_vp9;
+ subcribe_prop = core->platform_data->dec_output_prop_vp9;
+ } else {
+ return -EINVAL;
+ }
+ } else {
+ return -EINVAL;
+ }
+
+ for (i = 0; i < subscribe_prop_size; i++)
+ payload[i + 1] = subcribe_prop[i];
+
+ return iris_hfi_session_subscribe_mode(inst,
+ HFI_CMD_SUBSCRIBE_MODE,
+ plane,
+ HFI_PAYLOAD_U32_ARRAY,
+ &payload[0],
+ (subscribe_prop_size + 1) * sizeof(u32));
+}
+
+static int vdec_set_colorformat(struct iris_inst *inst)
+{
+ u32 hfi_colorformat;
+ u32 pixelformat;
+
+ pixelformat = inst->fmt_dst->fmt.pix_mp.pixelformat;
+ hfi_colorformat = get_hfi_colorformat(pixelformat);
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_COLOR_FORMAT,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(OUTPUT_MPLANE),
+ HFI_PAYLOAD_U32,
+ &hfi_colorformat,
+ sizeof(u32));
+}
+
+static int vdec_set_linear_stride_scanline(struct iris_inst *inst)
+{
+ u32 stride_y, scanline_y, stride_uv, scanline_uv;
+ u32 pixelformat;
+ u32 payload[2];
+
+ pixelformat = inst->fmt_dst->fmt.pix_mp.pixelformat;
+
+ if (!is_linear_colorformat(pixelformat))
+ return 0;
+
+ stride_y = inst->fmt_dst->fmt.pix_mp.width;
+ scanline_y = inst->fmt_dst->fmt.pix_mp.height;
+ stride_uv = stride_y;
+ scanline_uv = scanline_y / 2;
+
+ payload[0] = stride_y << 16 | scanline_y;
+ payload[1] = stride_uv << 16 | scanline_uv;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_LINEAR_STRIDE_SCANLINE,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(OUTPUT_MPLANE),
+ HFI_PAYLOAD_U64,
+ &payload,
+ sizeof(u64));
+}
+
+static int vdec_set_ubwc_stride_scanline(struct iris_inst *inst)
+{
+ u32 meta_stride_y, meta_scanline_y, meta_stride_uv, meta_scanline_uv;
+ u32 stride_y, scanline_y, stride_uv, scanline_uv;
+ u32 pix_fmt, width, height;
+ u32 payload[4];
+
+ pix_fmt = inst->fmt_dst->fmt.pix_mp.pixelformat;
+ width = inst->fmt_dst->fmt.pix_mp.width;
+ height = inst->fmt_dst->fmt.pix_mp.height;
+
+ if (is_linear_colorformat(pix_fmt))
+ return 0;
+
+ if (pix_fmt == V4L2_PIX_FMT_QC08C) {
+ stride_y = ALIGN(width, 128);
+ scanline_y = ALIGN(height, 32);
+ stride_uv = ALIGN(width, 128);
+ scanline_uv = ALIGN((height + 1) >> 1, 32);
+ meta_stride_y = ALIGN(DIV_ROUND_UP(width, 32), 64);
+ meta_scanline_y = ALIGN(DIV_ROUND_UP(height, 8), 16);
+ meta_stride_uv = ALIGN(DIV_ROUND_UP((width + 1) >> 1, 16), 64);
+ meta_scanline_uv = ALIGN(DIV_ROUND_UP((height + 1) >> 1, 8), 16);
+ } else {
+ stride_y = ALIGN(ALIGN(width, 192) * 4 / 3, 256);
+ scanline_y = ALIGN(height, 16);
+ stride_uv = ALIGN(ALIGN(width, 192) * 4 / 3, 256);
+ scanline_uv = ALIGN((height + 1) >> 1, 16);
+ meta_stride_y = ALIGN(DIV_ROUND_UP(width, 48), 64);
+ meta_scanline_y = ALIGN(DIV_ROUND_UP(height, 4), 16);
+ meta_stride_uv = ALIGN(DIV_ROUND_UP((width + 1) >> 1, 24), 64);
+ meta_scanline_uv = ALIGN(DIV_ROUND_UP((height + 1) >> 1, 4), 16);
+ }
+
+ payload[0] = stride_y << 16 | scanline_y;
+ payload[1] = stride_uv << 16 | scanline_uv;
+ payload[2] = meta_stride_y << 16 | meta_scanline_y;
+ payload[3] = meta_stride_uv << 16 | meta_scanline_uv;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_UBWC_STRIDE_SCANLINE,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(OUTPUT_MPLANE),
+ HFI_PAYLOAD_U32_ARRAY,
+ &payload[0],
+ sizeof(u32) * 4);
+}
+
+int vdec_set_output_property(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = vdec_set_colorformat(inst);
+ if (ret)
+ return ret;
+
+ ret = vdec_set_linear_stride_scanline(inst);
+ if (ret)
+ return ret;
+
+ return vdec_set_ubwc_stride_scanline(inst);
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
index a2f159d..6b0306c 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
@@ -14,5 +14,7 @@ int vdec_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f);
int vdec_try_fmt(struct iris_inst *inst, struct v4l2_format *f);
int vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f);
int vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscription *sub);
+int vdec_subscribe_property(struct iris_inst *inst, u32 plane);
+int vdec_set_output_property(struct iris_inst *inst);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
index abd11fa..fc12bde 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
@@ -206,6 +206,14 @@ struct platform_data {
u32 core_data_size;
struct plat_inst_cap *inst_cap_data;
u32 inst_cap_data_size;
+ const u32 *dec_input_prop;
+ unsigned int dec_input_prop_size;
+ const u32 *dec_output_prop_avc;
+ unsigned int dec_output_prop_size_avc;
+ const u32 *dec_output_prop_hevc;
+ unsigned int dec_output_prop_size_hevc;
+ const u32 *dec_output_prop_vp9;
+ unsigned int dec_output_prop_size_vp9;
};

int init_platform(struct iris_core *core);
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
index 85bc677..6a4bfa3 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
@@ -382,6 +382,26 @@ static struct format_capability format_data_sm8550 = {
.color_format_info_size = ARRAY_SIZE(color_format_data_sm8550),
};

+static const u32 sm8550_vdec_input_properties[] = {
+ HFI_PROP_NO_OUTPUT,
+};
+
+static const u32 sm8550_vdec_output_properties_avc[] = {
+ HFI_PROP_PICTURE_TYPE,
+ HFI_PROP_DPB_LIST,
+ HFI_PROP_CABAC_SESSION,
+};
+
+static const u32 sm8550_vdec_output_properties_hevc[] = {
+ HFI_PROP_PICTURE_TYPE,
+ HFI_PROP_DPB_LIST,
+};
+
+static const u32 sm8550_vdec_output_properties_vp9[] = {
+ HFI_PROP_PICTURE_TYPE,
+ HFI_PROP_DPB_LIST,
+};
+
struct platform_data sm8550_data = {
.bus_tbl = sm8550_bus_table,
.bus_tbl_size = ARRAY_SIZE(sm8550_bus_table),
@@ -409,4 +429,13 @@ struct platform_data sm8550_data = {
.inst_cap_data_size = ARRAY_SIZE(instance_cap_data_sm8550),
.ubwc_config = ubwc_config_sm8550,
.format_data = &format_data_sm8550,
+
+ .dec_input_prop = sm8550_vdec_input_properties,
+ .dec_input_prop_size = ARRAY_SIZE(sm8550_vdec_input_properties),
+ .dec_output_prop_avc = sm8550_vdec_output_properties_avc,
+ .dec_output_prop_size_avc = ARRAY_SIZE(sm8550_vdec_output_properties_avc),
+ .dec_output_prop_hevc = sm8550_vdec_output_properties_hevc,
+ .dec_output_prop_size_hevc = ARRAY_SIZE(sm8550_vdec_output_properties_hevc),
+ .dec_output_prop_vp9 = sm8550_vdec_output_properties_vp9,
+ .dec_output_prop_size_vp9 = ARRAY_SIZE(sm8550_vdec_output_properties_vp9),
};
--
2.7.4

2023-12-18 11:40:08

[permalink] [raw]

Subject: [PATCH v2 19/34] media: iris: implement HFI to queue and release buffers

Introduce and implement HFIs to queue and release
all input, output and internal buffers to/from firmware.

HFI_CMD_BUFFER - to submit the buffer to firmware.
HFI_CMD_BUFFER with HFI_BUF_HOST_FLAG_RELEASE - to request
firmware to release the buffer.

Signed-off-by: Dikshita Agarwal <[email protected]>
---
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 25 +++++++
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 85 ++++++++++++++++++++++
drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 4 +
.../platform/qcom/vcodec/iris/iris_hfi_packet.c | 78 ++++++++++++++++++++
.../platform/qcom/vcodec/iris/iris_hfi_packet.h | 15 ++++
5 files changed, 207 insertions(+)

diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index 9dd5f11..c044f78 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -13,6 +13,7 @@
#define HFI_CMD_INIT 0x01000001
#define HFI_CMD_OPEN 0x01000003
#define HFI_CMD_CLOSE 0x01000004
+#define HFI_CMD_BUFFER 0x01000009

#define HFI_PROP_IMAGE_VERSION 0x03000001

@@ -60,4 +61,28 @@ struct hfi_debug_header {
u32 reserved[2];
};

+enum hfi_buffer_type {
+ HFI_BUFFER_BITSTREAM = 0x00000001,
+ HFI_BUFFER_RAW = 0x00000002,
+ HFI_BUFFER_METADATA = 0x00000003,
+ HFI_BUFFER_SUBCACHE = 0x00000004,
+ HFI_BUFFER_PARTIAL_DATA = 0x00000005,
+ HFI_BUFFER_DPB = 0x00000006,
+ HFI_BUFFER_BIN = 0x00000007,
+ HFI_BUFFER_LINE = 0x00000008,
+ HFI_BUFFER_ARP = 0x00000009,
+ HFI_BUFFER_COMV = 0x0000000A,
+ HFI_BUFFER_NON_COMV = 0x0000000B,
+ HFI_BUFFER_PERSIST = 0x0000000C,
+ HFI_BUFFER_VPSS = 0x0000000D,
+};
+
+enum hfi_buffer_host_flags {
+ HFI_BUF_HOST_FLAG_NONE = 0x00000000,
+ HFI_BUF_HOST_FLAG_RELEASE = 0x00000001,
+ HFI_BUF_HOST_FLAG_READONLY = 0x00000010,
+ HFI_BUF_HOST_FLAG_CODEC_CONFIG = 0x00000100,
+ HFI_BUF_HOST_FLAGS_CB_NON_SECURE = 0x00000200,
+};
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index 24ddb98..8f1e456 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -309,3 +309,88 @@ int iris_hfi_set_property(struct iris_inst *inst,

return ret;
}
+
+int iris_hfi_queue_buffer(struct iris_inst *inst,
+ struct iris_buffer *buffer)
+{
+ struct hfi_buffer hfi_buffer;
+ struct iris_core *core;
+ int ret;
+
+ if (!inst->packet)
+ return -EINVAL;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ ret = get_hfi_buffer(buffer, &hfi_buffer);
+ if (ret)
+ goto unlock;
+
+ ret = hfi_packet_session_command(inst,
+ HFI_CMD_BUFFER,
+ HFI_HOST_FLAGS_INTR_REQUIRED,
+ get_hfi_port_from_buffer_type(buffer->type),
+ inst->session_id,
+ HFI_PAYLOAD_STRUCTURE,
+ &hfi_buffer,
+ sizeof(hfi_buffer));
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+int iris_hfi_release_buffer(struct iris_inst *inst,
+ struct iris_buffer *buffer)
+{
+ struct hfi_buffer hfi_buffer;
+ struct iris_core *core;
+ int ret;
+
+ if (!inst->packet || !buffer)
+ return -EINVAL;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ ret = get_hfi_buffer(buffer, &hfi_buffer);
+ if (ret)
+ goto unlock;
+
+ hfi_buffer.flags |= HFI_BUF_HOST_FLAG_RELEASE;
+
+ ret = hfi_packet_session_command(inst,
+ HFI_CMD_BUFFER,
+ (HFI_HOST_FLAGS_RESPONSE_REQUIRED |
+ HFI_HOST_FLAGS_INTR_REQUIRED),
+ get_hfi_port_from_buffer_type(buffer->type),
+ inst->session_id,
+ HFI_PAYLOAD_STRUCTURE,
+ &hfi_buffer,
+ sizeof(hfi_buffer));
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
index bf991bb..4aefdc4 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
@@ -19,5 +19,9 @@ int iris_hfi_set_property(struct iris_inst *inst,

irqreturn_t iris_hfi_isr(int irq, void *data);
irqreturn_t iris_hfi_isr_handler(int irq, void *data);
+int iris_hfi_queue_buffer(struct iris_inst *inst,
+ struct iris_buffer *buffer);
+int iris_hfi_release_buffer(struct iris_inst *inst,
+ struct iris_buffer *buffer);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
index 749d978..a3544d8 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
@@ -4,9 +4,87 @@
*/

#include "iris_core.h"
+#include "iris_helpers.h"
#include "iris_hfi_packet.h"
#include "hfi_defines.h"

+u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type)
+{
+ u32 hfi_port = HFI_PORT_NONE;
+
+ switch (buffer_type) {
+ case BUF_INPUT:
+ case BUF_BIN:
+ case BUF_COMV:
+ case BUF_NON_COMV:
+ case BUF_LINE:
+ hfi_port = HFI_PORT_BITSTREAM;
+ break;
+ case BUF_OUTPUT:
+ case BUF_DPB:
+ hfi_port = HFI_PORT_RAW;
+ break;
+ case BUF_PERSIST:
+ hfi_port = HFI_PORT_NONE;
+ break;
+ default:
+ break;
+ }
+
+ return hfi_port;
+}
+
+static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type)
+{
+ switch (buffer_type) {
+ case BUF_INPUT:
+ return HFI_BUFFER_BITSTREAM;
+ case BUF_OUTPUT:
+ return HFI_BUFFER_RAW;
+ case BUF_BIN:
+ return HFI_BUFFER_BIN;
+ case BUF_COMV:
+ return HFI_BUFFER_COMV;
+ case BUF_NON_COMV:
+ return HFI_BUFFER_NON_COMV;
+ case BUF_LINE:
+ return HFI_BUFFER_LINE;
+ case BUF_DPB:
+ return HFI_BUFFER_DPB;
+ case BUF_PERSIST:
+ return HFI_BUFFER_PERSIST;
+ default:
+ return 0;
+ }
+}
+
+int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf)
+{
+ memset(buf, 0, sizeof(*buf));
+ buf->type = hfi_buf_type_from_driver(buffer->type);
+ buf->index = buffer->index;
+ buf->base_address = buffer->device_addr;
+ buf->addr_offset = 0;
+ buf->buffer_size = buffer->buffer_size;
+ /*
+ * for decoder input buffers, firmware (BSE HW) needs 256 aligned
+ * buffer size otherwise it will truncate or ignore the data after 256
+ * aligned size which may lead to error concealment
+ */
+ if (buffer->type == BUF_INPUT)
+ buf->buffer_size = ALIGN(buffer->buffer_size, 256);
+ buf->data_offset = buffer->data_offset;
+ buf->data_size = buffer->data_size;
+ if (buffer->attr & BUF_ATTR_READ_ONLY)
+ buf->flags |= HFI_BUF_HOST_FLAG_READONLY;
+ if (buffer->attr & BUF_ATTR_PENDING_RELEASE)
+ buf->flags |= HFI_BUF_HOST_FLAG_RELEASE;
+ buf->flags |= HFI_BUF_HOST_FLAGS_CB_NON_SECURE;
+ buf->timestamp = buffer->timestamp;
+
+ return 0;
+}
+
static int hfi_create_header(u8 *packet, u32 packet_size, u32 session_id,
u32 header_id)
{
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
index bea7ed9..bf18553 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
@@ -27,6 +27,19 @@ struct hfi_packet {
u32 reserved[2];
};

+struct hfi_buffer {
+ u32 type;
+ u32 index;
+ u64 base_address;
+ u32 addr_offset;
+ u32 buffer_size;
+ u32 data_offset;
+ u32 data_size;
+ u64 timestamp;
+ u32 flags;
+ u32 reserved[5];
+};
+
enum hfi_packet_host_flags {
HFI_HOST_FLAGS_NONE = 0x00000000,
HFI_HOST_FLAGS_INTR_REQUIRED = 0x00000001,
@@ -66,6 +79,8 @@ enum hfi_packet_port_type {
HFI_PORT_RAW = 0x00000002,
};

+u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type);
+int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf);
int hfi_packet_sys_init(struct iris_core *core,
u8 *pkt, u32 pkt_size);
int hfi_packet_image_version(struct iris_core *core,
--
2.7.4

2023-12-18 11:45:24

[permalink] [raw]

Subject: [PATCH v2 34/34] media: iris: add power management for encoder

Hardware specific power sequence include programming specific
sequence of registers. Implements this sequence for iris3.

Also, implement iris3 Encoder specific calculation for clock
and bus bandwidth which depends on hardware configuration,
codec format, resolution and frame rate.

Signed-off-by: Dikshita Agarwal <[email protected]>
---
.../media/platform/qcom/vcodec/iris/iris_ctrls.c | 2 +
.../media/platform/qcom/vcodec/iris/iris_power.c | 15 ++--
.../platform/qcom/vcodec/iris/platform_common.h | 3 +
.../platform/qcom/vcodec/iris/platform_sm8550.c | 14 +++
.../platform/qcom/vcodec/iris/vpu_iris3_power.c | 100 +++++++++++++++------
5 files changed, 103 insertions(+), 31 deletions(-)

diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
index 559b0dd..1a2e305 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
@@ -799,6 +799,8 @@ int decide_quality_mode(struct iris_inst *inst)
if (mbpf <= max_hq_mbpf && mbps <= max_hq_mbps)
mode = MAX_QUALITY_MODE;

+ inst->cap[QUALITY_MODE].value = mode;
+
return mode;
}

diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_power.c b/drivers/media/platform/qcom/vcodec/iris/iris_power.c
index 77439e3..be71751 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_power.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_power.c
@@ -48,11 +48,16 @@ static int iris_vote_buses(struct iris_inst *inst)
vote_data->height = inp_f->fmt.pix_mp.height;
vote_data->fps = inst->max_rate;

- if (is_linear_colorformat(out_f->fmt.pix_mp.pixelformat)) {
- vote_data->color_formats[0] = V4L2_PIX_FMT_NV12;
- vote_data->color_formats[1] = out_f->fmt.pix_mp.pixelformat;
- } else {
- vote_data->color_formats[0] = out_f->fmt.pix_mp.pixelformat;
+ if (inst->domain == ENCODER) {
+ vote_data->color_formats[0] =
+ v4l2_colorformat_to_driver(inst, inst->fmt_src->fmt.pix_mp.pixelformat);
+ } else if (inst->domain == DECODER) {
+ if (is_linear_colorformat(out_f->fmt.pix_mp.pixelformat)) {
+ vote_data->color_formats[0] = V4L2_PIX_FMT_NV12;
+ vote_data->color_formats[1] = out_f->fmt.pix_mp.pixelformat;
+ } else {
+ vote_data->color_formats[0] = out_f->fmt.pix_mp.pixelformat;
+ }
}

call_session_op(core, calc_bw, inst, vote_data);
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
index 42ac662..e9b5c99 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
@@ -174,6 +174,7 @@ enum plat_inst_cap_type {
INPUT_BUF_HOST_MAX_COUNT,
STAGE,
PIPE,
+ QUALITY_MODE,
POC,
CODED_FRAMES,
BIT_DEPTH,
@@ -279,6 +280,8 @@ struct format_capability {
struct platform_data {
const struct bus_info *bus_tbl;
unsigned int bus_tbl_size;
+ const struct bw_info *bw_tbl_enc;
+ unsigned int bw_tbl_enc_size;
const struct bw_info *bw_tbl_dec;
unsigned int bw_tbl_dec_size;
const char * const *pd_tbl;
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
index 17c5856..d47c985 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
@@ -1005,6 +1005,11 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
set_pipe},

+ {QUALITY_MODE, ENC, CODECS_ALL,
+ MAX_QUALITY_MODE,
+ POWER_SAVE_MODE, 1,
+ POWER_SAVE_MODE},
+
{POC, DEC, H264, 0, 2, 1, 1,
0,
HFI_PROP_PIC_ORDER_CNT_TYPE},
@@ -1051,6 +1056,13 @@ static const char * const sm8550_pd_table[] = { "iris-ctl", "vcodec", NULL };

static const char * const sm8550_opp_pd_table[] = { "mxc", "mmcx", NULL };

+static const struct bw_info sm8550_bw_table_enc[] = {
+ { 1944000, 1491000, 2693000 }, /* 3840x2160@60 */
+ { 972000, 765000, 1366000 }, /* 3840x2160@30 */
+ { 489600, 557000, 780000 }, /* 1920x1080@60 */
+ { 244800, 288000, 399000 }, /* 1920x1080@30 */
+};
+
static const struct bw_info sm8550_bw_table_dec[] = {
{ 2073600, 1608000, 2742000 }, /* 4096x2160@60 */
{ 1036800, 826000, 1393000 }, /* 4096x2160@30 */
@@ -1133,6 +1145,8 @@ struct platform_data sm8550_data = {
.clk_rst_tbl = sm8550_clk_reset_table,
.clk_rst_tbl_size = ARRAY_SIZE(sm8550_clk_reset_table),

+ .bw_tbl_enc = sm8550_bw_table_enc,
+ .bw_tbl_enc_size = ARRAY_SIZE(sm8550_bw_table_enc),
.bw_tbl_dec = sm8550_bw_table_dec,
.bw_tbl_dec_size = ARRAY_SIZE(sm8550_bw_table_dec),

diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
index 2828227..8497b84 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
@@ -11,12 +11,13 @@

u64 iris_calc_freq_iris3(struct iris_inst *inst, u32 data_size)
{
+ u32 operating_rate, vsp_factor_num = 1, vsp_factor_den = 1;
u64 vsp_cycles = 0, vpp_cycles = 0, fw_cycles = 0;
u64 fw_vpp_cycles = 0, bitrate = 0, freq = 0;
+ u32 vpp_cycles_per_mb, mbs_per_second;
u32 base_cycles = 0, fps, mbpf;
u32 height = 0, width = 0;
struct v4l2_format *inp_f;
- u32 mbs_per_second;

inp_f = inst->fmt_src;
width = max(inp_f->fmt.pix_mp.width, inst->crop.width);
@@ -29,32 +30,74 @@ u64 iris_calc_freq_iris3(struct iris_inst *inst, u32 data_size)
fw_cycles = fps * inst->cap[MB_CYCLES_FW].value;
fw_vpp_cycles = fps * inst->cap[MB_CYCLES_FW_VPP].value;

- vpp_cycles = mbs_per_second * inst->cap[MB_CYCLES_VPP].value /
- inst->cap[PIPE].value;
- vpp_cycles += max(vpp_cycles / 20, fw_vpp_cycles);
-
- if (inst->cap[PIPE].value > 1)
- vpp_cycles += div_u64(vpp_cycles * 59, 1000);
-
- base_cycles = inst->cap[MB_CYCLES_VSP].value;
- bitrate = fps * data_size * 8;
- vsp_cycles = bitrate;
-
- if (inst->codec == VP9) {
- vsp_cycles = div_u64(vsp_cycles * 170, 100);
- } else if (inst->cap[ENTROPY_MODE].value ==
- V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC) {
- vsp_cycles = div_u64(vsp_cycles * 135, 100);
- } else {
- base_cycles = 0;
- vsp_cycles = div_u64(vsp_cycles, 2);
- }
- vsp_cycles = div_u64(vsp_cycles * 21, 20);
+ if (inst->domain == ENCODER) {
+ vpp_cycles_per_mb =
+ inst->cap[QUALITY_MODE].value == POWER_SAVE_MODE ?
+ inst->cap[MB_CYCLES_LP].value :
+ inst->cap[MB_CYCLES_VPP].value;
+
+ vpp_cycles = mbs_per_second * vpp_cycles_per_mb / inst->cap[PIPE].value;
+
+ if (inst->cap[B_FRAME].value > 1)
+ vpp_cycles += (vpp_cycles / 4) + (vpp_cycles / 8);
+ else if (inst->cap[B_FRAME].value)
+ vpp_cycles += vpp_cycles / 4;
+
+ vpp_cycles += max(div_u64(vpp_cycles, 20), fw_vpp_cycles);
+ if (inst->cap[PIPE].value > 1)
+ vpp_cycles += div_u64(vpp_cycles, 100);
+
+ operating_rate = inst->cap[OPERATING_RATE].value >> 16;
+ if (operating_rate > (inst->cap[FRAME_RATE].value >> 16) &&
+ (inst->cap[FRAME_RATE].value >> 16)) {
+ vsp_factor_num = operating_rate;
+ vsp_factor_den = inst->cap[FRAME_RATE].value >> 16;
+ }
+ vsp_cycles = div_u64(((u64)inst->cap[BIT_RATE].value * vsp_factor_num),
+ vsp_factor_den);
+
+ base_cycles = inst->cap[MB_CYCLES_VSP].value;
+ if (inst->cap[ENTROPY_MODE].value ==
+ V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC) {
+ vsp_cycles = div_u64(vsp_cycles * 135, 100);
+ } else {
+ base_cycles = 0;
+ vsp_cycles = div_u64(vsp_cycles, 2);
+ }
+ vsp_cycles = div_u64(vsp_cycles * 21, 20);
+
+ if (inst->cap[STAGE].value == STAGE_1)
+ vsp_cycles = vsp_cycles * 3;

- if (inst->cap[STAGE].value == STAGE_1)
- vsp_cycles = vsp_cycles * 3;
+ vsp_cycles += mbs_per_second * base_cycles;
+ } else if (inst->domain == DECODER) {
+ vpp_cycles = mbs_per_second * inst->cap[MB_CYCLES_VPP].value /
+ inst->cap[PIPE].value;
+ vpp_cycles += max(vpp_cycles / 20, fw_vpp_cycles);
+
+ if (inst->cap[PIPE].value > 1)
+ vpp_cycles += div_u64(vpp_cycles * 59, 1000);
+
+ base_cycles = inst->cap[MB_CYCLES_VSP].value;
+ bitrate = fps * data_size * 8;
+ vsp_cycles = bitrate;
+
+ if (inst->codec == VP9) {
+ vsp_cycles = div_u64(vsp_cycles * 170, 100);
+ } else if (inst->cap[ENTROPY_MODE].value ==
+ V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC) {
+ vsp_cycles = div_u64(vsp_cycles * 135, 100);
+ } else {
+ base_cycles = 0;
+ vsp_cycles = div_u64(vsp_cycles, 2);
+ }
+ vsp_cycles = div_u64(vsp_cycles * 21, 20);
+
+ if (inst->cap[STAGE].value == STAGE_1)
+ vsp_cycles = vsp_cycles * 3;

vsp_cycles += mbs_per_second * base_cycles;
+ }

freq = max3(vpp_cycles, vsp_cycles, fw_cycles);

@@ -78,8 +121,13 @@ int iris_calc_bw_iris3(struct iris_inst *inst, struct bus_vote_data *data)
if (mbps == 0)
return 0;

- bw_tbl = core->platform_data->bw_tbl_dec;
- num_rows = core->platform_data->bw_tbl_dec_size;
+ if (inst->domain == DECODER) {
+ bw_tbl = core->platform_data->bw_tbl_dec;
+ num_rows = core->platform_data->bw_tbl_dec_size;
+ } else if (inst->domain == ENCODER) {
+ bw_tbl = core->platform_data->bw_tbl_enc;
+ num_rows = core->platform_data->bw_tbl_enc_size;
+ }

if (!bw_tbl || num_rows == 0)
return 0;
--
2.7.4

2023-12-18 11:45:46

[permalink] [raw]

Subject: [PATCH v2 22/34] media: iris: introduce instance states

Introduce different states for instances. State transitions
are defined, based on which they would be allowed/rejected/ignored.

IRIS_INST_OPEN - video instance is opened.
IRIS_INST_INPUT_STREAMING - stream on is completed on output plane.
IRIS_INST_OUTPUT_STREAMING - stream on is completed on capture
plane.
IRIS_INST_STREAMING - stream on is completed on both output and
capture planes.
IRIS_INST_CLOSE - video instance is closed.
IRIS_INST_ERROR - error state.

|
v
-------------
+---------| OPEN |--------- +
| ------------- |
| ^ ^ |
| / \ |
| / \ |
| v v |
| ----------- ----------- |
| | INPUT OUTPUT | |
|---| STREAMING STREAMING |---|
| ----------- ----------- |
| ^ ^ |
| \ / |
| \ / |
| v v |
| ------------- |
|--------| STREAMING |-----------|
| ------------- |
| | |
| | |
| v |
| ----------- |
+-------->| CLOSE |<----------+
| ----------- |
| | |
| | |
| v |
| ---------- |
+-------->| ERROR |<-----------+
----------

Signed-off-by: Dikshita Agarwal <[email protected]>
---
.../platform/qcom/vcodec/iris/iris_instance.h | 2 +
.../media/platform/qcom/vcodec/iris/iris_state.c | 86 ++++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_state.h | 22 ++++++
drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 3 +
.../media/platform/qcom/vcodec/iris/iris_vidc.c | 5 ++
5 files changed, 118 insertions(+)

diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
index 5d4c856..275efa8 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -37,6 +37,7 @@
* @codec: codec type
* @buffers: structure of buffer info
* @fw_min_count: minimnum count of buffers needed by fw
+ * @state: instance state
*/

struct iris_inst {
@@ -60,6 +61,7 @@ struct iris_inst {
enum codec_type codec;
struct iris_buffers_info buffers;
u32 fw_min_count;
+ enum iris_inst_state state;
};

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
index 83bbc6b..9bf79a0 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_state.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
@@ -69,3 +69,89 @@ int iris_change_core_state(struct iris_core *core,

return ret;
}
+
+struct iris_inst_state_change_allow {
+ enum iris_inst_state from;
+ enum iris_inst_state to;
+ enum state_change allow;
+};
+
+static enum state_change iris_allow_inst_state_change(struct iris_inst *inst,
+ enum iris_inst_state req_state)
+{
+ enum state_change allow = STATE_CHANGE_ALLOW;
+ int cnt;
+ static struct iris_inst_state_change_allow state[] = {
+ /* from, to, allow */
+ {IRIS_INST_OPEN, IRIS_INST_OPEN, STATE_CHANGE_IGNORE },
+ {IRIS_INST_OPEN, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_ALLOW },
+ {IRIS_INST_OPEN, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_ALLOW },
+ {IRIS_INST_OPEN, IRIS_INST_STREAMING, STATE_CHANGE_DISALLOW },
+ {IRIS_INST_OPEN, IRIS_INST_CLOSE, STATE_CHANGE_ALLOW },
+ {IRIS_INST_OPEN, IRIS_INST_ERROR, STATE_CHANGE_ALLOW },
+
+ {IRIS_INST_INPUT_STREAMING, IRIS_INST_OPEN, STATE_CHANGE_ALLOW },
+ {IRIS_INST_INPUT_STREAMING, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_IGNORE },
+ {IRIS_INST_INPUT_STREAMING, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_DISALLOW },
+ {IRIS_INST_INPUT_STREAMING, IRIS_INST_STREAMING, STATE_CHANGE_ALLOW },
+ {IRIS_INST_INPUT_STREAMING, IRIS_INST_CLOSE, STATE_CHANGE_ALLOW },
+ {IRIS_INST_INPUT_STREAMING, IRIS_INST_ERROR, STATE_CHANGE_ALLOW },
+
+ {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_OPEN, STATE_CHANGE_ALLOW },
+ {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_DISALLOW },
+ {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_IGNORE },
+ {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_STREAMING, STATE_CHANGE_ALLOW },
+ {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_CLOSE, STATE_CHANGE_ALLOW },
+ {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_ERROR, STATE_CHANGE_ALLOW },
+
+ {IRIS_INST_STREAMING, IRIS_INST_OPEN, STATE_CHANGE_DISALLOW },
+ {IRIS_INST_STREAMING, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_ALLOW },
+ {IRIS_INST_STREAMING, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_ALLOW },
+ {IRIS_INST_STREAMING, IRIS_INST_STREAMING, STATE_CHANGE_IGNORE },
+ {IRIS_INST_STREAMING, IRIS_INST_CLOSE, STATE_CHANGE_ALLOW },
+ {IRIS_INST_STREAMING, IRIS_INST_ERROR, STATE_CHANGE_ALLOW },
+
+ {IRIS_INST_CLOSE, IRIS_INST_OPEN, STATE_CHANGE_DISALLOW },
+ {IRIS_INST_CLOSE, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_DISALLOW },
+ {IRIS_INST_CLOSE, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_DISALLOW },
+ {IRIS_INST_CLOSE, IRIS_INST_STREAMING, STATE_CHANGE_DISALLOW },
+ {IRIS_INST_CLOSE, IRIS_INST_CLOSE, STATE_CHANGE_IGNORE },
+ {IRIS_INST_CLOSE, IRIS_INST_ERROR, STATE_CHANGE_IGNORE },
+
+ {IRIS_INST_ERROR, IRIS_INST_OPEN, STATE_CHANGE_IGNORE },
+ {IRIS_INST_ERROR, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_IGNORE },
+ {IRIS_INST_ERROR, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_IGNORE },
+ {IRIS_INST_ERROR, IRIS_INST_STREAMING, STATE_CHANGE_IGNORE },
+ {IRIS_INST_ERROR, IRIS_INST_CLOSE, STATE_CHANGE_IGNORE },
+ {IRIS_INST_ERROR, IRIS_INST_ERROR, STATE_CHANGE_IGNORE },
+ };
+
+ for (cnt = 0; cnt < ARRAY_SIZE(state); cnt++) {
+ if (state[cnt].from == inst->state && state[cnt].to == req_state) {
+ allow = state[cnt].allow;
+ break;
+ }
+ }
+
+ return allow;
+}
+
+int iris_inst_change_state(struct iris_inst *inst,
+ enum iris_inst_state request_state)
+{
+ enum state_change allow;
+
+ if (IS_SESSION_ERROR(inst))
+ return 0;
+
+ if (inst->state == request_state)
+ return 0;
+
+ allow = iris_allow_inst_state_change(inst, request_state);
+ if (allow != STATE_CHANGE_ALLOW)
+ return (allow == STATE_CHANGE_DISALLOW ? -EINVAL : 0);
+
+ inst->state = request_state;
+
+ return 0;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.h b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
index ee20842..6db95a1 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_state.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
@@ -7,6 +7,7 @@
#define _IRIS_STATE_H_

struct iris_core;
+struct iris_inst;

enum iris_core_state {
IRIS_CORE_DEINIT,
@@ -15,8 +16,29 @@ enum iris_core_state {
IRIS_CORE_ERROR,
};

+enum iris_inst_state {
+ IRIS_INST_OPEN,
+ IRIS_INST_INPUT_STREAMING,
+ IRIS_INST_OUTPUT_STREAMING,
+ IRIS_INST_STREAMING,
+ IRIS_INST_CLOSE,
+ IRIS_INST_ERROR,
+};
+
+enum state_change {
+ STATE_CHANGE_ALLOW,
+ STATE_CHANGE_DISALLOW,
+ STATE_CHANGE_IGNORE,
+};
+
+#define IS_SESSION_ERROR(inst) \
+((inst)->state == IRIS_INST_ERROR)
+
bool core_in_valid_state(struct iris_core *core);
int iris_change_core_state(struct iris_core *core,
enum iris_core_state request_state);

+int iris_inst_change_state(struct iris_inst *inst,
+ enum iris_inst_state request_state);
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
index a57b5fb..8f499a9 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
@@ -34,6 +34,9 @@ int iris_vb2_queue_setup(struct vb2_queue *q,
else
f = inst->fmt_dst;

+ if (inst->state == IRIS_INST_STREAMING)
+ return -EINVAL;
+
if (*num_planes) {
if (*num_planes != f->fmt.pix_mp.num_planes ||
sizes[0] < f->fmt.pix_mp.plane_fmt[0].sizeimage)
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
index 124333a..68ba75f 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
@@ -168,6 +168,7 @@ int vidc_open(struct file *filp)

inst->core = core;
inst->session_id = hash32_ptr(inst);
+ iris_inst_change_state(inst, IRIS_INST_OPEN);
mutex_init(&inst->ctx_q_lock);

ret = vidc_add_session(inst);
@@ -244,6 +245,7 @@ int vidc_close(struct file *filp)
v4l2_ctrl_handler_free(&inst->ctrl_handler);
vdec_inst_deinit(inst);
close_session(inst);
+ iris_inst_change_state(inst, IRIS_INST_CLOSE);
vidc_vb2_queue_deinit(inst);
vidc_v4l2_fh_deinit(inst);
vidc_remove_session(inst);
@@ -295,6 +297,9 @@ static __poll_t vidc_poll(struct file *filp, struct poll_table_struct *pt)
if (!inst)
return EPOLLERR;

+ if (IS_SESSION_ERROR(inst))
+ return EPOLLERR;
+
poll_wait(filp, &inst->fh.wait, pt);
poll_wait(filp, &inst->vb2q_src->done_wq, pt);
poll_wait(filp, &inst->vb2q_dst->done_wq, pt);
--
2.7.4

2023-12-18 11:46:01

[permalink] [raw]

Subject: [PATCH v2 05/34] media: MAINTAINERS: Add Qualcomm Iris video accelerator driver

Add an entry for Iris video encoder/decoder accelerator driver.

Signed-off-by: Vikash Garodia <[email protected]>
Signed-off-by: Dikshita Agarwal <[email protected]>
---
MAINTAINERS | 11 +++++++++++
1 file changed, 11 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 10a981a..a727144 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -17980,6 +17980,17 @@ S: Maintained
F: Documentation/devicetree/bindings/regulator/vqmmc-ipq4019-regulator.yaml
F: drivers/regulator/vqmmc-ipq4019-regulator.c

+QUALCOMM IRIS VIDEO ACCELERATOR DRIVER
+M: Vikash Garodia <[email protected]>
+M: Dikshita Agarwal <[email protected]>
+R: Abhinav Kumar <[email protected]>
+L: [email protected]
+L: [email protected]
+S: Maintained
+T: git git://linuxtv.org/media_tree.git
+F: Documentation/devicetree/bindings/media/qcom,*-iris.yaml
+F: drivers/media/platform/qcom/iris/
+
QUALCOMM NAND CONTROLLER DRIVER
M: Manivannan Sadhasivam <[email protected]>
L: [email protected]
--
2.7.4

2023-12-18 11:46:56

[permalink] [raw]

Subject: [PATCH v2 21/34] media: iris: implement internal buffer management

Implement functions for creating, queueing, releasing
and destroying buffers for internal usage.
During any bitstream parameter change, these internal buffers
needs to be reallocated and requeued. To optimize the same,
driver checks for reusability of any buffer and skip
re-allocation/re-queue.

Signed-off-by: Dikshita Agarwal <[email protected]>
---
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 2 +
.../media/platform/qcom/vcodec/iris/iris_buffer.c | 312 +++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_buffer.h | 12 +
drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 4 +
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 3 +
5 files changed, 333 insertions(+)

diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index c044f78..ce4eaff 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -53,6 +53,8 @@

#define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169

+#define HFI_PROP_COMV_BUFFER_COUNT 0x03000193
+
#define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001

struct hfi_debug_header {
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
index 6d4e722..2a3989c 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
@@ -6,8 +6,22 @@
#include "../buffers.h"
#include "iris_buffer.h"
#include "iris_helpers.h"
+#include "iris_hfi.h"
+#include "hfi_defines.h"
+#include "iris_hfi_packet.h"
#include "iris_instance.h"

+static const u32 dec_ip_int_buf_type[] = {
+ BUF_BIN,
+ BUF_COMV,
+ BUF_NON_COMV,
+ BUF_LINE,
+};
+
+static const u32 dec_op_int_buf_type[] = {
+ BUF_DPB,
+};
+
static int input_min_count(struct iris_inst *inst)
{
return MIN_BUFFERS;
@@ -225,3 +239,301 @@ int iris_free_buffers(struct iris_inst *inst,

return 0;
}
+
+static int iris_get_internal_buf_info(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type)
+{
+ struct iris_buffers *buffers;
+ struct iris_core *core;
+ u32 buf_count;
+ u32 buf_size;
+
+ core = inst->core;
+
+ buf_size = call_session_op(core, int_buf_size,
+ inst, buffer_type);
+
+ buf_count = iris_get_buf_min_count(inst, buffer_type);
+
+ buffers = iris_get_buffer_list(inst, buffer_type);
+ if (!buffers)
+ return -EINVAL;
+
+ if (buf_size && buf_size <= buffers->size &&
+ buf_count && buf_count <= buffers->min_count) {
+ buffers->reuse = true;
+ } else {
+ buffers->reuse = false;
+ buffers->size = buf_size;
+ buffers->min_count = buf_count;
+ }
+
+ return 0;
+}
+
+int iris_get_internal_buffers(struct iris_inst *inst,
+ u32 plane)
+{
+ int ret = 0;
+ u32 i = 0;
+
+ if (plane == INPUT_MPLANE) {
+ for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
+ ret = iris_get_internal_buf_info(inst, dec_ip_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
+ } else {
+ return iris_get_internal_buf_info(inst, BUF_DPB);
+ }
+
+ return ret;
+}
+
+static int iris_create_internal_buffer(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type, u32 index)
+{
+ struct iris_buffers *buffers;
+ struct iris_buffer *buffer;
+ struct iris_core *core;
+
+ core = inst->core;
+
+ buffers = iris_get_buffer_list(inst, buffer_type);
+ if (!buffers)
+ return -EINVAL;
+
+ if (!buffers->size)
+ return 0;
+
+ buffer = kzalloc(sizeof(*buffer), GFP_KERNEL);
+ if (!buffer)
+ return -ENOMEM;
+
+ INIT_LIST_HEAD(&buffer->list);
+ buffer->type = buffer_type;
+ buffer->index = index;
+ buffer->buffer_size = buffers->size;
+ buffer->dma_attrs = DMA_ATTR_WRITE_COMBINE | DMA_ATTR_NO_KERNEL_MAPPING;
+ list_add_tail(&buffer->list, &buffers->list);
+
+ buffer->kvaddr = dma_alloc_attrs(core->dev, buffer->buffer_size,
+ &buffer->device_addr, GFP_KERNEL, buffer->dma_attrs);
+
+ if (!buffer->kvaddr)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static int iris_create_internal_buffers(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type)
+{
+ struct iris_buffers *buffers;
+ int ret = 0;
+ int i;
+
+ buffers = iris_get_buffer_list(inst, buffer_type);
+ if (!buffers)
+ return -EINVAL;
+
+ if (buffers->reuse)
+ return 0;
+
+ for (i = 0; i < buffers->min_count; i++) {
+ ret = iris_create_internal_buffer(inst, buffer_type, i);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
+int iris_create_input_internal_buffers(struct iris_inst *inst)
+{
+ int ret = 0;
+ u32 i = 0;
+
+ for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
+ ret = iris_create_internal_buffers(inst, dec_ip_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
+int iris_create_output_internal_buffers(struct iris_inst *inst)
+{
+ return iris_create_internal_buffers(inst, BUF_DPB);
+}
+
+static int set_num_comv(struct iris_inst *inst)
+{
+ u32 num_comv;
+
+ num_comv = inst->cap[NUM_COMV].value;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_COMV_BUFFER_COUNT,
+ HFI_HOST_FLAGS_NONE,
+ HFI_PORT_BITSTREAM,
+ HFI_PAYLOAD_U32,
+ &num_comv,
+ sizeof(u32));
+}
+
+static int iris_queue_internal_buffers(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type)
+{
+ struct iris_buffer *buffer, *dummy;
+ struct iris_buffers *buffers;
+ int ret = 0;
+
+ if (buffer_type == BUF_COMV) {
+ ret = set_num_comv(inst);
+ if (ret)
+ return ret;
+ }
+
+ buffers = iris_get_buffer_list(inst, buffer_type);
+ if (!buffers)
+ return -EINVAL;
+
+ list_for_each_entry_safe(buffer, dummy, &buffers->list, list) {
+ if (buffer->attr & BUF_ATTR_PENDING_RELEASE)
+ continue;
+ if (buffer->attr & BUF_ATTR_QUEUED)
+ continue;
+ ret = iris_hfi_queue_buffer(inst, buffer);
+ if (ret)
+ return ret;
+ buffer->attr |= BUF_ATTR_QUEUED;
+ }
+
+ return ret;
+}
+
+int iris_queue_input_internal_buffers(struct iris_inst *inst)
+{
+ int ret = 0;
+ u32 i;
+
+ for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
+ ret = iris_queue_internal_buffers(inst, dec_ip_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
+int iris_queue_output_internal_buffers(struct iris_inst *inst)
+{
+ return iris_queue_internal_buffers(inst, BUF_DPB);
+}
+
+int iris_destroy_internal_buffer(struct iris_inst *inst,
+ struct iris_buffer *buffer)
+{
+ struct iris_buffer *buf, *dummy;
+ struct iris_buffers *buffers;
+ struct iris_core *core;
+
+ core = inst->core;
+
+ buffers = iris_get_buffer_list(inst, buffer->type);
+ if (!buffers)
+ return -EINVAL;
+
+ list_for_each_entry_safe(buf, dummy, &buffers->list, list) {
+ if (buf->device_addr == buffer->device_addr) {
+ list_del(&buf->list);
+ dma_free_attrs(core->dev, buf->buffer_size, buf->kvaddr,
+ buf->device_addr, buf->dma_attrs);
+ buf->kvaddr = NULL;
+ buf->device_addr = 0;
+ kfree(buf);
+ break;
+ }
+ }
+
+ return 0;
+}
+
+int iris_destroy_internal_buffers(struct iris_inst *inst,
+ u32 plane)
+{
+ struct iris_buffer *buf, *dummy;
+ struct iris_buffers *buffers;
+ const u32 *internal_buf_type;
+ int ret = 0;
+ u32 i, len;
+
+ if (plane == INPUT_MPLANE) {
+ internal_buf_type = dec_ip_int_buf_type;
+ len = ARRAY_SIZE(dec_ip_int_buf_type);
+ } else {
+ internal_buf_type = dec_op_int_buf_type;
+ len = ARRAY_SIZE(dec_op_int_buf_type);
+ }
+
+ for (i = 0; i < len; i++) {
+ buffers = iris_get_buffer_list(inst, internal_buf_type[i]);
+ if (!buffers)
+ return -EINVAL;
+
+ if (buffers->reuse)
+ continue;
+
+ list_for_each_entry_safe(buf, dummy, &buffers->list, list) {
+ ret = iris_destroy_internal_buffer(inst, buf);
+ if (ret)
+ return ret;
+ }
+ }
+
+ return ret;
+}
+
+static int iris_release_internal_buffers(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type)
+{
+ struct iris_buffer *buffer, *dummy;
+ struct iris_buffers *buffers;
+ int ret = 0;
+
+ buffers = iris_get_buffer_list(inst, buffer_type);
+ if (!buffers)
+ return -EINVAL;
+
+ if (buffers->reuse)
+ return 0;
+
+ list_for_each_entry_safe(buffer, dummy, &buffers->list, list) {
+ if (buffer->attr & BUF_ATTR_PENDING_RELEASE)
+ continue;
+ if (!(buffer->attr & BUF_ATTR_QUEUED))
+ continue;
+ ret = iris_hfi_release_buffer(inst, buffer);
+ if (ret)
+ return ret;
+ buffer->attr |= BUF_ATTR_PENDING_RELEASE;
+ }
+
+ return ret;
+}
+
+int iris_release_input_internal_buffers(struct iris_inst *inst)
+{
+ int ret = 0;
+ u32 i = 0;
+
+ for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
+ ret = iris_release_internal_buffers(inst, dec_ip_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
index 1cd76a9..ece894e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
@@ -46,4 +46,16 @@ int iris_allocate_buffers(struct iris_inst *inst,
int iris_free_buffers(struct iris_inst *inst,
enum iris_buffer_type buf_type);

+int iris_get_internal_buffers(struct iris_inst *inst,
+ u32 plane);
+int iris_create_input_internal_buffers(struct iris_inst *inst);
+int iris_create_output_internal_buffers(struct iris_inst *inst);
+int iris_queue_input_internal_buffers(struct iris_inst *inst);
+int iris_queue_output_internal_buffers(struct iris_inst *inst);
+int iris_destroy_internal_buffer(struct iris_inst *inst,
+ struct iris_buffer *buffer);
+int iris_destroy_internal_buffers(struct iris_inst *inst,
+ u32 plane);
+int iris_release_input_internal_buffers(struct iris_inst *inst);
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
index 4aefdc4..ca2339e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
@@ -16,6 +16,10 @@ int iris_hfi_session_close(struct iris_inst *inst);
int iris_hfi_set_property(struct iris_inst *inst,
u32 packet_type, u32 flag, u32 plane, u32 payload_type,
void *payload, u32 payload_size);
+int iris_hfi_queue_buffer(struct iris_inst *inst,
+ struct iris_buffer *buffer);
+int iris_hfi_release_buffer(struct iris_inst *inst,
+ struct iris_buffer *buffer);

irqreturn_t iris_hfi_isr(int irq, void *data);
irqreturn_t iris_hfi_isr_handler(int irq, void *data);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index b131a50..566048e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -39,6 +39,9 @@ void vdec_inst_init(struct iris_inst *inst)
f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT;
f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+ inst->buffers.output.min_count = iris_get_buf_min_count(inst, BUF_OUTPUT);
+ inst->buffers.output.actual_count = inst->buffers.output.min_count;
+ inst->buffers.output.size = f->fmt.pix_mp.plane_fmt[0].sizeimage;
inst->fw_min_count = 0;
}

--
2.7.4

2023-12-18 11:47:02

[permalink] [raw]

Subject: [PATCH v2 30/34] media: iris: register video encoder device to platform driver

Register/unregister video Encoder device.

Signed-off-by: Dikshita Agarwal <[email protected]>
---
.../media/platform/qcom/vcodec/iris/iris_common.h | 5 +++
.../media/platform/qcom/vcodec/iris/iris_core.h | 2 ++
.../media/platform/qcom/vcodec/iris/iris_probe.c | 42 ++++++++++++++++------
3 files changed, 38 insertions(+), 11 deletions(-)

diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
index 3ab4767..b1273d0 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
@@ -30,6 +30,11 @@

#define INPUT_TIMER_LIST_SIZE 30

+enum domain_type {
+ ENCODER = BIT(0),
+ DECODER = BIT(1),
+};
+
enum codec_type {
H264 = BIT(0),
HEVC = BIT(1),
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index 50553d2..c56eb24 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -23,6 +23,7 @@
* @irq: iris irq
* @v4l2_dev: a holder for v4l2 device structure
* @vdev_dec: iris video device structure for decoder
+ * @vdev_enc: iris video device structure for encoder
* @v4l2_file_ops: iris v4l2 file ops
* @v4l2_ioctl_ops: iris v4l2 ioctl ops
* @bus_tbl: table of iris buses
@@ -66,6 +67,7 @@ struct iris_core {
int irq;
struct v4l2_device v4l2_dev;
struct video_device *vdev_dec;
+ struct video_device *vdev_enc;
const struct v4l2_file_operations *v4l2_file_ops;
const struct v4l2_ioctl_ops *v4l2_ioctl_ops;
struct bus_info *bus_tbl;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
index 2571e27..b487e83 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -31,16 +31,15 @@ static int init_iris_isr(struct iris_core *core)
return ret;
}

-static int iris_register_video_device(struct iris_core *core)
+static int iris_register_video_device(struct iris_core *core, enum domain_type type)
{
struct video_device *vdev;
- int ret;
+ int ret = 0;

vdev = video_device_alloc();
if (!vdev)
return -ENOMEM;

- strscpy(vdev->name, "qcom-iris-decoder", sizeof(vdev->name));
vdev->release = video_device_release;
vdev->fops = core->v4l2_file_ops;
vdev->ioctl_ops = core->v4l2_ioctl_ops;
@@ -48,11 +47,24 @@ static int iris_register_video_device(struct iris_core *core)
vdev->v4l2_dev = &core->v4l2_dev;
vdev->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;

- ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
- if (ret)
- goto err_vdev_release;
+ if (type == DECODER) {
+ strscpy(vdev->name, "qcom-iris-decoder", sizeof(vdev->name));
+
+ ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
+ if (ret)
+ goto err_vdev_release;
+
+ core->vdev_dec = vdev;
+ } else if (type == ENCODER) {
+ strscpy(vdev->name, "qcom-iris-encoder", sizeof(vdev->name));
+
+ ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
+ if (ret)
+ goto err_vdev_release;
+
+ core->vdev_enc = vdev;
+ }

- core->vdev_dec = vdev;
video_set_drvdata(vdev, core);

return ret;
@@ -80,6 +92,8 @@ static void iris_remove(struct platform_device *pdev)

video_unregister_device(core->vdev_dec);

+ video_unregister_device(core->vdev_enc);
+
v4l2_device_unregister(&core->v4l2_dev);

iris_pm_put(core, false);
@@ -185,17 +199,21 @@ static int iris_probe(struct platform_device *pdev)
if (ret)
goto err_runtime_disable;

- ret = iris_register_video_device(core);
+ ret = iris_register_video_device(core, DECODER);
if (ret)
goto err_v4l2_unreg;

+ ret = iris_register_video_device(core, ENCODER);
+ if (ret)
+ goto err_vdev_unreg_dec;
+
platform_set_drvdata(pdev, core);

dma_mask = core->cap[DMA_MASK].value;

ret = dma_set_mask_and_coherent(dev, dma_mask);
if (ret)
- goto err_vdev_unreg;
+ goto err_vdev_unreg_enc;

dma_set_max_seg_size(&pdev->dev, (unsigned int)DMA_BIT_MASK(32));
dma_set_seg_boundary(&pdev->dev, (unsigned long)DMA_BIT_MASK(64));
@@ -206,7 +224,7 @@ static int iris_probe(struct platform_device *pdev)
if (ret) {
dev_err_probe(core->dev, ret,
"%s: interface queues init failed\n", __func__);
- goto err_vdev_unreg;
+ goto err_vdev_unreg_enc;
}

ret = iris_pm_get(core);
@@ -236,7 +254,9 @@ static int iris_probe(struct platform_device *pdev)
hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr,
&core->command_queue, &core->message_queue,
&core->debug_queue);
-err_vdev_unreg:
+err_vdev_unreg_enc:
+ video_unregister_device(core->vdev_enc);
+err_vdev_unreg_dec:
video_unregister_device(core->vdev_dec);
err_v4l2_unreg:
v4l2_device_unregister(&core->v4l2_dev);
--
2.7.4

2023-12-18 11:50:06

[permalink] [raw]

Subject: [PATCH v2 18/34] media: iris: introduce and implement iris vb2 mem ops

From: Vikash Garodia <[email protected]>

Implement the iris vb2 mem ops for buffer management for
DMABUF streaming mode. Update video driver buffer
with dma buf information.

Signed-off-by: Vikash Garodia <[email protected]>
Signed-off-by: Dikshita Agarwal <[email protected]>
---
.../media/platform/qcom/vcodec/iris/iris_probe.c | 1 +
drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 147 +++++++++++++++++++++
drivers/media/platform/qcom/vcodec/iris/iris_vb2.h | 10 ++
.../media/platform/qcom/vcodec/iris/iris_vidc.c | 11 ++
4 files changed, 169 insertions(+)

diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
index 50fb93e..bf484a3 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -233,5 +233,6 @@ static struct platform_driver qcom_iris_driver = {
};

module_platform_driver(qcom_iris_driver);
+MODULE_IMPORT_NS(DMA_BUF);
MODULE_DESCRIPTION("Qualcomm Iris video driver");
MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
index b040d27..a57b5fb 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
@@ -69,3 +69,150 @@ int iris_vb2_queue_setup(struct vb2_queue *q,

return ret;
}
+
+void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev,
+ struct dma_buf *dbuf, unsigned long size)
+{
+ enum iris_buffer_type buf_type;
+ struct iris_buffers *buffers;
+ struct iris_buffer *iter;
+ struct iris_buffer *buf;
+ struct iris_inst *inst;
+ bool found = false;
+
+ if (!vb || !dev || !dbuf || !vb->vb2_queue)
+ return ERR_PTR(-EINVAL);
+
+ inst = vb->vb2_queue->drv_priv;
+
+ buf_type = v4l2_type_to_driver(vb->type);
+
+ buffers = iris_get_buffer_list(inst, buf_type);
+ if (!buffers)
+ return NULL;
+
+ list_for_each_entry(iter, &buffers->list, list) {
+ if (iter->index == vb->index) {
+ found = true;
+ buf = iter;
+ break;
+ }
+ }
+
+ if (!found)
+ return NULL;
+
+ buf->inst = inst;
+ buf->dmabuf = dbuf;
+
+ buf->attach = dma_buf_attach(dbuf, dev);
+ if (IS_ERR(buf->attach)) {
+ buf->attach = NULL;
+ return NULL;
+ }
+
+ return buf;
+}
+
+int iris_vb2_map_dmabuf(void *buf_priv)
+{
+ struct iris_buffer *buf = buf_priv;
+ struct iris_core *core;
+ struct iris_inst *inst;
+
+ if (!buf || !buf->inst)
+ return -EINVAL;
+
+ inst = buf->inst;
+ core = inst->core;
+
+ if (!buf->attach) {
+ dev_err(core->dev, "trying to map a non attached buffer\n");
+ return -EINVAL;
+ }
+
+ buf->sg_table = dma_buf_map_attachment(buf->attach, DMA_BIDIRECTIONAL);
+ if (IS_ERR(buf->sg_table))
+ return -EINVAL;
+
+ if (!buf->sg_table->sgl) {
+ dma_buf_unmap_attachment(buf->attach, buf->sg_table, DMA_BIDIRECTIONAL);
+ buf->sg_table = NULL;
+ return -EINVAL;
+ }
+
+ buf->device_addr = sg_dma_address(buf->sg_table->sgl);
+
+ return 0;
+}
+
+void iris_vb2_unmap_dmabuf(void *buf_priv)
+{
+ struct iris_buffer *buf = buf_priv;
+ struct iris_core *core;
+ struct iris_inst *inst;
+
+ if (!buf || !buf->inst)
+ return;
+
+ inst = buf->inst;
+ core = inst->core;
+
+ if (!buf->attach) {
+ dev_err(core->dev, "trying to unmap a non attached buffer\n");
+ return;
+ }
+
+ if (!buf->sg_table) {
+ dev_err(core->dev, "dmabuf buffer is already unmapped\n");
+ return;
+ }
+
+ if (buf->attach && buf->sg_table) {
+ dma_buf_unmap_attachment(buf->attach, buf->sg_table, DMA_BIDIRECTIONAL);
+ buf->sg_table = NULL;
+ buf->device_addr = 0x0;
+ }
+}
+
+void iris_vb2_detach_dmabuf(void *buf_priv)
+{
+ struct iris_buffer *buf = buf_priv;
+ struct iris_core *core;
+ struct iris_inst *inst;
+
+ if (!buf || !buf->inst)
+ return;
+
+ inst = buf->inst;
+ core = inst->core;
+
+ if (buf->sg_table) {
+ dev_err(core->dev, "trying to detach an unmapped buffer\n");
+ dma_buf_unmap_attachment(buf->attach, buf->sg_table, DMA_BIDIRECTIONAL);
+ buf->sg_table = NULL;
+ }
+
+ if (buf->attach && buf->dmabuf) {
+ dma_buf_detach(buf->dmabuf, buf->attach);
+ buf->attach = NULL;
+ }
+
+ buf->dmabuf = NULL;
+ buf->inst = NULL;
+}
+
+void *iris_vb2_alloc(struct vb2_buffer *vb, struct device *dev,
+ unsigned long size)
+{
+ return (void *)0xdeadbeef;
+}
+
+void iris_vb2_put(void *buf_priv)
+{
+}
+
+int iris_vb2_mmap(void *buf_priv, struct vm_area_struct *vma)
+{
+ return 0;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
index 8a8e1039..4342034 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
@@ -12,4 +12,14 @@ int iris_vb2_queue_setup(struct vb2_queue *q,
unsigned int *num_buffers, unsigned int *num_planes,
unsigned int sizes[], struct device *alloc_devs[]);

+/* vb2_mem_ops */
+void *iris_vb2_alloc(struct vb2_buffer *vb, struct device *dev, unsigned long size);
+void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev, struct dma_buf *dbuf,
+ unsigned long size);
+void iris_vb2_put(void *buf_priv);
+int iris_vb2_mmap(void *buf_priv, struct vm_area_struct *vma);
+void iris_vb2_detach_dmabuf(void *buf_priv);
+int iris_vb2_map_dmabuf(void *buf_priv);
+void iris_vb2_unmap_dmabuf(void *buf_priv);
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
index 410de720..124333a 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
@@ -320,10 +320,21 @@ static const struct vb2_ops iris_vb2_ops = {
.queue_setup = iris_vb2_queue_setup,
};

+static struct vb2_mem_ops iris_vb2_mem_ops = {
+ .alloc = iris_vb2_alloc,
+ .put = iris_vb2_put,
+ .mmap = iris_vb2_mmap,
+ .attach_dmabuf = iris_vb2_attach_dmabuf,
+ .detach_dmabuf = iris_vb2_detach_dmabuf,
+ .map_dmabuf = iris_vb2_map_dmabuf,
+ .unmap_dmabuf = iris_vb2_unmap_dmabuf,
+};
+
int init_ops(struct iris_core *core)
{
core->v4l2_file_ops = &v4l2_file_ops;
core->vb2_ops = &iris_vb2_ops;
+ core->vb2_mem_ops = &iris_vb2_mem_ops;

return 0;
}
--
2.7.4

2023-12-18 11:50:30

[permalink] [raw]

Subject: [PATCH v2 32/34] media: iris: implement iris v4l2 ioctl ops supported by encoder

Implement all iris v4l2 ioctls ops supported by encoder.
Add state checks to ensure ioctl are allowed in valid
instance state only. Codec format can be changed by client
during s_fmt. Update the v4l2 control values according
to the updated codec format.

Signed-off-by: Dikshita Agarwal <[email protected]>
---
drivers/media/platform/qcom/vcodec/iris/Makefile | 1 +
.../media/platform/qcom/vcodec/iris/iris_core.h | 8 +-
.../media/platform/qcom/vcodec/iris/iris_ctrls.c | 59 ++-
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 89 +++-
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 3 +
.../platform/qcom/vcodec/iris/iris_instance.h | 5 +
.../media/platform/qcom/vcodec/iris/iris_probe.c | 3 +-
.../media/platform/qcom/vcodec/iris/iris_state.c | 5 +-
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 80 +---
.../media/platform/qcom/vcodec/iris/iris_venc.c | 525 +++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_venc.h | 24 +
.../media/platform/qcom/vcodec/iris/iris_vidc.c | 415 ++++++++++++++--
.../platform/qcom/vcodec/iris/platform_common.h | 2 +
.../platform/qcom/vcodec/iris/platform_sm8550.c | 1 +
14 files changed, 1088 insertions(+), 132 deletions(-)
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_venc.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_venc.h

diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
index 4c8f8f6..b95d627 100644
--- a/drivers/media/platform/qcom/vcodec/iris/Makefile
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -6,6 +6,7 @@ iris-objs += iris_probe.o \
iris_vidc.o \
iris_vb2.o \
iris_vdec.o \
+ iris_venc.o \
iris_state.o \
iris_ctrls.o \
iris_helpers.o \
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index c56eb24..baced21 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -25,7 +25,8 @@
* @vdev_dec: iris video device structure for decoder
* @vdev_enc: iris video device structure for encoder
* @v4l2_file_ops: iris v4l2 file ops
- * @v4l2_ioctl_ops: iris v4l2 ioctl ops
+ * @v4l2_ioctl_ops_dec: iris v4l2 ioctl ops for decoder
+ * @v4l2_ioctl_ops_enc: iris v4l2 ioctl ops for encoder
* @bus_tbl: table of iris buses
* @bus_count: count of iris buses
* @power_domain_tbl: table of iris power domains
@@ -52,6 +53,7 @@
* @packet_id: id of packet
* @vpu_ops: a pointer to vpu ops
* @session_ops: a pointer to session level ops
+ * @enc_codecs_count: supported codec count for encoder
* @dec_codecs_count: supported codec count for decoder
* @platform_data: a structure for platform data
* @cap: an array for supported core capabilities
@@ -69,7 +71,8 @@ struct iris_core {
struct video_device *vdev_dec;
struct video_device *vdev_enc;
const struct v4l2_file_operations *v4l2_file_ops;
- const struct v4l2_ioctl_ops *v4l2_ioctl_ops;
+ const struct v4l2_ioctl_ops *v4l2_ioctl_ops_dec;
+ const struct v4l2_ioctl_ops *v4l2_ioctl_ops_enc;
struct bus_info *bus_tbl;
u32 bus_count;
struct power_domain_info *power_domain_tbl;
@@ -97,6 +100,7 @@ struct iris_core {
const struct vpu_ops *vpu_ops;
const struct vpu_session_ops *session_ops;
u32 dec_codecs_count;
+ u32 enc_codecs_count;
struct platform_data *platform_data;
struct plat_core_cap cap[CORE_CAP_MAX + 1];
struct plat_inst_caps *inst_caps;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
index a648cc1..af99ac73 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
@@ -217,7 +217,7 @@ static int set_dynamic_property(struct iris_inst *inst,
return ret;
}

-static int vdec_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
+static int iris_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
{
enum plat_inst_cap_type cap_id;
struct iris_inst *inst = NULL;
@@ -242,7 +242,7 @@ static int vdec_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
return ret;
}

-static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl)
+static int iris_op_s_ctrl(struct v4l2_ctrl *ctrl)
{
struct cap_entry *entry = NULL, *temp = NULL;
struct list_head children_list, firmware_list;
@@ -273,7 +273,8 @@ static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl)

cap[cap_id].flags |= CAP_FLAG_CLIENT_SET;

- if (!inst->vb2q_src->streaming) {
+ if ((inst->domain == ENCODER && !inst->vb2q_dst->streaming) ||
+ (inst->domain == DECODER && !inst->vb2q_src->streaming)) {
inst->cap[cap_id].value = ctrl->val;
} else {
ret = adjust_dynamic_property(inst, cap_id, ctrl,
@@ -300,16 +301,16 @@ static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl)
}

static const struct v4l2_ctrl_ops ctrl_ops = {
- .s_ctrl = vdec_op_s_ctrl,
- .g_volatile_ctrl = vdec_op_g_volatile_ctrl,
+ .s_ctrl = iris_op_s_ctrl,
+ .g_volatile_ctrl = iris_op_g_volatile_ctrl,
};

int ctrls_init(struct iris_inst *inst, bool init)
{
int num_ctrls = 0, ctrl_idx = 0;
+ u64 codecs_count, step_or_mask;
struct plat_inst_cap *cap;
struct iris_core *core;
- u64 step_or_mask;
int idx = 0;
int ret = 0;

@@ -324,8 +325,11 @@ int ctrls_init(struct iris_inst *inst, bool init)
return -EINVAL;

if (init) {
+ codecs_count = inst->domain == ENCODER ?
+ core->enc_codecs_count :
+ core->dec_codecs_count;
ret = v4l2_ctrl_handler_init(&inst->ctrl_handler,
- INST_CAP_MAX * core->dec_codecs_count);
+ INST_CAP_MAX * codecs_count);
if (ret)
return ret;
}
@@ -451,29 +455,49 @@ static int update_inst_capability(struct plat_inst_cap *in,
int iris_init_instance_caps(struct iris_core *core)
{
struct plat_inst_cap *inst_plat_cap_data = NULL;
- u8 dec_codecs_count = 0;
- int num_inst_cap;
- u32 dec_valid_codecs;
+ u8 enc_codecs_count = 0, dec_codecs_count = 0;
+ u32 enc_valid_codecs, dec_valid_codecs;
int i, j, check_bit = 0;
+ u8 codecs_count = 0;
+ int num_inst_cap;
int ret = 0;

inst_plat_cap_data = core->platform_data->inst_cap_data;
if (!inst_plat_cap_data)
return -EINVAL;

+ enc_valid_codecs = core->cap[ENC_CODECS].value;
+ enc_codecs_count = hweight32(enc_valid_codecs);
+ core->enc_codecs_count = enc_codecs_count;
+
dec_valid_codecs = core->cap[DEC_CODECS].value;
dec_codecs_count = hweight32(dec_valid_codecs);
core->dec_codecs_count = dec_codecs_count;

+ codecs_count = enc_codecs_count + dec_codecs_count;
core->inst_caps = devm_kzalloc(core->dev,
- dec_codecs_count * sizeof(struct plat_inst_caps),
+ codecs_count * sizeof(struct plat_inst_caps),
GFP_KERNEL);
if (!core->inst_caps)
return -ENOMEM;

- for (i = 0; i < dec_codecs_count; i++) {
+ for (i = 0; i < enc_codecs_count; i++) {
+ while (check_bit < (sizeof(enc_valid_codecs) * 8)) {
+ if (enc_valid_codecs & BIT(check_bit)) {
+ core->inst_caps[i].domain = ENCODER;
+ core->inst_caps[i].codec = enc_valid_codecs &
+ BIT(check_bit);
+ check_bit++;
+ break;
+ }
+ check_bit++;
+ }
+ }
+
+ for (; i < codecs_count; i++) {
while (check_bit < (sizeof(dec_valid_codecs) * 8)) {
if (dec_valid_codecs & BIT(check_bit)) {
+ core->inst_caps[i].domain = DECODER;
core->inst_caps[i].codec = dec_valid_codecs &
BIT(check_bit);
check_bit++;
@@ -486,9 +510,9 @@ int iris_init_instance_caps(struct iris_core *core)
num_inst_cap = core->platform_data->inst_cap_data_size;

for (i = 0; i < num_inst_cap; i++) {
- for (j = 0; j < dec_codecs_count; j++) {
- if ((inst_plat_cap_data[i].codec &
- core->inst_caps[j].codec)) {
+ for (j = 0; j < codecs_count; j++) {
+ if ((inst_plat_cap_data[i].domain & core->inst_caps[j].domain) &&
+ (inst_plat_cap_data[i].codec & core->inst_caps[j].codec)) {
ret = update_inst_capability(&inst_plat_cap_data[i],
&core->inst_caps[j]);
if (ret)
@@ -503,11 +527,14 @@ int iris_init_instance_caps(struct iris_core *core)
int get_inst_capability(struct iris_inst *inst)
{
struct iris_core *core;
+ u32 codecs_count = 0;
int i;

core = inst->core;

- for (i = 0; i < core->dec_codecs_count; i++) {
+ codecs_count = core->enc_codecs_count + core->dec_codecs_count;
+
+ for (i = 0; i < codecs_count; i++) {
if (core->inst_caps[i].codec == inst->codec) {
memcpy(&inst->cap[0], &core->inst_caps[i].cap[0],
(INST_CAP_MAX + 1) * sizeof(struct plat_inst_cap));
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index d9d22e2..c84bb51 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -7,6 +7,7 @@

#include "hfi_defines.h"
#include "iris_core.h"
+#include "iris_ctrls.h"
#include "iris_helpers.h"
#include "iris_hfi.h"
#include "iris_hfi_packet.h"
@@ -273,7 +274,7 @@ int close_session(struct iris_inst *inst)
return ret;
}

-static int check_core_mbps_mbpf(struct iris_inst *inst)
+int check_core_mbps_mbpf(struct iris_inst *inst)
{
u32 mbpf = 0, mbps = 0, total_mbpf = 0, total_mbps = 0;
struct iris_core *core;
@@ -284,7 +285,9 @@ static int check_core_mbps_mbpf(struct iris_inst *inst)

mutex_lock(&core->lock);
list_for_each_entry(instance, &core->instances, list) {
- fps = inst->cap[QUEUED_RATE].value >> 16;
+ fps = max3(inst->cap[QUEUED_RATE].value >> 16,
+ inst->cap[FRAME_RATE].value >> 16,
+ inst->cap[OPERATING_RATE].value >> 16);
mbpf = get_mbpf(inst);
mbps = mbpf * fps;
total_mbpf += mbpf;
@@ -814,6 +817,88 @@ int session_streamoff(struct iris_inst *inst, u32 plane)
return ret;
}

+int process_resume(struct iris_inst *inst)
+{
+ enum iris_inst_sub_state clear_sub_state = IRIS_INST_SUB_NONE;
+ int ret;
+
+ if (inst->sub_state & IRIS_INST_SUB_DRC &&
+ inst->sub_state & IRIS_INST_SUB_DRC_LAST) {
+ clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST;
+
+ if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
+ ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE);
+ if (ret)
+ return ret;
+ clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE;
+ }
+ if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) {
+ ret = iris_hfi_resume(inst, OUTPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE);
+ if (ret)
+ return ret;
+ clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE;
+ }
+ } else if (inst->sub_state & IRIS_INST_SUB_DRAIN &&
+ inst->sub_state & IRIS_INST_SUB_DRAIN_LAST) {
+ clear_sub_state = IRIS_INST_SUB_DRAIN | IRIS_INST_SUB_DRAIN_LAST;
+ if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
+ ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_DRAIN);
+ if (ret)
+ return ret;
+ clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE;
+ }
+ if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) {
+ ret = iris_hfi_resume(inst, OUTPUT_MPLANE, HFI_CMD_DRAIN);
+ if (ret)
+ return ret;
+ clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE;
+ }
+ }
+
+ ret = iris_inst_change_sub_state(inst, clear_sub_state, 0);
+
+ return ret;
+}
+
+int codec_change(struct iris_inst *inst, u32 v4l2_codec)
+{
+ bool session_init = false;
+ int ret;
+
+ if (!inst->codec)
+ session_init = true;
+
+ if (inst->codec &&
+ ((inst->domain == DECODER && inst->fmt_src->fmt.pix_mp.pixelformat == v4l2_codec) ||
+ (inst->domain == ENCODER && inst->fmt_dst->fmt.pix_mp.pixelformat == v4l2_codec)))
+ return 0;
+
+ inst->codec = v4l2_codec_to_driver(inst, v4l2_codec);
+ if (!inst->codec)
+ return -EINVAL;
+
+ if (inst->domain == DECODER)
+ inst->fmt_src->fmt.pix_mp.pixelformat = v4l2_codec;
+ else if (inst->domain == ENCODER)
+ inst->fmt_dst->fmt.pix_mp.pixelformat = v4l2_codec;
+
+ ret = get_inst_capability(inst);
+ if (ret)
+ return ret;
+
+ ret = ctrls_init(inst, session_init);
+ if (ret)
+ return ret;
+
+ ret = update_buffer_count(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = update_buffer_count(inst, OUTPUT_MPLANE);
+
+ return ret;
+}
+
int iris_pm_get(struct iris_core *core)
{
int ret;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
index c628b2e..39cec8c 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -41,6 +41,7 @@ enum codec_type v4l2_codec_to_driver(struct iris_inst *inst, u32 v4l2_codec);
u32 v4l2_colorformat_from_driver(struct iris_inst *inst, enum colorformat_type colorformat);
enum colorformat_type v4l2_colorformat_to_driver(struct iris_inst *inst, u32 v4l2_colorformat);
struct vb2_queue *get_vb2q(struct iris_inst *inst, u32 type);
+int check_core_mbps_mbpf(struct iris_inst *inst);
int check_session_supported(struct iris_inst *inst);

struct iris_buffer *get_driver_buf(struct iris_inst *inst, u32 plane, u32 index);
@@ -51,6 +52,8 @@ int iris_vb2_buffer_done(struct iris_inst *inst,
int iris_release_nonref_buffers(struct iris_inst *inst);
void iris_destroy_buffers(struct iris_inst *inst);
int session_streamoff(struct iris_inst *inst, u32 plane);
+int process_resume(struct iris_inst *inst);
+int codec_change(struct iris_inst *inst, u32 v4l2_codec);
int iris_pm_get(struct iris_core *core);
int iris_pm_put(struct iris_core *core, bool autosuspend);
int iris_pm_get_put(struct iris_core *core);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
index 70f4c7d..a1547c5 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -8,6 +8,7 @@

#include <media/v4l2-ctrls.h>

+#include "hfi_defines.h"
#include "iris_buffer.h"
#include "iris_common.h"
#include "iris_core.h"
@@ -29,6 +30,7 @@
* @fmt_dst: structure of v4l2_format for destination
* @ctrl_handler: reference of v4l2 ctrl handler
* @crop: structure of crop info
+ * @compose: structure of compose info
* @packet: HFI packet
* @packet_size: HFI packet size
* @completions: structure of signal completions
@@ -36,6 +38,7 @@
* @num_ctrls: supported number of controls
* @caps_list: list head of capability
* @codec: codec type
+ * @domain: domain type: encoder or decoder
* @buffers: structure of buffer info
* @fw_min_count: minimnum count of buffers needed by fw
* @state: instance state
@@ -70,6 +73,7 @@ struct iris_inst {
struct v4l2_format *fmt_dst;
struct v4l2_ctrl_handler ctrl_handler;
struct rect_desc crop;
+ struct rect_desc compose;
void *packet;
u32 packet_size;
struct completion completions[MAX_SIGNAL];
@@ -77,6 +81,7 @@ struct iris_inst {
u32 num_ctrls;
struct list_head caps_list;
enum codec_type codec;
+ enum domain_type domain;
struct iris_buffers_info buffers;
u32 fw_min_count;
enum iris_inst_state state;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
index b487e83..49d2701 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -42,13 +42,13 @@ static int iris_register_video_device(struct iris_core *core, enum domain_type t

vdev->release = video_device_release;
vdev->fops = core->v4l2_file_ops;
- vdev->ioctl_ops = core->v4l2_ioctl_ops;
vdev->vfl_dir = VFL_DIR_M2M;
vdev->v4l2_dev = &core->v4l2_dev;
vdev->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;

if (type == DECODER) {
strscpy(vdev->name, "qcom-iris-decoder", sizeof(vdev->name));
+ vdev->ioctl_ops = core->v4l2_ioctl_ops_dec;

ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
if (ret)
@@ -57,6 +57,7 @@ static int iris_register_video_device(struct iris_core *core, enum domain_type t
core->vdev_dec = vdev;
} else if (type == ENCODER) {
strscpy(vdev->name, "qcom-iris-encoder", sizeof(vdev->name));
+ vdev->ioctl_ops = core->v4l2_ioctl_ops_enc;

ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
if (ret)
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
index 952ba2a..e853aec 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_state.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
@@ -198,8 +198,9 @@ bool allow_s_ctrl(struct iris_inst *inst, u32 cap_id)
{
return ((inst->state == IRIS_INST_OPEN) ||
((inst->cap[cap_id].flags & CAP_FLAG_DYNAMIC_ALLOWED) &&
- (inst->state == IRIS_INST_INPUT_STREAMING ||
- inst->state == IRIS_INST_STREAMING)));
+ ((inst->state == IRIS_INST_INPUT_STREAMING && inst->domain == DECODER) ||
+ (inst->state == IRIS_INST_OUTPUT_STREAMING && inst->domain == ENCODER) ||
+ (inst->state == IRIS_INST_STREAMING))));
}

int iris_inst_state_change_streamon(struct iris_inst *inst, u32 plane)
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index 371615e..300d0e9 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -22,39 +22,6 @@ struct vdec_prop_type_handle {
int (*handle)(struct iris_inst *inst);
};

-static int vdec_codec_change(struct iris_inst *inst, u32 v4l2_codec)
-{
- bool session_init = false;
- int ret;
-
- if (!inst->codec)
- session_init = true;
-
- if (inst->codec && inst->fmt_src->fmt.pix_mp.pixelformat == v4l2_codec)
- return 0;
-
- inst->codec = v4l2_codec_to_driver(inst, v4l2_codec);
- if (!inst->codec)
- return -EINVAL;
-
- inst->fmt_src->fmt.pix_mp.pixelformat = v4l2_codec;
- ret = get_inst_capability(inst);
- if (ret)
- return ret;
-
- ret = ctrls_init(inst, session_init);
- if (ret)
- return ret;
-
- ret = update_buffer_count(inst, INPUT_MPLANE);
- if (ret)
- return ret;
-
- ret = update_buffer_count(inst, OUTPUT_MPLANE);
-
- return ret;
-}
-
int vdec_inst_init(struct iris_inst *inst)
{
struct v4l2_format *f;
@@ -93,7 +60,7 @@ int vdec_inst_init(struct iris_inst *inst)
inst->buffers.output.size = f->fmt.pix_mp.plane_fmt[0].sizeimage;
inst->fw_min_count = 0;

- ret = vdec_codec_change(inst, inst->fmt_src->fmt.pix_mp.pixelformat);
+ ret = codec_change(inst, inst->fmt_src->fmt.pix_mp.pixelformat);

return ret;
}
@@ -233,7 +200,7 @@ int vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f)
if (f->type == INPUT_MPLANE) {
if (inst->fmt_src->fmt.pix_mp.pixelformat !=
f->fmt.pix_mp.pixelformat) {
- ret = vdec_codec_change(inst, f->fmt.pix_mp.pixelformat);
+ ret = codec_change(inst, f->fmt.pix_mp.pixelformat);
if (ret)
return ret;
}
@@ -1304,49 +1271,6 @@ int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2)
return ret;
}

-static int process_resume(struct iris_inst *inst)
-{
- enum iris_inst_sub_state clear_sub_state = IRIS_INST_SUB_NONE;
- int ret;
-
- if (inst->sub_state & IRIS_INST_SUB_DRC &&
- inst->sub_state & IRIS_INST_SUB_DRC_LAST) {
- clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST;
-
- if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
- ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE);
- if (ret)
- return ret;
- clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE;
- }
- if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) {
- ret = iris_hfi_resume(inst, OUTPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE);
- if (ret)
- return ret;
- clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE;
- }
- } else if (inst->sub_state & IRIS_INST_SUB_DRAIN &&
- inst->sub_state & IRIS_INST_SUB_DRAIN_LAST) {
- clear_sub_state = IRIS_INST_SUB_DRAIN | IRIS_INST_SUB_DRAIN_LAST;
- if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
- ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_DRAIN);
- if (ret)
- return ret;
- clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE;
- }
- if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) {
- ret = iris_hfi_resume(inst, OUTPUT_MPLANE, HFI_CMD_DRAIN);
- if (ret)
- return ret;
- clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE;
- }
- }
-
- ret = iris_inst_change_sub_state(inst, clear_sub_state, 0);
-
- return ret;
-}
-
int vdec_start_cmd(struct iris_inst *inst)
{
int ret;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_venc.c b/drivers/media/platform/qcom/vcodec/iris/iris_venc.c
new file mode 100644
index 0000000..802db40
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_venc.c
@@ -0,0 +1,525 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "hfi_defines.h"
+#include "iris_buffer.h"
+#include "iris_common.h"
+#include "iris_ctrls.h"
+#include "iris_helpers.h"
+#include "iris_hfi.h"
+#include "iris_hfi_packet.h"
+#include "iris_power.h"
+#include "iris_venc.h"
+
+int venc_inst_init(struct iris_inst *inst)
+{
+ struct v4l2_format *f;
+ int ret;
+
+ inst->fmt_src = kzalloc(sizeof(*inst->fmt_src), GFP_KERNEL);
+ if (!inst->fmt_src)
+ return -ENOMEM;
+
+ inst->fmt_dst = kzalloc(sizeof(*inst->fmt_dst), GFP_KERNEL);
+ if (!inst->fmt_dst)
+ return -ENOMEM;
+
+ inst->vb2q_src = kzalloc(sizeof(*inst->vb2q_src), GFP_KERNEL);
+ if (!inst->vb2q_src)
+ return -ENOMEM;
+
+ inst->vb2q_dst = kzalloc(sizeof(*inst->vb2q_dst), GFP_KERNEL);
+ if (!inst->vb2q_dst)
+ return -ENOMEM;
+
+ f = inst->fmt_dst;
+ f->type = OUTPUT_MPLANE;
+ f->fmt.pix_mp.width = DEFAULT_WIDTH;
+ f->fmt.pix_mp.height = DEFAULT_HEIGHT;
+ f->fmt.pix_mp.pixelformat = V4L2_PIX_FMT_H264;
+ f->fmt.pix_mp.num_planes = 1;
+ f->fmt.pix_mp.plane_fmt[0].bytesperline = 0;
+ f->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_OUTPUT);
+ f->fmt.pix_mp.field = V4L2_FIELD_NONE;
+ f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT;
+ f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT;
+ f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+ inst->buffers.output.min_count = iris_get_buf_min_count(inst, BUF_OUTPUT);
+ inst->buffers.output.actual_count = inst->buffers.output.min_count;
+ inst->buffers.output.size = f->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+ inst->crop.left = 0;
+ inst->crop.top = 0;
+ inst->crop.width = f->fmt.pix_mp.width;
+ inst->crop.height = f->fmt.pix_mp.height;
+
+ inst->compose.left = 0;
+ inst->compose.top = 0;
+ inst->compose.width = f->fmt.pix_mp.width;
+ inst->compose.height = f->fmt.pix_mp.height;
+
+ f = inst->fmt_src;
+ f->type = INPUT_MPLANE;
+ f->fmt.pix_mp.pixelformat = V4L2_PIX_FMT_QC08C;
+ f->fmt.pix_mp.width = DEFAULT_WIDTH;
+ f->fmt.pix_mp.height = DEFAULT_HEIGHT;
+ f->fmt.pix_mp.num_planes = 1;
+ f->fmt.pix_mp.plane_fmt[0].bytesperline = ALIGN(DEFAULT_WIDTH, 128);
+ f->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_INPUT);
+ f->fmt.pix_mp.field = V4L2_FIELD_NONE;
+ f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT;
+ f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT;
+ f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+ inst->buffers.input.min_count = iris_get_buf_min_count(inst, BUF_INPUT);
+ inst->buffers.input.actual_count = inst->buffers.input.min_count;
+ inst->buffers.input.size = f->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+ inst->hfi_rc_type = HFI_RC_VBR_CFR;
+ inst->hfi_layer_type = HFI_HIER_P_SLIDING_WINDOW;
+
+ ret = codec_change(inst, inst->fmt_dst->fmt.pix_mp.pixelformat);
+
+ return ret;
+}
+
+void venc_inst_deinit(struct iris_inst *inst)
+{
+ kfree(inst->fmt_dst);
+ kfree(inst->fmt_src);
+}
+
+int venc_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f)
+{
+ struct iris_core *core;
+ u32 array[32] = {0};
+ u32 i = 0;
+
+ core = inst->core;
+
+ if (f->type == OUTPUT_MPLANE) {
+ u32 codecs = core->cap[ENC_CODECS].value;
+ u32 idx = 0;
+
+ for (i = 0; i <= 31; i++) {
+ if (codecs & BIT(i)) {
+ if (idx >= ARRAY_SIZE(array))
+ break;
+ array[idx] = codecs & BIT(i);
+ idx++;
+ }
+ }
+ if (!array[f->index])
+ return -EINVAL;
+ f->pixelformat = v4l2_codec_from_driver(inst, array[f->index]);
+ if (!f->pixelformat)
+ return -EINVAL;
+ f->flags = V4L2_FMT_FLAG_COMPRESSED;
+ strscpy(f->description, "codec", sizeof(f->description));
+ } else if (f->type == INPUT_MPLANE) {
+ u32 formats = inst->cap[PIX_FMTS].step_or_mask;
+ u32 idx = 0;
+
+ for (i = 0; i <= 31; i++) {
+ if (formats & BIT(i)) {
+ if (idx >= ARRAY_SIZE(array))
+ break;
+ array[idx] = formats & BIT(i);
+ idx++;
+ }
+ }
+ if (!array[f->index])
+ return -EINVAL;
+ f->pixelformat = v4l2_colorformat_from_driver(inst, array[f->index]);
+ if (!f->pixelformat)
+ return -EINVAL;
+ strscpy(f->description, "colorformat", sizeof(f->description));
+ }
+
+ memset(f->reserved, 0, sizeof(f->reserved));
+
+ return 0;
+}
+
+int venc_try_fmt(struct iris_inst *inst, struct v4l2_format *f)
+{
+ struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
+ u32 pix_fmt;
+
+ memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
+
+ if (f->type == INPUT_MPLANE) {
+ pix_fmt = v4l2_colorformat_to_driver(inst, f->fmt.pix_mp.pixelformat);
+ if (!pix_fmt) {
+ f->fmt.pix_mp.pixelformat = inst->fmt_src->fmt.pix_mp.pixelformat;
+ f->fmt.pix_mp.width = inst->fmt_src->fmt.pix_mp.width;
+ f->fmt.pix_mp.height = inst->fmt_src->fmt.pix_mp.height;
+ }
+ } else if (f->type == OUTPUT_MPLANE) {
+ pix_fmt = v4l2_codec_to_driver(inst, f->fmt.pix_mp.pixelformat);
+ if (!pix_fmt) {
+ f->fmt.pix_mp.width = inst->fmt_dst->fmt.pix_mp.width;
+ f->fmt.pix_mp.height = inst->fmt_dst->fmt.pix_mp.height;
+ f->fmt.pix_mp.pixelformat = inst->fmt_dst->fmt.pix_mp.pixelformat;
+ }
+ } else {
+ return -EINVAL;
+ }
+
+ if (pixmp->field == V4L2_FIELD_ANY)
+ pixmp->field = V4L2_FIELD_NONE;
+ pixmp->num_planes = 1;
+
+ return 0;
+}
+
+static int venc_s_fmt_output(struct iris_inst *inst, struct v4l2_format *f)
+{
+ struct v4l2_format *fmt;
+ enum codec_type codec;
+ u32 codec_align;
+ u32 width, height;
+ int ret = 0;
+
+ venc_try_fmt(inst, f);
+
+ fmt = inst->fmt_dst;
+ if (fmt->fmt.pix_mp.pixelformat != f->fmt.pix_mp.pixelformat) {
+ ret = codec_change(inst, f->fmt.pix_mp.pixelformat);
+ if (ret)
+ return ret;
+ }
+ fmt->type = OUTPUT_MPLANE;
+
+ codec = v4l2_codec_to_driver(inst, f->fmt.pix_mp.pixelformat);
+
+ codec_align = (codec == HEVC) ? 32 : 16;
+ width = inst->compose.width;
+ height = inst->compose.height;
+ if (inst->cap[ROTATION].value == 90 || inst->cap[ROTATION].value == 270) {
+ width = inst->compose.height;
+ height = inst->compose.width;
+ }
+ fmt->fmt.pix_mp.width = ALIGN(width, codec_align);
+ fmt->fmt.pix_mp.height = ALIGN(height, codec_align);
+ fmt->fmt.pix_mp.num_planes = 1;
+ fmt->fmt.pix_mp.plane_fmt[0].bytesperline = 0;
+ fmt->fmt.pix_mp.plane_fmt[0].sizeimage =
+ iris_get_buffer_size(inst, BUF_OUTPUT);
+ if (f->fmt.pix_mp.colorspace != V4L2_COLORSPACE_DEFAULT &&
+ f->fmt.pix_mp.colorspace != V4L2_COLORSPACE_REC709)
+ f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT;
+ fmt->fmt.pix_mp.colorspace = f->fmt.pix_mp.colorspace;
+ fmt->fmt.pix_mp.xfer_func = f->fmt.pix_mp.xfer_func;
+ fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc;
+ fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization;
+ inst->buffers.output.min_count = iris_get_buf_min_count(inst, BUF_OUTPUT);
+ if (inst->buffers.output.actual_count <
+ inst->buffers.output.min_count) {
+ inst->buffers.output.actual_count =
+ inst->buffers.output.min_count;
+ }
+ inst->buffers.output.size =
+ fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+ memcpy(f, fmt, sizeof(struct v4l2_format));
+
+ return ret;
+}
+
+static int venc_s_fmt_input(struct iris_inst *inst, struct v4l2_format *f)
+{
+ u32 pix_fmt, width, height, size, bytesperline;
+ struct v4l2_format *fmt, *output_fmt;
+ int ret = 0;
+
+ venc_try_fmt(inst, f);
+
+ pix_fmt = v4l2_colorformat_to_driver(inst, f->fmt.pix_mp.pixelformat);
+ inst->cap[PIX_FMTS].value = pix_fmt;
+
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+
+ bytesperline = pix_fmt == FMT_TP10C ?
+ ALIGN(ALIGN(f->fmt.pix_mp.width, 192) * 4 / 3, 256) :
+ ALIGN(f->fmt.pix_mp.width, 128);
+
+ fmt = inst->fmt_src;
+ fmt->type = INPUT_MPLANE;
+ fmt->fmt.pix_mp.width = width;
+ fmt->fmt.pix_mp.height = height;
+ fmt->fmt.pix_mp.num_planes = 1;
+ fmt->fmt.pix_mp.pixelformat = f->fmt.pix_mp.pixelformat;
+ fmt->fmt.pix_mp.plane_fmt[0].bytesperline = bytesperline;
+ size = iris_get_buffer_size(inst, BUF_INPUT);
+ fmt->fmt.pix_mp.plane_fmt[0].sizeimage = size;
+ fmt->fmt.pix_mp.colorspace = f->fmt.pix_mp.colorspace;
+ fmt->fmt.pix_mp.xfer_func = f->fmt.pix_mp.xfer_func;
+ fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc;
+ fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization;
+
+ output_fmt = inst->fmt_dst;
+ output_fmt->fmt.pix_mp.colorspace = fmt->fmt.pix_mp.colorspace;
+ output_fmt->fmt.pix_mp.xfer_func = fmt->fmt.pix_mp.xfer_func;
+ output_fmt->fmt.pix_mp.ycbcr_enc = fmt->fmt.pix_mp.ycbcr_enc;
+ output_fmt->fmt.pix_mp.quantization = fmt->fmt.pix_mp.quantization;
+
+ inst->buffers.input.min_count = iris_get_buf_min_count(inst, BUF_INPUT);
+ if (inst->buffers.input.actual_count <
+ inst->buffers.input.min_count) {
+ inst->buffers.input.actual_count =
+ inst->buffers.input.min_count;
+ }
+ inst->buffers.input.size = size;
+
+ if (f->fmt.pix_mp.width != inst->crop.width ||
+ f->fmt.pix_mp.height != inst->crop.height) {
+ inst->crop.top = 0;
+ inst->crop.left = 0;
+ inst->crop.width = f->fmt.pix_mp.width;
+ inst->crop.height = f->fmt.pix_mp.height;
+
+ inst->compose.top = 0;
+ inst->compose.left = 0;
+ inst->compose.width = f->fmt.pix_mp.width;
+ inst->compose.height = f->fmt.pix_mp.height;
+
+ ret = venc_s_fmt_output(inst, output_fmt);
+ if (ret)
+ return ret;
+ }
+
+ memcpy(f, fmt, sizeof(struct v4l2_format));
+
+ return ret;
+}
+
+int venc_s_fmt(struct iris_inst *inst, struct v4l2_format *f)
+{
+ int ret;
+
+ if (f->type == INPUT_MPLANE)
+ ret = venc_s_fmt_input(inst, f);
+ else if (f->type == OUTPUT_MPLANE)
+ ret = venc_s_fmt_output(inst, f);
+ else
+ ret = -EINVAL;
+
+ return ret;
+}
+
+int venc_s_selection(struct iris_inst *inst, struct v4l2_selection *s)
+{
+ struct v4l2_format *output_fmt;
+ int ret;
+
+ if (s->type != INPUT_MPLANE && s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
+ return -EINVAL;
+
+ switch (s->target) {
+ case V4L2_SEL_TGT_CROP:
+ if (s->r.left || s->r.top) {
+ s->r.left = 0;
+ s->r.top = 0;
+ }
+ if (s->r.width > inst->fmt_src->fmt.pix_mp.width)
+ s->r.width = inst->fmt_src->fmt.pix_mp.width;
+
+ if (s->r.height > inst->fmt_src->fmt.pix_mp.height)
+ s->r.height = inst->fmt_src->fmt.pix_mp.height;
+
+ inst->crop.left = s->r.left;
+ inst->crop.top = s->r.top;
+ inst->crop.width = s->r.width;
+ inst->crop.height = s->r.height;
+ inst->compose.left = inst->crop.left;
+ inst->compose.top = inst->crop.top;
+ inst->compose.width = inst->crop.width;
+ inst->compose.height = inst->crop.height;
+ output_fmt = inst->fmt_dst;
+ ret = venc_s_fmt_output(inst, output_fmt);
+ if (ret)
+ return ret;
+ break;
+ case V4L2_SEL_TGT_COMPOSE:
+ if (s->r.left < inst->crop.left)
+ s->r.left = inst->crop.left;
+
+ if (s->r.top < inst->crop.top)
+ s->r.top = inst->crop.top;
+
+ if (s->r.width > inst->crop.width)
+ s->r.width = inst->crop.width;
+
+ if (s->r.height > inst->crop.height)
+ s->r.height = inst->crop.height;
+ inst->compose.left = s->r.left;
+ inst->compose.top = s->r.top;
+ inst->compose.width = s->r.width;
+ inst->compose.height = s->r.height;
+
+ output_fmt = inst->fmt_dst;
+ ret = venc_s_fmt_output(inst, output_fmt);
+ if (ret)
+ return ret;
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+
+ return ret;
+}
+
+int venc_s_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm)
+{
+ struct v4l2_fract *timeperframe = NULL;
+ u32 q16_rate, max_rate, default_rate;
+ u64 us_per_frame = 0, input_rate = 0;
+ bool is_frame_rate = false;
+ int ret = 0;
+
+ if (s_parm->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ timeperframe = &s_parm->parm.output.timeperframe;
+ max_rate = inst->cap[OPERATING_RATE].max >> 16;
+ default_rate = inst->cap[OPERATING_RATE].value >> 16;
+ s_parm->parm.output.capability = V4L2_CAP_TIMEPERFRAME;
+ } else {
+ timeperframe = &s_parm->parm.capture.timeperframe;
+ is_frame_rate = true;
+ max_rate = inst->cap[FRAME_RATE].max >> 16;
+ default_rate = inst->cap[FRAME_RATE].value >> 16;
+ s_parm->parm.capture.capability = V4L2_CAP_TIMEPERFRAME;
+ }
+
+ if (!timeperframe->denominator || !timeperframe->numerator) {
+ if (!timeperframe->numerator)
+ timeperframe->numerator = 1;
+ if (!timeperframe->denominator)
+ timeperframe->denominator = default_rate;
+ }
+
+ us_per_frame = timeperframe->numerator * (u64)USEC_PER_SEC;
+ us_per_frame = div64_u64(us_per_frame, timeperframe->denominator);
+
+ if (!us_per_frame) {
+ ret = -EINVAL;
+ goto exit;
+ }
+
+ input_rate = (u64)USEC_PER_SEC;
+ input_rate = div64_u64(input_rate, us_per_frame);
+
+ q16_rate = (u32)input_rate << 16;
+ if (is_frame_rate)
+ inst->cap[FRAME_RATE].value = q16_rate;
+ else
+ inst->cap[OPERATING_RATE].value = q16_rate;
+
+ if ((s_parm->type == INPUT_MPLANE && inst->vb2q_src->streaming) ||
+ (s_parm->type == OUTPUT_MPLANE && inst->vb2q_dst->streaming)) {
+ ret = check_core_mbps_mbpf(inst);
+ if (ret)
+ goto reset_rate;
+ ret = input_rate > max_rate;
+ if (ret) {
+ ret = -ENOMEM;
+ goto reset_rate;
+ }
+ }
+
+ if (is_frame_rate)
+ inst->cap[FRAME_RATE].flags |= CAP_FLAG_CLIENT_SET;
+ else
+ inst->cap[OPERATING_RATE].flags |= CAP_FLAG_CLIENT_SET;
+
+ if (inst->vb2q_dst->streaming) {
+ ret = iris_hfi_set_property(inst,
+ HFI_PROP_FRAME_RATE,
+ HFI_HOST_FLAGS_NONE,
+ HFI_PORT_BITSTREAM,
+ HFI_PAYLOAD_Q16,
+ &q16_rate,
+ sizeof(u32));
+ if (ret)
+ goto exit;
+ }
+
+ return ret;
+
+reset_rate:
+ if (ret) {
+ if (is_frame_rate)
+ inst->cap[FRAME_RATE].value = default_rate << 16;
+ else
+ inst->cap[OPERATING_RATE].value = default_rate << 16;
+ }
+exit:
+ return ret;
+}
+
+int venc_g_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm)
+{
+ struct v4l2_fract *timeperframe = NULL;
+
+ if (s_parm->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ timeperframe = &s_parm->parm.output.timeperframe;
+ timeperframe->numerator = 1;
+ timeperframe->denominator =
+ inst->cap[OPERATING_RATE].value >> 16;
+ s_parm->parm.output.capability = V4L2_CAP_TIMEPERFRAME;
+ } else {
+ timeperframe = &s_parm->parm.capture.timeperframe;
+ timeperframe->numerator = 1;
+ timeperframe->denominator =
+ inst->cap[FRAME_RATE].value >> 16;
+ s_parm->parm.capture.capability = V4L2_CAP_TIMEPERFRAME;
+ }
+
+ return 0;
+}
+
+int venc_subscribe_event(struct iris_inst *inst,
+ const struct v4l2_event_subscription *sub)
+{
+ int ret;
+
+ switch (sub->type) {
+ case V4L2_EVENT_EOS:
+ ret = v4l2_event_subscribe(&inst->fh, sub, MAX_EVENTS, NULL);
+ break;
+ case V4L2_EVENT_CTRL:
+ ret = v4l2_ctrl_subscribe_event(&inst->fh, sub);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return ret;
+}
+
+int venc_start_cmd(struct iris_inst *inst)
+{
+ vb2_clear_last_buffer_dequeued(inst->vb2q_dst);
+
+ return process_resume(inst);
+}
+
+int venc_stop_cmd(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = iris_hfi_drain(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = iris_inst_change_sub_state(inst, 0, IRIS_INST_SUB_DRAIN);
+
+ iris_scale_power(inst);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_venc.h b/drivers/media/platform/qcom/vcodec/iris/iris_venc.h
new file mode 100644
index 0000000..24da63f
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_venc.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_VENC_H_
+#define _IRIS_VENC_H_
+
+#include "iris_instance.h"
+
+int venc_inst_init(struct iris_inst *inst);
+void venc_inst_deinit(struct iris_inst *inst);
+int venc_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f);
+int venc_try_fmt(struct iris_inst *inst, struct v4l2_format *f);
+int venc_s_fmt(struct iris_inst *inst, struct v4l2_format *f);
+int venc_s_selection(struct iris_inst *inst, struct v4l2_selection *s);
+int venc_s_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm);
+int venc_g_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm);
+int venc_subscribe_event(struct iris_inst *inst,
+ const struct v4l2_event_subscription *sub);
+int venc_start_cmd(struct iris_inst *inst);
+int venc_stop_cmd(struct iris_inst *inst);
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
index 437d6b4..aa19978 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
@@ -15,6 +15,7 @@
#include "iris_instance.h"
#include "iris_power.h"
#include "iris_vdec.h"
+#include "iris_venc.h"
#include "iris_vidc.h"
#include "iris_vb2.h"

@@ -30,7 +31,11 @@ static int vidc_v4l2_fh_init(struct iris_inst *inst)
if (inst->fh.vdev)
return -EINVAL;

- v4l2_fh_init(&inst->fh, core->vdev_dec);
+ if (inst->domain == ENCODER)
+ v4l2_fh_init(&inst->fh, core->vdev_enc);
+ else if (inst->domain == DECODER)
+ v4l2_fh_init(&inst->fh, core->vdev_dec);
+
inst->fh.ctrl_handler = &inst->ctrl_handler;
v4l2_fh_add(&inst->fh);

@@ -160,9 +165,20 @@ int vidc_open(struct file *filp)
{
struct iris_core *core = video_drvdata(filp);
struct iris_inst *inst = NULL;
+ struct video_device *vdev;
+ u32 session_type = 0;
int i = 0;
int ret;

+ vdev = video_devdata(filp);
+ if (strcmp(vdev->name, "qcom-iris-decoder") == 0)
+ session_type = DECODER;
+ else if (strcmp(vdev->name, "qcom-iris-encoder") == 0)
+ session_type = ENCODER;
+
+ if (session_type != DECODER && session_type != ENCODER)
+ return -EINVAL;
+
ret = iris_pm_get(core);
if (ret)
return ret;
@@ -182,6 +198,7 @@ int vidc_open(struct file *filp)
}

inst->core = core;
+ inst->domain = session_type;
inst->session_id = hash32_ptr(inst);
inst->ipsc_properties_set = false;
inst->opsc_properties_set = false;
@@ -213,7 +230,12 @@ int vidc_open(struct file *filp)
if (ret)
goto fail_remove_session;

- vdec_inst_init(inst);
+ if (inst->domain == DECODER)
+ ret = vdec_inst_init(inst);
+ else if (inst->domain == ENCODER)
+ ret = venc_inst_init(inst);
+ if (ret)
+ goto fail_fh_deinit;

ret = vidc_vb2_queue_init(inst);
if (ret)
@@ -238,7 +260,11 @@ int vidc_open(struct file *filp)
iris_core_deinit(core);
vidc_vb2_queue_deinit(inst);
fail_inst_deinit:
- vdec_inst_deinit(inst);
+ if (inst->domain == DECODER)
+ vdec_inst_deinit(inst);
+ else if (inst->domain == ENCODER)
+ venc_inst_deinit(inst);
+fail_fh_deinit:
vidc_v4l2_fh_deinit(inst);
fail_remove_session:
vidc_remove_session(inst);
@@ -264,7 +290,11 @@ int vidc_close(struct file *filp)
core = inst->core;

v4l2_ctrl_handler_free(&inst->ctrl_handler);
- vdec_inst_deinit(inst);
+ if (inst->domain == DECODER)
+ vdec_inst_deinit(inst);
+ else if (inst->domain == ENCODER)
+ venc_inst_deinit(inst);
+
mutex_lock(&inst->lock);
iris_pm_get(core);
close_session(inst);
@@ -342,7 +372,7 @@ static __poll_t vidc_poll(struct file *filp, struct poll_table_struct *pt)
static int vidc_enum_fmt(struct file *filp, void *fh, struct v4l2_fmtdesc *f)
{
struct iris_inst *inst;
- int ret;
+ int ret = 0;

inst = get_vidc_inst(filp, fh);
if (!inst)
@@ -354,7 +384,10 @@ static int vidc_enum_fmt(struct file *filp, void *fh, struct v4l2_fmtdesc *f)
goto unlock;
}

- ret = vdec_enum_fmt(inst, f);
+ if (inst->domain == DECODER)
+ ret = vdec_enum_fmt(inst, f);
+ else if (inst->domain == ENCODER)
+ ret = venc_enum_fmt(inst, f);

unlock:
mutex_unlock(&inst->lock);
@@ -365,7 +398,7 @@ static int vidc_enum_fmt(struct file *filp, void *fh, struct v4l2_fmtdesc *f)
static int vidc_try_fmt(struct file *filp, void *fh, struct v4l2_format *f)
{
struct iris_inst *inst;
- int ret;
+ int ret = 0;

inst = get_vidc_inst(filp, fh);
if (!inst)
@@ -382,7 +415,10 @@ static int vidc_try_fmt(struct file *filp, void *fh, struct v4l2_format *f)
goto unlock;
}

- ret = vdec_try_fmt(inst, f);
+ if (inst->domain == DECODER)
+ ret = vdec_try_fmt(inst, f);
+ else if (inst->domain == ENCODER)
+ ret = venc_try_fmt(inst, f);

unlock:
mutex_unlock(&inst->lock);
@@ -393,7 +429,7 @@ static int vidc_try_fmt(struct file *filp, void *fh, struct v4l2_format *f)
static int vidc_s_fmt(struct file *filp, void *fh, struct v4l2_format *f)
{
struct iris_inst *inst;
- int ret;
+ int ret = 0;

inst = get_vidc_inst(filp, fh);
if (!inst)
@@ -410,7 +446,10 @@ static int vidc_s_fmt(struct file *filp, void *fh, struct v4l2_format *f)
goto unlock;
}

- ret = vdec_s_fmt(inst, f);
+ if (inst->domain == DECODER)
+ ret = vdec_s_fmt(inst, f);
+ else if (inst->domain == ENCODER)
+ ret = venc_s_fmt(inst, f);

unlock:
mutex_unlock(&inst->lock);
@@ -488,6 +527,70 @@ static int vidc_enum_framesizes(struct file *filp, void *fh,
return ret;
}

+static int vidc_enum_frameintervals(struct file *filp, void *fh,
+ struct v4l2_frmivalenum *fival)
+
+{
+ enum colorformat_type colorfmt;
+ struct iris_inst *inst;
+ struct iris_core *core;
+ u32 fps, mbpf;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !fival)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (inst->domain == DECODER) {
+ ret = -ENOTTY;
+ goto unlock;
+ }
+
+ core = inst->core;
+
+ if (fival->index) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ colorfmt = v4l2_colorformat_to_driver(inst, fival->pixel_format);
+ if (colorfmt == FMT_NONE) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (fival->width > inst->cap[FRAME_WIDTH].max ||
+ fival->width < inst->cap[FRAME_WIDTH].min ||
+ fival->height > inst->cap[FRAME_HEIGHT].max ||
+ fival->height < inst->cap[FRAME_HEIGHT].min) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ mbpf = NUM_MBS_PER_FRAME(fival->height, fival->width);
+ fps = core->cap[MAX_MBPS].value / mbpf;
+
+ fival->type = V4L2_FRMIVAL_TYPE_STEPWISE;
+ fival->stepwise.min.numerator = 1;
+ fival->stepwise.min.denominator =
+ min_t(u32, fps, inst->cap[FRAME_RATE].max);
+ fival->stepwise.max.numerator = 1;
+ fival->stepwise.max.denominator = 1;
+ fival->stepwise.step.numerator = 1;
+ fival->stepwise.step.denominator = inst->cap[FRAME_RATE].max;
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
static int vidc_reqbufs(struct file *filp, void *fh, struct v4l2_requestbuffers *b)
{
struct vb2_queue *vb2q = NULL;
@@ -751,7 +854,11 @@ static int vidc_querycap(struct file *filp, void *fh, struct v4l2_capability *ca
strscpy(cap->driver, VIDC_DRV_NAME, sizeof(cap->driver));
strscpy(cap->bus_info, VIDC_BUS_NAME, sizeof(cap->bus_info));
memset(cap->reserved, 0, sizeof(cap->reserved));
- strscpy(cap->card, "iris_decoder", sizeof(cap->card));
+
+ if (inst->domain == DECODER)
+ strscpy(cap->card, "iris_decoder", sizeof(cap->card));
+ else if (inst->domain == ENCODER)
+ strscpy(cap->card, "iris_encoder", sizeof(cap->card));

unlock:
mutex_unlock(&inst->lock);
@@ -839,7 +946,7 @@ static int vidc_querymenu(struct file *filp, void *fh, struct v4l2_querymenu *qm
static int vidc_subscribe_event(struct v4l2_fh *fh, const struct v4l2_event_subscription *sub)
{
struct iris_inst *inst;
- int ret;
+ int ret = 0;

inst = container_of(fh, struct iris_inst, fh);

@@ -849,7 +956,10 @@ static int vidc_subscribe_event(struct v4l2_fh *fh, const struct v4l2_event_subs
goto unlock;
}

- ret = vdec_subscribe_event(inst, sub);
+ if (inst->domain == DECODER)
+ ret = vdec_subscribe_event(inst, sub);
+ else if (inst->domain == ENCODER)
+ ret = venc_subscribe_event(inst, sub);

unlock:
mutex_unlock(&inst->lock);
@@ -893,28 +1003,152 @@ static int vidc_g_selection(struct file *filp, void *fh, struct v4l2_selection *
goto unlock;
}

- if (s->type != OUTPUT_MPLANE && s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE) {
+ if (s->type != OUTPUT_MPLANE && s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE &&
+ inst->domain == DECODER) {
ret = -EINVAL;
goto unlock;
}

- switch (s->target) {
- case V4L2_SEL_TGT_CROP_BOUNDS:
- case V4L2_SEL_TGT_CROP_DEFAULT:
- case V4L2_SEL_TGT_CROP:
- case V4L2_SEL_TGT_COMPOSE_BOUNDS:
- case V4L2_SEL_TGT_COMPOSE_PADDED:
- case V4L2_SEL_TGT_COMPOSE_DEFAULT:
- case V4L2_SEL_TGT_COMPOSE:
- s->r.left = inst->crop.left;
- s->r.top = inst->crop.top;
- s->r.width = inst->crop.width;
- s->r.height = inst->crop.height;
- break;
- default:
+ if (s->type != INPUT_MPLANE && s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT &&
+ inst->domain == ENCODER) {
ret = -EINVAL;
+ goto unlock;
}

+ if (inst->domain == DECODER) {
+ switch (s->target) {
+ case V4L2_SEL_TGT_CROP_BOUNDS:
+ case V4L2_SEL_TGT_CROP_DEFAULT:
+ case V4L2_SEL_TGT_CROP:
+ case V4L2_SEL_TGT_COMPOSE_BOUNDS:
+ case V4L2_SEL_TGT_COMPOSE_PADDED:
+ case V4L2_SEL_TGT_COMPOSE_DEFAULT:
+ case V4L2_SEL_TGT_COMPOSE:
+ s->r.left = inst->crop.left;
+ s->r.top = inst->crop.top;
+ s->r.width = inst->crop.width;
+ s->r.height = inst->crop.height;
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+ } else if (inst->domain == ENCODER) {
+ switch (s->target) {
+ case V4L2_SEL_TGT_CROP_BOUNDS:
+ case V4L2_SEL_TGT_CROP_DEFAULT:
+ case V4L2_SEL_TGT_CROP:
+ s->r.left = inst->crop.left;
+ s->r.top = inst->crop.top;
+ s->r.width = inst->crop.width;
+ s->r.height = inst->crop.height;
+ break;
+ case V4L2_SEL_TGT_COMPOSE_BOUNDS:
+ case V4L2_SEL_TGT_COMPOSE_PADDED:
+ case V4L2_SEL_TGT_COMPOSE_DEFAULT:
+ case V4L2_SEL_TGT_COMPOSE:
+ s->r.left = inst->compose.left;
+ s->r.top = inst->compose.top;
+ s->r.width = inst->compose.width;
+ s->r.height = inst->compose.height;
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+ }
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_s_selection(struct file *filp, void *fh, struct v4l2_selection *s)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !s)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+ if (inst->domain == DECODER) {
+ ret = -EINVAL;
+ goto unlock;
+ } else if (inst->domain == ENCODER) {
+ ret = venc_s_selection(inst, s);
+ }
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_s_parm(struct file *filp, void *fh, struct v4l2_streamparm *a)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !a)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+ a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (inst->domain == ENCODER)
+ ret = venc_s_param(inst, a);
+ else
+ ret = -EINVAL;
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_g_parm(struct file *filp, void *fh, struct v4l2_streamparm *a)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !a)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+ a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (inst->domain == ENCODER)
+ ret = venc_g_param(inst, a);
+ else
+ ret = -EINVAL;
+
unlock:
mutex_unlock(&inst->lock);

@@ -955,6 +1189,39 @@ static int vidc_try_dec_cmd(struct file *filp, void *fh,
return ret;
}

+static int vidc_try_enc_cmd(struct file *filp, void *fh,
+ struct v4l2_encoder_cmd *enc)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !enc)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (inst->domain != ENCODER) {
+ ret = -ENOTTY;
+ goto unlock;
+ }
+
+ if (enc->cmd != V4L2_ENC_CMD_STOP && enc->cmd != V4L2_ENC_CMD_START) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+ enc->flags = 0;
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
static int vidc_dec_cmd(struct file *filp, void *fh,
struct v4l2_decoder_cmd *dec)
{
@@ -1002,6 +1269,60 @@ static int vidc_dec_cmd(struct file *filp, void *fh,
return ret;
}

+static int vidc_enc_cmd(struct file *filp, void *fh,
+ struct v4l2_encoder_cmd *enc)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !enc)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (inst->domain != ENCODER) {
+ ret = -ENOTTY;
+ goto unlock;
+ }
+
+ if (enc->cmd != V4L2_ENC_CMD_START &&
+ enc->cmd != V4L2_ENC_CMD_STOP) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (enc->cmd == V4L2_ENC_CMD_STOP && inst->state == IRIS_INST_OPEN) {
+ ret = 0;
+ goto unlock;
+ }
+
+ if (!allow_cmd(inst, enc->cmd)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ ret = iris_pm_get(inst->core);
+ if (ret)
+ goto unlock;
+
+ if (enc->cmd == V4L2_ENC_CMD_START)
+ ret = venc_start_cmd(inst);
+ else if (enc->cmd == V4L2_ENC_CMD_STOP)
+ ret = venc_stop_cmd(inst);
+
+ iris_pm_put(inst->core, true);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
static struct v4l2_file_operations v4l2_file_ops = {
.owner = THIS_MODULE,
.open = vidc_open,
@@ -1027,7 +1348,7 @@ static struct vb2_mem_ops iris_vb2_mem_ops = {
.unmap_dmabuf = iris_vb2_unmap_dmabuf,
};

-static const struct v4l2_ioctl_ops v4l2_ioctl_ops = {
+static const struct v4l2_ioctl_ops v4l2_ioctl_ops_dec = {
.vidioc_enum_fmt_vid_cap = vidc_enum_fmt,
.vidioc_enum_fmt_vid_out = vidc_enum_fmt,
.vidioc_try_fmt_vid_cap_mplane = vidc_try_fmt,
@@ -1055,12 +1376,44 @@ static const struct v4l2_ioctl_ops v4l2_ioctl_ops = {
.vidioc_decoder_cmd = vidc_dec_cmd,
};

+static const struct v4l2_ioctl_ops v4l2_ioctl_ops_enc = {
+ .vidioc_enum_fmt_vid_cap = vidc_enum_fmt,
+ .vidioc_enum_fmt_vid_out = vidc_enum_fmt,
+ .vidioc_try_fmt_vid_cap_mplane = vidc_try_fmt,
+ .vidioc_try_fmt_vid_out_mplane = vidc_try_fmt,
+ .vidioc_s_fmt_vid_cap_mplane = vidc_s_fmt,
+ .vidioc_s_fmt_vid_out_mplane = vidc_s_fmt,
+ .vidioc_g_fmt_vid_cap_mplane = vidc_g_fmt,
+ .vidioc_g_fmt_vid_out_mplane = vidc_g_fmt,
+ .vidioc_enum_framesizes = vidc_enum_framesizes,
+ .vidioc_enum_frameintervals = vidc_enum_frameintervals,
+ .vidioc_reqbufs = vidc_reqbufs,
+ .vidioc_querybuf = vidc_querybuf,
+ .vidioc_create_bufs = vidc_create_bufs,
+ .vidioc_prepare_buf = vidc_prepare_buf,
+ .vidioc_qbuf = vidc_qbuf,
+ .vidioc_dqbuf = vidc_dqbuf,
+ .vidioc_streamon = vidc_streamon,
+ .vidioc_streamoff = vidc_streamoff,
+ .vidioc_querycap = vidc_querycap,
+ .vidioc_queryctrl = vidc_queryctrl,
+ .vidioc_querymenu = vidc_querymenu,
+ .vidioc_subscribe_event = vidc_subscribe_event,
+ .vidioc_unsubscribe_event = vidc_unsubscribe_event,
+ .vidioc_g_selection = vidc_g_selection,
+ .vidioc_s_selection = vidc_s_selection,
+ .vidioc_s_parm = vidc_s_parm,
+ .vidioc_g_parm = vidc_g_parm,
+ .vidioc_try_encoder_cmd = vidc_try_enc_cmd,
+ .vidioc_encoder_cmd = vidc_enc_cmd,
+};
+
int init_ops(struct iris_core *core)
{
core->v4l2_file_ops = &v4l2_file_ops;
core->vb2_ops = &iris_vb2_ops;
core->vb2_mem_ops = &iris_vb2_mem_ops;
- core->v4l2_ioctl_ops = &v4l2_ioctl_ops;
-
+ core->v4l2_ioctl_ops_dec = &v4l2_ioctl_ops_dec;
+ core->v4l2_ioctl_ops_enc = &v4l2_ioctl_ops_enc;
return 0;
}
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
index 443894c..effecbb 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
@@ -115,6 +115,7 @@ struct iris_inst_power {
enum plat_core_cap_type {
CORE_CAP_NONE = 0,
DEC_CODECS,
+ ENC_CODECS,
MAX_SESSION_COUNT,
MAX_MBPF,
MAX_MBPS,
@@ -249,6 +250,7 @@ struct plat_inst_cap {

struct plat_inst_caps {
enum codec_type codec;
+ enum domain_type domain;
struct plat_inst_cap cap[INST_CAP_MAX + 1];
};

diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
index 6d5192a..ef0aad7 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
@@ -63,6 +63,7 @@ static struct color_format_info color_format_data_sm8550[] = {

static struct plat_core_cap core_data_sm8550[] = {
{DEC_CODECS, H264 | HEVC | VP9},
+ {ENC_CODECS, H264 | HEVC},
{MAX_SESSION_COUNT, 16},
{MAX_MBPF, 278528}, /* ((8192x4352)/256) * 2 */
{MAX_MBPS, 7833600}, /* max_load 7680x4320@60fps */
--
2.7.4

2023-12-18 11:50:45

[permalink] [raw]

Subject: [PATCH v2 31/34] media: iris: add platform specific instance capabilities for encoder

Capabilities are set of video specifications and features supported
by a specific platform(SOC). Each capability is defined with
min, max, range, default value and corresponding HFI.

Also, platform data defines different resource details for
a specific platform(SOC). This change defines resource tables
for sm8550 platform data and use for initializing these resources.

Add Children, Set, Adjust functions for to each capability for
Encoder.

Signed-off-by: Dikshita Agarwal <[email protected]>
---
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 128 ++-
.../media/platform/qcom/vcodec/iris/iris_common.h | 4 +
.../media/platform/qcom/vcodec/iris/iris_ctrls.c | 981 ++++++++++++++++++++-
.../media/platform/qcom/vcodec/iris/iris_ctrls.h | 35 +
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 16 +-
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 53 ++
drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 7 +-
.../platform/qcom/vcodec/iris/iris_hfi_packet.c | 10 +-
.../platform/qcom/vcodec/iris/iris_hfi_packet.h | 5 +
.../platform/qcom/vcodec/iris/iris_hfi_response.c | 6 +
.../platform/qcom/vcodec/iris/iris_instance.h | 6 +
.../platform/qcom/vcodec/iris/platform_common.h | 74 +-
.../platform/qcom/vcodec/iris/platform_sm8550.c | 768 +++++++++++++++-
.../platform/qcom/vcodec/iris/vpu_iris3_power.c | 3 +
14 files changed, 2024 insertions(+), 72 deletions(-)

diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index 872674e..bc32c99 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -45,6 +45,8 @@ enum hfi_property_mode_type {
#define HFI_BITMASK_BITSTREAM_WIDTH 0xffff0000
#define HFI_BITMASK_BITSTREAM_HEIGHT 0x0000ffff

+#define HFI_LEVEL_NONE 0xFFFFFFFF
+
#define HFI_BITMASK_FRAME_MBS_ONLY_FLAG 0x00000001

#define HFI_TRUE 0x00000001
@@ -68,14 +70,14 @@ enum hfi_codec_type {
#define HFI_PROP_CODEC 0x03000100

enum hfi_color_format {
- HFI_COLOR_FMT_OPAQUE = 0,
- HFI_COLOR_FMT_NV12 = 1,
- HFI_COLOR_FMT_NV12_UBWC = 2,
- HFI_COLOR_FMT_P010 = 3,
- HFI_COLOR_FMT_TP10_UBWC = 4,
- HFI_COLOR_FMT_RGBA8888 = 5,
- HFI_COLOR_FMT_RGBA8888_UBWC = 6,
- HFI_COLOR_FMT_NV21 = 7,
+ HFI_COLOR_FMT_OPAQUE = 0,
+ HFI_COLOR_FMT_NV12 = 1,
+ HFI_COLOR_FMT_NV12_UBWC = 2,
+ HFI_COLOR_FMT_P010 = 3,
+ HFI_COLOR_FMT_TP10_UBWC = 4,
+ HFI_COLOR_FMT_RGBA8888 = 5,
+ HFI_COLOR_FMT_RGBA8888_UBWC = 6,
+ HFI_COLOR_FMT_NV21 = 7,
};

#define HFI_PROP_COLOR_FORMAT 0x03000101
@@ -104,14 +106,108 @@ enum hfi_color_format {

#define HFI_PROP_CABAC_SESSION 0x03000121

+#define HFI_PROP_8X8_TRANSFORM 0x03000122
+
#define HFI_PROP_BUFFER_HOST_MAX_COUNT 0x03000123

#define HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT 0x03000124

#define HFI_PROP_PIC_ORDER_CNT_TYPE 0x03000128

+enum hfi_syncframe_request_mode {
+ HFI_SYNC_FRAME_REQUEST_WITHOUT_SEQ_HDR = 0x00000001,
+ HFI_SYNC_FRAME_REQUEST_WITH_PREFIX_SEQ_HDR = 0x00000002,
+};
+
+enum hfi_rate_control {
+ HFI_RC_VBR_CFR = 0x00000000,
+ HFI_RC_CBR_CFR = 0x00000001,
+ HFI_RC_CQ = 0x00000002,
+ HFI_RC_OFF = 0x00000003,
+ HFI_RC_CBR_VFR = 0x00000004,
+ HFI_RC_LOSSLESS = 0x00000005,
+};
+
+#define HFI_PROP_RATE_CONTROL 0x0300012a
+
+#define HFI_PROP_QP_PACKED 0x0300012e
+
+#define HFI_PROP_MIN_QP_PACKED 0x0300012f
+
+#define HFI_PROP_MAX_QP_PACKED 0x03000130
+
+enum hfi_layer_enc_type {
+ HFI_HIER_P_SLIDING_WINDOW = 0x1,
+ HFI_HIER_P_HYBRID_LTR = 0x2,
+ HFI_HIER_B = 0x3,
+};
+
+#define HFI_PROP_IR_RANDOM_PERIOD 0x03000131
+
+#define HFI_PROP_MULTI_SLICE_MB_COUNT 0x03000132
+
+#define HFI_PROP_MULTI_SLICE_BYTES_COUNT 0x03000133
+
+#define HFI_PROP_LTR_COUNT 0x03000134
+
+#define HFI_PROP_LTR_MARK 0x03000135
+
+#define HFI_PROP_LTR_USE 0x03000136
+
+#define HFI_PROP_LAYER_ENCODING_TYPE 0x03000138
+
+#define HFI_PROP_LAYER_COUNT 0x03000139
+
+#define HFI_PROP_TOTAL_BITRATE 0x0300013b
+
+#define HFI_PROP_BITRATE_LAYER1 0x0300013c
+
+#define HFI_PROP_BITRATE_LAYER2 0x0300013d
+
+#define HFI_PROP_BITRATE_LAYER3 0x0300013e
+
+#define HFI_PROP_BITRATE_LAYER4 0x0300013f
+
+#define HFI_PROP_BITRATE_LAYER5 0x03000140
+
+#define HFI_PROP_BITRATE_LAYER6 0x03000141
+
+#define HFI_PROP_BASELAYER_PRIORITYID 0x03000142
+
+#define HFI_PROP_REQUEST_SYNC_FRAME 0x03000145
+
+#define HFI_PROP_MAX_GOP_FRAMES 0x03000146
+
+#define HFI_PROP_MAX_B_FRAMES 0x03000147
+
#define HFI_PROP_QUALITY_MODE 0x03000148

+enum hfi_seq_header_mode {
+ HFI_SEQ_HEADER_SEPERATE_FRAME = 0x00000001,
+ HFI_SEQ_HEADER_JOINED_WITH_1ST_FRAME = 0x00000002,
+ HFI_SEQ_HEADER_PREFIX_WITH_SYNC_FRAME = 0x00000004,
+ HFI_SEQ_HEADER_METADATA = 0x00000008,
+};
+
+#define HFI_PROP_SEQ_HEADER_MODE 0x03000149
+
+enum hfi_rotation {
+ HFI_ROTATION_NONE = 0x00000000,
+ HFI_ROTATION_90 = 0x00000001,
+ HFI_ROTATION_180 = 0x00000002,
+ HFI_ROTATION_270 = 0x00000003,
+};
+
+#define HFI_PROP_ROTATION 0x0300014b
+
+enum hfi_flip {
+ HFI_DISABLE_FLIP = 0x00000000,
+ HFI_HORIZONTAL_FLIP = 0x00000001,
+ HFI_VERTICAL_FLIP = 0x00000002,
+};
+
+#define HFI_PROP_FLIP 0x0300014c
+
enum hfi_color_primaries {
HFI_PRIMARIES_RESERVED = 0,
HFI_PRIMARIES_BT709 = 1,
@@ -182,7 +278,7 @@ enum hfi_picture_type {

#define HFI_PROP_PICTURE_TYPE 0x03000162

-#define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168
+#define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168

#define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169

@@ -190,6 +286,12 @@ enum hfi_picture_type {

#define HFI_PROP_DPB_LIST 0x0300017A

+#define HFI_PROP_TOTAL_PEAK_BITRATE 0x0300017C
+
+#define HFI_PROP_MAINTAIN_MIN_QUALITY 0x0300017D
+
+#define HFI_PROP_IR_CYCLIC_PERIOD 0x0300017E
+
#define HFI_PROP_UBWC_STRIDE_SCANLINE 0x03000190

#define HFI_PROP_COMV_BUFFER_COUNT 0x03000193
@@ -214,7 +316,7 @@ enum hfi_picture_type {

#define HFI_SYSTEM_ERROR_BEGIN 0x05000000

-#define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001
+#define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001

#define HFI_SYSTEM_ERROR_END 0x05FFFFFF

@@ -233,9 +335,9 @@ enum hfi_picture_type {
#define HFI_INFORMATION_END 0x06FFFFFF

struct hfi_debug_header {
- u32 size;
- u32 debug_level;
- u32 reserved[2];
+ u32 size;
+ u32 debug_level;
+ u32 reserved[2];
};

enum hfi_buffer_type {
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
index b1273d0..ca5406a 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
@@ -30,6 +30,10 @@

#define INPUT_TIMER_LIST_SIZE 30

+#define CABAC_MAX_BITRATE 160000000
+
+#define CAVLC_MAX_BITRATE 220000000
+
enum domain_type {
ENCODER = BIT(0),
DECODER = BIT(1),
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
index 94fff74..a648cc1 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
@@ -611,6 +611,45 @@ int prepare_dependency_list(struct iris_inst *inst)
return ret;
}

+static inline bool is_layer_bitrate_set(struct iris_inst *inst)
+{
+ u32 layer_br_caps[6] = {L0_BR, L1_BR, L2_BR, L3_BR, L4_BR, L5_BR};
+ u32 cap_id = 0, i, enh_layer_count;
+
+ enh_layer_count = inst->cap[ENH_LAYER_COUNT].value;
+
+ for (i = 0; i <= enh_layer_count; i++) {
+ if (i >= ARRAY_SIZE(layer_br_caps))
+ break;
+
+ cap_id = layer_br_caps[i];
+ if (!(inst->cap[cap_id].flags & CAP_FLAG_CLIENT_SET))
+ return false;
+ }
+
+ return true;
+}
+
+static inline u32 get_cumulative_bitrate(struct iris_inst *inst)
+{
+ u32 layer_br_caps[6] = {L0_BR, L1_BR, L2_BR, L3_BR, L4_BR, L5_BR};
+ u32 cumulative_br = 0;
+ s32 enh_layer_count;
+ u32 cap_id = 0;
+ int i;
+
+ enh_layer_count = inst->cap[ENH_LAYER_COUNT].value;
+
+ for (i = 0; i <= enh_layer_count; i++) {
+ if (i >= ARRAY_SIZE(layer_br_caps))
+ break;
+ cap_id = layer_br_caps[i];
+ cumulative_br += inst->cap[cap_id].value;
+ }
+
+ return cumulative_br;
+}
+
int set_u32_enum(struct iris_inst *inst,
enum plat_inst_cap_type cap_id)
{
@@ -635,6 +674,18 @@ int set_u32(struct iris_inst *inst,
&hfi_value, sizeof(u32));
}

+int set_q16(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_value = inst->cap[cap_id].value;
+ u32 hfi_id = inst->cap[cap_id].hfi_id;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_Q16,
+ &hfi_value, sizeof(u32));
+}
+
int set_stage(struct iris_inst *inst,
enum plat_inst_cap_type cap_id)
{
@@ -662,7 +713,7 @@ int set_pipe(struct iris_inst *inst,
{
u32 work_route, hfi_id;

- work_route = inst->cap[PIPE].value;
+ work_route = inst->cap[cap_id].value;
hfi_id = inst->cap[cap_id].hfi_id;

return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
@@ -671,6 +722,419 @@ int set_pipe(struct iris_inst *inst,
&work_route, sizeof(u32));
}

+int set_level(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_value = inst->cap[cap_id].value;
+ u32 hfi_id = inst->cap[cap_id].hfi_id;
+
+ if (!(inst->cap[cap_id].flags & CAP_FLAG_CLIENT_SET))
+ hfi_value = HFI_LEVEL_NONE;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32_ENUM,
+ &hfi_value, sizeof(u32));
+}
+
+int set_req_sync_frame(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_id, hfi_val;
+ s32 prepend_sps_pps;
+
+ prepend_sps_pps = inst->cap[PREPEND_SPSPPS_TO_IDR].value;
+ hfi_id = inst->cap[cap_id].hfi_id;
+
+ if (prepend_sps_pps)
+ hfi_val = HFI_SYNC_FRAME_REQUEST_WITH_PREFIX_SEQ_HDR;
+ else
+ hfi_val = HFI_SYNC_FRAME_REQUEST_WITHOUT_SEQ_HDR;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32_ENUM,
+ &hfi_val, sizeof(u32));
+}
+
+int set_flip(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id)
+{
+ u32 hflip, vflip, ret = 0;
+
+ u32 hfi_value = HFI_DISABLE_FLIP;
+ u32 hfi_id = inst->cap[cap_id].hfi_id;
+
+ hflip = inst->cap[HFLIP].value;
+ vflip = inst->cap[VFLIP].value;
+
+ if (hflip)
+ hfi_value |= HFI_HORIZONTAL_FLIP;
+
+ if (vflip)
+ hfi_value |= HFI_VERTICAL_FLIP;
+
+ if (inst->vb2q_dst->streaming) {
+ if (hfi_value != HFI_DISABLE_FLIP) {
+ ret = set_req_sync_frame(inst, REQUEST_I_FRAME);
+ if (ret)
+ return ret;
+ }
+ }
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32_ENUM,
+ &hfi_value, sizeof(u32));
+}
+
+int set_rotation(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_id, hfi_val;
+
+ hfi_id = inst->cap[cap_id].hfi_id;
+
+ switch (inst->cap[cap_id].value) {
+ case 0:
+ hfi_val = HFI_ROTATION_NONE;
+ break;
+ case 90:
+ hfi_val = HFI_ROTATION_90;
+ break;
+ case 180:
+ hfi_val = HFI_ROTATION_180;
+ break;
+ case 270:
+ hfi_val = HFI_ROTATION_270;
+ break;
+ default:
+ hfi_val = HFI_ROTATION_NONE;
+ break;
+ }
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32,
+ &hfi_val, sizeof(u32));
+}
+
+int set_header_mode(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 header_mode, hfi_id, hfi_val;
+ s32 prepend_sps_pps;
+
+ prepend_sps_pps = inst->cap[PREPEND_SPSPPS_TO_IDR].value;
+ header_mode = inst->cap[cap_id].value;
+ hfi_id = inst->cap[cap_id].hfi_id;
+
+ if (prepend_sps_pps)
+ hfi_val = HFI_SEQ_HEADER_PREFIX_WITH_SYNC_FRAME;
+ else if (header_mode == V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME)
+ hfi_val = HFI_SEQ_HEADER_JOINED_WITH_1ST_FRAME;
+ else
+ hfi_val = HFI_SEQ_HEADER_SEPERATE_FRAME;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32_ENUM,
+ &hfi_val, sizeof(u32));
+}
+
+int set_gop_size(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_value, hfi_id;
+
+ if (inst->vb2q_dst->streaming) {
+ if (inst->hfi_layer_type == HFI_HIER_B)
+ return 0;
+ }
+
+ hfi_value = inst->cap[GOP_SIZE].value;
+ hfi_id = inst->cap[cap_id].hfi_id;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32,
+ &hfi_value, sizeof(u32));
+}
+
+int set_bitrate(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_id, hfi_val;
+
+ if (inst->cap[BIT_RATE].flags & CAP_FLAG_CLIENT_SET)
+ goto set_total_bitrate;
+
+ if (inst->vb2q_dst->streaming)
+ return 0;
+
+set_total_bitrate:
+ hfi_id = inst->cap[cap_id].hfi_id;
+ hfi_val = inst->cap[cap_id].value;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32,
+ &hfi_val, sizeof(u32));
+}
+
+int set_layer_bitrate(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_value = 0;
+ u32 hfi_id;
+
+ if (!inst->vb2q_dst->streaming)
+ return 0;
+
+ if (inst->cap[BIT_RATE].flags & CAP_FLAG_CLIENT_SET)
+ return 0;
+
+ if (!inst->cap[ENH_LAYER_COUNT].max ||
+ !is_layer_bitrate_set(inst))
+ return 0;
+
+ hfi_value = inst->cap[BIT_RATE].value;
+ hfi_id = inst->cap[BIT_RATE].hfi_id;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32,
+ &hfi_value, sizeof(u32));
+}
+
+int set_peak_bitrate(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_id, hfi_val;
+ s32 rc_mode;
+
+ hfi_id = inst->cap[cap_id].hfi_id;
+ hfi_val = inst->cap[cap_id].value;
+
+ rc_mode = inst->cap[BITRATE_MODE].value;
+ if (rc_mode != V4L2_MPEG_VIDEO_BITRATE_MODE_CBR)
+ return 0;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32,
+ &hfi_val, sizeof(u32));
+}
+
+int set_use_and_mark_ltr(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_id, hfi_val;
+
+ hfi_id = inst->cap[cap_id].hfi_id;
+ hfi_val = inst->cap[cap_id].value;
+
+ if (!inst->cap[LTR_COUNT].value ||
+ inst->cap[cap_id].value == INVALID_DEFAULT_MARK_OR_USE_LTR)
+ return 0;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32,
+ &hfi_val, sizeof(u32));
+}
+
+int set_ir_period(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_id = 0, hfi_val;
+
+ hfi_val = inst->cap[cap_id].value;
+
+ if (inst->cap[IR_TYPE].value ==
+ V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_RANDOM) {
+ hfi_id = HFI_PROP_IR_RANDOM_PERIOD;
+ } else if (inst->cap[IR_TYPE].value ==
+ V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_CYCLIC) {
+ hfi_id = HFI_PROP_IR_CYCLIC_PERIOD;
+ }
+
+ return iris_hfi_set_ir_period(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32,
+ &hfi_val, sizeof(u32));
+}
+
+int set_min_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ s32 i_frame_qp = 0, p_frame_qp = 0, b_frame_qp = 0, min_qp_enable = 0;
+ u32 i_qp_enable = 0, p_qp_enable = 0, b_qp_enable = 0;
+ u32 client_qp_enable = 0, hfi_value = 0;
+ u32 hfi_id;
+
+ if (inst->cap[MIN_FRAME_QP].flags & CAP_FLAG_CLIENT_SET)
+ min_qp_enable = 1;
+
+ if (min_qp_enable ||
+ (inst->cap[I_FRAME_MIN_QP].flags & CAP_FLAG_CLIENT_SET))
+ i_qp_enable = 1;
+ if (min_qp_enable ||
+ (inst->cap[P_FRAME_MIN_QP].flags & CAP_FLAG_CLIENT_SET))
+ p_qp_enable = 1;
+ if (min_qp_enable ||
+ (inst->cap[B_FRAME_MIN_QP].flags & CAP_FLAG_CLIENT_SET))
+ b_qp_enable = 1;
+
+ client_qp_enable = i_qp_enable | p_qp_enable << 1 | b_qp_enable << 2;
+ if (!client_qp_enable)
+ return 0;
+
+ i_frame_qp = max(inst->cap[I_FRAME_MIN_QP].value, inst->cap[MIN_FRAME_QP].value);
+ p_frame_qp = max(inst->cap[P_FRAME_MIN_QP].value, inst->cap[MIN_FRAME_QP].value);
+ b_frame_qp = max(inst->cap[B_FRAME_MIN_QP].value, inst->cap[MIN_FRAME_QP].value);
+
+ hfi_id = inst->cap[cap_id].hfi_id;
+ hfi_value = i_frame_qp | p_frame_qp << 8 | b_frame_qp << 16 |
+ client_qp_enable << 24;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_32_PACKED,
+ &hfi_value, sizeof(u32));
+}
+
+int set_max_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ s32 i_frame_qp = 0, p_frame_qp = 0, b_frame_qp = 0, max_qp_enable = 0;
+ u32 i_qp_enable = 0, p_qp_enable = 0, b_qp_enable = 0;
+ u32 client_qp_enable = 0, hfi_value = 0;
+ u32 hfi_id;
+
+ if (inst->cap[MAX_FRAME_QP].flags & CAP_FLAG_CLIENT_SET)
+ max_qp_enable = 1;
+
+ if (max_qp_enable ||
+ (inst->cap[I_FRAME_MAX_QP].flags & CAP_FLAG_CLIENT_SET))
+ i_qp_enable = 1;
+ if (max_qp_enable ||
+ (inst->cap[P_FRAME_MAX_QP].flags & CAP_FLAG_CLIENT_SET))
+ p_qp_enable = 1;
+ if (max_qp_enable ||
+ (inst->cap[B_FRAME_MAX_QP].flags & CAP_FLAG_CLIENT_SET))
+ b_qp_enable = 1;
+
+ client_qp_enable = i_qp_enable | p_qp_enable << 1 | b_qp_enable << 2;
+ if (!client_qp_enable)
+ return 0;
+
+ i_frame_qp = min(inst->cap[I_FRAME_MAX_QP].value, inst->cap[MAX_FRAME_QP].value);
+ p_frame_qp = min(inst->cap[P_FRAME_MAX_QP].value, inst->cap[MAX_FRAME_QP].value);
+ b_frame_qp = min(inst->cap[B_FRAME_MAX_QP].value, inst->cap[MAX_FRAME_QP].value);
+
+ hfi_id = inst->cap[cap_id].hfi_id;
+ hfi_value = i_frame_qp | p_frame_qp << 8 | b_frame_qp << 16 |
+ client_qp_enable << 24;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_32_PACKED,
+ &hfi_value, sizeof(u32));
+}
+
+int set_frame_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 i_qp_enable = 0, p_qp_enable = 0, b_qp_enable = 0;
+ u32 client_qp_enable = 0, hfi_value = 0;
+ s32 i_frame_qp = 0, p_frame_qp = 0, b_frame_qp = 0;
+ s32 rc_type = -1;
+ u32 hfi_id;
+
+ rc_type = inst->hfi_rc_type;
+ if (inst->vb2q_dst->streaming) {
+ if (rc_type != HFI_RC_OFF)
+ return 0;
+ }
+
+ if (rc_type == HFI_RC_OFF) {
+ i_qp_enable = 1;
+ p_qp_enable = 1;
+ b_qp_enable = 1;
+ } else {
+ if (inst->cap[I_FRAME_QP].flags & CAP_FLAG_CLIENT_SET)
+ i_qp_enable = 1;
+ if (inst->cap[P_FRAME_QP].flags & CAP_FLAG_CLIENT_SET)
+ p_qp_enable = 1;
+ if (inst->cap[B_FRAME_QP].flags & CAP_FLAG_CLIENT_SET)
+ b_qp_enable = 1;
+ }
+
+ client_qp_enable = i_qp_enable | p_qp_enable << 1 | b_qp_enable << 2;
+ if (!client_qp_enable)
+ return 0;
+
+ i_frame_qp = inst->cap[I_FRAME_QP].value;
+ p_frame_qp = inst->cap[P_FRAME_QP].value;
+ b_frame_qp = inst->cap[B_FRAME_QP].value;
+
+ hfi_id = inst->cap[cap_id].hfi_id;
+ hfi_value = i_frame_qp | p_frame_qp << 8 | b_frame_qp << 16 |
+ client_qp_enable << 24;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_32_PACKED,
+ &hfi_value, sizeof(u32));
+}
+
+int set_layer_count_and_type(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_layer_count, hfi_layer_type = 0;
+ int ret, hfi_id;
+
+ if (!inst->vb2q_dst->streaming) {
+ hfi_layer_type = inst->hfi_layer_type;
+ hfi_id = inst->cap[LAYER_TYPE].hfi_id;
+
+ ret = iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, LAYER_TYPE),
+ HFI_PAYLOAD_U32_ENUM,
+ &hfi_layer_type, sizeof(u32));
+ if (ret)
+ return ret;
+ } else {
+ if (inst->hfi_layer_type == HFI_HIER_B)
+ return 0;
+ }
+
+ hfi_id = inst->cap[ENH_LAYER_COUNT].hfi_id;
+ hfi_layer_count = inst->cap[ENH_LAYER_COUNT].value + 1;
+
+ ret = iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, ENH_LAYER_COUNT),
+ HFI_PAYLOAD_U32,
+ &hfi_layer_count, sizeof(u32));
+
+ return ret;
+}
+
+int set_slice_count(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_value = 0, set_cap_id = 0, hfi_id;
+ s32 slice_mode = -1;
+
+ slice_mode = inst->cap[SLICE_MODE].value;
+
+ if (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE)
+ return 0;
+
+ if (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB) {
+ hfi_value = (inst->codec == HEVC) ?
+ (inst->cap[SLICE_MAX_MB].value + 3) / 4 :
+ inst->cap[SLICE_MAX_MB].value;
+ set_cap_id = SLICE_MAX_MB;
+ } else if (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_BYTES) {
+ hfi_value = inst->cap[SLICE_MAX_BYTES].value;
+ set_cap_id = SLICE_MAX_BYTES;
+ }
+
+ hfi_id = inst->cap[set_cap_id].hfi_id;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, set_cap_id),
+ HFI_PAYLOAD_U32,
+ &hfi_value, sizeof(u32));
+}
+
int set_v4l2_properties(struct iris_inst *inst)
{
struct cap_entry *entry = NULL, *temp = NULL;
@@ -723,8 +1187,7 @@ int adjust_profile(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
u32 adjusted_value;
s32 pix_fmt = -1;

- adjusted_value = ctrl ? ctrl->val :
- inst->cap[PROFILE].value;
+ adjusted_value = ctrl ? ctrl->val : inst->cap[PROFILE].value;

pix_fmt = inst->cap[PIX_FMTS].value;

@@ -737,3 +1200,515 @@ int adjust_profile(struct iris_inst *inst, struct v4l2_ctrl *ctrl)

return 0;
}
+
+int adjust_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ u32 layer_br_caps[6] = {L0_BR, L1_BR, L2_BR, L3_BR, L4_BR, L5_BR};
+ u32 adjusted_value, cumulative_bitrate, cap_id, cap_val, i;
+ s32 layer_count, max_bitrate = 0, entropy_mode;
+
+ adjusted_value = ctrl ? ctrl->val : inst->cap[BIT_RATE].value;
+
+ if (inst->cap[BIT_RATE].flags & CAP_FLAG_CLIENT_SET) {
+ inst->cap[BIT_RATE].value = adjusted_value;
+ return 0;
+ }
+
+ entropy_mode = inst->cap[ENTROPY_MODE].value;
+
+ if (inst->codec == HEVC)
+ max_bitrate = CABAC_MAX_BITRATE;
+
+ if (inst->codec == H264) {
+ if (entropy_mode == V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC)
+ max_bitrate = CABAC_MAX_BITRATE;
+ else
+ max_bitrate = CAVLC_MAX_BITRATE;
+ }
+
+ if (inst->cap[BIT_RATE].value > max_bitrate)
+ inst->cap[BIT_RATE].value = max_bitrate;
+
+ layer_count = inst->cap[ENH_LAYER_COUNT].value;
+ if (!layer_count)
+ return 0;
+
+ if (!is_layer_bitrate_set(inst))
+ return 0;
+
+ cumulative_bitrate = get_cumulative_bitrate(inst);
+
+ if (cumulative_bitrate > max_bitrate) {
+ u32 decrement_in_value = 0;
+ u32 decrement_in_percent = ((cumulative_bitrate - max_bitrate) * 100) /
+ max_bitrate;
+
+ cumulative_bitrate = 0;
+ for (i = 0; i <= layer_count; i++) {
+ if (i >= ARRAY_SIZE(layer_br_caps))
+ break;
+ cap_id = layer_br_caps[i];
+ cap_val = inst->cap[cap_id].value;
+ decrement_in_value = (cap_val * decrement_in_percent) / 100;
+ cumulative_bitrate += (cap_val - decrement_in_value);
+ inst->cap[cap_id].value = cap_val - decrement_in_value;
+ }
+ inst->cap[BIT_RATE].value = cumulative_bitrate;
+ }
+
+ return 0;
+}
+
+int adjust_layer_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ u32 old_br = 0, new_br = 0, exceeded_br = 0;
+ u32 client_set_cap_id = INST_CAP_NONE;
+ u32 cumulative_bitrate = 0;
+ s32 max_bitrate;
+
+ if (!ctrl)
+ return 0;
+
+ if (inst->cap[BIT_RATE].flags & CAP_FLAG_CLIENT_SET ||
+ !inst->vb2q_dst->streaming)
+ return 0;
+
+ if (!inst->cap[ENH_LAYER_COUNT].max)
+ return -EINVAL;
+
+ if (!is_layer_bitrate_set(inst))
+ return 0;
+
+ client_set_cap_id = get_cap_id(inst, ctrl->id);
+ if (!is_valid_cap_id(client_set_cap_id))
+ return -EINVAL;
+
+ cumulative_bitrate = get_cumulative_bitrate(inst);
+ max_bitrate = inst->cap[BIT_RATE].max;
+ old_br = inst->cap[client_set_cap_id].value;
+ new_br = ctrl->val;
+
+ if ((cumulative_bitrate - old_br + new_br) > max_bitrate) {
+ exceeded_br = (cumulative_bitrate - old_br + new_br) - max_bitrate;
+ new_br = ctrl->val - exceeded_br;
+ }
+ inst->cap[client_set_cap_id].value = new_br;
+
+ inst->cap[BIT_RATE].value = get_cumulative_bitrate(inst);
+
+ return 0;
+}
+
+int adjust_peak_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ u32 adjusted_value;
+ s32 rc_mode, bitrate;
+
+ adjusted_value = ctrl ? ctrl->val : inst->cap[PEAK_BITRATE].value;
+
+ rc_mode = inst->cap[BITRATE_MODE].value;
+
+ if (rc_mode != V4L2_MPEG_VIDEO_BITRATE_MODE_CBR)
+ return 0;
+
+ bitrate = inst->cap[BIT_RATE].value;
+
+ if (inst->cap[PEAK_BITRATE].flags & CAP_FLAG_CLIENT_SET) {
+ if (adjusted_value < bitrate)
+ adjusted_value = bitrate;
+ } else {
+ adjusted_value = inst->cap[BIT_RATE].value;
+ }
+
+ inst->cap[PEAK_BITRATE].value = adjusted_value;
+
+ return 0;
+}
+
+int adjust_bitrate_mode(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 frame_rc, bitrate_mode, frame_skip;
+
+ bitrate_mode = inst->cap[BITRATE_MODE].value;
+ frame_rc = inst->cap[FRAME_RC_ENABLE].value;
+ frame_skip = inst->cap[FRAME_SKIP_MODE].value;
+
+ if (!frame_rc) {
+ inst->hfi_rc_type = HFI_RC_OFF;
+ return 0;
+ }
+
+ if (bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_VBR) {
+ inst->hfi_rc_type = HFI_RC_VBR_CFR;
+ } else if (bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CBR) {
+ if (frame_skip)
+ inst->hfi_rc_type = HFI_RC_CBR_VFR;
+ else
+ inst->hfi_rc_type = HFI_RC_CBR_CFR;
+ } else if (bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CQ) {
+ inst->hfi_rc_type = HFI_RC_CQ;
+ }
+
+ return 0;
+}
+
+int adjust_gop_size(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 adjusted_value, enh_layer_count;
+ u32 min_gop_size, num_subgops;
+
+ adjusted_value = ctrl ? ctrl->val : inst->cap[GOP_SIZE].value;
+
+ enh_layer_count = inst->cap[ENH_LAYER_COUNT].value;
+
+ if (!enh_layer_count)
+ goto exit;
+
+ /*
+ * Layer encoding needs GOP size to be multiple of subgop size
+ * And subgop size is 2 ^ number of enhancement layers.
+ */
+ min_gop_size = 1 << enh_layer_count;
+ num_subgops = (adjusted_value + (min_gop_size >> 1)) / min_gop_size;
+ if (num_subgops)
+ adjusted_value = num_subgops * min_gop_size;
+ else
+ adjusted_value = min_gop_size;
+
+exit:
+ inst->cap[GOP_SIZE].value = adjusted_value;
+
+ return 0;
+}
+
+int adjust_b_frame(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 adjusted_value, enh_layer_count = -1;
+ const u32 max_bframe_size = 7;
+
+ adjusted_value = ctrl ? ctrl->val : inst->cap[B_FRAME].value;
+
+ enh_layer_count = inst->cap[ENH_LAYER_COUNT].value;
+
+ if (!enh_layer_count || inst->hfi_layer_type != HFI_HIER_B) {
+ adjusted_value = 0;
+ goto exit;
+ }
+
+ adjusted_value = (1 << enh_layer_count) - 1;
+
+ if (adjusted_value > max_bframe_size)
+ adjusted_value = max_bframe_size;
+
+exit:
+ inst->cap[B_FRAME].value = adjusted_value;
+
+ return 0;
+}
+
+int adjust_ltr_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 adjusted_value, rc_mode, pix_fmt;
+
+ adjusted_value = ctrl ? ctrl->val : inst->cap[LTR_COUNT].value;
+
+ rc_mode = inst->cap[BITRATE_MODE].value;
+
+ if (rc_mode != V4L2_MPEG_VIDEO_BITRATE_MODE_CBR ||
+ inst->hfi_rc_type != HFI_RC_OFF)
+ adjusted_value = 0;
+
+ pix_fmt = inst->cap[PIX_FMTS].value;
+ if (is_10bit_colorformat(pix_fmt))
+ adjusted_value = 0;
+
+ inst->cap[LTR_COUNT].value = adjusted_value;
+
+ return 0;
+}
+
+int adjust_use_ltr(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 adjusted_value, ltr_count;
+
+ adjusted_value = ctrl ? ctrl->val : inst->cap[USE_LTR].value;
+
+ ltr_count = inst->cap[LTR_COUNT].value;
+ if (!ltr_count)
+ return 0;
+
+ /*
+ * USE_LTR is bitmask value, hence should be
+ * > 0 and <= (2 ^ LTR_COUNT) - 1
+ */
+ if (adjusted_value <= 0 ||
+ adjusted_value > (1 << ltr_count) - 1)
+ return 0;
+
+ inst->cap[USE_LTR].value = adjusted_value;
+
+ return 0;
+}
+
+int adjust_mark_ltr(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 adjusted_value, ltr_count;
+
+ adjusted_value = ctrl ? ctrl->val : inst->cap[MARK_LTR].value;
+
+ ltr_count = inst->cap[LTR_COUNT].value;
+ if (!ltr_count)
+ return 0;
+
+ if (adjusted_value < 0 || adjusted_value > ltr_count - 1)
+ return 0;
+
+ inst->cap[MARK_LTR].value = adjusted_value;
+
+ return 0;
+}
+
+int adjust_ir_period(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 adjusted_value, pix_fmt, rc_mode;
+
+ adjusted_value = ctrl ? ctrl->val : inst->cap[IR_PERIOD].value;
+
+ pix_fmt = inst->cap[PIX_FMTS].value;
+ if (is_10bit_colorformat(pix_fmt))
+ adjusted_value = 0;
+
+ rc_mode = inst->cap[BITRATE_MODE].value;
+ if (rc_mode != V4L2_MPEG_VIDEO_BITRATE_MODE_CBR)
+ adjusted_value = 0;
+
+ inst->cap[IR_PERIOD].value = adjusted_value;
+
+ return 0;
+}
+
+int adjust_min_quality(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 adjusted_value, pix_fmt, rc_mode, layer_count;
+ u32 width, height, frame_rate;
+ struct v4l2_format *f;
+
+ if (inst->vb2q_dst->streaming)
+ return 0;
+
+ adjusted_value = MAX_SUPPORTED_MIN_QUALITY;
+
+ rc_mode = inst->cap[BITRATE_MODE].value;
+ if (rc_mode != V4L2_MPEG_VIDEO_BITRATE_MODE_VBR)
+ adjusted_value = 0;
+
+ layer_count = inst->cap[ENH_LAYER_COUNT].value;
+ if (layer_count && inst->hfi_layer_type != HFI_HIER_B)
+ adjusted_value = 0;
+
+ pix_fmt = inst->cap[PIX_FMTS].value;
+ if (is_10bit_colorformat(pix_fmt))
+ adjusted_value = 0;
+
+ frame_rate = inst->cap[FRAME_RATE].value >> 16;
+ f = inst->fmt_dst;
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+
+ if (!res_is_less_than(width, height, 1920, 1080))
+ adjusted_value = 0;
+
+ if (frame_rate > 60)
+ adjusted_value = 0;
+
+ inst->cap[MIN_QUALITY].value = adjusted_value;
+
+ return 0;
+}
+
+static int adjust_static_layer_count_and_type(struct iris_inst *inst, s32 layer_count)
+{
+ bool hb_requested = false;
+ s32 max_enh_count = 0;
+
+ if (!layer_count)
+ goto exit;
+
+ if (inst->hfi_rc_type == HFI_RC_CQ) {
+ layer_count = 0;
+ goto exit;
+ }
+
+ if (inst->codec == H264) {
+ if (!inst->cap[LAYER_ENABLE].value) {
+ layer_count = 0;
+ goto exit;
+ }
+ hb_requested = inst->cap[LAYER_TYPE].value ==
+ V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_B;
+ } else if (inst->codec == HEVC) {
+ hb_requested = inst->cap[LAYER_TYPE].value ==
+ V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B;
+ }
+
+ if (hb_requested && inst->hfi_rc_type != HFI_RC_VBR_CFR) {
+ layer_count = 0;
+ goto exit;
+ }
+
+ inst->hfi_layer_type = hb_requested ? HFI_HIER_B :
+ (inst->codec == H264 && inst->hfi_rc_type == HFI_RC_VBR_CFR) ?
+ HFI_HIER_P_HYBRID_LTR : HFI_HIER_P_SLIDING_WINDOW;
+
+ max_enh_count = inst->hfi_layer_type == HFI_HIER_B ? MAX_ENH_LAYER_HB :
+ inst->hfi_layer_type == HFI_HIER_P_HYBRID_LTR ? MAX_AVC_ENH_LAYER_HYBRID_HP :
+ inst->hfi_layer_type == HFI_HIER_P_SLIDING_WINDOW ?
+ (inst->codec == H264 ? MAX_AVC_ENH_LAYER_SLIDING_WINDOW :
+ (inst->codec == HEVC && inst->hfi_rc_type == HFI_RC_VBR_CFR) ?
+ MAX_HEVC_VBR_ENH_LAYER_SLIDING_WINDOW :
+ MAX_HEVC_NON_VBR_ENH_LAYER_SLIDING_WINDOW) :
+ layer_count;
+
+ layer_count = min(layer_count, max_enh_count);
+
+exit:
+ inst->cap[ENH_LAYER_COUNT].value = layer_count;
+ inst->cap[ENH_LAYER_COUNT].max = layer_count;
+
+ return 0;
+}
+
+int adjust_layer_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 client_layer_count;
+ int ret = 0;
+
+ client_layer_count = ctrl ? ctrl->val : inst->cap[ENH_LAYER_COUNT].value;
+
+ if (!inst->vb2q_dst->streaming) {
+ ret = adjust_static_layer_count_and_type(inst, client_layer_count);
+ if (ret)
+ return ret;
+ } else {
+ if (inst->hfi_rc_type == HFI_RC_CBR_CFR ||
+ inst->hfi_rc_type == HFI_RC_CBR_VFR)
+ return ret;
+
+ if (inst->hfi_layer_type == HFI_HIER_P_HYBRID_LTR ||
+ inst->hfi_layer_type == HFI_HIER_P_SLIDING_WINDOW)
+ inst->cap[ENH_LAYER_COUNT].value =
+ min(client_layer_count, inst->cap[ENH_LAYER_COUNT].max);
+ }
+
+ return ret;
+}
+
+int adjust_entropy_mode(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 adjusted_value;
+ s32 profile = -1;
+
+ adjusted_value = ctrl ? ctrl->val : inst->cap[ENTROPY_MODE].value;
+
+ profile = inst->cap[PROFILE].value;
+ if (profile == V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE ||
+ profile == V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE)
+ adjusted_value = V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CAVLC;
+
+ inst->cap[ENTROPY_MODE].value = adjusted_value;
+
+ return 0;
+}
+
+int adjust_slice_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 adjusted_value, rc_type = -1, slice_mode, enh_layer_count = 0;
+ u32 slice_val, mbpf = 0, mbps = 0, max_mbpf = 0, max_mbps = 0, bitrate = 0;
+ u32 update_cap, max_avg_slicesize, output_width, output_height;
+ u32 min_width, min_height, max_width, max_height, fps;
+
+ slice_mode = ctrl ? ctrl->val : inst->cap[SLICE_MODE].value;
+ if (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE)
+ return 0;
+
+ bitrate = inst->cap[BIT_RATE].value;
+ enh_layer_count = inst->cap[ENH_LAYER_COUNT].value;
+ if (enh_layer_count && is_layer_bitrate_set(inst))
+ bitrate = get_cumulative_bitrate(inst);
+
+ rc_type = inst->hfi_rc_type;
+ fps = inst->cap[FRAME_RATE].value >> 16;
+ if (fps > MAX_SLICES_FRAME_RATE ||
+ (rc_type != HFI_RC_OFF && rc_type != HFI_RC_CBR_CFR &&
+ rc_type != HFI_RC_CBR_VFR && rc_type != HFI_RC_VBR_CFR)) {
+ adjusted_value = V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE;
+ update_cap = SLICE_MODE;
+ goto exit;
+ }
+
+ output_width = inst->fmt_dst->fmt.pix_mp.width;
+ output_height = inst->fmt_dst->fmt.pix_mp.height;
+
+ max_width = (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB) ?
+ MAX_MB_SLICE_WIDTH : MAX_BYTES_SLICE_WIDTH;
+ max_height = (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB) ?
+ MAX_MB_SLICE_HEIGHT : MAX_BYTES_SLICE_HEIGHT;
+ min_width = (inst->codec == HEVC) ?
+ MIN_HEVC_SLICE_WIDTH : MIN_AVC_SLICE_WIDTH;
+ min_height = MIN_SLICE_HEIGHT;
+
+ if (output_width < min_width || output_height < min_height ||
+ output_width > max_width || output_height > max_width) {
+ adjusted_value = V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE;
+ update_cap = SLICE_MODE;
+ goto exit;
+ }
+
+ mbpf = NUM_MBS_PER_FRAME(output_height, output_width);
+ mbps = mbpf * fps;
+ max_mbpf = NUM_MBS_PER_FRAME(max_height, max_width);
+ max_mbps = max_mbpf * MAX_SLICES_FRAME_RATE;
+
+ if (mbpf > max_mbpf || mbps > max_mbps) {
+ adjusted_value = V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE;
+ update_cap = SLICE_MODE;
+ goto exit;
+ }
+
+ if (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB) {
+ update_cap = SLICE_MAX_MB;
+ slice_val = inst->cap[SLICE_MAX_MB].value;
+ slice_val = max(slice_val, mbpf / MAX_SLICES_PER_FRAME);
+ } else {
+ slice_val = inst->cap[SLICE_MAX_BYTES].value;
+ update_cap = SLICE_MAX_BYTES;
+ if (rc_type != HFI_RC_OFF) {
+ max_avg_slicesize =
+ ((bitrate / fps) / 8) / MAX_SLICES_PER_FRAME;
+ slice_val = max(slice_val, max_avg_slicesize);
+ }
+ }
+ adjusted_value = slice_val;
+
+exit:
+ inst->cap[update_cap].value = adjusted_value;
+
+ return 0;
+}
+
+int adjust_transform_8x8(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 adjusted_value;
+ s32 profile = -1;
+
+ adjusted_value = ctrl ? ctrl->val : inst->cap[TRANSFORM_8X8].value;
+
+ profile = inst->cap[PROFILE].value;
+ if (profile != V4L2_MPEG_VIDEO_H264_PROFILE_HIGH &&
+ profile != V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH)
+ adjusted_value = 0;
+
+ inst->cap[TRANSFORM_8X8].value = adjusted_value;
+
+ return 0;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
index 28ce767..5421d9f 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
@@ -31,5 +31,40 @@ int get_inst_capability(struct iris_inst *inst);
int set_v4l2_properties(struct iris_inst *inst);
int adjust_v4l2_properties(struct iris_inst *inst);
int ctrls_init(struct iris_inst *inst, bool init);
+int set_q16(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_level(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_req_sync_frame(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_flip(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_rotation(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_header_mode(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_gop_size(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_bitrate(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_layer_bitrate(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_peak_bitrate(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id);
+int set_use_and_mark_ltr(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id);
+int set_ir_period(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id);
+int set_min_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_max_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_frame_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_layer_count_and_type(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_slice_count(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int adjust_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_layer_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_peak_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_bitrate_mode(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_gop_size(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_b_frame(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_ltr_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_use_ltr(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_mark_ltr(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_ir_period(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_min_quality(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_layer_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_entropy_mode(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_slice_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_transform_8x8(struct iris_inst *inst, struct v4l2_ctrl *ctrl);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index 395a189..d9d22e2 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -40,12 +40,20 @@ bool res_is_less_than(u32 width, u32 height,
u32 get_port_info(struct iris_inst *inst,
enum plat_inst_cap_type cap_id)
{
+ if (inst->cap[cap_id].flags & CAP_FLAG_INPUT_PORT &&
+ inst->cap[cap_id].flags & CAP_FLAG_OUTPUT_PORT) {
+ if (inst->vb2q_dst->streaming)
+ return get_hfi_port(INPUT_MPLANE);
+ else
+ return get_hfi_port(OUTPUT_MPLANE);
+ }
+
if (inst->cap[cap_id].flags & CAP_FLAG_INPUT_PORT)
- return HFI_PORT_BITSTREAM;
+ return get_hfi_port(INPUT_MPLANE);
else if (inst->cap[cap_id].flags & CAP_FLAG_OUTPUT_PORT)
- return HFI_PORT_RAW;
-
- return HFI_PORT_NONE;
+ return get_hfi_port(OUTPUT_MPLANE);
+ else
+ return HFI_PORT_NONE;
}

enum iris_buffer_type v4l2_type_to_driver(u32 type)
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index 2850fd5..00e598d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -668,6 +668,59 @@ int iris_hfi_set_property(struct iris_inst *inst,
return ret;
}

+int iris_hfi_set_ir_period(struct iris_inst *inst,
+ u32 packet_type, u32 flag, u32 plane, u32 payload_type,
+ void *payload, u32 payload_size)
+{
+ u32 sync_frame_req = 0;
+ struct iris_core *core;
+ int ret;
+
+ core = inst->core;
+
+ mutex_lock(&core->lock);
+
+ ret = hfi_create_header(inst->packet, inst->packet_size,
+ inst->session_id, core->header_id++);
+ if (ret)
+ goto exit;
+
+ if (!inst->ir_enabled) {
+ inst->ir_enabled = ((*(u32 *)payload > 0) ? true : false);
+ if (inst->ir_enabled && inst->vb2q_dst->streaming) {
+ sync_frame_req = HFI_SYNC_FRAME_REQUEST_WITH_PREFIX_SEQ_HDR;
+ ret = hfi_create_packet(inst->packet, inst->packet_size,
+ HFI_PROP_REQUEST_SYNC_FRAME,
+ HFI_HOST_FLAGS_NONE,
+ HFI_PAYLOAD_U32_ENUM,
+ HFI_PORT_BITSTREAM,
+ core->packet_id++,
+ &sync_frame_req,
+ sizeof(u32));
+ if (ret)
+ goto exit;
+ }
+ }
+
+ ret = hfi_create_packet(inst->packet, inst->packet_size,
+ packet_type,
+ HFI_HOST_FLAGS_NONE,
+ HFI_PAYLOAD_U32,
+ plane,
+ core->packet_id++,
+ payload,
+ sizeof(u32));
+ if (ret)
+ goto exit;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+exit:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
int iris_hfi_queue_buffer(struct iris_inst *inst,
struct iris_buffer *buffer)
{
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
index 465bfc5..95e0523 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
@@ -40,9 +40,8 @@ int prepare_pc(struct iris_core *core);

irqreturn_t iris_hfi_isr(int irq, void *data);
irqreturn_t iris_hfi_isr_handler(int irq, void *data);
-int iris_hfi_queue_buffer(struct iris_inst *inst,
- struct iris_buffer *buffer);
-int iris_hfi_release_buffer(struct iris_inst *inst,
- struct iris_buffer *buffer);
+int iris_hfi_set_ir_period(struct iris_inst *inst,
+ u32 packet_type, u32 flag, u32 plane, u32 payload_type,
+ void *payload, u32 payload_size);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
index fab206cf..dd27fa4 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
@@ -364,8 +364,8 @@ int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf)
return 0;
}

-static int hfi_create_header(u8 *packet, u32 packet_size, u32 session_id,
- u32 header_id)
+int hfi_create_header(u8 *packet, u32 packet_size, u32 session_id,
+ u32 header_id)
{
struct hfi_header *hdr = (struct hfi_header *)packet;

@@ -382,9 +382,9 @@ static int hfi_create_header(u8 *packet, u32 packet_size, u32 session_id,
return 0;
}

-static int hfi_create_packet(u8 *packet, u32 packet_size, u32 pkt_type,
- u32 pkt_flags, u32 payload_type, u32 port,
- u32 packet_id, void *payload, u32 payload_size)
+int hfi_create_packet(u8 *packet, u32 packet_size, u32 pkt_type,
+ u32 pkt_flags, u32 payload_type, u32 port,
+ u32 packet_id, void *payload, u32 payload_size)
{
struct hfi_header *hdr;
struct hfi_packet *pkt;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
index 849b585..82148b7 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
@@ -108,5 +108,10 @@ int hfi_packet_sys_interframe_powercollapse(struct iris_core *core,
u8 *pkt, u32 pkt_size);
int hfi_packet_sys_pc_prep(struct iris_core *core,
u8 *pkt, u32 pkt_size);
+int hfi_create_header(u8 *packet, u32 packet_size, u32 session_id,
+ u32 header_id);
+int hfi_create_packet(u8 *packet, u32 packet_size, u32 pkt_type,
+ u32 pkt_flags, u32 payload_type, u32 port,
+ u32 packet_id, void *payload, u32 payload_size);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
index a4d5d9c..1b667a5 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
@@ -758,6 +758,12 @@ static int handle_session_property(struct iris_inst *inst,
case HFI_PROP_PICTURE_TYPE:
inst->hfi_frame_info.picture_type = payload_ptr[0];
break;
+ case HFI_PROP_CABAC_SESSION:
+ if (payload_ptr[0] == 1)
+ inst->cap[ENTROPY_MODE].value = V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC;
+ else
+ inst->cap[ENTROPY_MODE].value = V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CAVLC;
+ break;
case HFI_PROP_DPB_LIST:
ret = handle_dpb_list_property(inst, pkt);
if (ret)
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
index 3c85e78..70f4c7d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -52,6 +52,9 @@
* @power: structure of power info
* @bus_data: structure of bus data
* @input_timer_list: list head of input timer
+ * @ir_enabled: boolean for intra refresh
+ * @hfi_rc_type: rate control type
+ * @hfi_layer_type: type of HFI layer encoding
*/

struct iris_inst {
@@ -90,6 +93,9 @@ struct iris_inst {
struct iris_inst_power power;
struct bus_vote_data bus_data;
struct list_head input_timer_list;
+ bool ir_enabled;
+ u32 hfi_rc_type;
+ u32 hfi_layer_type;
};

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
index 81de610..443894c 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
@@ -20,9 +20,29 @@ struct iris_inst;
#define BIT_DEPTH_8 (8 << 16 | 8)
#define BIT_DEPTH_10 (10 << 16 | 10)

-#define CODED_FRAMES_PROGRESSIVE 0x0
-#define CODED_FRAMES_INTERLACE 0x1
-#define MAX_NUM_CHILD 10
+#define CODED_FRAMES_PROGRESSIVE 0x0
+#define CODED_FRAMES_INTERLACE 0x1
+#define MAX_NUM_CHILD 10
+#define MAX_ENH_LAYER_HB 3
+#define MAX_HEVC_VBR_ENH_LAYER_SLIDING_WINDOW 5
+#define MAX_HEVC_NON_VBR_ENH_LAYER_SLIDING_WINDOW 3
+#define MAX_AVC_ENH_LAYER_SLIDING_WINDOW 3
+#define MAX_AVC_ENH_LAYER_HYBRID_HP 5
+#define MAX_SLICES_FRAME_RATE 60
+#define MAX_MB_SLICE_WIDTH 4096
+#define MAX_MB_SLICE_HEIGHT 2160
+#define MAX_BYTES_SLICE_WIDTH 1920
+#define MAX_BYTES_SLICE_HEIGHT 1088
+#define MIN_HEVC_SLICE_WIDTH 384
+#define MIN_AVC_SLICE_WIDTH 192
+#define MIN_SLICE_HEIGHT 128
+#define MAX_SLICES_PER_FRAME 10
+#define MIN_QP_8BIT 1
+#define MIN_SLICE_BYTE_SIZE 512
+#define MAX_SLICE_BYTE_SIZE ((MAX_BITRATE) >> 3)
+#define MAX_SLICE_MB_SIZE (((4096 + 15) >> 4) * ((2304 + 15) >> 4))
+#define INVALID_DEFAULT_MARK_OR_USE_LTR -1
+#define MAX_SUPPORTED_MIN_QUALITY 70

#define DEFAULT_MAX_HOST_BUF_COUNT 64
#define DEFAULT_MAX_HOST_BURST_BUF_COUNT 256
@@ -128,12 +148,17 @@ enum plat_inst_cap_type {
PIX_FMTS,
MBPF,
QUEUED_RATE,
+ FRAME_RATE,
+ OPERATING_RATE,
MB_CYCLES_VSP,
MB_CYCLES_VPP,
MB_CYCLES_LP,
MB_CYCLES_FW,
MB_CYCLES_FW_VPP,
NUM_COMV,
+ ENTROPY_MODE,
+ BASELAYER_PRIORITY,
+ IR_TYPE,
PROFILE,
LEVEL,
HEVC_TIER,
@@ -148,6 +173,48 @@ enum plat_inst_cap_type {
BIT_DEPTH,
DEFAULT_HEADER,
RAP_FRAME,
+ MIN_FRAME_QP,
+ MAX_FRAME_QP,
+ B_FRAME,
+ I_FRAME_QP,
+ P_FRAME_QP,
+ B_FRAME_QP,
+ BIT_RATE,
+ PEAK_BITRATE,
+ BITRATE_MODE,
+ FRAME_SKIP_MODE,
+ FRAME_RC_ENABLE,
+ GOP_SIZE,
+ MIN_QUALITY,
+ IR_PERIOD,
+ LTR_COUNT,
+ USE_LTR,
+ MARK_LTR,
+ I_FRAME_MIN_QP,
+ P_FRAME_MIN_QP,
+ B_FRAME_MIN_QP,
+ I_FRAME_MAX_QP,
+ P_FRAME_MAX_QP,
+ B_FRAME_MAX_QP,
+ LAYER_TYPE,
+ LAYER_ENABLE,
+ L0_BR,
+ L1_BR,
+ L2_BR,
+ L3_BR,
+ L4_BR,
+ L5_BR,
+ ENH_LAYER_COUNT,
+ TRANSFORM_8X8,
+ SLICE_MODE,
+ SLICE_MAX_MB,
+ SLICE_MAX_BYTES,
+ HFLIP,
+ VFLIP,
+ ROTATION,
+ HEADER_MODE,
+ PREPEND_SPSPPS_TO_IDR,
+ REQUEST_I_FRAME,
INST_CAP_MAX,
};

@@ -164,6 +231,7 @@ enum plat_inst_cap_flags {

struct plat_inst_cap {
enum plat_inst_cap_type cap_id;
+ enum domain_type domain;
enum codec_type codec;
s32 min;
s32 max;
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
index 7ae9715..6d5192a 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
@@ -13,11 +13,19 @@
#include "platform_common.h"
#include "resources.h"

-#define CODECS_ALL (H264 | HEVC | VP9)
-
-#define DEFAULT_FPS 30
-#define MINIMUM_FPS 1
-#define MAXIMUM_FPS 480
+#define CODECS_ALL (H264 | HEVC | VP9)
+#define ENC ENCODER
+#define DEC DECODER
+
+#define DEFAULT_FPS 30
+#define MINIMUM_FPS 1
+#define MAXIMUM_FPS 480
+#define MAX_BITRATE 245000000
+#define DEFAULT_BITRATE 20000000
+#define MAX_LTR_FRAME_COUNT 2
+#define MAX_BASE_LAYER_PRIORITY_ID 63
+#define MAX_QP 51
+#define DEFAULT_QP 20

static struct codec_info codec_data_sm8550[] = {
{
@@ -68,21 +76,37 @@ static struct plat_core_cap core_data_sm8550[] = {
};

static struct plat_inst_cap instance_cap_data_sm8550[] = {
- {FRAME_WIDTH, CODECS_ALL, 96, 8192, 1, 1920},
+ {FRAME_WIDTH, DEC, CODECS_ALL, 96, 8192, 1, 1920},
+
+ {FRAME_WIDTH, DEC, VP9, 96, 4096, 1, 1920},
+
+ {FRAME_WIDTH, ENC, CODECS_ALL, 128, 8192, 1, 1920},

- {FRAME_WIDTH, VP9, 96, 4096, 1, 1920},
+ {FRAME_WIDTH, ENC, HEVC, 96, 8192, 1, 1920},

- {FRAME_HEIGHT, CODECS_ALL, 96, 8192, 1, 1080},
+ {FRAME_HEIGHT, DEC, CODECS_ALL, 96, 8192, 1, 1080},

- {FRAME_HEIGHT, VP9, 96, 4096, 1, 1080},
+ {FRAME_HEIGHT, DEC, VP9, 96, 4096, 1, 1080},

- {PIX_FMTS, H264,
+ {FRAME_HEIGHT, ENC, CODECS_ALL, 128, 8192, 1, 1080},
+
+ {FRAME_HEIGHT, ENC, HEVC, 96, 8192, 1, 1080},
+
+ {PIX_FMTS, DEC, H264,
FMT_NV12,
FMT_NV12C,
FMT_NV12 | FMT_NV21 | FMT_NV12C,
FMT_NV12C},

- {PIX_FMTS, HEVC,
+ {PIX_FMTS, ENC, H264,
+ FMT_NV12,
+ FMT_NV12C,
+ FMT_NV12 | FMT_NV21 | FMT_NV12C,
+ FMT_NV12C,
+ 0, 0,
+ CAP_FLAG_NONE},
+
+ {PIX_FMTS, DEC, HEVC,
FMT_NV12,
FMT_TP10C,
FMT_NV12 | FMT_NV21 | FMT_NV12C | FMT_TP10C,
@@ -91,37 +115,73 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
CAP_FLAG_NONE,
{PROFILE}},

- {PIX_FMTS, VP9,
+ {PIX_FMTS, ENC, HEVC,
+ FMT_NV12,
+ FMT_TP10C,
+ FMT_NV12 | FMT_NV21 | FMT_NV12C | FMT_TP10C,
+ FMT_NV12C,
+ 0, 0,
+ CAP_FLAG_NONE,
+ {PROFILE, MIN_QUALITY, IR_PERIOD, LTR_COUNT}},
+
+ {PIX_FMTS, DEC, VP9,
FMT_NV12,
FMT_TP10C,
FMT_NV12 | FMT_NV21 | FMT_NV12C | FMT_TP10C,
FMT_NV12C},

- {MBPF, CODECS_ALL, 36, 138240, 1, 138240},
+ {MBPF, DEC, CODECS_ALL, 36, 138240, 1, 138240},

/* (4096 * 2304) / 256 */
- {MBPF, VP9, 36, 36864, 1, 36864},
+ {MBPF, DEC, VP9, 36, 36864, 1, 36864},
+
+ /* (8192 * 4320) / 256 */
+ {MBPF, ENC, CODECS_ALL, 64, 138240, 1, 138240},
+
+ {MBPF, ENC, HEVC, 36, 138240, 1, 138240},

- {QUEUED_RATE, CODECS_ALL,
+ {QUEUED_RATE, DEC | ENC, CODECS_ALL,
(MINIMUM_FPS << 16), INT_MAX,
1, (DEFAULT_FPS << 16)},

- {MB_CYCLES_VSP, CODECS_ALL, 25, 25, 1, 25},
+ {FRAME_RATE, ENC, CODECS_ALL,
+ (MINIMUM_FPS << 16), (MAXIMUM_FPS << 16),
+ 1, (DEFAULT_FPS << 16),
+ 0,
+ HFI_PROP_FRAME_RATE,
+ CAP_FLAG_OUTPUT_PORT,
+ {0},
+ NULL,
+ set_q16},
+
+ {OPERATING_RATE, ENC, CODECS_ALL,
+ (MINIMUM_FPS << 16), (MAXIMUM_FPS << 16),
+ 1, (DEFAULT_FPS << 16)},
+
+ {MB_CYCLES_VSP, DEC, CODECS_ALL, 25, 25, 1, 25},
+
+ {MB_CYCLES_VSP, DEC, VP9, 60, 60, 1, 60},
+
+ {MB_CYCLES_VSP, ENC, CODECS_ALL, 25, 25, 1, 25},

- {MB_CYCLES_VSP, VP9, 60, 60, 1, 60},
+ {MB_CYCLES_VPP, DEC, CODECS_ALL, 200, 200, 1, 200},

- {MB_CYCLES_VPP, CODECS_ALL, 200, 200, 1, 200},
+ {MB_CYCLES_VPP, ENC, CODECS_ALL, 675, 675, 1, 675},

- {MB_CYCLES_LP, CODECS_ALL, 200, 200, 1, 200},
+ {MB_CYCLES_LP, DEC, CODECS_ALL, 200, 200, 1, 200},

- {MB_CYCLES_FW, CODECS_ALL, 489583, 489583, 1, 489583},
+ {MB_CYCLES_LP, ENC, CODECS_ALL, 320, 320, 1, 320},

- {MB_CYCLES_FW_VPP, CODECS_ALL, 66234, 66234, 1, 66234},
+ {MB_CYCLES_FW, DEC | ENC, CODECS_ALL, 489583, 489583, 1, 489583},

- {NUM_COMV, CODECS_ALL,
+ {MB_CYCLES_FW_VPP, DEC, CODECS_ALL, 66234, 66234, 1, 66234},
+
+ {MB_CYCLES_FW_VPP, ENC, CODECS_ALL, 48405, 48405, 1, 48405},
+
+ {NUM_COMV, DEC, CODECS_ALL,
0, INT_MAX, 1, 0},

- {PROFILE, H264,
+ {PROFILE, DEC, H264,
V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE,
V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH,
BIT(V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) |
@@ -137,7 +197,23 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
set_u32_enum},

- {PROFILE, HEVC,
+ {PROFILE, ENC, H264,
+ V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE,
+ V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH,
+ BIT(V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) |
+ BIT(V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH) |
+ BIT(V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE) |
+ BIT(V4L2_MPEG_VIDEO_H264_PROFILE_MAIN) |
+ BIT(V4L2_MPEG_VIDEO_H264_PROFILE_HIGH),
+ V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
+ V4L2_CID_MPEG_VIDEO_H264_PROFILE,
+ HFI_PROP_PROFILE,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {ENTROPY_MODE, TRANSFORM_8X8},
+ NULL,
+ set_u32_enum},
+
+ {PROFILE, ENC | DEC, HEVC,
V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10,
BIT(V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN) |
@@ -151,7 +227,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
adjust_profile,
set_u32_enum},

- {PROFILE, VP9,
+ {PROFILE, DEC, VP9,
V4L2_MPEG_VIDEO_VP9_PROFILE_0,
V4L2_MPEG_VIDEO_VP9_PROFILE_2,
BIT(V4L2_MPEG_VIDEO_VP9_PROFILE_0) |
@@ -164,7 +240,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
set_u32_enum},

- {LEVEL, H264,
+ {LEVEL, DEC, H264,
V4L2_MPEG_VIDEO_H264_LEVEL_1_0,
V4L2_MPEG_VIDEO_H264_LEVEL_6_2,
BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_0) |
@@ -195,7 +271,36 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
set_u32_enum},

- {LEVEL, HEVC,
+ {LEVEL, ENC, H264,
+ V4L2_MPEG_VIDEO_H264_LEVEL_1_0,
+ V4L2_MPEG_VIDEO_H264_LEVEL_6_0,
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_0) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1B) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_1) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_2) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_3) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_0) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_1) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_2) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_0) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_1) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_2) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_0) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_1) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_2) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_0) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_1) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_2) |
+ BIT(V4L2_MPEG_VIDEO_H264_LEVEL_6_0),
+ V4L2_MPEG_VIDEO_H264_LEVEL_5_0,
+ V4L2_CID_MPEG_VIDEO_H264_LEVEL,
+ HFI_PROP_LEVEL,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {0},
+ NULL,
+ set_level},
+
+ {LEVEL, DEC, HEVC,
V4L2_MPEG_VIDEO_HEVC_LEVEL_1,
V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2,
BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_1) |
@@ -219,7 +324,31 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
set_u32_enum},

- {LEVEL, VP9,
+ {LEVEL, ENC, HEVC,
+ V4L2_MPEG_VIDEO_HEVC_LEVEL_1,
+ V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2,
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_1) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_2) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_2_1) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_3) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_3_1) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_4) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_4_1) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_2) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6_1) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2),
+ V4L2_MPEG_VIDEO_HEVC_LEVEL_5,
+ V4L2_CID_MPEG_VIDEO_HEVC_LEVEL,
+ HFI_PROP_LEVEL,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {0},
+ NULL,
+ set_level},
+
+ {LEVEL, DEC, VP9,
V4L2_MPEG_VIDEO_VP9_LEVEL_1_0,
V4L2_MPEG_VIDEO_VP9_LEVEL_6_0,
BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_1_0) |
@@ -242,7 +371,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
set_u32_enum},

- {HEVC_TIER, HEVC,
+ {HEVC_TIER, DEC | ENC, HEVC,
V4L2_MPEG_VIDEO_HEVC_TIER_MAIN,
V4L2_MPEG_VIDEO_HEVC_TIER_HIGH,
BIT(V4L2_MPEG_VIDEO_HEVC_TIER_MAIN) |
@@ -255,7 +384,566 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
set_u32_enum},

- {DISPLAY_DELAY_ENABLE, CODECS_ALL,
+ {HFLIP, ENC, CODECS_ALL,
+ 0, 1, 1, 0,
+ V4L2_CID_HFLIP,
+ HFI_PROP_FLIP,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ NULL,
+ set_flip},
+
+ {VFLIP, ENC, CODECS_ALL,
+ 0, 1, 1, 0,
+ V4L2_CID_VFLIP,
+ HFI_PROP_FLIP,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ NULL,
+ set_flip},
+
+ {ROTATION, ENC, CODECS_ALL,
+ 0, 270, 90, 0,
+ V4L2_CID_ROTATE,
+ HFI_PROP_ROTATION,
+ CAP_FLAG_OUTPUT_PORT,
+ {0},
+ NULL,
+ set_rotation},
+
+ {HEADER_MODE, ENC, CODECS_ALL,
+ V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE,
+ V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME,
+ BIT(V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE) |
+ BIT(V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME),
+ V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE,
+ V4L2_CID_MPEG_VIDEO_HEADER_MODE,
+ HFI_PROP_SEQ_HEADER_MODE,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {0},
+ NULL,
+ set_header_mode},
+
+ {PREPEND_SPSPPS_TO_IDR, ENC, CODECS_ALL,
+ 0, 1, 1, 0,
+ V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR},
+
+ {REQUEST_I_FRAME, ENC, H264 | HEVC,
+ 0, 0, 0, 0,
+ V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME,
+ HFI_PROP_REQUEST_SYNC_FRAME,
+ CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ NULL,
+ set_req_sync_frame},
+
+ {BIT_RATE, ENC, H264 | HEVC,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_BITRATE,
+ HFI_PROP_TOTAL_BITRATE,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {PEAK_BITRATE, L0_BR},
+ adjust_bitrate,
+ set_bitrate},
+
+ {PEAK_BITRATE, ENC, H264 | HEVC,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_BITRATE_PEAK,
+ HFI_PROP_TOTAL_PEAK_BITRATE,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ adjust_peak_bitrate,
+ set_peak_bitrate},
+
+ {BITRATE_MODE, ENC, H264 | HEVC,
+ V4L2_MPEG_VIDEO_BITRATE_MODE_VBR,
+ V4L2_MPEG_VIDEO_BITRATE_MODE_CBR,
+ BIT(V4L2_MPEG_VIDEO_BITRATE_MODE_VBR) |
+ BIT(V4L2_MPEG_VIDEO_BITRATE_MODE_CBR),
+ V4L2_MPEG_VIDEO_BITRATE_MODE_VBR,
+ V4L2_CID_MPEG_VIDEO_BITRATE_MODE,
+ HFI_PROP_RATE_CONTROL,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {LTR_COUNT, IR_PERIOD, I_FRAME_QP, P_FRAME_QP,
+ B_FRAME_QP, ENH_LAYER_COUNT, BIT_RATE,
+ MIN_QUALITY, PEAK_BITRATE, SLICE_MODE},
+ adjust_bitrate_mode,
+ set_u32_enum},
+
+ {FRAME_SKIP_MODE, ENC, H264 | HEVC,
+ V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_DISABLED,
+ V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_BUF_LIMIT,
+ BIT(V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_DISABLED) |
+ BIT(V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_LEVEL_LIMIT) |
+ BIT(V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_BUF_LIMIT),
+ V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_DISABLED,
+ V4L2_CID_MPEG_VIDEO_FRAME_SKIP_MODE,
+ 0,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+
+ {FRAME_RC_ENABLE, ENC, H264 | HEVC,
+ 0, 1, 1, 1,
+ V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE},
+
+ {GOP_SIZE, ENC, CODECS_ALL,
+ 0, INT_MAX, 1, 2 * DEFAULT_FPS - 1,
+ V4L2_CID_MPEG_VIDEO_GOP_SIZE,
+ HFI_PROP_MAX_GOP_FRAMES,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ adjust_gop_size,
+ set_gop_size},
+
+ {B_FRAME, ENC, H264 | HEVC,
+ 0, 7, 1, 0,
+ V4L2_CID_MPEG_VIDEO_B_FRAMES,
+ HFI_PROP_MAX_B_FRAMES,
+ CAP_FLAG_OUTPUT_PORT,
+ {0},
+ adjust_b_frame,
+ set_u32},
+
+ {LTR_COUNT, ENC, H264 | HEVC,
+ 0, MAX_LTR_FRAME_COUNT, 1, 0,
+ V4L2_CID_MPEG_VIDEO_LTR_COUNT,
+ HFI_PROP_LTR_COUNT,
+ CAP_FLAG_OUTPUT_PORT,
+ {0},
+ adjust_ltr_count,
+ set_u32},
+
+ {USE_LTR, ENC, H264 | HEVC,
+ 0,
+ ((1 << MAX_LTR_FRAME_COUNT) - 1),
+ 0, 0,
+ V4L2_CID_MPEG_VIDEO_USE_LTR_FRAMES,
+ HFI_PROP_LTR_USE,
+ CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ adjust_use_ltr,
+ set_use_and_mark_ltr},
+
+ {MARK_LTR, ENC, H264 | HEVC,
+ INVALID_DEFAULT_MARK_OR_USE_LTR,
+ (MAX_LTR_FRAME_COUNT - 1),
+ 1, INVALID_DEFAULT_MARK_OR_USE_LTR,
+ V4L2_CID_MPEG_VIDEO_FRAME_LTR_INDEX,
+ HFI_PROP_LTR_MARK,
+ CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ adjust_mark_ltr,
+ set_use_and_mark_ltr},
+
+ {BASELAYER_PRIORITY, ENC, H264,
+ 0, MAX_BASE_LAYER_PRIORITY_ID, 1, 0,
+ V4L2_CID_MPEG_VIDEO_BASELAYER_PRIORITY_ID,
+ HFI_PROP_BASELAYER_PRIORITYID,
+ CAP_FLAG_OUTPUT_PORT},
+
+ {IR_TYPE, ENC, H264 | HEVC,
+ V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_RANDOM,
+ V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_CYCLIC,
+ BIT(V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_RANDOM) |
+ BIT(V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_CYCLIC),
+ V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_RANDOM,
+ V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE,
+ 0,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+
+ {IR_PERIOD, ENC, H264 | HEVC,
+ 0, INT_MAX, 1, 0,
+ V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD,
+ 0,
+ CAP_FLAG_INPUT_PORT | CAP_FLAG_OUTPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ adjust_ir_period,
+ set_ir_period},
+
+ {MIN_QUALITY, ENC, H264 | HEVC,
+ 0, MAX_SUPPORTED_MIN_QUALITY, 70, MAX_SUPPORTED_MIN_QUALITY,
+ 0,
+ HFI_PROP_MAINTAIN_MIN_QUALITY,
+ CAP_FLAG_OUTPUT_PORT,
+ {0},
+ adjust_min_quality,
+ set_u32},
+
+ {MIN_FRAME_QP, ENC, H264,
+ MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT,
+ V4L2_CID_MPEG_VIDEO_H264_MIN_QP,
+ HFI_PROP_MIN_QP_PACKED,
+ CAP_FLAG_OUTPUT_PORT,
+ {0},
+ NULL,
+ set_min_qp},
+
+ {MIN_FRAME_QP, ENC, HEVC,
+ MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT,
+ V4L2_CID_MPEG_VIDEO_HEVC_MIN_QP,
+ HFI_PROP_MIN_QP_PACKED,
+ CAP_FLAG_OUTPUT_PORT,
+ {0},
+ NULL,
+ set_min_qp},
+
+ {MAX_FRAME_QP, ENC, H264,
+ MIN_QP_8BIT, MAX_QP, 1, MAX_QP,
+ V4L2_CID_MPEG_VIDEO_H264_MAX_QP,
+ HFI_PROP_MAX_QP_PACKED,
+ CAP_FLAG_OUTPUT_PORT,
+ {0},
+ NULL,
+ set_max_qp},
+
+ {MAX_FRAME_QP, ENC, HEVC,
+ MIN_QP_8BIT, MAX_QP, 1, MAX_QP,
+ V4L2_CID_MPEG_VIDEO_HEVC_MAX_QP,
+ HFI_PROP_MAX_QP_PACKED,
+ CAP_FLAG_OUTPUT_PORT,
+ {0},
+ NULL,
+ set_max_qp},
+
+ {I_FRAME_MIN_QP, ENC, H264,
+ MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT,
+ V4L2_CID_MPEG_VIDEO_H264_I_FRAME_MIN_QP},
+
+ {I_FRAME_MIN_QP, ENC, HEVC,
+ MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT,
+ V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_MIN_QP},
+
+ {P_FRAME_MIN_QP, ENC, H264,
+ MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT,
+ V4L2_CID_MPEG_VIDEO_H264_P_FRAME_MIN_QP},
+
+ {P_FRAME_MIN_QP, ENC, HEVC,
+ MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT,
+ V4L2_CID_MPEG_VIDEO_HEVC_P_FRAME_MIN_QP},
+
+ {B_FRAME_MIN_QP, ENC, H264,
+ MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT,
+ V4L2_CID_MPEG_VIDEO_H264_B_FRAME_MIN_QP},
+
+ {B_FRAME_MIN_QP, ENC, HEVC,
+ MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT,
+ V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_MIN_QP},
+
+ {I_FRAME_MAX_QP, ENC, H264,
+ MIN_QP_8BIT, MAX_QP, 1, MAX_QP,
+ V4L2_CID_MPEG_VIDEO_H264_I_FRAME_MAX_QP},
+
+ {I_FRAME_MAX_QP, ENC, HEVC,
+ MIN_QP_8BIT, MAX_QP, 1, MAX_QP,
+ V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_MAX_QP},
+
+ {P_FRAME_MAX_QP, ENC, H264,
+ MIN_QP_8BIT, MAX_QP, 1, MAX_QP,
+ V4L2_CID_MPEG_VIDEO_H264_P_FRAME_MAX_QP},
+
+ {P_FRAME_MAX_QP, ENC, HEVC,
+ MIN_QP_8BIT, MAX_QP, 1, MAX_QP,
+ V4L2_CID_MPEG_VIDEO_HEVC_P_FRAME_MAX_QP},
+
+ {B_FRAME_MAX_QP, ENC, H264,
+ MIN_QP_8BIT, MAX_QP, 1, MAX_QP,
+ V4L2_CID_MPEG_VIDEO_H264_B_FRAME_MAX_QP},
+
+ {B_FRAME_MAX_QP, ENC, HEVC,
+ MIN_QP_8BIT, MAX_QP, 1, MAX_QP,
+ V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_MAX_QP},
+
+ {I_FRAME_QP, ENC, HEVC,
+ MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP,
+ V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_QP,
+ HFI_PROP_QP_PACKED,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ NULL,
+ set_frame_qp},
+
+ {I_FRAME_QP, ENC, H264,
+ MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP,
+ V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP,
+ HFI_PROP_QP_PACKED,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ NULL,
+ set_frame_qp},
+
+ {P_FRAME_QP, ENC, HEVC,
+ MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP,
+ V4L2_CID_MPEG_VIDEO_HEVC_P_FRAME_QP,
+ HFI_PROP_QP_PACKED,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ NULL,
+ set_frame_qp},
+
+ {P_FRAME_QP, ENC, H264,
+ MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP,
+ V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP,
+ HFI_PROP_QP_PACKED,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ NULL,
+ set_frame_qp},
+
+ {B_FRAME_QP, ENC, HEVC,
+ MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP,
+ V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_QP,
+ HFI_PROP_QP_PACKED,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ NULL,
+ set_frame_qp},
+
+ {B_FRAME_QP, ENC, H264,
+ MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP,
+ V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP,
+ HFI_PROP_QP_PACKED,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ NULL,
+ set_frame_qp},
+
+ {LAYER_TYPE, ENC, HEVC,
+ V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B,
+ V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_P,
+ BIT(V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B) |
+ BIT(V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_P),
+ V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_P,
+ V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_TYPE,
+ HFI_PROP_LAYER_ENCODING_TYPE,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {LTR_COUNT}},
+
+ {LAYER_TYPE, ENC, H264,
+ V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_B,
+ V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_P,
+ BIT(V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_B) |
+ BIT(V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_P),
+ V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_P,
+ V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_TYPE,
+ HFI_PROP_LAYER_ENCODING_TYPE,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {LTR_COUNT}},
+
+ {LAYER_ENABLE, ENC, H264,
+ 0, 1, 1, 0,
+ V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING,
+ HFI_PROP_LAYER_ENCODING_TYPE,
+ CAP_FLAG_OUTPUT_PORT},
+
+ {LAYER_ENABLE, ENC, HEVC,
+ 0, 1, 1, 0,
+ 0,
+ 0,
+ CAP_FLAG_OUTPUT_PORT},
+
+ {ENH_LAYER_COUNT, ENC, HEVC,
+ 0, 5, 1, 0,
+ V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_LAYER,
+ HFI_PROP_LAYER_COUNT,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {GOP_SIZE, B_FRAME, BIT_RATE, MIN_QUALITY, SLICE_MODE,
+ LTR_COUNT},
+ adjust_layer_count,
+ set_layer_count_and_type},
+
+ {ENH_LAYER_COUNT, ENC, H264,
+ 0, 5, 1, 0,
+ V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER,
+ HFI_PROP_LAYER_COUNT,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {GOP_SIZE, B_FRAME, BIT_RATE, MIN_QUALITY, SLICE_MODE,
+ LTR_COUNT},
+ adjust_layer_count,
+ set_layer_count_and_type},
+
+ {L0_BR, ENC, H264,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L0_BR,
+ HFI_PROP_BITRATE_LAYER1,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {L1_BR},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {L0_BR, ENC, HEVC,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L0_BR,
+ HFI_PROP_BITRATE_LAYER1,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {L1_BR},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {L1_BR, ENC, H264,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L1_BR,
+ HFI_PROP_BITRATE_LAYER2,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {L2_BR},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {L1_BR, ENC, HEVC,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L1_BR,
+ HFI_PROP_BITRATE_LAYER2,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {L2_BR},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {L2_BR, ENC, H264,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L2_BR,
+ HFI_PROP_BITRATE_LAYER3,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {L3_BR},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {L2_BR, ENC, HEVC,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L2_BR,
+ HFI_PROP_BITRATE_LAYER3,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {L3_BR},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {L3_BR, ENC, H264,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L3_BR,
+ HFI_PROP_BITRATE_LAYER4,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {L4_BR},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {L3_BR, ENC, HEVC,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L3_BR,
+ HFI_PROP_BITRATE_LAYER4,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {L4_BR},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {L4_BR, ENC, H264,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L4_BR,
+ HFI_PROP_BITRATE_LAYER5,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {L5_BR},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {L4_BR, ENC, HEVC,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L4_BR,
+ HFI_PROP_BITRATE_LAYER5,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {L5_BR},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {L5_BR, ENC, H264,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L5_BR,
+ HFI_PROP_BITRATE_LAYER6,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {L5_BR, ENC, HEVC,
+ 1, MAX_BITRATE, 1, DEFAULT_BITRATE,
+ V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L5_BR,
+ HFI_PROP_BITRATE_LAYER6,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT |
+ CAP_FLAG_DYNAMIC_ALLOWED,
+ {0},
+ adjust_layer_bitrate,
+ set_layer_bitrate},
+
+ {ENTROPY_MODE, ENC, H264,
+ V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CAVLC,
+ V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC,
+ BIT(V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CAVLC) |
+ BIT(V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC),
+ V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC,
+ V4L2_CID_MPEG_VIDEO_H264_ENTROPY_MODE,
+ HFI_PROP_CABAC_SESSION,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {BIT_RATE},
+ adjust_entropy_mode,
+ set_u32},
+
+ {SLICE_MODE, ENC, H264 | HEVC,
+ V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE,
+ V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_BYTES,
+ BIT(V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE) |
+ BIT(V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB) |
+ BIT(V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_BYTES),
+ V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE,
+ V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MODE,
+ 0,
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {STAGE},
+ adjust_slice_count,
+ set_slice_count},
+
+ {SLICE_MAX_BYTES, ENC, H264 | HEVC,
+ MIN_SLICE_BYTE_SIZE, MAX_SLICE_BYTE_SIZE,
+ 1, MIN_SLICE_BYTE_SIZE,
+ V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MAX_BYTES,
+ HFI_PROP_MULTI_SLICE_BYTES_COUNT,
+ CAP_FLAG_OUTPUT_PORT},
+
+ {SLICE_MAX_MB, ENC, H264 | HEVC,
+ 1, MAX_SLICE_MB_SIZE, 1, 1,
+ V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MAX_MB,
+ HFI_PROP_MULTI_SLICE_MB_COUNT,
+ CAP_FLAG_OUTPUT_PORT},
+
+ {TRANSFORM_8X8, ENC, H264,
+ 0, 1, 1, 1,
+ V4L2_CID_MPEG_VIDEO_H264_8X8_TRANSFORM,
+ HFI_PROP_8X8_TRANSFORM,
+ CAP_FLAG_OUTPUT_PORT,
+ {0},
+ adjust_transform_8x8,
+ set_u32},
+
+ {DISPLAY_DELAY_ENABLE, DEC, CODECS_ALL,
0, 1, 1, 0,
V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY_ENABLE,
HFI_PROP_DECODE_ORDER_OUTPUT,
@@ -264,7 +952,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
NULL},

- {DISPLAY_DELAY, CODECS_ALL,
+ {DISPLAY_DELAY, DEC, CODECS_ALL,
0, 1, 1, 0,
V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY,
HFI_PROP_DECODE_ORDER_OUTPUT,
@@ -273,7 +961,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
NULL},

- {OUTPUT_ORDER, CODECS_ALL,
+ {OUTPUT_ORDER, DEC, CODECS_ALL,
0, 1, 1, 0,
0,
HFI_PROP_DECODE_ORDER_OUTPUT,
@@ -282,7 +970,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
adjust_output_order,
set_u32},

- {INPUT_BUF_HOST_MAX_COUNT, CODECS_ALL,
+ {INPUT_BUF_HOST_MAX_COUNT, ENC | DEC, CODECS_ALL,
DEFAULT_MAX_HOST_BUF_COUNT, DEFAULT_MAX_HOST_BURST_BUF_COUNT,
1, DEFAULT_MAX_HOST_BUF_COUNT,
0,
@@ -292,7 +980,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
set_u32},

- {STAGE, CODECS_ALL,
+ {STAGE, ENC | DEC, CODECS_ALL,
STAGE_1,
STAGE_2, 1,
STAGE_2,
@@ -303,7 +991,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
set_stage},

- {PIPE, CODECS_ALL,
+ {PIPE, ENC | DEC, CODECS_ALL,
PIPE_1,
PIPE_4, 1,
PIPE_4,
@@ -314,26 +1002,26 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
NULL,
set_pipe},

- {POC, H264, 0, 2, 1, 1,
+ {POC, DEC, H264, 0, 2, 1, 1,
0,
HFI_PROP_PIC_ORDER_CNT_TYPE},

- {CODED_FRAMES, H264 | HEVC,
+ {CODED_FRAMES, DEC, H264 | HEVC,
CODED_FRAMES_PROGRESSIVE, CODED_FRAMES_PROGRESSIVE,
0, CODED_FRAMES_PROGRESSIVE,
0,
HFI_PROP_CODED_FRAMES},

- {BIT_DEPTH, CODECS_ALL, BIT_DEPTH_8, BIT_DEPTH_10, 1, BIT_DEPTH_8,
+ {BIT_DEPTH, DEC, CODECS_ALL, BIT_DEPTH_8, BIT_DEPTH_10, 1, BIT_DEPTH_8,
0,
HFI_PROP_LUMA_CHROMA_BIT_DEPTH},

- {DEFAULT_HEADER, CODECS_ALL,
+ {DEFAULT_HEADER, DEC, CODECS_ALL,
0, 1, 1, 0,
0,
HFI_PROP_DEC_DEFAULT_HEADER},

- {RAP_FRAME, CODECS_ALL,
+ {RAP_FRAME, DEC, CODECS_ALL,
0, 1, 1, 1,
0,
HFI_PROP_DEC_START_FROM_RAP_FRAME,
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
index 5a02f24..58498af 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
@@ -42,6 +42,9 @@ u64 iris_calc_freq_iris3(struct iris_inst *inst, u32 data_size)

if (inst->codec == VP9) {
vsp_cycles = div_u64(vsp_cycles * 170, 100);
+ } else if (inst->cap[ENTROPY_MODE].value ==
+ V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC) {
+ vsp_cycles = div_u64(vsp_cycles * 135, 100);
} else {
base_cycles = 0;
vsp_cycles = div_u64(vsp_cycles, 2);
--
2.7.4

2023-12-18 11:51:32

[permalink] [raw]

Subject: [PATCH v2 29/34] media: iris: implement power management

From: Vikash Garodia <[email protected]>

Hardware specific power sequence include programming specific
sequence of registers. Implements this sequence for iris3.

Introduce video device suspend and resume using runtime PM.

Also, implement iris3 specific calculation for clock and
bus bandwidth which depends on hardware configuration,
codec format, resolution and frame rate.

Signed-off-by: Vikash Garodia <[email protected]>
Signed-off-by: Dikshita Agarwal <[email protected]>
---
drivers/media/platform/qcom/vcodec/iris/Makefile | 2 +
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 7 +
.../media/platform/qcom/vcodec/iris/iris_common.h | 7 +
.../media/platform/qcom/vcodec/iris/iris_core.h | 4 +
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 66 +++
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 4 +
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 175 +++++++-
drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 5 +
.../platform/qcom/vcodec/iris/iris_hfi_packet.c | 48 +++
.../platform/qcom/vcodec/iris/iris_hfi_packet.h | 4 +
.../platform/qcom/vcodec/iris/iris_hfi_response.c | 2 +
.../platform/qcom/vcodec/iris/iris_instance.h | 10 +
.../media/platform/qcom/vcodec/iris/iris_power.c | 173 ++++++++
.../media/platform/qcom/vcodec/iris/iris_power.h | 15 +
.../media/platform/qcom/vcodec/iris/iris_probe.c | 118 +++++-
drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 42 +-
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 7 +
.../media/platform/qcom/vcodec/iris/iris_vidc.c | 36 +-
.../platform/qcom/vcodec/iris/platform_common.h | 18 +
.../media/platform/qcom/vcodec/iris/resources.c | 280 +++++++++++++
.../media/platform/qcom/vcodec/iris/resources.h | 8 +
.../media/platform/qcom/vcodec/iris/vpu_common.c | 87 ++++
.../media/platform/qcom/vcodec/iris/vpu_common.h | 10 +
.../media/platform/qcom/vcodec/iris/vpu_iris3.c | 454 ++++++++++++++++++---
.../platform/qcom/vcodec/iris/vpu_iris3_power.c | 97 +++++
.../platform/qcom/vcodec/iris/vpu_iris3_power.h | 13 +
26 files changed, 1614 insertions(+), 78 deletions(-)
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_power.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_power.h
create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.h

diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
index 7e3d9f1..4c8f8f6 100644
--- a/drivers/media/platform/qcom/vcodec/iris/Makefile
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -13,10 +13,12 @@ iris-objs += iris_probe.o \
iris_hfi_response.o \
iris_hfi_packet.o \
iris_buffer.o \
+ iris_power.o \
resources.o \
vpu_common.o \
vpu_iris3.o \
vpu_iris3_buffer.o \
+ vpu_iris3_power.o \
platform_common.o \
platform_sm8550.o

diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index bbe8ca6..872674e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -12,6 +12,7 @@

#define HFI_CMD_BEGIN 0x01000000
#define HFI_CMD_INIT 0x01000001
+#define HFI_CMD_POWER_COLLAPSE 0x01000002
#define HFI_CMD_OPEN 0x01000003
#define HFI_CMD_CLOSE 0x01000004
#define HFI_CMD_START 0x01000005
@@ -46,10 +47,16 @@ enum hfi_property_mode_type {

#define HFI_BITMASK_FRAME_MBS_ONLY_FLAG 0x00000001

+#define HFI_TRUE 0x00000001
+
+#define HFI_FALSE 0x00000000
+
#define HFI_PROP_BEGIN 0x03000000

#define HFI_PROP_IMAGE_VERSION 0x03000001

+#define HFI_PROP_INTRA_FRAME_POWER_COLLAPSE 0x03000002
+
enum hfi_codec_type {
HFI_CODEC_DECODE_AVC = 1,
HFI_CODEC_ENCODE_AVC = 2,
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
index 24b322a..3ab4767 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
@@ -28,6 +28,8 @@
#define MAX_DPB_LIST_ARRAY_SIZE (16 * 4)
#define MAX_DPB_LIST_PAYLOAD_SIZE (16 * 4 * 4)

+#define INPUT_TIMER_LIST_SIZE 30
+
enum codec_type {
H264 = BIT(0),
HEVC = BIT(1),
@@ -128,4 +130,9 @@ struct iris_hfi_frame_info {
u32 overflow;
};

+struct iris_input_timer {
+ struct list_head list;
+ u64 time_us;
+};
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index 6452ea7..50553d2 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -101,6 +101,10 @@ struct iris_core {
struct list_head instances;
u32 intr_status;
char fw_version[IRIS_VERSION_LENGTH];
+ struct mutex pm_lock; /* lock for pm operations */
+ u32 skip_pc_count;
+ bool power_enabled;
+ struct iris_core_power power;
};

int iris_core_init(struct iris_core *core);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index a5b8aef..395a189 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -3,12 +3,15 @@
* Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
*/

+#include <linux/pm_runtime.h>
+
#include "hfi_defines.h"
#include "iris_core.h"
#include "iris_helpers.h"
#include "iris_hfi.h"
#include "iris_hfi_packet.h"
#include "iris_instance.h"
+#include "iris_power.h"
#include "iris_vidc.h"

int check_core_lock(struct iris_core *core)
@@ -479,6 +482,8 @@ int queue_deferred_buffers(struct iris_inst *inst, enum iris_buffer_type buf_typ
if (!buffers)
return -EINVAL;

+ iris_scale_power(inst);
+
list_for_each_entry(buf, &buffers->list, list) {
if (!(buf->attr & BUF_ATTR_DEFERRED))
continue;
@@ -775,6 +780,9 @@ int session_streamoff(struct iris_inst *inst, u32 plane)
if (ret)
goto error;

+ if (plane == INPUT_MPLANE)
+ iris_flush_input_timer(inst);
+
/* no more queued buffers after streamoff */
count = get_num_queued_buffers(inst, buffer_type);
if (count) {
@@ -797,3 +805,61 @@ int session_streamoff(struct iris_inst *inst, u32 plane)

return ret;
}
+
+int iris_pm_get(struct iris_core *core)
+{
+ int ret;
+
+ mutex_lock(&core->pm_lock);
+ ret = pm_runtime_resume_and_get(core->dev);
+ mutex_unlock(&core->pm_lock);
+
+ return ret;
+}
+
+int iris_pm_put(struct iris_core *core, bool autosuspend)
+{
+ int ret;
+
+ mutex_lock(&core->pm_lock);
+
+ if (autosuspend)
+ ret = pm_runtime_put_autosuspend(core->dev);
+ else
+ ret = pm_runtime_put_sync(core->dev);
+ if (ret > 0)
+ ret = 0;
+
+ mutex_unlock(&core->pm_lock);
+
+ return ret;
+}
+
+int iris_pm_get_put(struct iris_core *core)
+{
+ int ret = 0;
+
+ mutex_lock(&core->pm_lock);
+
+ if (pm_runtime_suspended(core->dev)) {
+ ret = pm_runtime_resume_and_get(core->dev);
+ if (ret < 0)
+ goto error;
+
+ ret = pm_runtime_put_autosuspend(core->dev);
+ }
+ if (ret > 0)
+ ret = 0;
+
+error:
+ mutex_unlock(&core->pm_lock);
+
+ return ret;
+}
+
+void iris_pm_touch(struct iris_core *core)
+{
+ mutex_lock(&core->pm_lock);
+ pm_runtime_mark_last_busy(core->dev);
+ mutex_unlock(&core->pm_lock);
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
index 3a9889e..c628b2e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -51,5 +51,9 @@ int iris_vb2_buffer_done(struct iris_inst *inst,
int iris_release_nonref_buffers(struct iris_inst *inst);
void iris_destroy_buffers(struct iris_inst *inst);
int session_streamoff(struct iris_inst *inst, u32 plane);
+int iris_pm_get(struct iris_core *core);
+int iris_pm_put(struct iris_core *core, bool autosuspend);
+int iris_pm_get_put(struct iris_core *core);
+void iris_pm_touch(struct iris_core *core);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index 2b3fa47..2850fd5 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -67,6 +67,41 @@ static bool validate_session(struct iris_core *core,
return valid;
}

+static int iris_power_off(struct iris_core *core)
+{
+ int ret;
+
+ if (!core->power_enabled)
+ return 0;
+
+ ret = call_vpu_op(core, power_off, core);
+ if (ret) {
+ dev_err(core->dev, "Failed to power off, err: %d\n", ret);
+ return ret;
+ }
+ core->power_enabled = false;
+
+ return ret;
+}
+
+static int iris_power_on(struct iris_core *core)
+{
+ int ret;
+
+ if (core->power_enabled)
+ return 0;
+
+ ret = call_vpu_op(core, power_on, core);
+ if (ret) {
+ dev_err(core->dev, "Failed to power on, err: %d\n", ret);
+ return ret;
+ }
+
+ core->power_enabled = true;
+
+ return ret;
+}
+
static int sys_init(struct iris_core *core)
{
int ret;
@@ -113,20 +148,24 @@ int iris_hfi_core_init(struct iris_core *core)
if (ret)
goto error;

+ ret = iris_power_on(core);
+ if (ret)
+ goto error;
+
core->use_tz = use_tz(core->dev);
if (!core->use_tz)
- goto error;
+ goto error_power_off;

pas_id = core->platform_data->pas_id;
fw_name = core->platform_data->fwname;
ret = load_fw(core->dev, fw_name, &mem_phys, &mem_size,
pas_id, core->use_tz);
if (ret)
- goto error;
+ goto error_power_off;

ret = auth_reset_fw(pas_id);
if (ret)
- goto error;
+ goto error_power_off;

cp_start = core->cap[CP_START].value;
cp_size = core->cap[CP_SIZE].value;
@@ -135,22 +174,24 @@ int iris_hfi_core_init(struct iris_core *core)
ret = protect_secure_region(cp_start, cp_size, cp_nonpixel_start,
cp_nonpixel_size, pas_id);
if (ret)
- goto error;
+ goto error_power_off;

ret = call_vpu_op(core, boot_firmware, core);
if (ret)
- goto error;
+ goto error_power_off;

ret = sys_init(core);
if (ret)
- goto error;
+ goto error_power_off;

ret = sys_image_version(core);
if (ret)
- goto error;
+ goto error_power_off;

return ret;

+error_power_off:
+ iris_power_off(core);
error:
dev_err(core->dev, "%s(): failed with ret %d\n", __func__, ret);

@@ -170,6 +211,8 @@ int iris_hfi_core_deinit(struct iris_core *core)

unload_fw(core->platform_data->pas_id);

+ iris_power_off(core);
+
return ret;
}

@@ -573,16 +616,24 @@ irqreturn_t iris_hfi_isr(int irq, void *data)
irqreturn_t iris_hfi_isr_handler(int irq, void *data)
{
struct iris_core *core = data;
+ int ret;

if (!core)
return IRQ_NONE;

+ ret = iris_pm_get(core);
+ if (ret)
+ goto exit;
+
mutex_lock(&core->lock);
call_vpu_op(core, clear_interrupt, core);
mutex_unlock(&core->lock);

__response_handler(core);

+ iris_pm_put(core, true);
+
+exit:
if (!call_vpu_op(core, watchdog, core, core->intr_status))
enable_irq(irq);

@@ -701,3 +752,113 @@ int iris_hfi_release_buffer(struct iris_inst *inst,

return ret;
}
+
+int prepare_pc(struct iris_core *core)
+{
+ int ret;
+
+ ret = hfi_packet_sys_pc_prep(core, core->packet, core->packet_size);
+ if (ret)
+ goto err_pc_prep;
+
+ ret = iris_hfi_queue_cmd_write(core, core->packet);
+ if (ret)
+ goto err_pc_prep;
+
+ return ret;
+
+err_pc_prep:
+ dev_err(core->dev, "Failed to prepare iris for power off\n");
+
+ return ret;
+}
+
+int iris_hfi_pm_suspend(struct iris_core *core)
+{
+ int ret;
+
+ ret = check_core_lock(core);
+ if (ret)
+ return ret;
+
+ if (!core_in_valid_state(core))
+ return -EINVAL;
+
+ if (!core->power_enabled)
+ return 0;
+
+ if (core->skip_pc_count >= MAX_PC_SKIP_COUNT) {
+ dev_err(core->dev, "Failed to PC for %d times\n", core->skip_pc_count);
+ core->skip_pc_count = 0;
+ iris_change_core_state(core, IRIS_CORE_ERROR);
+ iris_core_deinit_locked(core);
+ return -EINVAL;
+ }
+
+ iris_flush_debug_queue(core, core->packet, core->packet_size);
+
+ ret = call_vpu_op(core, prepare_pc, core);
+ if (ret) {
+ core->skip_pc_count++;
+ iris_pm_touch(core);
+ return -EAGAIN;
+ }
+
+ ret = set_hw_state(false);
+ if (ret)
+ return ret;
+
+ ret = iris_power_off(core);
+ if (ret)
+ return ret;
+
+ core->skip_pc_count = 0;
+
+ return ret;
+}
+
+int iris_hfi_pm_resume(struct iris_core *core)
+{
+ int ret;
+
+ ret = check_core_lock(core);
+ if (ret)
+ return ret;
+
+ if (!core_in_valid_state(core))
+ return -EINVAL;
+
+ if (core->power_enabled)
+ return 0;
+
+ ret = iris_power_on(core);
+ if (ret)
+ goto error;
+
+ ret = set_hw_state(true);
+ if (ret)
+ goto err_power_off;
+
+ ret = call_vpu_op(core, boot_firmware, core);
+ if (ret)
+ goto err_suspend_hw;
+
+ ret = hfi_packet_sys_interframe_powercollapse(core, core->packet, core->packet_size);
+ if (ret)
+ goto err_suspend_hw;
+
+ ret = iris_hfi_queue_cmd_write(core, core->packet);
+ if (ret)
+ goto err_suspend_hw;
+
+ return ret;
+
+err_suspend_hw:
+ set_hw_state(false);
+err_power_off:
+ iris_power_off(core);
+error:
+ dev_err(core->dev, "Failed to Resume\n");
+
+ return -EBUSY;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
index ab479a9..465bfc5 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
@@ -9,6 +9,8 @@
#include "iris_instance.h"
#include "iris_core.h"

+#define MAX_PC_SKIP_COUNT 10
+
int iris_hfi_core_init(struct iris_core *core);
int iris_hfi_core_deinit(struct iris_core *core);
int iris_hfi_session_open(struct iris_inst *inst);
@@ -32,6 +34,9 @@ int iris_hfi_queue_buffer(struct iris_inst *inst,
struct iris_buffer *buffer);
int iris_hfi_release_buffer(struct iris_inst *inst,
struct iris_buffer *buffer);
+int iris_hfi_pm_suspend(struct iris_core *core);
+int iris_hfi_pm_resume(struct iris_core *core);
+int prepare_pc(struct iris_core *core);

irqreturn_t iris_hfi_isr(int irq, void *data);
irqreturn_t iris_hfi_isr_handler(int irq, void *data);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
index 8dafd04..fab206cf 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
@@ -622,3 +622,51 @@ int hfi_packet_session_property(struct iris_inst *inst,

return ret;
}
+
+int hfi_packet_sys_interframe_powercollapse(struct iris_core *core,
+ u8 *pkt, u32 pkt_size)
+{
+ u32 payload = 0;
+ int ret;
+
+ ret = hfi_create_header(pkt, pkt_size,
+ 0 /*session_id*/,
+ core->header_id++);
+ if (ret)
+ return ret;
+
+ payload = HFI_TRUE;
+
+ ret = hfi_create_packet(pkt, pkt_size,
+ HFI_PROP_INTRA_FRAME_POWER_COLLAPSE,
+ HFI_HOST_FLAGS_NONE,
+ HFI_PAYLOAD_U32,
+ HFI_PORT_NONE,
+ core->packet_id++,
+ &payload,
+ sizeof(u32));
+
+ return ret;
+}
+
+int hfi_packet_sys_pc_prep(struct iris_core *core,
+ u8 *pkt, u32 pkt_size)
+{
+ int ret;
+
+ ret = hfi_create_header(pkt, pkt_size,
+ 0 /*session_id*/,
+ core->header_id++);
+ if (ret)
+ return ret;
+
+ ret = hfi_create_packet(pkt, pkt_size,
+ HFI_CMD_POWER_COLLAPSE,
+ HFI_HOST_FLAGS_NONE,
+ HFI_PAYLOAD_NONE,
+ HFI_PORT_NONE,
+ core->packet_id++,
+ NULL, 0);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
index 8999c28..849b585 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
@@ -104,5 +104,9 @@ int hfi_packet_session_command(struct iris_inst *inst, u32 pkt_type,
int hfi_packet_session_property(struct iris_inst *inst,
u32 pkt_type, u32 flags, u32 port,
u32 payload_type, void *payload, u32 payload_size);
+int hfi_packet_sys_interframe_powercollapse(struct iris_core *core,
+ u8 *pkt, u32 pkt_size);
+int hfi_packet_sys_pc_prep(struct iris_core *core,
+ u8 *pkt, u32 pkt_size);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
index 3548f9d..a4d5d9c 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
@@ -928,6 +928,8 @@ static int handle_response(struct iris_core *core, void *response)
struct hfi_header *hdr;
int ret;

+ iris_pm_touch(core);
+
hdr = (struct hfi_header *)response;
ret = validate_hdr_packet(core, hdr);
if (ret)
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
index 50154bf..3c85e78 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -47,6 +47,11 @@
* @dst_subcr_params: subscription params to fw on output port
* @dpb_list_payload: array of dpb buffers
* @once_per_session_set: boolean to set once per session property
+ * @max_rate: max input rate
+ * @max_input_data_size: max size of input data
+ * @power: structure of power info
+ * @bus_data: structure of bus data
+ * @input_timer_list: list head of input timer
*/

struct iris_inst {
@@ -80,6 +85,11 @@ struct iris_inst {
struct subscription_params dst_subcr_params;
u32 dpb_list_payload[MAX_DPB_LIST_ARRAY_SIZE];
bool once_per_session_set;
+ u32 max_rate;
+ u32 max_input_data_size;
+ struct iris_inst_power power;
+ struct bus_vote_data bus_data;
+ struct list_head input_timer_list;
};

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_power.c b/drivers/media/platform/qcom/vcodec/iris/iris_power.c
new file mode 100644
index 0000000..77439e3
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_power.c
@@ -0,0 +1,173 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "iris_power.h"
+#include "iris_helpers.h"
+#include "resources.h"
+
+static int iris_set_buses(struct iris_inst *inst)
+{
+ struct iris_inst *instance;
+ struct iris_core *core;
+ u64 total_bw_ddr = 0;
+ int ret;
+
+ core = inst->core;
+
+ mutex_lock(&core->lock);
+ list_for_each_entry(instance, &core->instances, list) {
+ if (!instance->max_input_data_size)
+ continue;
+
+ total_bw_ddr += instance->power.bus_bw;
+ }
+
+ ret = vote_buses(core, total_bw_ddr);
+
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+static int iris_vote_buses(struct iris_inst *inst)
+{
+ struct v4l2_format *out_f, *inp_f;
+ struct bus_vote_data *vote_data;
+ struct iris_core *core;
+
+ core = inst->core;
+
+ vote_data = &inst->bus_data;
+
+ out_f = inst->fmt_dst;
+ inp_f = inst->fmt_src;
+
+ vote_data->width = inp_f->fmt.pix_mp.width;
+ vote_data->height = inp_f->fmt.pix_mp.height;
+ vote_data->fps = inst->max_rate;
+
+ if (is_linear_colorformat(out_f->fmt.pix_mp.pixelformat)) {
+ vote_data->color_formats[0] = V4L2_PIX_FMT_NV12;
+ vote_data->color_formats[1] = out_f->fmt.pix_mp.pixelformat;
+ } else {
+ vote_data->color_formats[0] = out_f->fmt.pix_mp.pixelformat;
+ }
+
+ call_session_op(core, calc_bw, inst, vote_data);
+
+ inst->power.bus_bw = vote_data->bus_bw;
+
+ return iris_set_buses(inst);
+}
+
+static int iris_set_clocks(struct iris_inst *inst)
+{
+ struct iris_inst *instance;
+ struct iris_core *core;
+ int ret = 0;
+ u64 freq;
+
+ core = inst->core;
+
+ mutex_lock(&core->lock);
+
+ freq = 0;
+ list_for_each_entry(instance, &core->instances, list) {
+ if (!instance->max_input_data_size)
+ continue;
+
+ freq += instance->power.min_freq;
+ }
+
+ core->power.clk_freq = freq;
+
+ ret = opp_set_rate(core, freq);
+
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+static int iris_scale_clocks(struct iris_inst *inst)
+{
+ struct iris_buffer *vbuf;
+ struct iris_core *core;
+ u32 data_size = 0;
+
+ core = inst->core;
+
+ list_for_each_entry(vbuf, &inst->buffers.input.list, list)
+ data_size = max(data_size, vbuf->data_size);
+
+ inst->max_input_data_size = data_size;
+
+ inst->max_rate = inst->cap[QUEUED_RATE].value >> 16;
+
+ if (!inst->max_input_data_size)
+ return 0;
+
+ inst->power.min_freq = call_session_op(core, calc_freq, inst,
+ inst->max_input_data_size);
+ iris_set_clocks(inst);
+
+ return 0;
+}
+
+int iris_scale_power(struct iris_inst *inst)
+{
+ iris_scale_clocks(inst);
+ iris_vote_buses(inst);
+
+ return 0;
+}
+
+int iris_update_input_rate(struct iris_inst *inst, u64 time_us)
+{
+ struct iris_input_timer *prev_timer = NULL;
+ struct iris_input_timer *input_timer;
+ u64 input_timer_sum_us = 0;
+ u64 counter = 0;
+
+ input_timer = kzalloc(sizeof(*input_timer), GFP_KERNEL);
+ if (!input_timer)
+ return -ENOMEM;
+
+ input_timer->time_us = time_us;
+ INIT_LIST_HEAD(&input_timer->list);
+ list_add_tail(&input_timer->list, &inst->input_timer_list);
+ list_for_each_entry(input_timer, &inst->input_timer_list, list) {
+ if (prev_timer) {
+ input_timer_sum_us += input_timer->time_us - prev_timer->time_us;
+ counter++;
+ }
+ prev_timer = input_timer;
+ }
+
+ if (input_timer_sum_us && counter >= INPUT_TIMER_LIST_SIZE)
+ inst->cap[QUEUED_RATE].value =
+ (s32)(DIV64_U64_ROUND_CLOSEST(counter * 1000000,
+ input_timer_sum_us) << 16);
+
+ if (counter >= INPUT_TIMER_LIST_SIZE) {
+ input_timer = list_first_entry(&inst->input_timer_list,
+ struct iris_input_timer, list);
+ list_del_init(&input_timer->list);
+ kfree(input_timer);
+ }
+
+ return 0;
+}
+
+int iris_flush_input_timer(struct iris_inst *inst)
+{
+ struct iris_input_timer *input_timer, *dummy_timer;
+
+ list_for_each_entry_safe(input_timer, dummy_timer, &inst->input_timer_list, list) {
+ list_del_init(&input_timer->list);
+ kfree(input_timer);
+ }
+
+ return 0;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_power.h b/drivers/media/platform/qcom/vcodec/iris/iris_power.h
new file mode 100644
index 0000000..28dbd5f
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_power.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_POWER_H_
+#define _IRIS_POWER_H_
+
+#include "iris_instance.h"
+
+int iris_scale_power(struct iris_inst *inst);
+int iris_update_input_rate(struct iris_inst *inst, u64 time_us);
+int iris_flush_input_timer(struct iris_inst *inst);
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
index bf484a3..2571e27 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -6,14 +6,15 @@
#include <linux/module.h>
#include <linux/of_device.h>
#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>

#include "../hfi_queue.h"
#include "iris_core.h"
+#include "iris_ctrls.h"
#include "iris_helpers.h"
#include "iris_hfi.h"
-#include "resources.h"
#include "iris_vidc.h"
-#include "iris_ctrls.h"
+#include "resources.h"

static int init_iris_isr(struct iris_core *core)
{
@@ -70,6 +71,8 @@ static void iris_remove(struct platform_device *pdev)
if (!core)
return;

+ iris_pm_get(core);
+
iris_core_deinit(core);
hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr,
&core->command_queue, &core->message_queue,
@@ -79,7 +82,10 @@ static void iris_remove(struct platform_device *pdev)

v4l2_device_unregister(&core->v4l2_dev);

+ iris_pm_put(core, false);
+
mutex_destroy(&core->lock);
+ mutex_destroy(&core->pm_lock);
core->state = IRIS_CORE_DEINIT;
}

@@ -97,6 +103,7 @@ static int iris_probe(struct platform_device *pdev)

core->state = IRIS_CORE_DEINIT;
mutex_init(&core->lock);
+ mutex_init(&core->pm_lock);

core->packet_size = IFACEQ_CORE_PKT_SIZE;
core->packet = devm_kzalloc(core->dev, core->packet_size, GFP_KERNEL);
@@ -117,58 +124,66 @@ static int iris_probe(struct platform_device *pdev)
if (core->irq < 0)
return core->irq;

+ pm_runtime_set_autosuspend_delay(core->dev, AUTOSUSPEND_DELAY_VALUE);
+ pm_runtime_use_autosuspend(core->dev);
+ ret = devm_pm_runtime_enable(core->dev);
+ if (ret) {
+ dev_err(core->dev, "failed to enable runtime pm\n");
+ goto err_runtime_disable;
+ }
+
ret = init_iris_isr(core);
if (ret) {
dev_err_probe(core->dev, ret,
"%s: Failed to init isr with %d\n", __func__, ret);
- return ret;
+ goto err_runtime_disable;
}

ret = init_platform(core);
if (ret) {
dev_err_probe(core->dev, ret,
"%s: init platform failed with %d\n", __func__, ret);
- return ret;
+ goto err_runtime_disable;
}

ret = init_vpu(core);
if (ret) {
dev_err_probe(core->dev, ret,
"%s: init vpu failed with %d\n", __func__, ret);
- return ret;
+ goto err_runtime_disable;
}

ret = init_ops(core);
if (ret) {
dev_err_probe(core->dev, ret,
"%s: init ops failed with %d\n", __func__, ret);
- return ret;
+ goto err_runtime_disable;
}

ret = init_resources(core);
if (ret) {
dev_err_probe(core->dev, ret,
"%s: init resource failed with %d\n", __func__, ret);
- return ret;
+ goto err_runtime_disable;
}

ret = iris_init_core_caps(core);
if (ret) {
dev_err_probe(core->dev, ret,
"%s: init core caps failed with %d\n", __func__, ret);
- return ret;
+ goto err_runtime_disable;
}

ret = iris_init_instance_caps(core);
if (ret) {
dev_err_probe(core->dev, ret,
"%s: init inst caps failed with %d\n", __func__, ret);
- return ret;
+ goto err_runtime_disable;
}

ret = v4l2_device_register(dev, &core->v4l2_dev);
if (ret)
- return ret;
+ goto err_runtime_disable;

ret = iris_register_video_device(core);
if (ret)
@@ -194,14 +209,29 @@ static int iris_probe(struct platform_device *pdev)
goto err_vdev_unreg;
}

+ ret = iris_pm_get(core);
+ if (ret) {
+ dev_err_probe(core->dev, ret, "%s: failed to get runtime pm\n", __func__);
+ goto err_queue_deinit;
+ }
+
ret = iris_core_init(core);
if (ret) {
dev_err_probe(core->dev, ret, "%s: core init failed\n", __func__);
goto err_queue_deinit;
}

+ ret = iris_pm_put(core, false);
+ if (ret) {
+ pm_runtime_get_noresume(core->dev);
+ dev_err_probe(core->dev, ret, "%s: failed to put runtime pm\n", __func__);
+ goto err_core_deinit;
+ }
+
return ret;

+err_core_deinit:
+ iris_core_deinit(core);
err_queue_deinit:
hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr,
&core->command_queue, &core->message_queue,
@@ -210,10 +240,77 @@ static int iris_probe(struct platform_device *pdev)
video_unregister_device(core->vdev_dec);
err_v4l2_unreg:
v4l2_device_unregister(&core->v4l2_dev);
+err_runtime_disable:
+ pm_runtime_put_noidle(core->dev);
+ pm_runtime_set_suspended(core->dev);

return ret;
}

+static int iris_pm_suspend(struct device *dev)
+{
+ struct iris_core *core;
+ int ret;
+
+ if (!dev || !dev->driver)
+ return 0;
+
+ core = dev_get_drvdata(dev);
+
+ mutex_lock(&core->lock);
+ if (!core_in_valid_state(core)) {
+ ret = 0;
+ goto unlock;
+ }
+
+ if (!core->power_enabled) {
+ ret = 0;
+ goto unlock;
+ }
+
+ ret = iris_hfi_pm_suspend(core);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+static int iris_pm_resume(struct device *dev)
+{
+ struct iris_core *core;
+ int ret;
+
+ if (!dev || !dev->driver)
+ return 0;
+
+ core = dev_get_drvdata(dev);
+
+ mutex_lock(&core->lock);
+ if (!core_in_valid_state(core)) {
+ ret = 0;
+ goto unlock;
+ }
+
+ if (core->power_enabled) {
+ ret = 0;
+ goto unlock;
+ }
+
+ ret = iris_hfi_pm_resume(core);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+static const struct dev_pm_ops iris_pm_ops = {
+ SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
+ pm_runtime_force_resume)
+ SET_RUNTIME_PM_OPS(iris_pm_suspend, iris_pm_resume, NULL)
+};
+
static const struct of_device_id iris_dt_match[] = {
{
.compatible = "qcom,sm8550-iris",
@@ -229,6 +326,7 @@ static struct platform_driver qcom_iris_driver = {
.driver = {
.name = "qcom-iris",
.of_match_table = iris_dt_match,
+ .pm = &iris_pm_ops,
},
};

diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
index f34434fc..c0878f1 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
@@ -10,6 +10,7 @@
#include "iris_helpers.h"
#include "iris_hfi.h"
#include "iris_instance.h"
+#include "iris_power.h"
#include "iris_vb2.h"
#include "iris_vdec.h"

@@ -111,39 +112,52 @@ int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count)
goto error;
}

+ ret = iris_pm_get(inst->core);
+ if (ret)
+ goto error;
+
if (!inst->once_per_session_set) {
inst->once_per_session_set = true;
ret = iris_hfi_session_set_codec(inst);
if (ret)
- goto error;
+ goto err_pm_get;

ret = iris_hfi_session_set_default_header(inst);
if (ret)
- goto error;
+ goto err_pm_get;

ret = iris_alloc_and_queue_session_int_bufs(inst, BUF_PERSIST);
if (ret)
- goto error;
+ goto err_pm_get;
}

+ iris_scale_power(inst);
+
if (q->type == INPUT_MPLANE)
ret = vdec_streamon_input(inst);
else if (q->type == OUTPUT_MPLANE)
ret = vdec_streamon_output(inst);
if (ret)
- goto error;
+ goto err_pm_get;

buf_type = v4l2_type_to_driver(q->type);
if (!buf_type) {
ret = -EINVAL;
- goto error;
+ goto err_pm_get;
}

ret = queue_deferred_buffers(inst, buf_type);
if (ret)
+ goto err_pm_get;
+
+ ret = iris_pm_put(inst->core, true);
+ if (ret)
goto error;

return ret;
+
+err_pm_get:
+ iris_pm_put(inst->core, false);
error:
iris_inst_change_state(inst, IRIS_INST_ERROR);

@@ -165,6 +179,10 @@ void iris_vb2_stop_streaming(struct vb2_queue *q)
if (q->type != INPUT_MPLANE && q->type != OUTPUT_MPLANE)
goto error;

+ ret = iris_pm_get_put(inst->core);
+ if (ret)
+ goto error;
+
if (q->type == INPUT_MPLANE)
ret = session_streamoff(inst, INPUT_MPLANE);
else if (q->type == OUTPUT_MPLANE)
@@ -181,6 +199,8 @@ void iris_vb2_stop_streaming(struct vb2_queue *q)

void iris_vb2_buf_queue(struct vb2_buffer *vb2)
{
+ u64 ktime_ns = ktime_get_ns();
+ struct iris_core *core;
struct iris_inst *inst;
int ret;

@@ -188,11 +208,23 @@ void iris_vb2_buf_queue(struct vb2_buffer *vb2)
if (!inst || !inst->core)
return;

+ core = inst->core;
+
if (!vb2->planes[0].bytesused && vb2->type == INPUT_MPLANE) {
ret = -EINVAL;
goto exit;
}

+ if (vb2->type == INPUT_MPLANE) {
+ ret = iris_update_input_rate(inst, div_u64(ktime_ns, 1000));
+ if (ret)
+ goto exit;
+ }
+
+ ret = iris_pm_get_put(core);
+ if (ret)
+ goto exit;
+
ret = vdec_qbuf(inst, vb2);

exit:
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index 7884ba6..371615e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -12,6 +12,7 @@
#include "iris_helpers.h"
#include "iris_hfi.h"
#include "iris_hfi_packet.h"
+#include "iris_power.h"
#include "iris_vdec.h"

#define UNSPECIFIED_COLOR_FORMAT 5
@@ -1069,6 +1070,8 @@ static int process_streamon_input(struct iris_inst *inst)
enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE;
int ret;

+ iris_scale_power(inst);
+
ret = iris_hfi_start(inst, INPUT_MPLANE);
if (ret)
return ret;
@@ -1150,6 +1153,8 @@ static int process_streamon_output(struct iris_inst *inst)
bool drain_pending = false;
int ret;

+ iris_scale_power(inst);
+
if (inst->sub_state & IRIS_INST_SUB_DRC &&
inst->sub_state & IRIS_INST_SUB_DRC_LAST)
clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST;
@@ -1287,6 +1292,8 @@ int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2)
return 0;
}

+ iris_scale_power(inst);
+
ret = queue_buffer(inst, buf);
if (ret)
return ret;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
index d37ef04..437d6b4 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
@@ -9,12 +9,13 @@

#include "iris_buffer.h"
#include "iris_common.h"
+#include "iris_ctrls.h"
#include "iris_helpers.h"
#include "iris_hfi.h"
#include "iris_instance.h"
+#include "iris_power.h"
#include "iris_vdec.h"
#include "iris_vidc.h"
-#include "iris_ctrls.h"
#include "iris_vb2.h"

#define VIDC_DRV_NAME "iris_driver"
@@ -162,17 +163,23 @@ int vidc_open(struct file *filp)
int i = 0;
int ret;

- ret = iris_core_init(core);
+ ret = iris_pm_get(core);
if (ret)
return ret;

+ ret = iris_core_init(core);
+ if (ret)
+ goto fail_pm_put;
+
ret = iris_core_init_wait(core);
if (ret)
- return ret;
+ goto fail_pm_put;

inst = kzalloc(sizeof(*inst), GFP_KERNEL);
- if (!inst)
- return -ENOMEM;
+ if (!inst) {
+ ret = -ENOMEM;
+ goto fail_pm_put;
+ }

inst->core = core;
inst->session_id = hash32_ptr(inst);
@@ -198,6 +205,7 @@ int vidc_open(struct file *filp)
INIT_LIST_HEAD(&inst->buffers.persist.list);
INIT_LIST_HEAD(&inst->buffers.vpss.list);
INIT_LIST_HEAD(&inst->caps_list);
+ INIT_LIST_HEAD(&inst->input_timer_list);
for (i = 0; i < MAX_SIGNAL; i++)
init_completion(&inst->completions[i]);

@@ -211,11 +219,16 @@ int vidc_open(struct file *filp)
if (ret)
goto fail_inst_deinit;

+ iris_scale_power(inst);
+
ret = iris_hfi_session_open(inst);
if (ret) {
dev_err(core->dev, "%s: session open failed\n", __func__);
goto fail_core_deinit;
}
+
+ iris_pm_put(core, true);
+
filp->private_data = &inst->fh;

return 0;
@@ -233,6 +246,8 @@ int vidc_open(struct file *filp)
mutex_destroy(&inst->ctx_q_lock);
mutex_destroy(&inst->lock);
kfree(inst);
+fail_pm_put:
+ iris_pm_put(core, false);

return ret;
}
@@ -240,20 +255,25 @@ int vidc_open(struct file *filp)
int vidc_close(struct file *filp)
{
struct iris_inst *inst;
+ struct iris_core *core;

inst = get_vidc_inst(filp, NULL);
if (!inst)
return -EINVAL;

+ core = inst->core;
+
v4l2_ctrl_handler_free(&inst->ctrl_handler);
vdec_inst_deinit(inst);
mutex_lock(&inst->lock);
+ iris_pm_get(core);
close_session(inst);
iris_inst_change_state(inst, IRIS_INST_CLOSE);
vidc_vb2_queue_deinit(inst);
vidc_v4l2_fh_deinit(inst);
iris_destroy_buffers(inst);
vidc_remove_session(inst);
+ iris_pm_put(core, false);
mutex_unlock(&inst->lock);
mutex_destroy(&inst->ctx_q_lock);
mutex_destroy(&inst->lock);
@@ -965,11 +985,17 @@ static int vidc_dec_cmd(struct file *filp, void *fh,
goto unlock;
}

+ ret = iris_pm_get(inst->core);
+ if (ret)
+ goto unlock;
+
if (dec->cmd == V4L2_DEC_CMD_START)
ret = vdec_start_cmd(inst);
else if (dec->cmd == V4L2_DEC_CMD_STOP)
ret = vdec_stop_cmd(inst);

+ iris_pm_put(inst->core, true);
+
unlock:
mutex_unlock(&inst->lock);

diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
index 22a8f5b..81de610 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
@@ -15,6 +15,7 @@ struct iris_core;
struct iris_inst;

#define HW_RESPONSE_TIMEOUT_VALUE (1000)
+#define AUTOSUSPEND_DELAY_VALUE (HW_RESPONSE_TIMEOUT_VALUE + 500)

#define BIT_DEPTH_8 (8 << 16 | 8)
#define BIT_DEPTH_10 (10 << 16 | 10)
@@ -64,6 +65,11 @@ struct reg_preset_info {
u32 mask;
};

+struct iris_core_power {
+ u64 clk_freq;
+ u64 bus_bw;
+};
+
struct ubwc_config_data {
u32 max_channels;
u32 mal_length;
@@ -74,6 +80,18 @@ struct ubwc_config_data {
u32 bank_spreading;
};

+struct bus_vote_data {
+ u32 color_formats[2];
+ int height, width;
+ u32 fps;
+ u64 bus_bw;
+};
+
+struct iris_inst_power {
+ u64 min_freq;
+ u32 bus_bw;
+};
+
enum plat_core_cap_type {
CORE_CAP_NONE = 0,
DEC_CODECS,
diff --git a/drivers/media/platform/qcom/vcodec/iris/resources.c b/drivers/media/platform/qcom/vcodec/iris/resources.c
index 8cfdcd6..c9e3754 100644
--- a/drivers/media/platform/qcom/vcodec/iris/resources.c
+++ b/drivers/media/platform/qcom/vcodec/iris/resources.c
@@ -4,6 +4,8 @@
*/

#include <linux/clk.h>
+#include <linux/clk-provider.h>
+#include <linux/delay.h>
#include <linux/interconnect.h>
#include <linux/pm_domain.h>
#include <linux/pm_opp.h>
@@ -15,6 +17,8 @@
#include "platform_common.h"
#include "resources.h"

+#define BW_THRESHOLD 50000
+
static void iris_pd_release(void *res)
{
struct device *pd = (struct device *)res;
@@ -59,6 +63,34 @@ static int iris_opp_dl_get(struct device *dev, struct device *supplier)
return ret;
}

+int opp_set_rate(struct iris_core *core, u64 freq)
+{
+ unsigned long opp_freq = 0;
+ struct dev_pm_opp *opp;
+ int ret;
+
+ opp_freq = freq;
+
+ opp = dev_pm_opp_find_freq_ceil(core->dev, &opp_freq);
+ if (IS_ERR(opp)) {
+ opp = dev_pm_opp_find_freq_floor(core->dev, &opp_freq);
+ if (IS_ERR(opp)) {
+ dev_err(core->dev,
+ "unable to find freq %lld in opp table\n", freq);
+ return -EINVAL;
+ }
+ }
+ dev_pm_opp_put(opp);
+
+ ret = dev_pm_opp_set_rate(core->dev, opp_freq);
+ if (ret) {
+ dev_err(core->dev, "failed to set rate\n");
+ return ret;
+ }
+
+ return ret;
+}
+
static int init_bus(struct iris_core *core)
{
const struct bus_info *bus_tbl;
@@ -135,6 +167,10 @@ static int init_power_domains(struct iris_core *core)
}
}

+ ret = devm_pm_opp_set_clkname(core->dev, "vcodec_core");
+ if (ret)
+ return ret;
+
ret = devm_pm_opp_of_add_table(core->dev);
if (ret) {
dev_err(core->dev, "%s: failed to add opp table\n", __func__);
@@ -144,6 +180,56 @@ static int init_power_domains(struct iris_core *core)
return ret;
}

+int enable_power_domains(struct iris_core *core, const char *name)
+{
+ struct power_domain_info *pdinfo = NULL;
+ int ret;
+ u32 i;
+
+ ret = opp_set_rate(core, ULONG_MAX);
+ if (ret)
+ return ret;
+
+ core->pd_count = core->platform_data->pd_tbl_size;
+ for (i = 0; i < (core->pd_count - 1); i++) {
+ pdinfo = &core->power_domain_tbl[i];
+ if (strcmp(pdinfo->name, name))
+ continue;
+ ret = pm_runtime_get_sync(pdinfo->genpd_dev);
+ if (ret < 0)
+ return ret;
+ }
+
+ ret = opp_set_rate(core, ULONG_MAX);
+ if (ret)
+ return ret;
+
+ return ret;
+}
+
+int disable_power_domains(struct iris_core *core, const char *name)
+{
+ struct power_domain_info *pdinfo = NULL;
+ int ret;
+ u32 i;
+
+ ret = opp_set_rate(core, 0);
+ if (ret)
+ return ret;
+
+ core->pd_count = core->platform_data->pd_tbl_size;
+ for (i = 0; i < (core->pd_count - 1); i++) {
+ pdinfo = &core->power_domain_tbl[i];
+ if (strcmp(pdinfo->name, name))
+ continue;
+ ret = pm_runtime_put_sync(pdinfo->genpd_dev);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
static int init_clocks(struct iris_core *core)
{
const struct clock_info *clk_tbl;
@@ -204,6 +290,200 @@ static int init_reset_clocks(struct iris_core *core)
return 0;
}

+int unvote_buses(struct iris_core *core)
+{
+ struct bus_info *bus = NULL;
+ int ret = 0;
+ u32 i;
+
+ core->power.bus_bw = 0;
+ core->bus_count = core->platform_data->bus_tbl_size;
+
+ for (i = 0; i < core->bus_count; i++) {
+ bus = &core->bus_tbl[i];
+ if (!bus->icc)
+ return -EINVAL;
+
+ ret = icc_set_bw(bus->icc, 0, 0);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
+int vote_buses(struct iris_core *core, unsigned long bus_bw)
+{
+ unsigned long bw_kbps = 0, bw_prev = 0;
+ struct bus_info *bus = NULL;
+ int ret = 0;
+ u32 i;
+
+ core->bus_count = core->platform_data->bus_tbl_size;
+
+ for (i = 0; i < core->bus_count; i++) {
+ bus = &core->bus_tbl[i];
+ if (bus && bus->icc) {
+ if (!strcmp(bus->name, "iris-ddr")) {
+ bw_kbps = bus_bw;
+ bw_prev = core->power.bus_bw;
+ } else {
+ bw_kbps = bus->bw_max_kbps;
+ bw_prev = core->power.bus_bw ?
+ bw_kbps : 0;
+ }
+
+ bw_kbps = clamp_t(typeof(bw_kbps), bw_kbps,
+ bus->bw_min_kbps, bus->bw_max_kbps);
+
+ if (abs(bw_kbps - bw_prev) < BW_THRESHOLD && bw_prev)
+ continue;
+
+ ret = icc_set_bw(bus->icc, bw_kbps, 0);
+ if (ret)
+ return ret;
+
+ if (!strcmp(bus->name, "iris-ddr"))
+ core->power.bus_bw = bw_kbps;
+ }
+ }
+
+ return ret;
+}
+
+static int deassert_reset_control(struct iris_core *core)
+{
+ struct reset_info *rcinfo = NULL;
+ int ret = 0;
+ u32 i;
+
+ core->reset_count = core->platform_data->clk_rst_tbl_size;
+
+ for (i = 0; i < (core->reset_count - 1); i++) {
+ rcinfo = &core->reset_tbl[i];
+ ret = reset_control_deassert(rcinfo->rst);
+ if (ret) {
+ dev_err(core->dev, "deassert reset control failed. ret = %d\n", ret);
+ continue;
+ }
+ }
+
+ return ret;
+}
+
+static int assert_reset_control(struct iris_core *core)
+{
+ struct reset_info *rcinfo = NULL;
+ int ret = 0, cnt = 0;
+ u32 i;
+
+ core->reset_count = core->platform_data->clk_rst_tbl_size;
+
+ for (i = 0; i < (core->reset_count - 1); i++) {
+ rcinfo = &core->reset_tbl[i];
+ if (!rcinfo->rst)
+ return -EINVAL;
+
+ ret = reset_control_assert(rcinfo->rst);
+ if (ret) {
+ dev_err(core->dev, "failed to assert reset control %s, ret = %d\n",
+ rcinfo->name, ret);
+ goto deassert_reset_control;
+ }
+ cnt++;
+
+ usleep_range(1000, 1100);
+ }
+
+ return ret;
+deassert_reset_control:
+ for (i = 0; i < cnt; i++) {
+ rcinfo = &core->reset_tbl[i];
+ reset_control_deassert(rcinfo->rst);
+ }
+
+ return ret;
+}
+
+int reset_ahb2axi_bridge(struct iris_core *core)
+{
+ int ret;
+
+ ret = assert_reset_control(core);
+ if (ret)
+ return ret;
+
+ ret = deassert_reset_control(core);
+
+ return ret;
+}
+
+int disable_unprepare_clock(struct iris_core *core, const char *clk_name)
+{
+ struct clock_info *cl;
+ bool found = false;
+ u32 i;
+
+ core->clk_count = core->platform_data->clk_tbl_size;
+
+ for (i = 0; i < core->clk_count; i++) {
+ cl = &core->clock_tbl[i];
+ if (!cl->clk)
+ return -EINVAL;
+
+ if (strcmp(cl->name, clk_name))
+ continue;
+
+ found = true;
+ clk_disable_unprepare(cl->clk);
+ cl->prev = 0;
+ break;
+ }
+
+ if (!found)
+ return -EINVAL;
+
+ return 0;
+}
+
+int prepare_enable_clock(struct iris_core *core, const char *clk_name)
+{
+ struct clock_info *cl;
+ bool found = false;
+ int ret = 0;
+ u32 i;
+
+ core->clk_count = core->platform_data->clk_tbl_size;
+
+ for (i = 0; i < core->clk_count; i++) {
+ cl = &core->clock_tbl[i];
+ if (!cl->clk)
+ return -EINVAL;
+
+ if (strcmp(cl->name, clk_name))
+ continue;
+
+ found = true;
+
+ ret = clk_prepare_enable(cl->clk);
+ if (ret) {
+ dev_err(core->dev, "failed to enable clock %s\n", cl->name);
+ return ret;
+ }
+
+ if (!__clk_is_enabled(cl->clk)) {
+ clk_disable_unprepare(cl->clk);
+ return -EINVAL;
+ }
+ break;
+ }
+
+ if (!found)
+ return -EINVAL;
+
+ return ret;
+}
+
int init_resources(struct iris_core *core)
{
int ret;
diff --git a/drivers/media/platform/qcom/vcodec/iris/resources.h b/drivers/media/platform/qcom/vcodec/iris/resources.h
index d21bcc7e..09857ed 100644
--- a/drivers/media/platform/qcom/vcodec/iris/resources.h
+++ b/drivers/media/platform/qcom/vcodec/iris/resources.h
@@ -31,6 +31,14 @@ struct reset_info {
const char *name;
};

+int enable_power_domains(struct iris_core *core, const char *name);
+int disable_power_domains(struct iris_core *core, const char *name);
+int unvote_buses(struct iris_core *core);
+int vote_buses(struct iris_core *core, unsigned long bus_bw);
+int reset_ahb2axi_bridge(struct iris_core *core);
+int opp_set_rate(struct iris_core *core, u64 freq);
+int disable_unprepare_clock(struct iris_core *core, const char *clk_name);
+int prepare_enable_clock(struct iris_core *core, const char *clk_name);
int init_resources(struct iris_core *core);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_common.c b/drivers/media/platform/qcom/vcodec/iris/vpu_common.c
index 3282510..8ef1029 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_common.c
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_common.c
@@ -3,6 +3,8 @@
* Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
*/

+#include <linux/iopoll.h>
+
#include "vpu_iris3.h"
#include "iris_core.h"
#include "iris_helpers.h"
@@ -17,6 +19,9 @@ int write_register(struct iris_core *core, u32 reg, u32 value)
if (ret)
return ret;

+ if (!core->power_enabled)
+ return -EINVAL;
+
base_addr = core->reg_base;
base_addr += reg;
writel_relaxed(value, base_addr);
@@ -27,10 +32,46 @@ int write_register(struct iris_core *core, u32 reg, u32 value)
return ret;
}

+int write_register_masked(struct iris_core *core, u32 reg, u32 value, u32 mask)
+{
+ void __iomem *base_addr;
+ u32 prev_val, new_val;
+ int ret;
+
+ ret = check_core_lock(core);
+ if (ret)
+ return ret;
+
+ if (!core->power_enabled)
+ return -EINVAL;
+
+ base_addr = core->reg_base;
+ base_addr += reg;
+
+ prev_val = readl_relaxed(base_addr);
+ /*
+ * Memory barrier to ensure register read is correct
+ */
+ rmb();
+
+ new_val = (prev_val & ~mask) | (value & mask);
+
+ writel_relaxed(new_val, base_addr);
+ /*
+ * Memory barrier to make sure value is written into the register.
+ */
+ wmb();
+
+ return ret;
+}
+
int read_register(struct iris_core *core, u32 reg, u32 *value)
{
void __iomem *base_addr;

+ if (!core->power_enabled)
+ return -EINVAL;
+
base_addr = core->reg_base;

*value = readl_relaxed(base_addr + reg);
@@ -41,6 +82,52 @@ int read_register(struct iris_core *core, u32 reg, u32 *value)
return 0;
}

+int read_register_with_poll_timeout(struct iris_core *core, u32 reg,
+ u32 mask, u32 exp_val, u32 sleep_us,
+ u32 timeout_us)
+{
+ void __iomem *base_addr;
+ u32 val = 0;
+ int ret;
+
+ if (!core->power_enabled)
+ return -EINVAL;
+
+ base_addr = core->reg_base;
+
+ ret = readl_relaxed_poll_timeout(base_addr + reg, val, ((val & mask) == exp_val),
+ sleep_us, timeout_us);
+ /*
+ * Memory barrier to make sure value is read correctly from the
+ * register.
+ */
+ rmb();
+
+ return ret;
+}
+
+int set_preset_registers(struct iris_core *core)
+{
+ const struct reg_preset_info *reg_prst;
+ unsigned int prst_count;
+ int cnt, ret = 0;
+
+ reg_prst = core->platform_data->reg_prst_tbl;
+ prst_count = core->platform_data->reg_prst_tbl_size;
+
+ if (!reg_prst || !prst_count)
+ return 0;
+
+ for (cnt = 0; cnt < prst_count; cnt++) {
+ ret = write_register_masked(core, reg_prst[cnt].reg,
+ reg_prst[cnt].value, reg_prst[cnt].mask);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
static const struct compat_handle compat_handle[] = {
{
.compat = "qcom,sm8550-iris",
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h
index 7fba8c2..aef5606 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h
@@ -24,6 +24,9 @@ struct vpu_ops {
int (*raise_interrupt)(struct iris_core *core);
int (*clear_interrupt)(struct iris_core *core);
int (*watchdog)(struct iris_core *core, u32 intr_status);
+ int (*power_on)(struct iris_core *core);
+ int (*power_off)(struct iris_core *core);
+ int (*prepare_pc)(struct iris_core *core);
};

#define call_session_op(c, op, ...) \
@@ -32,11 +35,18 @@ struct vpu_ops {

struct vpu_session_ops {
int (*int_buf_size)(struct iris_inst *inst, enum iris_buffer_type type);
+ u64 (*calc_freq)(struct iris_inst *inst, u32 data_size);
+ int (*calc_bw)(struct iris_inst *inst, struct bus_vote_data *data);
};

int init_vpu(struct iris_core *core);

int write_register(struct iris_core *core, u32 reg, u32 value);
+int write_register_masked(struct iris_core *core, u32 reg, u32 value, u32 mask);
int read_register(struct iris_core *core, u32 reg, u32 *value);
+int read_register_with_poll_timeout(struct iris_core *core, u32 reg,
+ u32 mask, u32 exp_val, u32 sleep_us,
+ u32 timeout_us);
+int set_preset_registers(struct iris_core *core);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
index efea5aa..85ed38d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
@@ -5,73 +5,105 @@

#include <linux/delay.h>

+#include "iris_hfi.h"
#include "vpu_iris3.h"
#include "vpu_iris3_buffer.h"
+#include "vpu_iris3_power.h"

#define VIDEO_ARCH_LX 1

-#define CPU_BASE_OFFS_IRIS3 0x000A0000
-
-#define CPU_CS_BASE_OFFS_IRIS3 (CPU_BASE_OFFS_IRIS3)
-#define CPU_IC_BASE_OFFS_IRIS3 (CPU_BASE_OFFS_IRIS3)
-
-#define CPU_CS_VCICMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x24)
-#define CPU_CS_VCICMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x28)
-
-#define CPU_CS_A2HSOFTINTCLR_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x1C)
+#define VCODEC_BASE_OFFS_IRIS3 0x00000000
+#define AON_MVP_NOC_RESET 0x0001F000
+#define CPU_BASE_OFFS_IRIS3 0x000A0000
+#define AON_BASE_OFFS 0x000E0000
+#define CPU_CS_BASE_OFFS_IRIS3 (CPU_BASE_OFFS_IRIS3)
+#define CPU_IC_BASE_OFFS_IRIS3 (CPU_BASE_OFFS_IRIS3)

+#define CPU_CS_A2HSOFTINTCLR_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x1C)
+#define CPU_CS_VCICMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x24)
+#define CPU_CS_VCICMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x28)
/* HFI_CTRL_INIT */
-#define CPU_CS_SCIACMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x48)
-
+#define CPU_CS_SCIACMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x48)
/* HFI_CTRL_STATUS */
-#define CPU_CS_SCIACMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x4C)
-#define CPU_CS_SCIACMDARG0_HFI_CTRL_INIT_IDLE_MSG_BMSK_IRIS3 0x40000000
-
-#define CPU_CS_H2XSOFTINTEN_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x148)
-
-#define CPU_CS_X2RPMH_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x168)
-
-/* UC_REGION_ADDR */
-#define CPU_CS_SCIBARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x64)
-
-/* UC_REGION_ADDR */
-#define CPU_CS_SCIBARG2_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x68)
-
+#define CPU_CS_SCIACMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x4C)
/* HFI_QTBL_INFO */
-#define CPU_CS_SCIACMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x50)
-
+#define CPU_CS_SCIACMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x50)
/* HFI_QTBL_ADDR */
-#define CPU_CS_SCIACMDARG2_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x54)
-
+#define CPU_CS_SCIACMDARG2_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x54)
/* SFR_ADDR */
-#define CPU_CS_SCIBCMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x5C)
-
-#define UC_REGION_ADDR_IRIS3 CPU_CS_SCIBARG1_IRIS3
-#define UC_REGION_SIZE_IRIS3 CPU_CS_SCIBARG2_IRIS3
-
-#define QTBL_INFO_IRIS3 CPU_CS_SCIACMDARG1_IRIS3
-#define QTBL_ADDR_IRIS3 CPU_CS_SCIACMDARG2_IRIS3
-
-#define SFR_ADDR_IRIS3 CPU_CS_SCIBCMD_IRIS3
+#define CPU_CS_SCIBCMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x5C)
+#define CPU_CS_SCIBCMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x60)
+/* UC_REGION_ADDR */
+#define CPU_CS_SCIBARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x64)
+/* UC_REGION_ADDR */
+#define CPU_CS_SCIBARG2_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x68)
+#define CPU_CS_H2XSOFTINTEN_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x148)
+#define CPU_CS_AHB_BRIDGE_SYNC_RESET (CPU_CS_BASE_OFFS_IRIS3 + 0x160)
+#define CPU_CS_X2RPMH_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x168)

-#define CTRL_INIT_IRIS3 CPU_CS_SCIACMD_IRIS3
+#define CPU_IC_SOFTINT_IRIS3 (CPU_IC_BASE_OFFS_IRIS3 + 0x150)
+#define CPU_IC_SOFTINT_H2A_SHFT_IRIS3 0x0

-#define CTRL_STATUS_IRIS3 CPU_CS_SCIACMDARG0_IRIS3
-#define CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3 0xfe
-#define CTRL_ERROR_STATUS__M_IRIS3 \
- CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3
-#define CTRL_INIT_IDLE_MSG_BMSK_IRIS3 \
- CPU_CS_SCIACMDARG0_HFI_CTRL_INIT_IDLE_MSG_BMSK_IRIS3
+#define CPU_CS_SCIACMDARG0_HFI_CTRL_INIT_IDLE_MSG_BMSK_IRIS3 0x40000000
+#define CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3 0xfe
+#define CPU_CS_SCIACMDARG0_HFI_CTRL_PC_READY_IRIS3 0x100
+
+#define AON_WRAPPER_MVP_NOC_RESET_REQ (AON_MVP_NOC_RESET + 0x000)
+#define AON_WRAPPER_MVP_NOC_RESET_ACK (AON_MVP_NOC_RESET + 0x004)
+
+#define WRAPPER_BASE_OFFS_IRIS3 0x000B0000
+#define WRAPPER_INTR_STATUS_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x0C)
+#define WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3 0x8
+#define WRAPPER_INTR_STATUS_A2H_BMSK_IRIS3 0x4
+
+#define WRAPPER_INTR_MASK_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x10)
+#define WRAPPER_INTR_MASK_A2HWD_BMSK_IRIS3 0x8
+#define WRAPPER_INTR_MASK_A2HCPU_BMSK_IRIS3 0x4
+
+#define WRAPPER_DEBUG_BRIDGE_LPI_CONTROL_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x54)
+#define WRAPPER_DEBUG_BRIDGE_LPI_STATUS_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x58)
+#define WRAPPER_IRIS_CPU_NOC_LPI_CONTROL (WRAPPER_BASE_OFFS_IRIS3 + 0x5C)
+#define WRAPPER_IRIS_CPU_NOC_LPI_STATUS (WRAPPER_BASE_OFFS_IRIS3 + 0x60)
+#define WRAPPER_CORE_POWER_STATUS (WRAPPER_BASE_OFFS_IRIS3 + 0x80)
+#define WRAPPER_CORE_CLOCK_CONFIG_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x88)
+
+#define WRAPPER_TZ_BASE_OFFS 0x000C0000
+#define WRAPPER_TZ_CPU_STATUS (WRAPPER_TZ_BASE_OFFS + 0x10)
+#define WRAPPER_TZ_CTL_AXI_CLOCK_CONFIG (WRAPPER_TZ_BASE_OFFS + 0x14)
+#define WRAPPER_TZ_QNS4PDXFIFO_RESET (WRAPPER_TZ_BASE_OFFS + 0x18)
+
+#define CTRL_INIT_IRIS3 CPU_CS_SCIACMD_IRIS3
+#define CTRL_STATUS_IRIS3 CPU_CS_SCIACMDARG0_IRIS3
+#define CTRL_ERROR_STATUS__M_IRIS3 CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3
+#define CTRL_INIT_IDLE_MSG_BMSK_IRIS3 CPU_CS_SCIACMDARG0_HFI_CTRL_INIT_IDLE_MSG_BMSK_IRIS3
+#define CTRL_STATUS_PC_READY_IRIS3 CPU_CS_SCIACMDARG0_HFI_CTRL_PC_READY_IRIS3
+
+#define QTBL_INFO_IRIS3 CPU_CS_SCIACMDARG1_IRIS3
+#define QTBL_ADDR_IRIS3 CPU_CS_SCIACMDARG2_IRIS3
+#define SFR_ADDR_IRIS3 CPU_CS_SCIBCMD_IRIS3
+#define UC_REGION_ADDR_IRIS3 CPU_CS_SCIBARG1_IRIS3
+#define UC_REGION_SIZE_IRIS3 CPU_CS_SCIBARG2_IRIS3
+
+#define AON_WRAPPER_MVP_NOC_LPI_CONTROL (AON_BASE_OFFS)
+#define AON_WRAPPER_MVP_NOC_LPI_STATUS (AON_BASE_OFFS + 0x4)
+
+#define VCODEC_SS_IDLE_STATUSN (VCODEC_BASE_OFFS_IRIS3 + 0x70)
+
+static int interrupt_init_iris3(struct iris_core *core)
+{
+ u32 mask_val;
+ int ret;

-#define WRAPPER_BASE_OFFS_IRIS3 0x000B0000
-#define WRAPPER_INTR_STATUS_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x0C)
-#define WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3 0x8
-#define WRAPPER_INTR_STATUS_A2H_BMSK_IRIS3 0x4
+ ret = read_register(core, WRAPPER_INTR_MASK_IRIS3, &mask_val);
+ if (ret)
+ return ret;

-#define CPU_IC_SOFTINT_IRIS3 (CPU_IC_BASE_OFFS_IRIS3 + 0x150)
-#define CPU_IC_SOFTINT_H2A_SHFT_IRIS3 0x0
+ mask_val &= ~(WRAPPER_INTR_MASK_A2HWD_BMSK_IRIS3 |
+ WRAPPER_INTR_MASK_A2HCPU_BMSK_IRIS3);
+ ret = write_register(core, WRAPPER_INTR_MASK_IRIS3, mask_val);

-#define WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3 0x8
+ return ret;
+}

static int setup_ucregion_memory_map_iris3(struct iris_core *core)
{
@@ -196,15 +228,335 @@ static int watchdog_iris3(struct iris_core *core, u32 intr_status)
return 0;
}

+static bool is_iris3_hw_power_collapsed(struct iris_core *core)
+{
+ u32 value = 0, pwr_status = 0;
+ int ret;
+
+ ret = read_register(core, WRAPPER_CORE_POWER_STATUS, &value);
+ if (ret)
+ return false;
+
+ pwr_status = value & BIT(1);
+
+ return pwr_status ? false : true;
+}
+
+static int power_off_iris3_hardware(struct iris_core *core)
+{
+ u32 value = 0;
+ int ret, i;
+
+ if (is_iris3_hw_power_collapsed(core))
+ goto disable_power;
+
+ dev_err(core->dev, "Video hw is power ON\n");
+
+ ret = read_register(core, WRAPPER_CORE_CLOCK_CONFIG_IRIS3, &value);
+ if (ret)
+ goto disable_power;
+
+ if (value) {
+ ret = write_register(core, WRAPPER_CORE_CLOCK_CONFIG_IRIS3, 0);
+ if (ret)
+ goto disable_power;
+ }
+
+ for (i = 0; i < core->cap[NUM_VPP_PIPE].value; i++) {
+ ret = read_register_with_poll_timeout(core, VCODEC_SS_IDLE_STATUSN + 4 * i,
+ 0x400000, 0x400000, 2000, 20000);
+ }
+
+ ret = write_register(core, AON_WRAPPER_MVP_NOC_RESET_REQ, 0x3);
+ if (ret)
+ goto disable_power;
+
+ ret = read_register_with_poll_timeout(core, AON_WRAPPER_MVP_NOC_RESET_ACK,
+ 0x3, 0x3, 200, 2000);
+ ret = write_register(core, AON_WRAPPER_MVP_NOC_RESET_REQ, 0x0);
+ if (ret)
+ goto disable_power;
+
+ ret = read_register_with_poll_timeout(core, AON_WRAPPER_MVP_NOC_RESET_ACK,
+ 0x3, 0x0, 200, 2000);
+
+ ret = write_register(core, CPU_CS_AHB_BRIDGE_SYNC_RESET, 0x3);
+ if (ret)
+ goto disable_power;
+ ret = write_register(core, CPU_CS_AHB_BRIDGE_SYNC_RESET, 0x2);
+ if (ret)
+ goto disable_power;
+ ret = write_register(core, CPU_CS_AHB_BRIDGE_SYNC_RESET, 0x0);
+ if (ret)
+ goto disable_power;
+
+disable_power:
+ ret = disable_power_domains(core, "vcodec");
+ if (ret) {
+ dev_err(core->dev, "disable power domain vcodec failed\n");
+ ret = 0;
+ }
+
+ disable_unprepare_clock(core, "vcodec_core");
+ if (ret) {
+ dev_err(core->dev, "disable unprepare vcodec_core failed\n");
+ ret = 0;
+ }
+
+ return ret;
+}
+
+static int power_off_iris3_controller(struct iris_core *core)
+{
+ int ret;
+
+ ret = write_register(core, CPU_CS_X2RPMH_IRIS3, 0x3);
+ if (ret)
+ goto disable_power;
+
+ ret = write_register_masked(core, AON_WRAPPER_MVP_NOC_LPI_CONTROL,
+ 0x1, BIT(0));
+ if (ret)
+ goto disable_power;
+
+ ret = read_register_with_poll_timeout(core, AON_WRAPPER_MVP_NOC_LPI_STATUS,
+ 0x1, 0x1, 200, 2000);
+
+ ret = write_register_masked(core, WRAPPER_IRIS_CPU_NOC_LPI_CONTROL,
+ 0x1, BIT(0));
+ if (ret)
+ goto disable_power;
+
+ ret = read_register_with_poll_timeout(core, WRAPPER_IRIS_CPU_NOC_LPI_STATUS,
+ 0x1, 0x1, 200, 2000);
+
+ ret = write_register(core, WRAPPER_DEBUG_BRIDGE_LPI_CONTROL_IRIS3, 0x0);
+ if (ret)
+ goto disable_power;
+
+ ret = read_register_with_poll_timeout(core, WRAPPER_DEBUG_BRIDGE_LPI_STATUS_IRIS3,
+ 0xffffffff, 0x0, 200, 2000);
+
+ ret = write_register(core, WRAPPER_TZ_CTL_AXI_CLOCK_CONFIG, 0x3);
+ if (ret)
+ goto disable_power;
+
+ ret = write_register(core, WRAPPER_TZ_QNS4PDXFIFO_RESET, 0x1);
+ if (ret)
+ goto disable_power;
+
+ ret = write_register(core, WRAPPER_TZ_QNS4PDXFIFO_RESET, 0x0);
+ if (ret)
+ goto disable_power;
+
+ ret = write_register(core, WRAPPER_TZ_CTL_AXI_CLOCK_CONFIG, 0x0);
+ if (ret)
+ goto disable_power;
+
+disable_power:
+ disable_unprepare_clock(core, "core_clk");
+ if (ret) {
+ dev_err(core->dev, "disable unprepare core_clk failed\n");
+ ret = 0;
+ }
+
+ /* power down process */
+ ret = disable_power_domains(core, "iris-ctl");
+ if (ret) {
+ dev_err(core->dev, "disable power domain iris-ctl failed\n");
+ ret = 0;
+ }
+
+ return ret;
+}
+
+static int power_off_iris3(struct iris_core *core)
+{
+ if (!core->power_enabled)
+ return 0;
+
+ opp_set_rate(core, 0);
+ power_off_iris3_hardware(core);
+ power_off_iris3_controller(core);
+ unvote_buses(core);
+
+ if (!call_vpu_op(core, watchdog, core, core->intr_status))
+ disable_irq_nosync(core->irq);
+
+ core->power_enabled = false;
+
+ return 0;
+}
+
+static int power_on_iris3_controller(struct iris_core *core)
+{
+ int ret;
+
+ ret = enable_power_domains(core, "iris-ctl");
+ if (ret)
+ return ret;
+
+ ret = reset_ahb2axi_bridge(core);
+ if (ret)
+ goto err_disable_power;
+
+ ret = prepare_enable_clock(core, "gcc_video_axi0");
+ if (ret)
+ goto err_disable_power;
+
+ ret = prepare_enable_clock(core, "core_clk");
+ if (ret)
+ goto err_disable_clock;
+
+ return ret;
+
+err_disable_clock:
+ disable_unprepare_clock(core, "gcc_video_axi0");
+err_disable_power:
+ disable_power_domains(core, "iris-ctl");
+
+ return ret;
+}
+
+static int power_on_iris3_hardware(struct iris_core *core)
+{
+ int ret;
+
+ ret = enable_power_domains(core, "vcodec");
+ if (ret)
+ return ret;
+
+ ret = prepare_enable_clock(core, "vcodec_core");
+ if (ret)
+ goto err_disable_power;
+
+ return ret;
+
+err_disable_power:
+ disable_power_domains(core, "vcodec");
+
+ return ret;
+}
+
+static int power_on_iris3(struct iris_core *core)
+{
+ u32 freq = 0;
+ int ret;
+
+ if (core->power_enabled)
+ return 0;
+
+ if (!core_in_valid_state(core))
+ return -EINVAL;
+
+ ret = vote_buses(core, INT_MAX);
+ if (ret)
+ goto err;
+
+ ret = power_on_iris3_controller(core);
+ if (ret)
+ goto err_unvote_bus;
+
+ ret = power_on_iris3_hardware(core);
+ if (ret)
+ goto err_power_off_ctrl;
+
+ core->power_enabled = true;
+
+ freq = core->power.clk_freq ? core->power.clk_freq :
+ (u32)ULONG_MAX;
+
+ opp_set_rate(core, freq);
+
+ set_preset_registers(core);
+
+ interrupt_init_iris3(core);
+ core->intr_status = 0;
+ enable_irq(core->irq);
+
+ return ret;
+
+err_power_off_ctrl:
+ power_off_iris3_controller(core);
+err_unvote_bus:
+ unvote_buses(core);
+err:
+ core->power_enabled = false;
+
+ return ret;
+}
+
+static int prepare_pc_iris3(struct iris_core *core)
+{
+ u32 wfi_status = 0, idle_status = 0, pc_ready = 0;
+ u32 ctrl_status = 0;
+ int ret;
+
+ ret = read_register(core, CTRL_STATUS_IRIS3, &ctrl_status);
+ if (ret)
+ return ret;
+
+ pc_ready = ctrl_status & CTRL_STATUS_PC_READY_IRIS3;
+ idle_status = ctrl_status & BIT(30);
+
+ if (pc_ready)
+ return 0;
+
+ ret = read_register(core, WRAPPER_TZ_CPU_STATUS, &wfi_status);
+ if (ret)
+ return ret;
+
+ wfi_status &= BIT(0);
+ if (!wfi_status || !idle_status)
+ goto skip_power_off;
+
+ ret = prepare_pc(core);
+ if (ret)
+ goto skip_power_off;
+
+ ret = read_register_with_poll_timeout(core, CTRL_STATUS_IRIS3,
+ CTRL_STATUS_PC_READY_IRIS3,
+ CTRL_STATUS_PC_READY_IRIS3, 250, 2500);
+ if (ret)
+ goto skip_power_off;
+
+ ret = read_register_with_poll_timeout(core, WRAPPER_TZ_CPU_STATUS,
+ BIT(0), 0x1, 250, 2500);
+ if (ret)
+ goto skip_power_off;
+
+ return ret;
+
+skip_power_off:
+ ret = read_register(core, CTRL_STATUS_IRIS3, &ctrl_status);
+ if (ret)
+ return ret;
+
+ ret = read_register(core, WRAPPER_TZ_CPU_STATUS, &wfi_status);
+ if (ret)
+ return ret;
+
+ wfi_status &= BIT(0);
+ dev_err(core->dev, "Skip PC, wfi=%#x, idle=%#x, pcr=%#x, ctrl=%#x)\n",
+ wfi_status, idle_status, pc_ready, ctrl_status);
+
+ return -EAGAIN;
+}
+
static const struct vpu_ops iris3_ops = {
.boot_firmware = boot_firmware_iris3,
.raise_interrupt = raise_interrupt_iris3,
.clear_interrupt = clear_interrupt_iris3,
.watchdog = watchdog_iris3,
+ .power_on = power_on_iris3,
+ .power_off = power_off_iris3,
+ .prepare_pc = prepare_pc_iris3,
};

static const struct vpu_session_ops iris3_session_ops = {
.int_buf_size = iris_int_buf_size_iris3,
+ .calc_freq = iris_calc_freq_iris3,
+ .calc_bw = iris_calc_bw_iris3,
};

int init_iris3(struct iris_core *core)
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
new file mode 100644
index 0000000..5a02f24
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
@@ -0,0 +1,97 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "iris_core.h"
+#include "iris_instance.h"
+#include "iris_helpers.h"
+#include "platform_common.h"
+#include "vpu_iris3_power.h"
+
+u64 iris_calc_freq_iris3(struct iris_inst *inst, u32 data_size)
+{
+ u64 vsp_cycles = 0, vpp_cycles = 0, fw_cycles = 0;
+ u64 fw_vpp_cycles = 0, bitrate = 0, freq = 0;
+ u32 base_cycles = 0, fps, mbpf;
+ u32 height = 0, width = 0;
+ struct v4l2_format *inp_f;
+ u32 mbs_per_second;
+
+ inp_f = inst->fmt_src;
+ width = max(inp_f->fmt.pix_mp.width, inst->crop.width);
+ height = max(inp_f->fmt.pix_mp.height, inst->crop.height);
+
+ mbpf = NUM_MBS_PER_FRAME(height, width);
+ fps = inst->max_rate;
+ mbs_per_second = mbpf * fps;
+
+ fw_cycles = fps * inst->cap[MB_CYCLES_FW].value;
+ fw_vpp_cycles = fps * inst->cap[MB_CYCLES_FW_VPP].value;
+
+ vpp_cycles = mbs_per_second * inst->cap[MB_CYCLES_VPP].value /
+ inst->cap[PIPE].value;
+ vpp_cycles += max(vpp_cycles / 20, fw_vpp_cycles);
+
+ if (inst->cap[PIPE].value > 1)
+ vpp_cycles += div_u64(vpp_cycles * 59, 1000);
+
+ base_cycles = inst->cap[MB_CYCLES_VSP].value;
+ bitrate = fps * data_size * 8;
+ vsp_cycles = bitrate;
+
+ if (inst->codec == VP9) {
+ vsp_cycles = div_u64(vsp_cycles * 170, 100);
+ } else {
+ base_cycles = 0;
+ vsp_cycles = div_u64(vsp_cycles, 2);
+ }
+ vsp_cycles = div_u64(vsp_cycles * 21, 20);
+
+ if (inst->cap[STAGE].value == STAGE_1)
+ vsp_cycles = vsp_cycles * 3;
+
+ vsp_cycles += mbs_per_second * base_cycles;
+
+ freq = max3(vpp_cycles, vsp_cycles, fw_cycles);
+
+ return freq;
+}
+
+int iris_calc_bw_iris3(struct iris_inst *inst, struct bus_vote_data *data)
+{
+ const struct bw_info *bw_tbl = NULL;
+ unsigned int num_rows = 0;
+ unsigned int i, mbs, mbps;
+ struct iris_core *core;
+
+ if (!data)
+ return 0;
+
+ core = inst->core;
+
+ mbs = (ALIGN(data->height, 16) / 16) * (ALIGN(data->width, 16) / 16);
+ mbps = mbs * data->fps;
+ if (mbps == 0)
+ return 0;
+
+ bw_tbl = core->platform_data->bw_tbl_dec;
+ num_rows = core->platform_data->bw_tbl_dec_size;
+
+ if (!bw_tbl || num_rows == 0)
+ return 0;
+
+ for (i = 0; i < num_rows; i++) {
+ if (i != 0 && mbps > bw_tbl[i].mbs_per_sec)
+ break;
+
+ if (is_10bit_colorformat(data->color_formats[0]))
+ data->bus_bw = bw_tbl[i].bw_ddr_10bit;
+ else
+ data->bus_bw = bw_tbl[i].bw_ddr;
+ }
+
+ dev_info(core->dev, "bus_bw %llu\n", data->bus_bw);
+
+ return 0;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.h b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.h
new file mode 100644
index 0000000..2a37a9e
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef __H_VPU_IRIS3_POWER_H__
+#define __H_VPU_IRIS3_POWER_H__
+
+u64 iris_calc_freq_iris3(struct iris_inst *inst, u32 data_size);
+int iris_calc_bw_iris3(struct iris_inst *inst,
+ struct bus_vote_data *vote_data);
+
+#endif
--
2.7.4

2023-12-18 11:52:10

[permalink] [raw]

Subject: [PATCH v2 14/34] media: iris: implement iris v4l2 file ops

From: Vikash Garodia <[email protected]>

Implements iris v4l2 file ops - open, close and poll.

Open:
Configure the vb2 queue and v4l2 file handler. Initialize
a video instance and add the instance to core instance list.
Open a session in video firmware via HFI_CMD_OPEN.

Close:
De-initialize the instance and remove it from core
instance list. Close the session in firmware via HFI_CMD_CLOSE.

Signed-off-by: Vikash Garodia <[email protected]>
Signed-off-by: Dikshita Agarwal <[email protected]>
---
drivers/media/platform/qcom/vcodec/iris/Makefile | 2 +
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 2 +
.../media/platform/qcom/vcodec/iris/iris_common.h | 26 ++
.../media/platform/qcom/vcodec/iris/iris_core.c | 42 ++-
.../media/platform/qcom/vcodec/iris/iris_core.h | 8 +
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 62 +++++
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 2 +
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 115 +++++++-
drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 3 +
.../platform/qcom/vcodec/iris/iris_hfi_packet.c | 28 ++
.../platform/qcom/vcodec/iris/iris_hfi_packet.h | 7 +
.../platform/qcom/vcodec/iris/iris_instance.h | 48 ++++
.../media/platform/qcom/vcodec/iris/iris_probe.c | 10 +
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 47 ++++
.../media/platform/qcom/vcodec/iris/iris_vdec.h | 14 +
.../media/platform/qcom/vcodec/iris/iris_vidc.c | 298 +++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_vidc.h | 15 ++
17 files changed, 719 insertions(+), 10 deletions(-)
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_common.h
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_instance.h
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.h

diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
index c50e3241..3c076d0 100644
--- a/drivers/media/platform/qcom/vcodec/iris/Makefile
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -3,6 +3,8 @@ iris-objs += ../hfi_queue.o ../firmware.o
iris-objs += iris_probe.o \
iris_state.o \
iris_core.o \
+ iris_vidc.o \
+ iris_vdec.o \
iris_state.o \
iris_helpers.o \
iris_hfi.o \
diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index fb383b2..423ba1a 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -9,6 +9,8 @@
#define HFI_VIDEO_ARCH_LX 0x1

#define HFI_CMD_INIT 0x01000001
+#define HFI_CMD_OPEN 0x01000003
+#define HFI_CMD_CLOSE 0x01000004

#define HFI_PROP_IMAGE_VERSION 0x03000001

diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
new file mode 100644
index 0000000..3e4dd71
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+#ifndef _IRIS_COMMON_H_
+#define _IRIS_COMMON_H_
+
+#include <media/v4l2-device.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-v4l2.h>
+
+#define INPUT_MPLANE V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE
+#define OUTPUT_MPLANE V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE
+#define DEFAULT_WIDTH 320
+#define DEFAULT_HEIGHT 240
+#define DEFAULT_BUF_SIZE 115200
+
+enum signal_session_response {
+ SIGNAL_CMD_STOP_INPUT = 0,
+ SIGNAL_CMD_STOP_OUTPUT,
+ SIGNAL_CMD_CLOSE,
+ MAX_SIGNAL,
+};
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.c b/drivers/media/platform/qcom/vcodec/iris/iris_core.c
index ba8960d..174fad9 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.c
@@ -3,12 +3,14 @@
* Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
*/

+#include <linux/delay.h>
+
#include "iris_core.h"
#include "iris_helpers.h"
#include "iris_hfi.h"
#include "iris_state.h"

-static int iris_core_deinit_locked(struct iris_core *core)
+int iris_core_deinit_locked(struct iris_core *core)
{
int ret;

@@ -68,3 +70,41 @@ int iris_core_init(struct iris_core *core)

return ret;
}
+
+int iris_core_init_wait(struct iris_core *core)
+{
+ const int interval = 10;
+ int max_tries, count = 0, ret = 0;
+
+ mutex_lock(&core->lock);
+ if (!core_in_valid_state(core)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (core->state == IRIS_CORE_INIT)
+ goto unlock;
+
+ max_tries = core->cap[HW_RESPONSE_TIMEOUT].value / interval;
+ while (count < max_tries) {
+ if (core->state != IRIS_CORE_INIT_WAIT)
+ break;
+ msleep(interval);
+ count++;
+ }
+
+ if (core->state == IRIS_CORE_INIT) {
+ ret = 0;
+ goto unlock;
+ } else {
+ iris_change_core_state(core, IRIS_CORE_ERROR);
+ iris_core_deinit_locked(core);
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index 3c8497a..b47520a 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -33,6 +33,8 @@
* @clk_count: count of iris clocks
* @reset_tbl: table of iris reset clocks
* @reset_count: count of iris reset clocks
+ * @vb2_ops: iris vb2 ops
+ * @vb2_mem_ops: iris vb2 memory ops
* @state: current state of core
* @iface_q_table: Interface queue table memory
* @command_queue: shared interface queue to send commands to firmware
@@ -49,6 +51,7 @@
* @vpu_ops: a pointer to vpu ops
* @platform_data: a structure for platform data
* @cap: an array for supported core capabilities
+ * @instances: a list_head of all instances
*/

struct iris_core {
@@ -67,6 +70,8 @@ struct iris_core {
u32 clk_count;
struct reset_info *reset_tbl;
u32 reset_count;
+ const struct vb2_ops *vb2_ops;
+ struct vb2_mem_ops *vb2_mem_ops;
enum iris_core_state state;
struct mem_desc iface_q_table;
struct iface_q_info command_queue;
@@ -83,9 +88,12 @@ struct iris_core {
const struct vpu_ops *vpu_ops;
struct platform_data *platform_data;
struct plat_core_cap cap[CORE_CAP_MAX + 1];
+ struct list_head instances;
};

int iris_core_init(struct iris_core *core);
+int iris_core_init_wait(struct iris_core *core);
int iris_core_deinit(struct iris_core *core);
+int iris_core_deinit_locked(struct iris_core *core);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index c31dfd5..872268d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -4,6 +4,8 @@
*/

#include "iris_helpers.h"
+#include "iris_hfi.h"
+#include "iris_instance.h"

int check_core_lock(struct iris_core *core)
{
@@ -32,3 +34,63 @@ int iris_init_core_caps(struct iris_core *core)

return 0;
}
+
+static int process_inst_timeout(struct iris_inst *inst)
+{
+ struct iris_inst *instance;
+ struct iris_core *core;
+ bool found = false;
+ int ret = 0;
+
+ core = inst->core;
+
+ mutex_lock(&core->lock);
+ list_for_each_entry(instance, &core->instances, list) {
+ if (instance == inst) {
+ found = true;
+ break;
+ }
+ }
+ if (!found) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ iris_change_core_state(core, IRIS_CORE_ERROR);
+
+ iris_core_deinit_locked(core);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+int close_session(struct iris_inst *inst)
+{
+ u32 hw_response_timeout_val;
+ bool wait_for_response;
+ struct iris_core *core;
+ int ret;
+
+ core = inst->core;
+ hw_response_timeout_val = core->cap[HW_RESPONSE_TIMEOUT].value;
+ wait_for_response = true;
+ ret = iris_hfi_session_close(inst);
+ if (ret)
+ wait_for_response = false;
+
+ kfree(inst->packet);
+ inst->packet = NULL;
+
+ if (wait_for_response) {
+ ret = wait_for_completion_timeout(&inst->completions[SIGNAL_CMD_CLOSE],
+ msecs_to_jiffies(hw_response_timeout_val));
+ if (!ret) {
+ ret = -ETIMEDOUT;
+ process_inst_timeout(inst);
+ }
+ }
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
index 4c7ddbf..b9a6485 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -7,8 +7,10 @@
#define _IRIS_HELPERS_H_

#include "iris_core.h"
+#include "iris_instance.h"

int check_core_lock(struct iris_core *core);
int iris_init_core_caps(struct iris_core *core);
+int close_session(struct iris_inst *inst);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index b0390b542..d144ae5 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -8,6 +8,7 @@
#include "iris_helpers.h"
#include "iris_hfi.h"
#include "iris_hfi_packet.h"
+#include "hfi_defines.h"
#include "vpu_common.h"

static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt)
@@ -43,6 +44,27 @@ static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt)
return ret;
}

+static bool validate_session(struct iris_core *core,
+ struct iris_inst *inst)
+{
+ struct iris_inst *temp;
+ bool valid = false;
+ int ret;
+
+ ret = check_core_lock(core);
+ if (ret)
+ return false;
+
+ list_for_each_entry(temp, &core->instances, list) {
+ if (temp == inst) {
+ valid = true;
+ break;
+ }
+ }
+
+ return valid;
+}
+
static int sys_init(struct iris_core *core)
{
int ret;
@@ -69,14 +91,6 @@ static int sys_image_version(struct iris_core *core)
return ret;
}

-#define CP_START 0
-#define CP_SIZE 0x25800000
-#define CP_NONPIXEL_START 0x01000000
-#define CP_NONPIXEL_SIZE 0x24800000
-
-#define FW_NAME "vpu30_4v.mbn"
-#define IRIS_PAS_ID 9
-
int iris_hfi_core_init(struct iris_core *core)
{
u32 cp_nonpixel_start, cp_nonpixel_size;
@@ -152,7 +166,90 @@ int iris_hfi_core_deinit(struct iris_core *core)
if (core->state == IRIS_CORE_DEINIT)
return 0;

- unload_fw(IRIS_PAS_ID);
+ unload_fw(core->platform_data->pas_id);
+
+ return ret;
+}
+
+int iris_hfi_session_open(struct iris_inst *inst)
+{
+ struct iris_core *core;
+ int ret;
+
+ inst->packet_size = 4096;
+ inst->packet = kzalloc(inst->packet_size, GFP_KERNEL);
+ if (!inst->packet)
+ return -ENOMEM;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto fail_free_packet;
+ }
+
+ ret = hfi_packet_session_command(inst,
+ HFI_CMD_OPEN,
+ HFI_HOST_FLAGS_RESPONSE_REQUIRED |
+ HFI_HOST_FLAGS_INTR_REQUIRED,
+ HFI_PORT_NONE,
+ 0,
+ HFI_PAYLOAD_U32,
+ &inst->session_id,
+ sizeof(u32));
+ if (ret)
+ goto fail_free_packet;
+
+ ret = iris_hfi_queue_cmd_write(core, inst->packet);
+ if (ret)
+ goto fail_free_packet;
+
+ mutex_unlock(&core->lock);
+
+ return ret;
+
+fail_free_packet:
+ kfree(inst->packet);
+ inst->packet = NULL;
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+int iris_hfi_session_close(struct iris_inst *inst)
+{
+ struct iris_core *core;
+ int ret;
+
+ if (!inst->packet)
+ return -EINVAL;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ ret = hfi_packet_session_command(inst,
+ HFI_CMD_CLOSE,
+ (HFI_HOST_FLAGS_RESPONSE_REQUIRED |
+ HFI_HOST_FLAGS_INTR_REQUIRED |
+ HFI_HOST_FLAGS_NON_DISCARDABLE),
+ HFI_PORT_NONE,
+ inst->session_id,
+ HFI_PAYLOAD_NONE,
+ NULL,
+ 0);
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);

return ret;
}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
index fcf9f28..8a057cc 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
@@ -6,9 +6,12 @@
#ifndef _IRIS_HFI_H_
#define _IRIS_HFI_H_

+#include "iris_instance.h"
#include "iris_core.h"

int iris_hfi_core_init(struct iris_core *core);
int iris_hfi_core_deinit(struct iris_core *core);
+int iris_hfi_session_open(struct iris_inst *inst);
+int iris_hfi_session_close(struct iris_inst *inst);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
index f31dd84..1ed572e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
@@ -211,3 +211,31 @@ int hfi_packet_image_version(struct iris_core *core,

return ret;
}
+
+int hfi_packet_session_command(struct iris_inst *inst, u32 pkt_type,
+ u32 flags, u32 port, u32 session_id,
+ u32 payload_type, void *payload,
+ u32 payload_size)
+{
+ struct iris_core *core;
+ int ret;
+
+ core = inst->core;
+
+ ret = hfi_create_header(inst->packet, inst->packet_size,
+ session_id, core->header_id++);
+ if (ret)
+ return ret;
+
+ ret = hfi_create_packet(inst->packet,
+ inst->packet_size,
+ pkt_type,
+ flags,
+ payload_type,
+ port,
+ core->packet_id++,
+ payload,
+ payload_size);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
index e36612c..9e476e9 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
@@ -6,6 +6,9 @@
#ifndef _IRIS_HFI_PACKET_H_
#define _IRIS_HFI_PACKET_H_

+#include "iris_core.h"
+#include "iris_instance.h"
+
struct hfi_header {
u32 size;
u32 session_id;
@@ -67,5 +70,9 @@ int hfi_packet_sys_init(struct iris_core *core,
u8 *pkt, u32 pkt_size);
int hfi_packet_image_version(struct iris_core *core,
u8 *pkt, u32 pkt_size);
+int hfi_packet_session_command(struct iris_inst *inst, u32 pkt_type,
+ u32 flags, u32 port, u32 session_id,
+ u32 payload_type, void *payload,
+ u32 payload_size);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
new file mode 100644
index 0000000..1bbb533
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -0,0 +1,48 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_INSTANCE_H_
+#define _IRIS_INSTANCE_H_
+
+#include <media/v4l2-ctrls.h>
+
+#include "iris_core.h"
+#include "iris_common.h"
+
+/**
+ * struct iris_inst - holds per video instance parameters
+ *
+ * @list: used for attach an instance to the core
+ * @core: pointer to core structure
+ * @session_id: id of current video session
+ * @vb2q_src: source vb2 queue
+ * @vb2q_dst: destination vb2 queue
+ * @ctx_q_lock: lock to serialize queues related ioctls
+ * @fh: reference of v4l2 file handler
+ * @fmt_src: structure of v4l2_format for source
+ * @fmt_dst: structure of v4l2_format for destination
+ * @ctrl_handler: reference of v4l2 ctrl handler
+ * @packet: HFI packet
+ * @packet_size: HFI packet size
+ * @completions: structure of signal completions
+ */
+
+struct iris_inst {
+ struct list_head list;
+ struct iris_core *core;
+ u32 session_id;
+ struct vb2_queue *vb2q_src;
+ struct vb2_queue *vb2q_dst;
+ struct mutex ctx_q_lock;/* lock to serialize queues related ioctls */
+ struct v4l2_fh fh;
+ struct v4l2_format *fmt_src;
+ struct v4l2_format *fmt_dst;
+ struct v4l2_ctrl_handler ctrl_handler;
+ void *packet;
+ u32 packet_size;
+ struct completion completions[MAX_SIGNAL];
+};
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
index 9d6a6c5..4f20da8 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -11,6 +11,7 @@
#include "iris_core.h"
#include "iris_helpers.h"
#include "resources.h"
+#include "iris_vidc.h"

static int iris_register_video_device(struct iris_core *core)
{
@@ -85,6 +86,8 @@ static int iris_probe(struct platform_device *pdev)
if (!core->packet)
return -ENOMEM;

+ INIT_LIST_HEAD(&core->instances);
+
core->reg_base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(core->reg_base))
return PTR_ERR(core->reg_base);
@@ -107,6 +110,13 @@ static int iris_probe(struct platform_device *pdev)
return ret;
}

+ ret = init_ops(core);
+ if (ret) {
+ dev_err_probe(core->dev, ret,
+ "%s: init ops failed with %d\n", __func__, ret);
+ return ret;
+ }
+
ret = init_resources(core);
if (ret) {
dev_err_probe(core->dev, ret,
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
new file mode 100644
index 0000000..984be34
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -0,0 +1,47 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "iris_common.h"
+#include "iris_vdec.h"
+
+void vdec_inst_init(struct iris_inst *inst)
+{
+ struct v4l2_format *f;
+
+ inst->fmt_src = kzalloc(sizeof(*inst->fmt_src), GFP_KERNEL);
+ inst->fmt_dst = kzalloc(sizeof(*inst->fmt_dst), GFP_KERNEL);
+ inst->vb2q_src = kzalloc(sizeof(*inst->vb2q_src), GFP_KERNEL);
+ inst->vb2q_dst = kzalloc(sizeof(*inst->vb2q_dst), GFP_KERNEL);
+
+ f = inst->fmt_src;
+ f->type = INPUT_MPLANE;
+ f->fmt.pix_mp.width = DEFAULT_WIDTH;
+ f->fmt.pix_mp.height = DEFAULT_HEIGHT;
+ f->fmt.pix_mp.pixelformat = V4L2_PIX_FMT_H264;
+ f->fmt.pix_mp.num_planes = 1;
+ f->fmt.pix_mp.plane_fmt[0].bytesperline = 0;
+ f->fmt.pix_mp.plane_fmt[0].sizeimage = DEFAULT_BUF_SIZE;
+ f->fmt.pix_mp.field = V4L2_FIELD_NONE;
+
+ f = inst->fmt_dst;
+ f->type = OUTPUT_MPLANE;
+ f->fmt.pix_mp.pixelformat = V4L2_PIX_FMT_QC08C;
+ f->fmt.pix_mp.width = ALIGN(DEFAULT_WIDTH, 128);
+ f->fmt.pix_mp.height = ALIGN(DEFAULT_HEIGHT, 32);
+ f->fmt.pix_mp.num_planes = 1;
+ f->fmt.pix_mp.plane_fmt[0].bytesperline = ALIGN(DEFAULT_WIDTH, 128);
+ f->fmt.pix_mp.plane_fmt[0].sizeimage = DEFAULT_BUF_SIZE;
+ f->fmt.pix_mp.field = V4L2_FIELD_NONE;
+ f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT;
+ f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT;
+ f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+}
+
+void vdec_inst_deinit(struct iris_inst *inst)
+{
+ kfree(inst->fmt_dst);
+ kfree(inst->fmt_src);
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
new file mode 100644
index 0000000..dc8f43f
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_VDEC_H_
+#define _IRIS_VDEC_H_
+
+#include "iris_instance.h"
+
+void vdec_inst_init(struct iris_inst *inst);
+void vdec_inst_deinit(struct iris_inst *inst);
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
new file mode 100644
index 0000000..d02da8b
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
@@ -0,0 +1,298 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "iris_common.h"
+#include "iris_helpers.h"
+#include "iris_hfi.h"
+#include "iris_instance.h"
+#include "iris_vdec.h"
+#include "iris_vidc.h"
+
+static int vidc_v4l2_fh_init(struct iris_inst *inst)
+{
+ struct iris_core *core;
+
+ core = inst->core;
+
+ if (inst->fh.vdev)
+ return -EINVAL;
+
+ v4l2_fh_init(&inst->fh, core->vdev_dec);
+ inst->fh.ctrl_handler = &inst->ctrl_handler;
+ v4l2_fh_add(&inst->fh);
+
+ return 0;
+}
+
+static int vidc_v4l2_fh_deinit(struct iris_inst *inst)
+{
+ if (!inst->fh.vdev)
+ return 0;
+
+ v4l2_fh_del(&inst->fh);
+ inst->fh.ctrl_handler = NULL;
+ v4l2_fh_exit(&inst->fh);
+
+ return 0;
+}
+
+static int vb2q_init(struct iris_inst *inst,
+ struct vb2_queue *q, enum v4l2_buf_type type)
+{
+ struct iris_core *core;
+
+ core = inst->core;
+
+ q->lock = &inst->ctx_q_lock;
+ q->type = type;
+ q->io_modes = VB2_MMAP | VB2_DMABUF;
+ q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ q->ops = core->vb2_ops;
+ q->mem_ops = core->vb2_mem_ops;
+ q->drv_priv = inst;
+ q->copy_timestamp = 1;
+ q->min_buffers_needed = 0;
+ return vb2_queue_init(q);
+}
+
+static int vidc_vb2_queue_init(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = vb2q_init(inst, inst->vb2q_src, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = vb2q_init(inst, inst->vb2q_dst, OUTPUT_MPLANE);
+ if (ret)
+ goto fail_vb2q_src_deinit;
+
+ return ret;
+
+fail_vb2q_src_deinit:
+ vb2_queue_release(inst->vb2q_src);
+
+ return ret;
+}
+
+static int vidc_vb2_queue_deinit(struct iris_inst *inst)
+{
+ vb2_queue_release(inst->vb2q_src);
+ kfree(inst->vb2q_src);
+ inst->vb2q_src = NULL;
+
+ vb2_queue_release(inst->vb2q_dst);
+ kfree(inst->vb2q_dst);
+ inst->vb2q_dst = NULL;
+
+ return 0;
+}
+
+static int vidc_add_session(struct iris_inst *inst)
+{
+ struct iris_core *core;
+ struct iris_inst *i;
+ u32 count = 0;
+ int ret = 0;
+
+ core = inst->core;
+
+ mutex_lock(&core->lock);
+ if (core->state != IRIS_CORE_INIT) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+ list_for_each_entry(i, &core->instances, list)
+ count++;
+
+ if (count < core->cap[MAX_SESSION_COUNT].value)
+ list_add_tail(&inst->list, &core->instances);
+ else
+ ret = -EAGAIN;
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+static int vidc_remove_session(struct iris_inst *inst)
+{
+ struct iris_inst *i, *temp;
+ struct iris_core *core;
+
+ core = inst->core;
+
+ mutex_lock(&core->lock);
+ list_for_each_entry_safe(i, temp, &core->instances, list) {
+ if (i->session_id == inst->session_id) {
+ list_del_init(&i->list);
+ break;
+ }
+ }
+ mutex_unlock(&core->lock);
+
+ return 0;
+}
+
+static struct iris_inst *get_vidc_inst(struct file *filp, void *fh)
+{
+ if (!filp || !filp->private_data)
+ return NULL;
+
+ return container_of(filp->private_data,
+ struct iris_inst, fh);
+}
+
+int vidc_open(struct file *filp)
+{
+ struct iris_core *core = video_drvdata(filp);
+ struct iris_inst *inst = NULL;
+ int i = 0;
+ int ret;
+
+ ret = iris_core_init(core);
+ if (ret)
+ return ret;
+
+ ret = iris_core_init_wait(core);
+ if (ret)
+ return ret;
+
+ inst = kzalloc(sizeof(*inst), GFP_KERNEL);
+ if (!inst)
+ return -ENOMEM;
+
+ inst->core = core;
+ inst->session_id = hash32_ptr(inst);
+ mutex_init(&inst->ctx_q_lock);
+ for (i = 0; i < MAX_SIGNAL; i++)
+ init_completion(&inst->completions[i]);
+
+ ret = vidc_add_session(inst);
+ if (ret)
+ goto fail_free_inst;
+
+ ret = vidc_v4l2_fh_init(inst);
+ if (ret)
+ goto fail_remove_session;
+
+ vdec_inst_init(inst);
+
+ ret = vidc_vb2_queue_init(inst);
+ if (ret)
+ goto fail_inst_deinit;
+
+ ret = iris_hfi_session_open(inst);
+ if (ret) {
+ dev_err(core->dev, "%s: session open failed\n", __func__);
+ goto fail_core_deinit;
+ }
+ filp->private_data = &inst->fh;
+
+ return 0;
+
+fail_core_deinit:
+ iris_core_deinit(core);
+ vidc_vb2_queue_deinit(inst);
+fail_inst_deinit:
+ vdec_inst_deinit(inst);
+ vidc_v4l2_fh_deinit(inst);
+fail_remove_session:
+ vidc_remove_session(inst);
+fail_free_inst:
+ mutex_destroy(&inst->ctx_q_lock);
+ kfree(inst);
+
+ return ret;
+}
+
+int vidc_close(struct file *filp)
+{
+ struct iris_inst *inst;
+
+ inst = get_vidc_inst(filp, NULL);
+ if (!inst)
+ return -EINVAL;
+
+ vdec_inst_deinit(inst);
+ close_session(inst);
+ vidc_vb2_queue_deinit(inst);
+ vidc_v4l2_fh_deinit(inst);
+ vidc_remove_session(inst);
+ mutex_destroy(&inst->ctx_q_lock);
+ kfree(inst);
+
+ filp->private_data = NULL;
+
+ return 0;
+}
+
+static __poll_t get_poll_flags(struct iris_inst *inst, u32 plane)
+{
+ struct vb2_buffer *vb = NULL;
+ struct vb2_queue *q = NULL;
+ unsigned long flags = 0;
+ __poll_t poll = 0;
+
+ if (plane == INPUT_MPLANE)
+ q = inst->vb2q_src;
+ else if (plane == OUTPUT_MPLANE)
+ q = inst->vb2q_dst;
+
+ if (!q)
+ return EPOLLERR;
+
+ spin_lock_irqsave(&q->done_lock, flags);
+ if (!list_empty(&q->done_list))
+ vb = list_first_entry(&q->done_list, struct vb2_buffer,
+ done_entry);
+ if (vb && (vb->state == VB2_BUF_STATE_DONE ||
+ vb->state == VB2_BUF_STATE_ERROR)) {
+ if (plane == OUTPUT_MPLANE)
+ poll |= EPOLLIN | EPOLLRDNORM;
+ else if (plane == INPUT_MPLANE)
+ poll |= EPOLLOUT | EPOLLWRNORM;
+ }
+ spin_unlock_irqrestore(&q->done_lock, flags);
+
+ return poll;
+}
+
+static __poll_t vidc_poll(struct file *filp, struct poll_table_struct *pt)
+{
+ struct iris_inst *inst;
+ __poll_t poll = 0;
+
+ inst = get_vidc_inst(filp, NULL);
+ if (!inst)
+ return EPOLLERR;
+
+ poll_wait(filp, &inst->fh.wait, pt);
+ poll_wait(filp, &inst->vb2q_src->done_wq, pt);
+ poll_wait(filp, &inst->vb2q_dst->done_wq, pt);
+
+ if (v4l2_event_pending(&inst->fh))
+ poll |= EPOLLPRI;
+
+ poll |= get_poll_flags(inst, INPUT_MPLANE);
+ poll |= get_poll_flags(inst, OUTPUT_MPLANE);
+
+ return poll;
+}
+
+static const struct v4l2_file_operations v4l2_file_ops = {
+ .owner = THIS_MODULE,
+ .open = vidc_open,
+ .release = vidc_close,
+ .unlocked_ioctl = video_ioctl2,
+ .poll = vidc_poll,
+};
+
+int init_ops(struct iris_core *core)
+{
+ core->v4l2_file_ops = &v4l2_file_ops;
+
+ return 0;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.h b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.h
new file mode 100644
index 0000000..aebff44
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_VIDC_H_
+#define _IRIS_VIDC_H_
+
+#include "iris_core.h"
+
+int init_ops(struct iris_core *core);
+int vidc_open(struct file *filp);
+int vidc_close(struct file *filp);
+
+#endif
--
2.7.4

2023-12-18 11:53:06

[permalink] [raw]

Subject: [PATCH v2 23/34] media: iris: implement iris v4l2 ioctl ops supported by decoder

From: Vikash Garodia <[email protected]>

Implement all iris v4l2 ioctls ops supported by decoder.
Add state checks to ensure ioctl are allowed in valid
instance state only. Codec format can be changed by client
during s_fmt. Update the v4l2 control values according
to the updated codec format.

Signed-off-by: Vikash Garodia <[email protected]>
Signed-off-by: Dikshita Agarwal <[email protected]>
---
.../media/platform/qcom/vcodec/iris/iris_buffer.c | 23 +
.../media/platform/qcom/vcodec/iris/iris_buffer.h | 1 +
.../media/platform/qcom/vcodec/iris/iris_common.h | 2 +
.../media/platform/qcom/vcodec/iris/iris_ctrls.c | 48 +-
.../media/platform/qcom/vcodec/iris/iris_ctrls.h | 6 +-
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 212 +++++++
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 8 +
.../platform/qcom/vcodec/iris/iris_instance.h | 4 +
.../media/platform/qcom/vcodec/iris/iris_state.c | 38 ++
.../media/platform/qcom/vcodec/iris/iris_state.h | 6 +
drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 17 +
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 281 ++++++++-
.../media/platform/qcom/vcodec/iris/iris_vdec.h | 6 +-
.../media/platform/qcom/vcodec/iris/iris_vidc.c | 632 ++++++++++++++++++++-
.../platform/qcom/vcodec/iris/platform_common.h | 18 +
.../platform/qcom/vcodec/iris/platform_sm8550.c | 42 ++
16 files changed, 1323 insertions(+), 21 deletions(-)

diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
index 2a3989c..1ee840e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
@@ -55,6 +55,29 @@ static int output_min_count(struct iris_inst *inst)
return output_min_count;
}

+int update_buffer_count(struct iris_inst *inst, u32 plane)
+{
+ switch (plane) {
+ case INPUT_MPLANE:
+ inst->buffers.input.min_count = input_min_count(inst);
+ if (inst->buffers.input.actual_count < inst->buffers.input.min_count)
+ inst->buffers.input.actual_count = inst->buffers.input.min_count;
+
+ break;
+ case OUTPUT_MPLANE:
+ if (!inst->vb2q_src->streaming)
+ inst->buffers.output.min_count = output_min_count(inst);
+ if (inst->buffers.output.actual_count < inst->buffers.output.min_count)
+ inst->buffers.output.actual_count = inst->buffers.output.min_count;
+
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static u32 internal_buffer_count(struct iris_inst *inst,
enum iris_buffer_type buffer_type)
{
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
index ece894e..bdef15f 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
@@ -34,6 +34,7 @@ struct iris_buffers_info {
struct iris_buffers vpss;
};

+int update_buffer_count(struct iris_inst *inst, u32 plane);
int iris_get_buf_min_count(struct iris_inst *inst,
enum iris_buffer_type buffer_type);
int iris_get_buffer_size(struct iris_inst *inst,
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
index a83d1c1..6b771f8 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
@@ -17,6 +17,8 @@
#define DEFAULT_HEIGHT 240
#define DEFAULT_BSE_VPP_DELAY 2

+#define MAX_EVENTS 30
+
#define MB_IN_PIXEL (16 * 16)

#define NUM_MBS_4k (((4096 + 15) >> 4) * ((2304 + 15) >> 4))
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
index 6b7aeaa..28977e8 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
@@ -209,15 +209,15 @@ static int vdec_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
struct iris_inst *inst = NULL;
int ret = 0;

+ inst = container_of(ctrl->handler, struct iris_inst, ctrl_handler);
switch (ctrl->id) {
case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE:
- ctrl->val = MIN_CAPTURE_BUFFERS;
+ ctrl->val = inst->buffers.output.min_count;
break;
case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT:
- ctrl->val = MIN_OUTPUT_BUFFERS;
+ ctrl->val = inst->buffers.input.min_count;
break;
default:
- inst = container_of(ctrl->handler, struct iris_inst, ctrl_handler);
cap_id = get_cap_id(inst, ctrl->id);
if (is_valid_cap_id(cap_id))
ctrl->val = inst->cap[cap_id].value;
@@ -232,12 +232,17 @@ static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl)
{
struct cap_entry *entry = NULL, *temp = NULL;
struct list_head children_list, firmware_list;
+ struct ctrl_data *priv_ctrl_data;
enum plat_inst_cap_type cap_id;
bool cap_present[INST_CAP_MAX];
struct plat_inst_cap *cap;
struct iris_inst *inst;
int ret = 0;

+ priv_ctrl_data = ctrl->priv ? ctrl->priv : NULL;
+ if (priv_ctrl_data && priv_ctrl_data->skip_s_ctrl)
+ return 0;
+
inst = container_of(ctrl->handler, struct iris_inst, ctrl_handler);
cap = &inst->cap[0];

@@ -249,6 +254,9 @@ static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl)
if (!is_valid_cap_id(cap_id))
return -EINVAL;

+ if (!allow_s_ctrl(inst, cap_id))
+ return -EBUSY;
+
cap[cap_id].flags |= CAP_FLAG_CLIENT_SET;

if (!inst->vb2q_src->streaming) {
@@ -282,11 +290,12 @@ static const struct v4l2_ctrl_ops ctrl_ops = {
.g_volatile_ctrl = vdec_op_g_volatile_ctrl,
};

-int ctrls_init(struct iris_inst *inst)
+int ctrls_init(struct iris_inst *inst, bool init)
{
int num_ctrls = 0, ctrl_idx = 0;
struct plat_inst_cap *cap;
struct iris_core *core;
+ u64 step_or_mask;
int idx = 0;
int ret = 0;

@@ -300,10 +309,12 @@ int ctrls_init(struct iris_inst *inst)
if (!num_ctrls)
return -EINVAL;

- ret = v4l2_ctrl_handler_init(&inst->ctrl_handler,
- INST_CAP_MAX * core->dec_codecs_count);
- if (ret)
- return ret;
+ if (init) {
+ ret = v4l2_ctrl_handler_init(&inst->ctrl_handler,
+ INST_CAP_MAX * core->dec_codecs_count);
+ if (ret)
+ return ret;
+ }

for (idx = 0; idx < INST_CAP_MAX; idx++) {
struct v4l2_ctrl *ctrl;
@@ -316,6 +327,27 @@ int ctrls_init(struct iris_inst *inst)
goto error;
}

+ if (!init) {
+ struct ctrl_data ctrl_priv_data;
+
+ ctrl = v4l2_ctrl_find(&inst->ctrl_handler, cap[idx].v4l2_id);
+ if (ctrl) {
+ step_or_mask = (cap[idx].flags & CAP_FLAG_MENU) ?
+ ~(cap[idx].step_or_mask) :
+ cap[idx].step_or_mask;
+ memset(&ctrl_priv_data, 0, sizeof(ctrl_priv_data));
+ ctrl_priv_data.skip_s_ctrl = true;
+ ctrl->priv = &ctrl_priv_data;
+ v4l2_ctrl_modify_range(ctrl,
+ cap[idx].min,
+ cap[idx].max,
+ step_or_mask,
+ cap[idx].value);
+ ctrl->priv = NULL;
+ continue;
+ }
+ }
+
if (cap[idx].flags & CAP_FLAG_MENU) {
ctrl = v4l2_ctrl_new_std_menu(&inst->ctrl_handler,
&ctrl_ops,
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
index 0f67f4f..22ee6c4b 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
@@ -13,6 +13,10 @@ struct cap_entry {
enum plat_inst_cap_type cap_id;
};

+struct ctrl_data {
+ bool skip_s_ctrl;
+};
+
int set_u32_enum(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
int set_stage(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
int set_pipe(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
@@ -25,6 +29,6 @@ int iris_init_instance_caps(struct iris_core *core);
int iris_init_core_caps(struct iris_core *core);
int get_inst_capability(struct iris_inst *inst);
int adjust_v4l2_properties(struct iris_inst *inst);
-int ctrls_init(struct iris_inst *inst);
+int ctrls_init(struct iris_inst *inst, bool init);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index 335885f..ff44cda 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -79,6 +79,112 @@ bool is_split_mode_enabled(struct iris_inst *inst)
return false;
}

+inline bool is_10bit_colorformat(enum colorformat_type colorformat)
+{
+ return colorformat == FMT_TP10C;
+}
+
+inline bool is_8bit_colorformat(enum colorformat_type colorformat)
+{
+ return colorformat == FMT_NV12 ||
+ colorformat == FMT_NV12C ||
+ colorformat == FMT_NV21;
+}
+
+u32 v4l2_codec_from_driver(struct iris_inst *inst, enum codec_type codec)
+{
+ const struct codec_info *codec_info;
+ struct iris_core *core;
+ u32 v4l2_codec = 0;
+ u32 i, size;
+
+ core = inst->core;
+ codec_info = core->platform_data->format_data->codec_info;
+ size = core->platform_data->format_data->codec_info_size;
+
+ for (i = 0; i < size; i++) {
+ if (codec_info[i].codec == codec)
+ return codec_info[i].v4l2_codec;
+ }
+
+ return v4l2_codec;
+}
+
+enum codec_type v4l2_codec_to_driver(struct iris_inst *inst, u32 v4l2_codec)
+{
+ const struct codec_info *codec_info;
+ enum codec_type codec = 0;
+ struct iris_core *core;
+ u32 i, size;
+
+ core = inst->core;
+ codec_info = core->platform_data->format_data->codec_info;
+ size = core->platform_data->format_data->codec_info_size;
+
+ for (i = 0; i < size; i++) {
+ if (codec_info[i].v4l2_codec == v4l2_codec)
+ return codec_info[i].codec;
+ }
+
+ return codec;
+}
+
+u32 v4l2_colorformat_from_driver(struct iris_inst *inst, enum colorformat_type colorformat)
+{
+ const struct color_format_info *color_format_info;
+ u32 v4l2_colorformat = 0;
+ struct iris_core *core;
+ u32 i, size;
+
+ core = inst->core;
+ color_format_info = core->platform_data->format_data->color_format_info;
+ size = core->platform_data->format_data->color_format_info_size;
+
+ for (i = 0; i < size; i++) {
+ if (color_format_info[i].color_format == colorformat)
+ return color_format_info[i].v4l2_color_format;
+ }
+
+ return v4l2_colorformat;
+}
+
+enum colorformat_type v4l2_colorformat_to_driver(struct iris_inst *inst, u32 v4l2_colorformat)
+{
+ const struct color_format_info *color_format_info;
+ enum colorformat_type colorformat = 0;
+ struct iris_core *core;
+ u32 i, size;
+
+ core = inst->core;
+ color_format_info = core->platform_data->format_data->color_format_info;
+ size = core->platform_data->format_data->color_format_info_size;
+
+ for (i = 0; i < size; i++) {
+ if (color_format_info[i].v4l2_color_format == v4l2_colorformat)
+ return color_format_info[i].color_format;
+ }
+
+ return colorformat;
+}
+
+struct vb2_queue *get_vb2q(struct iris_inst *inst, u32 type)
+{
+ struct vb2_queue *vb2q = NULL;
+
+ switch (type) {
+ case INPUT_MPLANE:
+ vb2q = inst->vb2q_src;
+ break;
+ case OUTPUT_MPLANE:
+ vb2q = inst->vb2q_dst;
+ break;
+ default:
+ return NULL;
+ }
+
+ return vb2q;
+}
+
static int process_inst_timeout(struct iris_inst *inst)
{
struct iris_inst *instance;
@@ -128,13 +234,119 @@ int close_session(struct iris_inst *inst)
inst->packet = NULL;

if (wait_for_response) {
+ mutex_unlock(&inst->lock);
ret = wait_for_completion_timeout(&inst->completions[SIGNAL_CMD_CLOSE],
msecs_to_jiffies(hw_response_timeout_val));
if (!ret) {
ret = -ETIMEDOUT;
process_inst_timeout(inst);
}
+ mutex_lock(&inst->lock);
}

return ret;
}
+
+static int check_core_mbps_mbpf(struct iris_inst *inst)
+{
+ u32 mbpf = 0, mbps = 0, total_mbpf = 0, total_mbps = 0;
+ struct iris_core *core;
+ struct iris_inst *instance;
+ u32 fps;
+
+ core = inst->core;
+
+ mutex_lock(&core->lock);
+ list_for_each_entry(instance, &core->instances, list) {
+ fps = inst->cap[QUEUED_RATE].value >> 16;
+ mbpf = get_mbpf(inst);
+ mbps = mbpf * fps;
+ total_mbpf += mbpf;
+ total_mbps += mbps;
+ }
+ mutex_unlock(&core->lock);
+
+ if (total_mbps > core->cap[MAX_MBPS].value ||
+ total_mbpf > core->cap[MAX_MBPF].value)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static int check_inst_mbpf(struct iris_inst *inst)
+{
+ u32 mbpf = 0, max_mbpf = 0;
+
+ max_mbpf = inst->cap[MBPF].max;
+ mbpf = get_mbpf(inst);
+ if (mbpf > max_mbpf)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static int check_resolution_supported(struct iris_inst *inst)
+{
+ u32 width = 0, height = 0, min_width, min_height,
+ max_width, max_height;
+
+ width = inst->fmt_src->fmt.pix_mp.width;
+ height = inst->fmt_src->fmt.pix_mp.height;
+
+ min_width = inst->cap[FRAME_WIDTH].min;
+ max_width = inst->cap[FRAME_WIDTH].max;
+ min_height = inst->cap[FRAME_HEIGHT].min;
+ max_height = inst->cap[FRAME_HEIGHT].max;
+
+ if (!(min_width <= width && width <= max_width) ||
+ !(min_height <= height && height <= max_height))
+ return -EINVAL;
+
+ return 0;
+}
+
+static int check_max_sessions(struct iris_inst *inst)
+{
+ struct iris_core *core;
+ u32 num_sessions = 0;
+ struct iris_inst *i;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+ list_for_each_entry(i, &core->instances, list) {
+ num_sessions++;
+ }
+ mutex_unlock(&core->lock);
+
+ if (num_sessions > core->cap[MAX_SESSION_COUNT].value)
+ return -ENOMEM;
+
+ return 0;
+}
+
+int check_session_supported(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = check_core_mbps_mbpf(inst);
+ if (ret)
+ goto exit;
+
+ ret = check_inst_mbpf(inst);
+ if (ret)
+ goto exit;
+
+ ret = check_resolution_supported(inst);
+ if (ret)
+ goto exit;
+
+ ret = check_max_sessions(inst);
+ if (ret)
+ goto exit;
+
+ return ret;
+exit:
+ dev_err(inst->core->dev, "current session not supported(%d)\n", ret);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
index 9e85510..fe85d23 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -28,6 +28,14 @@ int get_mbpf(struct iris_inst *inst);
int close_session(struct iris_inst *inst);

bool is_linear_colorformat(u32 colorformat);
+bool is_10bit_colorformat(enum colorformat_type colorformat);
+bool is_8bit_colorformat(enum colorformat_type colorformat);
bool is_split_mode_enabled(struct iris_inst *inst);
+u32 v4l2_codec_from_driver(struct iris_inst *inst, enum codec_type codec);
+enum codec_type v4l2_codec_to_driver(struct iris_inst *inst, u32 v4l2_codec);
+u32 v4l2_colorformat_from_driver(struct iris_inst *inst, enum colorformat_type colorformat);
+enum colorformat_type v4l2_colorformat_to_driver(struct iris_inst *inst, u32 v4l2_colorformat);
+struct vb2_queue *get_vb2q(struct iris_inst *inst, u32 type);
+int check_session_supported(struct iris_inst *inst);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
index 275efa8..365f844 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -23,6 +23,7 @@
* @vb2q_src: source vb2 queue
* @vb2q_dst: destination vb2 queue
* @ctx_q_lock: lock to serialize queues related ioctls
+ * @lock: lock to seralise forward and reverse threads
* @fh: reference of v4l2 file handler
* @fmt_src: structure of v4l2_format for source
* @fmt_dst: structure of v4l2_format for destination
@@ -38,6 +39,7 @@
* @buffers: structure of buffer info
* @fw_min_count: minimnum count of buffers needed by fw
* @state: instance state
+ * @ipsc_properties_set: boolean to set ipsc properties to fw
*/

struct iris_inst {
@@ -47,6 +49,7 @@ struct iris_inst {
struct vb2_queue *vb2q_src;
struct vb2_queue *vb2q_dst;
struct mutex ctx_q_lock;/* lock to serialize queues related ioctls */
+ struct mutex lock;
struct v4l2_fh fh;
struct v4l2_format *fmt_src;
struct v4l2_format *fmt_dst;
@@ -62,6 +65,7 @@ struct iris_inst {
struct iris_buffers_info buffers;
u32 fw_min_count;
enum iris_inst_state state;
+ bool ipsc_properties_set;
};

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
index 9bf79a0..4cf6b69 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_state.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
@@ -155,3 +155,41 @@ int iris_inst_change_state(struct iris_inst *inst,

return 0;
}
+
+bool allow_s_fmt(struct iris_inst *inst, u32 type)
+{
+ return (inst->state == IRIS_INST_OPEN) ||
+ (type == OUTPUT_MPLANE && inst->state == IRIS_INST_INPUT_STREAMING) ||
+ (type == INPUT_MPLANE && inst->state == IRIS_INST_OUTPUT_STREAMING);
+}
+
+bool allow_reqbufs(struct iris_inst *inst, u32 type)
+{
+ return (inst->state == IRIS_INST_OPEN) ||
+ (type == OUTPUT_MPLANE && inst->state == IRIS_INST_INPUT_STREAMING) ||
+ (type == INPUT_MPLANE && inst->state == IRIS_INST_OUTPUT_STREAMING);
+}
+
+bool allow_streamon(struct iris_inst *inst, u32 type)
+{
+ return (type == INPUT_MPLANE && inst->state == IRIS_INST_OPEN) ||
+ (type == INPUT_MPLANE && inst->state == IRIS_INST_OUTPUT_STREAMING) ||
+ (type == OUTPUT_MPLANE && inst->state == IRIS_INST_OPEN) ||
+ (type == OUTPUT_MPLANE && inst->state == IRIS_INST_INPUT_STREAMING);
+}
+
+bool allow_streamoff(struct iris_inst *inst, u32 type)
+{
+ return (type == INPUT_MPLANE && inst->state == IRIS_INST_INPUT_STREAMING) ||
+ (type == INPUT_MPLANE && inst->state == IRIS_INST_STREAMING) ||
+ (type == OUTPUT_MPLANE && inst->state == IRIS_INST_OUTPUT_STREAMING) ||
+ (type == OUTPUT_MPLANE && inst->state == IRIS_INST_STREAMING);
+}
+
+bool allow_s_ctrl(struct iris_inst *inst, u32 cap_id)
+{
+ return ((inst->state == IRIS_INST_OPEN) ||
+ ((inst->cap[cap_id].flags & CAP_FLAG_DYNAMIC_ALLOWED) &&
+ (inst->state == IRIS_INST_INPUT_STREAMING ||
+ inst->state == IRIS_INST_STREAMING)));
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.h b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
index 6db95a1..35263e8 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_state.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
@@ -41,4 +41,10 @@ int iris_change_core_state(struct iris_core *core,
int iris_inst_change_state(struct iris_inst *inst,
enum iris_inst_state request_state);

+bool allow_s_fmt(struct iris_inst *inst, u32 type);
+bool allow_reqbufs(struct iris_inst *inst, u32 type);
+bool allow_streamon(struct iris_inst *inst, u32 type);
+bool allow_streamoff(struct iris_inst *inst, u32 type);
+bool allow_s_ctrl(struct iris_inst *inst, u32 cap_id);
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
index 8f499a9..66b5295 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
@@ -4,6 +4,7 @@
*/

#include "iris_buffer.h"
+#include "iris_ctrls.h"
#include "iris_core.h"
#include "iris_helpers.h"
#include "iris_instance.h"
@@ -47,6 +48,22 @@ int iris_vb2_queue_setup(struct vb2_queue *q,
if (!buffer_type)
return -EINVAL;

+ if (list_empty(&inst->caps_list)) {
+ ret = prepare_dependency_list(inst);
+ if (ret)
+ return ret;
+ }
+
+ if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ ret = adjust_v4l2_properties(inst);
+ if (ret)
+ return ret;
+ }
+
+ ret = check_session_supported(inst);
+ if (ret)
+ return ret;
+
ret = iris_free_buffers(inst, buffer_type);
if (ret)
return ret;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index 566048e..64067d5 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -3,13 +3,51 @@
* Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
*/

+#include <media/v4l2-event.h>
+
#include "iris_buffer.h"
#include "iris_common.h"
+#include "iris_ctrls.h"
+#include "iris_helpers.h"
#include "iris_vdec.h"

-void vdec_inst_init(struct iris_inst *inst)
+static int vdec_codec_change(struct iris_inst *inst, u32 v4l2_codec)
+{
+ bool session_init = false;
+ int ret;
+
+ if (!inst->codec)
+ session_init = true;
+
+ if (inst->codec && inst->fmt_src->fmt.pix_mp.pixelformat == v4l2_codec)
+ return 0;
+
+ inst->codec = v4l2_codec_to_driver(inst, v4l2_codec);
+ if (!inst->codec)
+ return -EINVAL;
+
+ inst->fmt_src->fmt.pix_mp.pixelformat = v4l2_codec;
+ ret = get_inst_capability(inst);
+ if (ret)
+ return ret;
+
+ ret = ctrls_init(inst, session_init);
+ if (ret)
+ return ret;
+
+ ret = update_buffer_count(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = update_buffer_count(inst, OUTPUT_MPLANE);
+
+ return ret;
+}
+
+int vdec_inst_init(struct iris_inst *inst)
{
struct v4l2_format *f;
+ int ret;

inst->fmt_src = kzalloc(sizeof(*inst->fmt_src), GFP_KERNEL);
inst->fmt_dst = kzalloc(sizeof(*inst->fmt_dst), GFP_KERNEL);
@@ -43,6 +81,10 @@ void vdec_inst_init(struct iris_inst *inst)
inst->buffers.output.actual_count = inst->buffers.output.min_count;
inst->buffers.output.size = f->fmt.pix_mp.plane_fmt[0].sizeimage;
inst->fw_min_count = 0;
+
+ ret = vdec_codec_change(inst, inst->fmt_src->fmt.pix_mp.pixelformat);
+
+ return ret;
}

void vdec_inst_deinit(struct iris_inst *inst)
@@ -50,3 +92,240 @@ void vdec_inst_deinit(struct iris_inst *inst)
kfree(inst->fmt_dst);
kfree(inst->fmt_src);
}
+
+static int vdec_check_colorformat_supported(struct iris_inst *inst,
+ enum colorformat_type colorformat)
+{
+ bool supported = true;
+
+ if (!inst->vb2q_src->streaming)
+ return true;
+
+ if (inst->cap[BIT_DEPTH].value == BIT_DEPTH_8 &&
+ !is_8bit_colorformat(colorformat))
+ supported = false;
+ if (inst->cap[BIT_DEPTH].value == BIT_DEPTH_10 &&
+ !is_10bit_colorformat(colorformat))
+ supported = false;
+ if (inst->cap[CODED_FRAMES].value == CODED_FRAMES_INTERLACE)
+ supported = false;
+
+ return supported;
+}
+
+int vdec_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f)
+{
+ struct iris_core *core;
+ u32 array[32] = {0};
+ u32 i = 0;
+
+ if (f->index >= ARRAY_SIZE(array))
+ return -EINVAL;
+
+ core = inst->core;
+ if (f->type == INPUT_MPLANE) {
+ u32 codecs = core->cap[DEC_CODECS].value;
+ u32 codecs_count = hweight32(codecs);
+ u32 idx = 0;
+
+ for (i = 0; i <= codecs_count; i++) {
+ if (codecs & BIT(i)) {
+ if (idx >= ARRAY_SIZE(array))
+ break;
+ array[idx] = codecs & BIT(i);
+ idx++;
+ }
+ }
+ if (!array[f->index])
+ return -EINVAL;
+ f->pixelformat = v4l2_codec_from_driver(inst, array[f->index]);
+ f->flags = V4L2_FMT_FLAG_COMPRESSED;
+ strscpy(f->description, "codec", sizeof(f->description));
+ } else if (f->type == OUTPUT_MPLANE) {
+ u32 formats = inst->cap[PIX_FMTS].step_or_mask;
+ u32 idx = 0;
+
+ for (i = 0; i <= 31; i++) {
+ if (formats & BIT(i)) {
+ if (idx >= ARRAY_SIZE(array))
+ break;
+ if (vdec_check_colorformat_supported(inst, formats & BIT(i))) {
+ array[idx] = formats & BIT(i);
+ idx++;
+ }
+ }
+ }
+ if (!array[f->index])
+ return -EINVAL;
+ f->pixelformat = v4l2_colorformat_from_driver(inst, array[f->index]);
+ strscpy(f->description, "colorformat", sizeof(f->description));
+ }
+
+ if (!f->pixelformat)
+ return -EINVAL;
+
+ memset(f->reserved, 0, sizeof(f->reserved));
+
+ return 0;
+}
+
+int vdec_try_fmt(struct iris_inst *inst, struct v4l2_format *f)
+{
+ struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
+ struct v4l2_format *f_inst;
+ u32 pix_fmt;
+
+ memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
+ if (f->type == INPUT_MPLANE) {
+ pix_fmt = v4l2_codec_to_driver(inst, f->fmt.pix_mp.pixelformat);
+ if (!pix_fmt) {
+ f_inst = inst->fmt_src;
+ f->fmt.pix_mp.width = f_inst->fmt.pix_mp.width;
+ f->fmt.pix_mp.height = f_inst->fmt.pix_mp.height;
+ f->fmt.pix_mp.pixelformat = f_inst->fmt.pix_mp.pixelformat;
+ pix_fmt = v4l2_codec_to_driver(inst, f->fmt.pix_mp.pixelformat);
+ }
+ } else if (f->type == OUTPUT_MPLANE) {
+ pix_fmt = v4l2_colorformat_to_driver(inst, f->fmt.pix_mp.pixelformat);
+ if (!pix_fmt) {
+ f_inst = inst->fmt_dst;
+ f->fmt.pix_mp.pixelformat = f_inst->fmt.pix_mp.pixelformat;
+ f->fmt.pix_mp.width = f_inst->fmt.pix_mp.width;
+ f->fmt.pix_mp.height = f_inst->fmt.pix_mp.height;
+ }
+ if (inst->vb2q_src->streaming) {
+ f_inst = inst->fmt_src;
+ f->fmt.pix_mp.height = f_inst->fmt.pix_mp.height;
+ f->fmt.pix_mp.width = f_inst->fmt.pix_mp.width;
+ }
+ } else {
+ return -EINVAL;
+ }
+
+ if (pixmp->field == V4L2_FIELD_ANY)
+ pixmp->field = V4L2_FIELD_NONE;
+
+ pixmp->num_planes = 1;
+
+ return 0;
+}
+
+int vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f)
+{
+ struct v4l2_format *fmt, *output_fmt;
+ enum colorformat_type colorformat;
+ u32 codec_align, stride = 0;
+ int ret = 0;
+
+ vdec_try_fmt(inst, f);
+
+ if (f->type == INPUT_MPLANE) {
+ if (inst->fmt_src->fmt.pix_mp.pixelformat !=
+ f->fmt.pix_mp.pixelformat) {
+ ret = vdec_codec_change(inst, f->fmt.pix_mp.pixelformat);
+ if (ret)
+ return ret;
+ }
+
+ fmt = inst->fmt_src;
+ fmt->type = INPUT_MPLANE;
+
+ codec_align = inst->fmt_src->fmt.pix_mp.pixelformat ==
+ V4L2_PIX_FMT_HEVC ? 32 : 16;
+ fmt->fmt.pix_mp.width = ALIGN(f->fmt.pix_mp.width, codec_align);
+ fmt->fmt.pix_mp.height = ALIGN(f->fmt.pix_mp.height, codec_align);
+ fmt->fmt.pix_mp.num_planes = 1;
+ fmt->fmt.pix_mp.plane_fmt[0].bytesperline = 0;
+ fmt->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_INPUT);
+ inst->buffers.input.min_count = iris_get_buf_min_count(inst, BUF_INPUT);
+ if (inst->buffers.input.actual_count <
+ inst->buffers.input.min_count) {
+ inst->buffers.input.actual_count =
+ inst->buffers.input.min_count;
+ }
+ inst->buffers.input.size =
+ fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+ fmt->fmt.pix_mp.colorspace = f->fmt.pix_mp.colorspace;
+ fmt->fmt.pix_mp.xfer_func = f->fmt.pix_mp.xfer_func;
+ fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc;
+ fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization;
+
+ output_fmt = inst->fmt_dst;
+ output_fmt->fmt.pix_mp.colorspace = f->fmt.pix_mp.colorspace;
+ output_fmt->fmt.pix_mp.xfer_func = f->fmt.pix_mp.xfer_func;
+ output_fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc;
+ output_fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization;
+
+ inst->crop.left = 0;
+ inst->crop.top = 0;
+ inst->crop.width = f->fmt.pix_mp.width;
+ inst->crop.height = f->fmt.pix_mp.height;
+ } else if (f->type == OUTPUT_MPLANE) {
+ fmt = inst->fmt_dst;
+ fmt->type = OUTPUT_MPLANE;
+ if (inst->vb2q_src->streaming) {
+ f->fmt.pix_mp.height = inst->fmt_src->fmt.pix_mp.height;
+ f->fmt.pix_mp.width = inst->fmt_src->fmt.pix_mp.width;
+ }
+ fmt->fmt.pix_mp.pixelformat = f->fmt.pix_mp.pixelformat;
+ codec_align = f->fmt.pix_mp.pixelformat == V4L2_PIX_FMT_QC10C ? 192 : 128;
+ fmt->fmt.pix_mp.width = ALIGN(f->fmt.pix_mp.width, codec_align);
+ codec_align = f->fmt.pix_mp.pixelformat == V4L2_PIX_FMT_QC10C ? 16 : 32;
+ fmt->fmt.pix_mp.height = ALIGN(f->fmt.pix_mp.height, codec_align);
+ fmt->fmt.pix_mp.num_planes = 1;
+ if (f->fmt.pix_mp.pixelformat == V4L2_PIX_FMT_QC10C) {
+ stride = ALIGN(f->fmt.pix_mp.width, 192);
+ fmt->fmt.pix_mp.plane_fmt[0].bytesperline = ALIGN(stride * 4 / 3, 256);
+ } else {
+ fmt->fmt.pix_mp.plane_fmt[0].bytesperline = ALIGN(f->fmt.pix_mp.width, 128);
+ }
+ fmt->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_OUTPUT);
+
+ if (!inst->vb2q_src->streaming)
+ inst->buffers.output.min_count = iris_get_buf_min_count(inst, BUF_OUTPUT);
+ if (inst->buffers.output.actual_count <
+ inst->buffers.output.min_count) {
+ inst->buffers.output.actual_count =
+ inst->buffers.output.min_count;
+ }
+
+ colorformat = v4l2_colorformat_to_driver(inst, fmt->fmt.pix_mp.pixelformat);
+ inst->buffers.output.size =
+ fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+ inst->cap[PIX_FMTS].value = colorformat;
+
+ if (!inst->vb2q_src->streaming) {
+ inst->crop.top = 0;
+ inst->crop.left = 0;
+ inst->crop.width = f->fmt.pix_mp.width;
+ inst->crop.height = f->fmt.pix_mp.height;
+ }
+ } else {
+ return -EINVAL;
+ }
+ memcpy(f, fmt, sizeof(*fmt));
+
+ return ret;
+}
+
+int vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscription *sub)
+{
+ int ret = 0;
+
+ switch (sub->type) {
+ case V4L2_EVENT_EOS:
+ ret = v4l2_event_subscribe(&inst->fh, sub, MAX_EVENTS, NULL);
+ break;
+ case V4L2_EVENT_SOURCE_CHANGE:
+ ret = v4l2_src_change_event_subscribe(&inst->fh, sub);
+ break;
+ case V4L2_EVENT_CTRL:
+ ret = v4l2_ctrl_subscribe_event(&inst->fh, sub);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
index dc8f43f..a2f159d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
@@ -8,7 +8,11 @@

#include "iris_instance.h"

-void vdec_inst_init(struct iris_inst *inst);
+int vdec_inst_init(struct iris_inst *inst);
void vdec_inst_deinit(struct iris_inst *inst);
+int vdec_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f);
+int vdec_try_fmt(struct iris_inst *inst, struct v4l2_format *f);
+int vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f);
+int vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscription *sub);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
index 68ba75f..5c76821 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
@@ -3,6 +3,11 @@
* Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
*/

+#include <linux/videodev2.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-ioctl.h>
+
+#include "iris_buffer.h"
#include "iris_common.h"
#include "iris_helpers.h"
#include "iris_hfi.h"
@@ -12,6 +17,9 @@
#include "iris_ctrls.h"
#include "iris_vb2.h"

+#define VIDC_DRV_NAME "iris_driver"
+#define VIDC_BUS_NAME "platform:iris_bus"
+
static int vidc_v4l2_fh_init(struct iris_inst *inst)
{
struct iris_core *core;
@@ -169,6 +177,7 @@ int vidc_open(struct file *filp)
inst->core = core;
inst->session_id = hash32_ptr(inst);
iris_inst_change_state(inst, IRIS_INST_OPEN);
+ mutex_init(&inst->lock);
mutex_init(&inst->ctx_q_lock);

ret = vidc_add_session(inst);
@@ -200,14 +209,6 @@ int vidc_open(struct file *filp)
if (ret)
goto fail_inst_deinit;

- ret = get_inst_capability(inst);
- if (ret)
- goto fail_queue_deinit;
-
- ret = ctrls_init(inst);
- if (ret)
- goto fail_queue_deinit;
-
ret = iris_hfi_session_open(inst);
if (ret) {
dev_err(core->dev, "%s: session open failed\n", __func__);
@@ -220,7 +221,6 @@ int vidc_open(struct file *filp)
fail_core_deinit:
v4l2_ctrl_handler_free(&inst->ctrl_handler);
iris_core_deinit(core);
-fail_queue_deinit:
vidc_vb2_queue_deinit(inst);
fail_inst_deinit:
vdec_inst_deinit(inst);
@@ -229,6 +229,7 @@ int vidc_open(struct file *filp)
vidc_remove_session(inst);
fail_free_inst:
mutex_destroy(&inst->ctx_q_lock);
+ mutex_destroy(&inst->lock);
kfree(inst);

return ret;
@@ -244,14 +245,16 @@ int vidc_close(struct file *filp)

v4l2_ctrl_handler_free(&inst->ctrl_handler);
vdec_inst_deinit(inst);
+ mutex_lock(&inst->lock);
close_session(inst);
iris_inst_change_state(inst, IRIS_INST_CLOSE);
vidc_vb2_queue_deinit(inst);
vidc_v4l2_fh_deinit(inst);
vidc_remove_session(inst);
+ mutex_unlock(&inst->lock);
mutex_destroy(&inst->ctx_q_lock);
+ mutex_destroy(&inst->lock);
kfree(inst);
-
filp->private_data = NULL;

return 0;
@@ -313,6 +316,588 @@ static __poll_t vidc_poll(struct file *filp, struct poll_table_struct *pt)
return poll;
}

+static int vidc_enum_fmt(struct file *filp, void *fh, struct v4l2_fmtdesc *f)
+{
+ struct iris_inst *inst;
+ int ret;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ ret = vdec_enum_fmt(inst, f);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_try_fmt(struct file *filp, void *fh, struct v4l2_format *f)
+{
+ struct iris_inst *inst;
+ int ret;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (!allow_s_fmt(inst, f->type)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ ret = vdec_try_fmt(inst, f);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_s_fmt(struct file *filp, void *fh, struct v4l2_format *f)
+{
+ struct iris_inst *inst;
+ int ret;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (!allow_s_fmt(inst, f->type)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ ret = vdec_s_fmt(inst, f);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_g_fmt(struct file *filp, void *fh, struct v4l2_format *f)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (f->type == INPUT_MPLANE)
+ memcpy(f, inst->fmt_src, sizeof(*f));
+ else if (f->type == OUTPUT_MPLANE)
+ memcpy(f, inst->fmt_dst, sizeof(*f));
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_enum_framesizes(struct file *filp, void *fh,
+ struct v4l2_frmsizeenum *fsize)
+{
+ enum colorformat_type colorfmt;
+ struct iris_inst *inst;
+ enum codec_type codec;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !fsize)
+ return -EINVAL;
+
+ if (fsize->index)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ codec = v4l2_codec_to_driver(inst, fsize->pixel_format);
+ if (!codec) {
+ colorfmt = v4l2_colorformat_to_driver(inst, fsize->pixel_format);
+ if (colorfmt == FMT_NONE) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+ }
+
+ fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
+ fsize->stepwise.min_width = inst->cap[FRAME_WIDTH].min;
+ fsize->stepwise.max_width = inst->cap[FRAME_WIDTH].max;
+ fsize->stepwise.step_width = inst->cap[FRAME_WIDTH].step_or_mask;
+ fsize->stepwise.min_height = inst->cap[FRAME_HEIGHT].min;
+ fsize->stepwise.max_height = inst->cap[FRAME_HEIGHT].max;
+ fsize->stepwise.step_height = inst->cap[FRAME_HEIGHT].step_or_mask;
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_reqbufs(struct file *filp, void *fh, struct v4l2_requestbuffers *b)
+{
+ struct vb2_queue *vb2q = NULL;
+ struct iris_inst *inst;
+ int ret;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !b)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (!allow_reqbufs(inst, b->type)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ vb2q = get_vb2q(inst, b->type);
+ if (!vb2q) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ ret = vb2_reqbufs(vb2q, b);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_querybuf(struct file *filp, void *fh, struct v4l2_buffer *b)
+{
+ struct vb2_queue *vb2q = NULL;
+ struct iris_inst *inst;
+ int ret;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !b)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ vb2q = get_vb2q(inst, b->type);
+ if (!vb2q) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ ret = vb2_querybuf(vb2q, b);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_create_bufs(struct file *filp, void *fh, struct v4l2_create_buffers *b)
+{
+ struct iris_inst *inst;
+ struct vb2_queue *vb2q;
+ struct v4l2_format *f;
+ int ret;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !b)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ f = &b->format;
+ vb2q = get_vb2q(inst, f->type);
+ if (!vb2q) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ ret = vb2_create_bufs(vb2q, b);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_prepare_buf(struct file *filp, void *fh, struct v4l2_buffer *b)
+{
+ struct video_device *vdev;
+ struct iris_inst *inst;
+ struct vb2_queue *vb2q;
+ int ret;
+
+ inst = get_vidc_inst(filp, fh);
+ vdev = video_devdata(filp);
+ if (!inst || !vdev || !vdev->v4l2_dev->mdev)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ vb2q = get_vb2q(inst, b->type);
+ if (!vb2q) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ ret = vb2_prepare_buf(vb2q, vdev->v4l2_dev->mdev, b);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_qbuf(struct file *filp, void *fh, struct v4l2_buffer *b)
+{
+ struct video_device *vdev;
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ vdev = video_devdata(filp);
+ if (!inst || !b)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (b->type == INPUT_MPLANE)
+ ret = vb2_qbuf(inst->vb2q_src, vdev->v4l2_dev->mdev, b);
+ else if (b->type == OUTPUT_MPLANE)
+ ret = vb2_qbuf(inst->vb2q_dst, vdev->v4l2_dev->mdev, b);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_dqbuf(struct file *filp, void *fh, struct v4l2_buffer *b)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !b)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (b->type == INPUT_MPLANE)
+ ret = vb2_dqbuf(inst->vb2q_src, b, true);
+ else if (b->type == OUTPUT_MPLANE)
+ ret = vb2_dqbuf(inst->vb2q_dst, b, true);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_streamon(struct file *filp, void *fh, enum v4l2_buf_type type)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (!allow_streamon(inst, type)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (type == INPUT_MPLANE)
+ ret = vb2_streamon(inst->vb2q_src, type);
+ else if (type == OUTPUT_MPLANE)
+ ret = vb2_streamon(inst->vb2q_dst, type);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_streamoff(struct file *filp, void *fh, enum v4l2_buf_type type)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (!allow_streamoff(inst, type)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (type == INPUT_MPLANE)
+ ret = vb2_streamoff(inst->vb2q_src, type);
+ else if (type == OUTPUT_MPLANE)
+ ret = vb2_streamoff(inst->vb2q_dst, type);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_querycap(struct file *filp, void *fh, struct v4l2_capability *cap)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ strscpy(cap->driver, VIDC_DRV_NAME, sizeof(cap->driver));
+ strscpy(cap->bus_info, VIDC_BUS_NAME, sizeof(cap->bus_info));
+ memset(cap->reserved, 0, sizeof(cap->reserved));
+ strscpy(cap->card, "iris_decoder", sizeof(cap->card));
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_queryctrl(struct file *filp, void *fh, struct v4l2_queryctrl *q_ctrl)
+{
+ struct v4l2_ctrl *ctrl;
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !q_ctrl)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ ctrl = v4l2_ctrl_find(&inst->ctrl_handler, q_ctrl->id);
+ if (!ctrl) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ q_ctrl->minimum = ctrl->minimum;
+ q_ctrl->maximum = ctrl->maximum;
+ q_ctrl->default_value = ctrl->default_value;
+ q_ctrl->flags = 0;
+ q_ctrl->step = ctrl->step;
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_querymenu(struct file *filp, void *fh, struct v4l2_querymenu *qmenu)
+{
+ struct v4l2_ctrl *ctrl;
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !qmenu)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ ctrl = v4l2_ctrl_find(&inst->ctrl_handler, qmenu->id);
+ if (!ctrl) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (ctrl->type != V4L2_CTRL_TYPE_MENU) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (qmenu->index < ctrl->minimum || qmenu->index > ctrl->maximum) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (ctrl->menu_skip_mask & (1 << qmenu->index)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_subscribe_event(struct v4l2_fh *fh, const struct v4l2_event_subscription *sub)
+{
+ struct iris_inst *inst;
+ int ret;
+
+ inst = container_of(fh, struct iris_inst, fh);
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ ret = vdec_subscribe_event(inst, sub);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_unsubscribe_event(struct v4l2_fh *fh, const struct v4l2_event_subscription *sub)
+{
+ struct iris_inst *inst;
+ int ret;
+
+ inst = container_of(fh, struct iris_inst, fh);
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ ret = v4l2_event_unsubscribe(&inst->fh, sub);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_g_selection(struct file *filp, void *fh, struct v4l2_selection *s)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !s)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (s->type != OUTPUT_MPLANE && s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ switch (s->target) {
+ case V4L2_SEL_TGT_CROP_BOUNDS:
+ case V4L2_SEL_TGT_CROP_DEFAULT:
+ case V4L2_SEL_TGT_CROP:
+ case V4L2_SEL_TGT_COMPOSE_BOUNDS:
+ case V4L2_SEL_TGT_COMPOSE_PADDED:
+ case V4L2_SEL_TGT_COMPOSE_DEFAULT:
+ case V4L2_SEL_TGT_COMPOSE:
+ s->r.left = inst->crop.left;
+ s->r.top = inst->crop.top;
+ s->r.width = inst->crop.width;
+ s->r.height = inst->crop.height;
+ break;
+ default:
+ ret = -EINVAL;
+ }
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
static const struct v4l2_file_operations v4l2_file_ops = {
.owner = THIS_MODULE,
.open = vidc_open,
@@ -335,11 +920,38 @@ static struct vb2_mem_ops iris_vb2_mem_ops = {
.unmap_dmabuf = iris_vb2_unmap_dmabuf,
};

+static const struct v4l2_ioctl_ops v4l2_ioctl_ops = {
+ .vidioc_enum_fmt_vid_cap = vidc_enum_fmt,
+ .vidioc_enum_fmt_vid_out = vidc_enum_fmt,
+ .vidioc_try_fmt_vid_cap_mplane = vidc_try_fmt,
+ .vidioc_try_fmt_vid_out_mplane = vidc_try_fmt,
+ .vidioc_s_fmt_vid_cap_mplane = vidc_s_fmt,
+ .vidioc_s_fmt_vid_out_mplane = vidc_s_fmt,
+ .vidioc_g_fmt_vid_cap_mplane = vidc_g_fmt,
+ .vidioc_g_fmt_vid_out_mplane = vidc_g_fmt,
+ .vidioc_enum_framesizes = vidc_enum_framesizes,
+ .vidioc_reqbufs = vidc_reqbufs,
+ .vidioc_querybuf = vidc_querybuf,
+ .vidioc_create_bufs = vidc_create_bufs,
+ .vidioc_prepare_buf = vidc_prepare_buf,
+ .vidioc_qbuf = vidc_qbuf,
+ .vidioc_dqbuf = vidc_dqbuf,
+ .vidioc_streamon = vidc_streamon,
+ .vidioc_streamoff = vidc_streamoff,
+ .vidioc_querycap = vidc_querycap,
+ .vidioc_queryctrl = vidc_queryctrl,
+ .vidioc_querymenu = vidc_querymenu,
+ .vidioc_subscribe_event = vidc_subscribe_event,
+ .vidioc_unsubscribe_event = vidc_unsubscribe_event,
+ .vidioc_g_selection = vidc_g_selection,
+};
+
int init_ops(struct iris_core *core)
{
core->v4l2_file_ops = &v4l2_file_ops;
core->vb2_ops = &iris_vb2_ops;
core->vb2_mem_ops = &iris_vb2_mem_ops;
+ core->v4l2_ioctl_ops = &v4l2_ioctl_ops;

return 0;
}
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
index e242614..abd11fa 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
@@ -166,6 +166,23 @@ struct plat_inst_caps {
struct plat_inst_cap cap[INST_CAP_MAX + 1];
};

+struct codec_info {
+ u32 v4l2_codec;
+ enum codec_type codec;
+};
+
+struct color_format_info {
+ u32 v4l2_color_format;
+ enum colorformat_type color_format;
+};
+
+struct format_capability {
+ struct codec_info *codec_info;
+ u32 codec_info_size;
+ struct color_format_info *color_format_info;
+ u32 color_format_info_size;
+};
+
struct platform_data {
const struct bus_info *bus_tbl;
unsigned int bus_tbl_size;
@@ -182,6 +199,7 @@ struct platform_data {
const struct reg_preset_info *reg_prst_tbl;
unsigned int reg_prst_tbl_size;
struct ubwc_config_data *ubwc_config;
+ struct format_capability *format_data;
const char *fwname;
u32 pas_id;
struct plat_core_cap *core_data;
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
index 0759ac5..85bc677 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
@@ -19,6 +19,40 @@
#define MINIMUM_FPS 1
#define MAXIMUM_FPS 480

+static struct codec_info codec_data_sm8550[] = {
+ {
+ .v4l2_codec = V4L2_PIX_FMT_H264,
+ .codec = H264,
+ },
+ {
+ .v4l2_codec = V4L2_PIX_FMT_HEVC,
+ .codec = HEVC,
+ },
+ {
+ .v4l2_codec = V4L2_PIX_FMT_VP9,
+ .codec = VP9,
+ },
+};
+
+static struct color_format_info color_format_data_sm8550[] = {
+ {
+ .v4l2_color_format = V4L2_PIX_FMT_NV12,
+ .color_format = FMT_NV12,
+ },
+ {
+ .v4l2_color_format = V4L2_PIX_FMT_NV21,
+ .color_format = FMT_NV21,
+ },
+ {
+ .v4l2_color_format = V4L2_PIX_FMT_QC08C,
+ .color_format = FMT_NV12C,
+ },
+ {
+ .v4l2_color_format = V4L2_PIX_FMT_QC10C,
+ .color_format = FMT_TP10C,
+ },
+};
+
static struct plat_core_cap core_data_sm8550[] = {
{DEC_CODECS, H264 | HEVC | VP9},
{MAX_SESSION_COUNT, 16},
@@ -341,6 +375,13 @@ static struct ubwc_config_data ubwc_config_sm8550[] = {
UBWC_CONFIG(8, 32, 16, 0, 1, 1, 1),
};

+static struct format_capability format_data_sm8550 = {
+ .codec_info = codec_data_sm8550,
+ .codec_info_size = ARRAY_SIZE(codec_data_sm8550),
+ .color_format_info = color_format_data_sm8550,
+ .color_format_info_size = ARRAY_SIZE(color_format_data_sm8550),
+};
+
struct platform_data sm8550_data = {
.bus_tbl = sm8550_bus_table,
.bus_tbl_size = ARRAY_SIZE(sm8550_bus_table),
@@ -367,4 +408,5 @@ struct platform_data sm8550_data = {
.inst_cap_data = instance_cap_data_sm8550,
.inst_cap_data_size = ARRAY_SIZE(instance_cap_data_sm8550),
.ubwc_config = ubwc_config_sm8550,
+ .format_data = &format_data_sm8550,
};
--
2.7.4

2023-12-18 11:54:45

[permalink] [raw]

Subject: [PATCH v2 20/34] media: iris: add video hardware internal buffer count and size calculation

Video driver needs various kind of internal buffers for
frame processing. Internal buffer size calculation depends
on hardware architecture, color format, resolution and codec.

Add APIs to calculate min count and size of different
internal buffers for different codecs for iris3.

Signed-off-by: Dikshita Agarwal <[email protected]>
---
drivers/media/platform/qcom/vcodec/iris/Makefile | 1 +
.../media/platform/qcom/vcodec/iris/iris_buffer.c | 48 ++
.../media/platform/qcom/vcodec/iris/iris_common.h | 1 +
.../media/platform/qcom/vcodec/iris/iris_core.h | 2 +
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 13 +
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 4 +
.../platform/qcom/vcodec/iris/iris_instance.h | 2 +
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 1 +
.../media/platform/qcom/vcodec/iris/vpu_common.h | 8 +
.../media/platform/qcom/vcodec/iris/vpu_iris3.c | 6 +
.../platform/qcom/vcodec/iris/vpu_iris3_buffer.c | 201 +++++
.../platform/qcom/vcodec/iris/vpu_iris3_buffer.h | 845 +++++++++++++++++++++
12 files changed, 1132 insertions(+)
create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h

diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
index a94e36b..7e3d9f1 100644
--- a/drivers/media/platform/qcom/vcodec/iris/Makefile
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -16,6 +16,7 @@ iris-objs += iris_probe.o \
resources.o \
vpu_common.o \
vpu_iris3.o \
+ vpu_iris3_buffer.o \
platform_common.o \
platform_sm8550.o

diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
index b9cffbf..6d4e722 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
@@ -17,6 +17,14 @@ static int output_min_count(struct iris_inst *inst)
{
int output_min_count;

+ /* fw_min_count > 0 indicates reconfig event has already arrived */
+ if (inst->fw_min_count) {
+ if (is_split_mode_enabled(inst) && inst->codec == VP9)
+ return min_t(u32, 4, inst->fw_min_count);
+ else
+ return inst->fw_min_count;
+ }
+
switch (inst->codec) {
case H264:
case HEVC:
@@ -33,6 +41,38 @@ static int output_min_count(struct iris_inst *inst)
return output_min_count;
}

+static u32 internal_buffer_count(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type)
+{
+ u32 count = 0;
+
+ if (buffer_type == BUF_BIN || buffer_type == BUF_LINE ||
+ buffer_type == BUF_PERSIST) {
+ count = 1;
+ } else if (buffer_type == BUF_COMV || buffer_type == BUF_NON_COMV) {
+ if (inst->codec == H264 || inst->codec == HEVC)
+ count = 1;
+ else
+ count = 0;
+ } else {
+ count = 0;
+ }
+
+ return count;
+}
+
+static int dpb_count(struct iris_inst *inst)
+{
+ int count = 0;
+
+ if (is_split_mode_enabled(inst)) {
+ count = inst->fw_min_count ?
+ inst->fw_min_count : inst->buffers.output.min_count;
+ }
+
+ return count;
+}
+
int iris_get_buf_min_count(struct iris_inst *inst,
enum iris_buffer_type buffer_type)
{
@@ -41,6 +81,14 @@ int iris_get_buf_min_count(struct iris_inst *inst,
return input_min_count(inst);
case BUF_OUTPUT:
return output_min_count(inst);
+ case BUF_BIN:
+ case BUF_COMV:
+ case BUF_NON_COMV:
+ case BUF_LINE:
+ case BUF_PERSIST:
+ return internal_buffer_count(inst, buffer_type);
+ case BUF_DPB:
+ return dpb_count(inst);
default:
return 0;
}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
index 4edadc3..a83d1c1 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
@@ -15,6 +15,7 @@
#define OUTPUT_MPLANE V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE
#define DEFAULT_WIDTH 320
#define DEFAULT_HEIGHT 240
+#define DEFAULT_BSE_VPP_DELAY 2

#define MB_IN_PIXEL (16 * 16)

diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index 30f7ad7..154991c 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -50,6 +50,7 @@
* @header_id: id of packet header
* @packet_id: id of packet
* @vpu_ops: a pointer to vpu ops
+ * @session_ops: a pointer to session level ops
* @dec_codecs_count: supported codec count for decoder
* @platform_data: a structure for platform data
* @cap: an array for supported core capabilities
@@ -91,6 +92,7 @@ struct iris_core {
u32 header_id;
u32 packet_id;
const struct vpu_ops *vpu_ops;
+ const struct vpu_session_ops *session_ops;
u32 dec_codecs_count;
struct platform_data *platform_data;
struct plat_core_cap cap[CORE_CAP_MAX + 1];
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index 54a7851..335885f 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -66,6 +66,19 @@ int get_mbpf(struct iris_inst *inst)
return NUM_MBS_PER_FRAME(height, width);
}

+bool is_linear_colorformat(u32 colorformat)
+{
+ return colorformat == V4L2_PIX_FMT_NV12 || colorformat == V4L2_PIX_FMT_NV21;
+}
+
+bool is_split_mode_enabled(struct iris_inst *inst)
+{
+ if (is_linear_colorformat(inst->fmt_dst->fmt.pix_mp.pixelformat))
+ return true;
+
+ return false;
+}
+
static int process_inst_timeout(struct iris_inst *inst)
{
struct iris_inst *instance;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
index 3bae969..9e85510 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -12,6 +12,7 @@
#include "iris_core.h"
#include "iris_instance.h"
#include "iris_buffer.h"
+#include "iris_instance.h"
#include "platform_common.h"

#define NUM_MBS_PER_FRAME(__height, __width) \
@@ -26,4 +27,7 @@ enum iris_buffer_type v4l2_type_to_driver(u32 type);
int get_mbpf(struct iris_inst *inst);
int close_session(struct iris_inst *inst);

+bool is_linear_colorformat(u32 colorformat);
+bool is_split_mode_enabled(struct iris_inst *inst);
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
index a5c6cb48..5d4c856 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -36,6 +36,7 @@
* @caps_list: list head of capability
* @codec: codec type
* @buffers: structure of buffer info
+ * @fw_min_count: minimnum count of buffers needed by fw
*/

struct iris_inst {
@@ -58,6 +59,7 @@ struct iris_inst {
struct list_head caps_list;
enum codec_type codec;
struct iris_buffers_info buffers;
+ u32 fw_min_count;
};

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index 1b957a13..b131a50 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -39,6 +39,7 @@ void vdec_inst_init(struct iris_inst *inst)
f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT;
f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+ inst->fw_min_count = 0;
}

void vdec_inst_deinit(struct iris_inst *inst)
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h
index 6512039..7fba8c2 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h
@@ -26,6 +26,14 @@ struct vpu_ops {
int (*watchdog)(struct iris_core *core, u32 intr_status);
};

+#define call_session_op(c, op, ...) \
+ (((c) && (c)->session_ops && (c)->session_ops->op) ? \
+ ((c)->session_ops->op(__VA_ARGS__)) : 0)
+
+struct vpu_session_ops {
+ int (*int_buf_size)(struct iris_inst *inst, enum iris_buffer_type type);
+};
+
int init_vpu(struct iris_core *core);

int write_register(struct iris_core *core, u32 reg, u32 value);
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
index a34d0ed..efea5aa 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
@@ -6,6 +6,7 @@
#include <linux/delay.h>

#include "vpu_iris3.h"
+#include "vpu_iris3_buffer.h"

#define VIDEO_ARCH_LX 1

@@ -202,9 +203,14 @@ static const struct vpu_ops iris3_ops = {
.watchdog = watchdog_iris3,
};

+static const struct vpu_session_ops iris3_session_ops = {
+ .int_buf_size = iris_int_buf_size_iris3,
+};
+
int init_iris3(struct iris_core *core)
{
core->vpu_ops = &iris3_ops;
+ core->session_ops = &iris3_session_ops;

return 0;
}
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c
new file mode 100644
index 0000000..44f9342
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c
@@ -0,0 +1,201 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include "iris_core.h"
+#include "iris_helpers.h"
+#include "iris_instance.h"
+#include "vpu_iris3_buffer.h"
+
+static u32 dec_bin_size_iris3(struct iris_inst *inst)
+{
+ u32 width, height, num_vpp_pipes;
+ struct iris_core *core;
+ struct v4l2_format *f;
+ u32 size = 0;
+
+ core = inst->core;
+
+ num_vpp_pipes = core->cap[NUM_VPP_PIPE].value;
+
+ f = inst->fmt_src;
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+
+ if (inst->codec == H264)
+ size = hfi_buffer_bin_h264d(width, height, num_vpp_pipes);
+ else if (inst->codec == HEVC)
+ size = hfi_buffer_bin_h265d(width, height, num_vpp_pipes);
+ else if (inst->codec == VP9)
+ size = hfi_buffer_bin_vp9d(width, height,
+ num_vpp_pipes);
+ return size;
+}
+
+static u32 dec_comv_size_iris3(struct iris_inst *inst)
+{
+ u32 width, height, num_comv;
+ struct v4l2_format *f;
+ u32 size = 0;
+
+ f = inst->fmt_src;
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+
+ num_comv = inst->buffers.output.min_count;
+
+ if (inst->codec == H264)
+ size = hfi_buffer_comv_h264d(width, height, num_comv);
+ else if (inst->codec == HEVC)
+ size = hfi_buffer_comv_h265d(width, height, num_comv);
+
+ inst->cap[NUM_COMV].value = num_comv;
+
+ return size;
+}
+
+static u32 dec_non_comv_size_iris3(struct iris_inst *inst)
+{
+ u32 width, height, num_vpp_pipes;
+ struct iris_core *core;
+ struct v4l2_format *f;
+ u32 size = 0;
+
+ core = inst->core;
+
+ num_vpp_pipes = core->cap[NUM_VPP_PIPE].value;
+
+ f = inst->fmt_src;
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+
+ if (inst->codec == H264)
+ size = hfi_buffer_non_comv_h264d(width, height, num_vpp_pipes);
+ else if (inst->codec == HEVC)
+ size = hfi_buffer_non_comv_h265d(width, height, num_vpp_pipes);
+
+ return size;
+}
+
+static u32 dec_line_size_iris3(struct iris_inst *inst)
+{
+ u32 width, height, out_min_count, num_vpp_pipes;
+ struct iris_core *core;
+ struct v4l2_format *f;
+ u32 size = 0;
+ bool is_opb;
+
+ core = inst->core;
+ num_vpp_pipes = core->cap[NUM_VPP_PIPE].value;
+
+ is_opb = true;
+
+ f = inst->fmt_src;
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+ out_min_count = inst->buffers.output.min_count;
+ if (inst->codec == H264)
+ size = hfi_buffer_line_h264d(width, height, is_opb,
+ num_vpp_pipes);
+ else if (inst->codec == HEVC)
+ size = hfi_buffer_line_h265d(width, height, is_opb,
+ num_vpp_pipes);
+ else if (inst->codec == VP9)
+ size = hfi_buffer_line_vp9d(width, height, out_min_count,
+ is_opb, num_vpp_pipes);
+ return size;
+}
+
+static u32 dec_persist_size_iris3(struct iris_inst *inst)
+{
+ u32 size = 0;
+
+ if (inst->codec == H264)
+ size = hfi_buffer_persist_h264d(0);
+ else if (inst->codec == HEVC)
+ size = hfi_buffer_persist_h265d(0);
+ else if (inst->codec == VP9)
+ size = hfi_buffer_persist_vp9d();
+
+ return size;
+}
+
+static u32 dec_dpb_size_iris3(struct iris_inst *inst)
+{
+ struct v4l2_format *f;
+ u32 width, height;
+ u32 color_fmt;
+ u32 size = 0;
+
+ f = inst->fmt_dst;
+ color_fmt = f->fmt.pix_mp.pixelformat;
+ if (!is_linear_colorformat(color_fmt))
+ return size;
+
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+
+ if (color_fmt == V4L2_PIX_FMT_NV12 ||
+ color_fmt == V4L2_PIX_FMT_QC08C) {
+ size =
+ hfi_nv12_ubwc_il_calc_buf_size_v2(width, height,
+ ALIGN(width, 128),
+ ALIGN(height, 32),
+ ALIGN(width, 128),
+ ALIGN((height + 1) >> 1, 32),
+ ALIGN(DIV_ROUND_UP(width, 32), 64),
+ ALIGN(DIV_ROUND_UP(height, 8), 16),
+ ALIGN(DIV_ROUND_UP((width + 1) >> 1, 16), 64),
+ ALIGN(DIV_ROUND_UP((height + 1) >> 1, 8), 16));
+ } else if (color_fmt == V4L2_PIX_FMT_QC10C) {
+ size =
+ hfi_yuv420_tp10_ubwc_calc_buf_size(ALIGN(ALIGN(width, 192) * 4 / 3, 256),
+ ALIGN(height, 16),
+ ALIGN(ALIGN(width, 192) * 4 / 3, 256),
+ ALIGN((height + 1) >> 1, 16),
+ ALIGN(DIV_ROUND_UP(width, 48), 64),
+ ALIGN(DIV_ROUND_UP(height, 4), 16),
+ ALIGN(DIV_ROUND_UP((width + 1) >> 1, 24), 64),
+ ALIGN(DIV_ROUND_UP((height + 1) >> 1, 4), 16));
+ }
+
+ return size;
+}
+
+struct iris_buf_type_handle {
+ enum iris_buffer_type type;
+ u32 (*handle)(struct iris_inst *inst);
+};
+
+int iris_int_buf_size_iris3(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type)
+{
+ const struct iris_buf_type_handle *buf_type_handle_arr = NULL;
+ u32 size = 0, buf_type_handle_size = 0;
+ int i;
+
+ static const struct iris_buf_type_handle dec_internal_buf_type_handle[] = {
+ {BUF_BIN, dec_bin_size_iris3 },
+ {BUF_COMV, dec_comv_size_iris3 },
+ {BUF_NON_COMV, dec_non_comv_size_iris3 },
+ {BUF_LINE, dec_line_size_iris3 },
+ {BUF_PERSIST, dec_persist_size_iris3 },
+ {BUF_DPB, dec_dpb_size_iris3 },
+ };
+
+ buf_type_handle_size = ARRAY_SIZE(dec_internal_buf_type_handle);
+ buf_type_handle_arr = dec_internal_buf_type_handle;
+
+ if (!buf_type_handle_arr || !buf_type_handle_size)
+ return size;
+
+ for (i = 0; i < buf_type_handle_size; i++) {
+ if (buf_type_handle_arr[i].type == buffer_type) {
+ size = buf_type_handle_arr[i].handle(inst);
+ break;
+ }
+ }
+
+ return size;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h
new file mode 100644
index 0000000..b520c79
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h
@@ -0,0 +1,845 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _VPU_IRIS3_BUFFER_H_
+#define _VPU_IRIS3_BUFFER_H_
+
+#include <linux/types.h>
+#include <linux/minmax.h>
+#include <linux/align.h>
+
+#include "iris_instance.h"
+
+#define DMA_ALIGNMENT 256
+
+#define BUFFER_ALIGNMENT_512_BYTES 512
+#define BUFFER_ALIGNMENT_256_BYTES 256
+#define BUFFER_ALIGNMENT_64_BYTES 64
+#define BUFFER_ALIGNMENT_32_BYTES 32
+#define BUFFER_ALIGNMENT_16_BYTES 16
+
+#define HFI_ALIGNMENT_4096 (4096)
+
+#define HFI_COL_FMT_NV12C_Y_TILE_HEIGHT (8)
+#define HFI_COL_FMT_NV12C_Y_TILE_WIDTH (32)
+#define HFI_COL_FMT_NV12C_UV_TILE_HEIGHT (8)
+#define HFI_COL_FMT_NV12C_UV_TILE_WIDTH (16)
+
+#define NUM_HW_PIC_BUF 32
+#define SIZE_HW_PIC(size_per_buf) (NUM_HW_PIC_BUF * (size_per_buf))
+
+#define MAX_TILE_COLUMNS 32
+
+#define LCU_MAX_SIZE_PELS 64
+#define LCU_MIN_SIZE_PELS 16
+
+#define HDR10_HIST_EXTRADATA_SIZE (4 * 1024)
+
+#define BIN_BUFFER_THRESHOLD (1280 * 736)
+
+#define VPP_CMD_MAX_SIZE (BIT(20))
+
+#define H264D_MAX_SLICE 1800
+
+#define SIZE_H264D_BUFTAB_T (256)
+#define SIZE_H264D_HW_PIC_T (BIT(11))
+#define SIZE_H264D_BSE_CMD_PER_BUF (32 * 4)
+#define SIZE_H264D_VPP_CMD_PER_BUF (512)
+
+#define NUM_SLIST_BUF_H264 (256 + 32)
+#define SIZE_SLIST_BUF_H264 (512)
+#define H264_DISPLAY_BUF_SIZE (3328)
+#define H264_NUM_FRM_INFO (66)
+
+#define H265_NUM_TILE_COL 32
+#define H265_NUM_TILE_ROW 128
+#define H265_NUM_TILE (H265_NUM_TILE_ROW * H265_NUM_TILE_COL + 1)
+#define SIZE_H265D_BSE_CMD_PER_BUF (16 * sizeof(u32))
+
+#define NUM_SLIST_BUF_H265 (80 + 20)
+#define SIZE_SLIST_BUF_H265 (BIT(10))
+#define H265_DISPLAY_BUF_SIZE (3072)
+#define H265_NUM_FRM_INFO (48)
+
+#define VP9_NUM_FRAME_INFO_BUF 32
+#define VP9_NUM_PROBABILITY_TABLE_BUF (VP9_NUM_FRAME_INFO_BUF + 4)
+#define VP9_PROB_TABLE_SIZE (3840)
+#define VP9_FRAME_INFO_BUF_SIZE (6144)
+
+#define VP9_UDC_HEADER_BUF_SIZE (3 * 128)
+#define MAX_SUPERFRAME_HEADER_LEN (34)
+#define CCE_TILE_OFFSET_SIZE ALIGN(32 * 4 * 4, BUFFER_ALIGNMENT_32_BYTES)
+
+#define SIZE_SEI_USERDATA (4096)
+#define SIZE_DOLBY_RPU_METADATA (41 * 1024)
+
+#define H264_CABAC_HDR_RATIO_HD_TOT 1
+#define H264_CABAC_RES_RATIO_HD_TOT 3
+
+#define H265D_MAX_SLICE 1200
+#define SIZE_H265D_HW_PIC_T SIZE_H264D_HW_PIC_T
+#define H265_CABAC_HDR_RATIO_HD_TOT 2
+#define H265_CABAC_RES_RATIO_HD_TOT 2
+#define SIZE_H265D_VPP_CMD_PER_BUF (256)
+
+#define VPX_DECODER_FRAME_CONCURENCY_LVL (2)
+#define VPX_DECODER_FRAME_BIN_HDR_BUDGET 1
+#define VPX_DECODER_FRAME_BIN_RES_BUDGET 3
+#define VPX_DECODER_FRAME_BIN_DENOMINATOR 2
+
+#define VPX_DECODER_FRAME_BIN_RES_BUDGET_RATIO (3 / 2)
+
+#define MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE 64
+#define MAX_FE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE 64
+#define MAX_FE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE 64
+
+#define MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE (128 / 8)
+#define MAX_SE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE (128 / 8)
+#define MAX_SE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE (128 / 8)
+
+#define MAX_PE_NBR_DATA_LCU64_LINE_BUFFER_SIZE (64 * 2 * 3)
+#define MAX_FE_NBR_DATA_LUMA_LINE_BUFFER_SIZE 640
+
+static inline
+u32 size_h264d_hw_bin_buffer(u32 frame_width, u32 frame_height,
+ u32 num_vpp_pipes)
+{
+ u32 size_yuv, size_bin_hdr, size_bin_res;
+
+ size_yuv = ((frame_width * frame_height) <= BIN_BUFFER_THRESHOLD) ?
+ ((BIN_BUFFER_THRESHOLD * 3) >> 1) :
+ ((frame_width * frame_height * 3) >> 1);
+ size_bin_hdr = size_yuv * H264_CABAC_HDR_RATIO_HD_TOT;
+ size_bin_res = size_yuv * H264_CABAC_RES_RATIO_HD_TOT;
+ size_bin_hdr = ALIGN(size_bin_hdr / num_vpp_pipes,
+ DMA_ALIGNMENT) * num_vpp_pipes;
+ size_bin_res = ALIGN(size_bin_res / num_vpp_pipes,
+ DMA_ALIGNMENT) * num_vpp_pipes;
+
+ return size_bin_hdr + size_bin_res;
+}
+
+static inline
+u32 hfi_buffer_bin_h264d(u32 frame_width, u32 frame_height,
+ u32 num_vpp_pipes)
+{
+ u32 n_aligned_w, n_aligned_h;
+
+ n_aligned_w =
+ ALIGN(frame_width, BUFFER_ALIGNMENT_16_BYTES);
+ n_aligned_h =
+ ALIGN(frame_height, BUFFER_ALIGNMENT_16_BYTES);
+
+ return size_h264d_hw_bin_buffer(n_aligned_w, n_aligned_h,
+ num_vpp_pipes);
+}
+
+static inline
+u32 size_h265d_hw_bin_buffer(u32 frame_width, u32 frame_height,
+ u32 num_vpp_pipes)
+{
+ u32 size_yuv, size_bin_hdr, size_bin_res;
+
+ size_yuv = ((frame_width * frame_height) <= BIN_BUFFER_THRESHOLD) ?
+ ((BIN_BUFFER_THRESHOLD * 3) >> 1) :
+ ((frame_width * frame_height * 3) >> 1);
+ size_bin_hdr = size_yuv * H265_CABAC_HDR_RATIO_HD_TOT;
+ size_bin_res = size_yuv * H265_CABAC_RES_RATIO_HD_TOT;
+ size_bin_hdr = ALIGN(size_bin_hdr / num_vpp_pipes, DMA_ALIGNMENT) *
+ num_vpp_pipes;
+ size_bin_res = ALIGN(size_bin_res / num_vpp_pipes, DMA_ALIGNMENT) *
+ num_vpp_pipes;
+
+ return size_bin_hdr + size_bin_res;
+}
+
+static inline
+u32 hfi_buffer_bin_h265d(u32 frame_width, u32 frame_height,
+ u32 num_vpp_pipes)
+{
+ u32 n_aligned_w, n_aligned_h;
+
+ n_aligned_w = ALIGN(frame_width, BUFFER_ALIGNMENT_16_BYTES);
+ n_aligned_h = ALIGN(frame_height, BUFFER_ALIGNMENT_16_BYTES);
+ return size_h265d_hw_bin_buffer(n_aligned_w, n_aligned_h,
+ num_vpp_pipes);
+}
+
+static inline
+u32 hfi_buffer_bin_vp9d(u32 frame_width, u32 frame_height,
+ u32 num_vpp_pipes)
+{
+ u32 _size_yuv, _size;
+
+ _size_yuv = ALIGN(frame_width, BUFFER_ALIGNMENT_16_BYTES) *
+ ALIGN(frame_height, BUFFER_ALIGNMENT_16_BYTES) * 3 / 2;
+ _size = ALIGN(((max_t(u32, _size_yuv, ((BIN_BUFFER_THRESHOLD * 3) >> 1)) *
+ VPX_DECODER_FRAME_BIN_HDR_BUDGET / VPX_DECODER_FRAME_BIN_DENOMINATOR *
+ VPX_DECODER_FRAME_CONCURENCY_LVL) / num_vpp_pipes),
+ DMA_ALIGNMENT) +
+ ALIGN(((max_t(u32, _size_yuv, ((BIN_BUFFER_THRESHOLD * 3) >> 1)) *
+ VPX_DECODER_FRAME_BIN_RES_BUDGET / VPX_DECODER_FRAME_BIN_DENOMINATOR *
+ VPX_DECODER_FRAME_CONCURENCY_LVL) / num_vpp_pipes),
+ DMA_ALIGNMENT);
+
+ return _size * num_vpp_pipes;
+}
+
+static inline
+u32 hfi_buffer_comv_h264d(u32 frame_width, u32 frame_height,
+ u32 _comv_bufcount)
+{
+ u32 frame_width_in_mbs = ((frame_width + 15) >> 4);
+ u32 frame_height_in_mbs = ((frame_height + 15) >> 4);
+ u32 col_mv_aligned_width = (frame_width_in_mbs << 7);
+ u32 col_zero_aligned_width = (frame_width_in_mbs << 2);
+ u32 col_zero_size = 0, size_colloc = 0;
+
+ col_mv_aligned_width =
+ ALIGN(col_mv_aligned_width, BUFFER_ALIGNMENT_16_BYTES);
+ col_zero_aligned_width =
+ ALIGN(col_zero_aligned_width, BUFFER_ALIGNMENT_16_BYTES);
+ col_zero_size = col_zero_aligned_width *
+ ((frame_height_in_mbs + 1) >> 1);
+ col_zero_size =
+ ALIGN(col_zero_size, BUFFER_ALIGNMENT_64_BYTES);
+ col_zero_size <<= 1;
+ col_zero_size =
+ ALIGN(col_zero_size, BUFFER_ALIGNMENT_512_BYTES);
+ size_colloc = col_mv_aligned_width *
+ ((frame_height_in_mbs + 1) >> 1);
+ size_colloc =
+ ALIGN(size_colloc, BUFFER_ALIGNMENT_64_BYTES);
+ size_colloc <<= 1;
+ size_colloc =
+ ALIGN(size_colloc, BUFFER_ALIGNMENT_512_BYTES);
+ size_colloc += (col_zero_size + SIZE_H264D_BUFTAB_T * 2);
+
+ return (size_colloc * (_comv_bufcount)) +
+ BUFFER_ALIGNMENT_512_BYTES;
+}
+
+static inline
+u32 hfi_buffer_comv_h265d(u32 frame_width, u32 frame_height,
+ u32 _comv_bufcount)
+{
+ u32 _size;
+
+ _size = ALIGN(((((frame_width + 15) >> 4) *
+ ((frame_height + 15) >> 4)) << 8),
+ BUFFER_ALIGNMENT_512_BYTES);
+ _size *= _comv_bufcount;
+ _size += BUFFER_ALIGNMENT_512_BYTES;
+
+ return _size;
+}
+
+static inline
+u32 size_h264d_bse_cmd_buf(u32 frame_width, u32 frame_height)
+{
+ u32 _height = ALIGN(frame_height,
+ BUFFER_ALIGNMENT_32_BYTES);
+ return min_t(u32, (((_height + 15) >> 4) * 48), H264D_MAX_SLICE) *
+ SIZE_H264D_BSE_CMD_PER_BUF;
+}
+
+static inline
+u32 size_h264d_vpp_cmd_buf(u32 frame_width, u32 frame_height)
+{
+ u32 _size, _height;
+
+ _height = ALIGN(frame_height, BUFFER_ALIGNMENT_32_BYTES);
+ _size = min_t(u32, (((_height + 15) >> 4) * 48), H264D_MAX_SLICE) *
+ SIZE_H264D_VPP_CMD_PER_BUF;
+
+ if (_size > VPP_CMD_MAX_SIZE)
+ _size = VPP_CMD_MAX_SIZE;
+
+ return _size;
+}
+
+static inline
+u32 hfi_buffer_non_comv_h264d(u32 frame_width, u32 frame_height,
+ u32 num_vpp_pipes)
+{
+ u32 _size_bse, _size_vpp, _size;
+
+ _size_bse = size_h264d_bse_cmd_buf(frame_width, frame_height);
+ _size_vpp = size_h264d_vpp_cmd_buf(frame_width, frame_height);
+ _size = ALIGN(_size_bse, DMA_ALIGNMENT) +
+ ALIGN(_size_vpp, DMA_ALIGNMENT) +
+ ALIGN(SIZE_HW_PIC(SIZE_H264D_HW_PIC_T), DMA_ALIGNMENT);
+
+ return ALIGN(_size, DMA_ALIGNMENT);
+}
+
+static inline
+u32 size_h265d_bse_cmd_buf(u32 frame_width, u32 frame_height)
+{
+ u32 _size;
+
+ _size = ALIGN(((ALIGN(frame_width, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS) *
+ (ALIGN(frame_height, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS)) *
+ NUM_HW_PIC_BUF, DMA_ALIGNMENT);
+ _size = min_t(u32, _size, H265D_MAX_SLICE + 1);
+ _size = 2 * _size * SIZE_H265D_BSE_CMD_PER_BUF;
+
+ return _size;
+}
+
+static inline
+u32 size_h265d_vpp_cmd_buf(u32 frame_width, u32 frame_height)
+{
+ u32 _size;
+
+ _size = ALIGN(((ALIGN(frame_width, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS) *
+ (ALIGN(frame_height, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS)) *
+ NUM_HW_PIC_BUF, DMA_ALIGNMENT);
+ _size = min_t(u32, _size, H265D_MAX_SLICE + 1);
+ _size = ALIGN(_size, 4);
+ _size = 2 * _size * SIZE_H265D_VPP_CMD_PER_BUF;
+ if (_size > VPP_CMD_MAX_SIZE)
+ _size = VPP_CMD_MAX_SIZE;
+
+ return _size;
+}
+
+static inline
+u32 hfi_buffer_non_comv_h265d(u32 frame_width, u32 frame_height,
+ u32 num_vpp_pipes)
+{
+ u32 _size_bse, _size_vpp, _size;
+
+ _size_bse = size_h265d_bse_cmd_buf(frame_width, frame_height);
+ _size_vpp = size_h265d_vpp_cmd_buf(frame_width, frame_height);
+ _size = ALIGN(_size_bse, DMA_ALIGNMENT) +
+ ALIGN(_size_vpp, DMA_ALIGNMENT) +
+ ALIGN(NUM_HW_PIC_BUF * 20 * 22 * 4, DMA_ALIGNMENT) +
+ ALIGN(2 * sizeof(u16) *
+ (ALIGN(frame_width, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS) *
+ (ALIGN(frame_height, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS),
+ DMA_ALIGNMENT) +
+ ALIGN(SIZE_HW_PIC(SIZE_H265D_HW_PIC_T),
+ DMA_ALIGNMENT) +
+ HDR10_HIST_EXTRADATA_SIZE;
+
+ return ALIGN(_size, DMA_ALIGNMENT);
+}
+
+static inline u32 size_h264d_lb_fe_top_data(u32 frame_width, u32 frame_height)
+{
+ return MAX_FE_NBR_DATA_LUMA_LINE_BUFFER_SIZE * ALIGN(frame_width, 16) * 3;
+}
+
+static inline u32 size_h264d_lb_fe_top_ctrl(u32 frame_width, u32 frame_height)
+{
+ return MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * ((frame_width + 15) >> 4);
+}
+
+static inline u32 size_h264d_lb_fe_left_ctrl(u32 frame_width, u32 frame_height)
+{
+ return MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * ((frame_height + 15) >> 4);
+}
+
+static inline u32 size_h264d_lb_se_top_ctrl(u32 frame_width, u32 frame_height)
+{
+ return MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * ((frame_width + 15) >> 4);
+}
+
+static inline u32 size_h264d_lb_se_left_ctrl(u32 frame_width, u32 frame_height)
+{
+ return MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * ((frame_height + 15) >> 4);
+}
+
+static inline u32 size_h264d_lb_pe_top_data(u32 frame_width, u32 frame_height)
+{
+ return MAX_PE_NBR_DATA_LCU64_LINE_BUFFER_SIZE * ((frame_width + 15) >> 4);
+}
+
+static inline u32 size_h264d_lb_vsp_top(u32 frame_width, u32 frame_height)
+{
+ return (((frame_width + 15) >> 4) << 7);
+}
+
+static inline u32 size_h264d_lb_recon_dma_metadata_wr(u32 frame_width, u32 frame_height)
+{
+ return ALIGN(frame_height, 16) * 32;
+}
+
+static inline u32 size_h264d_qp(u32 frame_width, u32 frame_height)
+{
+ return ((frame_width + 63) >> 6) * ((frame_height + 63) >> 6) * 128;
+}
+
+static inline
+u32 size_vpss_lb(u32 frame_width, u32 frame_height, u32 num_vpp_pipes)
+{
+ u32 vpss_4tap_left_buffer_size = 0, vpss_div2_left_buffer_size = 0;
+ u32 vpss_4tap_top_buffer_size = 0, vpss_div2_top_buffer_size = 0;
+ u32 opb_lb_wr_llb_y_buffer_size, opb_lb_wr_llb_uv_buffer_size;
+ u32 opb_wr_top_line_chroma_buffer_size;
+ u32 opb_wr_top_line_luma_buffer_size;
+ u32 macrotiling_size = 32, size;
+
+ opb_wr_top_line_luma_buffer_size =
+ ALIGN(frame_width, macrotiling_size) /
+ macrotiling_size * 256;
+ opb_wr_top_line_luma_buffer_size =
+ ALIGN(opb_wr_top_line_luma_buffer_size, DMA_ALIGNMENT) +
+ (MAX_TILE_COLUMNS - 1) * 256;
+ opb_wr_top_line_luma_buffer_size =
+ max_t(u32, opb_wr_top_line_luma_buffer_size,
+ (32 * ALIGN(frame_height, 8)));
+ opb_wr_top_line_chroma_buffer_size =
+ opb_wr_top_line_luma_buffer_size;
+ opb_lb_wr_llb_uv_buffer_size =
+ ALIGN((ALIGN(frame_height, 8) / (4 / 2)) * 64,
+ BUFFER_ALIGNMENT_32_BYTES);
+ opb_lb_wr_llb_y_buffer_size =
+ ALIGN((ALIGN(frame_height, 8) / (4 / 2)) * 64,
+ BUFFER_ALIGNMENT_32_BYTES);
+ size = num_vpp_pipes * 2 *
+ (vpss_4tap_top_buffer_size + vpss_div2_top_buffer_size) +
+ 2 * (vpss_4tap_left_buffer_size + vpss_div2_left_buffer_size) +
+ opb_wr_top_line_luma_buffer_size +
+ opb_wr_top_line_chroma_buffer_size +
+ opb_lb_wr_llb_uv_buffer_size +
+ opb_lb_wr_llb_y_buffer_size;
+
+ return size;
+}
+
+static inline
+u32 hfi_buffer_line_h264d(u32 frame_width, u32 frame_height,
+ bool is_opb, u32 num_vpp_pipes)
+{
+ u32 vpss_lb_size = 0;
+ u32 _size;
+
+ _size = ALIGN(size_h264d_lb_fe_top_data(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_h264d_lb_fe_top_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_h264d_lb_fe_left_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) * num_vpp_pipes +
+ ALIGN(size_h264d_lb_se_top_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_h264d_lb_se_left_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) * num_vpp_pipes +
+ ALIGN(size_h264d_lb_pe_top_data(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_h264d_lb_vsp_top(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_h264d_lb_recon_dma_metadata_wr(frame_width, frame_height),
+ DMA_ALIGNMENT) * 2 +
+ ALIGN(size_h264d_qp(frame_width, frame_height),
+ DMA_ALIGNMENT);
+ _size = ALIGN(_size, DMA_ALIGNMENT);
+ if (is_opb)
+ vpss_lb_size = size_vpss_lb(frame_width, frame_height,
+ num_vpp_pipes);
+
+ _size = ALIGN((_size + vpss_lb_size),
+ DMA_ALIGNMENT);
+
+ return _size;
+}
+
+static inline
+u32 size_h265d_lb_fe_top_data(u32 frame_width, u32 frame_height)
+{
+ return MAX_FE_NBR_DATA_LUMA_LINE_BUFFER_SIZE *
+ (ALIGN(frame_width, 64) + 8) * 2;
+}
+
+static inline
+u32 size_h265d_lb_fe_top_ctrl(u32 frame_width, u32 frame_height)
+{
+ return MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE *
+ (ALIGN(frame_width, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS);
+}
+
+static inline
+u32 size_h265d_lb_fe_left_ctrl(u32 frame_width, u32 frame_height)
+{
+ return MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE *
+ (ALIGN(frame_height, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS);
+}
+
+static inline
+u32 size_h265d_lb_se_top_ctrl(u32 frame_width, u32 frame_height)
+{
+ return (LCU_MAX_SIZE_PELS / 8 * (128 / 8)) * ((frame_width + 15) >> 4);
+}
+
+static inline
+u32 size_h265d_lb_se_left_ctrl(u32 frame_width, u32 frame_height)
+{
+ return max_t(u32, ((frame_height + 16 - 1) / 8) *
+ MAX_SE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE,
+ max_t(u32, ((frame_height + 32 - 1) / 8) *
+ MAX_SE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE,
+ ((frame_height + 64 - 1) / 8) *
+ MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE));
+}
+
+static inline
+u32 size_h265d_lb_pe_top_data(u32 frame_width, u32 frame_height)
+{
+ return MAX_PE_NBR_DATA_LCU64_LINE_BUFFER_SIZE *
+ (ALIGN(frame_width, LCU_MIN_SIZE_PELS) / LCU_MIN_SIZE_PELS);
+}
+
+static inline
+u32 size_h265d_lb_vsp_top(u32 frame_width, u32 frame_height)
+{
+ return ((frame_width + 63) >> 6) * 128;
+}
+
+static inline
+u32 size_h265d_lb_vsp_left(u32 frame_width, u32 frame_height)
+{
+ return ((frame_height + 63) >> 6) * 128;
+}
+
+static inline
+u32 size_h265d_lb_recon_dma_metadata_wr(u32 frame_width, u32 frame_height)
+{
+ return size_h264d_lb_recon_dma_metadata_wr(frame_width, frame_height);
+}
+
+static inline
+u32 size_h265d_qp(u32 frame_width, u32 frame_height)
+{
+ return size_h264d_qp(frame_width, frame_height);
+}
+
+static inline
+u32 hfi_buffer_line_h265d(u32 frame_width, u32 frame_height,
+ bool is_opb, u32 num_vpp_pipes)
+{
+ u32 vpss_lb_size = 0, _size;
+
+ _size = ALIGN(size_h265d_lb_fe_top_data(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_h265d_lb_fe_top_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_h265d_lb_fe_left_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) * num_vpp_pipes +
+ ALIGN(size_h265d_lb_se_left_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) * num_vpp_pipes +
+ ALIGN(size_h265d_lb_se_top_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_h265d_lb_pe_top_data(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_h265d_lb_vsp_top(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_h265d_lb_vsp_left(frame_width, frame_height),
+ DMA_ALIGNMENT) * num_vpp_pipes +
+ ALIGN(size_h265d_lb_recon_dma_metadata_wr(frame_width, frame_height),
+ DMA_ALIGNMENT) * 4 +
+ ALIGN(size_h265d_qp(frame_width, frame_height),
+ DMA_ALIGNMENT);
+ if (is_opb)
+ vpss_lb_size = size_vpss_lb(frame_width, frame_height, num_vpp_pipes);
+
+ return ALIGN((_size + vpss_lb_size), DMA_ALIGNMENT);
+}
+
+static inline
+u32 size_vpxd_lb_fe_left_ctrl(u32 frame_width, u32 frame_height)
+{
+ return max_t(u32, ((frame_height + 15) >> 4) * MAX_FE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE,
+ max_t(u32, ((frame_height + 31) >> 5) * MAX_FE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE,
+ ((frame_height + 63) >> 6) * MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE));
+}
+
+static inline
+u32 size_vpxd_lb_fe_top_ctrl(u32 frame_width, u32 frame_height)
+{
+ return ((ALIGN(frame_width, 64) + 8) * 10 * 2);
+}
+
+static inline
+u32 size_vpxd_lb_se_top_ctrl(u32 frame_width, u32 frame_height)
+{
+ return ((frame_width + 15) >> 4) * MAX_FE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE;
+}
+
+static inline
+u32 size_vpxd_lb_se_left_ctrl(u32 frame_width, u32 frame_height)
+{
+ return max_t(u32, ((frame_height + 15) >> 4) * MAX_SE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE,
+ max_t(u32, ((frame_height + 31) >> 5) * MAX_SE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE,
+ ((frame_height + 63) >> 6) * MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE));
+}
+
+static inline
+u32 size_vpxd_lb_recon_dma_metadata_wr(u32 frame_width, u32 frame_height)
+{
+ return ALIGN((ALIGN(frame_height, 8) / (4 / 2)) * 64,
+ BUFFER_ALIGNMENT_32_BYTES);
+}
+
+static inline
+u32 size_mp2d_lb_fe_top_data(u32 frame_width, u32 frame_height)
+{
+ return ((ALIGN(frame_width, 16) + 8) * 10 * 2);
+}
+
+static inline
+u32 size_vp9d_lb_fe_top_data(u32 frame_width, u32 frame_height)
+{
+ return (ALIGN(ALIGN(frame_width, 8), 64) + 8) * 10 * 2;
+}
+
+static inline
+u32 size_vp9d_lb_pe_top_data(u32 frame_width, u32 frame_height)
+{
+ return ((ALIGN(ALIGN(frame_width, 8), 64) >> 6) * 176);
+}
+
+static inline
+u32 size_vp9d_lb_vsp_top(u32 frame_width, u32 frame_height)
+{
+ return (((ALIGN(ALIGN(frame_width, 8), 64) >> 6) * 64 * 8) + 256);
+}
+
+static inline u32 hfi_iris3_vp9d_comv_size(void)
+{
+ return (((8192 + 63) >> 6) * ((4320 + 63) >> 6) * 8 * 8 * 2 * 8);
+}
+
+static inline
+u32 size_vp9d_qp(u32 frame_width, u32 frame_height)
+{
+ return size_h264d_qp(frame_width, frame_height);
+}
+
+static inline
+u32 hfi_iris3_vp9d_lb_size(u32 frame_width, u32 frame_height,
+ u32 num_vpp_pipes)
+{
+ return ALIGN(size_vpxd_lb_fe_left_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) * num_vpp_pipes +
+ ALIGN(size_vpxd_lb_se_left_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) * num_vpp_pipes +
+ ALIGN(size_vp9d_lb_vsp_top(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_vpxd_lb_fe_top_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ 2 * ALIGN(size_vpxd_lb_recon_dma_metadata_wr(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_vpxd_lb_se_top_ctrl(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_vp9d_lb_pe_top_data(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_vp9d_lb_fe_top_data(frame_width, frame_height),
+ DMA_ALIGNMENT) +
+ ALIGN(size_vp9d_qp(frame_width, frame_height),
+ DMA_ALIGNMENT);
+}
+
+static inline
+u32 hfi_buffer_line_vp9d(u32 frame_width, u32 frame_height,
+ u32 _yuv_bufcount_min, bool is_opb,
+ u32 num_vpp_pipes)
+{
+ u32 vpss_lb_size = 0;
+ u32 _lb_size = 0;
+
+ _lb_size = hfi_iris3_vp9d_lb_size(frame_width, frame_height,
+ num_vpp_pipes);
+
+ if (is_opb)
+ vpss_lb_size = size_vpss_lb(frame_width, frame_height,
+ num_vpp_pipes);
+
+ return _lb_size + vpss_lb_size;
+}
+
+static inline u32 hfi_buffer_persist_h264d(u32 rpu_enabled)
+{
+ return ALIGN(SIZE_SLIST_BUF_H264 * NUM_SLIST_BUF_H264 +
+ H264_DISPLAY_BUF_SIZE * H264_NUM_FRM_INFO +
+ NUM_HW_PIC_BUF * SIZE_SEI_USERDATA +
+ (rpu_enabled) * NUM_HW_PIC_BUF * SIZE_DOLBY_RPU_METADATA,
+ DMA_ALIGNMENT);
+}
+
+static inline
+u32 hfi_buffer_persist_h265d(u32 rpu_enabled)
+{
+ return ALIGN((SIZE_SLIST_BUF_H265 * NUM_SLIST_BUF_H265 +
+ H265_NUM_FRM_INFO * H265_DISPLAY_BUF_SIZE +
+ H265_NUM_TILE * sizeof(u32) +
+ NUM_HW_PIC_BUF * SIZE_SEI_USERDATA +
+ rpu_enabled * NUM_HW_PIC_BUF * SIZE_DOLBY_RPU_METADATA),
+ DMA_ALIGNMENT);
+}
+
+static inline u32 hfi_buffer_persist_vp9d(void)
+{
+ return ALIGN(VP9_NUM_PROBABILITY_TABLE_BUF * VP9_PROB_TABLE_SIZE,
+ DMA_ALIGNMENT) +
+ ALIGN(hfi_iris3_vp9d_comv_size(), DMA_ALIGNMENT) +
+ ALIGN(MAX_SUPERFRAME_HEADER_LEN, DMA_ALIGNMENT) +
+ ALIGN(VP9_UDC_HEADER_BUF_SIZE, DMA_ALIGNMENT) +
+ ALIGN(VP9_NUM_FRAME_INFO_BUF * CCE_TILE_OFFSET_SIZE,
+ DMA_ALIGNMENT) +
+ ALIGN(VP9_NUM_FRAME_INFO_BUF * VP9_FRAME_INFO_BUF_SIZE,
+ DMA_ALIGNMENT) +
+ HDR10_HIST_EXTRADATA_SIZE;
+}
+
+static inline
+u32 hfi_nv12_ubwc_il_calc_y_buf_size(u32 frame_width, u32 frame_height,
+ u32 stride_multiple,
+ u32 min_buf_height_multiple)
+{
+ u32 stride, buf_height;
+
+ stride = ALIGN(frame_width, stride_multiple);
+ buf_height = ALIGN(frame_height, min_buf_height_multiple);
+
+ return ALIGN(stride * buf_height, HFI_ALIGNMENT_4096);
+}
+
+static inline
+u32 hfi_nv12_ubwc_il_calc_uv_buf_size(u32 frame_width, u32 frame_height,
+ u32 stride_multiple,
+ u32 min_buf_height_multiple)
+{
+ u32 uv_stride, uv_buf_height;
+
+ uv_stride = ALIGN(frame_width, stride_multiple);
+ uv_buf_height = ALIGN(((frame_height + 1) >> 1),
+ min_buf_height_multiple);
+
+ return ALIGN(uv_stride * uv_buf_height, HFI_ALIGNMENT_4096);
+}
+
+static inline
+u32 hfi_ubwc_calc_metadata_plane_stride(u32 frame_width,
+ u32 metadata_stride_multiple,
+ u32 tile_width_in_pels)
+{
+ return ALIGN(((frame_width + (tile_width_in_pels - 1)) / tile_width_in_pels),
+ metadata_stride_multiple);
+}
+
+static inline
+u32 hfi_ubwc_metadata_plane_bufheight(u32 frame_height,
+ u32 metadata_height_multiple,
+ u32 tile_height_in_pels)
+{
+ return ALIGN(((frame_height + (tile_height_in_pels - 1)) / tile_height_in_pels),
+ metadata_height_multiple);
+}
+
+static inline
+u32 hfi_ubwc_uv_metadata_plane_stride(u32 frame_width,
+ u32 metadata_stride_multiple,
+ u32 tile_width_in_pels)
+{
+ return ALIGN(((((frame_width + 1) >> 1) + tile_width_in_pels - 1) /
+ tile_width_in_pels), metadata_stride_multiple);
+}
+
+static inline
+u32 hfi_ubwc_uv_metadata_plane_bufheight(u32 frame_height,
+ u32 metadata_height_multiple,
+ u32 tile_height_in_pels)
+{
+ return ALIGN(((((frame_height + 1) >> 1) + tile_height_in_pels - 1) /
+ tile_height_in_pels), metadata_height_multiple);
+}
+
+static inline
+u32 hfi_ubwc_metadata_plane_buffer_size(u32 _metadata_tride, u32 _metadata_buf_height)
+{
+ return ALIGN(_metadata_tride * _metadata_buf_height, HFI_ALIGNMENT_4096);
+}
+
+static inline
+u32 hfi_nv12_ubwc_il_calc_buf_size_v2(u32 frame_width,
+ u32 frame_height,
+ u32 y_stride_multiple,
+ u32 y_buffer_height_multiple,
+ u32 uv_stride_multiple,
+ u32 uv_buffer_height_multiple,
+ u32 y_metadata_stride_multiple,
+ u32 y_metadata_buffer_height_multiple,
+ u32 uv_metadata_stride_multiple,
+ u32 uv_metadata_buffer_height_multiple)
+{
+ u32 y_buf_size, uv_buf_size, y_meta_size, uv_meta_size;
+ u32 half_height = (frame_height + 1) >> 1;
+ u32 stride, _height;
+
+ y_buf_size =
+ hfi_nv12_ubwc_il_calc_y_buf_size(frame_width, half_height,
+ y_stride_multiple,
+ y_buffer_height_multiple);
+ uv_buf_size =
+ hfi_nv12_ubwc_il_calc_uv_buf_size(frame_width, half_height,
+ uv_stride_multiple,
+ uv_buffer_height_multiple);
+ stride =
+ hfi_ubwc_calc_metadata_plane_stride(frame_width,
+ y_metadata_stride_multiple,
+ HFI_COL_FMT_NV12C_Y_TILE_WIDTH);
+ _height =
+ hfi_ubwc_metadata_plane_bufheight(half_height,
+ y_metadata_buffer_height_multiple,
+ HFI_COL_FMT_NV12C_Y_TILE_HEIGHT);
+ y_meta_size = hfi_ubwc_metadata_plane_buffer_size(stride, _height);
+ stride =
+ hfi_ubwc_uv_metadata_plane_stride(frame_width,
+ uv_metadata_stride_multiple,
+ HFI_COL_FMT_NV12C_UV_TILE_WIDTH);
+ _height =
+ hfi_ubwc_uv_metadata_plane_bufheight(half_height,
+ uv_metadata_buffer_height_multiple,
+ HFI_COL_FMT_NV12C_UV_TILE_HEIGHT);
+ uv_meta_size = hfi_ubwc_metadata_plane_buffer_size(stride, _height);
+
+ return (y_buf_size + uv_buf_size + y_meta_size + uv_meta_size) << 1;
+}
+
+static inline
+u32 hfi_yuv420_tp10_ubwc_calc_y_buf_size(u32 y_stride,
+ u32 y_buf_height)
+{
+ return ALIGN(y_stride * y_buf_height, HFI_ALIGNMENT_4096);
+}
+
+static inline
+u32 hfi_yuv420_tp10_ubwc_calc_uv_buf_size(u32 uv_stride,
+ u32 uv_buf_height)
+{
+ return ALIGN(uv_stride * uv_buf_height, HFI_ALIGNMENT_4096);
+}
+
+static inline
+u32 hfi_yuv420_tp10_ubwc_calc_buf_size(u32 y_stride, u32 y_buf_height,
+ u32 uv_stride, u32 uv_buf_height,
+ u32 y_md_stride, u32 y_md_height,
+ u32 uv_md_stride, u32 uv_md_height)
+{
+ u32 y_data_size, uv_data_size, y_md_size, uv_md_size;
+
+ y_data_size = hfi_yuv420_tp10_ubwc_calc_y_buf_size(y_stride, y_buf_height);
+ uv_data_size = hfi_yuv420_tp10_ubwc_calc_uv_buf_size(uv_stride, uv_buf_height);
+ y_md_size = hfi_ubwc_metadata_plane_buffer_size(y_md_stride, y_md_height);
+ uv_md_size = hfi_ubwc_metadata_plane_buffer_size(uv_md_stride, uv_md_height);
+
+ return y_data_size + uv_data_size + y_md_size + uv_md_size;
+}
+
+int iris_int_buf_size_iris3(struct iris_inst *inst,
+ enum iris_buffer_type buffer_type);
+
+#endif
--
2.7.4

2023-12-18 11:55:53

[permalink] [raw]

Subject: [PATCH v2 27/34] media: iris: implement vb2 ops for buf_queue and firmware response

Implement vb2 ops for buf queue. Below are the different
buffer attributes:
BUF_ATTR_DEFERRED - buffer queued by client but not submitted
to firmware.
BUF_ATTR_READ_ONLY - processed buffer received from firmware
as read only. These buffers are held in firmware as reference
for future frame processing.
BUF_ATTR_PENDING_RELEASE - buffers requested to be released
from firmware.
BUF_ATTR_QUEUED - buffers submitted to firmware.
BUF_ATTR_DEQUEUED - buffers received from firmware.
BUF_ATTR_BUFFER_DONE - buffers sent back to vb2.

Buffers are submitted and received via HFI_CMD_BUFFER.
Firmware associates below flags during buffer response:
HFI_BUF_FW_FLAG_RELEASE_DONE - buffer released in firmware.
HFI_BUF_FW_FLAG_READONLY - buffer used as reference in firmware.

Input buffers dequeued from firmware are sent directly to vb2.

Output buffers if read only, are sent to vb2 and also maintained
in read only list. If the same read only buffer is received form
client, HFI_BUF_HOST_FLAG_READONLY is attached to the buffer and
submitted to firmware. Once the buffer is received from firmware
as non read only, it is removed from read only list.

Signed-off-by: Dikshita Agarwal <[email protected]>
---
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 6 +
.../media/platform/qcom/vcodec/iris/iris_buffer.h | 1 -
.../media/platform/qcom/vcodec/iris/iris_common.h | 9 +
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 369 +++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 11 +-
.../platform/qcom/vcodec/iris/iris_hfi_packet.c | 28 ++
.../platform/qcom/vcodec/iris/iris_hfi_packet.h | 2 +
.../platform/qcom/vcodec/iris/iris_hfi_response.c | 392 +++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_state.c | 8 +
.../media/platform/qcom/vcodec/iris/iris_state.h | 1 +
drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 81 +++++
drivers/media/platform/qcom/vcodec/iris/iris_vb2.h | 1 +
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 49 +++
.../media/platform/qcom/vcodec/iris/iris_vdec.h | 1 +
.../media/platform/qcom/vcodec/iris/iris_vidc.c | 2 +
15 files changed, 959 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index dfd1a4c..104ef9c 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -238,4 +238,10 @@ enum hfi_buffer_host_flags {
HFI_BUF_HOST_FLAGS_CB_NON_SECURE = 0x00000200,
};

+enum hfi_buffer_firmware_flags {
+ HFI_BUF_FW_FLAG_NONE = 0x00000000,
+ HFI_BUF_FW_FLAG_RELEASE_DONE = 0x00000001,
+ HFI_BUF_FW_FLAG_READONLY = 0x00000010,
+};
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
index 8769c3d..59c4a10 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
@@ -58,7 +58,6 @@ int iris_destroy_internal_buffer(struct iris_inst *inst,
int iris_destroy_internal_buffers(struct iris_inst *inst,
u32 plane);
int iris_release_input_internal_buffers(struct iris_inst *inst);
-
int iris_alloc_and_queue_session_int_bufs(struct iris_inst *inst,
enum iris_buffer_type buffer_type);

diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
index 0fbd139..5fd96d3 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
@@ -23,6 +23,8 @@

#define NUM_MBS_4k (((4096 + 15) >> 4) * ((2304 + 15) >> 4))

+#define MAX_DPB_COUNT 32
+
#define MAX_DPB_LIST_ARRAY_SIZE (16 * 4)
#define MAX_DPB_LIST_PAYLOAD_SIZE (16 * 4 * 4)

@@ -69,6 +71,13 @@ enum iris_buffer_type {
BUF_VPSS,
};

+enum iris_buffer_flags {
+ BUF_FLAG_KEYFRAME = 0x00000008,
+ BUF_FLAG_PFRAME = 0x00000010,
+ BUF_FLAG_BFRAME = 0x00000020,
+ BUF_FLAG_ERROR = 0x00000040,
+};
+
enum iris_buffer_attributes {
BUF_ATTR_DEFERRED = BIT(0),
BUF_ATTR_READ_ONLY = BIT(1),
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index 7868566..a5b8aef 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -57,6 +57,18 @@ enum iris_buffer_type v4l2_type_to_driver(u32 type)
}
}

+u32 v4l2_type_from_driver(enum iris_buffer_type buffer_type)
+{
+ switch (buffer_type) {
+ case BUF_INPUT:
+ return INPUT_MPLANE;
+ case BUF_OUTPUT:
+ return OUTPUT_MPLANE;
+ default:
+ return 0;
+ }
+}
+
int get_mbpf(struct iris_inst *inst)
{
int height = 0, width = 0;
@@ -390,13 +402,359 @@ static int kill_session(struct iris_inst *inst)
return 0;
}

+struct iris_buffer *get_driver_buf(struct iris_inst *inst, u32 plane, u32 index)
+{
+ struct iris_buffer *iter = NULL;
+ struct iris_buffer *buf = NULL;
+ enum iris_buffer_type buf_type;
+ struct iris_buffers *buffers;
+
+ bool found = false;
+
+ buf_type = v4l2_type_to_driver(plane);
+ if (!buf_type)
+ return NULL;
+
+ buffers = iris_get_buffer_list(inst, buf_type);
+ if (!buffers)
+ return NULL;
+
+ list_for_each_entry(iter, &buffers->list, list) {
+ if (iter->index == index) {
+ found = true;
+ buf = iter;
+ break;
+ }
+ }
+
+ if (!found)
+ return NULL;
+
+ return buf;
+}
+
+static void process_requeued_readonly_buffers(struct iris_inst *inst,
+ struct iris_buffer *buf)
+{
+ struct iris_buffer *ro_buf, *dummy;
+
+ list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) {
+ if (ro_buf->device_addr != buf->device_addr)
+ continue;
+ if (ro_buf->attr & BUF_ATTR_READ_ONLY &&
+ !(ro_buf->attr & BUF_ATTR_PENDING_RELEASE)) {
+ buf->attr |= BUF_ATTR_READ_ONLY;
+
+ list_del_init(&ro_buf->list);
+ kfree(ro_buf);
+ break;
+ }
+ }
+}
+
+int queue_buffer(struct iris_inst *inst, struct iris_buffer *buf)
+{
+ int ret;
+
+ if (buf->type == BUF_OUTPUT)
+ process_requeued_readonly_buffers(inst, buf);
+
+ ret = iris_hfi_queue_buffer(inst, buf);
+ if (ret)
+ return ret;
+
+ buf->attr &= ~BUF_ATTR_DEFERRED;
+ buf->attr |= BUF_ATTR_QUEUED;
+
+ return ret;
+}
+
+int queue_deferred_buffers(struct iris_inst *inst, enum iris_buffer_type buf_type)
+{
+ struct iris_buffers *buffers;
+ struct iris_buffer *buf;
+ int ret = 0;
+
+ buffers = iris_get_buffer_list(inst, buf_type);
+ if (!buffers)
+ return -EINVAL;
+
+ list_for_each_entry(buf, &buffers->list, list) {
+ if (!(buf->attr & BUF_ATTR_DEFERRED))
+ continue;
+ ret = queue_buffer(inst, buf);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
+int iris_release_nonref_buffers(struct iris_inst *inst)
+{
+ u32 fw_ro_count = 0, nonref_ro_count = 0;
+ struct iris_buffer *ro_buf;
+ bool found = false;
+ int ret = 0;
+ int i = 0;
+
+ list_for_each_entry(ro_buf, &inst->buffers.read_only.list, list) {
+ if (!(ro_buf->attr & BUF_ATTR_READ_ONLY))
+ continue;
+ if (ro_buf->attr & BUF_ATTR_PENDING_RELEASE)
+ continue;
+ fw_ro_count++;
+ }
+
+ if (fw_ro_count <= MAX_DPB_COUNT)
+ return 0;
+
+ /*
+ * Mark the read only buffers present in read_only list as
+ * non-reference if it's not part of dpb_list_payload.
+ * dpb_list_payload details:
+ * payload[0-1] : 64 bits base_address of DPB-1
+ * payload[2] : 32 bits addr_offset of DPB-1
+ * payload[3] : 32 bits data_offset of DPB-1
+ */
+ list_for_each_entry(ro_buf, &inst->buffers.read_only.list, list) {
+ found = false;
+ if (!(ro_buf->attr & BUF_ATTR_READ_ONLY))
+ continue;
+ if (ro_buf->attr & BUF_ATTR_PENDING_RELEASE)
+ continue;
+ for (i = 0; (i + 3) < MAX_DPB_LIST_ARRAY_SIZE; i = i + 4) {
+ if (ro_buf->device_addr == inst->dpb_list_payload[i] &&
+ ro_buf->data_offset == inst->dpb_list_payload[i + 3]) {
+ found = true;
+ break;
+ }
+ }
+ if (!found)
+ nonref_ro_count++;
+ }
+
+ if (nonref_ro_count <= inst->buffers.output.min_count)
+ return 0;
+
+ list_for_each_entry(ro_buf, &inst->buffers.read_only.list, list) {
+ found = false;
+ if (!(ro_buf->attr & BUF_ATTR_READ_ONLY))
+ continue;
+ if (ro_buf->attr & BUF_ATTR_PENDING_RELEASE)
+ continue;
+ for (i = 0; (i + 3) < MAX_DPB_LIST_ARRAY_SIZE; i = i + 4) {
+ if (ro_buf->device_addr == inst->dpb_list_payload[i] &&
+ ro_buf->data_offset == inst->dpb_list_payload[i + 3]) {
+ found = true;
+ break;
+ }
+ }
+ if (!found) {
+ ro_buf->attr |= BUF_ATTR_PENDING_RELEASE;
+ ret = iris_hfi_release_buffer(inst, ro_buf);
+ if (ret)
+ return ret;
+ }
+ }
+
+ return ret;
+}
+
+int iris_vb2_buffer_done(struct iris_inst *inst,
+ struct iris_buffer *buf)
+{
+ struct vb2_v4l2_buffer *vbuf;
+ struct vb2_queue *q = NULL;
+ struct vb2_buffer *iter;
+ struct vb2_buffer *vb2;
+ int type, state;
+ bool found;
+
+ type = v4l2_type_from_driver(buf->type);
+ if (!type)
+ return -EINVAL;
+
+ if (type == INPUT_MPLANE)
+ q = inst->vb2q_src;
+ else if (type == OUTPUT_MPLANE)
+ q = inst->vb2q_dst;
+ if (!q || !q->streaming)
+ return -EINVAL;
+
+ found = false;
+ list_for_each_entry(iter, &q->queued_list, queued_entry) {
+ if (iter->state != VB2_BUF_STATE_ACTIVE)
+ continue;
+ if (iter->index == buf->index) {
+ found = true;
+ vb2 = iter;
+ break;
+ }
+ }
+ if (!found)
+ return -EINVAL;
+
+ if (buf->flags & BUF_FLAG_ERROR)
+ state = VB2_BUF_STATE_ERROR;
+ else
+ state = VB2_BUF_STATE_DONE;
+
+ vbuf = to_vb2_v4l2_buffer(vb2);
+ vbuf->flags = buf->flags;
+ vb2->timestamp = buf->timestamp;
+ vb2->planes[0].bytesused = buf->data_size + vb2->planes[0].data_offset;
+ vb2_buffer_done(vb2, state);
+
+ return 0;
+}
+
+static int iris_flush_deferred_buffers(struct iris_inst *inst,
+ enum iris_buffer_type type)
+{
+ struct iris_buffer *buf, *dummy;
+ struct iris_buffers *buffers;
+
+ buffers = iris_get_buffer_list(inst, type);
+ if (!buffers)
+ return -EINVAL;
+
+ list_for_each_entry_safe(buf, dummy, &buffers->list, list) {
+ if (buf->attr & BUF_ATTR_DEFERRED) {
+ if (!(buf->attr & BUF_ATTR_BUFFER_DONE)) {
+ buf->attr |= BUF_ATTR_BUFFER_DONE;
+ buf->data_size = 0;
+ iris_vb2_buffer_done(inst, buf);
+ }
+ }
+ }
+
+ return 0;
+}
+
+static int iris_flush_read_only_buffers(struct iris_inst *inst,
+ enum iris_buffer_type type)
+{
+ struct iris_buffer *ro_buf, *dummy;
+
+ if (type != BUF_OUTPUT)
+ return 0;
+
+ list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) {
+ if (ro_buf->attr & BUF_ATTR_READ_ONLY)
+ continue;
+ if (ro_buf->attach && ro_buf->sg_table)
+ dma_buf_unmap_attachment(ro_buf->attach, ro_buf->sg_table,
+ DMA_BIDIRECTIONAL);
+ if (ro_buf->attach && ro_buf->dmabuf)
+ dma_buf_detach(ro_buf->dmabuf, ro_buf->attach);
+ ro_buf->attach = NULL;
+ ro_buf->sg_table = NULL;
+ ro_buf->dmabuf = NULL;
+ ro_buf->device_addr = 0x0;
+ list_del_init(&ro_buf->list);
+ kfree(ro_buf);
+ }
+
+ return 0;
+}
+
+void iris_destroy_buffers(struct iris_inst *inst)
+{
+ struct iris_buffer *buf, *dummy;
+ struct iris_buffers *buffers;
+
+ static const enum iris_buffer_type ext_buf_types[] = {
+ BUF_INPUT,
+ BUF_OUTPUT,
+ };
+ static const enum iris_buffer_type internal_buf_types[] = {
+ BUF_BIN,
+ BUF_COMV,
+ BUF_NON_COMV,
+ BUF_LINE,
+ BUF_DPB,
+ BUF_PERSIST,
+ };
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(internal_buf_types); i++) {
+ buffers = iris_get_buffer_list(inst, internal_buf_types[i]);
+ if (!buffers)
+ continue;
+ list_for_each_entry_safe(buf, dummy, &buffers->list, list)
+ iris_destroy_internal_buffer(inst, buf);
+ }
+
+ list_for_each_entry_safe(buf, dummy, &inst->buffers.read_only.list, list) {
+ if (buf->attach && buf->sg_table)
+ dma_buf_unmap_attachment(buf->attach, buf->sg_table, DMA_BIDIRECTIONAL);
+ if (buf->attach && buf->dmabuf)
+ dma_buf_detach(buf->dmabuf, buf->attach);
+ list_del_init(&buf->list);
+ kfree(buf);
+ }
+
+ for (i = 0; i < ARRAY_SIZE(ext_buf_types); i++) {
+ buffers = iris_get_buffer_list(inst, ext_buf_types[i]);
+ if (!buffers)
+ continue;
+ list_for_each_entry_safe(buf, dummy, &buffers->list, list) {
+ if (buf->attach && buf->sg_table)
+ dma_buf_unmap_attachment(buf->attach, buf->sg_table,
+ DMA_BIDIRECTIONAL);
+ if (buf->attach && buf->dmabuf)
+ dma_buf_detach(buf->dmabuf, buf->attach);
+ list_del_init(&buf->list);
+ kfree(buf);
+ }
+ }
+}
+
+static int get_num_queued_buffers(struct iris_inst *inst,
+ enum iris_buffer_type type)
+{
+ struct iris_buffers *buffers;
+ struct iris_buffer *vbuf;
+ int count = 0;
+
+ if (type == BUF_INPUT)
+ buffers = &inst->buffers.input;
+ else if (type == BUF_OUTPUT)
+ buffers = &inst->buffers.output;
+ else
+ return count;
+
+ list_for_each_entry(vbuf, &buffers->list, list) {
+ if (vbuf->type != type)
+ continue;
+ if (!(vbuf->attr & BUF_ATTR_QUEUED))
+ continue;
+ count++;
+ }
+
+ return count;
+}
+
int session_streamoff(struct iris_inst *inst, u32 plane)
{
enum signal_session_response signal_type;
+ enum iris_buffer_type buffer_type;
u32 hw_response_timeout_val;
struct iris_core *core;
+ int count = 0;
int ret;

+ if (plane == INPUT_MPLANE) {
+ signal_type = SIGNAL_CMD_STOP_INPUT;
+ buffer_type = BUF_INPUT;
+ } else if (plane == OUTPUT_MPLANE) {
+ signal_type = SIGNAL_CMD_STOP_OUTPUT;
+ buffer_type = BUF_OUTPUT;
+ } else {
+ return -EINVAL;
+ }
+
ret = iris_hfi_stop(inst, plane);
if (ret)
goto error;
@@ -417,14 +775,25 @@ int session_streamoff(struct iris_inst *inst, u32 plane)
if (ret)
goto error;

+ /* no more queued buffers after streamoff */
+ count = get_num_queued_buffers(inst, buffer_type);
+ if (count) {
+ ret = -EINVAL;
+ goto error;
+ }
+
ret = iris_inst_state_change_streamoff(inst, plane);
if (ret)
goto error;

+ iris_flush_deferred_buffers(inst, buffer_type);
+ iris_flush_read_only_buffers(inst, buffer_type);
return 0;

error:
kill_session(inst);
+ iris_flush_deferred_buffers(inst, buffer_type);
+ iris_flush_read_only_buffers(inst, buffer_type);

return ret;
}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
index 47a017d..3a9889e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -24,6 +24,7 @@ bool res_is_less_than(u32 width, u32 height,
u32 get_port_info(struct iris_inst *inst,
enum plat_inst_cap_type cap_id);
enum iris_buffer_type v4l2_type_to_driver(u32 type);
+u32 v4l2_type_from_driver(enum iris_buffer_type buffer_type);
int get_mbpf(struct iris_inst *inst);
int close_session(struct iris_inst *inst);

@@ -34,7 +35,6 @@ bool is_split_mode_enabled(struct iris_inst *inst);
int signal_session_msg_receipt(struct iris_inst *inst,
enum signal_session_response cmd);
struct iris_inst *to_instance(struct iris_core *core, u32 session_id);
-int session_streamoff(struct iris_inst *inst, u32 plane);

u32 v4l2_codec_from_driver(struct iris_inst *inst, enum codec_type codec);
enum codec_type v4l2_codec_to_driver(struct iris_inst *inst, u32 v4l2_codec);
@@ -43,4 +43,13 @@ enum colorformat_type v4l2_colorformat_to_driver(struct iris_inst *inst, u32 v4l
struct vb2_queue *get_vb2q(struct iris_inst *inst, u32 type);
int check_session_supported(struct iris_inst *inst);

+struct iris_buffer *get_driver_buf(struct iris_inst *inst, u32 plane, u32 index);
+int queue_buffer(struct iris_inst *inst, struct iris_buffer *buf);
+int queue_deferred_buffers(struct iris_inst *inst, enum iris_buffer_type buf_type);
+int iris_vb2_buffer_done(struct iris_inst *inst,
+ struct iris_buffer *buf);
+int iris_release_nonref_buffers(struct iris_inst *inst);
+void iris_destroy_buffers(struct iris_inst *inst);
+int session_streamoff(struct iris_inst *inst, u32 plane);
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
index b8785a9..8dafd04 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
@@ -76,6 +76,34 @@ static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type)
}
}

+u32 hfi_buf_type_to_driver(enum hfi_buffer_type buf_type)
+{
+ switch (buf_type) {
+ case HFI_BUFFER_BITSTREAM:
+ return BUF_INPUT;
+ case HFI_BUFFER_RAW:
+ return BUF_OUTPUT;
+ case HFI_BUFFER_BIN:
+ return BUF_BIN;
+ case HFI_BUFFER_ARP:
+ return BUF_ARP;
+ case HFI_BUFFER_COMV:
+ return BUF_COMV;
+ case HFI_BUFFER_NON_COMV:
+ return BUF_NON_COMV;
+ case HFI_BUFFER_LINE:
+ return BUF_LINE;
+ case HFI_BUFFER_DPB:
+ return BUF_DPB;
+ case HFI_BUFFER_PERSIST:
+ return BUF_PERSIST;
+ case HFI_BUFFER_VPSS:
+ return BUF_VPSS;
+ default:
+ return 0;
+ }
+}
+
u32 get_hfi_codec(struct iris_inst *inst)
{
switch (inst->codec) {
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
index cf0960b6..8999c28 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
@@ -6,6 +6,7 @@
#ifndef _IRIS_HFI_PACKET_H_
#define _IRIS_HFI_PACKET_H_

+#include "hfi_defines.h"
#include "iris_core.h"
#include "iris_instance.h"

@@ -89,6 +90,7 @@ u32 get_hfi_matrix_coefficients(u32 coefficients);
u32 get_v4l2_color_primaries(u32 hfi_primaries);
u32 get_v4l2_transfer_char(u32 hfi_characterstics);
u32 get_v4l2_matrix_coefficients(u32 hfi_coefficients);
+u32 hfi_buf_type_to_driver(enum hfi_buffer_type buf_type);
int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf);

int hfi_packet_sys_init(struct iris_core *core,
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
index 4ca9314..b1236dd 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
@@ -4,6 +4,7 @@
*/

#include "hfi_defines.h"
+#include "iris_buffer.h"
#include "iris_helpers.h"
#include "iris_hfi_packet.h"
#include "iris_hfi_response.h"
@@ -21,6 +22,11 @@ struct iris_inst_hfi_range {
int (*handle)(struct iris_inst *inst, struct hfi_packet *pkt);
};

+struct iris_hfi_buffer_handle {
+ enum hfi_buffer_type type;
+ int (*handle)(struct iris_inst *inst, struct hfi_buffer *buffer);
+};
+
struct iris_hfi_packet_handle {
enum hfi_buffer_type type;
int (*handle)(struct iris_inst *inst, struct hfi_packet *pkt);
@@ -44,6 +50,94 @@ static void print_sfr_message(struct iris_core *core)
}
}

+static bool is_valid_hfi_buffer_type(u32 buffer_type)
+{
+ if (buffer_type != HFI_BUFFER_BITSTREAM &&
+ buffer_type != HFI_BUFFER_RAW &&
+ buffer_type != HFI_BUFFER_BIN &&
+ buffer_type != HFI_BUFFER_ARP &&
+ buffer_type != HFI_BUFFER_COMV &&
+ buffer_type != HFI_BUFFER_NON_COMV &&
+ buffer_type != HFI_BUFFER_LINE &&
+ buffer_type != HFI_BUFFER_DPB &&
+ buffer_type != HFI_BUFFER_PERSIST &&
+ buffer_type != HFI_BUFFER_VPSS) {
+ return false;
+ }
+
+ return true;
+}
+
+static bool is_valid_hfi_port(u32 port, u32 buffer_type)
+{
+ if (port == HFI_PORT_NONE &&
+ buffer_type != HFI_BUFFER_PERSIST)
+ return false;
+
+ if (port != HFI_PORT_BITSTREAM && port != HFI_PORT_RAW)
+ return false;
+
+ return true;
+}
+
+static int get_driver_buffer_flags(struct iris_inst *inst, u32 hfi_flags)
+{
+ u32 driver_flags = 0;
+
+ if (inst->hfi_frame_info.picture_type & HFI_PICTURE_IDR)
+ driver_flags |= BUF_FLAG_KEYFRAME;
+ else if (inst->hfi_frame_info.picture_type & HFI_PICTURE_P)
+ driver_flags |= BUF_FLAG_PFRAME;
+ else if (inst->hfi_frame_info.picture_type & HFI_PICTURE_B)
+ driver_flags |= BUF_FLAG_BFRAME;
+ else if (inst->hfi_frame_info.picture_type & HFI_PICTURE_I)
+ driver_flags |= BUF_FLAG_KEYFRAME;
+ else if (inst->hfi_frame_info.picture_type & HFI_PICTURE_CRA)
+ driver_flags |= BUF_FLAG_KEYFRAME;
+ else if (inst->hfi_frame_info.picture_type & HFI_PICTURE_BLA)
+ driver_flags |= BUF_FLAG_KEYFRAME;
+
+ if (inst->hfi_frame_info.data_corrupt)
+ driver_flags |= BUF_FLAG_ERROR;
+
+ if (inst->hfi_frame_info.overflow)
+ driver_flags |= BUF_FLAG_ERROR;
+
+ return driver_flags;
+}
+
+static bool validate_packet_payload(struct hfi_packet *pkt)
+{
+ u32 payload_size = 0;
+
+ switch (pkt->payload_info) {
+ case HFI_PAYLOAD_U32:
+ case HFI_PAYLOAD_S32:
+ case HFI_PAYLOAD_Q16:
+ case HFI_PAYLOAD_U32_ENUM:
+ case HFI_PAYLOAD_32_PACKED:
+ payload_size = 4;
+ break;
+ case HFI_PAYLOAD_U64:
+ case HFI_PAYLOAD_S64:
+ case HFI_PAYLOAD_64_PACKED:
+ payload_size = 8;
+ break;
+ case HFI_PAYLOAD_STRUCTURE:
+ if (pkt->type == HFI_CMD_BUFFER)
+ payload_size = sizeof(struct hfi_buffer);
+ break;
+ default:
+ payload_size = 0;
+ break;
+ }
+
+ if (pkt->size < sizeof(struct hfi_packet) + payload_size)
+ return false;
+
+ return true;
+}
+
static int validate_packet(u8 *response_pkt, u8 *core_resp_pkt, u32 core_resp_pkt_size)
{
u32 response_pkt_size = 0;
@@ -168,6 +262,293 @@ static int handle_session_close(struct iris_inst *inst,
return 0;
}

+static int handle_read_only_buffer(struct iris_inst *inst,
+ struct iris_buffer *buf)
+{
+ struct iris_buffer *ro_buf, *iter;
+ bool found = false;
+
+ list_for_each_entry(iter, &inst->buffers.read_only.list, list) {
+ if (iter->device_addr == buf->device_addr) {
+ found = true;
+ ro_buf = iter;
+ break;
+ }
+ }
+
+ if (!found) {
+ ro_buf = kzalloc(sizeof(*ro_buf), GFP_KERNEL);
+ if (!ro_buf)
+ return -ENOMEM;
+ ro_buf->index = -1;
+ ro_buf->inst = inst;
+ ro_buf->type = buf->type;
+ ro_buf->fd = buf->fd;
+ ro_buf->dmabuf = buf->dmabuf;
+ ro_buf->device_addr = buf->device_addr;
+ ro_buf->data_offset = buf->data_offset;
+ INIT_LIST_HEAD(&ro_buf->list);
+ list_add_tail(&ro_buf->list, &inst->buffers.read_only.list);
+ }
+ ro_buf->attr |= BUF_ATTR_READ_ONLY;
+
+ return 0;
+}
+
+static int handle_non_read_only_buffer(struct iris_inst *inst,
+ struct hfi_buffer *buffer)
+{
+ struct iris_buffer *ro_buf;
+
+ list_for_each_entry(ro_buf, &inst->buffers.read_only.list, list) {
+ if (ro_buf->device_addr == buffer->base_address) {
+ ro_buf->attr &= ~BUF_ATTR_READ_ONLY;
+ break;
+ }
+ }
+
+ return 0;
+}
+
+static int handle_release_output_buffer(struct iris_inst *inst,
+ struct hfi_buffer *buffer)
+{
+ struct iris_buffer *buf, *iter;
+ bool found = false;
+
+ list_for_each_entry(iter, &inst->buffers.read_only.list, list) {
+ if (iter->device_addr == buffer->base_address &&
+ iter->attr & BUF_ATTR_PENDING_RELEASE) {
+ found = true;
+ buf = iter;
+ break;
+ }
+ }
+ if (!found)
+ return -EINVAL;
+
+ buf->attr &= ~BUF_ATTR_READ_ONLY;
+ buf->attr &= ~BUF_ATTR_PENDING_RELEASE;
+
+ return 0;
+}
+
+static int handle_input_buffer(struct iris_inst *inst,
+ struct hfi_buffer *buffer)
+{
+ struct iris_buffers *buffers;
+ struct iris_buffer *buf, *iter;
+ bool found;
+
+ buffers = iris_get_buffer_list(inst, BUF_INPUT);
+ if (!buffers)
+ return -EINVAL;
+
+ found = false;
+ list_for_each_entry(iter, &buffers->list, list) {
+ if (iter->index == buffer->index) {
+ found = true;
+ buf = iter;
+ break;
+ }
+ }
+ if (!found)
+ return -EINVAL;
+
+ if (!(buf->attr & BUF_ATTR_QUEUED))
+ return 0;
+
+ buf->data_size = buffer->data_size;
+ buf->attr &= ~BUF_ATTR_QUEUED;
+ buf->attr |= BUF_ATTR_DEQUEUED;
+
+ buf->flags = get_driver_buffer_flags(inst, buffer->flags);
+
+ return 0;
+}
+
+static int handle_output_buffer(struct iris_inst *inst,
+ struct hfi_buffer *hfi_buffer)
+{
+ struct iris_buffers *buffers;
+ struct iris_buffer *buf, *iter;
+ bool found;
+ int ret = 0;
+
+ if (hfi_buffer->flags & HFI_BUF_FW_FLAG_RELEASE_DONE)
+ return handle_release_output_buffer(inst, hfi_buffer);
+
+ if (!(hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY))
+ ret = handle_non_read_only_buffer(inst, hfi_buffer);
+
+ buffers = iris_get_buffer_list(inst, BUF_OUTPUT);
+ if (!buffers)
+ return -EINVAL;
+
+ found = false;
+ list_for_each_entry(iter, &buffers->list, list) {
+ if (!(iter->attr & BUF_ATTR_QUEUED))
+ continue;
+
+ found = (iter->index == hfi_buffer->index &&
+ iter->device_addr == hfi_buffer->base_address &&
+ iter->data_offset == hfi_buffer->data_offset);
+
+ if (found) {
+ buf = iter;
+ break;
+ }
+ }
+ if (!found)
+ return 0;
+
+ buf->data_offset = hfi_buffer->data_offset;
+ buf->data_size = hfi_buffer->data_size;
+ buf->timestamp = hfi_buffer->timestamp;
+
+ buf->attr &= ~BUF_ATTR_QUEUED;
+ buf->attr |= BUF_ATTR_DEQUEUED;
+
+ if (inst->buffers.dpb.size && hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY)
+ iris_inst_change_state(inst, IRIS_INST_ERROR);
+
+ if (hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY) {
+ buf->attr |= BUF_ATTR_READ_ONLY;
+ ret = handle_read_only_buffer(inst, buf);
+ } else {
+ buf->attr &= ~BUF_ATTR_READ_ONLY;
+ }
+
+ buf->flags = get_driver_buffer_flags(inst, hfi_buffer->flags);
+
+ return ret;
+}
+
+static int handle_dequeue_buffers(struct iris_inst *inst)
+{
+ struct iris_buffers *buffers;
+ struct iris_buffer *dummy;
+ struct iris_buffer *buf;
+ int ret = 0;
+ int i;
+ static const enum iris_buffer_type buffer_type[] = {
+ BUF_INPUT,
+ BUF_OUTPUT,
+ };
+
+ for (i = 0; i < ARRAY_SIZE(buffer_type); i++) {
+ buffers = iris_get_buffer_list(inst, buffer_type[i]);
+ if (!buffers)
+ return -EINVAL;
+
+ list_for_each_entry_safe(buf, dummy, &buffers->list, list) {
+ if (buf->attr & BUF_ATTR_DEQUEUED) {
+ buf->attr &= ~BUF_ATTR_DEQUEUED;
+ if (!(buf->attr & BUF_ATTR_BUFFER_DONE)) {
+ buf->attr |= BUF_ATTR_BUFFER_DONE;
+ ret = iris_vb2_buffer_done(inst, buf);
+ if (ret)
+ ret = 0;
+ }
+ }
+ }
+ }
+
+ return ret;
+}
+
+static int handle_release_internal_buffer(struct iris_inst *inst,
+ struct hfi_buffer *buffer)
+{
+ struct iris_buffers *buffers;
+ struct iris_buffer *buf, *iter;
+ int ret = 0;
+ bool found;
+
+ buffers = iris_get_buffer_list(inst, hfi_buf_type_to_driver(buffer->type));
+ if (!buffers)
+ return -EINVAL;
+
+ found = false;
+ list_for_each_entry(iter, &buffers->list, list) {
+ if (iter->device_addr == buffer->base_address) {
+ found = true;
+ buf = iter;
+ break;
+ }
+ }
+ if (!found)
+ return -EINVAL;
+
+ buf->attr &= ~BUF_ATTR_QUEUED;
+
+ if (buf->attr & BUF_ATTR_PENDING_RELEASE)
+ ret = iris_destroy_internal_buffer(inst, buf);
+
+ return ret;
+}
+
+static int handle_session_buffer(struct iris_inst *inst,
+ struct hfi_packet *pkt)
+{
+ struct hfi_buffer *buffer;
+ u32 hfi_handle_size = 0;
+ int i, ret = 0;
+ const struct iris_hfi_buffer_handle *hfi_handle_arr = NULL;
+ static const struct iris_hfi_buffer_handle input_hfi_handle[] = {
+ {HFI_BUFFER_BITSTREAM, handle_input_buffer },
+ {HFI_BUFFER_BIN, handle_release_internal_buffer },
+ {HFI_BUFFER_COMV, handle_release_internal_buffer },
+ {HFI_BUFFER_NON_COMV, handle_release_internal_buffer },
+ {HFI_BUFFER_LINE, handle_release_internal_buffer },
+ {HFI_BUFFER_PERSIST, handle_release_internal_buffer },
+ };
+ static const struct iris_hfi_buffer_handle output_hfi_handle[] = {
+ {HFI_BUFFER_RAW, handle_output_buffer },
+ {HFI_BUFFER_DPB, handle_release_internal_buffer },
+ };
+
+ if (pkt->payload_info == HFI_PAYLOAD_NONE)
+ return 0;
+
+ if (!validate_packet_payload(pkt)) {
+ iris_inst_change_state(inst, IRIS_INST_ERROR);
+ return 0;
+ }
+
+ buffer = (struct hfi_buffer *)((u8 *)pkt + sizeof(*pkt));
+ if (!is_valid_hfi_buffer_type(buffer->type))
+ return 0;
+
+ if (!is_valid_hfi_port(pkt->port, buffer->type))
+ return 0;
+
+ if (pkt->port == HFI_PORT_BITSTREAM) {
+ hfi_handle_size = ARRAY_SIZE(input_hfi_handle);
+ hfi_handle_arr = input_hfi_handle;
+ } else if (pkt->port == HFI_PORT_RAW) {
+ hfi_handle_size = ARRAY_SIZE(output_hfi_handle);
+ hfi_handle_arr = output_hfi_handle;
+ }
+
+ if (!hfi_handle_arr || !hfi_handle_size)
+ return -EINVAL;
+
+ for (i = 0; i < hfi_handle_size; i++) {
+ if (hfi_handle_arr[i].type == buffer->type) {
+ ret = hfi_handle_arr[i].handle(inst, buffer);
+ if (ret)
+ return ret;
+ break;
+ }
+ }
+
+ if (i == hfi_handle_size)
+ return -EINVAL;
+
+ return ret;
+}
+
static int handle_src_change(struct iris_inst *inst,
struct hfi_packet *pkt)
{
@@ -191,6 +572,7 @@ static int handle_session_command(struct iris_inst *inst,
static const struct iris_hfi_packet_handle hfi_pkt_handle[] = {
{HFI_CMD_OPEN, NULL },
{HFI_CMD_CLOSE, handle_session_close },
+ {HFI_CMD_BUFFER, handle_session_buffer },
{HFI_CMD_SETTINGS_CHANGE, handle_src_change },
{HFI_CMD_SUBSCRIBE_MODE, NULL },
};
@@ -386,6 +768,7 @@ static int handle_session_response(struct iris_core *core,
{
struct hfi_packet *packet;
struct iris_inst *inst;
+ bool dequeue = false;
u8 *pkt, *start_pkt;
int ret = 0;
int i, j;
@@ -423,6 +806,7 @@ static int handle_session_response(struct iris_core *core,
handle_session_error(inst, packet);

if (packet->type > be[i].begin && packet->type < be[i].end) {
+ dequeue |= (packet->type == HFI_CMD_BUFFER);
ret = be[i].handle(inst, packet);
if (ret)
iris_inst_change_state(inst, IRIS_INST_ERROR);
@@ -431,7 +815,15 @@ static int handle_session_response(struct iris_core *core,
}
}

+ if (dequeue) {
+ ret = handle_dequeue_buffers(inst);
+ if (ret)
+ goto unlock;
+ }
+
memset(&inst->hfi_frame_info, 0, sizeof(struct iris_hfi_frame_info));
+
+unlock:
mutex_unlock(&inst->lock);

return ret;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
index 6553029..04e7dc8 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_state.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
@@ -170,6 +170,14 @@ bool allow_reqbufs(struct iris_inst *inst, u32 type)
(type == INPUT_MPLANE && inst->state == IRIS_INST_OUTPUT_STREAMING);
}

+bool allow_qbuf(struct iris_inst *inst, u32 type)
+{
+ return (type == INPUT_MPLANE && inst->state == IRIS_INST_INPUT_STREAMING) ||
+ (type == INPUT_MPLANE && inst->state == IRIS_INST_STREAMING) ||
+ (type == OUTPUT_MPLANE && inst->state == IRIS_INST_OUTPUT_STREAMING) ||
+ (type == OUTPUT_MPLANE && inst->state == IRIS_INST_STREAMING);
+}
+
bool allow_streamon(struct iris_inst *inst, u32 type)
{
return (type == INPUT_MPLANE && inst->state == IRIS_INST_OPEN) ||
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.h b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
index 28d5380..47558ed 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_state.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
@@ -43,6 +43,7 @@ int iris_inst_change_state(struct iris_inst *inst,

bool allow_s_fmt(struct iris_inst *inst, u32 type);
bool allow_reqbufs(struct iris_inst *inst, u32 type);
+bool allow_qbuf(struct iris_inst *inst, u32 type);
bool allow_streamon(struct iris_inst *inst, u32 type);
bool allow_streamoff(struct iris_inst *inst, u32 type);
bool allow_s_ctrl(struct iris_inst *inst, u32 cap_id);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
index d599366..f34434fc 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
@@ -95,6 +95,7 @@ int iris_vb2_queue_setup(struct vb2_queue *q,

int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count)
{
+ enum iris_buffer_type buf_type;
struct iris_inst *inst;
int ret = 0;

@@ -132,6 +133,16 @@ int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count)
if (ret)
goto error;

+ buf_type = v4l2_type_to_driver(q->type);
+ if (!buf_type) {
+ ret = -EINVAL;
+ goto error;
+ }
+
+ ret = queue_deferred_buffers(inst, buf_type);
+ if (ret)
+ goto error;
+
return ret;
error:
iris_inst_change_state(inst, IRIS_INST_ERROR);
@@ -168,9 +179,33 @@ void iris_vb2_stop_streaming(struct vb2_queue *q)
iris_inst_change_state(inst, IRIS_INST_ERROR);
}

+void iris_vb2_buf_queue(struct vb2_buffer *vb2)
+{
+ struct iris_inst *inst;
+ int ret;
+
+ inst = vb2_get_drv_priv(vb2->vb2_queue);
+ if (!inst || !inst->core)
+ return;
+
+ if (!vb2->planes[0].bytesused && vb2->type == INPUT_MPLANE) {
+ ret = -EINVAL;
+ goto exit;
+ }
+
+ ret = vdec_qbuf(inst, vb2);
+
+exit:
+ if (ret) {
+ iris_inst_change_state(inst, IRIS_INST_ERROR);
+ vb2_buffer_done(vb2, VB2_BUF_STATE_ERROR);
+ }
+}
+
void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev,
struct dma_buf *dbuf, unsigned long size)
{
+ struct iris_buffer *ro_buf, *dummy;
enum iris_buffer_type buf_type;
struct iris_buffers *buffers;
struct iris_buffer *iter;
@@ -203,6 +238,16 @@ void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev,
buf->inst = inst;
buf->dmabuf = dbuf;

+ if (buf->type == BUF_OUTPUT) {
+ list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) {
+ if (ro_buf->dmabuf != buf->dmabuf)
+ continue;
+ buf->attach = ro_buf->attach;
+ ro_buf->attach = NULL;
+ return buf;
+ }
+ }
+
buf->attach = dma_buf_attach(dbuf, dev);
if (IS_ERR(buf->attach)) {
buf->attach = NULL;
@@ -214,6 +259,7 @@ void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev,

int iris_vb2_map_dmabuf(void *buf_priv)
{
+ struct iris_buffer *ro_buf, *dummy;
struct iris_buffer *buf = buf_priv;
struct iris_core *core;
struct iris_inst *inst;
@@ -229,6 +275,17 @@ int iris_vb2_map_dmabuf(void *buf_priv)
return -EINVAL;
}

+ if (buf->type == BUF_OUTPUT) {
+ list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) {
+ if (ro_buf->dmabuf != buf->dmabuf)
+ continue;
+ buf->sg_table = ro_buf->sg_table;
+ buf->device_addr = ro_buf->device_addr;
+ ro_buf->sg_table = NULL;
+ return 0;
+ }
+ }
+
buf->sg_table = dma_buf_map_attachment(buf->attach, DMA_BIDIRECTIONAL);
if (IS_ERR(buf->sg_table))
return -EINVAL;
@@ -246,6 +303,7 @@ int iris_vb2_map_dmabuf(void *buf_priv)

void iris_vb2_unmap_dmabuf(void *buf_priv)
{
+ struct iris_buffer *ro_buf, *dummy;
struct iris_buffer *buf = buf_priv;
struct iris_core *core;
struct iris_inst *inst;
@@ -266,6 +324,17 @@ void iris_vb2_unmap_dmabuf(void *buf_priv)
return;
}

+ if (buf->type == BUF_OUTPUT) {
+ list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) {
+ if (ro_buf->dmabuf != buf->dmabuf)
+ continue;
+ ro_buf->sg_table = buf->sg_table;
+ buf->sg_table = NULL;
+ buf->device_addr = 0x0;
+ return;
+ }
+ }
+
if (buf->attach && buf->sg_table) {
dma_buf_unmap_attachment(buf->attach, buf->sg_table, DMA_BIDIRECTIONAL);
buf->sg_table = NULL;
@@ -275,6 +344,7 @@ void iris_vb2_unmap_dmabuf(void *buf_priv)

void iris_vb2_detach_dmabuf(void *buf_priv)
{
+ struct iris_buffer *ro_buf, *dummy;
struct iris_buffer *buf = buf_priv;
struct iris_core *core;
struct iris_inst *inst;
@@ -291,11 +361,22 @@ void iris_vb2_detach_dmabuf(void *buf_priv)
buf->sg_table = NULL;
}

+ if (buf->type == BUF_OUTPUT) {
+ list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) {
+ if (ro_buf->dmabuf != buf->dmabuf)
+ continue;
+ ro_buf->attach = buf->attach;
+ buf->attach = NULL;
+ goto exit;
+ }
+ }
+
if (buf->attach && buf->dmabuf) {
dma_buf_detach(buf->dmabuf, buf->attach);
buf->attach = NULL;
}

+exit:
buf->dmabuf = NULL;
buf->inst = NULL;
}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
index fc0e804..0757f01 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
@@ -13,6 +13,7 @@ int iris_vb2_queue_setup(struct vb2_queue *q,
unsigned int sizes[], struct device *alloc_devs[]);
int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count);
void iris_vb2_stop_streaming(struct vb2_queue *q);
+void iris_vb2_buf_queue(struct vb2_buffer *vb2);

/* vb2_mem_ops */
void *iris_vb2_alloc(struct vb2_buffer *vb, struct device *dev, unsigned long size);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index 0d8ca4b..7eb74c3 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -1193,3 +1193,52 @@ int vdec_streamon_output(struct iris_inst *inst)

return ret;
}
+
+static int vb2_buffer_to_driver(struct vb2_buffer *vb2,
+ struct iris_buffer *buf)
+{
+ struct vb2_v4l2_buffer *vbuf;
+
+ if (!vb2 || !buf)
+ return -EINVAL;
+
+ vbuf = to_vb2_v4l2_buffer(vb2);
+
+ buf->fd = vb2->planes[0].m.fd;
+ buf->data_offset = vb2->planes[0].data_offset;
+ buf->data_size = vb2->planes[0].bytesused - vb2->planes[0].data_offset;
+ buf->buffer_size = vb2->planes[0].length;
+ buf->timestamp = vb2->timestamp;
+ buf->flags = vbuf->flags;
+ buf->attr = 0;
+
+ return 0;
+}
+
+int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2)
+{
+ struct iris_buffer *buf = NULL;
+ int ret = 0;
+
+ buf = get_driver_buf(inst, vb2->type, vb2->index);
+ if (!buf)
+ return -EINVAL;
+
+ ret = vb2_buffer_to_driver(vb2, buf);
+ if (ret)
+ return ret;
+
+ if (!allow_qbuf(inst, vb2->type)) {
+ buf->attr |= BUF_ATTR_DEFERRED;
+ return 0;
+ }
+
+ ret = queue_buffer(inst, buf);
+ if (ret)
+ return ret;
+
+ if (vb2->type == OUTPUT_MPLANE)
+ ret = iris_release_nonref_buffers(inst);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
index 0722da1..d666c54 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
@@ -18,5 +18,6 @@ int vdec_init_src_change_param(struct iris_inst *inst);
int vdec_src_change(struct iris_inst *inst);
int vdec_streamon_input(struct iris_inst *inst);
int vdec_streamon_output(struct iris_inst *inst);
+int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
index 14d0077..0165340 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
@@ -252,6 +252,7 @@ int vidc_close(struct file *filp)
iris_inst_change_state(inst, IRIS_INST_CLOSE);
vidc_vb2_queue_deinit(inst);
vidc_v4l2_fh_deinit(inst);
+ iris_destroy_buffers(inst);
vidc_remove_session(inst);
mutex_unlock(&inst->lock);
mutex_destroy(&inst->ctx_q_lock);
@@ -912,6 +913,7 @@ static const struct vb2_ops iris_vb2_ops = {
.queue_setup = iris_vb2_queue_setup,
.start_streaming = iris_vb2_start_streaming,
.stop_streaming = iris_vb2_stop_streaming,
+ .buf_queue = iris_vb2_buf_queue,
};

static struct vb2_mem_ops iris_vb2_mem_ops = {
--
2.7.4

2023-12-18 11:56:48

[permalink] [raw]

Subject: [PATCH v2 25/34] media: iris: subscribe src change and handle firmware responses

From: Vikash Garodia <[email protected]>

Driver can subscribe to different bitstream parameters
for which it expects a response from firmware.
These are vital bitstream parameters which requires
reconfiguration of buffers or sending updated info
to client. Incase of any change in these subscribed
parameters, firmware sends a response, which is then
sent to client as source change event.

HFI_CMD_SUBSCRIBE_MODE with HFI_MODE_PORT_SETTINGS_CHANGE
- to subscribe for different bitstream parameters.

Below properties are set to firmware as part of this:
HFI_PROP_BITSTREAM_RESOLUTION
HFI_PROP_CROP_OFFSETS
HFI_PROP_LUMA_CHROMA_BIT_DEPTH
HFI_PROP_CODED_FRAMES
HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT
HFI_PROP_PIC_ORDER_CNT_TYPE
HFI_PROP_SIGNAL_COLOR_INFO
HFI_PROP_PROFILE
HFI_PROP_LEVEL
HFI_PROP_TIER

Add the handling of different responses
from firmware. Below are different types of responses:

System responses:
- System error - response in case of system error occurred.
- System property - response for system level property - sys image version.
- System init - response for video core init completion.

Session responses:
- Session error - response in case of any session level error.
- Session property - response for all session properties.
HFI_PROP_BITSTREAM_RESOLUTION
HFI_PROP_CROP_OFFSETS
HFI_PROP_LUMA_CHROMA_BIT_DEPTH
HFI_PROP_CODED_FRAMES
HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT
HFI_PROP_PIC_ORDER_CNT_TYPE
HFI_PROP_SIGNAL_COLOR_INFO
HFI_PROP_PROFILE
HFI_PROP_LEVEL
HFI_PROP_TIER

- Session command - response for session level commands.
HFI_CMD_OPEN - response for open competition.
HFI_CMD_CLOSE - response for open competition.
HFI_CMD_SETTINGS_CHANGE - response for change in subscribe parameters.
HFI_CMD_SUBSCRIBE_MODE - response for info on subscribed properties.

Signed-off-by: Vikash Garodia <[email protected]>
Signed-off-by: Dikshita Agarwal <[email protected]>
---
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 114 +++-
.../media/platform/qcom/vcodec/iris/iris_common.h | 25 +-
.../media/platform/qcom/vcodec/iris/iris_core.h | 2 +
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 29 +-
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 4 +
.../platform/qcom/vcodec/iris/iris_hfi_packet.c | 195 +++++++
.../platform/qcom/vcodec/iris/iris_hfi_packet.h | 11 +-
.../platform/qcom/vcodec/iris/iris_hfi_response.c | 364 ++++++++++++-
.../platform/qcom/vcodec/iris/iris_instance.h | 8 +
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 580 +++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_vdec.h | 4 +
.../platform/qcom/vcodec/iris/platform_common.h | 6 +
.../platform/qcom/vcodec/iris/platform_sm8550.c | 44 ++
13 files changed, 1375 insertions(+), 11 deletions(-)

diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index a6078a5..0ef6bad 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -10,6 +10,7 @@

#define HFI_VIDEO_ARCH_LX 0x1

+#define HFI_CMD_BEGIN 0x01000000
#define HFI_CMD_INIT 0x01000001
#define HFI_CMD_OPEN 0x01000003
#define HFI_CMD_CLOSE 0x01000004
@@ -31,7 +32,18 @@ enum hfi_property_mode_type {
HFI_MODE_PROPERTY = 0x00000002,
};

-#define HFI_CMD_SUBSCRIBE_MODE 0x0100000B
+#define HFI_CMD_SUBSCRIBE_MODE 0x0100000B
+#define HFI_CMD_SETTINGS_CHANGE 0x0100000C
+#define HFI_CMD_END 0x01FFFFFF
+
+#define HFI_BITMASK_BITSTREAM_WIDTH 0xffff0000
+#define HFI_BITMASK_BITSTREAM_HEIGHT 0x0000ffff
+
+#define HFI_BITMASK_FRAME_MBS_ONLY_FLAG 0x00000001
+
+#define HFI_PROP_BEGIN 0x03000000
+
+#define HFI_PROP_IMAGE_VERSION 0x03000001

enum hfi_color_format {
HFI_COLOR_FMT_OPAQUE = 0,
@@ -46,8 +58,12 @@ enum hfi_color_format {

#define HFI_PROP_COLOR_FORMAT 0x03000101

+#define HFI_PROP_BITSTREAM_RESOLUTION 0x03000103
+
#define HFI_PROP_LINEAR_STRIDE_SCANLINE 0x03000104

+#define HFI_PROP_CROP_OFFSETS 0x03000105
+
#define HFI_PROP_PROFILE 0x03000107

#define HFI_PROP_LEVEL 0x03000108
@@ -68,13 +84,83 @@ enum hfi_color_format {

#define HFI_PROP_BUFFER_HOST_MAX_COUNT 0x03000123

+#define HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT 0x03000124
+
#define HFI_PROP_PIC_ORDER_CNT_TYPE 0x03000128

+#define HFI_PROP_QUALITY_MODE 0x03000148
+
+enum hfi_color_primaries {
+ HFI_PRIMARIES_RESERVED = 0,
+ HFI_PRIMARIES_BT709 = 1,
+ HFI_PRIMARIES_UNSPECIFIED = 2,
+ HFI_PRIMARIES_BT470_SYSTEM_M = 4,
+ HFI_PRIMARIES_BT470_SYSTEM_BG = 5,
+ HFI_PRIMARIES_BT601_525 = 6,
+ HFI_PRIMARIES_SMPTE_ST240M = 7,
+ HFI_PRIMARIES_GENERIC_FILM = 8,
+ HFI_PRIMARIES_BT2020 = 9,
+ HFI_PRIMARIES_SMPTE_ST428_1 = 10,
+ HFI_PRIMARIES_SMPTE_RP431_2 = 11,
+ HFI_PRIMARIES_SMPTE_EG431_1 = 12,
+ HFI_PRIMARIES_SMPTE_EBU_TECH = 22,
+};
+
+enum hfi_transfer_characteristics {
+ HFI_TRANSFER_RESERVED = 0,
+ HFI_TRANSFER_BT709 = 1,
+ HFI_TRANSFER_UNSPECIFIED = 2,
+ HFI_TRANSFER_BT470_SYSTEM_M = 4,
+ HFI_TRANSFER_BT470_SYSTEM_BG = 5,
+ HFI_TRANSFER_BT601_525_OR_625 = 6,
+ HFI_TRANSFER_SMPTE_ST240M = 7,
+ HFI_TRANSFER_LINEAR = 8,
+ HFI_TRANSFER_LOG_100_1 = 9,
+ HFI_TRANSFER_LOG_SQRT = 10,
+ HFI_TRANSFER_XVYCC = 11,
+ HFI_TRANSFER_BT1361_0 = 12,
+ HFI_TRANSFER_SRGB_SYCC = 13,
+ HFI_TRANSFER_BT2020_14 = 14,
+ HFI_TRANSFER_BT2020_15 = 15,
+ HFI_TRANSFER_SMPTE_ST2084_PQ = 16,
+ HFI_TRANSFER_SMPTE_ST428_1 = 17,
+ HFI_TRANSFER_BT2100_2_HLG = 18,
+};
+
+enum hfi_matrix_coefficients {
+ HFI_MATRIX_COEFF_SRGB_SMPTE_ST428_1 = 0,
+ HFI_MATRIX_COEFF_BT709 = 1,
+ HFI_MATRIX_COEFF_UNSPECIFIED = 2,
+ HFI_MATRIX_COEFF_RESERVED = 3,
+ HFI_MATRIX_COEFF_FCC_TITLE_47 = 4,
+ HFI_MATRIX_COEFF_BT470_SYS_BG_OR_BT601_625 = 5,
+ HFI_MATRIX_COEFF_BT601_525_BT1358_525_OR_625 = 6,
+ HFI_MATRIX_COEFF_SMPTE_ST240 = 7,
+ HFI_MATRIX_COEFF_YCGCO = 8,
+ HFI_MATRIX_COEFF_BT2020_NON_CONSTANT = 9,
+ HFI_MATRIX_COEFF_BT2020_CONSTANT = 10,
+ HFI_MATRIX_COEFF_SMPTE_ST2085 = 11,
+ HFI_MATRIX_COEFF_SMPTE_CHROM_DERV_NON_CONSTANT = 12,
+ HFI_MATRIX_COEFF_SMPTE_CHROM_DERV_CONSTANT = 13,
+ HFI_MATRIX_COEFF_BT2100 = 14,
+};
+
+#define HFI_PROP_SIGNAL_COLOR_INFO 0x03000155
+
#define HFI_PROP_DECODE_ORDER_OUTPUT 0x0300015b

+enum hfi_picture_type {
+ HFI_PICTURE_IDR = 0x00000001,
+ HFI_PICTURE_P = 0x00000002,
+ HFI_PICTURE_B = 0x00000004,
+ HFI_PICTURE_I = 0x00000008,
+ HFI_PICTURE_CRA = 0x00000010,
+ HFI_PICTURE_BLA = 0x00000020,
+};
+
#define HFI_PROP_PICTURE_TYPE 0x03000162

-#define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168
+#define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168

#define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169

@@ -86,7 +172,29 @@ enum hfi_color_format {

#define HFI_PROP_COMV_BUFFER_COUNT 0x03000193

-#define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001
+#define HFI_PROP_END 0x03FFFFFF
+
+#define HFI_SESSION_ERROR_BEGIN 0x04000000
+
+#define HFI_ERROR_UNKNOWN_SESSION 0x04000001
+
+#define HFI_ERROR_MAX_SESSIONS 0x04000002
+
+#define HFI_ERROR_FATAL 0x04000003
+
+#define HFI_ERROR_INVALID_STATE 0x04000004
+
+#define HFI_ERROR_INSUFFICIENT_RESOURCES 0x04000005
+
+#define HFI_ERROR_BUFFER_NOT_SET 0x04000006
+
+#define HFI_SESSION_ERROR_END 0x04FFFFFF
+
+#define HFI_SYSTEM_ERROR_BEGIN 0x05000000
+
+#define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001
+
+#define HFI_SYSTEM_ERROR_END 0x05FFFFFF

struct hfi_debug_header {
u32 size;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
index 6b771f8..0fbd139 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
@@ -8,7 +8,6 @@
#include <media/v4l2-dev.h>
#include <media/v4l2-device.h>
#include <media/v4l2-event.h>
-#include <media/v4l2-ioctl.h>
#include <media/videobuf2-v4l2.h>

#define INPUT_MPLANE V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE
@@ -16,6 +15,7 @@
#define DEFAULT_WIDTH 320
#define DEFAULT_HEIGHT 240
#define DEFAULT_BSE_VPP_DELAY 2
+#define IRIS_VERSION_LENGTH 128

#define MAX_EVENTS 30

@@ -23,6 +23,9 @@

#define NUM_MBS_4k (((4096 + 15) >> 4) * ((2304 + 15) >> 4))

+#define MAX_DPB_LIST_ARRAY_SIZE (16 * 4)
+#define MAX_DPB_LIST_PAYLOAD_SIZE (16 * 4 * 4)
+
enum codec_type {
H264 = BIT(0),
HEVC = BIT(1),
@@ -95,4 +98,24 @@ struct iris_buffer {
struct dma_buf_attachment *attach;
};

+struct subscription_params {
+ u32 bitstream_resolution;
+ u32 crop_offsets[2];
+ u32 bit_depth;
+ u32 coded_frames;
+ u32 fw_min_count;
+ u32 pic_order_cnt;
+ u32 color_info;
+ u32 profile;
+ u32 level;
+ u32 tier;
+};
+
+struct iris_hfi_frame_info {
+ u32 picture_type;
+ u32 no_output;
+ u32 data_corrupt;
+ u32 overflow;
+};
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index 154991c..6452ea7 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -57,6 +57,7 @@
* @inst_caps: a pointer to supported instance capabilities
* @instances: a list_head of all instances
* @intr_status: interrupt status
+ * @fw_version: firmware version
*/

struct iris_core {
@@ -99,6 +100,7 @@ struct iris_core {
struct plat_inst_caps *inst_caps;
struct list_head instances;
u32 intr_status;
+ char fw_version[IRIS_VERSION_LENGTH];
};

int iris_core_init(struct iris_core *core);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index ff44cda..4cad673 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -3,6 +3,8 @@
* Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
*/

+#include "hfi_defines.h"
+#include "iris_core.h"
#include "iris_helpers.h"
#include "iris_hfi.h"
#include "iris_hfi_packet.h"
@@ -66,7 +68,7 @@ int get_mbpf(struct iris_inst *inst)
return NUM_MBS_PER_FRAME(height, width);
}

-bool is_linear_colorformat(u32 colorformat)
+inline bool is_linear_colorformat(u32 colorformat)
{
return colorformat == V4L2_PIX_FMT_NV12 || colorformat == V4L2_PIX_FMT_NV21;
}
@@ -350,3 +352,28 @@ int check_session_supported(struct iris_inst *inst)

return ret;
}
+
+int signal_session_msg_receipt(struct iris_inst *inst,
+ enum signal_session_response cmd)
+{
+ if (cmd < MAX_SIGNAL)
+ complete(&inst->completions[cmd]);
+
+ return 0;
+}
+
+struct iris_inst *to_instance(struct iris_core *core, u32 session_id)
+{
+ struct iris_inst *inst = NULL;
+
+ mutex_lock(&core->lock);
+ list_for_each_entry(inst, &core->instances, list) {
+ if (inst->session_id == session_id) {
+ mutex_unlock(&core->lock);
+ return inst;
+ }
+ }
+ mutex_unlock(&core->lock);
+
+ return NULL;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
index fe85d23..cb22adf 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -31,6 +31,10 @@ bool is_linear_colorformat(u32 colorformat);
bool is_10bit_colorformat(enum colorformat_type colorformat);
bool is_8bit_colorformat(enum colorformat_type colorformat);
bool is_split_mode_enabled(struct iris_inst *inst);
+int signal_session_msg_receipt(struct iris_inst *inst,
+ enum signal_session_response cmd);
+struct iris_inst *to_instance(struct iris_core *core, u32 session_id);
+
u32 v4l2_codec_from_driver(struct iris_inst *inst, enum codec_type codec);
enum codec_type v4l2_codec_to_driver(struct iris_inst *inst, u32 v4l2_codec);
u32 v4l2_colorformat_from_driver(struct iris_inst *inst, enum colorformat_type colorformat);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
index d4cdfcf..dc7157d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
@@ -100,6 +100,201 @@ u32 get_hfi_colorformat(u32 colorformat)
return hfi_colorformat;
}

+u32 get_hfi_color_primaries(u32 primaries)
+{
+ u32 hfi_primaries = HFI_PRIMARIES_RESERVED;
+
+ switch (primaries) {
+ case V4L2_COLORSPACE_DEFAULT:
+ hfi_primaries = HFI_PRIMARIES_RESERVED;
+ break;
+ case V4L2_COLORSPACE_REC709:
+ hfi_primaries = HFI_PRIMARIES_BT709;
+ break;
+ case V4L2_COLORSPACE_470_SYSTEM_M:
+ hfi_primaries = HFI_PRIMARIES_BT470_SYSTEM_M;
+ break;
+ case V4L2_COLORSPACE_470_SYSTEM_BG:
+ hfi_primaries = HFI_PRIMARIES_BT470_SYSTEM_BG;
+ break;
+ case V4L2_COLORSPACE_SMPTE170M:
+ hfi_primaries = HFI_PRIMARIES_BT601_525;
+ break;
+ case V4L2_COLORSPACE_SMPTE240M:
+ hfi_primaries = HFI_PRIMARIES_SMPTE_ST240M;
+ break;
+ case V4L2_COLORSPACE_BT2020:
+ hfi_primaries = HFI_PRIMARIES_BT2020;
+ break;
+ case V4L2_COLORSPACE_DCI_P3:
+ hfi_primaries = HFI_PRIMARIES_SMPTE_RP431_2;
+ break;
+ default:
+ break;
+ }
+
+ return hfi_primaries;
+}
+
+u32 get_hfi_transfer_char(u32 characterstics)
+{
+ u32 hfi_characterstics = HFI_TRANSFER_RESERVED;
+
+ switch (characterstics) {
+ case V4L2_XFER_FUNC_DEFAULT:
+ hfi_characterstics = HFI_TRANSFER_RESERVED;
+ break;
+ case V4L2_XFER_FUNC_709:
+ hfi_characterstics = HFI_TRANSFER_BT709;
+ break;
+ case V4L2_XFER_FUNC_SMPTE240M:
+ hfi_characterstics = HFI_TRANSFER_SMPTE_ST240M;
+ break;
+ case V4L2_XFER_FUNC_SRGB:
+ hfi_characterstics = HFI_TRANSFER_SRGB_SYCC;
+ break;
+ case V4L2_XFER_FUNC_SMPTE2084:
+ hfi_characterstics = HFI_TRANSFER_SMPTE_ST2084_PQ;
+ break;
+ default:
+ break;
+ }
+
+ return hfi_characterstics;
+}
+
+u32 get_hfi_matrix_coefficients(u32 coefficients)
+{
+ u32 hfi_coefficients = HFI_MATRIX_COEFF_RESERVED;
+
+ switch (coefficients) {
+ case V4L2_YCBCR_ENC_DEFAULT:
+ hfi_coefficients = HFI_MATRIX_COEFF_RESERVED;
+ break;
+ case V4L2_YCBCR_ENC_709:
+ hfi_coefficients = HFI_MATRIX_COEFF_BT709;
+ break;
+ case V4L2_YCBCR_ENC_XV709:
+ hfi_coefficients = HFI_MATRIX_COEFF_BT709;
+ break;
+ case V4L2_YCBCR_ENC_XV601:
+ hfi_coefficients = HFI_MATRIX_COEFF_BT470_SYS_BG_OR_BT601_625;
+ break;
+ case V4L2_YCBCR_ENC_601:
+ hfi_coefficients = HFI_MATRIX_COEFF_BT601_525_BT1358_525_OR_625;
+ break;
+ case V4L2_YCBCR_ENC_SMPTE240M:
+ hfi_coefficients = HFI_MATRIX_COEFF_SMPTE_ST240;
+ break;
+ case V4L2_YCBCR_ENC_BT2020:
+ hfi_coefficients = HFI_MATRIX_COEFF_BT2020_NON_CONSTANT;
+ break;
+ case V4L2_YCBCR_ENC_BT2020_CONST_LUM:
+ hfi_coefficients = HFI_MATRIX_COEFF_BT2020_CONSTANT;
+ break;
+ default:
+ break;
+ }
+
+ return hfi_coefficients;
+}
+
+u32 get_v4l2_color_primaries(u32 hfi_primaries)
+{
+ u32 primaries = V4L2_COLORSPACE_DEFAULT;
+
+ switch (hfi_primaries) {
+ case HFI_PRIMARIES_RESERVED:
+ primaries = V4L2_COLORSPACE_DEFAULT;
+ break;
+ case HFI_PRIMARIES_BT709:
+ primaries = V4L2_COLORSPACE_REC709;
+ break;
+ case HFI_PRIMARIES_BT470_SYSTEM_M:
+ primaries = V4L2_COLORSPACE_470_SYSTEM_M;
+ break;
+ case HFI_PRIMARIES_BT470_SYSTEM_BG:
+ primaries = V4L2_COLORSPACE_470_SYSTEM_BG;
+ break;
+ case HFI_PRIMARIES_BT601_525:
+ primaries = V4L2_COLORSPACE_SMPTE170M;
+ break;
+ case HFI_PRIMARIES_SMPTE_ST240M:
+ primaries = V4L2_COLORSPACE_SMPTE240M;
+ break;
+ case HFI_PRIMARIES_BT2020:
+ primaries = V4L2_COLORSPACE_BT2020;
+ break;
+ case V4L2_COLORSPACE_DCI_P3:
+ primaries = HFI_PRIMARIES_SMPTE_RP431_2;
+ break;
+ default:
+ break;
+ }
+
+ return primaries;
+}
+
+u32 get_v4l2_transfer_char(u32 hfi_characterstics)
+{
+ u32 characterstics = V4L2_XFER_FUNC_DEFAULT;
+
+ switch (hfi_characterstics) {
+ case HFI_TRANSFER_RESERVED:
+ characterstics = V4L2_XFER_FUNC_DEFAULT;
+ break;
+ case HFI_TRANSFER_BT709:
+ characterstics = V4L2_XFER_FUNC_709;
+ break;
+ case HFI_TRANSFER_SMPTE_ST240M:
+ characterstics = V4L2_XFER_FUNC_SMPTE240M;
+ break;
+ case HFI_TRANSFER_SRGB_SYCC:
+ characterstics = V4L2_XFER_FUNC_SRGB;
+ break;
+ case HFI_TRANSFER_SMPTE_ST2084_PQ:
+ characterstics = V4L2_XFER_FUNC_SMPTE2084;
+ break;
+ default:
+ break;
+ }
+
+ return characterstics;
+}
+
+u32 get_v4l2_matrix_coefficients(u32 hfi_coefficients)
+{
+ u32 coefficients = V4L2_YCBCR_ENC_DEFAULT;
+
+ switch (hfi_coefficients) {
+ case HFI_MATRIX_COEFF_RESERVED:
+ coefficients = V4L2_YCBCR_ENC_DEFAULT;
+ break;
+ case HFI_MATRIX_COEFF_BT709:
+ coefficients = V4L2_YCBCR_ENC_709;
+ break;
+ case HFI_MATRIX_COEFF_BT470_SYS_BG_OR_BT601_625:
+ coefficients = V4L2_YCBCR_ENC_XV601;
+ break;
+ case HFI_MATRIX_COEFF_BT601_525_BT1358_525_OR_625:
+ coefficients = V4L2_YCBCR_ENC_601;
+ break;
+ case HFI_MATRIX_COEFF_SMPTE_ST240:
+ coefficients = V4L2_YCBCR_ENC_SMPTE240M;
+ break;
+ case HFI_MATRIX_COEFF_BT2020_NON_CONSTANT:
+ coefficients = V4L2_YCBCR_ENC_BT2020;
+ break;
+ case HFI_MATRIX_COEFF_BT2020_CONSTANT:
+ coefficients = V4L2_YCBCR_ENC_BT2020_CONST_LUM;
+ break;
+ default:
+ break;
+ }
+
+ return coefficients;
+}
+
int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf)
{
memset(buf, 0, sizeof(*buf));
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
index 4276d6d..f813116 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
@@ -80,10 +80,17 @@ enum hfi_packet_port_type {
};

u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type);
-int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf);
-u32 get_hfi_colorformat(u32 colorformat);
u32 get_hfi_port(u32 plane);

+u32 get_hfi_colorformat(u32 colorformat);
+u32 get_hfi_color_primaries(u32 primaries);
+u32 get_hfi_transfer_char(u32 characterstics);
+u32 get_hfi_matrix_coefficients(u32 coefficients);
+u32 get_v4l2_color_primaries(u32 hfi_primaries);
+u32 get_v4l2_transfer_char(u32 hfi_characterstics);
+u32 get_v4l2_matrix_coefficients(u32 hfi_coefficients);
+int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf);
+
int hfi_packet_sys_init(struct iris_core *core,
u8 *pkt, u32 pkt_size);
int hfi_packet_image_version(struct iris_core *core,
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
index 829f3f6..4ca9314 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
@@ -4,8 +4,27 @@
*/

#include "hfi_defines.h"
+#include "iris_helpers.h"
#include "iris_hfi_packet.h"
#include "iris_hfi_response.h"
+#include "iris_vdec.h"
+
+struct iris_core_hfi_range {
+ u32 begin;
+ u32 end;
+ int (*handle)(struct iris_core *core, struct hfi_packet *pkt);
+};
+
+struct iris_inst_hfi_range {
+ u32 begin;
+ u32 end;
+ int (*handle)(struct iris_inst *inst, struct hfi_packet *pkt);
+};
+
+struct iris_hfi_packet_handle {
+ enum hfi_buffer_type type;
+ int (*handle)(struct iris_inst *inst, struct hfi_packet *pkt);
+};

static void print_sfr_message(struct iris_core *core)
{
@@ -25,8 +44,7 @@ static void print_sfr_message(struct iris_core *core)
}
}

-static int validate_packet(u8 *response_pkt, u8 *core_resp_pkt,
- u32 core_resp_pkt_size)
+static int validate_packet(u8 *response_pkt, u8 *core_resp_pkt, u32 core_resp_pkt_size)
{
u32 response_pkt_size = 0;
u8 *response_limit;
@@ -52,8 +70,7 @@ static int validate_packet(u8 *response_pkt, u8 *core_resp_pkt,
return 0;
}

-static int validate_hdr_packet(struct iris_core *core,
- struct hfi_header *hdr)
+static int validate_hdr_packet(struct iris_core *core, struct hfi_header *hdr)
{
struct hfi_packet *packet;
int i, ret = 0;
@@ -76,6 +93,45 @@ static int validate_hdr_packet(struct iris_core *core,
return ret;
}

+static int handle_session_error(struct iris_inst *inst,
+ struct hfi_packet *pkt)
+{
+ struct iris_core *core;
+ char *error;
+
+ core = inst->core;
+
+ switch (pkt->type) {
+ case HFI_ERROR_MAX_SESSIONS:
+ error = "exceeded max sessions";
+ break;
+ case HFI_ERROR_UNKNOWN_SESSION:
+ error = "unknown session id";
+ break;
+ case HFI_ERROR_INVALID_STATE:
+ error = "invalid operation for current state";
+ break;
+ case HFI_ERROR_INSUFFICIENT_RESOURCES:
+ error = "insufficient resources";
+ break;
+ case HFI_ERROR_BUFFER_NOT_SET:
+ error = "internal buffers not set";
+ break;
+ case HFI_ERROR_FATAL:
+ error = "fatal error";
+ break;
+ default:
+ error = "unknown";
+ break;
+ }
+
+ dev_err(core->dev, "session error received %#x: %s\n",
+ pkt->type, error);
+ iris_inst_change_state(inst, IRIS_INST_ERROR);
+
+ return 0;
+}
+
static int handle_system_error(struct iris_core *core,
struct hfi_packet *pkt)
{
@@ -86,6 +142,301 @@ static int handle_system_error(struct iris_core *core,
return 0;
}

+static int handle_system_init(struct iris_core *core,
+ struct hfi_packet *pkt)
+{
+ if (!(pkt->flags & HFI_FW_FLAGS_SUCCESS))
+ return 0;
+
+ mutex_lock(&core->lock);
+ if (pkt->packet_id != core->sys_init_id)
+ goto unlock;
+
+ iris_change_core_state(core, IRIS_CORE_INIT);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return 0;
+}
+
+static int handle_session_close(struct iris_inst *inst,
+ struct hfi_packet *pkt)
+{
+ signal_session_msg_receipt(inst, SIGNAL_CMD_CLOSE);
+
+ return 0;
+}
+
+static int handle_src_change(struct iris_inst *inst,
+ struct hfi_packet *pkt)
+{
+ int ret = 0;
+
+ if (pkt->port == HFI_PORT_BITSTREAM)
+ ret = vdec_src_change(inst);
+ else if (pkt->port == HFI_PORT_RAW)
+ ret = 0;
+
+ if (ret)
+ iris_inst_change_state(inst, IRIS_INST_ERROR);
+
+ return ret;
+}
+
+static int handle_session_command(struct iris_inst *inst,
+ struct hfi_packet *pkt)
+{
+ int i, ret = 0;
+ static const struct iris_hfi_packet_handle hfi_pkt_handle[] = {
+ {HFI_CMD_OPEN, NULL },
+ {HFI_CMD_CLOSE, handle_session_close },
+ {HFI_CMD_SETTINGS_CHANGE, handle_src_change },
+ {HFI_CMD_SUBSCRIBE_MODE, NULL },
+ };
+
+ for (i = 0; i < ARRAY_SIZE(hfi_pkt_handle); i++) {
+ if (hfi_pkt_handle[i].type == pkt->type) {
+ if (hfi_pkt_handle[i].handle) {
+ ret = hfi_pkt_handle[i].handle(inst, pkt);
+ if (ret)
+ return ret;
+ }
+ break;
+ }
+ }
+
+ if (i == ARRAY_SIZE(hfi_pkt_handle))
+ return -EINVAL;
+
+ return ret;
+}
+
+static int handle_dpb_list_property(struct iris_inst *inst,
+ struct hfi_packet *pkt)
+{
+ u8 *payload_start;
+ u32 payload_size;
+
+ payload_size = pkt->size - sizeof(*pkt);
+ payload_start = (u8 *)((u8 *)pkt + sizeof(*pkt));
+ memset(inst->dpb_list_payload, 0, MAX_DPB_LIST_ARRAY_SIZE);
+
+ if (payload_size > MAX_DPB_LIST_PAYLOAD_SIZE) {
+ iris_inst_change_state(inst, IRIS_INST_ERROR);
+ return -EINVAL;
+ }
+
+ memcpy(inst->dpb_list_payload, payload_start, payload_size);
+
+ return 0;
+}
+
+static int handle_session_property(struct iris_inst *inst,
+ struct hfi_packet *pkt)
+{
+ u32 *payload_ptr = NULL;
+ int ret = 0;
+
+ if (pkt->port != HFI_PORT_BITSTREAM)
+ return 0;
+
+ if (pkt->flags & HFI_FW_FLAGS_INFORMATION)
+ return 0;
+
+ payload_ptr = (u32 *)((u8 *)pkt + sizeof(*pkt));
+ if (!payload_ptr)
+ return -EINVAL;
+
+ switch (pkt->type) {
+ case HFI_PROP_BITSTREAM_RESOLUTION:
+ inst->src_subcr_params.bitstream_resolution = payload_ptr[0];
+ break;
+ case HFI_PROP_CROP_OFFSETS:
+ inst->src_subcr_params.crop_offsets[0] = payload_ptr[0];
+ inst->src_subcr_params.crop_offsets[1] = payload_ptr[1];
+ break;
+ case HFI_PROP_LUMA_CHROMA_BIT_DEPTH:
+ inst->src_subcr_params.bit_depth = payload_ptr[0];
+ break;
+ case HFI_PROP_CODED_FRAMES:
+ inst->src_subcr_params.coded_frames = payload_ptr[0];
+ break;
+ case HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT:
+ inst->src_subcr_params.fw_min_count = payload_ptr[0];
+ break;
+ case HFI_PROP_PIC_ORDER_CNT_TYPE:
+ inst->src_subcr_params.pic_order_cnt = payload_ptr[0];
+ break;
+ case HFI_PROP_SIGNAL_COLOR_INFO:
+ inst->src_subcr_params.color_info = payload_ptr[0];
+ break;
+ case HFI_PROP_PROFILE:
+ inst->src_subcr_params.profile = payload_ptr[0];
+ break;
+ case HFI_PROP_LEVEL:
+ inst->src_subcr_params.level = payload_ptr[0];
+ break;
+ case HFI_PROP_TIER:
+ inst->src_subcr_params.tier = payload_ptr[0];
+ break;
+ case HFI_PROP_PICTURE_TYPE:
+ inst->hfi_frame_info.picture_type = payload_ptr[0];
+ break;
+ case HFI_PROP_DPB_LIST:
+ ret = handle_dpb_list_property(inst, pkt);
+ if (ret)
+ break;
+ break;
+ case HFI_PROP_NO_OUTPUT:
+ inst->hfi_frame_info.no_output = 1;
+ break;
+ case HFI_PROP_QUALITY_MODE:
+ case HFI_PROP_STAGE:
+ case HFI_PROP_PIPE:
+ break;
+ default:
+ break;
+ }
+
+ return ret;
+}
+
+static int handle_image_version_property(struct iris_core *core,
+ struct hfi_packet *pkt)
+{
+ u8 *str_image_version;
+ u32 req_bytes;
+ u32 i = 0;
+
+ req_bytes = pkt->size - sizeof(*pkt);
+ if (req_bytes < IRIS_VERSION_LENGTH - 1)
+ return -EINVAL;
+
+ str_image_version = (u8 *)pkt + sizeof(*pkt);
+
+ for (i = 0; i < IRIS_VERSION_LENGTH - 1; i++) {
+ if (str_image_version[i] != '\0')
+ core->fw_version[i] = str_image_version[i];
+ else
+ core->fw_version[i] = ' ';
+ }
+ core->fw_version[i] = '\0';
+
+ return 0;
+}
+
+static int handle_system_property(struct iris_core *core,
+ struct hfi_packet *pkt)
+{
+ int ret = 0;
+
+ switch (pkt->type) {
+ case HFI_PROP_IMAGE_VERSION:
+ ret = handle_image_version_property(core, pkt);
+ break;
+ default:
+ break;
+ }
+
+ return ret;
+}
+
+static int handle_system_response(struct iris_core *core,
+ struct hfi_header *hdr)
+{
+ struct hfi_packet *packet;
+ u8 *pkt, *start_pkt;
+ int ret = 0;
+ int i, j;
+ static const struct iris_core_hfi_range be[] = {
+ {HFI_SYSTEM_ERROR_BEGIN, HFI_SYSTEM_ERROR_END, handle_system_error },
+ {HFI_PROP_BEGIN, HFI_PROP_END, handle_system_property },
+ {HFI_CMD_BEGIN, HFI_CMD_END, handle_system_init },
+ };
+
+ start_pkt = (u8 *)((u8 *)hdr + sizeof(*hdr));
+ for (i = 0; i < ARRAY_SIZE(be); i++) {
+ pkt = start_pkt;
+ for (j = 0; j < hdr->num_packets; j++) {
+ packet = (struct hfi_packet *)pkt;
+ if (packet->flags & HFI_FW_FLAGS_SYSTEM_ERROR) {
+ ret = handle_system_error(core, packet);
+ return ret;
+ }
+
+ if (packet->type > be[i].begin && packet->type < be[i].end) {
+ ret = be[i].handle(core, packet);
+ if (ret)
+ return ret;
+
+ if (packet->type > HFI_SYSTEM_ERROR_BEGIN &&
+ packet->type < HFI_SYSTEM_ERROR_END)
+ return 0;
+ }
+ pkt += packet->size;
+ }
+ }
+
+ return ret;
+}
+
+static int handle_session_response(struct iris_core *core,
+ struct hfi_header *hdr)
+{
+ struct hfi_packet *packet;
+ struct iris_inst *inst;
+ u8 *pkt, *start_pkt;
+ int ret = 0;
+ int i, j;
+ static const struct iris_inst_hfi_range be[] = {
+ {HFI_SESSION_ERROR_BEGIN, HFI_SESSION_ERROR_END, handle_session_error },
+ {HFI_PROP_BEGIN, HFI_PROP_END, handle_session_property },
+ {HFI_CMD_BEGIN, HFI_CMD_END, handle_session_command },
+ };
+
+ inst = to_instance(core, hdr->session_id);
+ if (!inst)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ memset(&inst->hfi_frame_info, 0, sizeof(struct iris_hfi_frame_info));
+
+ pkt = (u8 *)((u8 *)hdr + sizeof(*hdr));
+ for (i = 0; i < hdr->num_packets; i++) {
+ packet = (struct hfi_packet *)pkt;
+ if (packet->type == HFI_CMD_SETTINGS_CHANGE) {
+ if (packet->port == HFI_PORT_BITSTREAM) {
+ vdec_init_src_change_param(inst);
+ break;
+ }
+ }
+ pkt += packet->size;
+ }
+
+ start_pkt = (u8 *)((u8 *)hdr + sizeof(*hdr));
+ for (i = 0; i < ARRAY_SIZE(be); i++) {
+ pkt = start_pkt;
+ for (j = 0; j < hdr->num_packets; j++) {
+ packet = (struct hfi_packet *)pkt;
+ if (packet->flags & HFI_FW_FLAGS_SESSION_ERROR)
+ handle_session_error(inst, packet);
+
+ if (packet->type > be[i].begin && packet->type < be[i].end) {
+ ret = be[i].handle(inst, packet);
+ if (ret)
+ iris_inst_change_state(inst, IRIS_INST_ERROR);
+ }
+ pkt += packet->size;
+ }
+ }
+
+ memset(&inst->hfi_frame_info, 0, sizeof(struct iris_hfi_frame_info));
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
static int handle_response(struct iris_core *core, void *response)
{
struct hfi_header *hdr;
@@ -96,6 +447,11 @@ static int handle_response(struct iris_core *core, void *response)
if (ret)
return handle_system_error(core, NULL);

+ if (!hdr->session_id)
+ return handle_system_response(core, hdr);
+ else
+ return handle_session_response(core, hdr);
+
return ret;
}

diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
index 365f844..4f51d68 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -40,6 +40,10 @@
* @fw_min_count: minimnum count of buffers needed by fw
* @state: instance state
* @ipsc_properties_set: boolean to set ipsc properties to fw
+ * @hfi_frame_info: structure of frame info
+ * @src_subcr_params: subscription params to fw on input port
+ * @dst_subcr_params: subscription params to fw on output port
+ * @dpb_list_payload: array of dpb buffers
*/

struct iris_inst {
@@ -66,6 +70,10 @@ struct iris_inst {
u32 fw_min_count;
enum iris_inst_state state;
bool ipsc_properties_set;
+ struct iris_hfi_frame_info hfi_frame_info;
+ struct subscription_params src_subcr_params;
+ struct subscription_params dst_subcr_params;
+ u32 dpb_list_payload[MAX_DPB_LIST_ARRAY_SIZE];
};

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index 7d16c96..ac47fc0 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -14,6 +14,13 @@
#include "iris_hfi_packet.h"
#include "iris_vdec.h"

+#define UNSPECIFIED_COLOR_FORMAT 5
+
+struct vdec_prop_type_handle {
+ u32 type;
+ int (*handle)(struct iris_inst *inst);
+};
+
static int vdec_codec_change(struct iris_inst *inst, u32 v4l2_codec)
{
bool session_init = false;
@@ -376,6 +383,455 @@ int vdec_subscribe_property(struct iris_inst *inst, u32 plane)
(subscribe_prop_size + 1) * sizeof(u32));
}

+static int vdec_set_bitstream_resolution(struct iris_inst *inst)
+{
+ u32 resolution;
+
+ resolution = inst->fmt_src->fmt.pix_mp.width << 16 |
+ inst->fmt_src->fmt.pix_mp.height;
+ inst->src_subcr_params.bitstream_resolution = resolution;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_BITSTREAM_RESOLUTION,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(INPUT_MPLANE),
+ HFI_PAYLOAD_U32,
+ &resolution,
+ sizeof(u32));
+}
+
+static int vdec_set_crop_offsets(struct iris_inst *inst)
+{
+ u32 left_offset, top_offset, right_offset, bottom_offset;
+ u32 payload[2] = {0};
+
+ left_offset = inst->crop.left;
+ top_offset = inst->crop.top;
+ right_offset = (inst->fmt_src->fmt.pix_mp.width -
+ inst->crop.width);
+ bottom_offset = (inst->fmt_src->fmt.pix_mp.height -
+ inst->crop.height);
+
+ payload[0] = left_offset << 16 | top_offset;
+ payload[1] = right_offset << 16 | bottom_offset;
+ inst->src_subcr_params.crop_offsets[0] = payload[0];
+ inst->src_subcr_params.crop_offsets[1] = payload[1];
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_CROP_OFFSETS,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(INPUT_MPLANE),
+ HFI_PAYLOAD_64_PACKED,
+ &payload,
+ sizeof(u64));
+}
+
+static int vdec_set_bit_depth(struct iris_inst *inst)
+{
+ u32 bitdepth = 8 << 16 | 8;
+ u32 pix_fmt;
+
+ pix_fmt = inst->fmt_dst->fmt.pix_mp.pixelformat;
+ if (is_10bit_colorformat(pix_fmt))
+ bitdepth = 10 << 16 | 10;
+
+ inst->src_subcr_params.bit_depth = bitdepth;
+ inst->cap[BIT_DEPTH].value = bitdepth;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_LUMA_CHROMA_BIT_DEPTH,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(INPUT_MPLANE),
+ HFI_PAYLOAD_U32,
+ &bitdepth,
+ sizeof(u32));
+}
+
+static int vdec_set_coded_frames(struct iris_inst *inst)
+{
+ u32 coded_frames = 0;
+
+ if (inst->cap[CODED_FRAMES].value == CODED_FRAMES_PROGRESSIVE)
+ coded_frames = HFI_BITMASK_FRAME_MBS_ONLY_FLAG;
+ inst->src_subcr_params.coded_frames = coded_frames;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_CODED_FRAMES,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(INPUT_MPLANE),
+ HFI_PAYLOAD_U32,
+ &coded_frames,
+ sizeof(u32));
+}
+
+static int vdec_set_min_output_count(struct iris_inst *inst)
+{
+ u32 min_output;
+
+ min_output = inst->buffers.output.min_count;
+ inst->src_subcr_params.fw_min_count = min_output;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(INPUT_MPLANE),
+ HFI_PAYLOAD_U32,
+ &min_output,
+ sizeof(u32));
+}
+
+static int vdec_set_picture_order_count(struct iris_inst *inst)
+{
+ u32 poc = 0;
+
+ inst->src_subcr_params.pic_order_cnt = poc;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_PIC_ORDER_CNT_TYPE,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(INPUT_MPLANE),
+ HFI_PAYLOAD_U32,
+ &poc,
+ sizeof(u32));
+}
+
+static int vdec_set_colorspace(struct iris_inst *inst)
+{
+ u32 video_signal_type_present_flag = 0, color_info = 0;
+ u32 matrix_coeff = HFI_MATRIX_COEFF_RESERVED;
+ u32 video_format = UNSPECIFIED_COLOR_FORMAT;
+ struct v4l2_pix_format_mplane *pixmp = NULL;
+ u32 full_range = V4L2_QUANTIZATION_DEFAULT;
+ u32 transfer_char = HFI_TRANSFER_RESERVED;
+ u32 colour_description_present_flag = 0;
+ u32 primaries = HFI_PRIMARIES_RESERVED;
+
+ int ret;
+
+ if (inst->codec == VP9)
+ return 0;
+
+ pixmp = &inst->fmt_src->fmt.pix_mp;
+ if (pixmp->colorspace != V4L2_COLORSPACE_DEFAULT ||
+ pixmp->ycbcr_enc != V4L2_YCBCR_ENC_DEFAULT ||
+ pixmp->xfer_func != V4L2_XFER_FUNC_DEFAULT) {
+ colour_description_present_flag = 1;
+ video_signal_type_present_flag = 1;
+ primaries = get_hfi_color_primaries(pixmp->colorspace);
+ matrix_coeff = get_hfi_matrix_coefficients(pixmp->ycbcr_enc);
+ transfer_char = get_hfi_transfer_char(pixmp->xfer_func);
+ }
+
+ if (pixmp->quantization != V4L2_QUANTIZATION_DEFAULT) {
+ video_signal_type_present_flag = 1;
+ full_range = pixmp->quantization ==
+ V4L2_QUANTIZATION_FULL_RANGE ? 1 : 0;
+ }
+
+ color_info = (matrix_coeff & 0xFF) |
+ ((transfer_char << 8) & 0xFF00) |
+ ((primaries << 16) & 0xFF0000) |
+ ((colour_description_present_flag << 24) & 0x1000000) |
+ ((full_range << 25) & 0x2000000) |
+ ((video_format << 26) & 0x1C000000) |
+ ((video_signal_type_present_flag << 29) & 0x20000000);
+
+ inst->src_subcr_params.color_info = color_info;
+
+ ret = iris_hfi_set_property(inst,
+ HFI_PROP_SIGNAL_COLOR_INFO,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(INPUT_MPLANE),
+ HFI_PAYLOAD_32_PACKED,
+ &color_info,
+ sizeof(u32));
+
+ return ret;
+}
+
+static int vdec_set_profile(struct iris_inst *inst)
+{
+ u32 profile;
+
+ profile = inst->cap[PROFILE].value;
+ inst->src_subcr_params.profile = profile;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_PROFILE,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(INPUT_MPLANE),
+ HFI_PAYLOAD_U32_ENUM,
+ &profile,
+ sizeof(u32));
+}
+
+static int vdec_set_level(struct iris_inst *inst)
+{
+ u32 level;
+
+ level = inst->cap[LEVEL].value;
+ inst->src_subcr_params.level = level;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_LEVEL,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(INPUT_MPLANE),
+ HFI_PAYLOAD_U32_ENUM,
+ &level,
+ sizeof(u32));
+}
+
+static int vdec_set_tier(struct iris_inst *inst)
+{
+ u32 tier;
+
+ tier = inst->cap[HEVC_TIER].value;
+ inst->src_subcr_params.tier = tier;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_TIER,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(INPUT_MPLANE),
+ HFI_PAYLOAD_U32_ENUM,
+ &tier,
+ sizeof(u32));
+}
+
+int vdec_subscribe_src_change_param(struct iris_inst *inst)
+{
+ const u32 *src_change_param;
+ u32 src_change_param_size;
+ struct iris_core *core;
+ u32 payload[32] = {0};
+ int ret;
+ u32 i, j;
+
+ static const struct vdec_prop_type_handle prop_type_handle_arr[] = {
+ {HFI_PROP_BITSTREAM_RESOLUTION, vdec_set_bitstream_resolution },
+ {HFI_PROP_CROP_OFFSETS, vdec_set_crop_offsets },
+ {HFI_PROP_LUMA_CHROMA_BIT_DEPTH, vdec_set_bit_depth },
+ {HFI_PROP_CODED_FRAMES, vdec_set_coded_frames },
+ {HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT, vdec_set_min_output_count },
+ {HFI_PROP_PIC_ORDER_CNT_TYPE, vdec_set_picture_order_count },
+ {HFI_PROP_SIGNAL_COLOR_INFO, vdec_set_colorspace },
+ {HFI_PROP_PROFILE, vdec_set_profile },
+ {HFI_PROP_LEVEL, vdec_set_level },
+ {HFI_PROP_TIER, vdec_set_tier },
+ };
+
+ core = inst->core;
+
+ payload[0] = HFI_MODE_PORT_SETTINGS_CHANGE;
+ if (inst->codec == H264) {
+ src_change_param_size = core->platform_data->avc_subscribe_param_size;
+ src_change_param = core->platform_data->avc_subscribe_param;
+ } else if (inst->codec == HEVC) {
+ src_change_param_size = core->platform_data->hevc_subscribe_param_size;
+ src_change_param = core->platform_data->hevc_subscribe_param;
+ } else if (inst->codec == VP9) {
+ src_change_param_size = core->platform_data->vp9_subscribe_param_size;
+ src_change_param = core->platform_data->vp9_subscribe_param;
+ } else {
+ src_change_param = NULL;
+ return -EINVAL;
+ }
+
+ if (!src_change_param || !src_change_param_size)
+ return -EINVAL;
+
+ for (i = 0; i < src_change_param_size; i++)
+ payload[i + 1] = src_change_param[i];
+
+ ret = iris_hfi_session_subscribe_mode(inst,
+ HFI_CMD_SUBSCRIBE_MODE,
+ INPUT_MPLANE,
+ HFI_PAYLOAD_U32_ARRAY,
+ &payload[0],
+ ((src_change_param_size + 1) * sizeof(u32)));
+ if (ret)
+ return ret;
+
+ for (i = 0; i < src_change_param_size; i++) {
+ for (j = 0; j < ARRAY_SIZE(prop_type_handle_arr); j++) {
+ if (prop_type_handle_arr[j].type == src_change_param[i]) {
+ ret = prop_type_handle_arr[j].handle(inst);
+ if (ret)
+ return ret;
+ break;
+ }
+ }
+ }
+
+ return ret;
+}
+
+int vdec_init_src_change_param(struct iris_inst *inst)
+{
+ u32 left_offset, top_offset, right_offset, bottom_offset;
+ struct v4l2_pix_format_mplane *pixmp_ip, *pixmp_op;
+ u32 primaries, matrix_coeff, transfer_char;
+ struct subscription_params *subsc_params;
+ u32 colour_description_present_flag = 0;
+ u32 video_signal_type_present_flag = 0;
+ u32 full_range = 0, video_format = 0;
+
+ subsc_params = &inst->src_subcr_params;
+ pixmp_ip = &inst->fmt_src->fmt.pix_mp;
+ pixmp_op = &inst->fmt_dst->fmt.pix_mp;
+
+ subsc_params->bitstream_resolution =
+ pixmp_ip->width << 16 | pixmp_ip->height;
+
+ left_offset = inst->crop.left;
+ top_offset = inst->crop.top;
+ right_offset = (pixmp_ip->width - inst->crop.width);
+ bottom_offset = (pixmp_ip->height - inst->crop.height);
+ subsc_params->crop_offsets[0] =
+ left_offset << 16 | top_offset;
+ subsc_params->crop_offsets[1] =
+ right_offset << 16 | bottom_offset;
+
+ subsc_params->fw_min_count = inst->buffers.output.min_count;
+
+ primaries = get_hfi_color_primaries(pixmp_op->colorspace);
+ matrix_coeff = get_hfi_matrix_coefficients(pixmp_op->ycbcr_enc);
+ transfer_char = get_hfi_transfer_char(pixmp_op->xfer_func);
+ full_range = pixmp_op->quantization == V4L2_QUANTIZATION_FULL_RANGE ? 1 : 0;
+ subsc_params->color_info =
+ (matrix_coeff & 0xFF) |
+ ((transfer_char << 8) & 0xFF00) |
+ ((primaries << 16) & 0xFF0000) |
+ ((colour_description_present_flag << 24) & 0x1000000) |
+ ((full_range << 25) & 0x2000000) |
+ ((video_format << 26) & 0x1C000000) |
+ ((video_signal_type_present_flag << 29) & 0x20000000);
+
+ subsc_params->profile = inst->cap[PROFILE].value;
+ subsc_params->level = inst->cap[LEVEL].value;
+ subsc_params->tier = inst->cap[HEVC_TIER].value;
+ subsc_params->pic_order_cnt = inst->cap[POC].value;
+ subsc_params->bit_depth = inst->cap[BIT_DEPTH].value;
+ if (inst->cap[CODED_FRAMES].value ==
+ CODED_FRAMES_PROGRESSIVE)
+ subsc_params->coded_frames = HFI_BITMASK_FRAME_MBS_ONLY_FLAG;
+ else
+ subsc_params->coded_frames = 0;
+
+ return 0;
+}
+
+static int vdec_read_input_subcr_params(struct iris_inst *inst)
+{
+ struct v4l2_pix_format_mplane *pixmp_ip, *pixmp_op;
+ u32 primaries, matrix_coeff, transfer_char;
+ struct subscription_params subsc_params;
+ u32 colour_description_present_flag = 0;
+ u32 video_signal_type_present_flag = 0;
+ u32 full_range = 0;
+ u32 width, height;
+
+ subsc_params = inst->src_subcr_params;
+ pixmp_ip = &inst->fmt_src->fmt.pix_mp;
+ pixmp_op = &inst->fmt_dst->fmt.pix_mp;
+ width = (subsc_params.bitstream_resolution &
+ HFI_BITMASK_BITSTREAM_WIDTH) >> 16;
+ height = subsc_params.bitstream_resolution &
+ HFI_BITMASK_BITSTREAM_HEIGHT;
+
+ pixmp_ip->width = width;
+ pixmp_ip->height = height;
+
+ pixmp_op->width = pixmp_op->pixelformat == V4L2_PIX_FMT_QC10C ?
+ ALIGN(width, 192) : ALIGN(width, 128);
+ pixmp_op->height = pixmp_op->pixelformat == V4L2_PIX_FMT_QC10C ?
+ ALIGN(height, 16) : ALIGN(height, 32);
+ pixmp_op->plane_fmt[0].bytesperline =
+ pixmp_op->pixelformat == V4L2_PIX_FMT_QC10C ?
+ ALIGN(ALIGN(width, 192) * 4 / 3, 256) :
+ ALIGN(width, 128);
+ pixmp_op->plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_OUTPUT);
+
+ matrix_coeff = subsc_params.color_info & 0xFF;
+ transfer_char = (subsc_params.color_info & 0xFF00) >> 8;
+ primaries = (subsc_params.color_info & 0xFF0000) >> 16;
+ colour_description_present_flag =
+ (subsc_params.color_info & 0x1000000) >> 24;
+ full_range = (subsc_params.color_info & 0x2000000) >> 25;
+ video_signal_type_present_flag =
+ (subsc_params.color_info & 0x20000000) >> 29;
+
+ pixmp_op->colorspace = V4L2_COLORSPACE_DEFAULT;
+ pixmp_op->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+ pixmp_op->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ pixmp_op->quantization = V4L2_QUANTIZATION_DEFAULT;
+
+ if (video_signal_type_present_flag) {
+ pixmp_op->quantization =
+ full_range ?
+ V4L2_QUANTIZATION_FULL_RANGE :
+ V4L2_QUANTIZATION_LIM_RANGE;
+ if (colour_description_present_flag) {
+ pixmp_op->colorspace =
+ get_v4l2_color_primaries(primaries);
+ pixmp_op->xfer_func =
+ get_v4l2_transfer_char(transfer_char);
+ pixmp_op->ycbcr_enc =
+ get_v4l2_matrix_coefficients(matrix_coeff);
+ }
+ }
+
+ pixmp_ip->colorspace = pixmp_op->colorspace;
+ pixmp_ip->xfer_func = pixmp_op->xfer_func;
+ pixmp_ip->ycbcr_enc = pixmp_op->ycbcr_enc;
+ pixmp_ip->quantization = pixmp_op->quantization;
+
+ inst->crop.top = subsc_params.crop_offsets[0] & 0xFFFF;
+ inst->crop.left = (subsc_params.crop_offsets[0] >> 16) & 0xFFFF;
+ inst->crop.height = pixmp_ip->height -
+ (subsc_params.crop_offsets[1] & 0xFFFF) - inst->crop.top;
+ inst->crop.width = pixmp_ip->width -
+ ((subsc_params.crop_offsets[1] >> 16) & 0xFFFF) - inst->crop.left;
+
+ inst->cap[PROFILE].value = subsc_params.profile;
+ inst->cap[LEVEL].value = subsc_params.level;
+ inst->cap[HEVC_TIER].value = subsc_params.tier;
+ inst->cap[POC].value = subsc_params.pic_order_cnt;
+
+ if (subsc_params.bit_depth == BIT_DEPTH_8)
+ inst->cap[BIT_DEPTH].value = BIT_DEPTH_8;
+ else
+ inst->cap[BIT_DEPTH].value = BIT_DEPTH_10;
+
+ if (subsc_params.coded_frames & HFI_BITMASK_FRAME_MBS_ONLY_FLAG)
+ inst->cap[CODED_FRAMES].value = CODED_FRAMES_PROGRESSIVE;
+ else
+ inst->cap[CODED_FRAMES].value = CODED_FRAMES_INTERLACE;
+
+ inst->fw_min_count = subsc_params.fw_min_count;
+ inst->buffers.output.min_count = iris_get_buf_min_count(inst, BUF_OUTPUT);
+
+ return 0;
+}
+
+int vdec_src_change(struct iris_inst *inst)
+{
+ struct v4l2_event event = {0};
+ u32 ret;
+
+ if (!inst->vb2q_src->streaming)
+ return 0;
+
+ ret = vdec_read_input_subcr_params(inst);
+ if (ret)
+ return ret;
+
+ event.type = V4L2_EVENT_SOURCE_CHANGE;
+ event.u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION;
+ v4l2_event_queue_fh(&inst->fh, &event);
+
+ return ret;
+}
+
static int vdec_set_colorformat(struct iris_inst *inst)
{
u32 hfi_colorformat;
@@ -483,3 +939,127 @@ int vdec_set_output_property(struct iris_inst *inst)

return vdec_set_ubwc_stride_scanline(inst);
}
+
+int vdec_subscribe_dst_change_param(struct iris_inst *inst)
+{
+ u32 prop_type, payload_size, payload_type;
+ struct subscription_params subsc_params;
+ const u32 *dst_change_param = NULL;
+ u32 dst_change_param_size = 0;
+ struct iris_core *core;
+ u32 payload[32] = {0};
+ int ret;
+ u32 i;
+
+ core = inst->core;
+
+ payload[0] = HFI_MODE_PORT_SETTINGS_CHANGE;
+ if (inst->codec == H264) {
+ dst_change_param_size = core->platform_data->avc_subscribe_param_size;
+ dst_change_param = core->platform_data->avc_subscribe_param;
+ } else if (inst->codec == HEVC) {
+ dst_change_param_size = core->platform_data->hevc_subscribe_param_size;
+ dst_change_param = core->platform_data->hevc_subscribe_param;
+ } else if (inst->codec == VP9) {
+ dst_change_param_size = core->platform_data->vp9_subscribe_param_size;
+ dst_change_param = core->platform_data->vp9_subscribe_param;
+ } else {
+ dst_change_param = NULL;
+ return -EINVAL;
+ }
+
+ if (!dst_change_param || !dst_change_param_size)
+ return -EINVAL;
+
+ payload[0] = HFI_MODE_PORT_SETTINGS_CHANGE;
+ for (i = 0; i < dst_change_param_size; i++)
+ payload[i + 1] = dst_change_param[i];
+
+ ret = iris_hfi_session_subscribe_mode(inst,
+ HFI_CMD_SUBSCRIBE_MODE,
+ OUTPUT_MPLANE,
+ HFI_PAYLOAD_U32_ARRAY,
+ &payload[0],
+ ((dst_change_param_size + 1) * sizeof(u32)));
+ if (ret)
+ return ret;
+
+ subsc_params = inst->dst_subcr_params;
+ for (i = 0; i < dst_change_param_size; i++) {
+ payload[0] = 0;
+ payload[1] = 0;
+ payload_size = 0;
+ payload_type = 0;
+ prop_type = dst_change_param[i];
+ switch (prop_type) {
+ case HFI_PROP_BITSTREAM_RESOLUTION:
+ payload[0] = subsc_params.bitstream_resolution;
+ payload_size = sizeof(u32);
+ payload_type = HFI_PAYLOAD_U32;
+ break;
+ case HFI_PROP_CROP_OFFSETS:
+ payload[0] = subsc_params.crop_offsets[0];
+ payload[1] = subsc_params.crop_offsets[1];
+ payload_size = sizeof(u64);
+ payload_type = HFI_PAYLOAD_64_PACKED;
+ break;
+ case HFI_PROP_LUMA_CHROMA_BIT_DEPTH:
+ payload[0] = subsc_params.bit_depth;
+ payload_size = sizeof(u32);
+ payload_type = HFI_PAYLOAD_U32;
+ break;
+ case HFI_PROP_CODED_FRAMES:
+ payload[0] = subsc_params.coded_frames;
+ payload_size = sizeof(u32);
+ payload_type = HFI_PAYLOAD_U32;
+ break;
+ case HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT:
+ payload[0] = subsc_params.fw_min_count;
+ payload_size = sizeof(u32);
+ payload_type = HFI_PAYLOAD_U32;
+ break;
+ case HFI_PROP_PIC_ORDER_CNT_TYPE:
+ payload[0] = subsc_params.pic_order_cnt;
+ payload_size = sizeof(u32);
+ payload_type = HFI_PAYLOAD_U32;
+ break;
+ case HFI_PROP_SIGNAL_COLOR_INFO:
+ payload[0] = subsc_params.color_info;
+ payload_size = sizeof(u32);
+ payload_type = HFI_PAYLOAD_U32;
+ break;
+ case HFI_PROP_PROFILE:
+ payload[0] = subsc_params.profile;
+ payload_size = sizeof(u32);
+ payload_type = HFI_PAYLOAD_U32;
+ break;
+ case HFI_PROP_LEVEL:
+ payload[0] = subsc_params.level;
+ payload_size = sizeof(u32);
+ payload_type = HFI_PAYLOAD_U32;
+ break;
+ case HFI_PROP_TIER:
+ payload[0] = subsc_params.tier;
+ payload_size = sizeof(u32);
+ payload_type = HFI_PAYLOAD_U32;
+ break;
+ default:
+ prop_type = 0;
+ ret = -EINVAL;
+ break;
+ }
+ if (prop_type) {
+ ret = iris_hfi_set_property(inst,
+ prop_type,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(OUTPUT_MPLANE),
+ payload_type,
+ &payload,
+ payload_size);
+ if (ret)
+ return ret;
+ }
+ }
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
index 6b0306c..e0db653 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
@@ -16,5 +16,9 @@ int vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f);
int vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscription *sub);
int vdec_subscribe_property(struct iris_inst *inst, u32 plane);
int vdec_set_output_property(struct iris_inst *inst);
+int vdec_init_src_change_param(struct iris_inst *inst);
+int vdec_src_change(struct iris_inst *inst);
+int vdec_subscribe_src_change_param(struct iris_inst *inst);
+int vdec_subscribe_dst_change_param(struct iris_inst *inst);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
index fc12bde..22a8f5b 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
@@ -206,6 +206,12 @@ struct platform_data {
u32 core_data_size;
struct plat_inst_cap *inst_cap_data;
u32 inst_cap_data_size;
+ const u32 *avc_subscribe_param;
+ unsigned int avc_subscribe_param_size;
+ const u32 *hevc_subscribe_param;
+ unsigned int hevc_subscribe_param_size;
+ const u32 *vp9_subscribe_param;
+ unsigned int vp9_subscribe_param_size;
const u32 *dec_input_prop;
unsigned int dec_input_prop_size;
const u32 *dec_output_prop_avc;
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
index 6a4bfa3..7ae9715 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
@@ -382,6 +382,38 @@ static struct format_capability format_data_sm8550 = {
.color_format_info_size = ARRAY_SIZE(color_format_data_sm8550),
};

+static const u32 sm8550_vdec_src_change_param_avc[] = {
+ HFI_PROP_BITSTREAM_RESOLUTION,
+ HFI_PROP_CROP_OFFSETS,
+ HFI_PROP_CODED_FRAMES,
+ HFI_PROP_LUMA_CHROMA_BIT_DEPTH,
+ HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT,
+ HFI_PROP_PIC_ORDER_CNT_TYPE,
+ HFI_PROP_PROFILE,
+ HFI_PROP_LEVEL,
+ HFI_PROP_SIGNAL_COLOR_INFO,
+};
+
+static const u32 sm8550_vdec_src_change_param_hevc[] = {
+ HFI_PROP_BITSTREAM_RESOLUTION,
+ HFI_PROP_CROP_OFFSETS,
+ HFI_PROP_LUMA_CHROMA_BIT_DEPTH,
+ HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT,
+ HFI_PROP_PROFILE,
+ HFI_PROP_LEVEL,
+ HFI_PROP_TIER,
+ HFI_PROP_SIGNAL_COLOR_INFO,
+};
+
+static const u32 sm8550_vdec_src_change_param_vp9[] = {
+ HFI_PROP_BITSTREAM_RESOLUTION,
+ HFI_PROP_CROP_OFFSETS,
+ HFI_PROP_LUMA_CHROMA_BIT_DEPTH,
+ HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT,
+ HFI_PROP_PROFILE,
+ HFI_PROP_LEVEL,
+};
+
static const u32 sm8550_vdec_input_properties[] = {
HFI_PROP_NO_OUTPUT,
};
@@ -430,6 +462,18 @@ struct platform_data sm8550_data = {
.ubwc_config = ubwc_config_sm8550,
.format_data = &format_data_sm8550,

+ .avc_subscribe_param =
+ sm8550_vdec_src_change_param_avc,
+ .avc_subscribe_param_size =
+ ARRAY_SIZE(sm8550_vdec_src_change_param_avc),
+ .hevc_subscribe_param =
+ sm8550_vdec_src_change_param_hevc,
+ .hevc_subscribe_param_size =
+ ARRAY_SIZE(sm8550_vdec_src_change_param_hevc),
+ .vp9_subscribe_param =
+ sm8550_vdec_src_change_param_vp9,
+ .vp9_subscribe_param_size =
+ ARRAY_SIZE(sm8550_vdec_src_change_param_vp9),
.dec_input_prop = sm8550_vdec_input_properties,
.dec_input_prop_size = ARRAY_SIZE(sm8550_vdec_input_properties),
.dec_output_prop_avc = sm8550_vdec_output_properties_avc,
--
2.7.4

2023-12-18 11:57:00

[permalink] [raw]

Subject: [PATCH v2 16/34] media: iris: implement iris v4l2_ctrl_ops and prepare capabilities

Implement iris v4l2_ctrl_ops - s_ctrl and g_volatile_ctrl.

Core capability structure has all the supported capabilities
for all supported codecs. Initializes instance capability
from core capability based on the codec type.

Add following to each capability:
Children: define dependencies for a specific capability.
Adjust: define function to adjust the value of capability
based on client configuration or dependency with
other capability.
Set: define function to set the adjusted value to firmware.

Prepare dependency graph for all inter-dependent capabilities.
This is used to adjust the value of capabilities and set the
same to firmware.

Signed-off-by: Dikshita Agarwal <[email protected]>
---
drivers/media/platform/qcom/vcodec/iris/Makefile | 1 +
.../media/platform/qcom/vcodec/iris/iris_core.h | 4 +
.../media/platform/qcom/vcodec/iris/iris_ctrls.c | 679 +++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_ctrls.h | 30 +
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 31 +-
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 13 +-
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 29 +
drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 3 +
.../platform/qcom/vcodec/iris/iris_hfi_packet.c | 26 +
.../platform/qcom/vcodec/iris/iris_hfi_packet.h | 3 +
.../platform/qcom/vcodec/iris/iris_instance.h | 9 +
.../media/platform/qcom/vcodec/iris/iris_probe.c | 8 +
.../media/platform/qcom/vcodec/iris/iris_vidc.c | 16 +
.../platform/qcom/vcodec/iris/platform_common.h | 13 +
.../platform/qcom/vcodec/iris/platform_sm8550.c | 84 ++-
15 files changed, 921 insertions(+), 28 deletions(-)
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h

diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
index 85fdf63..7fdee5b 100644
--- a/drivers/media/platform/qcom/vcodec/iris/Makefile
+++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
@@ -6,6 +6,7 @@ iris-objs += iris_probe.o \
iris_vidc.o \
iris_vdec.o \
iris_state.o \
+ iris_ctrls.o \
iris_helpers.o \
iris_hfi.o \
iris_hfi_response.o \
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
index c125bce..30f7ad7 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
@@ -50,8 +50,10 @@
* @header_id: id of packet header
* @packet_id: id of packet
* @vpu_ops: a pointer to vpu ops
+ * @dec_codecs_count: supported codec count for decoder
* @platform_data: a structure for platform data
* @cap: an array for supported core capabilities
+ * @inst_caps: a pointer to supported instance capabilities
* @instances: a list_head of all instances
* @intr_status: interrupt status
*/
@@ -89,8 +91,10 @@ struct iris_core {
u32 header_id;
u32 packet_id;
const struct vpu_ops *vpu_ops;
+ u32 dec_codecs_count;
struct platform_data *platform_data;
struct plat_core_cap cap[CORE_CAP_MAX + 1];
+ struct plat_inst_caps *inst_caps;
struct list_head instances;
u32 intr_status;
};
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
new file mode 100644
index 0000000..6b7aeaa
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
@@ -0,0 +1,679 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+#include <linux/vmalloc.h>
+#include <linux/types.h>
+#include <linux/list.h>
+
+#include "platform_common.h"
+#include "iris_helpers.h"
+#include "iris_hfi_packet.h"
+#include "iris_hfi.h"
+#include "iris_ctrls.h"
+
+#define MIN_CAPTURE_BUFFERS 4
+#define MIN_OUTPUT_BUFFERS 4
+
+static inline bool are_all_childrens_visited(struct plat_inst_cap *cap,
+ bool lookup[INST_CAP_MAX])
+{
+ bool found = true;
+ int i;
+
+ for (i = 0; i < MAX_NUM_CHILD; i++) {
+ if (cap->children[i] == INST_CAP_NONE)
+ continue;
+
+ if (!lookup[cap->children[i]]) {
+ found = false;
+ break;
+ }
+ }
+
+ return found;
+}
+
+static bool is_valid_cap_id(enum plat_inst_cap_type cap_id)
+{
+ return cap_id > INST_CAP_NONE && cap_id < INST_CAP_MAX;
+}
+
+static enum plat_inst_cap_type get_cap_id(struct iris_inst *inst, u32 id)
+{
+ enum plat_inst_cap_type iter = INST_CAP_NONE;
+ enum plat_inst_cap_type cap_id = INST_CAP_NONE;
+
+ do {
+ if (inst->cap[iter].v4l2_id == id) {
+ cap_id = inst->cap[iter].cap_id;
+ break;
+ }
+ iter++;
+ } while (iter < INST_CAP_MAX);
+
+ return cap_id;
+}
+
+static int add_node_list(struct list_head *list, enum plat_inst_cap_type cap_id)
+{
+ struct cap_entry *entry = NULL;
+
+ entry = kzalloc(sizeof(*entry), GFP_KERNEL);
+ if (!entry)
+ return -ENOMEM;
+
+ INIT_LIST_HEAD(&entry->list);
+ entry->cap_id = cap_id;
+ list_add(&entry->list, list);
+
+ return 0;
+}
+
+static int add_children(struct list_head *list,
+ struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id)
+{
+ struct plat_inst_cap *cap;
+ int i, ret = 0;
+
+ cap = &inst->cap[cap_id];
+
+ for (i = 0; i < MAX_NUM_CHILD; i++) {
+ if (!cap->children[i])
+ break;
+
+ if (!is_valid_cap_id(cap->children[i]))
+ continue;
+
+ ret = add_node_list(list, cap->children[i]);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
+static int adjust_cap(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id,
+ struct v4l2_ctrl *ctrl)
+{
+ struct plat_inst_cap *cap;
+
+ cap = &inst->cap[cap_id];
+ if (!inst->cap[cap_id].cap_id)
+ return 0;
+
+ if (!cap->adjust) {
+ if (ctrl)
+ inst->cap[cap_id].value = ctrl->val;
+ return 0;
+ }
+
+ return cap->adjust(inst, ctrl);
+}
+
+static int adjust_dynamic_property(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id,
+ struct v4l2_ctrl *ctrl,
+ struct list_head *firmware_list,
+ struct list_head *children_list,
+ bool cap_present[INST_CAP_MAX])
+{
+ struct cap_entry *entry = NULL, *temp = NULL;
+ struct plat_inst_cap *cap;
+ s32 prev_value;
+ int ret;
+
+ cap = &inst->cap[0];
+
+ if (!(cap[cap_id].flags & CAP_FLAG_DYNAMIC_ALLOWED))
+ return -EBUSY;
+
+ prev_value = cap[cap_id].value;
+ ret = adjust_cap(inst, cap_id, ctrl);
+ if (ret)
+ return ret;
+
+ ret = add_node_list(firmware_list, cap_id);
+ if (ret)
+ return ret;
+ cap_present[cap->cap_id] = true;
+
+ if (cap[cap_id].value == prev_value)
+ return 0;
+
+ ret = add_children(children_list, inst, cap_id);
+ if (ret)
+ return ret;
+
+ list_for_each_entry_safe(entry, temp, children_list, list) {
+ if (!cap[entry->cap_id].adjust) {
+ list_del_init(&entry->list);
+ kfree(entry);
+ continue;
+ }
+
+ prev_value = cap[entry->cap_id].value;
+ ret = adjust_cap(inst, entry->cap_id, NULL);
+ if (ret)
+ return ret;
+
+ if (cap[entry->cap_id].value != prev_value) {
+ if (!cap_present[cap->cap_id]) {
+ ret = add_node_list(firmware_list, cap_id);
+ if (ret)
+ return ret;
+ cap_present[cap->cap_id] = true;
+ }
+
+ ret = add_children(children_list, inst, entry->cap_id);
+ if (ret)
+ return ret;
+ }
+
+ list_del_init(&entry->list);
+ kfree(entry);
+ }
+
+ if (!list_empty(children_list))
+ return -EINVAL;
+
+ return ret;
+}
+
+static int set_dynamic_property(struct iris_inst *inst,
+ struct list_head *firmware_list)
+{
+ struct cap_entry *entry = NULL, *temp = NULL;
+ struct plat_inst_cap *cap;
+ int ret = 0;
+
+ list_for_each_entry_safe(entry, temp, firmware_list, list) {
+ cap = &inst->cap[entry->cap_id];
+ if (cap->set) {
+ ret = cap->set(inst, entry->cap_id);
+ if (ret)
+ return -EINVAL;
+ }
+ list_del_init(&entry->list);
+ kfree(entry);
+ }
+
+ return ret;
+}
+
+static int vdec_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
+{
+ enum plat_inst_cap_type cap_id;
+ struct iris_inst *inst = NULL;
+ int ret = 0;
+
+ switch (ctrl->id) {
+ case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE:
+ ctrl->val = MIN_CAPTURE_BUFFERS;
+ break;
+ case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT:
+ ctrl->val = MIN_OUTPUT_BUFFERS;
+ break;
+ default:
+ inst = container_of(ctrl->handler, struct iris_inst, ctrl_handler);
+ cap_id = get_cap_id(inst, ctrl->id);
+ if (is_valid_cap_id(cap_id))
+ ctrl->val = inst->cap[cap_id].value;
+ else
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
+static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct cap_entry *entry = NULL, *temp = NULL;
+ struct list_head children_list, firmware_list;
+ enum plat_inst_cap_type cap_id;
+ bool cap_present[INST_CAP_MAX];
+ struct plat_inst_cap *cap;
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = container_of(ctrl->handler, struct iris_inst, ctrl_handler);
+ cap = &inst->cap[0];
+
+ INIT_LIST_HEAD(&firmware_list);
+ INIT_LIST_HEAD(&children_list);
+ memset(&cap_present, 0, sizeof(cap_present));
+
+ cap_id = get_cap_id(inst, ctrl->id);
+ if (!is_valid_cap_id(cap_id))
+ return -EINVAL;
+
+ cap[cap_id].flags |= CAP_FLAG_CLIENT_SET;
+
+ if (!inst->vb2q_src->streaming) {
+ inst->cap[cap_id].value = ctrl->val;
+ } else {
+ ret = adjust_dynamic_property(inst, cap_id, ctrl,
+ &firmware_list, &children_list,
+ cap_present);
+ if (ret)
+ goto free_list;
+
+ ret = set_dynamic_property(inst, &firmware_list);
+ }
+
+free_list:
+ list_for_each_entry_safe(entry, temp, &children_list, list) {
+ list_del_init(&entry->list);
+ kfree(entry);
+ }
+
+ list_for_each_entry_safe(entry, temp, &firmware_list, list) {
+ list_del_init(&entry->list);
+ kfree(entry);
+ }
+
+ return ret;
+}
+
+static const struct v4l2_ctrl_ops ctrl_ops = {
+ .s_ctrl = vdec_op_s_ctrl,
+ .g_volatile_ctrl = vdec_op_g_volatile_ctrl,
+};
+
+int ctrls_init(struct iris_inst *inst)
+{
+ int num_ctrls = 0, ctrl_idx = 0;
+ struct plat_inst_cap *cap;
+ struct iris_core *core;
+ int idx = 0;
+ int ret = 0;
+
+ core = inst->core;
+ cap = &inst->cap[0];
+
+ for (idx = 0; idx < INST_CAP_MAX; idx++) {
+ if (cap[idx].v4l2_id)
+ num_ctrls++;
+ }
+ if (!num_ctrls)
+ return -EINVAL;
+
+ ret = v4l2_ctrl_handler_init(&inst->ctrl_handler,
+ INST_CAP_MAX * core->dec_codecs_count);
+ if (ret)
+ return ret;
+
+ for (idx = 0; idx < INST_CAP_MAX; idx++) {
+ struct v4l2_ctrl *ctrl;
+
+ if (!cap[idx].v4l2_id)
+ continue;
+
+ if (ctrl_idx >= num_ctrls) {
+ ret = -EINVAL;
+ goto error;
+ }
+
+ if (cap[idx].flags & CAP_FLAG_MENU) {
+ ctrl = v4l2_ctrl_new_std_menu(&inst->ctrl_handler,
+ &ctrl_ops,
+ cap[idx].v4l2_id,
+ cap[idx].max,
+ ~(cap[idx].step_or_mask),
+ cap[idx].value);
+ } else {
+ ctrl = v4l2_ctrl_new_std(&inst->ctrl_handler,
+ &ctrl_ops,
+ cap[idx].v4l2_id,
+ cap[idx].min,
+ cap[idx].max,
+ cap[idx].step_or_mask,
+ cap[idx].value);
+ }
+ if (!ctrl) {
+ ret = -EINVAL;
+ goto error;
+ }
+
+ ret = inst->ctrl_handler.error;
+ if (ret)
+ goto error;
+
+ if ((cap[idx].flags & CAP_FLAG_VOLATILE) ||
+ (ctrl->id == V4L2_CID_MIN_BUFFERS_FOR_CAPTURE ||
+ ctrl->id == V4L2_CID_MIN_BUFFERS_FOR_OUTPUT))
+ ctrl->flags |= V4L2_CTRL_FLAG_VOLATILE;
+
+ ctrl->flags |= V4L2_CTRL_FLAG_EXECUTE_ON_WRITE;
+ ctrl_idx++;
+ }
+ inst->num_ctrls = num_ctrls;
+
+ return 0;
+error:
+ v4l2_ctrl_handler_free(&inst->ctrl_handler);
+
+ return ret;
+}
+
+int iris_init_core_caps(struct iris_core *core)
+{
+ struct plat_core_cap *core_platform_data;
+ int i, num_core_caps;
+
+ core_platform_data = core->platform_data->core_data;
+ if (!core_platform_data)
+ return -EINVAL;
+
+ num_core_caps = core->platform_data->core_data_size;
+
+ for (i = 0; i < num_core_caps && i < CORE_CAP_MAX; i++) {
+ core->cap[core_platform_data[i].type].type = core_platform_data[i].type;
+ core->cap[core_platform_data[i].type].value = core_platform_data[i].value;
+ }
+
+ return 0;
+}
+
+static int update_inst_capability(struct plat_inst_cap *in,
+ struct plat_inst_caps *capability)
+{
+ if (!in || !capability)
+ return -EINVAL;
+
+ if (in->cap_id >= INST_CAP_MAX)
+ return -EINVAL;
+
+ capability->cap[in->cap_id].cap_id = in->cap_id;
+ capability->cap[in->cap_id].min = in->min;
+ capability->cap[in->cap_id].max = in->max;
+ capability->cap[in->cap_id].step_or_mask = in->step_or_mask;
+ capability->cap[in->cap_id].value = in->value;
+ capability->cap[in->cap_id].flags = in->flags;
+ capability->cap[in->cap_id].v4l2_id = in->v4l2_id;
+ capability->cap[in->cap_id].hfi_id = in->hfi_id;
+ memcpy(capability->cap[in->cap_id].children, in->children,
+ sizeof(capability->cap[in->cap_id].children));
+ capability->cap[in->cap_id].adjust = in->adjust;
+ capability->cap[in->cap_id].set = in->set;
+
+ return 0;
+}
+
+int iris_init_instance_caps(struct iris_core *core)
+{
+ struct plat_inst_cap *inst_plat_cap_data = NULL;
+ u8 dec_codecs_count = 0;
+ int num_inst_cap;
+ u32 dec_valid_codecs;
+ int i, j, check_bit = 0;
+ int ret = 0;
+
+ inst_plat_cap_data = core->platform_data->inst_cap_data;
+ if (!inst_plat_cap_data)
+ return -EINVAL;
+
+ dec_valid_codecs = core->cap[DEC_CODECS].value;
+ dec_codecs_count = hweight32(dec_valid_codecs);
+ core->dec_codecs_count = dec_codecs_count;
+
+ core->inst_caps = devm_kzalloc(core->dev,
+ dec_codecs_count * sizeof(struct plat_inst_caps),
+ GFP_KERNEL);
+ if (!core->inst_caps)
+ return -ENOMEM;
+
+ for (i = 0; i < dec_codecs_count; i++) {
+ while (check_bit < (sizeof(dec_valid_codecs) * 8)) {
+ if (dec_valid_codecs & BIT(check_bit)) {
+ core->inst_caps[i].codec = dec_valid_codecs &
+ BIT(check_bit);
+ check_bit++;
+ break;
+ }
+ check_bit++;
+ }
+ }
+
+ num_inst_cap = core->platform_data->inst_cap_data_size;
+
+ for (i = 0; i < num_inst_cap; i++) {
+ for (j = 0; j < dec_codecs_count; j++) {
+ if ((inst_plat_cap_data[i].codec &
+ core->inst_caps[j].codec)) {
+ ret = update_inst_capability(&inst_plat_cap_data[i],
+ &core->inst_caps[j]);
+ if (ret)
+ return ret;
+ }
+ }
+ }
+
+ return ret;
+}
+
+int get_inst_capability(struct iris_inst *inst)
+{
+ struct iris_core *core;
+ int i;
+
+ core = inst->core;
+
+ for (i = 0; i < core->dec_codecs_count; i++) {
+ if (core->inst_caps[i].codec == inst->codec) {
+ memcpy(&inst->cap[0], &core->inst_caps[i].cap[0],
+ (INST_CAP_MAX + 1) * sizeof(struct plat_inst_cap));
+ }
+ }
+
+ return 0;
+}
+
+int prepare_dependency_list(struct iris_inst *inst)
+{
+ struct cap_entry *entry = NULL, *temp = NULL;
+ struct plat_inst_cap *cap, *temp_cap;
+ int caps_to_prepare, pending_list_counter,
+ pending_at_start = 0;
+ struct list_head prepared_list, pending_list;
+ bool is_prepared[INST_CAP_MAX];
+ bool is_pending[INST_CAP_MAX];
+ int i, ret = 0;
+
+ cap = &inst->cap[0];
+
+ if (!list_empty(&inst->caps_list))
+ return 0;
+
+ INIT_LIST_HEAD(&prepared_list);
+ INIT_LIST_HEAD(&pending_list);
+ memset(&is_prepared, 0, sizeof(is_prepared));
+ memset(&is_pending, 0, sizeof(is_pending));
+
+ for (i = 1; i < INST_CAP_MAX; i++) {
+ temp_cap = &cap[i];
+ if (!is_valid_cap_id(temp_cap->cap_id))
+ continue;
+
+ if (!temp_cap->children[0]) {
+ if (!is_prepared[temp_cap->cap_id]) {
+ ret = add_node_list(&prepared_list, temp_cap->cap_id);
+ if (ret)
+ goto free_list;
+ is_prepared[temp_cap->cap_id] = true;
+ }
+ } else {
+ if (!is_pending[temp_cap->cap_id]) {
+ ret = add_node_list(&pending_list, temp_cap->cap_id);
+ if (ret)
+ goto free_list;
+ is_pending[temp_cap->cap_id] = true;
+ }
+ }
+ }
+
+ list_for_each_entry(entry, &pending_list, list)
+ pending_at_start++;
+
+ caps_to_prepare = pending_at_start;
+ pending_list_counter = pending_at_start;
+
+ list_for_each_entry_safe(entry, temp, &pending_list, list) {
+ list_del_init(&entry->list);
+ is_pending[entry->cap_id] = false;
+ pending_list_counter--;
+ temp_cap = &cap[entry->cap_id];
+
+ if (are_all_childrens_visited(temp_cap, is_prepared)) {
+ list_add(&entry->list, &prepared_list);
+ is_prepared[entry->cap_id] = true;
+ caps_to_prepare--;
+ } else {
+ list_add_tail(&entry->list, &pending_list);
+ is_pending[entry->cap_id] = true;
+ }
+
+ if (!pending_list_counter) {
+ if (pending_at_start == caps_to_prepare) {
+ ret = -EINVAL;
+ goto free_list;
+ }
+ pending_at_start = caps_to_prepare;
+ pending_list_counter = caps_to_prepare;
+ }
+ }
+
+ if (!list_empty(&pending_list)) {
+ ret = -EINVAL;
+ goto free_list;
+ }
+
+ list_replace_init(&prepared_list, &inst->caps_list);
+
+free_list:
+ list_for_each_entry_safe(entry, temp, &pending_list, list) {
+ list_del_init(&entry->list);
+ kfree(entry);
+ }
+ list_for_each_entry_safe(entry, temp, &prepared_list, list) {
+ list_del_init(&entry->list);
+ kfree(entry);
+ }
+
+ return ret;
+}
+
+int set_u32_enum(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_value = inst->cap[cap_id].value;
+ u32 hfi_id = inst->cap[cap_id].hfi_id;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32_ENUM,
+ &hfi_value, sizeof(u32));
+}
+
+int set_u32(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id)
+{
+ u32 hfi_value = inst->cap[cap_id].value;
+ u32 hfi_id = inst->cap[cap_id].hfi_id;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32,
+ &hfi_value, sizeof(u32));
+}
+
+int set_stage(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id)
+{
+ struct v4l2_format *inp_f;
+ u32 work_mode = STAGE_2;
+ u32 width, height;
+ u32 hfi_id;
+
+ hfi_id = inst->cap[cap_id].hfi_id;
+
+ inp_f = inst->fmt_src;
+ height = inp_f->fmt.pix_mp.height;
+ width = inp_f->fmt.pix_mp.width;
+ if (res_is_less_than(width, height, 1280, 720))
+ work_mode = STAGE_1;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32,
+ &work_mode, sizeof(u32));
+}
+
+int set_pipe(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id)
+{
+ u32 work_route, hfi_id;
+
+ work_route = inst->cap[PIPE].value;
+ hfi_id = inst->cap[cap_id].hfi_id;
+
+ return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
+ get_port_info(inst, cap_id),
+ HFI_PAYLOAD_U32,
+ &work_route, sizeof(u32));
+}
+
+int adjust_v4l2_properties(struct iris_inst *inst)
+{
+ struct cap_entry *entry = NULL, *temp = NULL;
+ int ret = 0;
+
+ list_for_each_entry_safe(entry, temp, &inst->caps_list, list) {
+ ret = adjust_cap(inst, entry->cap_id, NULL);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
+int adjust_output_order(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ s32 display_delay = -1, display_delay_enable = -1;
+ u32 adjusted_value;
+
+ adjusted_value = ctrl ? ctrl->val :
+ inst->cap[OUTPUT_ORDER].value;
+
+ display_delay = inst->cap[DISPLAY_DELAY].value;
+ display_delay_enable = inst->cap[DISPLAY_DELAY_ENABLE].value;
+
+ if (display_delay_enable && !display_delay)
+ adjusted_value = 1;
+
+ inst->cap[OUTPUT_ORDER].value = adjusted_value;
+
+ return 0;
+}
+
+int adjust_profile(struct iris_inst *inst, struct v4l2_ctrl *ctrl)
+{
+ u32 adjusted_value;
+ s32 pix_fmt = -1;
+
+ adjusted_value = ctrl ? ctrl->val :
+ inst->cap[PROFILE].value;
+
+ pix_fmt = inst->cap[PIX_FMTS].value;
+
+ if (pix_fmt == FMT_TP10C)
+ adjusted_value = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10;
+ else
+ adjusted_value = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN;
+
+ inst->cap[PROFILE].value = adjusted_value;
+
+ return 0;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
new file mode 100644
index 0000000..0f67f4f
--- /dev/null
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef _IRIS_CTRLS_H_
+#define _IRIS_CTRLS_H_
+
+#include "iris_instance.h"
+
+struct cap_entry {
+ struct list_head list;
+ enum plat_inst_cap_type cap_id;
+};
+
+int set_u32_enum(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_stage(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_pipe(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int set_u32(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int adjust_output_order(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int adjust_profile(struct iris_inst *inst, struct v4l2_ctrl *ctrl);
+int vidc_ctrl_handler_deinit(struct iris_inst *inst);
+int prepare_dependency_list(struct iris_inst *inst);
+int iris_init_instance_caps(struct iris_core *core);
+int iris_init_core_caps(struct iris_core *core);
+int get_inst_capability(struct iris_inst *inst);
+int adjust_v4l2_properties(struct iris_inst *inst);
+int ctrls_init(struct iris_inst *inst);
+
+#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index 872268d..8d8bc3a 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -5,6 +5,7 @@

#include "iris_helpers.h"
#include "iris_hfi.h"
+#include "iris_hfi_packet.h"
#include "iris_instance.h"

int check_core_lock(struct iris_core *core)
@@ -16,23 +17,29 @@ int check_core_lock(struct iris_core *core)
return fatal ? -EINVAL : 0;
}

-int iris_init_core_caps(struct iris_core *core)
+bool res_is_less_than(u32 width, u32 height,
+ u32 ref_width, u32 ref_height)
{
- struct plat_core_cap *core_platform_data;
- int i, num_core_caps;
+ u32 num_mbs = NUM_MBS_PER_FRAME(height, width);
+ u32 max_side = max(ref_width, ref_height);

- core_platform_data = core->platform_data->core_data;
- if (!core_platform_data)
- return -EINVAL;
+ if (num_mbs < NUM_MBS_PER_FRAME(ref_height, ref_width) &&
+ width < max_side &&
+ height < max_side)
+ return true;

- num_core_caps = core->platform_data->core_data_size;
+ return false;
+}

- for (i = 0; i < num_core_caps && i < CORE_CAP_MAX; i++) {
- core->cap[core_platform_data[i].type].type = core_platform_data[i].type;
- core->cap[core_platform_data[i].type].value = core_platform_data[i].value;
- }
+u32 get_port_info(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id)
+{
+ if (inst->cap[cap_id].flags & CAP_FLAG_INPUT_PORT)
+ return HFI_PORT_BITSTREAM;
+ else if (inst->cap[cap_id].flags & CAP_FLAG_OUTPUT_PORT)
+ return HFI_PORT_RAW;

- return 0;
+ return HFI_PORT_NONE;
}

static int process_inst_timeout(struct iris_inst *inst)
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
index b9a6485..60c79124 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -6,11 +6,22 @@
#ifndef _IRIS_HELPERS_H_
#define _IRIS_HELPERS_H_

+#include <linux/align.h>
+#include <linux/types.h>
+
#include "iris_core.h"
#include "iris_instance.h"
+#include "platform_common.h"
+
+#define NUM_MBS_PER_FRAME(__height, __width) \
+ ((ALIGN(__height, 16) / 16) * (ALIGN(__width, 16) / 16))

int check_core_lock(struct iris_core *core);
-int iris_init_core_caps(struct iris_core *core);
+bool res_is_less_than(u32 width, u32 height,
+ u32 ref_width, u32 ref_height);
+u32 get_port_info(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id);
+
int close_session(struct iris_inst *inst);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index 4b8d8c7..24ddb98 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -10,6 +10,7 @@
#include "iris_hfi.h"
#include "iris_hfi_packet.h"
#include "iris_hfi_response.h"
+#include "iris_helpers.h"
#include "vpu_common.h"

static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt)
@@ -280,3 +281,31 @@ irqreturn_t iris_hfi_isr_handler(int irq, void *data)

return IRQ_HANDLED;
}
+
+int iris_hfi_set_property(struct iris_inst *inst,
+ u32 packet_type, u32 flag, u32 plane, u32 payload_type,
+ void *payload, u32 payload_size)
+{
+ struct iris_core *core;
+ int ret;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ ret = hfi_packet_session_property(inst,
+ packet_type,
+ flag,
+ plane,
+ payload_type,
+ payload,
+ payload_size);
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
index 8a62986..bf991bb 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
@@ -13,6 +13,9 @@ int iris_hfi_core_init(struct iris_core *core);
int iris_hfi_core_deinit(struct iris_core *core);
int iris_hfi_session_open(struct iris_inst *inst);
int iris_hfi_session_close(struct iris_inst *inst);
+int iris_hfi_set_property(struct iris_inst *inst,
+ u32 packet_type, u32 flag, u32 plane, u32 payload_type,
+ void *payload, u32 payload_size);

irqreturn_t iris_hfi_isr(int irq, void *data);
irqreturn_t iris_hfi_isr_handler(int irq, void *data);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
index 1ed572e..749d978 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
@@ -239,3 +239,29 @@ int hfi_packet_session_command(struct iris_inst *inst, u32 pkt_type,

return ret;
}
+
+int hfi_packet_session_property(struct iris_inst *inst,
+ u32 pkt_type, u32 flags, u32 port,
+ u32 payload_type, void *payload, u32 payload_size)
+{
+ struct iris_core *core;
+ int ret;
+
+ core = inst->core;
+
+ ret = hfi_create_header(inst->packet, inst->packet_size,
+ inst->session_id, core->header_id++);
+ if (ret)
+ return ret;
+
+ ret = hfi_create_packet(inst->packet, inst->packet_size,
+ pkt_type,
+ flags,
+ payload_type,
+ port,
+ core->packet_id++,
+ payload,
+ payload_size);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
index 9e476e9..bea7ed9 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
@@ -74,5 +74,8 @@ int hfi_packet_session_command(struct iris_inst *inst, u32 pkt_type,
u32 flags, u32 port, u32 session_id,
u32 payload_type, void *payload,
u32 payload_size);
+int hfi_packet_session_property(struct iris_inst *inst,
+ u32 pkt_type, u32 flags, u32 port,
+ u32 payload_type, void *payload, u32 payload_size);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
index 1bbb533..f6a3066 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -10,6 +10,7 @@

#include "iris_core.h"
#include "iris_common.h"
+#include "platform_common.h"

/**
* struct iris_inst - holds per video instance parameters
@@ -27,6 +28,10 @@
* @packet: HFI packet
* @packet_size: HFI packet size
* @completions: structure of signal completions
+ * @cap: array of supported instance capabilities
+ * @num_ctrls: supported number of controls
+ * @caps_list: list head of capability
+ * @codec: codec type
*/

struct iris_inst {
@@ -43,6 +48,10 @@ struct iris_inst {
void *packet;
u32 packet_size;
struct completion completions[MAX_SIGNAL];
+ struct plat_inst_cap cap[INST_CAP_MAX + 1];
+ u32 num_ctrls;
+ struct list_head caps_list;
+ enum codec_type codec;
};

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
index 8c591da..50fb93e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
@@ -13,6 +13,7 @@
#include "iris_hfi.h"
#include "resources.h"
#include "iris_vidc.h"
+#include "iris_ctrls.h"

static int init_iris_isr(struct iris_core *core)
{
@@ -158,6 +159,13 @@ static int iris_probe(struct platform_device *pdev)
return ret;
}

+ ret = iris_init_instance_caps(core);
+ if (ret) {
+ dev_err_probe(core->dev, ret,
+ "%s: init inst caps failed with %d\n", __func__, ret);
+ return ret;
+ }
+
ret = v4l2_device_register(dev, &core->v4l2_dev);
if (ret)
return ret;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
index d02da8b..3a26edb 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
@@ -9,6 +9,7 @@
#include "iris_instance.h"
#include "iris_vdec.h"
#include "iris_vidc.h"
+#include "iris_ctrls.h"

static int vidc_v4l2_fh_init(struct iris_inst *inst)
{
@@ -174,6 +175,10 @@ int vidc_open(struct file *filp)
if (ret)
goto fail_free_inst;

+ INIT_LIST_HEAD(&inst->caps_list);
+ for (i = 0; i < MAX_SIGNAL; i++)
+ init_completion(&inst->completions[i]);
+
ret = vidc_v4l2_fh_init(inst);
if (ret)
goto fail_remove_session;
@@ -184,6 +189,14 @@ int vidc_open(struct file *filp)
if (ret)
goto fail_inst_deinit;

+ ret = get_inst_capability(inst);
+ if (ret)
+ goto fail_queue_deinit;
+
+ ret = ctrls_init(inst);
+ if (ret)
+ goto fail_queue_deinit;
+
ret = iris_hfi_session_open(inst);
if (ret) {
dev_err(core->dev, "%s: session open failed\n", __func__);
@@ -194,7 +207,9 @@ int vidc_open(struct file *filp)
return 0;

fail_core_deinit:
+ v4l2_ctrl_handler_free(&inst->ctrl_handler);
iris_core_deinit(core);
+fail_queue_deinit:
vidc_vb2_queue_deinit(inst);
fail_inst_deinit:
vdec_inst_deinit(inst);
@@ -216,6 +231,7 @@ int vidc_close(struct file *filp)
if (!inst)
return -EINVAL;

+ v4l2_ctrl_handler_free(&inst->ctrl_handler);
vdec_inst_deinit(inst);
close_session(inst);
vidc_vb2_queue_deinit(inst);
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
index e478b02..8305c65 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
@@ -7,8 +7,10 @@
#define _PLATFORM_COMMON_H_

#include <linux/bits.h>
+#include <media/v4l2-ctrls.h>

struct iris_core;
+struct iris_inst;

#define HW_RESPONSE_TIMEOUT_VALUE (1000)

@@ -17,6 +19,7 @@ struct iris_core;

#define CODED_FRAMES_PROGRESSIVE 0x0
#define CODED_FRAMES_INTERLACE 0x1
+#define MAX_NUM_CHILD 10

#define DEFAULT_MAX_HOST_BUF_COUNT 64
#define DEFAULT_MAX_HOST_BURST_BUF_COUNT 256
@@ -163,6 +166,16 @@ struct plat_inst_cap {
u32 v4l2_id;
u32 hfi_id;
enum plat_inst_cap_flags flags;
+ enum plat_inst_cap_type children[MAX_NUM_CHILD];
+ int (*adjust)(struct iris_inst *inst,
+ struct v4l2_ctrl *ctrl);
+ int (*set)(struct iris_inst *inst,
+ enum plat_inst_cap_type cap_id);
+};
+
+struct plat_inst_caps {
+ enum codec_type codec;
+ struct plat_inst_cap cap[INST_CAP_MAX + 1];
};

struct platform_data {
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
index c75017e..0759ac5 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
@@ -9,6 +9,7 @@
#include <media/v4l2-ctrls.h>

#include "hfi_defines.h"
+#include "iris_ctrls.h"
#include "platform_common.h"
#include "resources.h"

@@ -47,7 +48,16 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
FMT_NV12 | FMT_NV21 | FMT_NV12C,
FMT_NV12C},

- {PIX_FMTS, HEVC | VP9,
+ {PIX_FMTS, HEVC,
+ FMT_NV12,
+ FMT_TP10C,
+ FMT_NV12 | FMT_NV21 | FMT_NV12C | FMT_TP10C,
+ FMT_NV12C,
+ 0, 0,
+ CAP_FLAG_NONE,
+ {PROFILE}},
+
+ {PIX_FMTS, VP9,
FMT_NV12,
FMT_TP10C,
FMT_NV12 | FMT_NV21 | FMT_NV12C | FMT_TP10C,
@@ -88,7 +98,10 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
V4L2_CID_MPEG_VIDEO_H264_PROFILE,
HFI_PROP_PROFILE,
- CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {0},
+ NULL,
+ set_u32_enum},

{PROFILE, HEVC,
V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
@@ -99,7 +112,10 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
V4L2_CID_MPEG_VIDEO_HEVC_PROFILE,
HFI_PROP_PROFILE,
- CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {0},
+ adjust_profile,
+ set_u32_enum},

{PROFILE, VP9,
V4L2_MPEG_VIDEO_VP9_PROFILE_0,
@@ -109,7 +125,10 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
V4L2_MPEG_VIDEO_VP9_PROFILE_0,
V4L2_CID_MPEG_VIDEO_VP9_PROFILE,
HFI_PROP_PROFILE,
- CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {0},
+ NULL,
+ set_u32_enum},

{LEVEL, H264,
V4L2_MPEG_VIDEO_H264_LEVEL_1_0,
@@ -137,7 +156,10 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
V4L2_MPEG_VIDEO_H264_LEVEL_6_1,
V4L2_CID_MPEG_VIDEO_H264_LEVEL,
HFI_PROP_LEVEL,
- CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {0},
+ NULL,
+ set_u32_enum},

{LEVEL, HEVC,
V4L2_MPEG_VIDEO_HEVC_LEVEL_1,
@@ -158,7 +180,10 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
V4L2_MPEG_VIDEO_HEVC_LEVEL_6_1,
V4L2_CID_MPEG_VIDEO_HEVC_LEVEL,
HFI_PROP_LEVEL,
- CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {0},
+ NULL,
+ set_u32_enum},

{LEVEL, VP9,
V4L2_MPEG_VIDEO_VP9_LEVEL_1_0,
@@ -178,7 +203,10 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
V4L2_MPEG_VIDEO_VP9_LEVEL_6_0,
V4L2_CID_MPEG_VIDEO_VP9_LEVEL,
HFI_PROP_LEVEL,
- CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {0},
+ NULL,
+ set_u32_enum},

{HEVC_TIER, HEVC,
V4L2_MPEG_VIDEO_HEVC_TIER_MAIN,
@@ -188,46 +216,69 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
V4L2_MPEG_VIDEO_HEVC_TIER_HIGH,
V4L2_CID_MPEG_VIDEO_HEVC_TIER,
HFI_PROP_TIER,
- CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU},
+ CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU,
+ {0},
+ NULL,
+ set_u32_enum},

{DISPLAY_DELAY_ENABLE, CODECS_ALL,
0, 1, 1, 0,
V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY_ENABLE,
HFI_PROP_DECODE_ORDER_OUTPUT,
- CAP_FLAG_INPUT_PORT},
+ CAP_FLAG_INPUT_PORT,
+ {OUTPUT_ORDER},
+ NULL,
+ NULL},

{DISPLAY_DELAY, CODECS_ALL,
0, 1, 1, 0,
V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY,
HFI_PROP_DECODE_ORDER_OUTPUT,
- CAP_FLAG_INPUT_PORT},
+ CAP_FLAG_INPUT_PORT,
+ {OUTPUT_ORDER},
+ NULL,
+ NULL},

{OUTPUT_ORDER, CODECS_ALL,
0, 1, 1, 0,
0,
HFI_PROP_DECODE_ORDER_OUTPUT,
- CAP_FLAG_INPUT_PORT},
+ CAP_FLAG_INPUT_PORT,
+ {0},
+ adjust_output_order,
+ set_u32},

{INPUT_BUF_HOST_MAX_COUNT, CODECS_ALL,
DEFAULT_MAX_HOST_BUF_COUNT, DEFAULT_MAX_HOST_BURST_BUF_COUNT,
1, DEFAULT_MAX_HOST_BUF_COUNT,
0,
HFI_PROP_BUFFER_HOST_MAX_COUNT,
- CAP_FLAG_INPUT_PORT},
+ CAP_FLAG_INPUT_PORT,
+ {0},
+ NULL,
+ set_u32},

{STAGE, CODECS_ALL,
STAGE_1,
STAGE_2, 1,
STAGE_2,
0,
- HFI_PROP_STAGE},
+ HFI_PROP_STAGE,
+ CAP_FLAG_NONE,
+ {0},
+ NULL,
+ set_stage},

{PIPE, CODECS_ALL,
PIPE_1,
PIPE_4, 1,
PIPE_4,
0,
- HFI_PROP_PIPE},
+ HFI_PROP_PIPE,
+ CAP_FLAG_NONE,
+ {0},
+ NULL,
+ set_pipe},

{POC, H264, 0, 2, 1, 1,
0,
@@ -252,7 +303,10 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = {
0, 1, 1, 1,
0,
HFI_PROP_DEC_START_FROM_RAP_FRAME,
- CAP_FLAG_INPUT_PORT},
+ CAP_FLAG_INPUT_PORT,
+ {0},
+ NULL,
+ set_u32},
};

static const struct bus_info sm8550_bus_table[] = {
--
2.7.4

2023-12-18 11:57:14

[permalink] [raw]

Subject: [PATCH v2 28/34] media: iris: add instance sub states and implement DRC and Drain sequence

From: Vikash Garodia <[email protected]>

Implement decoder commands for CMD_START and CMD_STOP.
HFI_CMD_DRAIN - set to firmware as part of stop command
to initiate drain.
HFI_CMD_RESUME - set to firmware as part of start command
to resume the buffer processing on specific plane.

Also, introduce different sub states for instances.
Sub state transitions are defined, based on which they would
be allowed/rejected/ignored.

- IRIS_INST_SUB_DRAIN - indicates drain is in progress.
- IRIS_INST_SUB_DRC - indicates source change is received from
firmware and source change event is sent to client.
- IRIS_INST_SUB_DRAIN_LAST - indicates last buffer is received
from firmware as part of drain sequence .
- IRIS_INST_SUB_DRC_LAST - indicates last buffer is received
from firmware as part of source change.
- IRIS_INST_SUB_INPUT_PAUSE - source change is received form
firmware. This indicates that firmware is paused to process
any further input frames.
- IRIS_INST_SUB_OUTPUT_PAUSE - last buffer is received form
firmware as part of drain or drc sequence . This indicates
that firmware is paused to process any further output frames.

Signed-off-by: Vikash Garodia <[email protected]>
Signed-off-by: Dikshita Agarwal <[email protected]>
---
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 19 ++
.../media/platform/qcom/vcodec/iris/iris_buffer.c | 56 +++++-
.../media/platform/qcom/vcodec/iris/iris_buffer.h | 3 +-
.../media/platform/qcom/vcodec/iris/iris_common.h | 1 +
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 118 ++++++++++++
drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 4 +-
.../platform/qcom/vcodec/iris/iris_hfi_response.c | 106 ++++++++++-
.../platform/qcom/vcodec/iris/iris_instance.h | 2 +
.../media/platform/qcom/vcodec/iris/iris_state.c | 211 +++++++++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_state.h | 24 +++
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 145 ++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_vdec.h | 2 +
.../media/platform/qcom/vcodec/iris/iris_vidc.c | 79 +++++++-
13 files changed, 760 insertions(+), 10 deletions(-)

diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index 104ef9c..bbe8ca6 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -16,7 +16,10 @@
#define HFI_CMD_CLOSE 0x01000004
#define HFI_CMD_START 0x01000005
#define HFI_CMD_STOP 0x01000006
+#define HFI_CMD_DRAIN 0x01000007
+#define HFI_CMD_RESUME 0x01000008
#define HFI_CMD_BUFFER 0x01000009
+#define HFI_CMD_PAUSE 0x01000011

#define HFI_PROP_IMAGE_VERSION 0x03000001

@@ -208,6 +211,20 @@ enum hfi_picture_type {

#define HFI_SYSTEM_ERROR_END 0x05FFFFFF

+#define HFI_INFORMATION_BEGIN 0x06000000
+
+#define HFI_INFO_UNSUPPORTED 0x06000001
+
+#define HFI_INFO_DATA_CORRUPT 0x06000002
+
+#define HFI_INFO_BUFFER_OVERFLOW 0x06000004
+
+#define HFI_INFO_HFI_FLAG_DRAIN_LAST 0x06000006
+
+#define HFI_INFO_HFI_FLAG_PSC_LAST 0x06000007
+
+#define HFI_INFORMATION_END 0x06FFFFFF
+
struct hfi_debug_header {
u32 size;
u32 debug_level;
@@ -242,6 +259,8 @@ enum hfi_buffer_firmware_flags {
HFI_BUF_FW_FLAG_NONE = 0x00000000,
HFI_BUF_FW_FLAG_RELEASE_DONE = 0x00000001,
HFI_BUF_FW_FLAG_READONLY = 0x00000010,
+ HFI_BUF_FW_FLAG_LAST = 0x10000000,
+ HFI_BUF_FW_FLAG_PSC_LAST = 0x20000000,
};

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
index ab25026..f85b52c 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
@@ -547,7 +547,7 @@ static int iris_release_internal_buffers(struct iris_inst *inst,
return ret;
}

-int iris_release_input_internal_buffers(struct iris_inst *inst)
+static int iris_release_input_internal_buffers(struct iris_inst *inst)
{
int ret = 0;
u32 i = 0;
@@ -581,3 +581,57 @@ int iris_alloc_and_queue_session_int_bufs(struct iris_inst *inst,

return ret;
}
+
+int iris_alloc_and_queue_input_int_bufs(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = iris_get_internal_buffers(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = iris_release_input_internal_buffers(inst);
+ if (ret)
+ return ret;
+
+ ret = iris_create_input_internal_buffers(inst);
+ if (ret)
+ return ret;
+
+ ret = iris_queue_input_internal_buffers(inst);
+
+ return ret;
+}
+
+int iris_alloc_and_queue_additional_dpb_buffers(struct iris_inst *inst)
+{
+ struct iris_buffers *buffers;
+ struct iris_buffer *buffer;
+ int cur_min_count = 0;
+ int ret;
+ int i;
+
+ ret = iris_get_internal_buf_info(inst, BUF_DPB);
+ if (ret)
+ return ret;
+
+ buffers = iris_get_buffer_list(inst, BUF_DPB);
+ if (!buffers)
+ return -EINVAL;
+
+ list_for_each_entry(buffer, &buffers->list, list)
+ cur_min_count++;
+
+ if (cur_min_count >= buffers->min_count)
+ return 0;
+
+ for (i = cur_min_count; i < buffers->min_count; i++) {
+ ret = iris_create_internal_buffers(inst, BUF_DPB);
+ if (ret)
+ return ret;
+ }
+
+ ret = iris_queue_internal_buffers(inst, BUF_DPB);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
index 59c4a10..6ab5f22 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
@@ -57,8 +57,9 @@ int iris_destroy_internal_buffer(struct iris_inst *inst,
struct iris_buffer *buffer);
int iris_destroy_internal_buffers(struct iris_inst *inst,
u32 plane);
-int iris_release_input_internal_buffers(struct iris_inst *inst);
int iris_alloc_and_queue_session_int_bufs(struct iris_inst *inst,
enum iris_buffer_type buffer_type);
+int iris_alloc_and_queue_input_int_bufs(struct iris_inst *inst);
+int iris_alloc_and_queue_additional_dpb_buffers(struct iris_inst *inst);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
index 5fd96d3..24b322a 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
@@ -76,6 +76,7 @@ enum iris_buffer_flags {
BUF_FLAG_PFRAME = 0x00000010,
BUF_FLAG_BFRAME = 0x00000020,
BUF_FLAG_ERROR = 0x00000040,
+ BUF_FLAG_LAST = 0x00100000,
};

enum iris_buffer_attributes {
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index e0da3be..2b3fa47 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -445,6 +445,124 @@ int iris_hfi_session_subscribe_mode(struct iris_inst *inst,
return ret;
}

+int iris_hfi_pause(struct iris_inst *inst, u32 plane)
+{
+ struct iris_core *core;
+ int ret = 0;
+
+ if (!inst->packet)
+ return -EINVAL;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (plane != INPUT_MPLANE && plane != OUTPUT_MPLANE)
+ goto unlock;
+
+ ret = hfi_packet_session_command(inst,
+ HFI_CMD_PAUSE,
+ (HFI_HOST_FLAGS_RESPONSE_REQUIRED |
+ HFI_HOST_FLAGS_INTR_REQUIRED),
+ get_hfi_port(plane),
+ inst->session_id,
+ HFI_PAYLOAD_NONE,
+ NULL,
+ 0);
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+int iris_hfi_resume(struct iris_inst *inst, u32 plane, u32 payload)
+{
+ struct iris_core *core;
+ int ret = 0;
+
+ if (!inst->packet)
+ return -EINVAL;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (plane != INPUT_MPLANE && plane != OUTPUT_MPLANE)
+ goto unlock;
+
+ ret = hfi_packet_session_command(inst,
+ HFI_CMD_RESUME,
+ (HFI_HOST_FLAGS_RESPONSE_REQUIRED |
+ HFI_HOST_FLAGS_INTR_REQUIRED),
+ get_hfi_port(plane),
+ inst->session_id,
+ HFI_PAYLOAD_U32,
+ &payload,
+ sizeof(u32));
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
+int iris_hfi_drain(struct iris_inst *inst, u32 plane)
+{
+ struct iris_core *core;
+ int ret = 0;
+
+ if (!inst->packet)
+ return -EINVAL;
+
+ core = inst->core;
+ mutex_lock(&core->lock);
+
+ if (!validate_session(core, inst)) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (plane != INPUT_MPLANE)
+ goto unlock;
+
+ ret = hfi_packet_session_command(inst,
+ HFI_CMD_DRAIN,
+ (HFI_HOST_FLAGS_RESPONSE_REQUIRED |
+ HFI_HOST_FLAGS_INTR_REQUIRED |
+ HFI_HOST_FLAGS_NON_DISCARDABLE),
+ get_hfi_port(plane),
+ inst->session_id,
+ HFI_PAYLOAD_NONE,
+ NULL,
+ 0);
+ if (ret)
+ goto unlock;
+
+ ret = iris_hfi_queue_cmd_write(inst->core, inst->packet);
+
+unlock:
+ mutex_unlock(&core->lock);
+
+ return ret;
+}
+
irqreturn_t iris_hfi_isr(int irq, void *data)
{
disable_irq_nosync(irq);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
index f054c2d..ab479a9 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
@@ -25,7 +25,9 @@ int iris_hfi_session_set_default_header(struct iris_inst *inst);

int iris_hfi_start(struct iris_inst *inst, u32 plane);
int iris_hfi_stop(struct iris_inst *inst, u32 plane);
-
+int iris_hfi_drain(struct iris_inst *inst, u32 plane);
+int iris_hfi_pause(struct iris_inst *inst, u32 plane);
+int iris_hfi_resume(struct iris_inst *inst, u32 plane, u32 cmd);
int iris_hfi_queue_buffer(struct iris_inst *inst,
struct iris_buffer *buffer);
int iris_hfi_release_buffer(struct iris_inst *inst,
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
index b1236dd..3548f9d 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
@@ -103,6 +103,10 @@ static int get_driver_buffer_flags(struct iris_inst *inst, u32 hfi_flags)
if (inst->hfi_frame_info.overflow)
driver_flags |= BUF_FLAG_ERROR;

+ if (hfi_flags & HFI_BUF_FW_FLAG_LAST ||
+ hfi_flags & HFI_BUF_FW_FLAG_PSC_LAST)
+ driver_flags |= BUF_FLAG_LAST;
+
return driver_flags;
}

@@ -187,6 +191,46 @@ static int validate_hdr_packet(struct iris_core *core, struct hfi_header *hdr)
return ret;
}

+static int handle_session_info(struct iris_inst *inst,
+ struct hfi_packet *pkt)
+{
+ struct iris_core *core;
+ int ret = 0;
+ char *info;
+
+ core = inst->core;
+
+ switch (pkt->type) {
+ case HFI_INFO_UNSUPPORTED:
+ info = "unsupported";
+ break;
+ case HFI_INFO_DATA_CORRUPT:
+ info = "data corrupt";
+ inst->hfi_frame_info.data_corrupt = 1;
+ break;
+ case HFI_INFO_BUFFER_OVERFLOW:
+ info = "buffer overflow";
+ inst->hfi_frame_info.overflow = 1;
+ break;
+ case HFI_INFO_HFI_FLAG_DRAIN_LAST:
+ info = "drain last flag";
+ ret = iris_inst_sub_state_change_drain_last(inst);
+ break;
+ case HFI_INFO_HFI_FLAG_PSC_LAST:
+ info = "drc last flag";
+ ret = iris_inst_sub_state_change_drc_last(inst);
+ break;
+ default:
+ info = "unknown";
+ break;
+ }
+
+ dev_dbg(core->dev, "session info received %#x: %s\n",
+ pkt->type, info);
+
+ return ret;
+}
+
static int handle_session_error(struct iris_inst *inst,
struct hfi_packet *pkt)
{
@@ -375,9 +419,21 @@ static int handle_output_buffer(struct iris_inst *inst,
bool found;
int ret = 0;

+ if (hfi_buffer->flags & HFI_BUF_FW_FLAG_LAST) {
+ ret = iris_inst_sub_state_change_drain_last(inst);
+ if (ret)
+ return ret;
+ }
+
if (hfi_buffer->flags & HFI_BUF_FW_FLAG_RELEASE_DONE)
return handle_release_output_buffer(inst, hfi_buffer);

+ if (hfi_buffer->flags & HFI_BUF_FW_FLAG_PSC_LAST) {
+ ret = iris_inst_sub_state_change_drc_last(inst);
+ if (ret)
+ return ret;
+ }
+
if (!(hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY))
ret = handle_non_read_only_buffer(inst, hfi_buffer);

@@ -488,6 +544,26 @@ static int handle_release_internal_buffer(struct iris_inst *inst,
return ret;
}

+static int handle_session_stop(struct iris_inst *inst,
+ struct hfi_packet *pkt)
+{
+ int ret = 0;
+ enum signal_session_response signal_type = -1;
+
+ if (pkt->port == HFI_PORT_RAW) {
+ signal_type = SIGNAL_CMD_STOP_OUTPUT;
+ ret = iris_inst_sub_state_change_pause(inst, OUTPUT_MPLANE);
+ } else if (pkt->port == HFI_PORT_BITSTREAM) {
+ signal_type = SIGNAL_CMD_STOP_INPUT;
+ ret = iris_inst_sub_state_change_pause(inst, INPUT_MPLANE);
+ }
+
+ if (signal_type != -1)
+ signal_session_msg_receipt(inst, signal_type);
+
+ return ret;
+}
+
static int handle_session_buffer(struct iris_inst *inst,
struct hfi_packet *pkt)
{
@@ -549,18 +625,30 @@ static int handle_session_buffer(struct iris_inst *inst,
return ret;
}

+static int handle_session_drain(struct iris_inst *inst,
+ struct hfi_packet *pkt)
+{
+ int ret = 0;
+
+ if (inst->sub_state & IRIS_INST_SUB_DRAIN)
+ ret = iris_inst_change_sub_state(inst, 0, IRIS_INST_SUB_INPUT_PAUSE);
+
+ return ret;
+}
+
static int handle_src_change(struct iris_inst *inst,
struct hfi_packet *pkt)
{
- int ret = 0;
+ int ret;

- if (pkt->port == HFI_PORT_BITSTREAM)
- ret = vdec_src_change(inst);
- else if (pkt->port == HFI_PORT_RAW)
- ret = 0;
+ if (pkt->port != HFI_PORT_BITSTREAM)
+ return 0;

+ ret = iris_inst_sub_state_change_drc(inst);
if (ret)
- iris_inst_change_state(inst, IRIS_INST_ERROR);
+ return ret;
+
+ ret = vdec_src_change(inst);

return ret;
}
@@ -572,9 +660,14 @@ static int handle_session_command(struct iris_inst *inst,
static const struct iris_hfi_packet_handle hfi_pkt_handle[] = {
{HFI_CMD_OPEN, NULL },
{HFI_CMD_CLOSE, handle_session_close },
+ {HFI_CMD_START, NULL },
+ {HFI_CMD_STOP, handle_session_stop },
+ {HFI_CMD_DRAIN, handle_session_drain },
{HFI_CMD_BUFFER, handle_session_buffer },
{HFI_CMD_SETTINGS_CHANGE, handle_src_change },
{HFI_CMD_SUBSCRIBE_MODE, NULL },
+ {HFI_CMD_PAUSE, NULL },
+ {HFI_CMD_RESUME, NULL },
};

for (i = 0; i < ARRAY_SIZE(hfi_pkt_handle); i++) {
@@ -774,6 +867,7 @@ static int handle_session_response(struct iris_core *core,
int i, j;
static const struct iris_inst_hfi_range be[] = {
{HFI_SESSION_ERROR_BEGIN, HFI_SESSION_ERROR_END, handle_session_error },
+ {HFI_INFORMATION_BEGIN, HFI_INFORMATION_END, handle_session_info },
{HFI_PROP_BEGIN, HFI_PROP_END, handle_session_property },
{HFI_CMD_BEGIN, HFI_CMD_END, handle_session_command },
};
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
index cddf143..50154bf 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h
@@ -39,6 +39,7 @@
* @buffers: structure of buffer info
* @fw_min_count: minimnum count of buffers needed by fw
* @state: instance state
+ * @sub_state: instance sub state
* @ipsc_properties_set: boolean to set ipsc properties to fw
* @opsc_properties_set: boolean to set opsc properties to fw
* @hfi_frame_info: structure of frame info
@@ -71,6 +72,7 @@ struct iris_inst {
struct iris_buffers_info buffers;
u32 fw_min_count;
enum iris_inst_state state;
+ enum iris_inst_sub_state sub_state;
bool ipsc_properties_set;
bool opsc_properties_set;
struct iris_hfi_frame_info hfi_frame_info;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
index 04e7dc8..952ba2a 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_state.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c
@@ -239,3 +239,214 @@ int iris_inst_state_change_streamoff(struct iris_inst *inst, u32 plane)

return iris_inst_change_state(inst, new_state);
}
+
+struct iris_inst_sub_state_change_allow {
+ enum iris_inst_state state;
+ enum state_change allow;
+ u32 sub_state_mask;
+};
+
+static int iris_inst_allow_sub_state(struct iris_inst *inst, enum iris_inst_sub_state sub_state)
+{
+ int i;
+ static struct iris_inst_sub_state_change_allow sub_state_allow[] = {
+ {IRIS_INST_OPEN, STATE_CHANGE_DISALLOW, IRIS_INST_SUB_DRC |
+ IRIS_INST_SUB_DRAIN |
+ IRIS_INST_SUB_DRC_LAST |
+ IRIS_INST_SUB_DRAIN_LAST |
+ IRIS_INST_SUB_INPUT_PAUSE |
+ IRIS_INST_SUB_OUTPUT_PAUSE},
+
+ {IRIS_INST_INPUT_STREAMING, STATE_CHANGE_DISALLOW, IRIS_INST_SUB_DRC_LAST |
+ IRIS_INST_SUB_DRAIN_LAST |
+ IRIS_INST_SUB_OUTPUT_PAUSE},
+
+ {IRIS_INST_INPUT_STREAMING, STATE_CHANGE_ALLOW, IRIS_INST_SUB_DRC |
+ IRIS_INST_SUB_DRAIN |
+ IRIS_INST_SUB_INPUT_PAUSE },
+
+ {IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_DISALLOW, IRIS_INST_SUB_DRC |
+ IRIS_INST_SUB_DRAIN |
+ IRIS_INST_SUB_INPUT_PAUSE },
+
+ {IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_ALLOW, IRIS_INST_SUB_DRC_LAST |
+ IRIS_INST_SUB_DRAIN_LAST |
+ IRIS_INST_SUB_OUTPUT_PAUSE},
+
+ {IRIS_INST_STREAMING, STATE_CHANGE_ALLOW, IRIS_INST_SUB_DRC |
+ IRIS_INST_SUB_DRAIN |
+ IRIS_INST_SUB_DRC_LAST |
+ IRIS_INST_SUB_DRAIN_LAST |
+ IRIS_INST_SUB_INPUT_PAUSE |
+ IRIS_INST_SUB_OUTPUT_PAUSE},
+
+ {IRIS_INST_CLOSE, STATE_CHANGE_ALLOW, IRIS_INST_SUB_DRC |
+ IRIS_INST_SUB_DRAIN |
+ IRIS_INST_SUB_DRC_LAST |
+ IRIS_INST_SUB_DRAIN_LAST |
+ IRIS_INST_SUB_INPUT_PAUSE |
+ IRIS_INST_SUB_OUTPUT_PAUSE},
+
+ {IRIS_INST_ERROR, STATE_CHANGE_ALLOW, IRIS_INST_SUB_DRC |
+ IRIS_INST_SUB_DRAIN |
+ IRIS_INST_SUB_DRC_LAST |
+ IRIS_INST_SUB_DRAIN_LAST |
+ IRIS_INST_SUB_INPUT_PAUSE |
+ IRIS_INST_SUB_OUTPUT_PAUSE},
+ };
+
+ if (!sub_state)
+ return 0;
+
+ for (i = 0; i < ARRAY_SIZE(sub_state_allow); i++) {
+ if (sub_state_allow[i].state != inst->state)
+ continue;
+
+ if (sub_state_allow[i].allow == STATE_CHANGE_DISALLOW)
+ if (sub_state_allow[i].sub_state_mask & sub_state) {
+ dev_dbg(inst->core->dev, "state %d with disallowed sub state %x\n",
+ inst->state, sub_state);
+ return -EINVAL;
+ }
+
+ if (sub_state_allow[i].allow == STATE_CHANGE_ALLOW) {
+ if ((sub_state_allow[i].sub_state_mask & sub_state) != sub_state)
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+int iris_inst_change_sub_state(struct iris_inst *inst,
+ enum iris_inst_sub_state clear_sub_state,
+ enum iris_inst_sub_state set_sub_state)
+{
+ enum iris_inst_sub_state prev_sub_state;
+ int ret;
+
+ if (IS_SESSION_ERROR(inst))
+ return 0;
+
+ if (!clear_sub_state && !set_sub_state)
+ return 0;
+
+ if ((clear_sub_state & set_sub_state) ||
+ set_sub_state > IRIS_INST_MAX_SUB_STATE_VALUE ||
+ clear_sub_state > IRIS_INST_MAX_SUB_STATE_VALUE)
+ return -EINVAL;
+
+ prev_sub_state = inst->sub_state;
+
+ ret = iris_inst_allow_sub_state(inst, set_sub_state);
+ if (ret)
+ return ret;
+
+ inst->sub_state |= set_sub_state;
+ inst->sub_state &= ~clear_sub_state;
+
+ if (inst->sub_state != prev_sub_state)
+ dev_dbg(inst->core->dev, "state %d and sub state changed to %x\n",
+ inst->sub_state, inst->sub_state);
+
+ return 0;
+}
+
+int iris_inst_sub_state_change_drc(struct iris_inst *inst)
+{
+ enum iris_inst_sub_state set_sub_state = 0;
+
+ if (inst->sub_state & IRIS_INST_SUB_DRC)
+ return STATE_CHANGE_DISALLOW;
+
+ if (inst->state == IRIS_INST_INPUT_STREAMING ||
+ inst->state == IRIS_INST_OPEN)
+ set_sub_state = IRIS_INST_SUB_INPUT_PAUSE;
+ else
+ set_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_INPUT_PAUSE;
+
+ return iris_inst_change_sub_state(inst, 0, set_sub_state);
+}
+
+int iris_inst_sub_state_change_drain_last(struct iris_inst *inst)
+{
+ enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE;
+
+ if (inst->sub_state & IRIS_INST_SUB_DRAIN_LAST)
+ return STATE_CHANGE_DISALLOW;
+
+ if (!(inst->sub_state & IRIS_INST_SUB_DRAIN) ||
+ !(inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE))
+ return STATE_CHANGE_DISALLOW;
+
+ set_sub_state = IRIS_INST_SUB_DRAIN_LAST | IRIS_INST_SUB_OUTPUT_PAUSE;
+
+ return iris_inst_change_sub_state(inst, 0, set_sub_state);
+}
+
+int iris_inst_sub_state_change_drc_last(struct iris_inst *inst)
+{
+ enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE;
+
+ if (inst->sub_state & IRIS_INST_SUB_DRC_LAST)
+ return STATE_CHANGE_DISALLOW;
+
+ if (!(inst->sub_state & IRIS_INST_SUB_DRC) ||
+ !(inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE))
+ return STATE_CHANGE_DISALLOW;
+
+ set_sub_state = IRIS_INST_SUB_DRC_LAST | IRIS_INST_SUB_OUTPUT_PAUSE;
+
+ return iris_inst_change_sub_state(inst, 0, set_sub_state);
+}
+
+int iris_inst_sub_state_change_pause(struct iris_inst *inst, u32 plane)
+{
+ enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE;
+
+ if (plane == INPUT_MPLANE) {
+ if (inst->sub_state & IRIS_INST_SUB_DRC &&
+ !(inst->sub_state & IRIS_INST_SUB_DRC_LAST))
+ return STATE_CHANGE_DISALLOW;
+
+ if (inst->sub_state & IRIS_INST_SUB_DRAIN &&
+ !(inst->sub_state & IRIS_INST_SUB_DRAIN_LAST))
+ return STATE_CHANGE_DISALLOW;
+
+ set_sub_state = IRIS_INST_SUB_INPUT_PAUSE;
+ } else {
+ set_sub_state = IRIS_INST_SUB_OUTPUT_PAUSE;
+ }
+
+ return iris_inst_change_sub_state(inst, 0, set_sub_state);
+}
+
+bool is_drc_pending(struct iris_inst *inst)
+{
+ return inst->sub_state & IRIS_INST_SUB_DRC &&
+ inst->sub_state & IRIS_INST_SUB_DRC_LAST;
+}
+
+bool is_drain_pending(struct iris_inst *inst)
+{
+ return inst->sub_state & IRIS_INST_SUB_DRAIN &&
+ inst->sub_state & IRIS_INST_SUB_DRAIN_LAST;
+}
+
+bool allow_cmd(struct iris_inst *inst, u32 cmd)
+{
+ if (cmd == V4L2_DEC_CMD_START) {
+ if (inst->state == IRIS_INST_INPUT_STREAMING ||
+ inst->state == IRIS_INST_OUTPUT_STREAMING ||
+ inst->state == IRIS_INST_STREAMING)
+ if (is_drc_pending(inst) || is_drain_pending(inst))
+ return true;
+ } else if (cmd == V4L2_DEC_CMD_STOP) {
+ if (inst->state == IRIS_INST_INPUT_STREAMING ||
+ inst->state == IRIS_INST_STREAMING)
+ if (inst->sub_state != IRIS_INST_SUB_DRAIN)
+ return true;
+ }
+
+ return false;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.h b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
index 47558ed..76db8f5 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_state.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.h
@@ -25,6 +25,19 @@ enum iris_inst_state {
IRIS_INST_ERROR,
};

+#define IRIS_INST_SUB_NONE 0
+#define IRIS_INST_SUB_STATES 6
+#define IRIS_INST_MAX_SUB_STATE_VALUE ((1 << IRIS_INST_SUB_STATES) - 1)
+
+enum iris_inst_sub_state {
+ IRIS_INST_SUB_DRAIN = BIT(0),
+ IRIS_INST_SUB_DRC = BIT(1),
+ IRIS_INST_SUB_DRAIN_LAST = BIT(2),
+ IRIS_INST_SUB_DRC_LAST = BIT(3),
+ IRIS_INST_SUB_INPUT_PAUSE = BIT(4),
+ IRIS_INST_SUB_OUTPUT_PAUSE = BIT(5),
+};
+
enum state_change {
STATE_CHANGE_ALLOW,
STATE_CHANGE_DISALLOW,
@@ -40,6 +53,9 @@ int iris_change_core_state(struct iris_core *core,

int iris_inst_change_state(struct iris_inst *inst,
enum iris_inst_state request_state);
+int iris_inst_change_sub_state(struct iris_inst *inst,
+ enum iris_inst_sub_state clear_sub_state,
+ enum iris_inst_sub_state set_sub_state);

bool allow_s_fmt(struct iris_inst *inst, u32 type);
bool allow_reqbufs(struct iris_inst *inst, u32 type);
@@ -51,4 +67,12 @@ bool allow_s_ctrl(struct iris_inst *inst, u32 cap_id);
int iris_inst_state_change_streamon(struct iris_inst *inst, u32 plane);
int iris_inst_state_change_streamoff(struct iris_inst *inst, u32 plane);

+int iris_inst_sub_state_change_drc(struct iris_inst *inst);
+int iris_inst_sub_state_change_drain_last(struct iris_inst *inst);
+int iris_inst_sub_state_change_drc_last(struct iris_inst *inst);
+int iris_inst_sub_state_change_pause(struct iris_inst *inst, u32 plane);
+bool is_drc_pending(struct iris_inst *inst);
+bool is_drain_pending(struct iris_inst *inst);
+bool allow_cmd(struct iris_inst *inst, u32 cmd);
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index 7eb74c3..7884ba6 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -1066,16 +1066,35 @@ static int vdec_subscribe_dst_change_param(struct iris_inst *inst)

static int process_streamon_input(struct iris_inst *inst)
{
+ enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE;
int ret;

ret = iris_hfi_start(inst, INPUT_MPLANE);
if (ret)
return ret;

+ if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
+ ret = iris_inst_change_sub_state(inst, IRIS_INST_SUB_INPUT_PAUSE, 0);
+ if (ret)
+ return ret;
+ }
+
+ if (inst->sub_state & IRIS_INST_SUB_DRC ||
+ inst->sub_state & IRIS_INST_SUB_DRAIN) {
+ if (!(inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE)) {
+ ret = iris_hfi_pause(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+ set_sub_state = IRIS_INST_SUB_INPUT_PAUSE;
+ }
+ }
+
ret = iris_inst_state_change_streamon(inst, INPUT_MPLANE);
if (ret)
return ret;

+ ret = iris_inst_change_sub_state(inst, 0, set_sub_state);
+
return ret;
}

@@ -1127,16 +1146,51 @@ int vdec_streamon_input(struct iris_inst *inst)

static int process_streamon_output(struct iris_inst *inst)
{
+ enum iris_inst_sub_state clear_sub_state = IRIS_INST_SUB_NONE;
+ bool drain_pending = false;
int ret;

+ if (inst->sub_state & IRIS_INST_SUB_DRC &&
+ inst->sub_state & IRIS_INST_SUB_DRC_LAST)
+ clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST;
+
+ if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
+ ret = iris_alloc_and_queue_input_int_bufs(inst);
+ if (ret)
+ return ret;
+ ret = set_stage(inst, STAGE);
+ if (ret)
+ return ret;
+ ret = set_pipe(inst, PIPE);
+ if (ret)
+ return ret;
+ }
+
+ drain_pending = inst->sub_state & IRIS_INST_SUB_DRAIN &&
+ inst->sub_state & IRIS_INST_SUB_DRAIN_LAST;
+
+ if (!drain_pending && inst->state == IRIS_INST_INPUT_STREAMING) {
+ if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
+ ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE);
+ if (ret)
+ return ret;
+ clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE;
+ }
+ }
+
ret = iris_hfi_start(inst, OUTPUT_MPLANE);
if (ret)
return ret;

+ if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE)
+ clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE;
+
ret = iris_inst_state_change_streamon(inst, OUTPUT_MPLANE);
if (ret)
return ret;

+ ret = iris_inst_change_sub_state(inst, clear_sub_state, 0);
+
return ret;
}

@@ -1242,3 +1296,94 @@ int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2)

return ret;
}
+
+static int process_resume(struct iris_inst *inst)
+{
+ enum iris_inst_sub_state clear_sub_state = IRIS_INST_SUB_NONE;
+ int ret;
+
+ if (inst->sub_state & IRIS_INST_SUB_DRC &&
+ inst->sub_state & IRIS_INST_SUB_DRC_LAST) {
+ clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST;
+
+ if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
+ ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE);
+ if (ret)
+ return ret;
+ clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE;
+ }
+ if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) {
+ ret = iris_hfi_resume(inst, OUTPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE);
+ if (ret)
+ return ret;
+ clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE;
+ }
+ } else if (inst->sub_state & IRIS_INST_SUB_DRAIN &&
+ inst->sub_state & IRIS_INST_SUB_DRAIN_LAST) {
+ clear_sub_state = IRIS_INST_SUB_DRAIN | IRIS_INST_SUB_DRAIN_LAST;
+ if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
+ ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_DRAIN);
+ if (ret)
+ return ret;
+ clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE;
+ }
+ if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) {
+ ret = iris_hfi_resume(inst, OUTPUT_MPLANE, HFI_CMD_DRAIN);
+ if (ret)
+ return ret;
+ clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE;
+ }
+ }
+
+ ret = iris_inst_change_sub_state(inst, clear_sub_state, 0);
+
+ return ret;
+}
+
+int vdec_start_cmd(struct iris_inst *inst)
+{
+ int ret;
+
+ vb2_clear_last_buffer_dequeued(inst->vb2q_dst);
+
+ if (inst->sub_state & IRIS_INST_SUB_DRC &&
+ inst->sub_state & IRIS_INST_SUB_DRC_LAST &&
+ inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
+ ret = iris_alloc_and_queue_input_int_bufs(inst);
+ if (ret)
+ return ret;
+
+ ret = set_stage(inst, STAGE);
+ if (ret)
+ return ret;
+
+ ret = set_pipe(inst, PIPE);
+ if (ret)
+ return ret;
+ }
+
+ ret = iris_alloc_and_queue_additional_dpb_buffers(inst);
+ if (ret)
+ return ret;
+
+ ret = queue_deferred_buffers(inst, BUF_OUTPUT);
+ if (ret)
+ return ret;
+
+ ret = process_resume(inst);
+
+ return ret;
+}
+
+int vdec_stop_cmd(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = iris_hfi_drain(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = iris_inst_change_sub_state(inst, 0, IRIS_INST_SUB_DRAIN);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
index d666c54..974b95e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
@@ -19,5 +19,7 @@ int vdec_src_change(struct iris_inst *inst);
int vdec_streamon_input(struct iris_inst *inst);
int vdec_streamon_output(struct iris_inst *inst);
int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2);
+int vdec_start_cmd(struct iris_inst *inst);
+int vdec_stop_cmd(struct iris_inst *inst);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
index 0165340..d37ef04 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
@@ -901,7 +901,82 @@ static int vidc_g_selection(struct file *filp, void *fh, struct v4l2_selection *
return ret;
}

-static const struct v4l2_file_operations v4l2_file_ops = {
+static int vidc_try_dec_cmd(struct file *filp, void *fh,
+ struct v4l2_decoder_cmd *dec)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !dec)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (dec->cmd != V4L2_DEC_CMD_STOP && dec->cmd != V4L2_DEC_CMD_START) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+ dec->flags = 0;
+ if (dec->cmd == V4L2_DEC_CMD_STOP) {
+ dec->stop.pts = 0;
+ } else if (dec->cmd == V4L2_DEC_CMD_START) {
+ dec->start.speed = 0;
+ dec->start.format = V4L2_DEC_START_FMT_NONE;
+ }
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static int vidc_dec_cmd(struct file *filp, void *fh,
+ struct v4l2_decoder_cmd *dec)
+{
+ struct iris_inst *inst;
+ int ret = 0;
+
+ inst = get_vidc_inst(filp, fh);
+ if (!inst || !dec)
+ return -EINVAL;
+
+ mutex_lock(&inst->lock);
+ if (IS_SESSION_ERROR(inst)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (dec->cmd != V4L2_DEC_CMD_START &&
+ dec->cmd != V4L2_DEC_CMD_STOP) {
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ if (inst->state == IRIS_INST_OPEN)
+ goto unlock;
+
+ if (!allow_cmd(inst, dec->cmd)) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ if (dec->cmd == V4L2_DEC_CMD_START)
+ ret = vdec_start_cmd(inst);
+ else if (dec->cmd == V4L2_DEC_CMD_STOP)
+ ret = vdec_stop_cmd(inst);
+
+unlock:
+ mutex_unlock(&inst->lock);
+
+ return ret;
+}
+
+static struct v4l2_file_operations v4l2_file_ops = {
.owner = THIS_MODULE,
.open = vidc_open,
.release = vidc_close,
@@ -950,6 +1025,8 @@ static const struct v4l2_ioctl_ops v4l2_ioctl_ops = {
.vidioc_subscribe_event = vidc_subscribe_event,
.vidioc_unsubscribe_event = vidc_unsubscribe_event,
.vidioc_g_selection = vidc_g_selection,
+ .vidioc_try_decoder_cmd = vidc_try_dec_cmd,
+ .vidioc_decoder_cmd = vidc_dec_cmd,
};

int init_ops(struct iris_core *core)
--
2.7.4

2023-12-18 12:01:13

[permalink] [raw]

Subject: [PATCH v2 33/34] media: iris: add vb2 streaming and buffer ops for encoder

During stream on, set some mandatory properties
to firmware to start a session. Set all v4l2 properties set
by client, to firmware prepared with the dependency graph.
Also, configure the hardware internal buffers required
for frame processing. Send HFI_CMD_START on capture and
output planes to start the processing on respective planes.
The instance state is changed accordingly.

During stream off, send HFI_CMD_STOP to firmware which is
a synchronous command. After the response is received from
firmware, the session is closed on firmware and
instance state is changed accordingly.

Buffers are submitted and received via HFI_CMD_BUFFER.

Add Internal buffer size calculations for Encoder.

Signed-off-by: Dikshita Agarwal <[email protected]>
---
.../media/platform/qcom/vcodec/iris/hfi_defines.h | 11 +
.../media/platform/qcom/vcodec/iris/iris_buffer.c | 296 ++++++++--
.../media/platform/qcom/vcodec/iris/iris_ctrls.c | 49 +-
.../media/platform/qcom/vcodec/iris/iris_ctrls.h | 1 +
.../media/platform/qcom/vcodec/iris/iris_helpers.c | 169 +++++-
.../media/platform/qcom/vcodec/iris/iris_helpers.h | 6 +
drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 22 +-
.../platform/qcom/vcodec/iris/iris_hfi_packet.c | 137 +++--
.../platform/qcom/vcodec/iris/iris_hfi_packet.h | 9 +-
.../platform/qcom/vcodec/iris/iris_hfi_response.c | 143 +++--
drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 62 +-
.../media/platform/qcom/vcodec/iris/iris_vdec.c | 137 +----
.../media/platform/qcom/vcodec/iris/iris_venc.c | 423 ++++++++++++++
.../media/platform/qcom/vcodec/iris/iris_venc.h | 3 +
.../platform/qcom/vcodec/iris/platform_common.h | 5 +
.../platform/qcom/vcodec/iris/platform_sm8550.c | 2 +
.../platform/qcom/vcodec/iris/vpu_iris3_buffer.c | 198 ++++++-
.../platform/qcom/vcodec/iris/vpu_iris3_buffer.h | 624 +++++++++++++++++++++
.../platform/qcom/vcodec/iris/vpu_iris3_power.c | 2 +-
19 files changed, 1992 insertions(+), 307 deletions(-)

diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
index bc32c99..59765de 100644
--- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
+++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
@@ -284,6 +284,10 @@ enum hfi_picture_type {

#define HFI_PROP_NO_OUTPUT 0x0300016a

+#define HFI_PROP_BUFFER_MARK 0x0300016c
+
+#define HFI_PROP_RAW_RESOLUTION 0x03000178
+
#define HFI_PROP_DPB_LIST 0x0300017A

#define HFI_PROP_TOTAL_PEAK_BITRATE 0x0300017C
@@ -372,4 +376,11 @@ enum hfi_buffer_firmware_flags {
HFI_BUF_FW_FLAG_PSC_LAST = 0x20000000,
};

+enum hfi_hevc_profile_type {
+ HFI_H265_PROFILE_MAIN = 0,
+ HFI_H265_PROFILE_MAIN_STILL_PICTURE = 1,
+ HFI_H265_PROFILE_MAIN_10 = 2,
+ HFI_H265_PROFILE_MAIN_10_STILL_PICTURE = 3,
+};
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
index f85b52c..18418e4e 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
@@ -10,6 +10,7 @@
#include "hfi_defines.h"
#include "iris_hfi_packet.h"
#include "iris_instance.h"
+#include "vpu_iris3_buffer.h"

static const u32 dec_ip_int_buf_type[] = {
BUF_BIN,
@@ -22,15 +23,51 @@ static const u32 dec_op_int_buf_type[] = {
BUF_DPB,
};

+static const u32 enc_ip_int_buf_type[] = {
+ BUF_VPSS,
+};
+
+static const u32 enc_op_int_buf_type[] = {
+ BUF_BIN,
+ BUF_COMV,
+ BUF_NON_COMV,
+ BUF_LINE,
+ BUF_DPB,
+};
+
static int input_min_count(struct iris_inst *inst)
{
- return MIN_BUFFERS;
+ u32 input_min_count = 0;
+ u32 total_hb_layer = 0;
+
+ if (inst->domain == DECODER) {
+ input_min_count = MIN_BUFFERS;
+ } else if (inst->domain == ENCODER) {
+ total_hb_layer = is_hierb_type_requested(inst) ?
+ inst->cap[ENH_LAYER_COUNT].value + 1 : 0;
+ if (inst->codec == H264 &&
+ !inst->cap[LAYER_ENABLE].value) {
+ total_hb_layer = 0;
+ }
+ input_min_count =
+ hfi_iris3_enc_min_input_buf_count(total_hb_layer);
+ } else {
+ return 0;
+ }
+
+ return input_min_count;
}

static int output_min_count(struct iris_inst *inst)
{
int output_min_count;

+ if (inst->domain != DECODER && inst->domain != ENCODER)
+ return 0;
+
+ if (inst->domain == ENCODER)
+ return MIN_BUFFERS;
+
/* fw_min_count > 0 indicates reconfig event has already arrived */
if (inst->fw_min_count) {
if (is_split_mode_enabled(inst) && inst->codec == VP9)
@@ -83,16 +120,21 @@ static u32 internal_buffer_count(struct iris_inst *inst,
{
u32 count = 0;

- if (buffer_type == BUF_BIN || buffer_type == BUF_LINE ||
- buffer_type == BUF_PERSIST) {
- count = 1;
- } else if (buffer_type == BUF_COMV || buffer_type == BUF_NON_COMV) {
- if (inst->codec == H264 || inst->codec == HEVC)
+ if (inst->domain == ENCODER)
+ return 1;
+
+ if (inst->domain == DECODER) {
+ if (buffer_type == BUF_BIN || buffer_type == BUF_LINE ||
+ buffer_type == BUF_PERSIST) {
count = 1;
- else
+ } else if (buffer_type == BUF_COMV || buffer_type == BUF_NON_COMV) {
+ if (inst->codec == H264 || inst->codec == HEVC)
+ count = 1;
+ else
+ count = 0;
+ } else {
count = 0;
- } else {
- count = 0;
+ }
}

return count;
@@ -102,6 +144,9 @@ static int dpb_count(struct iris_inst *inst)
{
int count = 0;

+ if (inst->domain == ENCODER)
+ return get_recon_buf_count(inst);
+
if (is_split_mode_enabled(inst)) {
count = inst->fw_min_count ?
inst->fw_min_count : inst->buffers.output.min_count;
@@ -123,6 +168,7 @@ int iris_get_buf_min_count(struct iris_inst *inst,
case BUF_NON_COMV:
case BUF_LINE:
case BUF_PERSIST:
+ case BUF_ARP:
return internal_buffer_count(inst, buffer_type);
case BUF_DPB:
return dpb_count(inst);
@@ -131,7 +177,7 @@ int iris_get_buf_min_count(struct iris_inst *inst,
}
}

-static u32 input_buffer_size(struct iris_inst *inst)
+static u32 dec_input_buffer_size(struct iris_inst *inst)
{
u32 base_res_mbs = NUM_MBS_4k;
u32 frame_size, num_mbs;
@@ -163,7 +209,7 @@ static u32 input_buffer_size(struct iris_inst *inst)
return ALIGN(frame_size, SZ_4K);
}

-static u32 output_buffer_size(struct iris_inst *inst)
+static u32 dec_output_buffer_size(struct iris_inst *inst)
{
struct v4l2_format *f;
u32 size;
@@ -175,17 +221,97 @@ static u32 output_buffer_size(struct iris_inst *inst)
return size;
}

+static u32 enc_input_buffer_size(struct iris_inst *inst)
+{
+ struct v4l2_format *f;
+ u32 size;
+
+ f = inst->fmt_src;
+
+ size = video_raw_buffer_size(f->fmt.pix_mp.pixelformat, f->fmt.pix_mp.width,
+ f->fmt.pix_mp.height);
+ return size;
+}
+
+static inline
+u32 hfi_buffer_bitstream_enc(u32 frame_width, u32 frame_height,
+ u32 rc_type, bool is_ten_bit)
+{
+ u32 aligned_width, aligned_height, bitstream_size, yuv_size;
+
+ aligned_width = ALIGN(frame_width, 32);
+ aligned_height = ALIGN(frame_height, 32);
+ bitstream_size = aligned_width * aligned_height * 3;
+ yuv_size = (aligned_width * aligned_height * 3) >> 1;
+ if (aligned_width * aligned_height > (4096 * 2176))
+ /* bitstream_size = 0.25 * yuv_size; */
+ bitstream_size = (bitstream_size >> 3);
+ else if (aligned_width * aligned_height > (1280 * 720))
+ /* bitstream_size = 0.5 * yuv_size; */
+ bitstream_size = (bitstream_size >> 2);
+
+ if ((rc_type == HFI_RC_CQ || rc_type == HFI_RC_OFF) &&
+ bitstream_size < yuv_size)
+ bitstream_size = (bitstream_size << 1);
+
+ if (is_ten_bit)
+ bitstream_size = (bitstream_size) + (bitstream_size >> 2);
+
+ return ALIGN(bitstream_size, HFI_ALIGNMENT_4096);
+}
+
+static u32 enc_output_buffer_size(struct iris_inst *inst)
+{
+ u32 hfi_rc_type = HFI_RC_VBR_CFR;
+ enum codec_type codec;
+ int bitrate_mode, frame_rc;
+ bool is_ten_bit = false;
+ struct v4l2_format *f;
+ u32 frame_size;
+
+ f = inst->fmt_dst;
+ codec = v4l2_codec_to_driver(inst, f->fmt.pix_mp.pixelformat);
+ if (codec == HEVC)
+ is_ten_bit = true;
+
+ bitrate_mode = inst->cap[BITRATE_MODE].value;
+ frame_rc = inst->cap[FRAME_RC_ENABLE].value;
+ if (!frame_rc)
+ hfi_rc_type = HFI_RC_OFF;
+ else if (bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CQ)
+ hfi_rc_type = HFI_RC_CQ;
+
+ frame_size = hfi_buffer_bitstream_enc(f->fmt.pix_mp.width,
+ f->fmt.pix_mp.height,
+ hfi_rc_type, is_ten_bit);
+
+ return frame_size;
+}
+
int iris_get_buffer_size(struct iris_inst *inst,
enum iris_buffer_type buffer_type)
{
- switch (buffer_type) {
- case BUF_INPUT:
- return input_buffer_size(inst);
- case BUF_OUTPUT:
- return output_buffer_size(inst);
- default:
- return 0;
+ if (inst->domain == DECODER) {
+ switch (buffer_type) {
+ case BUF_INPUT:
+ return dec_input_buffer_size(inst);
+ case BUF_OUTPUT:
+ return dec_output_buffer_size(inst);
+ default:
+ break;
+ }
+ } else if (inst->domain == ENCODER) {
+ switch (buffer_type) {
+ case BUF_INPUT:
+ return enc_input_buffer_size(inst);
+ case BUF_OUTPUT:
+ return enc_output_buffer_size(inst);
+ default:
+ break;
+ }
}
+
+ return 0;
}

struct iris_buffers *iris_get_buffer_list(struct iris_inst *inst,
@@ -300,14 +426,30 @@ int iris_get_internal_buffers(struct iris_inst *inst,
int ret = 0;
u32 i = 0;

- if (plane == INPUT_MPLANE) {
- for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
- ret = iris_get_internal_buf_info(inst, dec_ip_int_buf_type[i]);
- if (ret)
- return ret;
+ if (inst->domain == DECODER) {
+ if (plane == INPUT_MPLANE) {
+ for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
+ ret = iris_get_internal_buf_info(inst, dec_ip_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
+ } else {
+ return iris_get_internal_buf_info(inst, BUF_DPB);
+ }
+ } else if (inst->domain == ENCODER) {
+ if (plane == INPUT_MPLANE) {
+ for (i = 0; i < ARRAY_SIZE(enc_ip_int_buf_type); i++) {
+ ret = iris_get_internal_buf_info(inst, enc_ip_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
+ } else {
+ for (i = 0; i < ARRAY_SIZE(enc_op_int_buf_type); i++) {
+ ret = iris_get_internal_buf_info(inst, enc_op_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
}
- } else {
- return iris_get_internal_buf_info(inst, BUF_DPB);
}

return ret;
@@ -377,10 +519,18 @@ int iris_create_input_internal_buffers(struct iris_inst *inst)
int ret = 0;
u32 i = 0;

- for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
- ret = iris_create_internal_buffers(inst, dec_ip_int_buf_type[i]);
- if (ret)
- return ret;
+ if (inst->domain == DECODER) {
+ for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
+ ret = iris_create_internal_buffers(inst, dec_ip_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
+ } else if (inst->domain == ENCODER) {
+ for (i = 0; i < ARRAY_SIZE(enc_ip_int_buf_type); i++) {
+ ret = iris_create_internal_buffers(inst, enc_ip_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
}

return ret;
@@ -388,7 +538,20 @@ int iris_create_input_internal_buffers(struct iris_inst *inst)

int iris_create_output_internal_buffers(struct iris_inst *inst)
{
- return iris_create_internal_buffers(inst, BUF_DPB);
+ int ret = 0;
+ u32 i = 0;
+
+ if (inst->domain == DECODER) {
+ return iris_create_internal_buffers(inst, BUF_DPB);
+ } else if (inst->domain == ENCODER) {
+ for (i = 0; i < ARRAY_SIZE(enc_op_int_buf_type); i++) {
+ ret = iris_create_internal_buffers(inst, enc_op_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
+ }
+
+ return ret;
}

static int set_num_comv(struct iris_inst *inst)
@@ -413,7 +576,7 @@ static int iris_queue_internal_buffers(struct iris_inst *inst,
struct iris_buffers *buffers;
int ret = 0;

- if (buffer_type == BUF_COMV) {
+ if (inst->domain == DECODER && buffer_type == BUF_COMV) {
ret = set_num_comv(inst);
if (ret)
return ret;
@@ -442,10 +605,18 @@ int iris_queue_input_internal_buffers(struct iris_inst *inst)
int ret = 0;
u32 i;

- for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
- ret = iris_queue_internal_buffers(inst, dec_ip_int_buf_type[i]);
- if (ret)
- return ret;
+ if (inst->domain == DECODER) {
+ for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
+ ret = iris_queue_internal_buffers(inst, dec_ip_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
+ } else if (inst->domain == ENCODER) {
+ for (i = 0; i < ARRAY_SIZE(enc_ip_int_buf_type); i++) {
+ ret = iris_queue_internal_buffers(inst, enc_ip_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
}

return ret;
@@ -453,7 +624,20 @@ int iris_queue_input_internal_buffers(struct iris_inst *inst)

int iris_queue_output_internal_buffers(struct iris_inst *inst)
{
- return iris_queue_internal_buffers(inst, BUF_DPB);
+ int ret = 0;
+ u32 i = 0;
+
+ if (inst->domain == DECODER) {
+ return iris_queue_internal_buffers(inst, BUF_DPB);
+ } else if (inst->domain == ENCODER) {
+ for (i = 0; i < ARRAY_SIZE(enc_op_int_buf_type); i++) {
+ ret = iris_queue_internal_buffers(inst, enc_op_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
+ }
+
+ return ret;
}

int iris_destroy_internal_buffer(struct iris_inst *inst,
@@ -489,16 +673,26 @@ int iris_destroy_internal_buffers(struct iris_inst *inst,
{
struct iris_buffer *buf, *dummy;
struct iris_buffers *buffers;
- const u32 *internal_buf_type;
+ const u32 *internal_buf_type = NULL;
int ret = 0;
- u32 i, len;
-
- if (plane == INPUT_MPLANE) {
- internal_buf_type = dec_ip_int_buf_type;
- len = ARRAY_SIZE(dec_ip_int_buf_type);
- } else {
- internal_buf_type = dec_op_int_buf_type;
- len = ARRAY_SIZE(dec_op_int_buf_type);
+ u32 i, len = 0;
+
+ if (inst->domain == DECODER) {
+ if (plane == INPUT_MPLANE) {
+ internal_buf_type = dec_ip_int_buf_type;
+ len = ARRAY_SIZE(dec_ip_int_buf_type);
+ } else {
+ internal_buf_type = dec_op_int_buf_type;
+ len = ARRAY_SIZE(dec_op_int_buf_type);
+ }
+ } else if (inst->domain == ENCODER) {
+ if (plane == INPUT_MPLANE) {
+ internal_buf_type = enc_ip_int_buf_type;
+ len = ARRAY_SIZE(enc_ip_int_buf_type);
+ } else {
+ internal_buf_type = enc_op_int_buf_type;
+ len = ARRAY_SIZE(enc_op_int_buf_type);
+ }
}

for (i = 0; i < len; i++) {
@@ -552,10 +746,12 @@ static int iris_release_input_internal_buffers(struct iris_inst *inst)
int ret = 0;
u32 i = 0;

- for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
- ret = iris_release_internal_buffers(inst, dec_ip_int_buf_type[i]);
- if (ret)
- return ret;
+ if (inst->domain == DECODER) {
+ for (i = 0; i < ARRAY_SIZE(dec_ip_int_buf_type); i++) {
+ ret = iris_release_internal_buffers(inst, dec_ip_int_buf_type[i]);
+ if (ret)
+ return ret;
+ }
}

return ret;
@@ -566,7 +762,7 @@ int iris_alloc_and_queue_session_int_bufs(struct iris_inst *inst,
{
int ret;

- if (buffer_type != BUF_PERSIST)
+ if (buffer_type != BUF_ARP && buffer_type != BUF_PERSIST)
return -EINVAL;

ret = iris_get_internal_buf_info(inst, buffer_type);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
index af99ac73..559b0dd 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
@@ -723,11 +723,20 @@ int set_stage(struct iris_inst *inst,

hfi_id = inst->cap[cap_id].hfi_id;

- inp_f = inst->fmt_src;
- height = inp_f->fmt.pix_mp.height;
- width = inp_f->fmt.pix_mp.width;
- if (res_is_less_than(width, height, 1280, 720))
- work_mode = STAGE_1;
+ if (inst->domain == DECODER) {
+ inp_f = inst->fmt_src;
+ height = inp_f->fmt.pix_mp.height;
+ width = inp_f->fmt.pix_mp.width;
+ if (res_is_less_than(width, height, 1280, 720))
+ work_mode = STAGE_1;
+ } else if (inst->domain == ENCODER) {
+ if (inst->cap[SLICE_MODE].value ==
+ V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_BYTES)
+ work_mode = STAGE_1;
+
+ if (!inst->cap[GOP_SIZE].value)
+ work_mode = STAGE_2;
+ }

return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
get_port_info(inst, cap_id),
@@ -743,6 +752,12 @@ int set_pipe(struct iris_inst *inst,
work_route = inst->cap[cap_id].value;
hfi_id = inst->cap[cap_id].hfi_id;

+ if (inst->domain == ENCODER) {
+ if (inst->cap[SLICE_MODE].value ==
+ V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_BYTES)
+ work_route = PIPE_1;
+ }
+
return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE,
get_port_info(inst, cap_id),
HFI_PAYLOAD_U32,
@@ -763,6 +778,30 @@ int set_level(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
&hfi_value, sizeof(u32));
}

+int decide_quality_mode(struct iris_inst *inst)
+{
+ u32 fps, mbpf, mbps, max_hq_mbpf, max_hq_mbps;
+ u32 mode = POWER_SAVE_MODE;
+ struct iris_core *core;
+
+ if (inst->domain != ENCODER)
+ return 0;
+
+ mbpf = NUM_MBS_PER_FRAME(inst->crop.height, inst->crop.width);
+ fps = max3(inst->cap[QUEUED_RATE].value >> 16,
+ inst->cap[FRAME_RATE].value >> 16,
+ inst->cap[OPERATING_RATE].value >> 16);
+ mbps = mbpf * fps;
+ core = inst->core;
+ max_hq_mbpf = core->cap[MAX_MBPF_HQ].value;
+ max_hq_mbps = core->cap[MAX_MBPS_HQ].value;
+
+ if (mbpf <= max_hq_mbpf && mbps <= max_hq_mbps)
+ mode = MAX_QUALITY_MODE;
+
+ return mode;
+}
+
int set_req_sync_frame(struct iris_inst *inst, enum plat_inst_cap_type cap_id)
{
u32 hfi_id, hfi_val;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
index 5421d9f..404d98c 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
@@ -33,6 +33,7 @@ int adjust_v4l2_properties(struct iris_inst *inst);
int ctrls_init(struct iris_inst *inst, bool init);
int set_q16(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
int set_level(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
+int decide_quality_mode(struct iris_inst *inst);
int set_req_sync_frame(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
int set_flip(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
int set_rotation(struct iris_inst *inst, enum plat_inst_cap_type cap_id);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
index c84bb51..a770157 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
@@ -44,15 +44,15 @@ u32 get_port_info(struct iris_inst *inst,
if (inst->cap[cap_id].flags & CAP_FLAG_INPUT_PORT &&
inst->cap[cap_id].flags & CAP_FLAG_OUTPUT_PORT) {
if (inst->vb2q_dst->streaming)
- return get_hfi_port(INPUT_MPLANE);
+ return get_hfi_port(inst, INPUT_MPLANE);
else
- return get_hfi_port(OUTPUT_MPLANE);
+ return get_hfi_port(inst, OUTPUT_MPLANE);
}

if (inst->cap[cap_id].flags & CAP_FLAG_INPUT_PORT)
- return get_hfi_port(INPUT_MPLANE);
+ return get_hfi_port(inst, INPUT_MPLANE);
else if (inst->cap[cap_id].flags & CAP_FLAG_OUTPUT_PORT)
- return get_hfi_port(OUTPUT_MPLANE);
+ return get_hfi_port(inst, OUTPUT_MPLANE);
else
return HFI_PORT_NONE;
}
@@ -86,9 +86,14 @@ int get_mbpf(struct iris_inst *inst)
int height = 0, width = 0;
struct v4l2_format *inp_f;

- inp_f = inst->fmt_src;
- width = max(inp_f->fmt.pix_mp.width, inst->crop.width);
- height = max(inp_f->fmt.pix_mp.height, inst->crop.height);
+ if (inst->domain == DECODER) {
+ inp_f = inst->fmt_src;
+ width = max(inp_f->fmt.pix_mp.width, inst->crop.width);
+ height = max(inp_f->fmt.pix_mp.height, inst->crop.height);
+ } else if (inst->domain == ENCODER) {
+ width = inst->crop.width;
+ height = inst->crop.height;
+ }

return NUM_MBS_PER_FRAME(height, width);
}
@@ -100,6 +105,9 @@ inline bool is_linear_colorformat(u32 colorformat)

bool is_split_mode_enabled(struct iris_inst *inst)
{
+ if (inst->domain != DECODER)
+ return false;
+
if (is_linear_colorformat(inst->fmt_dst->fmt.pix_mp.pixelformat))
return true;

@@ -118,6 +126,24 @@ inline bool is_8bit_colorformat(enum colorformat_type colorformat)
colorformat == FMT_NV21;
}

+inline bool is_scaling_enabled(struct iris_inst *inst)
+{
+ return inst->crop.left != inst->compose.left ||
+ inst->crop.top != inst->compose.top ||
+ inst->crop.width != inst->compose.width ||
+ inst->crop.height != inst->compose.height;
+}
+
+inline bool is_hierb_type_requested(struct iris_inst *inst)
+{
+ return (inst->codec == H264 &&
+ inst->cap[LAYER_TYPE].value ==
+ V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_B) ||
+ (inst->codec == HEVC &&
+ inst->cap[LAYER_TYPE].value ==
+ V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B);
+}
+
u32 v4l2_codec_from_driver(struct iris_inst *inst, enum codec_type codec)
{
const struct codec_info *codec_info;
@@ -319,17 +345,24 @@ static int check_resolution_supported(struct iris_inst *inst)
u32 width = 0, height = 0, min_width, min_height,
max_width, max_height;

- width = inst->fmt_src->fmt.pix_mp.width;
- height = inst->fmt_src->fmt.pix_mp.height;
+ if (inst->domain == DECODER) {
+ width = inst->fmt_src->fmt.pix_mp.width;
+ height = inst->fmt_src->fmt.pix_mp.height;
+ } else if (inst->domain == ENCODER) {
+ width = inst->crop.width;
+ height = inst->crop.height;
+ }

min_width = inst->cap[FRAME_WIDTH].min;
max_width = inst->cap[FRAME_WIDTH].max;
min_height = inst->cap[FRAME_HEIGHT].min;
max_height = inst->cap[FRAME_HEIGHT].max;

- if (!(min_width <= width && width <= max_width) ||
- !(min_height <= height && height <= max_height))
- return -EINVAL;
+ if (inst->domain == DECODER || inst->domain == ENCODER) {
+ if (!(min_width <= width && width <= max_width) ||
+ !(min_height <= height && height <= max_height))
+ return -EINVAL;
+ }

return 0;
}
@@ -470,7 +503,7 @@ int queue_buffer(struct iris_inst *inst, struct iris_buffer *buf)
{
int ret;

- if (buf->type == BUF_OUTPUT)
+ if (inst->domain == DECODER && buf->type == BUF_OUTPUT)
process_requeued_readonly_buffers(inst, buf);

ret = iris_hfi_queue_buffer(inst, buf);
@@ -653,7 +686,7 @@ static int iris_flush_read_only_buffers(struct iris_inst *inst,
{
struct iris_buffer *ro_buf, *dummy;

- if (type != BUF_OUTPUT)
+ if (inst->domain != DECODER || type != BUF_OUTPUT)
return 0;

list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) {
@@ -899,6 +932,114 @@ int codec_change(struct iris_inst *inst, u32 v4l2_codec)
return ret;
}

+int process_streamon_input(struct iris_inst *inst)
+{
+ enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE;
+ int ret;
+
+ iris_scale_power(inst);
+
+ ret = iris_hfi_start(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
+ ret = iris_inst_change_sub_state(inst, IRIS_INST_SUB_INPUT_PAUSE, 0);
+ if (ret)
+ return ret;
+ }
+
+ if (inst->sub_state & IRIS_INST_SUB_DRC ||
+ inst->sub_state & IRIS_INST_SUB_DRAIN) {
+ if (!(inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE)) {
+ ret = iris_hfi_pause(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+ set_sub_state = IRIS_INST_SUB_INPUT_PAUSE;
+ }
+ }
+
+ ret = iris_inst_state_change_streamon(inst, INPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = iris_inst_change_sub_state(inst, 0, set_sub_state);
+
+ return ret;
+}
+
+int process_streamon_output(struct iris_inst *inst)
+{
+ enum iris_inst_sub_state clear_sub_state = IRIS_INST_SUB_NONE;
+ bool drain_pending = false;
+ int ret;
+
+ iris_scale_power(inst);
+
+ if (inst->sub_state & IRIS_INST_SUB_DRC &&
+ inst->sub_state & IRIS_INST_SUB_DRC_LAST)
+ clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST;
+
+ if (inst->domain == DECODER && inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
+ ret = iris_alloc_and_queue_input_int_bufs(inst);
+ if (ret)
+ return ret;
+ ret = set_stage(inst, STAGE);
+ if (ret)
+ return ret;
+ ret = set_pipe(inst, PIPE);
+ if (ret)
+ return ret;
+ }
+
+ drain_pending = inst->sub_state & IRIS_INST_SUB_DRAIN &&
+ inst->sub_state & IRIS_INST_SUB_DRAIN_LAST;
+
+ if (!drain_pending && inst->state == IRIS_INST_INPUT_STREAMING) {
+ if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
+ ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE);
+ if (ret)
+ return ret;
+ clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE;
+ }
+ }
+
+ ret = iris_hfi_start(inst, OUTPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE)
+ clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE;
+
+ ret = iris_inst_state_change_streamon(inst, OUTPUT_MPLANE);
+ if (ret)
+ return ret;
+
+ ret = iris_inst_change_sub_state(inst, clear_sub_state, 0);
+
+ return ret;
+}
+
+int vb2_buffer_to_driver(struct vb2_buffer *vb2, struct iris_buffer *buf)
+{
+ struct vb2_v4l2_buffer *vbuf;
+
+ if (!vb2 || !buf)
+ return -EINVAL;
+
+ vbuf = to_vb2_v4l2_buffer(vb2);
+
+ buf->fd = vb2->planes[0].m.fd;
+ buf->data_offset = vb2->planes[0].data_offset;
+ buf->data_size = vb2->planes[0].bytesused - vb2->planes[0].data_offset;
+ buf->buffer_size = vb2->planes[0].length;
+ buf->timestamp = vb2->timestamp;
+ buf->flags = vbuf->flags;
+ buf->attr = 0;
+
+ return 0;
+}
+
int iris_pm_get(struct iris_core *core)
{
int ret;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
index 39cec8c..bf8448a 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
@@ -31,7 +31,9 @@ int close_session(struct iris_inst *inst);
bool is_linear_colorformat(u32 colorformat);
bool is_10bit_colorformat(enum colorformat_type colorformat);
bool is_8bit_colorformat(enum colorformat_type colorformat);
+bool is_scaling_enabled(struct iris_inst *inst);
bool is_split_mode_enabled(struct iris_inst *inst);
+bool is_hierb_type_requested(struct iris_inst *inst);
int signal_session_msg_receipt(struct iris_inst *inst,
enum signal_session_response cmd);
struct iris_inst *to_instance(struct iris_core *core, u32 session_id);
@@ -54,6 +56,10 @@ void iris_destroy_buffers(struct iris_inst *inst);
int session_streamoff(struct iris_inst *inst, u32 plane);
int process_resume(struct iris_inst *inst);
int codec_change(struct iris_inst *inst, u32 v4l2_codec);
+int process_streamon_input(struct iris_inst *inst);
+int process_streamon_output(struct iris_inst *inst);
+int vb2_buffer_to_driver(struct vb2_buffer *vb2,
+ struct iris_buffer *buf);
int iris_pm_get(struct iris_core *core);
int iris_pm_put(struct iris_core *core, bool autosuspend);
int iris_pm_get_put(struct iris_core *core);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
index 00e598d..4a54293 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
@@ -356,7 +356,7 @@ int iris_hfi_session_set_default_header(struct iris_inst *inst)
ret = hfi_packet_session_property(inst,
HFI_PROP_DEC_DEFAULT_HEADER,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(INPUT_MPLANE),
+ get_hfi_port(inst, INPUT_MPLANE),
HFI_PAYLOAD_U32,
&default_header,
sizeof(u32));
@@ -394,7 +394,7 @@ int iris_hfi_start(struct iris_inst *inst, u32 plane)
HFI_CMD_START,
(HFI_HOST_FLAGS_RESPONSE_REQUIRED |
HFI_HOST_FLAGS_INTR_REQUIRED),
- get_hfi_port(plane),
+ get_hfi_port(inst, plane),
inst->session_id,
HFI_PAYLOAD_NONE,
NULL,
@@ -434,7 +434,7 @@ int iris_hfi_stop(struct iris_inst *inst, u32 plane)
(HFI_HOST_FLAGS_RESPONSE_REQUIRED |
HFI_HOST_FLAGS_INTR_REQUIRED |
HFI_HOST_FLAGS_NON_DISCARDABLE),
- get_hfi_port(plane),
+ get_hfi_port(inst, plane),
inst->session_id,
HFI_PAYLOAD_NONE,
NULL,
@@ -472,7 +472,7 @@ int iris_hfi_session_subscribe_mode(struct iris_inst *inst,
cmd,
(HFI_HOST_FLAGS_RESPONSE_REQUIRED |
HFI_HOST_FLAGS_INTR_REQUIRED),
- get_hfi_port(plane),
+ get_hfi_port(inst, plane),
inst->session_id,
payload_type,
payload,
@@ -511,7 +511,7 @@ int iris_hfi_pause(struct iris_inst *inst, u32 plane)
HFI_CMD_PAUSE,
(HFI_HOST_FLAGS_RESPONSE_REQUIRED |
HFI_HOST_FLAGS_INTR_REQUIRED),
- get_hfi_port(plane),
+ get_hfi_port(inst, plane),
inst->session_id,
HFI_PAYLOAD_NONE,
NULL,
@@ -550,7 +550,7 @@ int iris_hfi_resume(struct iris_inst *inst, u32 plane, u32 payload)
HFI_CMD_RESUME,
(HFI_HOST_FLAGS_RESPONSE_REQUIRED |
HFI_HOST_FLAGS_INTR_REQUIRED),
- get_hfi_port(plane),
+ get_hfi_port(inst, plane),
inst->session_id,
HFI_PAYLOAD_U32,
&payload,
@@ -590,7 +590,7 @@ int iris_hfi_drain(struct iris_inst *inst, u32 plane)
(HFI_HOST_FLAGS_RESPONSE_REQUIRED |
HFI_HOST_FLAGS_INTR_REQUIRED |
HFI_HOST_FLAGS_NON_DISCARDABLE),
- get_hfi_port(plane),
+ get_hfi_port(inst, plane),
inst->session_id,
HFI_PAYLOAD_NONE,
NULL,
@@ -739,14 +739,14 @@ int iris_hfi_queue_buffer(struct iris_inst *inst,
goto unlock;
}

- ret = get_hfi_buffer(buffer, &hfi_buffer);
+ ret = get_hfi_buffer(inst, buffer, &hfi_buffer);
if (ret)
goto unlock;

ret = hfi_packet_session_command(inst,
HFI_CMD_BUFFER,
HFI_HOST_FLAGS_INTR_REQUIRED,
- get_hfi_port_from_buffer_type(buffer->type),
+ get_hfi_port_from_buffer_type(inst, buffer->type),
inst->session_id,
HFI_PAYLOAD_STRUCTURE,
&hfi_buffer,
@@ -780,7 +780,7 @@ int iris_hfi_release_buffer(struct iris_inst *inst,
goto unlock;
}

- ret = get_hfi_buffer(buffer, &hfi_buffer);
+ ret = get_hfi_buffer(inst, buffer, &hfi_buffer);
if (ret)
goto unlock;

@@ -790,7 +790,7 @@ int iris_hfi_release_buffer(struct iris_inst *inst,
HFI_CMD_BUFFER,
(HFI_HOST_FLAGS_RESPONSE_REQUIRED |
HFI_HOST_FLAGS_INTR_REQUIRED),
- get_hfi_port_from_buffer_type(buffer->type),
+ get_hfi_port_from_buffer_type(inst, buffer->type),
inst->session_id,
HFI_PAYLOAD_STRUCTURE,
&hfi_buffer,
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
index dd27fa4..e9f3749 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
@@ -3,64 +3,108 @@
* Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
*/

+#include "iris_common.h"
#include "iris_core.h"
#include "iris_helpers.h"
#include "iris_hfi_packet.h"
#include "hfi_defines.h"

-u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type)
+u32 get_hfi_port_from_buffer_type(struct iris_inst *inst, enum iris_buffer_type buffer_type)
{
u32 hfi_port = HFI_PORT_NONE;

- switch (buffer_type) {
- case BUF_INPUT:
- case BUF_BIN:
- case BUF_COMV:
- case BUF_NON_COMV:
- case BUF_LINE:
- hfi_port = HFI_PORT_BITSTREAM;
- break;
- case BUF_OUTPUT:
- case BUF_DPB:
- hfi_port = HFI_PORT_RAW;
- break;
- case BUF_PERSIST:
- hfi_port = HFI_PORT_NONE;
- break;
- default:
- break;
+ if (inst->domain == DECODER) {
+ switch (buffer_type) {
+ case BUF_INPUT:
+ case BUF_BIN:
+ case BUF_COMV:
+ case BUF_NON_COMV:
+ case BUF_LINE:
+ hfi_port = HFI_PORT_BITSTREAM;
+ break;
+ case BUF_OUTPUT:
+ case BUF_DPB:
+ hfi_port = HFI_PORT_RAW;
+ break;
+ case BUF_PERSIST:
+ hfi_port = HFI_PORT_NONE;
+ break;
+ default:
+ break;
+ }
+ } else if (inst->domain == ENCODER) {
+ switch (buffer_type) {
+ case BUF_INPUT:
+ case BUF_VPSS:
+ hfi_port = HFI_PORT_RAW;
+ break;
+ case BUF_OUTPUT:
+ case BUF_BIN:
+ case BUF_COMV:
+ case BUF_NON_COMV:
+ case BUF_LINE:
+ case BUF_DPB:
+ hfi_port = HFI_PORT_BITSTREAM;
+ break;
+ case BUF_ARP:
+ hfi_port = HFI_PORT_NONE;
+ break;
+ default:
+ break;
+ }
}

return hfi_port;
}

-u32 get_hfi_port(u32 plane)
+u32 get_hfi_port(struct iris_inst *inst, u32 plane)
{
u32 hfi_port = HFI_PORT_NONE;

- switch (plane) {
- case INPUT_MPLANE:
- hfi_port = HFI_PORT_BITSTREAM;
- break;
- case OUTPUT_MPLANE:
- hfi_port = HFI_PORT_RAW;
- break;
- default:
- break;
+ if (inst->domain == DECODER) {
+ switch (plane) {
+ case INPUT_MPLANE:
+ hfi_port = HFI_PORT_BITSTREAM;
+ break;
+ case OUTPUT_MPLANE:
+ hfi_port = HFI_PORT_RAW;
+ break;
+ default:
+ break;
+ }
+ } else if (inst->domain == ENCODER) {
+ switch (plane) {
+ case INPUT_MPLANE:
+ hfi_port = HFI_PORT_RAW;
+ break;
+ case OUTPUT_MPLANE:
+ hfi_port = HFI_PORT_BITSTREAM;
+ break;
+ default:
+ break;
+ }
}

return hfi_port;
}

-static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type)
+static u32 hfi_buf_type_from_driver(enum domain_type domain, enum iris_buffer_type buffer_type)
{
switch (buffer_type) {
case BUF_INPUT:
- return HFI_BUFFER_BITSTREAM;
+ if (domain == DECODER)
+ return HFI_BUFFER_BITSTREAM;
+ else
+ return HFI_BUFFER_RAW;
case BUF_OUTPUT:
- return HFI_BUFFER_RAW;
+ if (domain == DECODER)
+ return HFI_BUFFER_RAW;
+ else
+ return HFI_BUFFER_BITSTREAM;
case BUF_BIN:
return HFI_BUFFER_BIN;
+ case BUF_ARP:
+ return HFI_BUFFER_ARP;
case BUF_COMV:
return HFI_BUFFER_COMV;
case BUF_NON_COMV:
@@ -76,13 +120,19 @@ static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type)
}
}

-u32 hfi_buf_type_to_driver(enum hfi_buffer_type buf_type)
+u32 hfi_buf_type_to_driver(enum domain_type domain, enum hfi_buffer_type buf_type)
{
switch (buf_type) {
case HFI_BUFFER_BITSTREAM:
- return BUF_INPUT;
+ if (domain == DECODER)
+ return BUF_INPUT;
+ else
+ return BUF_OUTPUT;
case HFI_BUFFER_RAW:
- return BUF_OUTPUT;
+ if (domain == DECODER)
+ return BUF_OUTPUT;
+ else
+ return BUF_INPUT;
case HFI_BUFFER_BIN:
return BUF_BIN;
case HFI_BUFFER_ARP:
@@ -108,9 +158,15 @@ u32 get_hfi_codec(struct iris_inst *inst)
{
switch (inst->codec) {
case H264:
- return HFI_CODEC_DECODE_AVC;
+ if (inst->domain == ENCODER)
+ return HFI_CODEC_ENCODE_AVC;
+ else
+ return HFI_CODEC_DECODE_AVC;
case HEVC:
- return HFI_CODEC_DECODE_HEVC;
+ if (inst->domain == ENCODER)
+ return HFI_CODEC_ENCODE_HEVC;
+ else
+ return HFI_CODEC_DECODE_HEVC;
case VP9:
return HFI_CODEC_DECODE_VP9;
default:
@@ -337,10 +393,11 @@ u32 get_v4l2_matrix_coefficients(u32 hfi_coefficients)
return coefficients;
}

-int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf)
+int get_hfi_buffer(struct iris_inst *inst,
+ struct iris_buffer *buffer, struct hfi_buffer *buf)
{
- memset(buf, 0, sizeof(*buf));
- buf->type = hfi_buf_type_from_driver(buffer->type);
+ memset(buf, 0, sizeof(struct hfi_buffer));
+ buf->type = hfi_buf_type_from_driver(inst->domain, buffer->type);
buf->index = buffer->index;
buf->base_address = buffer->device_addr;
buf->addr_offset = 0;
@@ -350,7 +407,7 @@ int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf)
* buffer size otherwise it will truncate or ignore the data after 256
* aligned size which may lead to error concealment
*/
- if (buffer->type == BUF_INPUT)
+ if (inst->domain == DECODER && buffer->type == BUF_INPUT)
buf->buffer_size = ALIGN(buffer->buffer_size, 256);
buf->data_offset = buffer->data_offset;
buf->data_size = buffer->data_size;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
index 82148b7..734a070 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
@@ -80,8 +80,8 @@ enum hfi_packet_port_type {
HFI_PORT_RAW = 0x00000002,
};

-u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type);
-u32 get_hfi_port(u32 plane);
+u32 get_hfi_port_from_buffer_type(struct iris_inst *inst, enum iris_buffer_type buffer_type);
+u32 get_hfi_port(struct iris_inst *inst, u32 plane);
u32 get_hfi_colorformat(u32 colorformat);
u32 get_hfi_codec(struct iris_inst *inst);
u32 get_hfi_color_primaries(u32 primaries);
@@ -90,8 +90,9 @@ u32 get_hfi_matrix_coefficients(u32 coefficients);
u32 get_v4l2_color_primaries(u32 hfi_primaries);
u32 get_v4l2_transfer_char(u32 hfi_characterstics);
u32 get_v4l2_matrix_coefficients(u32 hfi_coefficients);
-u32 hfi_buf_type_to_driver(enum hfi_buffer_type buf_type);
-int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf);
+u32 hfi_buf_type_to_driver(enum domain_type domain, enum hfi_buffer_type buf_type);
+int get_hfi_buffer(struct iris_inst *inst,
+ struct iris_buffer *buffer, struct hfi_buffer *buf);

int hfi_packet_sys_init(struct iris_core *core,
u8 *pkt, u32 pkt_size);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
index 1b667a5..08bdb8f 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
@@ -71,7 +71,8 @@ static bool is_valid_hfi_buffer_type(u32 buffer_type)
static bool is_valid_hfi_port(u32 port, u32 buffer_type)
{
if (port == HFI_PORT_NONE &&
- buffer_type != HFI_BUFFER_PERSIST)
+ buffer_type != HFI_BUFFER_ARP &&
+ buffer_type != HFI_BUFFER_PERSIST)
return false;

if (port != HFI_PORT_BITSTREAM && port != HFI_PORT_RAW)
@@ -103,8 +104,9 @@ static int get_driver_buffer_flags(struct iris_inst *inst, u32 hfi_flags)
if (inst->hfi_frame_info.overflow)
driver_flags |= BUF_FLAG_ERROR;

- if (hfi_flags & HFI_BUF_FW_FLAG_LAST ||
- hfi_flags & HFI_BUF_FW_FLAG_PSC_LAST)
+ if ((inst->domain == ENCODER && (hfi_flags & HFI_BUF_FW_FLAG_LAST)) ||
+ (inst->domain == DECODER && (hfi_flags & HFI_BUF_FW_FLAG_LAST ||
+ hfi_flags & HFI_BUF_FW_FLAG_PSC_LAST)))
driver_flags |= BUF_FLAG_LAST;

return driver_flags;
@@ -309,9 +311,12 @@ static int handle_session_close(struct iris_inst *inst,
static int handle_read_only_buffer(struct iris_inst *inst,
struct iris_buffer *buf)
{
- struct iris_buffer *ro_buf, *iter;
+ struct iris_buffer *ro_buf, *iter = NULL;
bool found = false;

+ if (inst->domain != DECODER && inst->domain != ENCODER)
+ return 0;
+
list_for_each_entry(iter, &inst->buffers.read_only.list, list) {
if (iter->device_addr == buf->device_addr) {
found = true;
@@ -344,6 +349,9 @@ static int handle_non_read_only_buffer(struct iris_inst *inst,
{
struct iris_buffer *ro_buf;

+ if (inst->domain != DECODER && inst->domain != ENCODER)
+ return 0;
+
list_for_each_entry(ro_buf, &inst->buffers.read_only.list, list) {
if (ro_buf->device_addr == buffer->base_address) {
ro_buf->attr &= ~BUF_ATTR_READ_ONLY;
@@ -414,9 +422,9 @@ static int handle_input_buffer(struct iris_inst *inst,
static int handle_output_buffer(struct iris_inst *inst,
struct hfi_buffer *hfi_buffer)
{
- struct iris_buffers *buffers;
struct iris_buffer *buf, *iter;
- bool found;
+ struct iris_buffers *buffers;
+ bool found, fatal = false;
int ret = 0;

if (hfi_buffer->flags & HFI_BUF_FW_FLAG_LAST) {
@@ -425,17 +433,19 @@ static int handle_output_buffer(struct iris_inst *inst,
return ret;
}

- if (hfi_buffer->flags & HFI_BUF_FW_FLAG_RELEASE_DONE)
- return handle_release_output_buffer(inst, hfi_buffer);
+ if (inst->domain == DECODER) {
+ if (hfi_buffer->flags & HFI_BUF_FW_FLAG_RELEASE_DONE)
+ return handle_release_output_buffer(inst, hfi_buffer);

- if (hfi_buffer->flags & HFI_BUF_FW_FLAG_PSC_LAST) {
- ret = iris_inst_sub_state_change_drc_last(inst);
- if (ret)
- return ret;
- }
+ if (hfi_buffer->flags & HFI_BUF_FW_FLAG_PSC_LAST) {
+ ret = iris_inst_sub_state_change_drc_last(inst);
+ if (ret)
+ return ret;
+ }

- if (!(hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY))
- ret = handle_non_read_only_buffer(inst, hfi_buffer);
+ if (!(hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY))
+ ret = handle_non_read_only_buffer(inst, hfi_buffer);
+ }

buffers = iris_get_buffer_list(inst, BUF_OUTPUT);
if (!buffers)
@@ -446,9 +456,12 @@ static int handle_output_buffer(struct iris_inst *inst,
if (!(iter->attr & BUF_ATTR_QUEUED))
continue;

- found = (iter->index == hfi_buffer->index &&
- iter->device_addr == hfi_buffer->base_address &&
- iter->data_offset == hfi_buffer->data_offset);
+ if (inst->domain == DECODER)
+ found = (iter->index == hfi_buffer->index &&
+ iter->device_addr == hfi_buffer->base_address &&
+ iter->data_offset == hfi_buffer->data_offset);
+ else
+ found = iter->index == hfi_buffer->index;

if (found) {
buf = iter;
@@ -465,14 +478,27 @@ static int handle_output_buffer(struct iris_inst *inst,
buf->attr &= ~BUF_ATTR_QUEUED;
buf->attr |= BUF_ATTR_DEQUEUED;

- if (inst->buffers.dpb.size && hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY)
- iris_inst_change_state(inst, IRIS_INST_ERROR);
+ if (inst->domain == ENCODER) {
+ if (inst->hfi_frame_info.data_corrupt)
+ fatal = true;
+ if (inst->hfi_frame_info.overflow) {
+ if (!hfi_buffer->data_size && inst->hfi_rc_type == HFI_RC_CBR_CFR)
+ fatal = true;
+ }
+ if (fatal)
+ iris_inst_change_state(inst, IRIS_INST_ERROR);
+ }
+
+ if (inst->domain == DECODER) {
+ if (inst->buffers.dpb.size && hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY)
+ iris_inst_change_state(inst, IRIS_INST_ERROR);

- if (hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY) {
- buf->attr |= BUF_ATTR_READ_ONLY;
- ret = handle_read_only_buffer(inst, buf);
- } else {
- buf->attr &= ~BUF_ATTR_READ_ONLY;
+ if (hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY) {
+ buf->attr |= BUF_ATTR_READ_ONLY;
+ ret = handle_read_only_buffer(inst, buf);
+ } else {
+ buf->attr &= ~BUF_ATTR_READ_ONLY;
+ }
}

buf->flags = get_driver_buffer_flags(inst, hfi_buffer->flags);
@@ -521,7 +547,7 @@ static int handle_release_internal_buffer(struct iris_inst *inst,
int ret = 0;
bool found;

- buffers = iris_get_buffer_list(inst, hfi_buf_type_to_driver(buffer->type));
+ buffers = iris_get_buffer_list(inst, hfi_buf_type_to_driver(inst->domain, buffer->type));
if (!buffers)
return -EINVAL;

@@ -550,12 +576,22 @@ static int handle_session_stop(struct iris_inst *inst,
int ret = 0;
enum signal_session_response signal_type = -1;

- if (pkt->port == HFI_PORT_RAW) {
- signal_type = SIGNAL_CMD_STOP_OUTPUT;
- ret = iris_inst_sub_state_change_pause(inst, OUTPUT_MPLANE);
- } else if (pkt->port == HFI_PORT_BITSTREAM) {
- signal_type = SIGNAL_CMD_STOP_INPUT;
- ret = iris_inst_sub_state_change_pause(inst, INPUT_MPLANE);
+ if (inst->domain == DECODER) {
+ if (pkt->port == HFI_PORT_RAW) {
+ signal_type = SIGNAL_CMD_STOP_OUTPUT;
+ ret = iris_inst_sub_state_change_pause(inst, OUTPUT_MPLANE);
+ } else if (pkt->port == HFI_PORT_BITSTREAM) {
+ signal_type = SIGNAL_CMD_STOP_INPUT;
+ ret = iris_inst_sub_state_change_pause(inst, INPUT_MPLANE);
+ }
+ } else if (inst->domain == ENCODER) {
+ if (pkt->port == HFI_PORT_RAW) {
+ signal_type = SIGNAL_CMD_STOP_INPUT;
+ ret = iris_inst_sub_state_change_pause(inst, INPUT_MPLANE);
+ } else if (pkt->port == HFI_PORT_BITSTREAM) {
+ signal_type = SIGNAL_CMD_STOP_OUTPUT;
+ ret = iris_inst_sub_state_change_pause(inst, OUTPUT_MPLANE);
+ }
}

if (signal_type != -1)
@@ -571,7 +607,7 @@ static int handle_session_buffer(struct iris_inst *inst,
u32 hfi_handle_size = 0;
int i, ret = 0;
const struct iris_hfi_buffer_handle *hfi_handle_arr = NULL;
- static const struct iris_hfi_buffer_handle input_hfi_handle[] = {
+ static const struct iris_hfi_buffer_handle dec_input_hfi_handle[] = {
{HFI_BUFFER_BITSTREAM, handle_input_buffer },
{HFI_BUFFER_BIN, handle_release_internal_buffer },
{HFI_BUFFER_COMV, handle_release_internal_buffer },
@@ -579,10 +615,24 @@ static int handle_session_buffer(struct iris_inst *inst,
{HFI_BUFFER_LINE, handle_release_internal_buffer },
{HFI_BUFFER_PERSIST, handle_release_internal_buffer },
};
- static const struct iris_hfi_buffer_handle output_hfi_handle[] = {
+ static const struct iris_hfi_buffer_handle dec_output_hfi_handle[] = {
{HFI_BUFFER_RAW, handle_output_buffer },
{HFI_BUFFER_DPB, handle_release_internal_buffer },
};
+ static const struct iris_hfi_buffer_handle enc_input_hfi_handle[] = {
+ {HFI_BUFFER_RAW, handle_input_buffer },
+ {HFI_BUFFER_VPSS, handle_release_internal_buffer },
+ };
+ static const struct iris_hfi_buffer_handle enc_output_hfi_handle[] = {
+ {HFI_BUFFER_BITSTREAM, handle_output_buffer },
+ {HFI_BUFFER_BIN, handle_release_internal_buffer },
+ {HFI_BUFFER_COMV, handle_release_internal_buffer },
+ {HFI_BUFFER_NON_COMV, handle_release_internal_buffer },
+ {HFI_BUFFER_LINE, handle_release_internal_buffer },
+ {HFI_BUFFER_ARP, handle_release_internal_buffer },
+ {HFI_BUFFER_DPB, handle_release_internal_buffer },
+ };
+

if (pkt->payload_info == HFI_PAYLOAD_NONE)
return 0;
@@ -599,12 +649,22 @@ static int handle_session_buffer(struct iris_inst *inst,
if (!is_valid_hfi_port(pkt->port, buffer->type))
return 0;

- if (pkt->port == HFI_PORT_BITSTREAM) {
- hfi_handle_size = ARRAY_SIZE(input_hfi_handle);
- hfi_handle_arr = input_hfi_handle;
- } else if (pkt->port == HFI_PORT_RAW) {
- hfi_handle_size = ARRAY_SIZE(output_hfi_handle);
- hfi_handle_arr = output_hfi_handle;
+ if (inst->domain == DECODER) {
+ if (pkt->port == HFI_PORT_BITSTREAM) {
+ hfi_handle_size = ARRAY_SIZE(dec_input_hfi_handle);
+ hfi_handle_arr = dec_input_hfi_handle;
+ } else if (pkt->port == HFI_PORT_RAW) {
+ hfi_handle_size = ARRAY_SIZE(dec_output_hfi_handle);
+ hfi_handle_arr = dec_output_hfi_handle;
+ }
+ } else if (inst->domain == ENCODER) {
+ if (pkt->port == HFI_PORT_RAW) {
+ hfi_handle_size = ARRAY_SIZE(enc_input_hfi_handle);
+ hfi_handle_arr = enc_input_hfi_handle;
+ } else if (pkt->port == HFI_PORT_BITSTREAM) {
+ hfi_handle_size = ARRAY_SIZE(enc_output_hfi_handle);
+ hfi_handle_arr = enc_output_hfi_handle;
+ }
}

if (!hfi_handle_arr || !hfi_handle_size)
@@ -693,6 +753,9 @@ static int handle_dpb_list_property(struct iris_inst *inst,
u8 *payload_start;
u32 payload_size;

+ if (inst->domain != DECODER)
+ return -EINVAL;
+
payload_size = pkt->size - sizeof(*pkt);
payload_start = (u8 *)((u8 *)pkt + sizeof(*pkt));
memset(inst->dpb_list_payload, 0, MAX_DPB_LIST_ARRAY_SIZE);
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
index c0878f1..1a434ff 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
@@ -13,6 +13,7 @@
#include "iris_power.h"
#include "iris_vb2.h"
#include "iris_vdec.h"
+#include "iris_venc.h"

int iris_vb2_queue_setup(struct vb2_queue *q,
unsigned int *num_buffers, unsigned int *num_planes,
@@ -58,7 +59,8 @@ int iris_vb2_queue_setup(struct vb2_queue *q,
return ret;
}

- if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ if ((inst->domain == DECODER && q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) ||
+ (inst->domain == ENCODER && q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)) {
ret = adjust_v4l2_properties(inst);
if (ret)
return ret;
@@ -112,6 +114,11 @@ int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count)
goto error;
}

+ if (inst->domain != DECODER && inst->domain != ENCODER) {
+ ret = -EINVAL;
+ goto error;
+ }
+
ret = iris_pm_get(inst->core);
if (ret)
goto error;
@@ -122,21 +129,34 @@ int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count)
if (ret)
goto err_pm_get;

- ret = iris_hfi_session_set_default_header(inst);
- if (ret)
- goto err_pm_get;
-
- ret = iris_alloc_and_queue_session_int_bufs(inst, BUF_PERSIST);
- if (ret)
- goto err_pm_get;
+ if (inst->domain == ENCODER) {
+ ret = iris_alloc_and_queue_session_int_bufs(inst, BUF_ARP);
+ if (ret)
+ goto err_pm_get;
+ } else if (inst->domain == DECODER) {
+ ret = iris_hfi_session_set_default_header(inst);
+ if (ret)
+ goto err_pm_get;
+
+ ret = iris_alloc_and_queue_session_int_bufs(inst, BUF_PERSIST);
+ if (ret)
+ goto err_pm_get;
+ }
}

iris_scale_power(inst);

- if (q->type == INPUT_MPLANE)
- ret = vdec_streamon_input(inst);
- else if (q->type == OUTPUT_MPLANE)
- ret = vdec_streamon_output(inst);
+ if (q->type == INPUT_MPLANE) {
+ if (inst->domain == DECODER)
+ ret = vdec_streamon_input(inst);
+ else if (inst->domain == ENCODER)
+ ret = venc_streamon_input(inst);
+ } else if (q->type == OUTPUT_MPLANE) {
+ if (inst->domain == DECODER)
+ ret = vdec_streamon_output(inst);
+ else if (inst->domain == ENCODER)
+ ret = venc_streamon_output(inst);
+ }
if (ret)
goto err_pm_get;

@@ -179,6 +199,11 @@ void iris_vb2_stop_streaming(struct vb2_queue *q)
if (q->type != INPUT_MPLANE && q->type != OUTPUT_MPLANE)
goto error;

+ if (inst->domain != DECODER && inst->domain != ENCODER) {
+ ret = -EINVAL;
+ goto error;
+ }
+
ret = iris_pm_get_put(inst->core);
if (ret)
goto error;
@@ -225,7 +250,10 @@ void iris_vb2_buf_queue(struct vb2_buffer *vb2)
if (ret)
goto exit;

- ret = vdec_qbuf(inst, vb2);
+ if (inst->domain == DECODER)
+ ret = vdec_qbuf(inst, vb2);
+ else if (inst->domain == ENCODER)
+ ret = venc_qbuf(inst, vb2);

exit:
if (ret) {
@@ -270,7 +298,7 @@ void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev,
buf->inst = inst;
buf->dmabuf = dbuf;

- if (buf->type == BUF_OUTPUT) {
+ if (inst->domain == DECODER && buf->type == BUF_OUTPUT) {
list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) {
if (ro_buf->dmabuf != buf->dmabuf)
continue;
@@ -307,7 +335,7 @@ int iris_vb2_map_dmabuf(void *buf_priv)
return -EINVAL;
}

- if (buf->type == BUF_OUTPUT) {
+ if (inst->domain == DECODER && buf->type == BUF_OUTPUT) {
list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) {
if (ro_buf->dmabuf != buf->dmabuf)
continue;
@@ -356,7 +384,7 @@ void iris_vb2_unmap_dmabuf(void *buf_priv)
return;
}

- if (buf->type == BUF_OUTPUT) {
+ if (inst->domain == DECODER && buf->type == BUF_OUTPUT) {
list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) {
if (ro_buf->dmabuf != buf->dmabuf)
continue;
@@ -393,7 +421,7 @@ void iris_vb2_detach_dmabuf(void *buf_priv)
buf->sg_table = NULL;
}

- if (buf->type == BUF_OUTPUT) {
+ if (inst->domain == DECODER && buf->type == BUF_OUTPUT) {
list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) {
if (ro_buf->dmabuf != buf->dmabuf)
continue;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
index 300d0e9..b2d4739 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
@@ -362,7 +362,7 @@ static int vdec_set_bitstream_resolution(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_BITSTREAM_RESOLUTION,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(INPUT_MPLANE),
+ get_hfi_port(inst, INPUT_MPLANE),
HFI_PAYLOAD_U32,
&resolution,
sizeof(u32));
@@ -388,7 +388,7 @@ static int vdec_set_crop_offsets(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_CROP_OFFSETS,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(INPUT_MPLANE),
+ get_hfi_port(inst, INPUT_MPLANE),
HFI_PAYLOAD_64_PACKED,
&payload,
sizeof(u64));
@@ -409,7 +409,7 @@ static int vdec_set_bit_depth(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_LUMA_CHROMA_BIT_DEPTH,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(INPUT_MPLANE),
+ get_hfi_port(inst, INPUT_MPLANE),
HFI_PAYLOAD_U32,
&bitdepth,
sizeof(u32));
@@ -426,7 +426,7 @@ static int vdec_set_coded_frames(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_CODED_FRAMES,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(INPUT_MPLANE),
+ get_hfi_port(inst, INPUT_MPLANE),
HFI_PAYLOAD_U32,
&coded_frames,
sizeof(u32));
@@ -442,7 +442,7 @@ static int vdec_set_min_output_count(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(INPUT_MPLANE),
+ get_hfi_port(inst, INPUT_MPLANE),
HFI_PAYLOAD_U32,
&min_output,
sizeof(u32));
@@ -457,7 +457,7 @@ static int vdec_set_picture_order_count(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_PIC_ORDER_CNT_TYPE,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(INPUT_MPLANE),
+ get_hfi_port(inst, INPUT_MPLANE),
HFI_PAYLOAD_U32,
&poc,
sizeof(u32));
@@ -509,7 +509,7 @@ static int vdec_set_colorspace(struct iris_inst *inst)
ret = iris_hfi_set_property(inst,
HFI_PROP_SIGNAL_COLOR_INFO,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(INPUT_MPLANE),
+ get_hfi_port(inst, INPUT_MPLANE),
HFI_PAYLOAD_32_PACKED,
&color_info,
sizeof(u32));
@@ -527,7 +527,7 @@ static int vdec_set_profile(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_PROFILE,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(INPUT_MPLANE),
+ get_hfi_port(inst, INPUT_MPLANE),
HFI_PAYLOAD_U32_ENUM,
&profile,
sizeof(u32));
@@ -543,7 +543,7 @@ static int vdec_set_level(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_LEVEL,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(INPUT_MPLANE),
+ get_hfi_port(inst, INPUT_MPLANE),
HFI_PAYLOAD_U32_ENUM,
&level,
sizeof(u32));
@@ -559,7 +559,7 @@ static int vdec_set_tier(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_TIER,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(INPUT_MPLANE),
+ get_hfi_port(inst, INPUT_MPLANE),
HFI_PAYLOAD_U32_ENUM,
&tier,
sizeof(u32));
@@ -811,7 +811,7 @@ static int vdec_set_colorformat(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_COLOR_FORMAT,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(OUTPUT_MPLANE),
+ get_hfi_port(inst, OUTPUT_MPLANE),
HFI_PAYLOAD_U32,
&hfi_colorformat,
sizeof(u32));
@@ -839,7 +839,7 @@ static int vdec_set_linear_stride_scanline(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_LINEAR_STRIDE_SCANLINE,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(OUTPUT_MPLANE),
+ get_hfi_port(inst, OUTPUT_MPLANE),
HFI_PAYLOAD_U64,
&payload,
sizeof(u64));
@@ -887,7 +887,7 @@ static int vdec_set_ubwc_stride_scanline(struct iris_inst *inst)
return iris_hfi_set_property(inst,
HFI_PROP_UBWC_STRIDE_SCANLINE,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(OUTPUT_MPLANE),
+ get_hfi_port(inst, OUTPUT_MPLANE),
HFI_PAYLOAD_U32_ARRAY,
&payload[0],
sizeof(u32) * 4);
@@ -1020,7 +1020,7 @@ static int vdec_subscribe_dst_change_param(struct iris_inst *inst)
ret = iris_hfi_set_property(inst,
prop_type,
HFI_HOST_FLAGS_NONE,
- get_hfi_port(OUTPUT_MPLANE),
+ get_hfi_port(inst, OUTPUT_MPLANE),
payload_type,
&payload,
payload_size);
@@ -1032,42 +1032,6 @@ static int vdec_subscribe_dst_change_param(struct iris_inst *inst)
return ret;
}

-static int process_streamon_input(struct iris_inst *inst)
-{
- enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE;
- int ret;
-
- iris_scale_power(inst);
-
- ret = iris_hfi_start(inst, INPUT_MPLANE);
- if (ret)
- return ret;
-
- if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
- ret = iris_inst_change_sub_state(inst, IRIS_INST_SUB_INPUT_PAUSE, 0);
- if (ret)
- return ret;
- }
-
- if (inst->sub_state & IRIS_INST_SUB_DRC ||
- inst->sub_state & IRIS_INST_SUB_DRAIN) {
- if (!(inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE)) {
- ret = iris_hfi_pause(inst, INPUT_MPLANE);
- if (ret)
- return ret;
- set_sub_state = IRIS_INST_SUB_INPUT_PAUSE;
- }
- }
-
- ret = iris_inst_state_change_streamon(inst, INPUT_MPLANE);
- if (ret)
- return ret;
-
- ret = iris_inst_change_sub_state(inst, 0, set_sub_state);
-
- return ret;
-}
-
int vdec_streamon_input(struct iris_inst *inst)
{
int ret;
@@ -1114,58 +1078,6 @@ int vdec_streamon_input(struct iris_inst *inst)
return ret;
}

-static int process_streamon_output(struct iris_inst *inst)
-{
- enum iris_inst_sub_state clear_sub_state = IRIS_INST_SUB_NONE;
- bool drain_pending = false;
- int ret;
-
- iris_scale_power(inst);
-
- if (inst->sub_state & IRIS_INST_SUB_DRC &&
- inst->sub_state & IRIS_INST_SUB_DRC_LAST)
- clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST;
-
- if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
- ret = iris_alloc_and_queue_input_int_bufs(inst);
- if (ret)
- return ret;
- ret = set_stage(inst, STAGE);
- if (ret)
- return ret;
- ret = set_pipe(inst, PIPE);
- if (ret)
- return ret;
- }
-
- drain_pending = inst->sub_state & IRIS_INST_SUB_DRAIN &&
- inst->sub_state & IRIS_INST_SUB_DRAIN_LAST;
-
- if (!drain_pending && inst->state == IRIS_INST_INPUT_STREAMING) {
- if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) {
- ret = iris_hfi_resume(inst, INPUT_MPLANE, HFI_CMD_SETTINGS_CHANGE);
- if (ret)
- return ret;
- clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE;
- }
- }
-
- ret = iris_hfi_start(inst, OUTPUT_MPLANE);
- if (ret)
- return ret;
-
- if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE)
- clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE;
-
- ret = iris_inst_state_change_streamon(inst, OUTPUT_MPLANE);
- if (ret)
- return ret;
-
- ret = iris_inst_change_sub_state(inst, clear_sub_state, 0);
-
- return ret;
-}
-
int vdec_streamon_output(struct iris_inst *inst)
{
int ret;
@@ -1220,27 +1132,6 @@ int vdec_streamon_output(struct iris_inst *inst)
return ret;
}

-static int vb2_buffer_to_driver(struct vb2_buffer *vb2,
- struct iris_buffer *buf)
-{
- struct vb2_v4l2_buffer *vbuf;
-
- if (!vb2 || !buf)
- return -EINVAL;
-
- vbuf = to_vb2_v4l2_buffer(vb2);
-
- buf->fd = vb2->planes[0].m.fd;
- buf->data_offset = vb2->planes[0].data_offset;
- buf->data_size = vb2->planes[0].bytesused - vb2->planes[0].data_offset;
- buf->buffer_size = vb2->planes[0].length;
- buf->timestamp = vb2->timestamp;
- buf->flags = vbuf->flags;
- buf->attr = 0;
-
- return 0;
-}
-
int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2)
{
struct iris_buffer *buf = NULL;
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_venc.c b/drivers/media/platform/qcom/vcodec/iris/iris_venc.c
index 802db40..28c6b20 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_venc.c
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_venc.c
@@ -13,6 +13,23 @@
#include "iris_power.h"
#include "iris_venc.h"

+#define SCALE_FACTOR 8
+#define UNSPECIFIED_COLOR_FORMAT 5
+
+static const u32 enc_input_properties[] = {
+ HFI_PROP_NO_OUTPUT,
+};
+
+static const u32 enc_output_properties[] = {
+ HFI_PROP_PICTURE_TYPE,
+ HFI_PROP_BUFFER_MARK,
+};
+
+struct venc_prop_type_handle {
+ u32 type;
+ int (*handle)(struct iris_inst *inst);
+};
+
int venc_inst_init(struct iris_inst *inst)
{
struct v4l2_format *f;
@@ -523,3 +540,409 @@ int venc_stop_cmd(struct iris_inst *inst)

return ret;
}
+
+int venc_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2)
+{
+ struct iris_buffer *buf = NULL;
+ int ret;
+
+ buf = get_driver_buf(inst, vb2->type, vb2->index);
+ if (!buf)
+ return -EINVAL;
+
+ ret = vb2_buffer_to_driver(vb2, buf);
+ if (ret)
+ return ret;
+
+ if (!allow_qbuf(inst, vb2->type)) {
+ buf->attr |= BUF_ATTR_DEFERRED;
+ return 0;
+ }
+
+ iris_scale_power(inst);
+
+ return queue_buffer(inst, buf);
+}
+
+static int check_scaling_supported(struct iris_inst *inst)
+{
+ u32 iwidth, owidth, iheight, oheight;
+
+ if (!(inst->crop.left != inst->compose.left ||
+ inst->crop.top != inst->compose.top ||
+ inst->crop.width != inst->compose.width ||
+ inst->crop.height != inst->compose.height))
+ return 0;
+
+ iwidth = inst->crop.width;
+ iheight = inst->crop.height;
+ owidth = inst->compose.width;
+ oheight = inst->compose.height;
+
+ if (owidth > iwidth || oheight > iheight)
+ return -EINVAL;
+
+ if (iwidth > owidth * SCALE_FACTOR || iheight > oheight * SCALE_FACTOR)
+ return -EINVAL;
+
+ return 0;
+}
+
+static int venc_set_colorformat(struct iris_inst *inst)
+{
+ u32 hfi_colorformat;
+ u32 pixelformat;
+
+ pixelformat = inst->fmt_src->fmt.pix_mp.pixelformat;
+ hfi_colorformat = get_hfi_colorformat(pixelformat);
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_COLOR_FORMAT,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(inst, INPUT_MPLANE),
+ HFI_PAYLOAD_U32_ENUM,
+ &hfi_colorformat,
+ sizeof(u32));
+}
+
+static int venc_set_stride_scanline(struct iris_inst *inst)
+{
+ u32 color_format, stride_y, scanline_y;
+ u32 stride_uv = 0, scanline_uv = 0;
+ u32 payload[2];
+
+ color_format = inst->cap[PIX_FMTS].value;
+ if (!is_linear_colorformat(color_format))
+ return 0;
+
+ stride_y = color_format == FMT_TP10C ?
+ ALIGN(inst->fmt_src->fmt.pix_mp.width, 192) :
+ ALIGN(inst->fmt_src->fmt.pix_mp.width, 128);
+ scanline_y = color_format == FMT_TP10C ?
+ ALIGN(inst->fmt_src->fmt.pix_mp.height, 16) :
+ ALIGN(inst->fmt_src->fmt.pix_mp.height, 32);
+
+ if (color_format == FMT_NV12 ||
+ color_format == FMT_NV21) {
+ stride_uv = stride_y;
+ scanline_uv = scanline_y / 2;
+ }
+
+ payload[0] = stride_y << 16 | scanline_y;
+ payload[1] = stride_uv << 16 | scanline_uv;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_LINEAR_STRIDE_SCANLINE,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(inst, INPUT_MPLANE),
+ HFI_PAYLOAD_64_PACKED,
+ &payload,
+ sizeof(u64));
+}
+
+static int venc_set_raw_resolution(struct iris_inst *inst)
+{
+ u32 resolution;
+
+ resolution = (inst->fmt_src->fmt.pix_mp.width << 16) |
+ inst->fmt_src->fmt.pix_mp.height;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_RAW_RESOLUTION,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(inst, INPUT_MPLANE),
+ HFI_PAYLOAD_32_PACKED,
+ &resolution,
+ sizeof(u32));
+}
+
+static int venc_set_bitstream_resolution(struct iris_inst *inst)
+{
+ u32 resolution;
+
+ resolution = (inst->fmt_dst->fmt.pix_mp.width << 16) |
+ inst->fmt_dst->fmt.pix_mp.height;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_BITSTREAM_RESOLUTION,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(inst, OUTPUT_MPLANE),
+ HFI_PAYLOAD_32_PACKED,
+ &resolution,
+ sizeof(u32));
+}
+
+static int venc_set_inp_crop_offsets(struct iris_inst *inst)
+{
+ u32 left_offset, top_offset, right_offset, bottom_offset;
+ u32 crop[2] = {0};
+ u32 width, height;
+
+ left_offset = inst->crop.left;
+ top_offset = inst->crop.top;
+ width = inst->crop.width;
+ height = inst->crop.height;
+
+ right_offset = (inst->fmt_src->fmt.pix_mp.width - width);
+ bottom_offset = (inst->fmt_src->fmt.pix_mp.height - height);
+
+ crop[0] = left_offset << 16 | top_offset;
+ crop[1] = right_offset << 16 | bottom_offset;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_CROP_OFFSETS,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(inst, INPUT_MPLANE),
+ HFI_PAYLOAD_64_PACKED,
+ &crop,
+ sizeof(u64));
+}
+
+static int venc_set_out_crop_offsets(struct iris_inst *inst)
+{
+ u32 left_offset, top_offset, right_offset, bottom_offset;
+ u32 crop[2] = {0};
+ u32 width, height;
+
+ left_offset = inst->compose.left;
+ top_offset = inst->compose.top;
+ width = inst->compose.width;
+ height = inst->compose.height;
+ if (inst->cap[ROTATION].value == 90 || inst->cap[ROTATION].value == 270) {
+ width = inst->compose.height;
+ height = inst->compose.width;
+ }
+
+ right_offset = (inst->fmt_dst->fmt.pix_mp.width - width);
+ bottom_offset = (inst->fmt_dst->fmt.pix_mp.height - height);
+
+ crop[0] = left_offset << 16 | top_offset;
+ crop[1] = right_offset << 16 | bottom_offset;
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_CROP_OFFSETS,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(inst, OUTPUT_MPLANE),
+ HFI_PAYLOAD_64_PACKED,
+ &crop,
+ sizeof(u64));
+}
+
+static int venc_set_colorspace(struct iris_inst *inst)
+{
+ u32 video_signal_type_present_flag = 0, payload = 0;
+ u32 matrix_coeff = HFI_MATRIX_COEFF_RESERVED;
+ u32 video_format = UNSPECIFIED_COLOR_FORMAT;
+ struct v4l2_pix_format_mplane *pixmp = NULL;
+ u32 transfer_char = HFI_TRANSFER_RESERVED;
+ u32 colour_description_present_flag = 0;
+ u32 primaries = HFI_PRIMARIES_RESERVED;
+ u32 full_range = 0;
+
+ pixmp = &inst->fmt_src->fmt.pix_mp;
+ if (pixmp->colorspace != V4L2_COLORSPACE_DEFAULT ||
+ pixmp->ycbcr_enc != V4L2_YCBCR_ENC_DEFAULT ||
+ pixmp->xfer_func != V4L2_XFER_FUNC_DEFAULT) {
+ colour_description_present_flag = 1;
+ video_signal_type_present_flag = 1;
+ primaries = get_hfi_color_primaries(pixmp->colorspace);
+ matrix_coeff = get_hfi_matrix_coefficients(pixmp->ycbcr_enc);
+ transfer_char = get_hfi_transfer_char(pixmp->xfer_func);
+ }
+
+ if (pixmp->quantization != V4L2_QUANTIZATION_DEFAULT) {
+ video_signal_type_present_flag = 1;
+ full_range = pixmp->quantization ==
+ V4L2_QUANTIZATION_FULL_RANGE ? 1 : 0;
+ }
+
+ payload = (matrix_coeff & 0xFF) |
+ ((transfer_char << 8) & 0xFF00) |
+ ((primaries << 16) & 0xFF0000) |
+ ((colour_description_present_flag << 24) & 0x1000000) |
+ ((full_range << 25) & 0x2000000) |
+ ((video_format << 26) & 0x1C000000) |
+ ((video_signal_type_present_flag << 29) & 0x20000000);
+
+ return iris_hfi_set_property(inst,
+ HFI_PROP_SIGNAL_COLOR_INFO,
+ HFI_HOST_FLAGS_NONE,
+ get_hfi_port(inst, INPUT_MPLANE),
+ HFI_PAYLOAD_32_PACKED,
+ &payload,
+ sizeof(u32));
+}
+
+static int venc_set_quality_mode(struct iris_inst *inst)
+{
+ u32 mode;
+
+ mode = decide_quality_mode(inst);
+ return iris_hfi_set_property(inst,
+ HFI_PROP_QUALITY_MODE,
+ HFI_HOST_FLAGS_NONE,
+ HFI_PORT_BITSTREAM,
+ HFI_PAYLOAD_U32_ENUM,
+ &mode,
+ sizeof(u32));
+}
+
+static int venc_set_input_properties(struct iris_inst *inst)
+{
+ int j, ret;
+ static const struct venc_prop_type_handle prop_type_handle_arr[] = {
+ {HFI_PROP_COLOR_FORMAT, venc_set_colorformat },
+ {HFI_PROP_RAW_RESOLUTION, venc_set_raw_resolution },
+ {HFI_PROP_CROP_OFFSETS, venc_set_inp_crop_offsets },
+ {HFI_PROP_LINEAR_STRIDE_SCANLINE, venc_set_stride_scanline },
+ {HFI_PROP_SIGNAL_COLOR_INFO, venc_set_colorspace },
+ };
+
+ for (j = 0; j < ARRAY_SIZE(prop_type_handle_arr); j++) {
+ ret = prop_type_handle_arr[j].handle(inst);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
+static int venc_property_subscription(struct iris_inst *inst, u32 plane)
+{
+ u32 payload[32] = {0};
+ u32 payload_size = 0;
+ u32 i;
+
+ payload[0] = HFI_MODE_PROPERTY;
+ if (plane == INPUT_MPLANE) {
+ for (i = 0; i < ARRAY_SIZE(enc_input_properties); i++)
+ payload[i + 1] = enc_input_properties[i];
+ payload_size = (ARRAY_SIZE(enc_input_properties) + 1) *
+ sizeof(u32);
+ } else if (plane == OUTPUT_MPLANE) {
+ for (i = 0; i < ARRAY_SIZE(enc_output_properties); i++)
+ payload[i + 1] = enc_output_properties[i];
+ payload_size = (ARRAY_SIZE(enc_output_properties) + 1) *
+ sizeof(u32);
+ } else {
+ return -EINVAL;
+ }
+
+ return iris_hfi_session_subscribe_mode(inst,
+ HFI_CMD_SUBSCRIBE_MODE,
+ plane,
+ HFI_PAYLOAD_U32_ARRAY,
+ &payload[0],
+ payload_size);
+}
+
+int venc_streamon_input(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = check_session_supported(inst);
+ if (ret)
+ goto error;
+
+ ret = check_scaling_supported(inst);
+ if (ret)
+ goto error;
+
+ ret = venc_set_input_properties(inst);
+ if (ret)
+ goto error;
+
+ ret = iris_get_internal_buffers(inst, INPUT_MPLANE);
+ if (ret)
+ goto error;
+
+ ret = iris_destroy_internal_buffers(inst, INPUT_MPLANE);
+ if (ret)
+ goto error;
+
+ ret = iris_create_input_internal_buffers(inst);
+ if (ret)
+ goto error;
+
+ ret = iris_queue_input_internal_buffers(inst);
+ if (ret)
+ goto error;
+
+ ret = venc_property_subscription(inst, INPUT_MPLANE);
+ if (ret)
+ goto error;
+
+ ret = process_streamon_input(inst);
+ if (ret)
+ goto error;
+
+ return ret;
+
+error:
+ return ret;
+}
+
+static int venc_set_output_properties(struct iris_inst *inst)
+{
+ int j, ret;
+ static const struct venc_prop_type_handle prop_type_handle_arr[] = {
+ {HFI_PROP_BITSTREAM_RESOLUTION, venc_set_bitstream_resolution },
+ {HFI_PROP_CROP_OFFSETS, venc_set_out_crop_offsets },
+ };
+
+ for (j = 0; j < ARRAY_SIZE(prop_type_handle_arr); j++) {
+ ret = prop_type_handle_arr[j].handle(inst);
+ if (ret)
+ return ret;
+ }
+
+ return ret;
+}
+
+int venc_streamon_output(struct iris_inst *inst)
+{
+ int ret;
+
+ ret = venc_set_output_properties(inst);
+ if (ret)
+ goto error;
+
+ ret = set_v4l2_properties(inst);
+ if (ret)
+ goto error;
+
+ ret = venc_set_quality_mode(inst);
+ if (ret)
+ goto error;
+
+ ret = iris_get_internal_buffers(inst, OUTPUT_MPLANE);
+ if (ret)
+ goto error;
+
+ ret = iris_destroy_internal_buffers(inst, OUTPUT_MPLANE);
+ if (ret)
+ goto error;
+
+ ret = iris_create_output_internal_buffers(inst);
+ if (ret)
+ goto error;
+
+ ret = iris_queue_output_internal_buffers(inst);
+ if (ret)
+ goto error;
+
+ ret = venc_property_subscription(inst, OUTPUT_MPLANE);
+ if (ret)
+ goto error;
+
+ ret = process_streamon_output(inst);
+ if (ret)
+ goto error;
+
+ return ret;
+
+error:
+ session_streamoff(inst, OUTPUT_MPLANE);
+
+ return ret;
+}
diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_venc.h b/drivers/media/platform/qcom/vcodec/iris/iris_venc.h
index 24da63f..b289750 100644
--- a/drivers/media/platform/qcom/vcodec/iris/iris_venc.h
+++ b/drivers/media/platform/qcom/vcodec/iris/iris_venc.h
@@ -20,5 +20,8 @@ int venc_subscribe_event(struct iris_inst *inst,
const struct v4l2_event_subscription *sub);
int venc_start_cmd(struct iris_inst *inst);
int venc_stop_cmd(struct iris_inst *inst);
+int venc_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2);
+int venc_streamon_input(struct iris_inst *inst);
+int venc_streamon_output(struct iris_inst *inst);

#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
index effecbb..42ac662 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h
@@ -71,6 +71,11 @@ enum pipe_type {
PIPE_4 = 4,
};

+enum quality_mode {
+ MAX_QUALITY_MODE = 0x1,
+ POWER_SAVE_MODE = 0x2,
+};
+
extern struct platform_data sm8550_data;

struct bw_info {
diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
index ef0aad7..17c5856 100644
--- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
+++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
@@ -74,6 +74,8 @@ static struct plat_core_cap core_data_sm8550[] = {
{CP_SIZE, 0x25800000},
{CP_NONPIXEL_START, 0x01000000},
{CP_NONPIXEL_SIZE, 0x24800000},
+ {MAX_MBPF_HQ, 8160}, /* ((1920x1088)/256) */
+ {MAX_MBPS_HQ, 489600}, /* ((1920x1088)/256)@60fps */
};

static struct plat_inst_cap instance_cap_data_sm8550[] = {
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c
index 44f9342..9882f00 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c
@@ -163,6 +163,186 @@ static u32 dec_dpb_size_iris3(struct iris_inst *inst)
return size;
}

+static u32 enc_bin_size_iris3(struct iris_inst *inst)
+{
+ u32 width, height, num_vpp_pipes, stage, profile;
+ struct iris_core *core;
+ struct v4l2_format *f;
+ u32 size = 0;
+
+ core = inst->core;
+
+ num_vpp_pipes = core->cap[NUM_VPP_PIPE].value;
+ stage = inst->cap[STAGE].value;
+ f = inst->fmt_dst;
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+ profile = inst->cap[PROFILE].value;
+
+ if (inst->codec == H264)
+ size = hfi_buffer_bin_h264e(inst->hfi_rc_type, width, height,
+ stage, num_vpp_pipes, profile);
+ else if (inst->codec == HEVC)
+ size = hfi_buffer_bin_h265e(inst->hfi_rc_type, width, height,
+ stage, num_vpp_pipes, profile);
+
+ return size;
+}
+
+u32 get_recon_buf_count(struct iris_inst *inst)
+{
+ s32 n_bframe, ltr_count, hp_layers = 0, hb_layers = 0;
+ bool is_hybrid_hp = false;
+ u32 num_buf_recon = 0;
+ u32 hfi_codec = 0;
+
+ n_bframe = inst->cap[B_FRAME].value;
+ ltr_count = inst->cap[LTR_COUNT].value;
+
+ if (inst->hfi_layer_type == HFI_HIER_B) {
+ hb_layers = inst->cap[ENH_LAYER_COUNT].value + 1;
+ } else {
+ hp_layers = inst->cap[ENH_LAYER_COUNT].value + 1;
+ if (inst->hfi_layer_type == HFI_HIER_P_HYBRID_LTR)
+ is_hybrid_hp = true;
+ }
+
+ if (inst->codec == H264)
+ hfi_codec = HFI_CODEC_ENCODE_AVC;
+ else if (inst->codec == HEVC)
+ hfi_codec = HFI_CODEC_ENCODE_HEVC;
+
+ num_buf_recon = hfi_iris3_enc_recon_buf_count(n_bframe, ltr_count,
+ hp_layers, hb_layers,
+ is_hybrid_hp, hfi_codec);
+
+ return num_buf_recon;
+}
+
+static u32 enc_comv_size_iris3(struct iris_inst *inst)
+{
+ u32 width, height, num_recon = 0;
+ struct v4l2_format *f;
+ u32 size = 0;
+
+ f = inst->fmt_dst;
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+
+ num_recon = get_recon_buf_count(inst);
+ if (inst->codec == H264)
+ size = hfi_buffer_comv_h264e(width, height, num_recon);
+ else if (inst->codec == HEVC)
+ size = hfi_buffer_comv_h265e(width, height, num_recon);
+
+ return size;
+}
+
+static u32 enc_non_comv_size_iris3(struct iris_inst *inst)
+{
+ u32 width, height, num_vpp_pipes;
+ struct iris_core *core;
+ struct v4l2_format *f;
+ u32 size = 0;
+
+ core = inst->core;
+
+ num_vpp_pipes = core->cap[NUM_VPP_PIPE].value;
+ f = inst->fmt_dst;
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+
+ if (inst->codec == H264)
+ size = hfi_buffer_non_comv_h264e(width, height, num_vpp_pipes);
+ else if (inst->codec == HEVC)
+ size = hfi_buffer_non_comv_h265e(width, height, num_vpp_pipes);
+
+ return size;
+}
+
+static u32 enc_line_size_iris3(struct iris_inst *inst)
+{
+ u32 width, height, pixfmt, num_vpp_pipes;
+ struct iris_core *core;
+ bool is_tenbit = false;
+ struct v4l2_format *f;
+ u32 size = 0;
+
+ core = inst->core;
+ num_vpp_pipes = core->cap[NUM_VPP_PIPE].value;
+ pixfmt = inst->cap[PIX_FMTS].value;
+
+ f = inst->fmt_dst;
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+ is_tenbit = (pixfmt == FMT_TP10C);
+
+ if (inst->codec == H264)
+ size = hfi_buffer_line_h264e(width, height, is_tenbit, num_vpp_pipes);
+ else if (inst->codec == HEVC)
+ size = hfi_buffer_line_h265e(width, height, is_tenbit, num_vpp_pipes);
+
+ return size;
+}
+
+static u32 enc_dpb_size_iris3(struct iris_inst *inst)
+{
+ u32 width, height, pixfmt;
+ struct v4l2_format *f;
+ bool is_tenbit;
+ u32 size = 0;
+
+ f = inst->fmt_dst;
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+
+ pixfmt = inst->cap[PIX_FMTS].value;
+ is_tenbit = (pixfmt == FMT_TP10C);
+
+ if (inst->codec == H264)
+ size = hfi_buffer_dpb_h264e(width, height);
+ else if (inst->codec == HEVC)
+ size = hfi_buffer_dpb_h265e(width, height, is_tenbit);
+
+ return size;
+}
+
+static u32 enc_arp_size_iris3(struct iris_inst *inst)
+{
+ u32 size = 0;
+
+ HFI_BUFFER_ARP_ENC(size);
+
+ return size;
+}
+
+static u32 enc_vpss_size_iris3(struct iris_inst *inst)
+{
+ bool ds_enable = false, is_tenbit = false;
+ struct v4l2_format *f;
+ u32 width, height;
+ u32 size = 0;
+
+ ds_enable = is_scaling_enabled(inst);
+
+ f = inst->fmt_dst;
+ if (inst->cap[ROTATION].value == 90 ||
+ inst->cap[ROTATION].value == 270) {
+ width = f->fmt.pix_mp.height;
+ height = f->fmt.pix_mp.width;
+ } else {
+ width = f->fmt.pix_mp.width;
+ height = f->fmt.pix_mp.height;
+ }
+
+ f = inst->fmt_src;
+ is_tenbit = is_10bit_colorformat(f->fmt.pix_mp.pixelformat);
+
+ size = hfi_buffer_vpss_enc(width, height, ds_enable, 0, is_tenbit);
+
+ return size;
+}
+
struct iris_buf_type_handle {
enum iris_buffer_type type;
u32 (*handle)(struct iris_inst *inst);
@@ -183,9 +363,23 @@ int iris_int_buf_size_iris3(struct iris_inst *inst,
{BUF_PERSIST, dec_persist_size_iris3 },
{BUF_DPB, dec_dpb_size_iris3 },
};
+ static const struct iris_buf_type_handle enc_internal_buf_type_handle[] = {
+ {BUF_BIN, enc_bin_size_iris3 },
+ {BUF_COMV, enc_comv_size_iris3 },
+ {BUF_NON_COMV, enc_non_comv_size_iris3 },
+ {BUF_LINE, enc_line_size_iris3 },
+ {BUF_DPB, enc_dpb_size_iris3 },
+ {BUF_ARP, enc_arp_size_iris3 },
+ {BUF_VPSS, enc_vpss_size_iris3 },
+ };

- buf_type_handle_size = ARRAY_SIZE(dec_internal_buf_type_handle);
- buf_type_handle_arr = dec_internal_buf_type_handle;
+ if (inst->domain == DECODER) {
+ buf_type_handle_size = ARRAY_SIZE(dec_internal_buf_type_handle);
+ buf_type_handle_arr = dec_internal_buf_type_handle;
+ } else if (inst->domain == ENCODER) {
+ buf_type_handle_size = ARRAY_SIZE(enc_internal_buf_type_handle);
+ buf_type_handle_arr = enc_internal_buf_type_handle;
+ }

if (!buf_type_handle_arr || !buf_type_handle_size)
return size;
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h
index b520c79..c63b3ac 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h
@@ -14,8 +14,10 @@

#define DMA_ALIGNMENT 256

+#define BUFFER_ALIGNMENT_4096_BYTES 4096
#define BUFFER_ALIGNMENT_512_BYTES 512
#define BUFFER_ALIGNMENT_256_BYTES 256
+#define BUFFER_ALIGNMENT_128_BYTES 128
#define BUFFER_ALIGNMENT_64_BYTES 64
#define BUFFER_ALIGNMENT_32_BYTES 32
#define BUFFER_ALIGNMENT_16_BYTES 16
@@ -26,6 +28,8 @@
#define HFI_COL_FMT_NV12C_Y_TILE_WIDTH (32)
#define HFI_COL_FMT_NV12C_UV_TILE_HEIGHT (8)
#define HFI_COL_FMT_NV12C_UV_TILE_WIDTH (16)
+#define HFI_COL_FMT_TP10C_Y_TILE_HEIGHT (4)
+#define HFI_COL_FMT_TP10C_Y_TILE_WIDTH (48)

#define NUM_HW_PIC_BUF 32
#define SIZE_HW_PIC(size_per_buf) (NUM_HW_PIC_BUF * (size_per_buf))
@@ -102,6 +106,27 @@
#define MAX_PE_NBR_DATA_LCU64_LINE_BUFFER_SIZE (64 * 2 * 3)
#define MAX_FE_NBR_DATA_LUMA_LINE_BUFFER_SIZE 640

+#define HFI_BUFFER_ARP_ENC(size) (size = 204800)
+
+#define HFI_MAX_COL_FRAME 6
+#define HFI_VENUS_WIDTH_ALIGNMENT 128
+#define HFI_VENUS_HEIGHT_ALIGNMENT 32
+#define VENUS_METADATA_STRIDE_MULTIPLE 64
+#define VENUS_METADATA_HEIGHT_MULTIPLE 16
+
+#ifndef SYSTEM_LAL_TILE10
+#define SYSTEM_LAL_TILE10 192
+#endif
+
+#define SIZE_SLICE_CMD_BUFFER (ALIGN(20480, BUFFER_ALIGNMENT_256_BYTES))
+#define SIZE_SPS_PPS_SLICE_HDR (2048 + 4096)
+
+#define SIZE_BSE_SLICE_CMD_BUF ((((8192 << 2) + 7) & (~7)) * 3)
+#define SIZE_LAMBDA_LUT (256 * 11)
+
+#define HFI_WORKMODE_1 1
+#define HFI_WORKMODE_2 2
+
static inline
u32 size_h264d_hw_bin_buffer(u32 frame_width, u32 frame_height,
u32 num_vpp_pipes)
@@ -842,4 +867,603 @@ u32 hfi_yuv420_tp10_ubwc_calc_buf_size(u32 y_stride, u32 y_buf_height,
int iris_int_buf_size_iris3(struct iris_inst *inst,
enum iris_buffer_type buffer_type);

+static inline
+u32 hfi_iris3_enc_recon_buf_count(s32 n_bframe, s32 ltr_count,
+ s32 _total_hp_layers,
+ s32 _total_hb_layers,
+ bool hybrid_hp,
+ u32 codec_standard)
+{
+ u32 num_ref = 1;
+
+ if (n_bframe)
+ num_ref = 2;
+
+ if (_total_hp_layers > 1) {
+ if (hybrid_hp)
+ num_ref = (_total_hp_layers + 1) >> 1;
+ else if (codec_standard == HFI_CODEC_ENCODE_HEVC)
+ num_ref = (_total_hp_layers + 1) >> 1;
+ else if (codec_standard == HFI_CODEC_ENCODE_AVC &&
+ _total_hp_layers < 4)
+ num_ref = (_total_hp_layers - 1);
+ else
+ num_ref = _total_hp_layers;
+ }
+
+ if (ltr_count)
+ num_ref = num_ref + ltr_count;
+
+ if (_total_hb_layers > 1) {
+ if (codec_standard == HFI_CODEC_ENCODE_HEVC)
+ num_ref = (_total_hb_layers);
+ else if (codec_standard == HFI_CODEC_ENCODE_AVC)
+ num_ref = (1 << (_total_hb_layers - 2)) + 1;
+ }
+
+ return num_ref + 1;
+}
+
+static inline
+u32 size_bin_bitstream_enc(u32 rc_type, u32 frame_width, u32 frame_height,
+ u32 work_mode, u32 lcu_size, u32 profile)
+{
+ u32 size_aligned_width = 0, size_aligned_height = 0;
+ u32 bitstream_size_eval = 0;
+
+ size_aligned_width = ALIGN((frame_width), lcu_size);
+ size_aligned_height = ALIGN((frame_height), lcu_size);
+ if (work_mode == HFI_WORKMODE_2) {
+ if (rc_type == HFI_RC_CQ || rc_type == HFI_RC_OFF) {
+ bitstream_size_eval = (((size_aligned_width) *
+ (size_aligned_height) * 3) >> 1);
+ } else {
+ bitstream_size_eval = ((size_aligned_width) *
+ (size_aligned_height) * 3);
+ if ((size_aligned_width * size_aligned_height) > (4096 * 2176))
+ bitstream_size_eval >>= 3;
+ else if ((size_aligned_width * size_aligned_height) > (480 * 320))
+ bitstream_size_eval >>= 2;
+
+ if (profile == HFI_H265_PROFILE_MAIN_10 ||
+ profile == HFI_H265_PROFILE_MAIN_10_STILL_PICTURE)
+ bitstream_size_eval = (bitstream_size_eval * 5 >> 2);
+ }
+ } else {
+ bitstream_size_eval = size_aligned_width * size_aligned_height * 3;
+ }
+
+ return ALIGN(bitstream_size_eval, BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline
+u32 size_enc_single_pipe(u32 rc_type, u32 bitbin_size, u32 num_vpp_pipes,
+ u32 frame_width, u32 frame_height, u32 lcu_size)
+{
+ u32 size_single_pipe_eval = 0, sao_bin_buffer_size = 0;
+ u32 size_aligned_width = 0, size_aligned_height = 0;
+ u32 _padded_bin_sz = 0;
+
+ size_aligned_width = ALIGN((frame_width), lcu_size);
+ size_aligned_height = ALIGN((frame_height), lcu_size);
+ if ((size_aligned_width * size_aligned_height) > (3840 * 2160))
+ size_single_pipe_eval = (bitbin_size / num_vpp_pipes);
+ else if (num_vpp_pipes > 2)
+ size_single_pipe_eval = bitbin_size / 2;
+ else
+ size_single_pipe_eval = bitbin_size;
+
+ sao_bin_buffer_size =
+ (64 * ((((frame_width) + BUFFER_ALIGNMENT_32_BYTES) *
+ ((frame_height) + BUFFER_ALIGNMENT_32_BYTES)) >> 10)) + 384;
+ _padded_bin_sz = ALIGN(size_single_pipe_eval, BUFFER_ALIGNMENT_256_BYTES);
+ size_single_pipe_eval = sao_bin_buffer_size + _padded_bin_sz;
+ size_single_pipe_eval = ALIGN(size_single_pipe_eval, BUFFER_ALIGNMENT_256_BYTES);
+
+ return size_single_pipe_eval;
+}
+
+static inline
+u32 hfi_buffer_bin_enc(u32 rc_type, u32 frame_width, u32 frame_height, u32 lcu_size,
+ u32 work_mode, u32 num_vpp_pipes, u32 profile)
+{
+ u32 bitstream_size = 0, total_bitbin_buffers = 0;
+ u32 size_single_pipe = 0, bitbin_size = 0;
+ u32 _size = 0;
+
+ bitstream_size = size_bin_bitstream_enc(rc_type, frame_width,
+ frame_height, work_mode,
+ lcu_size, profile);
+
+ if (work_mode == HFI_WORKMODE_2) {
+ total_bitbin_buffers = 3;
+ bitbin_size = bitstream_size * 12 / 10;
+ bitbin_size = ALIGN(bitbin_size, BUFFER_ALIGNMENT_256_BYTES);
+ } else if ((lcu_size == 16) || (num_vpp_pipes > 1)) {
+ total_bitbin_buffers = 1;
+ bitbin_size = bitstream_size;
+ }
+
+ if (total_bitbin_buffers > 0) {
+ size_single_pipe = size_enc_single_pipe(rc_type, bitbin_size,
+ num_vpp_pipes, frame_width,
+ frame_height, lcu_size);
+ bitbin_size = size_single_pipe * num_vpp_pipes;
+ _size = ALIGN(bitbin_size, BUFFER_ALIGNMENT_256_BYTES) *
+ total_bitbin_buffers + 512;
+ } else {
+ /* Avoid 512 Bytes allocation in case of 1Pipe HEVC Direct Mode*/
+ _size = 0;
+ }
+
+ return _size;
+}
+
+static inline
+u32 hfi_buffer_bin_h264e(u32 rc_type, u32 frame_width, u32 frame_height,
+ u32 work_mode, u32 num_vpp_pipes, u32 profile)
+{
+ return hfi_buffer_bin_enc(rc_type, frame_width, frame_height, 16,
+ work_mode, num_vpp_pipes, profile);
+}
+
+static inline
+u32 hfi_buffer_bin_h265e(u32 rc_type, u32 frame_width, u32 frame_height,
+ u32 work_mode, u32 num_vpp_pipes, u32 profile)
+{
+ return hfi_buffer_bin_enc(rc_type, frame_width, frame_height, 32,
+ work_mode, num_vpp_pipes, profile);
+}
+
+static inline
+u32 size_enc_slice_info_buf(u32 num_lcu_in_frame)
+{
+ return ALIGN((256 + (num_lcu_in_frame << 4)), BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline
+u32 size_line_buf_ctrl(u32 frame_width_coded)
+{
+ return ALIGN(frame_width_coded, BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline
+u32 size_line_buf_ctrl_id2(u32 frame_width_coded)
+{
+ return ALIGN(frame_width_coded, BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline
+u32 size_linebuff_data(bool is_ten_bit, u32 frame_width_coded)
+{
+ u32 _size = 0;
+
+ _size = is_ten_bit ?
+ (((((10 * (frame_width_coded) + 1024) + (BUFFER_ALIGNMENT_256_BYTES - 1)) &
+ (~(BUFFER_ALIGNMENT_256_BYTES - 1))) * 1) +
+ (((((10 * (frame_width_coded) + 1024) >> 1) + (BUFFER_ALIGNMENT_256_BYTES - 1)) &
+ (~(BUFFER_ALIGNMENT_256_BYTES - 1))) * 2)) :
+ (((((8 * (frame_width_coded) + 1024) + (BUFFER_ALIGNMENT_256_BYTES - 1)) &
+ (~(BUFFER_ALIGNMENT_256_BYTES - 1))) * 1) +
+ (((((8 * (frame_width_coded) + 1024) >> 1) + (BUFFER_ALIGNMENT_256_BYTES - 1)) &
+ (~(BUFFER_ALIGNMENT_256_BYTES - 1))) * 2));
+
+ return _size;
+}
+
+static inline
+u32 size_left_linebuff_ctrl(u32 standard, u32 frame_height_coded,
+ u32 num_vpp_pipes_enc)
+{
+ u32 _size = 0;
+
+ _size = standard == HFI_CODEC_ENCODE_HEVC ?
+ (((frame_height_coded) +
+ (BUFFER_ALIGNMENT_32_BYTES)) / BUFFER_ALIGNMENT_32_BYTES * 4 * 16) :
+ (((frame_height_coded) + 15) / 16 * 5 * 16);
+
+ if ((num_vpp_pipes_enc) > 1) {
+ _size += BUFFER_ALIGNMENT_512_BYTES;
+ _size = ALIGN(_size, BUFFER_ALIGNMENT_512_BYTES) *
+ num_vpp_pipes_enc;
+ }
+
+ return ALIGN(_size, BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline
+u32 size_left_linebuff_recon_pix(bool is_ten_bit, u32 frame_height_coded,
+ u32 num_vpp_pipes_enc)
+{
+ return (((is_ten_bit + 1) * 2 * (frame_height_coded) + BUFFER_ALIGNMENT_256_BYTES) +
+ (BUFFER_ALIGNMENT_256_BYTES << (num_vpp_pipes_enc - 1)) - 1) &
+ (~((BUFFER_ALIGNMENT_256_BYTES << (num_vpp_pipes_enc - 1)) - 1)) * 1;
+}
+
+static inline
+u32 size_top_linebuff_ctrl_fe(u32 frame_width_coded, u32 standard)
+{
+ u32 _size = 0;
+
+ _size = standard == HFI_CODEC_ENCODE_HEVC ?
+ (64 * ((frame_width_coded) >> 5)) :
+ (BUFFER_ALIGNMENT_256_BYTES + 16 * ((frame_width_coded) >> 4));
+
+ return ALIGN(_size, BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline
+u32 size_left_linebuff_ctrl_fe(u32 frame_height_coded, u32 num_vpp_pipes_enc)
+{
+ return (((BUFFER_ALIGNMENT_256_BYTES + 64 * ((frame_height_coded) >> 4)) +
+ (BUFFER_ALIGNMENT_256_BYTES << (num_vpp_pipes_enc - 1)) - 1) &
+ (~((BUFFER_ALIGNMENT_256_BYTES << (num_vpp_pipes_enc - 1)) - 1)) * 1) *
+ num_vpp_pipes_enc;
+}
+
+static inline
+u32 size_left_linebuff_metadata_recon_y(u32 frame_height_coded,
+ bool is_ten_bit,
+ u32 num_vpp_pipes_enc)
+{
+ u32 _size = 0;
+
+ _size = ((BUFFER_ALIGNMENT_256_BYTES + 64 * ((frame_height_coded) /
+ (8 * (is_ten_bit ? 4 : 8)))));
+ _size = ALIGN(_size, BUFFER_ALIGNMENT_256_BYTES);
+
+ return (_size * num_vpp_pipes_enc);
+}
+
+static inline
+u32 size_left_linebuff_metadata_recon_uv(u32 frame_height_coded,
+ bool is_ten_bit,
+ u32 num_vpp_pipes_enc)
+{
+ u32 _size = 0;
+
+ _size = ((BUFFER_ALIGNMENT_256_BYTES + 64 * ((frame_height_coded) /
+ (4 * (is_ten_bit ? 4 : 8)))));
+ _size = ALIGN(_size, BUFFER_ALIGNMENT_256_BYTES);
+
+ return (_size * num_vpp_pipes_enc);
+}
+
+static inline
+u32 size_linebuff_recon_pix(bool is_ten_bit, u32 frame_width_coded)
+{
+ return ALIGN(((is_ten_bit ? 3 : 2) * (frame_width_coded)),
+ BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline
+u32 size_frame_rc_buf_size(u32 standard, u32 frame_height_coded,
+ u32 num_vpp_pipes_enc)
+{
+ u32 _size = 0;
+
+ _size = (standard == HFI_CODEC_ENCODE_HEVC) ?
+ (256 + 16 * (14 + ((((frame_height_coded) >> 5) + 7) >> 3))) :
+ (256 + 16 * (14 + ((((frame_height_coded) >> 4) + 7) >> 3)));
+ _size *= 11;
+ if (num_vpp_pipes_enc > 1)
+ _size = ALIGN(_size, BUFFER_ALIGNMENT_256_BYTES) * num_vpp_pipes_enc;
+
+ return ALIGN(_size, BUFFER_ALIGNMENT_512_BYTES) * HFI_MAX_COL_FRAME;
+}
+
+static inline u32 enc_bitcnt_buf_size(u32 num_lcu_in_frame)
+{
+ return ALIGN((256 + (4 * (num_lcu_in_frame))), BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline u32 enc_bitmap_buf_size(u32 num_lcu_in_frame)
+{
+ return ALIGN((256 + ((num_lcu_in_frame) >> 3)), BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline u32 size_line_buf_sde(u32 frame_width_coded)
+{
+ return ALIGN((256 + (16 * ((frame_width_coded) >> 4))), BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline u32 size_override_buf(u32 num_lcumb)
+{
+ return ALIGN(((16 * (((num_lcumb) + 7) >> 3))), BUFFER_ALIGNMENT_256_BYTES) * 2;
+}
+
+static inline u32 size_ir_buf(u32 num_lcu_in_frame)
+{
+ return ALIGN((((((num_lcu_in_frame) << 1) + 7) & (~7)) * 3), BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline
+u32 size_vpss_line_buf(u32 num_vpp_pipes_enc, u32 frame_height_coded,
+ u32 frame_width_coded)
+{
+ return ALIGN(((((((8192) >> 2) << 5) * (num_vpp_pipes_enc)) + 64) +
+ (((((max_t(u32, (frame_width_coded),
+ (frame_height_coded)) + 3) >> 2) << 5) + 256) * 16)),
+ BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline
+u32 size_top_line_buf_first_stg_sao(u32 frame_width_coded)
+{
+ return ALIGN((16 * ((frame_width_coded) >> 5)), BUFFER_ALIGNMENT_256_BYTES);
+}
+
+static inline
+u32 hfi_buffer_line_enc(u32 frame_width, u32 frame_height, bool is_ten_bit,
+ u32 num_vpp_pipes_enc, u32 lcu_size, u32 standard)
+{
+ u32 line_buff_data_size = 0, left_line_buff_ctrl_size = 0;
+ u32 frame_width_coded = 0, frame_height_coded = 0;
+ u32 left_line_buff_metadata_recon__uv__size = 0;
+ u32 left_line_buff_metadata_recon__y__size = 0;
+ u32 width_in_lcus = 0, height_in_lcus = 0;
+ u32 left_line_buff_recon_pix_size = 0;
+ u32 top_line_buff_ctrl_fe_size = 0;
+ u32 line_buff_recon_pix_size = 0;
+
+ width_in_lcus = ((frame_width) + (lcu_size) - 1) / (lcu_size);
+ height_in_lcus = ((frame_height) + (lcu_size) - 1) / (lcu_size);
+ frame_width_coded = width_in_lcus * (lcu_size);
+ frame_height_coded = height_in_lcus * (lcu_size);
+ line_buff_data_size = size_linebuff_data(is_ten_bit, frame_width_coded);
+ left_line_buff_ctrl_size =
+ size_left_linebuff_ctrl(standard, frame_height_coded, num_vpp_pipes_enc);
+ left_line_buff_recon_pix_size =
+ size_left_linebuff_recon_pix(is_ten_bit, frame_height_coded,
+ num_vpp_pipes_enc);
+ top_line_buff_ctrl_fe_size =
+ size_top_linebuff_ctrl_fe(frame_width_coded, standard);
+ left_line_buff_metadata_recon__y__size =
+ size_left_linebuff_metadata_recon_y(frame_height_coded, is_ten_bit,
+ num_vpp_pipes_enc);
+ left_line_buff_metadata_recon__uv__size =
+ size_left_linebuff_metadata_recon_uv(frame_height_coded, is_ten_bit,
+ num_vpp_pipes_enc);
+ line_buff_recon_pix_size = size_linebuff_recon_pix(is_ten_bit, frame_width_coded);
+
+ return size_line_buf_ctrl(frame_width_coded) +
+ size_line_buf_ctrl_id2(frame_width_coded) +
+ line_buff_data_size +
+ left_line_buff_ctrl_size +
+ left_line_buff_recon_pix_size +
+ top_line_buff_ctrl_fe_size +
+ left_line_buff_metadata_recon__y__size +
+ left_line_buff_metadata_recon__uv__size +
+ line_buff_recon_pix_size +
+ size_left_linebuff_ctrl_fe(frame_height_coded,
+ num_vpp_pipes_enc) +
+ size_line_buf_sde(frame_width_coded) +
+ size_vpss_line_buf(num_vpp_pipes_enc, frame_height_coded,
+ frame_width_coded) +
+ size_top_line_buf_first_stg_sao(frame_width_coded);
+}
+
+static inline
+u32 hfi_buffer_line_h264e(u32 frame_width, u32 frame_height, bool is_ten_bit,
+ u32 num_vpp_pipes)
+{
+ return hfi_buffer_line_enc(frame_width, frame_height, 0,
+ num_vpp_pipes, 16,
+ HFI_CODEC_ENCODE_AVC);
+}
+
+static inline
+u32 hfi_buffer_line_h265e(u32 frame_width, u32 frame_height, bool is_ten_bit,
+ u32 num_vpp_pipes)
+{
+ return hfi_buffer_line_enc(frame_width, frame_height,
+ is_ten_bit, num_vpp_pipes, 32,
+ HFI_CODEC_ENCODE_HEVC);
+}
+
+static inline
+u32 hfi_buffer_comv_enc(u32 frame_width, u32 frame_height, u32 lcu_size,
+ u32 num_recon, u32 standard)
+{
+ u32 width_in_lcus, height_in_lcus, num_lcu_in_frame;
+ u32 size_colloc_mv = 0, size_colloc_rc = 0;
+ u32 mb_height = ((frame_height) + 15) >> 4;
+ u32 mb_width = ((frame_width) + 15) >> 4;
+
+ width_in_lcus = ((frame_width) + (lcu_size) - 1) / (lcu_size);
+ height_in_lcus = ((frame_height) + (lcu_size) - 1) / (lcu_size);
+ num_lcu_in_frame = width_in_lcus * height_in_lcus;
+ size_colloc_mv = (standard == HFI_CODEC_ENCODE_HEVC) ?
+ (16 * ((num_lcu_in_frame << 2) + BUFFER_ALIGNMENT_32_BYTES)) :
+ (3 * 16 * (width_in_lcus * height_in_lcus + BUFFER_ALIGNMENT_32_BYTES));
+ size_colloc_mv = ALIGN(size_colloc_mv, BUFFER_ALIGNMENT_256_BYTES) * num_recon;
+ size_colloc_rc = (((mb_width + 7) >> 3) * 16 * 2 * mb_height);
+ size_colloc_rc = ALIGN(size_colloc_rc, BUFFER_ALIGNMENT_256_BYTES) * HFI_MAX_COL_FRAME;
+
+ return size_colloc_mv + size_colloc_rc;
+}
+
+static inline
+u32 hfi_buffer_comv_h264e(u32 frame_width, u32 frame_height, u32 num_recon)
+{
+ return hfi_buffer_comv_enc(frame_width, frame_height, 16,
+ num_recon, HFI_CODEC_ENCODE_AVC);
+}
+
+static inline
+u32 hfi_buffer_comv_h265e(u32 frame_width, u32 frame_height, u32 num_recon)
+{
+ return hfi_buffer_comv_enc(frame_width, frame_height, 32,
+ num_recon, HFI_CODEC_ENCODE_HEVC);
+}
+
+static inline
+u32 hfi_buffer_non_comv_enc(u32 frame_width, u32 frame_height,
+ u32 num_vpp_pipes_enc, u32 lcu_size, u32 standard)
+{
+ u32 frame_width_coded = 0, frame_height_coded = 0;
+ u32 width_in_lcus = 0, height_in_lcus = 0;
+ u32 num_lcu_in_frame = 0, num_lcumb = 0;
+ u32 frame_rc_buf_size = 0;
+
+ width_in_lcus = ((frame_width) + (lcu_size) - 1) / (lcu_size);
+ height_in_lcus = ((frame_height) + (lcu_size) - 1) / (lcu_size);
+ num_lcu_in_frame = width_in_lcus * height_in_lcus;
+ frame_width_coded = width_in_lcus * (lcu_size);
+ frame_height_coded = height_in_lcus * (lcu_size);
+ num_lcumb = (frame_height_coded / lcu_size) *
+ ((frame_width_coded + lcu_size * 8) / lcu_size);
+ frame_rc_buf_size =
+ size_frame_rc_buf_size(standard, frame_height_coded,
+ num_vpp_pipes_enc);
+ return size_enc_slice_info_buf(num_lcu_in_frame) +
+ SIZE_SLICE_CMD_BUFFER +
+ SIZE_SPS_PPS_SLICE_HDR +
+ frame_rc_buf_size +
+ enc_bitcnt_buf_size(num_lcu_in_frame) +
+ enc_bitmap_buf_size(num_lcu_in_frame) +
+ SIZE_BSE_SLICE_CMD_BUF +
+ SIZE_LAMBDA_LUT +
+ size_override_buf(num_lcumb) +
+ size_ir_buf(num_lcu_in_frame);
+}
+
+static inline
+u32 hfi_buffer_non_comv_h264e(u32 frame_width, u32 frame_height,
+ u32 num_vpp_pipes_enc)
+{
+ return hfi_buffer_non_comv_enc(frame_width, frame_height,
+ num_vpp_pipes_enc, 16,
+ HFI_CODEC_ENCODE_AVC);
+}
+
+static inline
+u32 hfi_buffer_non_comv_h265e(u32 frame_width, u32 frame_height,
+ u32 num_vpp_pipes_enc)
+{
+ return hfi_buffer_non_comv_enc(frame_width, frame_height,
+ num_vpp_pipes_enc, 32,
+ HFI_CODEC_ENCODE_HEVC);
+}
+
+static inline
+u32 size_enc_ref_buffer(u32 frame_width, u32 frame_height)
+{
+ u32 u_buffer_width = 0, u_buffer_height = 0;
+ u32 u_chroma_buffer_height = 0;
+
+ u_buffer_height = ALIGN(frame_height, HFI_VENUS_HEIGHT_ALIGNMENT);
+ u_chroma_buffer_height = frame_height >> 1;
+ u_chroma_buffer_height = ALIGN(u_chroma_buffer_height, HFI_VENUS_HEIGHT_ALIGNMENT);
+ u_buffer_width = ALIGN(frame_width, HFI_VENUS_WIDTH_ALIGNMENT);
+
+ return (u_buffer_height + u_chroma_buffer_height) * u_buffer_width;
+}
+
+static inline
+u32 size_enc_ten_bit_ref_buffer(u32 frame_width, u32 frame_height)
+{
+ u32 ref_buf_height = 0, ref_luma_stride_in_bytes = 0;
+ u32 chroma_size = 0, ref_buf_size = 0;
+ u32 u_ref_stride = 0, luma_size = 0;
+ u32 ref_chrm_height_in_bytes = 0;
+
+ ref_buf_height = (frame_height + (HFI_VENUS_HEIGHT_ALIGNMENT - 1)) &
+ (~(HFI_VENUS_HEIGHT_ALIGNMENT - 1));
+ ref_luma_stride_in_bytes = ((frame_width + SYSTEM_LAL_TILE10 - 1) / SYSTEM_LAL_TILE10) *
+ SYSTEM_LAL_TILE10;
+ u_ref_stride = 4 * (ref_luma_stride_in_bytes / 3);
+ u_ref_stride = (u_ref_stride + (BUFFER_ALIGNMENT_128_BYTES - 1)) &
+ (~(BUFFER_ALIGNMENT_128_BYTES - 1));
+ luma_size = ref_buf_height * u_ref_stride;
+ ref_chrm_height_in_bytes = (((frame_height + 1) >> 1) + (BUFFER_ALIGNMENT_32_BYTES - 1)) &
+ (~(BUFFER_ALIGNMENT_32_BYTES - 1));
+ chroma_size = u_ref_stride * ref_chrm_height_in_bytes;
+ luma_size = (luma_size + (BUFFER_ALIGNMENT_4096_BYTES - 1)) &
+ (~(BUFFER_ALIGNMENT_4096_BYTES - 1));
+ chroma_size = (chroma_size + (BUFFER_ALIGNMENT_4096_BYTES - 1)) &
+ (~(BUFFER_ALIGNMENT_4096_BYTES - 1));
+ ref_buf_size = luma_size + chroma_size;
+
+ return ref_buf_size;
+}
+
+static inline
+u32 hfi_buffer_dpb_enc(u32 frame_width, u32 frame_height, bool is_ten_bit)
+{
+ u32 metadata_stride, metadata_buf_height, meta_size_y, meta_size_c;
+ u32 ten_bit_ref_buf_size = 0, ref_buf_size = 0;
+ u32 _size = 0;
+
+ if (!is_ten_bit) {
+ ref_buf_size = size_enc_ref_buffer(frame_width, frame_height);
+ metadata_stride =
+ hfi_ubwc_calc_metadata_plane_stride(frame_width, 64,
+ HFI_COL_FMT_NV12C_Y_TILE_WIDTH);
+ metadata_buf_height =
+ hfi_ubwc_metadata_plane_bufheight(frame_height, 16,
+ HFI_COL_FMT_NV12C_Y_TILE_HEIGHT);
+ meta_size_y =
+ hfi_ubwc_metadata_plane_buffer_size(metadata_stride, metadata_buf_height);
+ meta_size_c =
+ hfi_ubwc_metadata_plane_buffer_size(metadata_stride, metadata_buf_height);
+ _size = ref_buf_size + meta_size_y + meta_size_c;
+ } else {
+ ten_bit_ref_buf_size = size_enc_ten_bit_ref_buffer(frame_width, frame_height);
+ metadata_stride =
+ hfi_ubwc_calc_metadata_plane_stride(frame_width,
+ VENUS_METADATA_STRIDE_MULTIPLE,
+ HFI_COL_FMT_TP10C_Y_TILE_WIDTH);
+ metadata_buf_height =
+ hfi_ubwc_metadata_plane_bufheight(frame_height,
+ VENUS_METADATA_HEIGHT_MULTIPLE,
+ HFI_COL_FMT_TP10C_Y_TILE_HEIGHT);
+ meta_size_y =
+ hfi_ubwc_metadata_plane_buffer_size(metadata_stride, metadata_buf_height);
+ meta_size_c =
+ hfi_ubwc_metadata_plane_buffer_size(metadata_stride, metadata_buf_height);
+ _size = ten_bit_ref_buf_size + meta_size_y + meta_size_c;
+ }
+
+ return _size;
+}
+
+static inline
+u32 hfi_buffer_dpb_h264e(u32 frame_width, u32 frame_height)
+{
+ return hfi_buffer_dpb_enc(frame_width, frame_height, 0);
+}
+
+static inline
+u32 hfi_buffer_dpb_h265e(u32 frame_width, u32 frame_height, bool is_ten_bit)
+{
+ return hfi_buffer_dpb_enc(frame_width, frame_height, is_ten_bit);
+}
+
+static inline
+u32 hfi_buffer_vpss_enc(u32 dswidth, u32 dsheight, bool ds_enable,
+ u32 blur, bool is_ten_bit)
+{
+ u32 vpss_size = 0;
+
+ if (ds_enable || blur)
+ vpss_size = hfi_buffer_dpb_enc(dswidth, dsheight, is_ten_bit);
+
+ return vpss_size;
+}
+
+static inline
+u32 hfi_iris3_enc_min_input_buf_count(u32 totalhblayers)
+{
+ u32 numinput = 3;
+
+ if (totalhblayers >= 2)
+ numinput = (1 << (totalhblayers - 1)) + 2;
+
+ return numinput;
+}
+
+u32 enc_output_buffer_size_iris3(struct iris_inst *inst);
+u32 get_recon_buf_count(struct iris_inst *inst);
+
#endif
diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
index 58498af..2828227 100644
--- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
+++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
@@ -54,7 +54,7 @@ u64 iris_calc_freq_iris3(struct iris_inst *inst, u32 data_size)
if (inst->cap[STAGE].value == STAGE_1)
vsp_cycles = vsp_cycles * 3;

- vsp_cycles += mbs_per_second * base_cycles;
+ vsp_cycles += mbs_per_second * base_cycles;

freq = max3(vpp_cycles, vsp_cycles, fw_cycles);

--
2.7.4

2023-12-18 12:13:36

[permalink] [raw]

Subject: Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver

On 12/18/2023 5:01 PM, Dikshita Agarwal wrote:
> This patch series introduces support for Qualcomm new video acceleration
> hardware architecture, used for video stream decoding/encoding. This driver
> is based on new communication protocol between video hardware and application
> processor.
> This driver comes with below capabilities:
> - V4L2 complaint video driver with M2M and STREAMING capability.
> - Supports H264, H265, VP9 decoders.
> - Supports H264, H265 encoders.
> This driver comes with below features:
> - Centralized resource and memory management.
> - Centralized management of core and instance states.
> - Defines platform specific capabilities and features. As a results, it provides
> a single point of control to enable/disable a given feature depending on
> specific platform capabilities.
> - Handles hardware interdependent configurations. For a given configuration from
> client, the driver checks for hardware dependent configuration/s and updates
> the same.
> - Handles multiple complex video scenarios involving state transitions - DRC,
> Drain, Seek, back to back DRC, DRC during Drain sequence, DRC during Seek
> sequence.
> - Introduces a flexible way for driver to subscribe for any property with
> hardware. Hardware would inform driver with those subscribed property with any
> change in value.
> - Introduces performance (clock and bus) model based on new hardware
> architecture.
> - Introduces multi thread safe design to handle communication between client and
> hardware.
> - Adapts encoder quality improvements available in new hardware architecture.
> - Implements asynchronous communication with hardware to achieve better
> experience in low latency usecases.
> - Supports multi stage hardware architecture for encode/decode.
> - Output and capture planes are controlled independently. Thereby providing a
> way to reconfigure individual plane.
> - Hardware packetization layer supports synchronization between configuration
> packet and data packet.
> - Introduces a flexibility to receive a hardware response for a given command
> packet.
> - Native hardware support of LAST flag which is mandatory to align with port
> reconfiguration and DRAIN sequence as per V4L guidelines.
> - Native hardware support for drain sequence.
>
> Changes done since v1[1]:
> - Patches are created as logical split instead of file by file.
> - Extracted common functionality between venus and iris driver and placed them
> under common folder vcodec.
> - Flattened directory structure.
> - Restructured the code in a simplified layer architecture.
> - Implemented runtime PM, and dropped explicit power collapse thread in driver.
> - Moved to standard kernel log print api.
> - Simplified the bus and clock calculation logic.
> - Removed debug features like dma tracker, frame stats, debugfs.
> - Merged v4l2 and vidc layer as vidc in driver.
> - Removed separate probe for context bank.
> - Use of_device_get_match_data to get platform data.
> - Replaced complex macros with inline functions.
> - Migrated venus to iris names.
> - Addressed many other comments received on [1].
>
> [1]: https://patchwork.kernel.org/project/linux-media/list/?series=770581
>
> Patch 1, adds dt binding for iris driver.
>
> Patches 2-4, are introducing vcodec folder and moving common APIs like fw
> load/unload, buffer size calcualtions, read/write to shared queues to common
> files which are being used by both venus and iris drivers.
> This also moves the venus driver folder to vcodec folder.
>
> Patch 5, adds the maintainers for iris driver.
>
> Patch 6-29, adds core iris driver and enable decoder.
>
> Patch 30-34, enable encoder functionality in iris driver.
>

Made a mistake while explaining the patches.
Please find the correct order below:

Patches 1-3, are introducing vcodec folder and moving common APIs like fw
load/unload, buffer size calcualtions, read/write to shared queues to
common files which are being used by both venus and iris drivers.
This also moves the venus driver folder to vcodec folder.

Patch 4, adds dt binding for iris driver.

Patch 5, adds the maintainers for iris driver.

Patch 6-29, adds core iris driver and enable decoder.

Patch 30-34, enable encoder functionality in iris driver.

Thanks,
Dikshita

> Static tools like checkpatch, smatch, dt_binding_check, sparse and Coccinelle
> run successfully with this driver.
>
> The output of v4l2-compliance test looks like:
>
> v4l2-compliance SHA: not available, 64 bits
>
> Cannot open device /dev/vido0, exiting.
> root@qemuarm64:/usr/bin# ./v4l2-compliance -d /dev/video0
> v4l2-compliance SHA: not available, 64 bits
>
> Compliance test for iris_driver device /dev/video0:
>
> Driver Info:
> Driver name : iris_driver
> Card type : iris_decoder
> Bus info : platform:iris_bus
> Driver version : 6.6.0
> Capabilities : 0x84204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x04204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
>
> Required ioctls:
> test VIDIOC_QUERYCAP: OK
>
> Allow for multiple opens:
> test second /dev/video0 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> fail: ../../../v4l-utils-1.18.1/utils/v4l2-compliance/v4l2-compliance.cpp(724): !ok
> test for unlimited opens: FAIL
>
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 0 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls:
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> test VIDIOC_QUERYCTRL: OK
> test VIDIOC_G/S_CTRL: OK
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 48 Private Controls: 0
>
> Format ioctls:
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK (Not Supported)
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> test VIDIOC_S_FMT: OK
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> test Cropping: OK
> test Composing: OK
> test Scaling: OK
>
> Codec ioctls:
> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK
>
> Buffer ioctls:
> testVIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> test VIDIOC_EXPBUF: OK (Not Supported)
> test Requests: OK (Not Supported)
>
> Total for iris_driver device /dev/video0: 44, Succeeded: 43, Failed: 1, Warnings: 0
>
> Compliance test for iris_driver device /dev/video1:
>
> Driver Info:
> Driver name : iris_driver
> Card type : iris_encoder
> Bus info : platform:iris_bus
> Driver version : 6.6.0
> Capabilities : 0x84204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x04204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
>
> Required ioctls:
> test VIDIOC_QUERYCAP: OK
>
> Allow for multiple opens:
> test second /dev/video1 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> fail: ../../../v4l-utils-1.18.1/utils/v4l2-compliance/v4l2-compliance.cpp(724): !ok
> test for unlimited opens: FAIL
>
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 0 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls:
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> test VIDIOC_QUERYCTRL: OK
> test VIDIOC_G/S_CTRL: OK
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 48 Private Controls: 0
>
> Format ioctls:
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> test VIDIOC_S_FMT: OK
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> test Cropping: OK
> test Composing: OK
> test Scaling: OK (Not Supported)
>
> Codec ioctls:
> test VIDIOC_(TRY_)ENCODER_CMD: OK
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>
> Buffer ioctls:
> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> test VIDIOC_EXPBUF: OK (Not Supported)
> test Requests: OK (Not Supported)
>
> Total for iris_driver device /dev/video1: 44, Succeeded: 43, Failed: 1, Warnings: 0
>
> Dikshita Agarwal (27):
> media: introduce common helpers for video firmware handling
> media: introduce common helpers for queues handling
> media: introduce common helpers for buffer size calculation
> dt-bindings: media: Add sm8550 dt schema
> media: MAINTAINERS: Add Qualcomm Iris video accelerator driver
> media: iris: register video device to platform driver
> media: iris: initialize power resources
> media: iris: introduce state machine for iris core
> media: iris: initialize shared queues for host and firmware
> communication
> media: iris: add PIL functionality for video firmware
> media: iris: introduce packetization layer for creating HFI packets
> media: iris: add video processing unit(VPU) specific register handling
> media: iris: introduce platform specific capabilities for core and
> instance
> media: iris: add handling for interrupt service routine(ISR) invoked
> by hardware
> media: iris: implement iris v4l2_ctrl_ops and prepare capabilities
> media: iris: implement vb2_ops queue setup
> media: iris: implement HFI to queue and release buffers
> media: iris: add video hardware internal buffer count and size
> calculation
> media: iris: implement internal buffer management
> media: iris: introduce instance states
> media: iris: implement vb2 streaming ops on capture and output planes
> media: iris: implement vb2 ops for buf_queue and firmware response
> media: iris: register video encoder device to platform driver
> media: iris: add platform specific instance capabilities for encoder
> media: iris: implement iris v4l2 ioctl ops supported by encoder
> media: iris: add vb2 streaming and buffer ops for encoder
> media: iris: add power management for encoder
>
> Vikash Garodia (7):
> media: iris: implement iris v4l2 file ops
> media: iris: introduce and implement iris vb2 mem ops
> media: iris: implement iris v4l2 ioctl ops supported by decoder
> media: iris: subscribe input and output properties to firmware
> media: iris: subscribe src change and handle firmware responses
> media: iris: add instance sub states and implement DRC and Drain
> sequence
> media: iris: implement power management
>
> .../bindings/media/qcom,sm8550-iris.yaml | 177 ++
> MAINTAINERS | 11 +
> drivers/media/platform/qcom/Kconfig | 3 +-
> drivers/media/platform/qcom/Makefile | 3 +-
> drivers/media/platform/qcom/vcodec/buffers.c | 103 ++
> drivers/media/platform/qcom/vcodec/buffers.h | 15 +
> drivers/media/platform/qcom/vcodec/firmware.c | 147 ++
> drivers/media/platform/qcom/vcodec/firmware.h | 21 +
> drivers/media/platform/qcom/vcodec/hfi_queue.c | 258 +++
> drivers/media/platform/qcom/vcodec/hfi_queue.h | 129 ++
> drivers/media/platform/qcom/vcodec/iris/Kconfig | 13 +
> drivers/media/platform/qcom/vcodec/iris/Makefile | 26 +
> .../media/platform/qcom/vcodec/iris/hfi_defines.h | 386 +++++
> .../media/platform/qcom/vcodec/iris/iris_buffer.c | 833 +++++++++
> .../media/platform/qcom/vcodec/iris/iris_buffer.h | 65 +
> .../media/platform/qcom/vcodec/iris/iris_common.h | 147 ++
> .../media/platform/qcom/vcodec/iris/iris_core.c | 110 ++
> .../media/platform/qcom/vcodec/iris/iris_core.h | 121 ++
> .../media/platform/qcom/vcodec/iris/iris_ctrls.c | 1782 ++++++++++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_ctrls.h | 71 +
> .../media/platform/qcom/vcodec/iris/iris_helpers.c | 1099 ++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_helpers.h | 68 +
> drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 917 ++++++++++
> drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 47 +
> .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 729 ++++++++
> .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 118 ++
> .../platform/qcom/vcodec/iris/iris_hfi_response.c | 1097 ++++++++++++
> .../platform/qcom/vcodec/iris/iris_hfi_response.h | 20 +
> .../platform/qcom/vcodec/iris/iris_instance.h | 106 ++
> .../media/platform/qcom/vcodec/iris/iris_power.c | 178 ++
> .../media/platform/qcom/vcodec/iris/iris_power.h | 15 +
> .../media/platform/qcom/vcodec/iris/iris_probe.c | 357 ++++
> .../media/platform/qcom/vcodec/iris/iris_state.c | 453 +++++
> .../media/platform/qcom/vcodec/iris/iris_state.h | 78 +
> drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 457 +++++
> drivers/media/platform/qcom/vcodec/iris/iris_vb2.h | 28 +
> .../media/platform/qcom/vcodec/iris/iris_vdec.c | 1211 +++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_vdec.h | 25 +
> .../media/platform/qcom/vcodec/iris/iris_venc.c | 948 +++++++++++
> .../media/platform/qcom/vcodec/iris/iris_venc.h | 27 +
> .../media/platform/qcom/vcodec/iris/iris_vidc.c | 1419 ++++++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_vidc.h | 15 +
> .../platform/qcom/vcodec/iris/platform_common.c | 29 +
> .../platform/qcom/vcodec/iris/platform_common.h | 323 ++++
> .../platform/qcom/vcodec/iris/platform_sm8550.c | 1190 +++++++++++++
> .../media/platform/qcom/vcodec/iris/resources.c | 506 ++++++
> .../media/platform/qcom/vcodec/iris/resources.h | 44 +
> .../media/platform/qcom/vcodec/iris/vpu_common.c | 158 ++
> .../media/platform/qcom/vcodec/iris/vpu_common.h | 52 +
> .../media/platform/qcom/vcodec/iris/vpu_iris3.c | 568 +++++++
> .../media/platform/qcom/vcodec/iris/vpu_iris3.h | 13 +
> .../platform/qcom/vcodec/iris/vpu_iris3_buffer.c | 395 +++++
> .../platform/qcom/vcodec/iris/vpu_iris3_buffer.h | 1469 ++++++++++++++++
> .../platform/qcom/vcodec/iris/vpu_iris3_power.c | 148 ++
> .../platform/qcom/vcodec/iris/vpu_iris3_power.h | 13 +
> .../media/platform/qcom/{ => vcodec}/venus/Kconfig | 0
> .../platform/qcom/{ => vcodec}/venus/Makefile | 5 +-
> .../media/platform/qcom/{ => vcodec}/venus/core.c | 102 +-
> .../media/platform/qcom/{ => vcodec}/venus/core.h | 0
> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.h | 0
> .../platform/qcom/vcodec/venus/firmware_no_tz.c | 194 +++
> .../platform/qcom/vcodec/venus/firmware_no_tz.h | 19 +
> .../platform/qcom/{ => vcodec}/venus/helpers.c | 172 +-
> .../platform/qcom/{ => vcodec}/venus/helpers.h | 2 +-
> .../media/platform/qcom/{ => vcodec}/venus/hfi.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/hfi.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_helper.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_parser.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_parser.h | 0
> .../qcom/{ => vcodec}/venus/hfi_plat_bufs.h | 4 +-
> .../qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c | 10 +-
> .../qcom/{ => vcodec}/venus/hfi_platform.c | 0
> .../qcom/{ => vcodec}/venus/hfi_platform.h | 0
> .../qcom/{ => vcodec}/venus/hfi_platform_v4.c | 0
> .../qcom/{ => vcodec}/venus/hfi_platform_v6.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_venus.c | 521 +-----
> .../platform/qcom/{ => vcodec}/venus/hfi_venus.h | 0
> .../qcom/{ => vcodec}/venus/hfi_venus_io.h | 0
> .../platform/qcom/{ => vcodec}/venus/pm_helpers.c | 0
> .../platform/qcom/{ => vcodec}/venus/pm_helpers.h | 0
> .../media/platform/qcom/{ => vcodec}/venus/vdec.c | 5 +-
> .../media/platform/qcom/{ => vcodec}/venus/vdec.h | 0
> .../platform/qcom/{ => vcodec}/venus/vdec_ctrls.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/venc.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/venc.h | 0
> .../platform/qcom/{ => vcodec}/venus/venc_ctrls.c | 0
> drivers/media/platform/qcom/venus/firmware.c | 363 ----
> drivers/media/platform/qcom/venus/firmware.h | 26 -
> 93 files changed, 19175 insertions(+), 989 deletions(-)
> create mode 100644 Documentation/devicetree/bindings/media/qcom,sm8550-iris.yaml
> create mode 100644 drivers/media/platform/qcom/vcodec/buffers.c
> create mode 100644 drivers/media/platform/qcom/vcodec/buffers.h
> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.c
> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.h
> create mode 100644 drivers/media/platform/qcom/vcodec/hfi_queue.c
> create mode 100644 drivers/media/platform/qcom/vcodec/hfi_queue.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/Kconfig
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/Makefile
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_common.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_core.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_core.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_instance.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_power.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_power.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_probe.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_state.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_state.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_venc.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_venc.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_common.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_common.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/resources.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/resources.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_common.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_common.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.h
> rename drivers/media/platform/qcom/{ => vcodec}/venus/Kconfig (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/Makefile (75%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.c (91%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.h (100%)
> create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
> create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.c (90%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.h (98%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_helper.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs.h (94%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c (99%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v4.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v6.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.c (73%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus_io.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.c (99%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec_ctrls.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc_ctrls.c (100%)
> delete mode 100644 drivers/media/platform/qcom/venus/firmware.c
> delete mode 100644 drivers/media/platform/qcom/venus/firmware.h
>

2023-12-18 14:38:49

[permalink] [raw]

Subject: Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver

On 18/12/2023 11:31, Dikshita Agarwal wrote:

> Static tools like checkpatch, smatch, dt_binding_check, sparse and Coccinelle
> run successfully with this driver.
>
> The output of v4l2-compliance test looks like:
>
> v4l2-compliance SHA: not available, 64 bits
>
> Cannot open device /dev/vido0, exiting.
> root@qemuarm64:/usr/bin# ./v4l2-compliance -d /dev/video0
> v4l2-compliance SHA: not available, 64 bits
>
> Compliance test for iris_driver device /dev/video0:
>
> Driver Info:
> Driver name : iris_driver
> Card type : iris_decoder
> Bus info : platform:iris_bus
> Driver version : 6.6.0
> Capabilities : 0x84204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x04204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format

Good to see some code sharing, is there no way to facilitate the new
code in existing venus ?

I applied your patches to -stable and was happy to find they didn't
break venus on rb5, at least for the superficial test I did [1]. The
names you export here look odd though.

"Driver name" and "Cart Type" should be something like "qcom-iris" and
"Qualcomm Iris video decoder"

v4l2-compliance -d /dev/video15
v4l2-compliance 1.24.1, 64 bits, 64-bit time_t

Compliance test for qcom-venus device /dev/video15:

Driver Info:
Driver name : qcom-venus
Card type : Qualcomm Venus video encoder
Bus info : platform:qcom-venus
Driver version : 6.7.0
Capabilities : 0x84204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Detected Stateful Encoder
[1]
https://file-examples.com/index.php/sample-video-files/sample-mp4-files/

ffplay -loglevel debug -codec:video h264_v4l2m2m -i sample-30s.mp4

> Driver Info:
> Driver name : iris_driver
> Card type : iris_encoder

Same comment on normalising names wrt to venus.

---
bod

2023-12-18 16:09:22

by Xilin Wu

[permalink] [raw]

Subject: Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver

Hi, I tested the patches on qcs8550 (which should be the same as
sm8550) and I seemed to get different
results with v4l2-compliance sometimes. Also, ffmpeg never seemed to work.

ffmpeg cmdline: sudo ffmpeg -loglevel debug -c:v h264_v4l2m2m -i
rickroll.mp4 -f null /dev/null
Partial output:

[h264_v4l2m2m @ 0xaaaabf7a22b0] probing device /dev/video0
[h264_v4l2m2m @ 0xaaaabf7a22b0] driver 'iris_driver' on card
'iris_decoder' in mplane mode
[h264_v4l2m2m @ 0xaaaabf7a22b0] Using device /dev/video0
[h264_v4l2m2m @ 0xaaaabf7a22b0] driver 'iris_driver' on card
'iris_decoder' in mplane mode
[h264_v4l2m2m @ 0xaaaabf7a22b0] requesting formats: output=H264/none
capture=Q08C/yuv420p
[h264_v4l2m2m @ 0xaaaabf7a22b0] output buffer[0] initialization
(Cannot allocate memory)
[h264_v4l2m2m @ 0xaaaabf7a22b0] output unmap plane (Invalid argument))
[h264_v4l2m2m @ 0xaaaabf7a22b0] no v4l2 output context's buffers
[h264_v4l2m2m @ 0xaaaabf7a22b0] can't configure decoder
[vist#0:0/h264 @ 0xaaaabf741e70] Error while opening decoder: Cannot
allocate memory
[vost#0:0/wrapped_avframe @ 0xaaaabf760570] Error initializing a
simple filtergraph
Error opening output file /dev/null.
Error opening output files: Cannot allocate memory
[AVIOContext @ 0xaaaabf742c30] Statistics: 1703936 bytes read, 44 seeks

After trying several times, it just fails:

[h264_v4l2m2m @ 0xaaaaf7f852b0] probing device /dev/video0
[h264_v4l2m2m @ 0xaaaaf7f852b0] probing device /dev/video1
[h264_v4l2m2m @ 0xaaaaf7f852b0] Could not find a valid device
[h264_v4l2m2m @ 0xaaaaf7f852b0] can't configure decoder
[vist#0:0/h264 @ 0xaaaaf7f24e70] Error while opening decoder: Invalid argument
[vost#0:0/wrapped_avframe @ 0xaaaaf7f43570] Error initializing a
simple filtergraph
Error opening output file /dev/null.
Error opening output files: Invalid argument

v4l2-compliance output:

[alarm@ayn-odin2 ~]$ sudo v4l2-compliance -d /dev/video0
v4l2-compliance 1.26.0, 64 bits, 64-bit time_t

Compliance test for iris_driver device /dev/video0:

Driver Info:
Driver name : iris_driver
Card type : iris_decoder
Bus info : platform:iris_bus
Driver version : 6.7.0
Capabilities : 0x84204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Detected Stateful Decoder

Required ioctls:
test VIDIOC_QUERYCAP: OK
test invalid ioctls: OK

Allow for multiple opens:
test second /dev/video0 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
fail: v4l2-compliance.cpp(763): !ok
test for unlimited opens: FAIL

Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK
test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 48 Private Controls: 0

Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK
test Composing: OK
test Scaling: OK (Not Supported)

Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK

Buffer ioctls:
fail: v4l2-test-buffers.cpp(553): ret != EINVAL (got 9)
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
test CREATE_BUFS maximum buffers: OK
fail: v4l2-test-buffers.cpp(814): VIDIOC_EXPBUF is
supported, but the V4L2_MEMORY_MMAP support is missing or
malfunctioning.
fail: v4l2-test-buffers.cpp(815): VIDIOC_EXPBUF is
supported, but the V4L2_MEMORY_MMAP support is missing, probably due
to earlier failing format tests.
test VIDIOC_EXPBUF: OK (Not Supported)
failed to stat file
fail: v4l2-test-buffers.cpp(2140): ret != EINVAL &&
ret != EBADR && ret != ENOTTY (got 9)
test Requests: FAIL

Total for iris_driver device /dev/video0: 46, Succeeded: 43, Failed:
3, Warnings: 0
[alarm@ayn-odin2 ~]$ sudo v4l2-compliance -d /dev/video1
v4l2-compliance 1.26.0, 64 bits, 64-bit time_t

Failed to open /dev/video1: Invalid argument
[alarm@ayn-odin2 ~]$ sudo v4l2-compliance -d /dev/video0
v4l2-compliance 1.26.0, 64 bits, 64-bit time_t

Failed to open /dev/video0: Invalid argument

2023-12-18 18:25:15

[permalink] [raw]

Subject: Re: [PATCH v2 01/34] media: introduce common helpers for video firmware handling

On 18/12/2023 13:31, Dikshita Agarwal wrote:
> Re-organize the video driver code by introducing a new folder
> 'vcodec' and placing 'venus' driver code inside that.
>
> Introduce common helpers for trustzone based firmware
> load/unload etc. which are placed in common folder
> i.e. 'vcodec'.
> Use these helpers in 'venus' driver. These helpers will be
> used by 'iris' driver as well which is introduced later
> in this patch series.

But why do you need to move the venus driver to subdir?

>
> Signed-off-by: Dikshita Agarwal <[email protected]>
> ---
> drivers/media/platform/qcom/Kconfig | 2 +-
> drivers/media/platform/qcom/Makefile | 2 +-
> drivers/media/platform/qcom/vcodec/firmware.c | 147 +++++++++
> drivers/media/platform/qcom/vcodec/firmware.h | 21 ++
> .../media/platform/qcom/{ => vcodec}/venus/Kconfig | 0
> .../platform/qcom/{ => vcodec}/venus/Makefile | 4 +-
> .../media/platform/qcom/{ => vcodec}/venus/core.c | 102 +++++-
> .../media/platform/qcom/{ => vcodec}/venus/core.h | 0
> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.h | 0
> .../platform/qcom/vcodec/venus/firmware_no_tz.c | 194 +++++++++++
> .../platform/qcom/vcodec/venus/firmware_no_tz.h | 19 ++
> .../platform/qcom/{ => vcodec}/venus/helpers.c | 0
> .../platform/qcom/{ => vcodec}/venus/helpers.h | 0
> .../media/platform/qcom/{ => vcodec}/venus/hfi.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/hfi.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_helper.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_parser.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_parser.h | 0
> .../qcom/{ => vcodec}/venus/hfi_plat_bufs.h | 0
> .../qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c | 0
> .../qcom/{ => vcodec}/venus/hfi_platform.c | 0
> .../qcom/{ => vcodec}/venus/hfi_platform.h | 0
> .../qcom/{ => vcodec}/venus/hfi_platform_v4.c | 0
> .../qcom/{ => vcodec}/venus/hfi_platform_v6.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_venus.c | 21 +-
> .../platform/qcom/{ => vcodec}/venus/hfi_venus.h | 0
> .../qcom/{ => vcodec}/venus/hfi_venus_io.h | 0
> .../platform/qcom/{ => vcodec}/venus/pm_helpers.c | 0
> .../platform/qcom/{ => vcodec}/venus/pm_helpers.h | 0
> .../media/platform/qcom/{ => vcodec}/venus/vdec.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/vdec.h | 0
> .../platform/qcom/{ => vcodec}/venus/vdec_ctrls.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/venc.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/venc.h | 0
> .../platform/qcom/{ => vcodec}/venus/venc_ctrls.c | 0
> drivers/media/platform/qcom/venus/firmware.c | 363 ---------------------
> drivers/media/platform/qcom/venus/firmware.h | 26 --
> 42 files changed, 492 insertions(+), 409 deletions(-)
> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.c
> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.h
> rename drivers/media/platform/qcom/{ => vcodec}/venus/Kconfig (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/Makefile (83%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.c (91%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.h (100%)
> create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
> create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_helper.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v4.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v6.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.c (99%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus_io.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec_ctrls.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc_ctrls.c (100%)
> delete mode 100644 drivers/media/platform/qcom/venus/firmware.c
> delete mode 100644 drivers/media/platform/qcom/venus/firmware.h
>
> diff --git a/drivers/media/platform/qcom/Kconfig b/drivers/media/platform/qcom/Kconfig
> index cc5799b..e94142f 100644
> --- a/drivers/media/platform/qcom/Kconfig
> +++ b/drivers/media/platform/qcom/Kconfig
> @@ -3,4 +3,4 @@
> comment "Qualcomm media platform drivers"
>
> source "drivers/media/platform/qcom/camss/Kconfig"
> -source "drivers/media/platform/qcom/venus/Kconfig"
> +source "drivers/media/platform/qcom/vcodec/venus/Kconfig"
> diff --git a/drivers/media/platform/qcom/Makefile b/drivers/media/platform/qcom/Makefile
> index 4f055c3..3d2d82b 100644
> --- a/drivers/media/platform/qcom/Makefile
> +++ b/drivers/media/platform/qcom/Makefile
> @@ -1,3 +1,3 @@
> # SPDX-License-Identifier: GPL-2.0-only
> obj-y += camss/
> -obj-y += venus/
> +obj-y += vcodec/venus/
> diff --git a/drivers/media/platform/qcom/vcodec/firmware.c b/drivers/media/platform/qcom/vcodec/firmware.c
> new file mode 100644
> index 0000000..dbc220a
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/firmware.c
> @@ -0,0 +1,147 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
> + */
> +
> +#include <linux/device.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/firmware.h>
> +#include <linux/kernel.h>
> +#include <linux/iommu.h>
> +#include <linux/of_device.h>
> +#include <linux/firmware/qcom/qcom_scm.h>
> +#include <linux/of_reserved_mem.h>
> +#include <linux/platform_device.h>
> +#include <linux/soc/qcom/mdt_loader.h>
> +
> +#include "firmware.h"
> +
> +bool use_tz(struct device *core_dev)

All these functions must get some sane prefix. Otherwise a generic
'use_tz' function is too polluting for the global namespace.

> +{
> + struct device_node *np;
> +
> + np = of_get_child_by_name(core_dev->of_node, "video-firmware");
> + if (!np)
> + return true;
> +
> + return false;
> +}
> +
> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
> + u32 cp_nonpixel_size, u32 pas_id)
> +{
> + int ret;
> + /*
> + * Clues for porting using downstream data:
> + * cp_start = 0
> + * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and not size!
> + * This works, as the non-secure context bank is placed
> + * contiguously right after the Content Protection region.
> + *
> + * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0]
> + * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1]
> + */
> + ret = qcom_scm_mem_protect_video_var(cp_start,
> + cp_size,
> + cp_nonpixel_start,
> + cp_nonpixel_size);
> + if (ret)
> + qcom_scm_pas_shutdown(pas_id);
> +
> + return ret;
> +}
> +
> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
> + size_t *mem_size, u32 pas_id, bool use_tz)
> +{
> + const struct firmware *firmware = NULL;
> + struct reserved_mem *rmem;
> + struct device_node *node;
> + void *mem_virt = NULL;
> + ssize_t fw_size = 0;
> + int ret;
> +
> + if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||

Why? Can you just depend on it?

> + (use_tz && !qcom_scm_is_available()))
> + return -EPROBE_DEFER;
> +
> + if (!fw_name || !(*fw_name))
> + return -EINVAL;
> +
> + *mem_phys = 0;
> + *mem_size = 0;
> +
> + node = of_parse_phandle(dev->of_node, "memory-region", 0);
> + if (!node) {
> + dev_err(dev, "no memory-region specified\n");
> + return -EINVAL;
> + }
> +
> + rmem = of_reserved_mem_lookup(node);
> + of_node_put(node);
> + if (!rmem) {
> + dev_err(dev, "failed to lookup reserved memory-region\n");
> + return -EINVAL;
> + }
> +
> + ret = request_firmware(&firmware, fw_name, dev);
> + if (ret) {
> + dev_err(dev, "%s: failed to request fw \"%s\", error %d\n",
> + __func__, fw_name, ret);
> + return ret;
> + }
> +
> + fw_size = qcom_mdt_get_size(firmware);
> + if (fw_size < 0) {
> + ret = fw_size;
> + dev_err(dev, "%s: out of bound fw image fw size: %ld\n",
> + __func__, fw_size);
> + goto err_release_fw;
> + }
> +
> + *mem_phys = rmem->base;
> + *mem_size = rmem->size;
> +
> + if (*mem_size < fw_size) {
> + ret = -EINVAL;
> + goto err_release_fw;
> + }
> +
> + mem_virt = memremap(*mem_phys, *mem_size, MEMREMAP_WC);
> + if (!mem_virt) {
> + dev_err(dev, "unable to remap fw memory region %pa size %#zx\n",
> + mem_phys, *mem_size);
> + goto err_release_fw;
> + }
> +
> + if (use_tz)
> + ret = qcom_mdt_load(dev, firmware, fw_name, pas_id, mem_virt,
> + *mem_phys, *mem_size, NULL);
> + else
> + ret = qcom_mdt_load_no_init(dev, firmware, fw_name, pas_id, mem_virt,
> + *mem_phys, *mem_size, NULL);
> + if (ret) {
> + dev_err(dev, "%s: error %d loading fw \"%s\"\n",
> + __func__, ret, fw_name);
> + }
> +
> + memunmap(mem_virt);
> +err_release_fw:
> + release_firmware(firmware);
> + return ret;
> +}
> +
> +int auth_reset_fw(u32 pas_id)
> +{
> + return qcom_scm_pas_auth_and_reset(pas_id);
> +}
> +
> +void unload_fw(u32 pas_id)
> +{
> + qcom_scm_pas_shutdown(pas_id);
> +}
> +
> +int set_hw_state(bool resume)
> +{
> + return qcom_scm_set_remote_state(resume, 0);
> +}
> diff --git a/drivers/media/platform/qcom/vcodec/firmware.h b/drivers/media/platform/qcom/vcodec/firmware.h
> new file mode 100644
> index 0000000..7d410a8
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/firmware.h
> @@ -0,0 +1,21 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
> + */
> +
> +#ifndef _FIRMWARE_H_
> +#define _FIRMWARE_H_
> +
> +#include <linux/device.h>
> +#include <linux/types.h>
> +
> +bool use_tz(struct device *core_dev);
> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
> + size_t *mem_size, u32 pas_id, bool use_tz);
> +int auth_reset_fw(u32 pas_id);
> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
> + u32 cp_nonpixel_size, u32 pas_id);
> +void unload_fw(u32 pas_id);
> +int set_hw_state(bool resume);
> +
> +#endif
> diff --git a/drivers/media/platform/qcom/venus/Kconfig b/drivers/media/platform/qcom/vcodec/venus/Kconfig
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/Kconfig
> rename to drivers/media/platform/qcom/vcodec/venus/Kconfig
> diff --git a/drivers/media/platform/qcom/venus/Makefile b/drivers/media/platform/qcom/vcodec/venus/Makefile
> similarity index 83%
> rename from drivers/media/platform/qcom/venus/Makefile
> rename to drivers/media/platform/qcom/vcodec/venus/Makefile
> index 91ee6be..f6f3a88 100644
> --- a/drivers/media/platform/qcom/venus/Makefile
> +++ b/drivers/media/platform/qcom/vcodec/venus/Makefile
> @@ -1,7 +1,9 @@
> # SPDX-License-Identifier: GPL-2.0
> # Makefile for Qualcomm Venus driver
>
> -venus-core-objs += core.o helpers.o firmware.o \
> +venus-core-objs += ../firmware.o
> +
> +venus-core-objs += core.o helpers.o firmware_no_tz.o \
> hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \
> hfi_parser.o pm_helpers.o dbgfs.o \
> hfi_platform.o hfi_platform_v4.o \
> diff --git a/drivers/media/platform/qcom/venus/core.c b/drivers/media/platform/qcom/vcodec/venus/core.c
> similarity index 91%
> rename from drivers/media/platform/qcom/venus/core.c
> rename to drivers/media/platform/qcom/vcodec/venus/core.c
> index 9cffe97..56d9a53 100644
> --- a/drivers/media/platform/qcom/venus/core.c
> +++ b/drivers/media/platform/qcom/vcodec/venus/core.c
> @@ -22,7 +22,8 @@
> #include <media/v4l2-ioctl.h>
>
> #include "core.h"
> -#include "firmware.h"
> +#include "../firmware.h"
> +#include "firmware_no_tz.h"
> #include "pm_helpers.h"
> #include "hfi_venus_io.h"
>
> @@ -86,6 +87,8 @@ static void venus_sys_error_handler(struct work_struct *work)
> struct venus_core *core =
> container_of(work, struct venus_core, work.work);
> int ret, i, max_attempts = RPM_WAIT_FOR_IDLE_MAX_ATTEMPTS;
> + const struct venus_resources *res = core->res;
> + const char *fwpath = NULL;
> const char *err_msg = "";
> bool failed = false;
>
> @@ -107,7 +110,10 @@ static void venus_sys_error_handler(struct work_struct *work)
>
> mutex_lock(&core->lock);
>
> - venus_shutdown(core);
> + if (core->use_tz)
> + unload_fw(VENUS_PAS_ID);
> + else
> + unload_fw_no_tz(core);

This is more than introducing helpers.

>
> venus_coredump(core);
>
> @@ -127,12 +133,39 @@ static void venus_sys_error_handler(struct work_struct *work)
> failed = true;
> }
>
> - ret = venus_boot(core);
> + ret = of_property_read_string_index(core->dev->of_node, "firmware-name", 0,
> + &fwpath);
> + if (ret)
> + fwpath = core->res->fwname;
> +
> + ret = load_fw(core->dev, fwpath, &core->fw.mem_phys, &core->fw.mem_size,
> + VENUS_PAS_ID, core->use_tz);

So, we had a nice local 'venus_boot'. Instead we now have a pile of code
with non-generic prefixes, etc. If you are introducing helpers, please
refrain from inlining of calling functions, etc. Just move the code to
your helpers.

NAK for the rest of the patch.

> if (ret && !failed) {
> - err_msg = "boot Venus";
> + err_msg = "load FW";
> failed = true;
> }
>
> + if (core->use_tz)
> + ret = auth_reset_fw(VENUS_PAS_ID);
> + else
> + ret = auth_reset_fw_no_tz(core, core->fw.mem_phys, core->fw.mem_size);
> + if (ret && !failed) {
> + err_msg = "Auth and Reset";
> + failed = true;
> + }
> +
> + if (core->use_tz && res->cp_size) {
> + ret = protect_secure_region(res->cp_start,
> + res->cp_size,
> + res->cp_nonpixel_start,
> + res->cp_nonpixel_size,
> + VENUS_PAS_ID);
> + if (ret && !failed) {
> + err_msg = "Protect CP Mem";
> + failed = true;
> + }
> + }
> +
> ret = hfi_core_resume(core, true);
> if (ret && !failed) {
> err_msg = "resume HFI";
> @@ -281,7 +314,9 @@ static irqreturn_t venus_isr_thread(int irq, void *dev_id)
>
> static int venus_probe(struct platform_device *pdev)
> {
> + const struct venus_resources *res;
> struct device *dev = &pdev->dev;
> + const char *fwpath = NULL;
> struct venus_core *core;
> int ret;
>
> @@ -362,14 +397,42 @@ static int venus_probe(struct platform_device *pdev)
> if (ret)
> goto err_runtime_disable;
>
> - ret = venus_firmware_init(core);
> + core->use_tz = use_tz(core->dev);
> +
> + if (!core->use_tz) {
> + ret = init_fw_no_tz(core);
> + if (ret)
> + goto err_of_depopulate;
> + }
> +
> + ret = of_property_read_string_index(dev->of_node, "firmware-name", 0,
> + &fwpath);
> if (ret)
> - goto err_of_depopulate;
> + fwpath = core->res->fwname;
>
> - ret = venus_boot(core);
> + ret = load_fw(core->dev, fwpath, &core->fw.mem_phys, &core->fw.mem_size,
> + VENUS_PAS_ID, core->use_tz);
> if (ret)
> goto err_firmware_deinit;
>
> + if (core->use_tz)
> + ret = auth_reset_fw(VENUS_PAS_ID);
> + else
> + ret = auth_reset_fw_no_tz(core, core->fw.mem_phys, core->fw.mem_size);
> + if (ret)
> + goto err_firmware_deinit;
> +
> + res = core->res;
> + if (core->use_tz && res->cp_size) {
> + ret = protect_secure_region(res->cp_start,
> + res->cp_size,
> + res->cp_nonpixel_start,
> + res->cp_nonpixel_size,
> + VENUS_PAS_ID);
> + if (ret)
> + goto err_firmware_deinit;
> + }
> +
> ret = hfi_core_resume(core, true);
> if (ret)
> goto err_venus_shutdown;
> @@ -399,9 +462,13 @@ static int venus_probe(struct platform_device *pdev)
> err_dev_unregister:
> v4l2_device_unregister(&core->v4l2_dev);
> err_venus_shutdown:
> - venus_shutdown(core);
> + if (core->use_tz)
> + unload_fw(VENUS_PAS_ID);
> + else
> + unload_fw_no_tz(core);
> err_firmware_deinit:
> - venus_firmware_deinit(core);
> + if (!core->use_tz)
> + deinit_fw_no_tz(core);
> err_of_depopulate:
> of_platform_depopulate(dev);
> err_runtime_disable:
> @@ -430,10 +497,15 @@ static void venus_remove(struct platform_device *pdev)
> ret = hfi_core_deinit(core, true);
> WARN_ON(ret);
>
> - venus_shutdown(core);
> + if (core->use_tz)
> + unload_fw(VENUS_PAS_ID);
> + else
> + unload_fw_no_tz(core);
> +
> of_platform_depopulate(dev);
>
> - venus_firmware_deinit(core);
> + if (!core->use_tz)
> + deinit_fw_no_tz(core);
>
> pm_runtime_put_sync(dev);
> pm_runtime_disable(dev);
> @@ -455,8 +527,12 @@ static void venus_core_shutdown(struct platform_device *pdev)
> struct venus_core *core = platform_get_drvdata(pdev);
>
> pm_runtime_get_sync(core->dev);
> - venus_shutdown(core);
> - venus_firmware_deinit(core);
> + if (core->use_tz) {
> + unload_fw(VENUS_PAS_ID);
> + } else {
> + unload_fw_no_tz(core);
> + deinit_fw_no_tz(core);
> + }
> pm_runtime_put_sync(core->dev);
> }
>
> diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/vcodec/venus/core.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/core.h
> rename to drivers/media/platform/qcom/vcodec/venus/core.h
> diff --git a/drivers/media/platform/qcom/venus/dbgfs.c b/drivers/media/platform/qcom/vcodec/venus/dbgfs.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/dbgfs.c
> rename to drivers/media/platform/qcom/vcodec/venus/dbgfs.c
> diff --git a/drivers/media/platform/qcom/venus/dbgfs.h b/drivers/media/platform/qcom/vcodec/venus/dbgfs.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/dbgfs.h
> rename to drivers/media/platform/qcom/vcodec/venus/dbgfs.h
> diff --git a/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
> new file mode 100644
> index 0000000..9dca6e23
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
> @@ -0,0 +1,194 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2017 Linaro Ltd.
> + */
> +
> +#include <linux/iommu.h>
> +#include <linux/of_device.h>
> +#include "core.h"
> +#include "firmware_no_tz.h"
> +#include "hfi_venus_io.h"
> +
> +#define VENUS_FW_MEM_SIZE (6 * SZ_1M)
> +#define VENUS_FW_START_ADDR 0x0
> +
> +int init_fw_no_tz(struct venus_core *core)
> +{
> + struct platform_device_info info;
> + struct iommu_domain *iommu_dom;
> + struct platform_device *pdev;
> + struct device_node *np;
> + int ret;
> +
> + np = of_get_child_by_name(core->dev->of_node, "video-firmware");
> +
> + memset(&info, 0, sizeof(info));
> + info.fwnode = &np->fwnode;
> + info.parent = core->dev;
> + info.name = np->name;
> + info.dma_mask = DMA_BIT_MASK(32);
> +
> + pdev = platform_device_register_full(&info);
> + if (IS_ERR(pdev)) {
> + of_node_put(np);
> + return PTR_ERR(pdev);
> + }
> +
> + pdev->dev.of_node = np;
> +
> + ret = of_dma_configure(&pdev->dev, np, true);
> + if (ret) {
> + dev_err(core->dev, "dma configure fail\n");
> + goto err_unregister;
> + }
> +
> + core->fw.dev = &pdev->dev;
> +
> + iommu_dom = iommu_domain_alloc(&platform_bus_type);
> + if (!iommu_dom) {
> + dev_err(core->fw.dev, "Failed to allocate iommu domain\n");
> + ret = -ENOMEM;
> + goto err_unregister;
> + }
> +
> + ret = iommu_attach_device(iommu_dom, core->fw.dev);
> + if (ret) {
> + dev_err(core->fw.dev, "could not attach device\n");
> + goto err_iommu_free;
> + }
> +
> + core->fw.iommu_domain = iommu_dom;
> +
> + of_node_put(np);
> +
> + return 0;
> +
> +err_iommu_free:
> + iommu_domain_free(iommu_dom);
> +err_unregister:
> + platform_device_unregister(pdev);
> + of_node_put(np);
> + return ret;
> +}
> +
> +void deinit_fw_no_tz(struct venus_core *core)
> +{
> + struct iommu_domain *iommu;
> +
> + if (!core->fw.dev)
> + return;
> +
> + iommu = core->fw.iommu_domain;
> +
> + iommu_detach_device(iommu, core->fw.dev);
> +
> + if (iommu) {
> + iommu_domain_free(iommu);
> + iommu = NULL;
> + }
> +
> + platform_device_unregister(to_platform_device(core->fw.dev));
> +}
> +
> +static void reset_cpu_no_tz(struct venus_core *core)
> +{
> + u32 fw_size = core->fw.mapped_mem_size;
> + void __iomem *wrapper_base;
> +
> + if (IS_IRIS2_1(core))
> + wrapper_base = core->wrapper_tz_base;
> + else
> + wrapper_base = core->wrapper_base;
> +
> + writel(0, wrapper_base + WRAPPER_FW_START_ADDR);
> + writel(fw_size, wrapper_base + WRAPPER_FW_END_ADDR);
> + writel(0, wrapper_base + WRAPPER_CPA_START_ADDR);
> + writel(fw_size, wrapper_base + WRAPPER_CPA_END_ADDR);
> + writel(fw_size, wrapper_base + WRAPPER_NONPIX_START_ADDR);
> + writel(fw_size, wrapper_base + WRAPPER_NONPIX_END_ADDR);
> +
> + if (IS_IRIS2_1(core)) {
> + /* Bring XTSS out of reset */
> + writel(0, wrapper_base + WRAPPER_TZ_XTSS_SW_RESET);
> + } else {
> + writel(0x0, wrapper_base + WRAPPER_CPU_CGC_DIS);
> + writel(0x0, wrapper_base + WRAPPER_CPU_CLOCK_CONFIG);
> +
> + /* Bring ARM9 out of reset */
> + writel(0, wrapper_base + WRAPPER_A9SS_SW_RESET);
> + }
> +}
> +
> +void set_hw_state_no_tz(struct venus_core *core, bool resume)
> +{
> + if (resume) {
> + reset_cpu_no_tz(core);
> + } else {
> + if (IS_IRIS2_1(core))
> + writel(WRAPPER_XTSS_SW_RESET_BIT,
> + core->wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
> + else
> + writel(WRAPPER_A9SS_SW_RESET_BIT,
> + core->wrapper_base + WRAPPER_A9SS_SW_RESET);
> + }
> +}
> +
> +int auth_reset_fw_no_tz(struct venus_core *core, phys_addr_t mem_phys,
> + size_t mem_size)
> +{
> + struct iommu_domain *iommu;
> + struct device *dev;
> + int ret;
> +
> + dev = core->fw.dev;
> + if (!dev)
> + return -EPROBE_DEFER;
> +
> + iommu = core->fw.iommu_domain;
> + core->fw.mapped_mem_size = mem_size;
> +
> + ret = iommu_map(iommu, VENUS_FW_START_ADDR, mem_phys, mem_size,
> + IOMMU_READ | IOMMU_WRITE | IOMMU_PRIV, GFP_KERNEL);
> + if (ret) {
> + dev_err(dev, "could not map video firmware region\n");
> + return ret;
> + }
> +
> + reset_cpu_no_tz(core);
> +
> + return 0;
> +}
> +
> +void unload_fw_no_tz(struct venus_core *core)
> +{
> + const size_t mapped = core->fw.mapped_mem_size;
> + struct iommu_domain *iommu;
> + size_t unmapped;
> + u32 reg;
> + struct device *dev = core->fw.dev;
> + void __iomem *wrapper_base = core->wrapper_base;
> + void __iomem *wrapper_tz_base = core->wrapper_tz_base;
> +
> + if (IS_IRIS2_1(core)) {
> + /* Assert the reset to XTSS */
> + reg = readl(wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
> + reg |= WRAPPER_XTSS_SW_RESET_BIT;
> + writel(reg, wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
> + } else {
> + /* Assert the reset to ARM9 */
> + reg = readl(wrapper_base + WRAPPER_A9SS_SW_RESET);
> + reg |= WRAPPER_A9SS_SW_RESET_BIT;
> + writel(reg, wrapper_base + WRAPPER_A9SS_SW_RESET);
> + }
> +
> + iommu = core->fw.iommu_domain;
> +
> + if (core->fw.mapped_mem_size && iommu) {
> + unmapped = iommu_unmap(iommu, VENUS_FW_START_ADDR, mapped);
> +
> + if (unmapped != mapped)
> + dev_err(dev, "failed to unmap firmware\n");
> + else
> + core->fw.mapped_mem_size = 0;
> + }
> +}
> diff --git a/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
> new file mode 100644
> index 0000000..5f008ef
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
> @@ -0,0 +1,19 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2017 Linaro Ltd.
> + */
> +#ifndef __FIRMWARE_NO_TZ_H__
> +#define __FIRMWARE_NO_TZ_H__
> +
> +struct device;
> +
> +#define VENUS_PAS_ID 9
> +
> +int init_fw_no_tz(struct venus_core *core);
> +void deinit_fw_no_tz(struct venus_core *core);
> +int auth_reset_fw_no_tz(struct venus_core *core, phys_addr_t mem_phys,
> + size_t mem_size);
> +void unload_fw_no_tz(struct venus_core *core);
> +void set_hw_state_no_tz(struct venus_core *core, bool resume);
> +
> +#endif
> diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/vcodec/venus/helpers.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/helpers.c
> rename to drivers/media/platform/qcom/vcodec/venus/helpers.c
> diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/vcodec/venus/helpers.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/helpers.h
> rename to drivers/media/platform/qcom/vcodec/venus/helpers.h
> diff --git a/drivers/media/platform/qcom/venus/hfi.c b/drivers/media/platform/qcom/vcodec/venus/hfi.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi.c
> rename to drivers/media/platform/qcom/vcodec/venus/hfi.c
> diff --git a/drivers/media/platform/qcom/venus/hfi.h b/drivers/media/platform/qcom/vcodec/venus/hfi.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi.h
> rename to drivers/media/platform/qcom/vcodec/venus/hfi.h
> diff --git a/drivers/media/platform/qcom/venus/hfi_cmds.c b/drivers/media/platform/qcom/vcodec/venus/hfi_cmds.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_cmds.c
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_cmds.c
> diff --git a/drivers/media/platform/qcom/venus/hfi_cmds.h b/drivers/media/platform/qcom/vcodec/venus/hfi_cmds.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_cmds.h
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_cmds.h
> diff --git a/drivers/media/platform/qcom/venus/hfi_helper.h b/drivers/media/platform/qcom/vcodec/venus/hfi_helper.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_helper.h
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_helper.h
> diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.c b/drivers/media/platform/qcom/vcodec/venus/hfi_msgs.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_msgs.c
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_msgs.c
> diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.h b/drivers/media/platform/qcom/vcodec/venus/hfi_msgs.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_msgs.h
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_msgs.h
> diff --git a/drivers/media/platform/qcom/venus/hfi_parser.c b/drivers/media/platform/qcom/vcodec/venus/hfi_parser.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_parser.c
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_parser.c
> diff --git a/drivers/media/platform/qcom/venus/hfi_parser.h b/drivers/media/platform/qcom/vcodec/venus/hfi_parser.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_parser.h
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_parser.h
> diff --git a/drivers/media/platform/qcom/venus/hfi_plat_bufs.h b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_plat_bufs.h
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
> diff --git a/drivers/media/platform/qcom/venus/hfi_plat_bufs_v6.c b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_plat_bufs_v6.c
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
> diff --git a/drivers/media/platform/qcom/venus/hfi_platform.c b/drivers/media/platform/qcom/vcodec/venus/hfi_platform.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_platform.c
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform.c
> diff --git a/drivers/media/platform/qcom/venus/hfi_platform.h b/drivers/media/platform/qcom/vcodec/venus/hfi_platform.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_platform.h
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform.h
> diff --git a/drivers/media/platform/qcom/venus/hfi_platform_v4.c b/drivers/media/platform/qcom/vcodec/venus/hfi_platform_v4.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_platform_v4.c
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform_v4.c
> diff --git a/drivers/media/platform/qcom/venus/hfi_platform_v6.c b/drivers/media/platform/qcom/vcodec/venus/hfi_platform_v6.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_platform_v6.c
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform_v6.c
> diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
> similarity index 99%
> rename from drivers/media/platform/qcom/venus/hfi_venus.c
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
> index f9437b6..5a68db9 100644
> --- a/drivers/media/platform/qcom/venus/hfi_venus.c
> +++ b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
> @@ -13,11 +13,12 @@
> #include <linux/slab.h>
>
> #include "core.h"
> +#include "../firmware.h"
> #include "hfi_cmds.h"
> #include "hfi_msgs.h"
> #include "hfi_venus.h"
> #include "hfi_venus_io.h"
> -#include "firmware.h"
> +#include "firmware_no_tz.h"
>
> #define HFI_MASK_QHDR_TX_TYPE 0xff000000
> #define HFI_MASK_QHDR_RX_TYPE 0x00ff0000
> @@ -635,7 +636,10 @@ static int venus_power_off(struct venus_hfi_device *hdev)
> if (!hdev->power_enabled)
> return 0;
>
> - ret = venus_set_hw_state_suspend(hdev->core);
> + if (hdev->core->use_tz)
> + ret = set_hw_state(false);
> + else
> + set_hw_state_no_tz(hdev->core, false);
> if (ret)
> return ret;
>
> @@ -655,7 +659,13 @@ static int venus_power_on(struct venus_hfi_device *hdev)
> if (hdev->power_enabled)
> return 0;
>
> - ret = venus_set_hw_state_resume(hdev->core);
> + if (hdev->core->use_tz) {
> + ret = set_hw_state(true);
> + if (ret == -EINVAL)
> + ret = 0;
> + } else {
> + set_hw_state_no_tz(hdev->core, true);
> + }
> if (ret)
> goto err;
>
> @@ -668,7 +678,10 @@ static int venus_power_on(struct venus_hfi_device *hdev)
> return 0;
>
> err_suspend:
> - venus_set_hw_state_suspend(hdev->core);
> + if (hdev->core->use_tz)
> + set_hw_state(false);
> + else
> + set_hw_state_no_tz(hdev->core, false);
> err:
> hdev->power_enabled = false;
> return ret;
> diff --git a/drivers/media/platform/qcom/venus/hfi_venus.h b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_venus.h
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus.h
> diff --git a/drivers/media/platform/qcom/venus/hfi_venus_io.h b/drivers/media/platform/qcom/vcodec/venus/hfi_venus_io.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/hfi_venus_io.h
> rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus_io.h
> diff --git a/drivers/media/platform/qcom/venus/pm_helpers.c b/drivers/media/platform/qcom/vcodec/venus/pm_helpers.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/pm_helpers.c
> rename to drivers/media/platform/qcom/vcodec/venus/pm_helpers.c
> diff --git a/drivers/media/platform/qcom/venus/pm_helpers.h b/drivers/media/platform/qcom/vcodec/venus/pm_helpers.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/pm_helpers.h
> rename to drivers/media/platform/qcom/vcodec/venus/pm_helpers.h
> diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/vcodec/venus/vdec.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/vdec.c
> rename to drivers/media/platform/qcom/vcodec/venus/vdec.c
> diff --git a/drivers/media/platform/qcom/venus/vdec.h b/drivers/media/platform/qcom/vcodec/venus/vdec.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/vdec.h
> rename to drivers/media/platform/qcom/vcodec/venus/vdec.h
> diff --git a/drivers/media/platform/qcom/venus/vdec_ctrls.c b/drivers/media/platform/qcom/vcodec/venus/vdec_ctrls.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/vdec_ctrls.c
> rename to drivers/media/platform/qcom/vcodec/venus/vdec_ctrls.c
> diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/vcodec/venus/venc.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/venc.c
> rename to drivers/media/platform/qcom/vcodec/venus/venc.c
> diff --git a/drivers/media/platform/qcom/venus/venc.h b/drivers/media/platform/qcom/vcodec/venus/venc.h
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/venc.h
> rename to drivers/media/platform/qcom/vcodec/venus/venc.h
> diff --git a/drivers/media/platform/qcom/venus/venc_ctrls.c b/drivers/media/platform/qcom/vcodec/venus/venc_ctrls.c
> similarity index 100%
> rename from drivers/media/platform/qcom/venus/venc_ctrls.c
> rename to drivers/media/platform/qcom/vcodec/venus/venc_ctrls.c
> diff --git a/drivers/media/platform/qcom/venus/firmware.c b/drivers/media/platform/qcom/venus/firmware.c
> deleted file mode 100644
> index fe7da2b..0000000
> --- a/drivers/media/platform/qcom/venus/firmware.c
> +++ /dev/null
> @@ -1,363 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0-only
> -/*
> - * Copyright (C) 2017 Linaro Ltd.
> - */
> -
> -#include <linux/device.h>
> -#include <linux/firmware.h>
> -#include <linux/kernel.h>
> -#include <linux/iommu.h>
> -#include <linux/io.h>
> -#include <linux/of.h>
> -#include <linux/of_address.h>
> -#include <linux/of_reserved_mem.h>
> -#include <linux/platform_device.h>
> -#include <linux/of_device.h>
> -#include <linux/firmware/qcom/qcom_scm.h>
> -#include <linux/sizes.h>
> -#include <linux/soc/qcom/mdt_loader.h>
> -
> -#include "core.h"
> -#include "firmware.h"
> -#include "hfi_venus_io.h"
> -
> -#define VENUS_PAS_ID 9
> -#define VENUS_FW_MEM_SIZE (6 * SZ_1M)
> -#define VENUS_FW_START_ADDR 0x0
> -
> -static void venus_reset_cpu(struct venus_core *core)
> -{
> - u32 fw_size = core->fw.mapped_mem_size;
> - void __iomem *wrapper_base;
> -
> - if (IS_IRIS2_1(core))
> - wrapper_base = core->wrapper_tz_base;
> - else
> - wrapper_base = core->wrapper_base;
> -
> - writel(0, wrapper_base + WRAPPER_FW_START_ADDR);
> - writel(fw_size, wrapper_base + WRAPPER_FW_END_ADDR);
> - writel(0, wrapper_base + WRAPPER_CPA_START_ADDR);
> - writel(fw_size, wrapper_base + WRAPPER_CPA_END_ADDR);
> - writel(fw_size, wrapper_base + WRAPPER_NONPIX_START_ADDR);
> - writel(fw_size, wrapper_base + WRAPPER_NONPIX_END_ADDR);
> -
> - if (IS_IRIS2_1(core)) {
> - /* Bring XTSS out of reset */
> - writel(0, wrapper_base + WRAPPER_TZ_XTSS_SW_RESET);
> - } else {
> - writel(0x0, wrapper_base + WRAPPER_CPU_CGC_DIS);
> - writel(0x0, wrapper_base + WRAPPER_CPU_CLOCK_CONFIG);
> -
> - /* Bring ARM9 out of reset */
> - writel(0, wrapper_base + WRAPPER_A9SS_SW_RESET);
> - }
> -}
> -
> -int venus_set_hw_state(struct venus_core *core, bool resume)
> -{
> - int ret;
> -
> - if (core->use_tz) {
> - ret = qcom_scm_set_remote_state(resume, 0);
> - if (resume && ret == -EINVAL)
> - ret = 0;
> - return ret;
> - }
> -
> - if (resume) {
> - venus_reset_cpu(core);
> - } else {
> - if (IS_IRIS2_1(core))
> - writel(WRAPPER_XTSS_SW_RESET_BIT,
> - core->wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
> - else
> - writel(WRAPPER_A9SS_SW_RESET_BIT,
> - core->wrapper_base + WRAPPER_A9SS_SW_RESET);
> - }
> -
> - return 0;
> -}
> -
> -static int venus_load_fw(struct venus_core *core, const char *fwname,
> - phys_addr_t *mem_phys, size_t *mem_size)
> -{
> - const struct firmware *mdt;
> - struct reserved_mem *rmem;
> - struct device_node *node;
> - struct device *dev;
> - ssize_t fw_size;
> - void *mem_va;
> - int ret;
> -
> - *mem_phys = 0;
> - *mem_size = 0;
> -
> - dev = core->dev;
> - node = of_parse_phandle(dev->of_node, "memory-region", 0);
> - if (!node) {
> - dev_err(dev, "no memory-region specified\n");
> - return -EINVAL;
> - }
> -
> - rmem = of_reserved_mem_lookup(node);
> - of_node_put(node);
> - if (!rmem) {
> - dev_err(dev, "failed to lookup reserved memory-region\n");
> - return -EINVAL;
> - }
> -
> - ret = request_firmware(&mdt, fwname, dev);
> - if (ret < 0)
> - return ret;
> -
> - fw_size = qcom_mdt_get_size(mdt);
> - if (fw_size < 0) {
> - ret = fw_size;
> - goto err_release_fw;
> - }
> -
> - *mem_phys = rmem->base;
> - *mem_size = rmem->size;
> -
> - if (*mem_size < fw_size || fw_size > VENUS_FW_MEM_SIZE) {
> - ret = -EINVAL;
> - goto err_release_fw;
> - }
> -
> - mem_va = memremap(*mem_phys, *mem_size, MEMREMAP_WC);
> - if (!mem_va) {
> - dev_err(dev, "unable to map memory region %pa size %#zx\n", mem_phys, *mem_size);
> - ret = -ENOMEM;
> - goto err_release_fw;
> - }
> -
> - if (core->use_tz)
> - ret = qcom_mdt_load(dev, mdt, fwname, VENUS_PAS_ID,
> - mem_va, *mem_phys, *mem_size, NULL);
> - else
> - ret = qcom_mdt_load_no_init(dev, mdt, fwname, VENUS_PAS_ID,
> - mem_va, *mem_phys, *mem_size, NULL);
> -
> - memunmap(mem_va);
> -err_release_fw:
> - release_firmware(mdt);
> - return ret;
> -}
> -
> -static int venus_boot_no_tz(struct venus_core *core, phys_addr_t mem_phys,
> - size_t mem_size)
> -{
> - struct iommu_domain *iommu;
> - struct device *dev;
> - int ret;
> -
> - dev = core->fw.dev;
> - if (!dev)
> - return -EPROBE_DEFER;
> -
> - iommu = core->fw.iommu_domain;
> - core->fw.mapped_mem_size = mem_size;
> -
> - ret = iommu_map(iommu, VENUS_FW_START_ADDR, mem_phys, mem_size,
> - IOMMU_READ | IOMMU_WRITE | IOMMU_PRIV, GFP_KERNEL);
> - if (ret) {
> - dev_err(dev, "could not map video firmware region\n");
> - return ret;
> - }
> -
> - venus_reset_cpu(core);
> -
> - return 0;
> -}
> -
> -static int venus_shutdown_no_tz(struct venus_core *core)
> -{
> - const size_t mapped = core->fw.mapped_mem_size;
> - struct iommu_domain *iommu;
> - size_t unmapped;
> - u32 reg;
> - struct device *dev = core->fw.dev;
> - void __iomem *wrapper_base = core->wrapper_base;
> - void __iomem *wrapper_tz_base = core->wrapper_tz_base;
> -
> - if (IS_IRIS2_1(core)) {
> - /* Assert the reset to XTSS */
> - reg = readl(wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
> - reg |= WRAPPER_XTSS_SW_RESET_BIT;
> - writel(reg, wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
> - } else {
> - /* Assert the reset to ARM9 */
> - reg = readl(wrapper_base + WRAPPER_A9SS_SW_RESET);
> - reg |= WRAPPER_A9SS_SW_RESET_BIT;
> - writel(reg, wrapper_base + WRAPPER_A9SS_SW_RESET);
> - }
> -
> - iommu = core->fw.iommu_domain;
> -
> - if (core->fw.mapped_mem_size && iommu) {
> - unmapped = iommu_unmap(iommu, VENUS_FW_START_ADDR, mapped);
> -
> - if (unmapped != mapped)
> - dev_err(dev, "failed to unmap firmware\n");
> - else
> - core->fw.mapped_mem_size = 0;
> - }
> -
> - return 0;
> -}
> -
> -int venus_boot(struct venus_core *core)
> -{
> - struct device *dev = core->dev;
> - const struct venus_resources *res = core->res;
> - const char *fwpath = NULL;
> - phys_addr_t mem_phys;
> - size_t mem_size;
> - int ret;
> -
> - if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
> - (core->use_tz && !qcom_scm_is_available()))
> - return -EPROBE_DEFER;
> -
> - ret = of_property_read_string_index(dev->of_node, "firmware-name", 0,
> - &fwpath);
> - if (ret)
> - fwpath = core->res->fwname;
> -
> - ret = venus_load_fw(core, fwpath, &mem_phys, &mem_size);
> - if (ret) {
> - dev_err(dev, "fail to load video firmware\n");
> - return -EINVAL;
> - }
> -
> - core->fw.mem_size = mem_size;
> - core->fw.mem_phys = mem_phys;
> -
> - if (core->use_tz)
> - ret = qcom_scm_pas_auth_and_reset(VENUS_PAS_ID);
> - else
> - ret = venus_boot_no_tz(core, mem_phys, mem_size);
> -
> - if (ret)
> - return ret;
> -
> - if (core->use_tz && res->cp_size) {
> - /*
> - * Clues for porting using downstream data:
> - * cp_start = 0
> - * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and not size!
> - * This works, as the non-secure context bank is placed
> - * contiguously right after the Content Protection region.
> - *
> - * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0]
> - * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1]
> - */
> - ret = qcom_scm_mem_protect_video_var(res->cp_start,
> - res->cp_size,
> - res->cp_nonpixel_start,
> - res->cp_nonpixel_size);
> - if (ret) {
> - qcom_scm_pas_shutdown(VENUS_PAS_ID);
> - dev_err(dev, "set virtual address ranges fail (%d)\n",
> - ret);
> - return ret;
> - }
> - }
> -
> - return 0;
> -}
> -
> -int venus_shutdown(struct venus_core *core)
> -{
> - int ret;
> -
> - if (core->use_tz)
> - ret = qcom_scm_pas_shutdown(VENUS_PAS_ID);
> - else
> - ret = venus_shutdown_no_tz(core);
> -
> - return ret;
> -}
> -
> -int venus_firmware_init(struct venus_core *core)
> -{
> - struct platform_device_info info;
> - struct iommu_domain *iommu_dom;
> - struct platform_device *pdev;
> - struct device_node *np;
> - int ret;
> -
> - np = of_get_child_by_name(core->dev->of_node, "video-firmware");
> - if (!np) {
> - core->use_tz = true;
> - return 0;
> - }
> -
> - memset(&info, 0, sizeof(info));
> - info.fwnode = &np->fwnode;
> - info.parent = core->dev;
> - info.name = np->name;
> - info.dma_mask = DMA_BIT_MASK(32);
> -
> - pdev = platform_device_register_full(&info);
> - if (IS_ERR(pdev)) {
> - of_node_put(np);
> - return PTR_ERR(pdev);
> - }
> -
> - pdev->dev.of_node = np;
> -
> - ret = of_dma_configure(&pdev->dev, np, true);
> - if (ret) {
> - dev_err(core->dev, "dma configure fail\n");
> - goto err_unregister;
> - }
> -
> - core->fw.dev = &pdev->dev;
> -
> - iommu_dom = iommu_domain_alloc(&platform_bus_type);
> - if (!iommu_dom) {
> - dev_err(core->fw.dev, "Failed to allocate iommu domain\n");
> - ret = -ENOMEM;
> - goto err_unregister;
> - }
> -
> - ret = iommu_attach_device(iommu_dom, core->fw.dev);
> - if (ret) {
> - dev_err(core->fw.dev, "could not attach device\n");
> - goto err_iommu_free;
> - }
> -
> - core->fw.iommu_domain = iommu_dom;
> -
> - of_node_put(np);
> -
> - return 0;
> -
> -err_iommu_free:
> - iommu_domain_free(iommu_dom);
> -err_unregister:
> - platform_device_unregister(pdev);
> - of_node_put(np);
> - return ret;
> -}
> -
> -void venus_firmware_deinit(struct venus_core *core)
> -{
> - struct iommu_domain *iommu;
> -
> - if (!core->fw.dev)
> - return;
> -
> - iommu = core->fw.iommu_domain;
> -
> - iommu_detach_device(iommu, core->fw.dev);
> -
> - if (core->fw.iommu_domain) {
> - iommu_domain_free(iommu);
> - core->fw.iommu_domain = NULL;
> - }
> -
> - platform_device_unregister(to_platform_device(core->fw.dev));
> -}
> diff --git a/drivers/media/platform/qcom/venus/firmware.h b/drivers/media/platform/qcom/venus/firmware.h
> deleted file mode 100644
> index aaccd84..0000000
> --- a/drivers/media/platform/qcom/venus/firmware.h
> +++ /dev/null
> @@ -1,26 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0-only */
> -/*
> - * Copyright (C) 2017 Linaro Ltd.
> - */
> -#ifndef __VENUS_FIRMWARE_H__
> -#define __VENUS_FIRMWARE_H__
> -
> -struct device;
> -
> -int venus_firmware_init(struct venus_core *core);
> -void venus_firmware_deinit(struct venus_core *core);
> -int venus_boot(struct venus_core *core);
> -int venus_shutdown(struct venus_core *core);
> -int venus_set_hw_state(struct venus_core *core, bool suspend);
> -
> -static inline int venus_set_hw_state_suspend(struct venus_core *core)
> -{
> - return venus_set_hw_state(core, false);
> -}
> -
> -static inline int venus_set_hw_state_resume(struct venus_core *core)
> -{
> - return venus_set_hw_state(core, true);
> -}
> -
> -#endif

--
With best wishes
Dmitry

2023-12-18 18:38:21

[permalink] [raw]

Subject: Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver

On 18/12/2023 13:31, Dikshita Agarwal wrote:
> This patch series introduces support for Qualcomm new video acceleration
> hardware architecture, used for video stream decoding/encoding. This driver
> is based on new communication protocol between video hardware and application
> processor.

This doesn't answer one important point, you have been asked for v1.
What is the actual change point between Venus and Iris? What has been
changed so much that it demands a separate driver. This is the main
question for the cover letter, which has not been answered so far.

From what I see from you bindings, the hardware is pretty close to what
we see in the latest venus generations. I asssme that there was a change
in the vcodec inteface to the firmware and other similar changes. Could
you please point out, which parts of Venus driver do no longer work or
are not applicable for sm8550?

> This driver comes with below capabilities:
> - V4L2 complaint video driver with M2M and STREAMING capability.
> - Supports H264, H265, VP9 decoders.
> - Supports H264, H265 encoders.
> This driver comes with below features:
> - Centralized resource and memory management.
> - Centralized management of core and instance states.
> - Defines platform specific capabilities and features. As a results, it provides
> a single point of control to enable/disable a given feature depending on
> specific platform capabilities.
> - Handles hardware interdependent configurations. For a given configuration from
> client, the driver checks for hardware dependent configuration/s and updates
> the same.
> - Handles multiple complex video scenarios involving state transitions - DRC,
> Drain, Seek, back to back DRC, DRC during Drain sequence, DRC during Seek
> sequence.
> - Introduces a flexible way for driver to subscribe for any property with
> hardware. Hardware would inform driver with those subscribed property with any
> change in value.
> - Introduces performance (clock and bus) model based on new hardware
> architecture.
> - Introduces multi thread safe design to handle communication between client and
> hardware.
> - Adapts encoder quality improvements available in new hardware architecture.
> - Implements asynchronous communication with hardware to achieve better
> experience in low latency usecases.
> - Supports multi stage hardware architecture for encode/decode.
> - Output and capture planes are controlled independently. Thereby providing a
> way to reconfigure individual plane.
> - Hardware packetization layer supports synchronization between configuration
> packet and data packet.
> - Introduces a flexibility to receive a hardware response for a given command
> packet.
> - Native hardware support of LAST flag which is mandatory to align with port
> reconfiguration and DRAIN sequence as per V4L guidelines.
> - Native hardware support for drain sequence.
>
> Changes done since v1[1]:
> - Patches are created as logical split instead of file by file.
> - Extracted common functionality between venus and iris driver and placed them
> under common folder vcodec.
> - Flattened directory structure.
> - Restructured the code in a simplified layer architecture.
> - Implemented runtime PM, and dropped explicit power collapse thread in driver.
> - Moved to standard kernel log print api.
> - Simplified the bus and clock calculation logic.
> - Removed debug features like dma tracker, frame stats, debugfs.
> - Merged v4l2 and vidc layer as vidc in driver.
> - Removed separate probe for context bank.
> - Use of_device_get_match_data to get platform data.
> - Replaced complex macros with inline functions.
> - Migrated venus to iris names.
> - Addressed many other comments received on [1].
>
> [1]: https://patchwork.kernel.org/project/linux-media/list/?series=770581
>
> Patch 1, adds dt binding for iris driver.
>
> Patches 2-4, are introducing vcodec folder and moving common APIs like fw
> load/unload, buffer size calcualtions, read/write to shared queues to common
> files which are being used by both venus and iris drivers.
> This also moves the venus driver folder to vcodec folder.
>
> Patch 5, adds the maintainers for iris driver.
>
> Patch 6-29, adds core iris driver and enable decoder.
>
> Patch 30-34, enable encoder functionality in iris driver.
>
> Static tools like checkpatch, smatch, dt_binding_check, sparse and Coccinelle
> run successfully with this driver.
>
> The output of v4l2-compliance test looks like:
>
> v4l2-compliance SHA: not available, 64 bits
>
> Cannot open device /dev/vido0, exiting.
> root@qemuarm64:/usr/bin# ./v4l2-compliance -d /dev/video0
> v4l2-compliance SHA: not available, 64 bits
>
> Compliance test for iris_driver device /dev/video0:
>
> Driver Info:
> Driver name : iris_driver
> Card type : iris_decoder
> Bus info : platform:iris_bus
> Driver version : 6.6.0
> Capabilities : 0x84204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x04204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
>
> Required ioctls:
> test VIDIOC_QUERYCAP: OK
>
> Allow for multiple opens:
> test second /dev/video0 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> fail: ../../../v4l-utils-1.18.1/utils/v4l2-compliance/v4l2-compliance.cpp(724): !ok
> test for unlimited opens: FAIL
>
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 0 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls:
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> test VIDIOC_QUERYCTRL: OK
> test VIDIOC_G/S_CTRL: OK
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 48 Private Controls: 0
>
> Format ioctls:
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK (Not Supported)
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> test VIDIOC_S_FMT: OK
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> test Cropping: OK
> test Composing: OK
> test Scaling: OK
>
> Codec ioctls:
> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK
>
> Buffer ioctls:
> testVIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> test VIDIOC_EXPBUF: OK (Not Supported)
> test Requests: OK (Not Supported)
>
> Total for iris_driver device /dev/video0: 44, Succeeded: 43, Failed: 1, Warnings: 0
>
> Compliance test for iris_driver device /dev/video1:
>
> Driver Info:
> Driver name : iris_driver
> Card type : iris_encoder
> Bus info : platform:iris_bus
> Driver version : 6.6.0
> Capabilities : 0x84204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x04204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
>
> Required ioctls:
> test VIDIOC_QUERYCAP: OK
>
> Allow for multiple opens:
> test second /dev/video1 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> fail: ../../../v4l-utils-1.18.1/utils/v4l2-compliance/v4l2-compliance.cpp(724): !ok
> test for unlimited opens: FAIL
>
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 0 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls:
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> test VIDIOC_QUERYCTRL: OK
> test VIDIOC_G/S_CTRL: OK
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 48 Private Controls: 0
>
> Format ioctls:
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> test VIDIOC_S_FMT: OK
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> test Cropping: OK
> test Composing: OK
> test Scaling: OK (Not Supported)
>
> Codec ioctls:
> test VIDIOC_(TRY_)ENCODER_CMD: OK
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>
> Buffer ioctls:
> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> test VIDIOC_EXPBUF: OK (Not Supported)
> test Requests: OK (Not Supported)
>
> Total for iris_driver device /dev/video1: 44, Succeeded: 43, Failed: 1, Warnings: 0
>
> Dikshita Agarwal (27):
> media: introduce common helpers for video firmware handling
> media: introduce common helpers for queues handling
> media: introduce common helpers for buffer size calculation
> dt-bindings: media: Add sm8550 dt schema
> media: MAINTAINERS: Add Qualcomm Iris video accelerator driver
> media: iris: register video device to platform driver
> media: iris: initialize power resources
> media: iris: introduce state machine for iris core
> media: iris: initialize shared queues for host and firmware
> communication
> media: iris: add PIL functionality for video firmware
> media: iris: introduce packetization layer for creating HFI packets
> media: iris: add video processing unit(VPU) specific register handling
> media: iris: introduce platform specific capabilities for core and
> instance
> media: iris: add handling for interrupt service routine(ISR) invoked
> by hardware
> media: iris: implement iris v4l2_ctrl_ops and prepare capabilities
> media: iris: implement vb2_ops queue setup
> media: iris: implement HFI to queue and release buffers
> media: iris: add video hardware internal buffer count and size
> calculation
> media: iris: implement internal buffer management
> media: iris: introduce instance states
> media: iris: implement vb2 streaming ops on capture and output planes
> media: iris: implement vb2 ops for buf_queue and firmware response
> media: iris: register video encoder device to platform driver
> media: iris: add platform specific instance capabilities for encoder
> media: iris: implement iris v4l2 ioctl ops supported by encoder
> media: iris: add vb2 streaming and buffer ops for encoder
> media: iris: add power management for encoder
>
> Vikash Garodia (7):
> media: iris: implement iris v4l2 file ops
> media: iris: introduce and implement iris vb2 mem ops
> media: iris: implement iris v4l2 ioctl ops supported by decoder
> media: iris: subscribe input and output properties to firmware
> media: iris: subscribe src change and handle firmware responses
> media: iris: add instance sub states and implement DRC and Drain
> sequence
> media: iris: implement power management
>
> .../bindings/media/qcom,sm8550-iris.yaml | 177 ++
> MAINTAINERS | 11 +
> drivers/media/platform/qcom/Kconfig | 3 +-
> drivers/media/platform/qcom/Makefile | 3 +-
> drivers/media/platform/qcom/vcodec/buffers.c | 103 ++
> drivers/media/platform/qcom/vcodec/buffers.h | 15 +
> drivers/media/platform/qcom/vcodec/firmware.c | 147 ++
> drivers/media/platform/qcom/vcodec/firmware.h | 21 +
> drivers/media/platform/qcom/vcodec/hfi_queue.c | 258 +++
> drivers/media/platform/qcom/vcodec/hfi_queue.h | 129 ++
> drivers/media/platform/qcom/vcodec/iris/Kconfig | 13 +
> drivers/media/platform/qcom/vcodec/iris/Makefile | 26 +
> .../media/platform/qcom/vcodec/iris/hfi_defines.h | 386 +++++
> .../media/platform/qcom/vcodec/iris/iris_buffer.c | 833 +++++++++
> .../media/platform/qcom/vcodec/iris/iris_buffer.h | 65 +
> .../media/platform/qcom/vcodec/iris/iris_common.h | 147 ++
> .../media/platform/qcom/vcodec/iris/iris_core.c | 110 ++
> .../media/platform/qcom/vcodec/iris/iris_core.h | 121 ++
> .../media/platform/qcom/vcodec/iris/iris_ctrls.c | 1782 ++++++++++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_ctrls.h | 71 +
> .../media/platform/qcom/vcodec/iris/iris_helpers.c | 1099 ++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_helpers.h | 68 +
> drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 917 ++++++++++
> drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 47 +
> .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 729 ++++++++
> .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 118 ++
> .../platform/qcom/vcodec/iris/iris_hfi_response.c | 1097 ++++++++++++
> .../platform/qcom/vcodec/iris/iris_hfi_response.h | 20 +
> .../platform/qcom/vcodec/iris/iris_instance.h | 106 ++
> .../media/platform/qcom/vcodec/iris/iris_power.c | 178 ++
> .../media/platform/qcom/vcodec/iris/iris_power.h | 15 +
> .../media/platform/qcom/vcodec/iris/iris_probe.c | 357 ++++
> .../media/platform/qcom/vcodec/iris/iris_state.c | 453 +++++
> .../media/platform/qcom/vcodec/iris/iris_state.h | 78 +
> drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 457 +++++
> drivers/media/platform/qcom/vcodec/iris/iris_vb2.h | 28 +
> .../media/platform/qcom/vcodec/iris/iris_vdec.c | 1211 +++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_vdec.h | 25 +
> .../media/platform/qcom/vcodec/iris/iris_venc.c | 948 +++++++++++
> .../media/platform/qcom/vcodec/iris/iris_venc.h | 27 +
> .../media/platform/qcom/vcodec/iris/iris_vidc.c | 1419 ++++++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_vidc.h | 15 +
> .../platform/qcom/vcodec/iris/platform_common.c | 29 +
> .../platform/qcom/vcodec/iris/platform_common.h | 323 ++++
> .../platform/qcom/vcodec/iris/platform_sm8550.c | 1190 +++++++++++++
> .../media/platform/qcom/vcodec/iris/resources.c | 506 ++++++
> .../media/platform/qcom/vcodec/iris/resources.h | 44 +
> .../media/platform/qcom/vcodec/iris/vpu_common.c | 158 ++
> .../media/platform/qcom/vcodec/iris/vpu_common.h | 52 +
> .../media/platform/qcom/vcodec/iris/vpu_iris3.c | 568 +++++++
> .../media/platform/qcom/vcodec/iris/vpu_iris3.h | 13 +
> .../platform/qcom/vcodec/iris/vpu_iris3_buffer.c | 395 +++++
> .../platform/qcom/vcodec/iris/vpu_iris3_buffer.h | 1469 ++++++++++++++++
> .../platform/qcom/vcodec/iris/vpu_iris3_power.c | 148 ++
> .../platform/qcom/vcodec/iris/vpu_iris3_power.h | 13 +
> .../media/platform/qcom/{ => vcodec}/venus/Kconfig | 0
> .../platform/qcom/{ => vcodec}/venus/Makefile | 5 +-
> .../media/platform/qcom/{ => vcodec}/venus/core.c | 102 +-
> .../media/platform/qcom/{ => vcodec}/venus/core.h | 0
> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.h | 0
> .../platform/qcom/vcodec/venus/firmware_no_tz.c | 194 +++
> .../platform/qcom/vcodec/venus/firmware_no_tz.h | 19 +
> .../platform/qcom/{ => vcodec}/venus/helpers.c | 172 +-
> .../platform/qcom/{ => vcodec}/venus/helpers.h | 2 +-
> .../media/platform/qcom/{ => vcodec}/venus/hfi.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/hfi.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_helper.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_parser.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_parser.h | 0
> .../qcom/{ => vcodec}/venus/hfi_plat_bufs.h | 4 +-
> .../qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c | 10 +-
> .../qcom/{ => vcodec}/venus/hfi_platform.c | 0
> .../qcom/{ => vcodec}/venus/hfi_platform.h | 0
> .../qcom/{ => vcodec}/venus/hfi_platform_v4.c | 0
> .../qcom/{ => vcodec}/venus/hfi_platform_v6.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_venus.c | 521 +-----
> .../platform/qcom/{ => vcodec}/venus/hfi_venus.h | 0
> .../qcom/{ => vcodec}/venus/hfi_venus_io.h | 0
> .../platform/qcom/{ => vcodec}/venus/pm_helpers.c | 0
> .../platform/qcom/{ => vcodec}/venus/pm_helpers.h | 0
> .../media/platform/qcom/{ => vcodec}/venus/vdec.c | 5 +-
> .../media/platform/qcom/{ => vcodec}/venus/vdec.h | 0
> .../platform/qcom/{ => vcodec}/venus/vdec_ctrls.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/venc.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/venc.h | 0
> .../platform/qcom/{ => vcodec}/venus/venc_ctrls.c | 0
> drivers/media/platform/qcom/venus/firmware.c | 363 ----
> drivers/media/platform/qcom/venus/firmware.h | 26 -
> 93 files changed, 19175 insertions(+), 989 deletions(-)
> create mode 100644 Documentation/devicetree/bindings/media/qcom,sm8550-iris.yaml
> create mode 100644 drivers/media/platform/qcom/vcodec/buffers.c
> create mode 100644 drivers/media/platform/qcom/vcodec/buffers.h
> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.c
> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.h
> create mode 100644 drivers/media/platform/qcom/vcodec/hfi_queue.c
> create mode 100644 drivers/media/platform/qcom/vcodec/hfi_queue.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/Kconfig
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/Makefile
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_common.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_core.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_core.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_instance.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_power.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_power.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_probe.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_state.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_state.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_venc.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_venc.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_common.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_common.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/resources.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/resources.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_common.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_common.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.h
> rename drivers/media/platform/qcom/{ => vcodec}/venus/Kconfig (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/Makefile (75%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.c (91%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.h (100%)
> create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
> create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.c (90%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.h (98%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_helper.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs.h (94%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c (99%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v4.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v6.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.c (73%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus_io.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.c (99%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec_ctrls.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc_ctrls.c (100%)
> delete mode 100644 drivers/media/platform/qcom/venus/firmware.c
> delete mode 100644 drivers/media/platform/qcom/venus/firmware.h
>

--
With best wishes
Dmitry

2023-12-18 18:40:26

[permalink] [raw]

Subject: Re: [PATCH v2 06/34] media: iris: register video device to platform driver

On 18/12/2023 13:32, Dikshita Agarwal wrote:
> Iris is a multi pipe based video acceleration hardware
> block that offloads video stream encoding and decoding
> from the application processor (AP). It supports H.264
> and H.265 encoding and decoding, as well as VP9 decoding.
> The AP communicates with hardware through a well defined
> protocol which provides fine-grained and asynchronous
> control over individual hardware features.
>
> This patch introduces basic probe and remove functions.

Documentation/process/submitting-patches.rst

Other than that LGTM

> It handles setting up a video device as well as registering
> it with the V4L2 subsystem.
>
> Signed-off-by: Dikshita Agarwal <[email protected]>
> ---
> drivers/media/platform/qcom/Kconfig | 1 +
> drivers/media/platform/qcom/Makefile | 1 +
> drivers/media/platform/qcom/vcodec/iris/Kconfig | 13 +++
> drivers/media/platform/qcom/vcodec/iris/Makefile | 3 +
> .../media/platform/qcom/vcodec/iris/iris_core.h | 34 +++++++
> .../media/platform/qcom/vcodec/iris/iris_probe.c | 111 +++++++++++++++++++++
> 6 files changed, 163 insertions(+)
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/Kconfig
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/Makefile
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_core.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_probe.c
>
> diff --git a/drivers/media/platform/qcom/Kconfig b/drivers/media/platform/qcom/Kconfig
> index e94142f..7c88837 100644
> --- a/drivers/media/platform/qcom/Kconfig
> +++ b/drivers/media/platform/qcom/Kconfig
> @@ -4,3 +4,4 @@ comment "Qualcomm media platform drivers"
>
> source "drivers/media/platform/qcom/camss/Kconfig"
> source "drivers/media/platform/qcom/vcodec/venus/Kconfig"
> +source "drivers/media/platform/qcom/vcodec/iris/Kconfig"
> diff --git a/drivers/media/platform/qcom/Makefile b/drivers/media/platform/qcom/Makefile
> index 3d2d82b..3c76969 100644
> --- a/drivers/media/platform/qcom/Makefile
> +++ b/drivers/media/platform/qcom/Makefile
> @@ -1,3 +1,4 @@
> # SPDX-License-Identifier: GPL-2.0-only
> obj-y += camss/
> obj-y += vcodec/venus/
> +obj-y += vcodec/iris/
> diff --git a/drivers/media/platform/qcom/vcodec/iris/Kconfig b/drivers/media/platform/qcom/vcodec/iris/Kconfig
> new file mode 100644
> index 0000000..850a5b4
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/iris/Kconfig
> @@ -0,0 +1,13 @@
> +config VIDEO_QCOM_IRIS
> + tristate "Qualcomm Iris V4L2 encoder/decoder driver"
> + depends on VIDEO_DEV
> + depends on ARCH_QCOM
> + select QCOM_MDT_LOADER if ARCH_QCOM
> + select QCOM_SCM
> + select DMABUF_HEAPS
> + select DMABUF_HEAPS_SYSTEM
> + help
> + This is a V4L2 driver for Qualcomm Iris video accelerator
> + hardware. It accelerates encoding and decoding operations
> + on various Qualcomm SoCs.
> + To compile this driver as a module choose m here.
> diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
> new file mode 100644
> index 0000000..5536ae0
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
> @@ -0,0 +1,3 @@
> +iris-objs += iris_probe.o
> +
> +obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o
> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
> new file mode 100644
> index 0000000..ab7fcee
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
> @@ -0,0 +1,34 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
> + */
> +
> +#ifndef _IRIS_CORE_H_
> +#define _IRIS_CORE_H_
> +
> +#include <linux/types.h>
> +#include <media/v4l2-device.h>
> +
> +/**
> + * struct iris_core - holds core parameters valid for all instances
> + *
> + * @dev: reference to device structure
> + * @reg_base: IO memory base address
> + * @irq: iris irq
> + * @v4l2_dev: a holder for v4l2 device structure
> + * @vdev_dec: iris video device structure for decoder
> + * @v4l2_file_ops: iris v4l2 file ops
> + * @v4l2_ioctl_ops: iris v4l2 ioctl ops
> + */
> +
> +struct iris_core {
> + struct device *dev;
> + void __iomem *reg_base;
> + int irq;
> + struct v4l2_device v4l2_dev;
> + struct video_device *vdev_dec;
> + const struct v4l2_file_operations *v4l2_file_ops;
> + const struct v4l2_ioctl_ops *v4l2_ioctl_ops;
> +};
> +
> +#endif
> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
> new file mode 100644
> index 0000000..2e93118
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
> @@ -0,0 +1,111 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
> + */
> +
> +#include <linux/module.h>
> +#include <linux/of_device.h>
> +#include <linux/platform_device.h>
> +
> +#include "iris_core.h"
> +
> +static int iris_register_video_device(struct iris_core *core)
> +{
> + struct video_device *vdev;
> + int ret;
> +
> + vdev = video_device_alloc();
> + if (!vdev)
> + return -ENOMEM;
> +
> + strscpy(vdev->name, "qcom-iris-decoder", sizeof(vdev->name));
> + vdev->release = video_device_release;
> + vdev->fops = core->v4l2_file_ops;
> + vdev->ioctl_ops = core->v4l2_ioctl_ops;
> + vdev->vfl_dir = VFL_DIR_M2M;
> + vdev->v4l2_dev = &core->v4l2_dev;
> + vdev->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
> +
> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> + if (ret)
> + goto err_vdev_release;
> +
> + core->vdev_dec = vdev;
> + video_set_drvdata(vdev, core);
> +
> + return ret;
> +
> +err_vdev_release:
> + video_device_release(vdev);
> +
> + return ret;
> +}
> +
> +static void iris_remove(struct platform_device *pdev)
> +{
> + struct iris_core *core;
> +
> + core = platform_get_drvdata(pdev);
> + if (!core)
> + return;
> +
> + video_unregister_device(core->vdev_dec);
> +
> + v4l2_device_unregister(&core->v4l2_dev);
> +}
> +
> +static int iris_probe(struct platform_device *pdev)
> +{
> + struct device *dev = &pdev->dev;
> + struct iris_core *core;
> + int ret;
> +
> + core = devm_kzalloc(&pdev->dev, sizeof(*core), GFP_KERNEL);
> + if (!core)
> + return -ENOMEM;
> + core->dev = dev;
> +
> + core->reg_base = devm_platform_ioremap_resource(pdev, 0);
> + if (IS_ERR(core->reg_base))
> + return PTR_ERR(core->reg_base);
> +
> + core->irq = platform_get_irq(pdev, 0);
> + if (core->irq < 0)
> + return core->irq;
> +
> + ret = v4l2_device_register(dev, &core->v4l2_dev);
> + if (ret)
> + return ret;
> +
> + ret = iris_register_video_device(core);
> + if (ret)
> + goto err_v4l2_unreg;
> +
> + platform_set_drvdata(pdev, core);
> +
> + return ret;
> +
> +err_v4l2_unreg:
> + v4l2_device_unregister(&core->v4l2_dev);
> +
> + return ret;
> +}
> +
> +static const struct of_device_id iris_dt_match[] = {
> + { .compatible = "qcom,sm8550-iris", },
> + { },
> +};
> +MODULE_DEVICE_TABLE(of, iris_dt_match);
> +
> +static struct platform_driver qcom_iris_driver = {
> + .probe = iris_probe,
> + .remove_new = iris_remove,
> + .driver = {
> + .name = "qcom-iris",
> + .of_match_table = iris_dt_match,
> + },
> +};
> +
> +module_platform_driver(qcom_iris_driver);
> +MODULE_DESCRIPTION("Qualcomm Iris video driver");
> +MODULE_LICENSE("GPL");

--
With best wishes
Dmitry

2023-12-18 18:47:14

[permalink] [raw]

Subject: Re: [PATCH v2 09/34] media: iris: initialize shared queues for host and firmware communication

On 18/12/2023 13:32, Dikshita Agarwal wrote:
> Shared queues are used for communication between driver and firmware.
> There are 3 types of queues:
> Command queue - driver to write any command to firmware.
> Message queue - firmware to send any response to driver.
> Debug queue - firmware to write debug message.
>
> Above queues are initialized and configured to firmware during probe.
>
> Signed-off-by: Dikshita Agarwal <[email protected]>
> ---
> drivers/media/platform/qcom/vcodec/iris/Makefile | 2 ++
> .../media/platform/qcom/vcodec/iris/iris_core.h | 11 ++++++++
> .../media/platform/qcom/vcodec/iris/iris_probe.c | 31 ++++++++++++++++++++++
> 3 files changed, 44 insertions(+)
>
> diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
> index 12a74de..59798e5d 100644
> --- a/drivers/media/platform/qcom/vcodec/iris/Makefile
> +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
> @@ -1,3 +1,5 @@
> +iris-objs += ../hfi_queue.o
> +
> iris-objs += iris_probe.o \
> resources.o \
> iris_state.o
> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
> index 56a5b7d..77124f9 100644
> --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
> @@ -9,6 +9,7 @@
> #include <linux/types.h>
> #include <media/v4l2-device.h>
>
> +#include "../hfi_queue.h"
> #include "iris_state.h"
>
> /**
> @@ -30,6 +31,11 @@
> * @reset_tbl: table of iris reset clocks
> * @reset_count: count of iris reset clocks
> * @state: current state of core
> + * @iface_q_table: Interface queue table memory
> + * @command_queue: shared interface queue to send commands to firmware
> + * @message_queue: shared interface queue to receive responses from firmware
> + * @debug_queue: shared interface queue to receive debug info from firmware
> + * @sfr: SFR register memory
> */
>
> struct iris_core {
> @@ -49,6 +55,11 @@ struct iris_core {
> struct reset_info *reset_tbl;
> u32 reset_count;
> enum iris_core_state state;
> + struct mem_desc iface_q_table;
> + struct iface_q_info command_queue;
> + struct iface_q_info message_queue;
> + struct iface_q_info debug_queue;
> + struct mem_desc sfr;
> };
>
> #endif
> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
> index 7bb9c92..fd349a3 100644
> --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
> @@ -7,6 +7,7 @@
> #include <linux/of_device.h>
> #include <linux/platform_device.h>
>
> +#include "../hfi_queue.h"
> #include "iris_core.h"
> #include "resources.h"
>
> @@ -50,6 +51,10 @@ static void iris_remove(struct platform_device *pdev)
> if (!core)
> return;
>
> + hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr,
> + &core->command_queue, &core->message_queue,
> + &core->debug_queue);
> +
> video_unregister_device(core->vdev_dec);
>
> v4l2_device_unregister(&core->v4l2_dev);
> @@ -59,6 +64,7 @@ static int iris_probe(struct platform_device *pdev)
> {
> struct device *dev = &pdev->dev;
> struct iris_core *core;
> + u64 dma_mask;
> int ret;
>
> core = devm_kzalloc(&pdev->dev, sizeof(*core), GFP_KERNEL);
> @@ -91,8 +97,33 @@ static int iris_probe(struct platform_device *pdev)
>
> platform_set_drvdata(pdev, core);
>
> + /*
> + * Specify the max value of address space, which can be used
> + * for buffer transactions.
> + */
> + dma_mask = DMA_BIT_MASK(32);
> + dma_mask &= ~BIT(29);
> +
> + ret = dma_set_mask_and_coherent(dev, dma_mask);
> + if (ret)
> + goto err_vdev_unreg;
> +
> + dma_set_max_seg_size(&pdev->dev, (unsigned int)DMA_BIT_MASK(32));
> + dma_set_seg_boundary(&pdev->dev, (unsigned long)DMA_BIT_MASK(64));

This isn't related to queues.

> +
> + ret = hfi_queue_init(core->dev, &core->iface_q_table, &core->sfr,
> + &core->command_queue, &core->message_queue,
> + &core->debug_queue, core);
> + if (ret) {
> + dev_err_probe(core->dev, ret,
> + "%s: interface queues init failed\n", __func__);
> + goto err_vdev_unreg;
> + }
> +
> return ret;
>
> +err_vdev_unreg:
> + iris_unregister_video_device(core);
> err_v4l2_unreg:
> v4l2_device_unregister(&core->v4l2_dev);
>

--
With best wishes
Dmitry

2023-12-18 21:33:41

[permalink] [raw]

Subject: Re: [PATCH v2 09/34] media: iris: initialize shared queues for host and firmware communication

On 12/18/23 12:32, Dikshita Agarwal wrote:
> Shared queues are used for communication between driver and firmware.
> There are 3 types of queues:
> Command queue - driver to write any command to firmware.
> Message queue - firmware to send any response to driver.
> Debug queue - firmware to write debug message.
>
> Above queues are initialized and configured to firmware during probe.
>
> Signed-off-by: Dikshita Agarwal <[email protected]>
> ---
[...]

> + ret = hfi_queue_init(core->dev, &core->iface_q_table, &core->sfr,
> + &core->command_queue, &core->message_queue,
> + &core->debug_queue, core);
> + if (ret) {
> + dev_err_probe(core->dev, ret,
> + "%s: interface queues init failed\n", __func__);
> + goto err_vdev_unreg;
> + }
> +
> return ret;
Like before, you're suppose to return dev_err_probe

Konrad

2023-12-18 21:51:26

[permalink] [raw]

Subject: Re: [PATCH v2 11/34] media: iris: introduce packetization layer for creating HFI packets

On 12/18/23 12:32, Dikshita Agarwal wrote:
> Host firmware interface (HFI) is well defined set of interfaces
> for communication between host driver and firmware.
> The command and responses are exchanged in form of packets.
> One or multiple packets are grouped under packet header.
> Each packet has packet type which describes the specific HFI
> and payload which holds the corresponding value for that HFI.
>
> Sys_init is the first packets sent to firmware, which initializes
> the firmware. Sys_image_version packet is to get the firmware
> version string.
>
> Signed-off-by: Dikshita Agarwal <[email protected]>
> ---
[...]

> struct iris_core {
> @@ -65,6 +70,11 @@ struct iris_core {
> struct mem_desc sfr;
> struct mutex lock; /* lock for core structure */
> unsigned int use_tz;
> + u8 *packet;
> + u32 packet_size;
I'm not sure it's necessary to always keep a reference to the last
packet in the core struct, especially since it needs to be allocated
first anyway

> + u32 sys_init_id;
This looks like a hyper-defensive measure against some firmware
overtaking attacks.. Or a way to spot random/unwanted resets of
the firmware core..

Is it actually necessary, or does this just serve as a debug
feature?

> + u32 header_id;
Similar to above..

> + u32 packet_id;
And here.

I performed some quick CTRL-F-agge around the series and this is
never reset.. Can the firmware cope with this? What if I watch a
veeeery long youtube video that ends up creating more than
(1<<32)-1 HFI packets while playing?

> +
> +enum hfi_packet_host_flags {
> + HFI_HOST_FLAGS_NONE = 0x00000000,
> + HFI_HOST_FLAGS_INTR_REQUIRED = 0x00000001,
> + HFI_HOST_FLAGS_RESPONSE_REQUIRED = 0x00000002,
> + HFI_HOST_FLAGS_NON_DISCARDABLE = 0x00000004,
> + HFI_HOST_FLAGS_GET_PROPERTY = 0x00000008,
BIT(n)?

> +};
> +
> +enum hfi_packet_firmware_flags {
> + HFI_FW_FLAGS_NONE = 0x00000000,
> + HFI_FW_FLAGS_SUCCESS = 0x00000001,
> + HFI_FW_FLAGS_INFORMATION = 0x00000002,
> + HFI_FW_FLAGS_SESSION_ERROR = 0x00000004,
> + HFI_FW_FLAGS_SYSTEM_ERROR = 0x00000008,
BIT(n)?

Konrad

2023-12-18 22:08:39

[permalink] [raw]

Subject: Re: [PATCH v2 13/34] media: iris: introduce platform specific capabilities for core and instance

On 12/18/23 12:32, Dikshita Agarwal wrote:
> Capabilities are set of video specifications and features supported
> by a specific platform(SOC). Each capability is defined with
> min, max, range, default value and corresponding HFI.
>
> Also, platform data defines different resource details for
> a specific platform(SOC). This change defines resource tables
> for sm8550 platform data and use for initializing these resources.
>
> Signed-off-by: Dikshita Agarwal <[email protected]>
> ---
[...]
[...]

> - ret = protect_secure_region(CP_START, CP_SIZE, CP_NONPIXEL_START,
> - CP_NONPIXEL_SIZE, IRIS_PAS_ID);
> + cp_start = core->cap[CP_START].value;
> + cp_size = core->cap[CP_SIZE].value;
> + cp_nonpixel_start = core->cap[CP_NONPIXEL_START].value;
> + cp_nonpixel_size = core->cap[CP_NONPIXEL_SIZE].value;
but you just hardcoded these a couple patches ago.. are they
variable after all?

[...]

> + {DEC_CODECS, H264 | HEVC | VP9},
> + {MAX_SESSION_COUNT, 16},
> + {MAX_MBPF, 278528}, /* ((8192x4352)/256) * 2 */
> + {MAX_MBPS, 7833600}, /* max_load 7680x4320@60fps */
> + {NUM_VPP_PIPE, 4},
> + {HW_RESPONSE_TIMEOUT, HW_RESPONSE_TIMEOUT_VALUE},
> + {DMA_MASK, GENMASK(31, 29) - 1},
> + {CP_START, 0},
> + {CP_SIZE, 0x25800000},
> + {CP_NONPIXEL_START, 0x01000000},
> + {CP_NONPIXEL_SIZE, 0x24800000},
Why this and not an array of enum-indexed u32s?

> +};
> +
> +static struct plat_inst_cap instance_cap_data_sm8550[] = {
you know all of the possible members here as well, you can just
create a struct of actual configurations instead of turning them
into generic capabilities that you have to parse either way at
some point

[...]

> +
> +static const struct bus_info sm8550_bus_table[] = {
> + { NULL, "iris-cnoc", 1000, 1000 },
> + { NULL, "iris-ddr", 1000, 15000000 },
This is a copy of the common data from the previous patches that you're
now dropping (why was it there in the first place then?). Is it specific
to this platform, or can it be reused?
> +};
> +
> +static const struct clock_info sm8550_clk_table[] = {
> + { NULL, "gcc_video_axi0", GCC_VIDEO_AXI0_CLK, 0 },
> + { NULL, "core_clk", VIDEO_CC_MVS0C_CLK, 0 },
> + { NULL, "vcodec_core", VIDEO_CC_MVS0_CLK, 1 },
> +};
Are these the input pad names?

> +
> +static const char * const sm8550_clk_reset_table[] = { "video_axi_reset", NULL };
Ditto

> +
> +static const char * const sm8550_pd_table[] = { "iris-ctl", "vcodec", NULL };
Ditto

> +
> +static const char * const sm8550_opp_pd_table[] = { "mxc", "mmcx", NULL };
Ditto

> +
> +static const struct bw_info sm8550_bw_table_dec[] = {
> + { 2073600, 1608000, 2742000 }, /* 4096x2160@60 */
> + { 1036800, 826000, 1393000 }, /* 4096x2160@30 */
> + { 489600, 567000, 723000 }, /* 1920x1080@60 */
> + { 244800, 294000, 372000 }, /* 1920x1080@30 */
> +};
> +
> +static const struct reg_preset_info sm8550_reg_preset_table[] = {
> + { 0xB0088, 0x0, 0x11 },
lowercase hex for non-defines, please
> +};
> +
> +static struct ubwc_config_data ubwc_config_sm8550[] = {
> + UBWC_CONFIG(8, 32, 16, 0, 1, 1, 1),
I think it would be far more telling to drop this #define and fill
in the values inline

> +};
> +
> +struct platform_data sm8550_data = {
> + .bus_tbl = sm8550_bus_table,
> + .bus_tbl_size = ARRAY_SIZE(sm8550_bus_table),
> + .clk_tbl = sm8550_clk_table,
> + .clk_tbl_size = ARRAY_SIZE(sm8550_clk_table),
> + .clk_rst_tbl = sm8550_clk_reset_table,
> + .clk_rst_tbl_size = ARRAY_SIZE(sm8550_clk_reset_table),
> +
> + .bw_tbl_dec = sm8550_bw_table_dec,
> + .bw_tbl_dec_size = ARRAY_SIZE(sm8550_bw_table_dec),
> +
> + .pd_tbl = sm8550_pd_table,
> + .pd_tbl_size = ARRAY_SIZE(sm8550_pd_table),
> + .opp_pd_tbl = sm8550_opp_pd_table,
> + .opp_pd_tbl_size = ARRAY_SIZE(sm8550_opp_pd_table),
> +
> + .reg_prst_tbl = sm8550_reg_preset_table,
> + .reg_prst_tbl_size = ARRAY_SIZE(sm8550_reg_preset_table),
> + .fwname = "vpu30_4v.mbn",
> + .pas_id = 9,
> +
> + .core_data = core_data_sm8550,
> + .core_data_size = ARRAY_SIZE(core_data_sm8550),
> + .inst_cap_data = instance_cap_data_sm8550,
> + .inst_cap_data_size = ARRAY_SIZE(instance_cap_data_sm8550),
> + .ubwc_config = ubwc_config_sm8550,
Unless any of these are going to be reused, please inline all of the
values here..

Konrad

2023-12-19 11:41:03

[permalink] [raw]

Subject: Re: [PATCH v2 01/34] media: introduce common helpers for video firmware handling

On 18/12/2023 11:31, Dikshita Agarwal wrote:
> Re-organize the video driver code by introducing a new folder
> 'vcodec' and placing 'venus' driver code inside that.
>
> Introduce common helpers for trustzone based firmware
> load/unload etc. which are placed in common folder
> i.e. 'vcodec'.
> Use these helpers in 'venus' driver. These helpers will be
> used by 'iris' driver as well which is introduced later
> in this patch series.
>
> Signed-off-by: Dikshita Agarwal <[email protected]>
> ---

This is a very large patch, I think it needs to be broken up into
smaller chunks.

#1 Introduce common helper functions
#2 Use common helper functions

Its alot of code to try to eat in the one go.

Could you consider making patches 1-3 a standalone series to reduce the
amount of code to review here ?

* 77e7025529d7c - (HEAD -> linux-stable-master-23-12-18-iris-v2) media:
iris: add power management for encoder (21 hours ago)

* ceb6a6f023fd3 - (tag: v6.7-rc6, linux-stable/master) Linux 6.7-rc6 (2
days ago)

git diff ceb6a6f023fd3 | wc -l

21243

Also I feel it wouild give more time for the changes to "digest" though
upstream users and to find any unintended bugs.

> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
> + size_t *mem_size, u32 pas_id, bool use_tz)
> +{
> + const struct firmware *firmware = NULL;
> + struct reserved_mem *rmem;
> + struct device_node *node;
> + void *mem_virt = NULL;
> + ssize_t fw_size = 0;
> + int ret;
> +
> + if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
> + (use_tz && !qcom_scm_is_available()))
> + return -EPROBE_DEFER;
> +
> + if (!fw_name || !(*fw_name))
> + return -EINVAL;

The parameter check should come before the qcom_scm_is_available()

No matter how many times you -EPROBE_DEFER -EINVAL is still -EINVAL.

> +
> + *mem_phys = 0;
> + *mem_size = 0;

I don't think you need this, you don't appear to use these variables
before you assign them below.

> +
> + *mem_phys = rmem->base;
> + *mem_size = rmem->size;

> +
> +int auth_reset_fw(u32 pas_id)
> +{
> + return qcom_scm_pas_auth_and_reset(pas_id);
> +}
> +
> +void unload_fw(u32 pas_id)
> +{
> + qcom_scm_pas_shutdown(pas_id);
> +}
> +

Do these wrapper functions add anything ? Some kind of validity check on
the pas_id otherwise I'm not sure these are justified.

> +int set_hw_state(bool resume)
> +{
> + return qcom_scm_set_remote_state(resume, 0);
> +}
> diff --git a/drivers/media/platform/qcom/vcodec/firmware.h b/drivers/media/platform/qcom/vcodec/firmware.h
> new file mode 100644
> index 0000000..7d410a8
> --- /dev/null

---
bod

2023-12-19 11:48:53

[permalink] [raw]

Subject: Re: [PATCH v2 15/34] media: iris: add handling for interrupt service routine(ISR) invoked by hardware

On 18.12.2023 12:32, Dikshita Agarwal wrote:
> Allocate interrupt resources, enable the interrupt line and IRQ handling.
> Register the IRQ handler to be called when interrupt occurs and
> the function to be called from IRQ handler thread.
> The threads invoke the driver's response handler which handles
> all different responses from firmware.
>
> Signed-off-by: Dikshita Agarwal <[email protected]>
> ---
[...]

> +
> +irqreturn_t iris_hfi_isr_handler(int irq, void *data)
> +{
> + struct iris_core *core = data;
> +
> + if (!core)
> + return IRQ_NONE;
Should this even be possible?

> +
> + mutex_lock(&core->lock);
> + call_vpu_op(core, clear_interrupt, core);
> + mutex_unlock(&core->lock);
> +
> + __response_handler(core);
> +
> + if (!call_vpu_op(core, watchdog, core, core->intr_status))
> + enable_irq(irq);
> +
> + return IRQ_HANDLED;
> +}
> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
> index 8a057cc..8a62986 100644
> --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
> @@ -14,4 +14,7 @@ int iris_hfi_core_deinit(struct iris_core *core);
> int iris_hfi_session_open(struct iris_inst *inst);
> int iris_hfi_session_close(struct iris_inst *inst);
>
> +irqreturn_t iris_hfi_isr(int irq, void *data);
> +irqreturn_t iris_hfi_isr_handler(int irq, void *data);
> +
> #endif
> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
> new file mode 100644
> index 0000000..829f3f6
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
> @@ -0,0 +1,184 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
> + */
> +
> +#include "hfi_defines.h"
> +#include "iris_hfi_packet.h"
> +#include "iris_hfi_response.h"
> +
> +static void print_sfr_message(struct iris_core *core)
I'm not sure how 'print' relates to what this function does

[...]

> +static int handle_system_error(struct iris_core *core,
> + struct hfi_packet *pkt)
> +{
> + print_sfr_message(core);
> +
> + iris_core_deinit(core);
Either take the return value of /\ into account, or make this function
void.

> +
> + return 0;
> +}

[...]

> +
> +struct sfr_buffer {
> + u32 bufsize;
> + u8 rg_data[];
Looks like you skipped static code checks.. Use __counted_by

[...]

> @@ -17,6 +17,8 @@
> #define CPU_CS_VCICMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x24)
> #define CPU_CS_VCICMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x28)
>
> +#define CPU_CS_A2HSOFTINTCLR_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x1C)
You're mixing upper and lowercase hex throughout your defines.

[...]

> +static int clear_interrupt_iris3(struct iris_core *core)
> +{
> + u32 intr_status = 0, mask = 0;
> + int ret;
> +
> + ret = read_register(core, WRAPPER_INTR_STATUS_IRIS3, &intr_status);
> + if (ret)
> + return ret;
> +
> + mask = (WRAPPER_INTR_STATUS_A2H_BMSK_IRIS3 |
> + WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3 |
> + CTRL_INIT_IDLE_MSG_BMSK_IRIS3);
unnecesary parentheses

> +
> + if (intr_status & mask)
> + core->intr_status |= intr_status;
> +
> + ret = write_register(core, CPU_CS_A2HSOFTINTCLR_IRIS3, 1);
> +
> + return ret;
why not return write_register directly?

Konrad

2023-12-19 11:50:50

[permalink] [raw]

Subject: Re: [PATCH v2 16/34] media: iris: implement iris v4l2_ctrl_ops and prepare capabilities

On 18.12.2023 12:32, Dikshita Agarwal wrote:
> Implement iris v4l2_ctrl_ops - s_ctrl and g_volatile_ctrl.
>
> Core capability structure has all the supported capabilities
> for all supported codecs. Initializes instance capability
> from core capability based on the codec type.
>
> Add following to each capability:
> Children: define dependencies for a specific capability.
> Adjust: define function to adjust the value of capability
> based on client configuration or dependency with
> other capability.
> Set: define function to set the adjusted value to firmware.
>
> Prepare dependency graph for all inter-dependent capabilities.
> This is used to adjust the value of capabilities and set the
> same to firmware.
>
> Signed-off-by: Dikshita Agarwal <[email protected]>
> ---
[...]

> +static inline bool are_all_childrens_visited(struct plat_inst_cap *cap,
> + bool lookup[INST_CAP_MAX])
he's making a list, he's checking it twice..

I still think moving to a predefined config struct instead would
be a good idea

Konrad

2023-12-19 11:57:19

[permalink] [raw]

Subject: Re: [PATCH v2 17/34] media: iris: implement vb2_ops queue setup

On 18.12.2023 12:32, Dikshita Agarwal wrote:
> Implement queue_setup vb2_ops.
> Calculate the buffer count and buffer size as par video
> hardware requirement and updates to client.
> Also, allocate the video driver buffers for output and
> capture plane.
>
> Signed-off-by: Dikshita Agarwal <[email protected]>
> ---
[...]

> +static int input_min_count(struct iris_inst *inst)
> +{
> + return MIN_BUFFERS;
> +}
Why is this a function?

> +
> +static int output_min_count(struct iris_inst *inst)
> +{
> + int output_min_count;
> +
> + switch (inst->codec) {
> + case H264:
> + case HEVC:
> + output_min_count = 4;
> + break;
> + case VP9:
> + output_min_count = 9;
> + break;
> + default:
> + output_min_count = 4;
> + break;
> + }

switch (inst->codec) {
case VP9:
return 9;
case H264:
case HEVC:
default:
return 4;
}

> +
> + return output_min_count;
> +}
> +
> +int iris_get_buf_min_count(struct iris_inst *inst,
> + enum iris_buffer_type buffer_type)
> +{
> + switch (buffer_type) {
> + case BUF_INPUT:
> + return input_min_count(inst);
> + case BUF_OUTPUT:
> + return output_min_count(inst);
> + default:
> + return 0;
> + }
> +}
> +
> +static u32 input_buffer_size(struct iris_inst *inst)
> +{
> + u32 base_res_mbs = NUM_MBS_4k;
> + u32 frame_size, num_mbs;
> + struct v4l2_format *f;
> + u32 div_factor = 1;
> + u32 codec;
> +
> + f = inst->fmt_src;
> + codec = f->fmt.pix_mp.pixelformat;
> +
> + num_mbs = get_mbpf(inst);
> + if (num_mbs > NUM_MBS_4k) {
> + div_factor = 4;
> + base_res_mbs = inst->cap[MBPF].value;
> + } else {
> + base_res_mbs = NUM_MBS_4k;
> + if (codec == V4L2_PIX_FMT_VP9)
> + div_factor = 1;
> + else
> + div_factor = 2;
> + }
> +
> + frame_size = base_res_mbs * MB_IN_PIXEL * 3 / 2 / div_factor;
that's a bit magic..

> +
> + /* multiply by 10/8 (1.25) to get size for 10 bit case */
misaligned

Konrad

2023-12-19 11:57:42

[permalink] [raw]

Subject: Re: [PATCH v2 14/34] media: iris: implement iris v4l2 file ops

On 18/12/2023 11:32, Dikshita Agarwal wrote:
> From: Vikash Garodia <[email protected]>
>
> Implements iris v4l2 file ops - open, close and poll.
>
> Open:
> Configure the vb2 queue and v4l2 file handler. Initialize
> a video instance and add the instance to core instance list.
> Open a session in video firmware via HFI_CMD_OPEN.
>
> Close:
> De-initialize the instance and remove it from core
> instance list. Close the session in firmware via HFI_CMD_CLOSE.
>
> Signed-off-by: Vikash Garodia <[email protected]>
> Signed-off-by: Dikshita Agarwal <[email protected]>
> ---
> drivers/media/platform/qcom/vcodec/iris/Makefile | 2 +
> .../media/platform/qcom/vcodec/iris/hfi_defines.h | 2 +
> .../media/platform/qcom/vcodec/iris/iris_common.h | 26 ++
> .../media/platform/qcom/vcodec/iris/iris_core.c | 42 ++-
> .../media/platform/qcom/vcodec/iris/iris_core.h | 8 +
> .../media/platform/qcom/vcodec/iris/iris_helpers.c | 62 +++++
> .../media/platform/qcom/vcodec/iris/iris_helpers.h | 2 +
> drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 115 +++++++-
> drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 3 +
> .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 28 ++
> .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 7 +
> .../platform/qcom/vcodec/iris/iris_instance.h | 48 ++++
> .../media/platform/qcom/vcodec/iris/iris_probe.c | 10 +
> .../media/platform/qcom/vcodec/iris/iris_vdec.c | 47 ++++
> .../media/platform/qcom/vcodec/iris/iris_vdec.h | 14 +
> .../media/platform/qcom/vcodec/iris/iris_vidc.c | 298 +++++++++++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_vidc.h | 15 ++
> 17 files changed, 719 insertions(+), 10 deletions(-)
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_common.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_instance.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.h
>
> diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile
> index c50e3241..3c076d0 100644
> --- a/drivers/media/platform/qcom/vcodec/iris/Makefile
> +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
> @@ -3,6 +3,8 @@ iris-objs += ../hfi_queue.o ../firmware.o
> iris-objs += iris_probe.o \
> iris_state.o \
> iris_core.o \
> + iris_vidc.o \
> + iris_vdec.o \
> iris_state.o \
> iris_helpers.o \
> iris_hfi.o \
> diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
> index fb383b2..423ba1a 100644
> --- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
> +++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
> @@ -9,6 +9,8 @@
> #define HFI_VIDEO_ARCH_LX 0x1
>
> #define HFI_CMD_INIT 0x01000001
> +#define HFI_CMD_OPEN 0x01000003
> +#define HFI_CMD_CLOSE 0x01000004
>
> #define HFI_PROP_IMAGE_VERSION 0x03000001
>
> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
> new file mode 100644
> index 0000000..3e4dd71
> --- /dev/null
> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h
> @@ -0,0 +1,26 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
> + */
> +#ifndef _IRIS_COMMON_H_
> +#define _IRIS_COMMON_H_
> +
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#define INPUT_MPLANE V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE
> +#define OUTPUT_MPLANE V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE
> +#define DEFAULT_WIDTH 320
> +#define DEFAULT_HEIGHT 240
> +#define DEFAULT_BUF_SIZE 115200
> +
> +enum signal_session_response {
> + SIGNAL_CMD_STOP_INPUT = 0,
> + SIGNAL_CMD_STOP_OUTPUT,
> + SIGNAL_CMD_CLOSE,
> + MAX_SIGNAL,
> +};
> +
> +#endif
> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.c b/drivers/media/platform/qcom/vcodec/iris/iris_core.c
> index ba8960d..174fad9 100644
> --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.c
> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.c
> @@ -3,12 +3,14 @@
> * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved.
> */
>
> +#include <linux/delay.h>
> +
> #include "iris_core.h"
> #include "iris_helpers.h"
> #include "iris_hfi.h"
> #include "iris_state.h"
>
> -static int iris_core_deinit_locked(struct iris_core *core)
> +int iris_core_deinit_locked(struct iris_core *core)
> {
> int ret;
>
> @@ -68,3 +70,41 @@ int iris_core_init(struct iris_core *core)
>
> return ret;
> }
> +
> +int iris_core_init_wait(struct iris_core *core)
> +{
> + const int interval = 10;
> + int max_tries, count = 0, ret = 0;
> +
> + mutex_lock(&core->lock);
> + if (!core_in_valid_state(core)) {
> + ret = -EINVAL;
> + goto unlock;
> + }
> +
> + if (core->state == IRIS_CORE_INIT)
> + goto unlock;
> +
> + max_tries = core->cap[HW_RESPONSE_TIMEOUT].value / interval;
> + while (count < max_tries) {
> + if (core->state != IRIS_CORE_INIT_WAIT)
> + break;
> + msleep(interval);
> + count++;
> + }

Is the intention here really to poll and sleep for 10 milliseconds and
if so why ?

Shouldn't there be some sort of bootup IRQ or event that we can trap ?

That's how it works in the venus driver right ? Just kick a workqueue
when the IRQ fires and have the iris_core_init_wait() do a
wait_for_completion_timeout().

> +int iris_hfi_session_open(struct iris_inst *inst)
> +{
> + struct iris_core *core;
> + int ret;
> +
> + inst->packet_size = 4096;
> + inst->packet = kzalloc(inst->packet_size, GFP_KERNEL);
> + if (!inst->packet)
> + return -ENOMEM;

Who is responsible for cleaning up this allocation ?

> +
> +fail_free_packet:
> + kfree(inst->packet);

freed here.

> +int iris_hfi_session_close(struct iris_inst *inst)
> +{
> }

but not here ?

Still finding this alot of code to go through.

Please try to break this up into smaller series.

---
bod

2023-12-19 11:58:48

[permalink] [raw]

Subject: Re: [PATCH v2 18/34] media: iris: introduce and implement iris vb2 mem ops

On 18.12.2023 12:32, Dikshita Agarwal wrote:
> From: Vikash Garodia <[email protected]>
>
> Implement the iris vb2 mem ops for buffer management for
> DMABUF streaming mode. Update video driver buffer
> with dma buf information.
>
> Signed-off-by: Vikash Garodia <[email protected]>
> Signed-off-by: Dikshita Agarwal <[email protected]>
> ---
[...]

> +
> +void *iris_vb2_alloc(struct vb2_buffer *vb, struct device *dev,
> + unsigned long size)
> +{
> + return (void *)0xdeadbeef;
> +}
> +
> +void iris_vb2_put(void *buf_priv)
> +{
> +}
> +
> +int iris_vb2_mmap(void *buf_priv, struct vm_area_struct *vma)
> +{
> + return 0;
> +}
Are you sure these are enough for this driver to function?

Konrad

2023-12-19 12:07:08

[permalink] [raw]

Subject: Re: [PATCH v2 20/34] media: iris: add video hardware internal buffer count and size calculation

On 18/12/2023 11:32, Dikshita Agarwal wrote:
> drivers/media/platform/qcom/vcodec/iris/Makefile | 1 +
> .../media/platform/qcom/vcodec/iris/iris_buffer.c | 48 ++
> .../media/platform/qcom/vcodec/iris/iris_common.h | 1 +
> .../media/platform/qcom/vcodec/iris/iris_core.h | 2 +
> .../media/platform/qcom/vcodec/iris/iris_helpers.c | 13 +
> .../media/platform/qcom/vcodec/iris/iris_helpers.h | 4 +
> .../platform/qcom/vcodec/iris/iris_instance.h | 2 +
> .../media/platform/qcom/vcodec/iris/iris_vdec.c | 1 +
> .../media/platform/qcom/vcodec/iris/vpu_common.h | 8 +
> .../media/platform/qcom/vcodec/iris/vpu_iris3.c | 6 +
> .../platform/qcom/vcodec/iris/vpu_iris3_buffer.c | 201 +++++
> .../platform/qcom/vcodec/iris/vpu_iris3_buffer.h | 845 +++++++++++++++++++++

So, unless the code you are adding effects existing upstream venus, I
think it would be a bit easier to review if you squashed down changes
that pertain to Iris only.

For example this patch seems to relate to Iris only, so why is it a
progressive change within your series.

Similar comment for "add vb2 streaming and buffer ops" and other "add
feature x" patches in this series.

If the change is contained to your own codebase, then progressive
changes are more noise than content.

Please try to squash down changes - to reduce the number of patches and
the total LOC being proposed here.

---
bod

2023-12-19 12:20:59

[permalink] [raw]

Subject: Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver

On 18/12/2023 18:38, Dmitry Baryshkov wrote:
> On 18/12/2023 13:31, Dikshita Agarwal wrote:
>> This patch series introduces support for Qualcomm new video acceleration
>> hardware architecture, used for video stream decoding/encoding. This
>> driver
>> is based on new communication protocol between video hardware and
>> application
>> processor.
>
> This doesn't answer one important point, you have been asked for v1.
> What is the actual change point between Venus and Iris? What has been
> changed so much that it demands a separate driver. This is the main
> question for the cover letter, which has not been answered so far.
>
> From what I see from you bindings, the hardware is pretty close to what
> we see in the latest venus generations. I asssme that there was a change
> in the vcodec inteface to the firmware and other similar changes. Could
> you please point out, which parts of Venus driver do no longer work or
> are not applicable for sm8550?

I'd like to hear this detail too.

Grazing on some of the patches here I saw were we adding vb2 buffer ops,
which is already supported by venus.

Similar question on booting cores and doing a busy/wait instead of a
rendezvous based on an IRQ.

I feel there's a version of this series that can probably unify the
codebases but, am open to being told how that is incorrect.

---
bod

2023-12-19 12:22:11

[permalink] [raw]

Subject: Re: [PATCH v2 27/34] media: iris: implement vb2 ops for buf_queue and firmware response

On 18.12.2023 12:32, Dikshita Agarwal wrote:
> Implement vb2 ops for buf queue. Below are the different
> buffer attributes:
> BUF_ATTR_DEFERRED - buffer queued by client but not submitted
> to firmware.
> BUF_ATTR_READ_ONLY - processed buffer received from firmware
> as read only. These buffers are held in firmware as reference
> for future frame processing.
> BUF_ATTR_PENDING_RELEASE - buffers requested to be released
> from firmware.
> BUF_ATTR_QUEUED - buffers submitted to firmware.
> BUF_ATTR_DEQUEUED - buffers received from firmware.
> BUF_ATTR_BUFFER_DONE - buffers sent back to vb2.
>
> Buffers are submitted and received via HFI_CMD_BUFFER.
> Firmware associates below flags during buffer response:
> HFI_BUF_FW_FLAG_RELEASE_DONE - buffer released in firmware.
> HFI_BUF_FW_FLAG_READONLY - buffer used as reference in firmware.
>
> Input buffers dequeued from firmware are sent directly to vb2.
>
> Output buffers if read only, are sent to vb2 and also maintained
> in read only list. If the same read only buffer is received form
> client, HFI_BUF_HOST_FLAG_READONLY is attached to the buffer and
> submitted to firmware. Once the buffer is received from firmware
> as non read only, it is removed from read only list.
>
> Signed-off-by: Dikshita Agarwal <[email protected]>
> ---
[...]

>
> +enum iris_buffer_flags {
> + BUF_FLAG_KEYFRAME = 0x00000008,
> + BUF_FLAG_PFRAME = 0x00000010,
> + BUF_FLAG_BFRAME = 0x00000020,
> + BUF_FLAG_ERROR = 0x00000040,
BIT(3), 4, 5, 6?

Konrad

2023-12-19 12:25:06

[permalink] [raw]

Subject: Re: [PATCH v2 29/34] media: iris: implement power management

On 18.12.2023 12:32, Dikshita Agarwal wrote:
> From: Vikash Garodia <[email protected]>
>
> Hardware specific power sequence include programming specific
> sequence of registers. Implements this sequence for iris3.
>
> Introduce video device suspend and resume using runtime PM.
>
> Also, implement iris3 specific calculation for clock and
> bus bandwidth which depends on hardware configuration,
> codec format, resolution and frame rate.
>
> Signed-off-by: Vikash Garodia <[email protected]>
> Signed-off-by: Dikshita Agarwal <[email protected]>
> ---
[...]

> @@ -97,6 +103,7 @@ static int iris_probe(struct platform_device *pdev)
return dev_err_probe, drop __func__

[...]

> + ret = readl_relaxed_poll_timeout(base_addr + reg, val, ((val & mask) == exp_val),
> + sleep_us, timeout_us);
> + /*
> + * Memory barrier to make sure value is read correctly from the
> + * register.
> + */
> + rmb();
just drop _relaxed and return directly

> +
> + return ret;
> +}
[...]

> + ret = read_register_with_poll_timeout(core, AON_WRAPPER_MVP_NOC_RESET_ACK,
> + 0x3, 0x3, 200, 2000);
that looks like a lot of bits/bitfields that deserve #defining in this
and some other functions

Konrad

2023-12-19 12:26:45

[permalink] [raw]

Subject: Re: [PATCH v2 34/34] media: iris: add power management for encoder

On 18.12.2023 12:32, Dikshita Agarwal wrote:
> Hardware specific power sequence include programming specific
> sequence of registers. Implements this sequence for iris3.
>
> Also, implement iris3 Encoder specific calculation for clock
> and bus bandwidth which depends on hardware configuration,
> codec format, resolution and frame rate.
>
> Signed-off-by: Dikshita Agarwal <[email protected]>
> ---
[...]

> +static const struct bw_info sm8550_bw_table_enc[] = {
> + { 1944000, 1491000, 2693000 }, /* 3840x2160@60 */
> + { 972000, 765000, 1366000 }, /* 3840x2160@30 */
> + { 489600, 557000, 780000 }, /* 1920x1080@60 */
> + { 244800, 288000, 399000 }, /* 1920x1080@30 */
> +};
can this be calculated at runtime by chance?

Konrad

2023-12-19 13:27:56

[permalink] [raw]

Subject: Re: [PATCH v2 01/34] media: introduce common helpers for video firmware handling

On Tue, 19 Dec 2023 at 13:40, Bryan O'Donoghue
<[email protected]> wrote:
>
> On 18/12/2023 11:31, Dikshita Agarwal wrote:
> > Re-organize the video driver code by introducing a new folder
> > 'vcodec' and placing 'venus' driver code inside that.
> >
> > Introduce common helpers for trustzone based firmware
> > load/unload etc. which are placed in common folder
> > i.e. 'vcodec'.
> > Use these helpers in 'venus' driver. These helpers will be
> > used by 'iris' driver as well which is introduced later
> > in this patch series.
> >
> > Signed-off-by: Dikshita Agarwal <[email protected]>
> > ---
>
> This is a very large patch, I think it needs to be broken up into
> smaller chunks.
>
> #1 Introduce common helper functions
> #2 Use common helper functions

This will make it harder to review. It's usually preferred to have a
single 'move' patch instead of two (add + remove). But I definitely
agree that the size of the patch is big. Somewhat it is related to the
fact that this doesn't only introduce helpers, but also reshuffles the
rest of the code.

>
> Its alot of code to try to eat in the one go.
>
> Could you consider making patches 1-3 a standalone series to reduce the
> amount of code to review here ?

This sounds like a good idea.

>
> * 77e7025529d7c - (HEAD -> linux-stable-master-23-12-18-iris-v2) media:
> iris: add power management for encoder (21 hours ago)
>
> * ceb6a6f023fd3 - (tag: v6.7-rc6, linux-stable/master) Linux 6.7-rc6 (2
> days ago)
>
> git diff ceb6a6f023fd3 | wc -l
>
> 21243
>
> Also I feel it wouild give more time for the changes to "digest" though
> upstream users and to find any unintended bugs.
>
> > +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
> > + size_t *mem_size, u32 pas_id, bool use_tz)
> > +{
> > + const struct firmware *firmware = NULL;
> > + struct reserved_mem *rmem;
> > + struct device_node *node;
> > + void *mem_virt = NULL;
> > + ssize_t fw_size = 0;
> > + int ret;
> > +
> > + if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
> > + (use_tz && !qcom_scm_is_available()))
> > + return -EPROBE_DEFER;
> > +
> > + if (!fw_name || !(*fw_name))
> > + return -EINVAL;
>
> The parameter check should come before the qcom_scm_is_available()
>
> No matter how many times you -EPROBE_DEFER -EINVAL is still -EINVAL.
>
> > +
> > + *mem_phys = 0;
> > + *mem_size = 0;
>
> I don't think you need this, you don't appear to use these variables
> before you assign them below.
>
>
> > +
> > + *mem_phys = rmem->base;
> > + *mem_size = rmem->size;
>
> > +
> > +int auth_reset_fw(u32 pas_id)
> > +{
> > + return qcom_scm_pas_auth_and_reset(pas_id);
> > +}
> > +
> > +void unload_fw(u32 pas_id)
> > +{
> > + qcom_scm_pas_shutdown(pas_id);
> > +}
> > +
>
> Do these wrapper functions add anything ? Some kind of validity check on
> the pas_id otherwise I'm not sure these are justified.
>
> > +int set_hw_state(bool resume)
> > +{
> > + return qcom_scm_set_remote_state(resume, 0);
> > +}
> > diff --git a/drivers/media/platform/qcom/vcodec/firmware.h b/drivers/media/platform/qcom/vcodec/firmware.h
> > new file mode 100644
> > index 0000000..7d410a8
> > --- /dev/null
>
> ---
> bod
>
>

--
With best wishes
Dmitry

2023-12-20 06:33:19

[permalink] [raw]

Subject: Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver

Hi Dmitry,

On 12/19/2023 12:08 AM, Dmitry Baryshkov wrote:
> On 18/12/2023 13:31, Dikshita Agarwal wrote:
>> This patch series introduces support for Qualcomm new video acceleration
>> hardware architecture, used for video stream decoding/encoding. This driver
>> is based on new communication protocol between video hardware and application
>> processor.
>
> This doesn't answer one important point, you have been asked for v1. What is the
> actual change point between Venus and Iris? What has been changed so much that
> it demands a separate driver. This is the main question for the cover letter,
> which has not been answered so far.
>
> From what I see from you bindings, the hardware is pretty close to what we see
> in the latest venus generations. I asssme that there was a change in the vcodec
> inteface to the firmware and other similar changes. Could you please point out,
> which parts of Venus driver do no longer work or are not applicable for sm8550

The motivation behind having a separate IRIS driver was discussed earlier in [1]
In the same discussion, it was ellaborated on how the impact would be with
change in the new firmware interface and other video layers in the driver. I can
add this in cover letter in the next revision.

We see some duplication of code and to handle the same, the series brings in a
common code reusability between iris and venus. Aligning the common peices of
venus and iris will be a work in progress, once we land the base driver for iris.

Again qualcomm video team does not have a plan to support sm8550/x1e80100 on
venus as the changes are too interleaved to absorb in venus driver. And there is
significant interest in community to start validating video driver on sm8550 or
x1e80100.

[1] https://lore.kernel.org/lkml/[email protected]/

Regards,
Vikash

2023-12-20 07:38:02

[permalink] [raw]

Subject: Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver

On Wed, 20 Dec 2023 at 08:32, Vikash Garodia <[email protected]> wrote:
>
> Hi Dmitry,
>
> On 12/19/2023 12:08 AM, Dmitry Baryshkov wrote:
> > On 18/12/2023 13:31, Dikshita Agarwal wrote:
> >> This patch series introduces support for Qualcomm new video acceleration
> >> hardware architecture, used for video stream decoding/encoding. This driver
> >> is based on new communication protocol between video hardware and application
> >> processor.
> >
> > This doesn't answer one important point, you have been asked for v1. What is the
> > actual change point between Venus and Iris? What has been changed so much that
> > it demands a separate driver. This is the main question for the cover letter,
> > which has not been answered so far.
> >
> > From what I see from you bindings, the hardware is pretty close to what we see
> > in the latest venus generations. I asssme that there was a change in the vcodec
> > inteface to the firmware and other similar changes. Could you please point out,
> > which parts of Venus driver do no longer work or are not applicable for sm8550
>
> The motivation behind having a separate IRIS driver was discussed earlier in [1]
> In the same discussion, it was ellaborated on how the impact would be with
> change in the new firmware interface and other video layers in the driver. I can
> add this in cover letter in the next revision.

Ok. So the changes cover the HFI interface. Is that correct?

> We see some duplication of code and to handle the same, the series brings in a
> common code reusability between iris and venus. Aligning the common peices of
> venus and iris will be a work in progress, once we land the base driver for iris.

This is not how it usually works. Especially not with the patches you
have posted.

I have the following suggestion how this story can continue:
You can _start_ by reworking venus driver, separating the HFI /
firmware / etc interface to an internal interface in the driver. Then
implement Iris as a plug in for that interface. I might be mistaken
here, but I think this is the way how this can be beneficial for both
the video en/decoding on both old and new platforms.

Short rationale:
The venus driver has a history of supported platforms. There is
already some kind of buffer management in place. Both developers and
testers have spent their effort on finding issues there. Sending new
driver means that we have to spend the same amount of efforts on this.
Moreover, even from the porter point of view. You are creating new
bindings for the new hardware. Which do not follow the
venus-common.yaml. And they do not follow the defined bindings for the
recent venus platforms. Which means that as a developer I have to care
about two different ways to describe nearly the same hardware.

> Again qualcomm video team does not have a plan to support sm8550/x1e80100 on
> venus as the changes are too interleaved to absorb in venus driver. And there is
> significant interest in community to start validating video driver on sm8550 or
> x1e80100.
>
> [1] https://lore.kernel.org/lkml/[email protected]/
>
> Regards,
> Vikash

--
With best wishes
Dmitry

2023-12-20 08:02:15

[permalink] [raw]

Subject: Re: [PATCH v2 01/34] media: introduce common helpers for video firmware handling

On 12/18/2023 11:54 PM, Dmitry Baryshkov wrote:
> On 18/12/2023 13:31, Dikshita Agarwal wrote:
>> Re-organize the video driver code by introducing a new folder
>> 'vcodec' and placing 'venus' driver code inside that.
>>
>> Introduce common helpers for trustzone based firmware
>> load/unload etc. which are placed in common folder
>> i.e. 'vcodec'.
>> Use these helpers in 'venus' driver. These helpers will be
>> used by 'iris' driver as well which is introduced later
>> in this patch series.
>
> But why do you need to move the venus driver to subdir?

Currently venus driver is present in drivers/media/platform/qcom folder
which also has camss folder. We introduced vcodec to keep common code and
moved venus inside that, to indicate that the common code is for vcodec
drivers i.e venus and iris. Keeping this in qcom folder would mean, common
code will be used for camss only which is not the case here.
>
>>
>> Signed-off-by: Dikshita Agarwal <[email protected]>
>> ---
>> drivers/media/platform/qcom/Kconfig                |   2 +-
>> drivers/media/platform/qcom/Makefile               |   2 +-
>> drivers/media/platform/qcom/vcodec/firmware.c      | 147 +++++++++
>> drivers/media/platform/qcom/vcodec/firmware.h      | 21 ++
>> .../media/platform/qcom/{ => vcodec}/venus/Kconfig |   0
>> .../platform/qcom/{ => vcodec}/venus/Makefile      |   4 +-
>> .../media/platform/qcom/{ => vcodec}/venus/core.c | 102 +++++-
>> .../media/platform/qcom/{ => vcodec}/venus/core.h |   0
>> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.c |   0
>> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.h |   0
>> .../platform/qcom/vcodec/venus/firmware_no_tz.c    | 194 +++++++++++
>> .../platform/qcom/vcodec/venus/firmware_no_tz.h    | 19 ++
>> .../platform/qcom/{ => vcodec}/venus/helpers.c     |   0
>> .../platform/qcom/{ => vcodec}/venus/helpers.h     |   0
>> .../media/platform/qcom/{ => vcodec}/venus/hfi.c   |   0
>> .../media/platform/qcom/{ => vcodec}/venus/hfi.h   |   0
>> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.c    |   0
>> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.h    |   0
>> .../platform/qcom/{ => vcodec}/venus/hfi_helper.h |   0
>> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.c    |   0
>> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.h    |   0
>> .../platform/qcom/{ => vcodec}/venus/hfi_parser.c |   0
>> .../platform/qcom/{ => vcodec}/venus/hfi_parser.h |   0
>> .../qcom/{ => vcodec}/venus/hfi_plat_bufs.h        |   0
>> .../qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c     |   0
>> .../qcom/{ => vcodec}/venus/hfi_platform.c         |   0
>> .../qcom/{ => vcodec}/venus/hfi_platform.h         |   0
>> .../qcom/{ => vcodec}/venus/hfi_platform_v4.c      |   0
>> .../qcom/{ => vcodec}/venus/hfi_platform_v6.c      |   0
>> .../platform/qcom/{ => vcodec}/venus/hfi_venus.c   | 21 +-
>> .../platform/qcom/{ => vcodec}/venus/hfi_venus.h   |   0
>> .../qcom/{ => vcodec}/venus/hfi_venus_io.h         |   0
>> .../platform/qcom/{ => vcodec}/venus/pm_helpers.c |   0
>> .../platform/qcom/{ => vcodec}/venus/pm_helpers.h |   0
>> .../media/platform/qcom/{ => vcodec}/venus/vdec.c |   0
>> .../media/platform/qcom/{ => vcodec}/venus/vdec.h |   0
>> .../platform/qcom/{ => vcodec}/venus/vdec_ctrls.c |   0
>> .../media/platform/qcom/{ => vcodec}/venus/venc.c |   0
>> .../media/platform/qcom/{ => vcodec}/venus/venc.h |   0
>> .../platform/qcom/{ => vcodec}/venus/venc_ctrls.c |   0
>> drivers/media/platform/qcom/venus/firmware.c       | 363
>> ---------------------
>> drivers/media/platform/qcom/venus/firmware.h       | 26 --
>> 42 files changed, 492 insertions(+), 409 deletions(-)
>> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.c
>> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.h
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/Kconfig (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/Makefile (83%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.c (91%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.h (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.c (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.h (100%)
>> create mode 100644
>> drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
>> create mode 100644
>> drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.c (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.h (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.c (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.h (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.c (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.h (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_helper.h (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.c (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.h (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.c (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.h (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs.h
>> (100%)
>> rename drivers/media/platform/qcom/{ =>
>> vcodec}/venus/hfi_plat_bufs_v6.c (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.c
>> (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.h
>> (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v4.c
>> (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v6.c
>> (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.c (99%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.h (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus_io.h
>> (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.c (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.h (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.c (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.h (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec_ctrls.c (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.c (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.h (100%)
>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc_ctrls.c (100%)
>> delete mode 100644 drivers/media/platform/qcom/venus/firmware.c
>> delete mode 100644 drivers/media/platform/qcom/venus/firmware.h
>>
>> diff --git a/drivers/media/platform/qcom/Kconfig
>> b/drivers/media/platform/qcom/Kconfig
>> index cc5799b..e94142f 100644
>> --- a/drivers/media/platform/qcom/Kconfig
>> +++ b/drivers/media/platform/qcom/Kconfig
>> @@ -3,4 +3,4 @@
>> comment "Qualcomm media platform drivers"
>> source "drivers/media/platform/qcom/camss/Kconfig"
>> -source "drivers/media/platform/qcom/venus/Kconfig"
>> +source "drivers/media/platform/qcom/vcodec/venus/Kconfig"
>> diff --git a/drivers/media/platform/qcom/Makefile
>> b/drivers/media/platform/qcom/Makefile
>> index 4f055c3..3d2d82b 100644
>> --- a/drivers/media/platform/qcom/Makefile
>> +++ b/drivers/media/platform/qcom/Makefile
>> @@ -1,3 +1,3 @@
>> # SPDX-License-Identifier: GPL-2.0-only
>> obj-y += camss/
>> -obj-y += venus/
>> +obj-y += vcodec/venus/
>> diff --git a/drivers/media/platform/qcom/vcodec/firmware.c
>> b/drivers/media/platform/qcom/vcodec/firmware.c
>> new file mode 100644
>> index 0000000..dbc220a
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vcodec/firmware.c
>> @@ -0,0 +1,147 @@
>> +// SPDX-License-Identifier: GPL-2.0-only
>> +/*
>> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
>> reserved.
>> + */
>> +
>> +#include <linux/device.h>
>> +#include <linux/dma-mapping.h>
>> +#include <linux/firmware.h>
>> +#include <linux/kernel.h>
>> +#include <linux/iommu.h>
>> +#include <linux/of_device.h>
>> +#include <linux/firmware/qcom/qcom_scm.h>
>> +#include <linux/of_reserved_mem.h>
>> +#include <linux/platform_device.h>
>> +#include <linux/soc/qcom/mdt_loader.h>
>> +
>> +#include "firmware.h"
>> +
>> +bool use_tz(struct device *core_dev)
>
> All these functions must get some sane prefix. Otherwise a generic 'use_tz'
> function is too polluting for the global namespace.
>
I understand, will check and do the needful.
>> +{
>> +    struct device_node *np;
>> +
>> +    np = of_get_child_by_name(core_dev->of_node, "video-firmware");
>> +    if (!np)
>> +        return true;
>> +
>> +    return false;
>> +}
>> +
>> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
>> +              u32 cp_nonpixel_size, u32 pas_id)
>> +{
>> +    int ret;
>> +    /*
>> +     * Clues for porting using downstream data:
>> +     * cp_start = 0
>> +     * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and not size!
>> +     *   This works, as the non-secure context bank is placed
>> +     *   contiguously right after the Content Protection region.
>> +     *
>> +     * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0]
>> +     * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1]
>> +     */
>> +    ret = qcom_scm_mem_protect_video_var(cp_start,
>> +                         cp_size,
>> +                         cp_nonpixel_start,
>> +                         cp_nonpixel_size);
>> +    if (ret)
>> +        qcom_scm_pas_shutdown(pas_id);
>> +
>> +    return ret;
>> +}
>> +
>> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
>> +        size_t *mem_size, u32 pas_id, bool use_tz)
>> +{
>> +    const struct firmware *firmware = NULL;
>> +    struct reserved_mem *rmem;
>> +    struct device_node *node;
>> +    void *mem_virt = NULL;
>> +    ssize_t fw_size = 0;
>> +    int ret;
>> +
>> +    if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
>
> Why? Can you just depend on it?
>
Sure, Will check this and get back.
>> +        (use_tz && !qcom_scm_is_available()))
>> +        return -EPROBE_DEFER;
>> +
>> +    if (!fw_name || !(*fw_name))
>> +        return -EINVAL;
>> +
>> +    *mem_phys = 0;
>> +    *mem_size = 0;
>> +
>> +    node = of_parse_phandle(dev->of_node, "memory-region", 0);
>> +    if (!node) {
>> +        dev_err(dev, "no memory-region specified\n");
>> +        return -EINVAL;
>> +    }
>> +
>> +    rmem = of_reserved_mem_lookup(node);
>> +    of_node_put(node);
>> +    if (!rmem) {
>> +        dev_err(dev, "failed to lookup reserved memory-region\n");
>> +        return -EINVAL;
>> +    }
>> +
>> +    ret = request_firmware(&firmware, fw_name, dev);
>> +    if (ret) {
>> +        dev_err(dev, "%s: failed to request fw \"%s\", error %d\n",
>> +            __func__, fw_name, ret);
>> +        return ret;
>> +    }
>> +
>> +    fw_size = qcom_mdt_get_size(firmware);
>> +    if (fw_size < 0) {
>> +        ret = fw_size;
>> +        dev_err(dev, "%s: out of bound fw image fw size: %ld\n",
>> +            __func__, fw_size);
>> +        goto err_release_fw;
>> +    }
>> +
>> +    *mem_phys = rmem->base;
>> +    *mem_size = rmem->size;
>> +
>> +    if (*mem_size < fw_size) {
>> +        ret = -EINVAL;
>> +        goto err_release_fw;
>> +    }
>> +
>> +    mem_virt = memremap(*mem_phys, *mem_size, MEMREMAP_WC);
>> +    if (!mem_virt) {
>> +        dev_err(dev, "unable to remap fw memory region %pa size %#zx\n",
>> +            mem_phys, *mem_size);
>> +        goto err_release_fw;
>> +    }
>> +
>> +    if (use_tz)
>> +        ret = qcom_mdt_load(dev, firmware, fw_name, pas_id, mem_virt,
>> +                    *mem_phys, *mem_size, NULL);
>> +    else
>> +        ret = qcom_mdt_load_no_init(dev, firmware, fw_name, pas_id,
>> mem_virt,
>> +                        *mem_phys, *mem_size, NULL);
>> +    if (ret) {
>> +        dev_err(dev, "%s: error %d loading fw \"%s\"\n",
>> +            __func__, ret, fw_name);
>> +    }
>> +
>> +    memunmap(mem_virt);
>> +err_release_fw:
>> +    release_firmware(firmware);
>> +    return ret;
>> +}
>> +
>> +int auth_reset_fw(u32 pas_id)
>> +{
>> +    return qcom_scm_pas_auth_and_reset(pas_id);
>> +}
>> +
>> +void unload_fw(u32 pas_id)
>> +{
>> +    qcom_scm_pas_shutdown(pas_id);
>> +}
>> +
>> +int set_hw_state(bool resume)
>> +{
>> +    return qcom_scm_set_remote_state(resume, 0);
>> +}
>> diff --git a/drivers/media/platform/qcom/vcodec/firmware.h
>> b/drivers/media/platform/qcom/vcodec/firmware.h
>> new file mode 100644
>> index 0000000..7d410a8
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vcodec/firmware.h
>> @@ -0,0 +1,21 @@
>> +/* SPDX-License-Identifier: GPL-2.0-only */
>> +/*
>> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
>> reserved.
>> + */
>> +
>> +#ifndef _FIRMWARE_H_
>> +#define _FIRMWARE_H_
>> +
>> +#include <linux/device.h>
>> +#include <linux/types.h>
>> +
>> +bool use_tz(struct device *core_dev);
>> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
>> +        size_t *mem_size, u32 pas_id, bool use_tz);
>> +int auth_reset_fw(u32 pas_id);
>> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
>> +              u32 cp_nonpixel_size, u32 pas_id);
>> +void unload_fw(u32 pas_id);
>> +int set_hw_state(bool resume);
>> +
>> +#endif
>> diff --git a/drivers/media/platform/qcom/venus/Kconfig
>> b/drivers/media/platform/qcom/vcodec/venus/Kconfig
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/Kconfig
>> rename to drivers/media/platform/qcom/vcodec/venus/Kconfig
>> diff --git a/drivers/media/platform/qcom/venus/Makefile
>> b/drivers/media/platform/qcom/vcodec/venus/Makefile
>> similarity index 83%
>> rename from drivers/media/platform/qcom/venus/Makefile
>> rename to drivers/media/platform/qcom/vcodec/venus/Makefile
>> index 91ee6be..f6f3a88 100644
>> --- a/drivers/media/platform/qcom/venus/Makefile
>> +++ b/drivers/media/platform/qcom/vcodec/venus/Makefile
>> @@ -1,7 +1,9 @@
>> # SPDX-License-Identifier: GPL-2.0
>> # Makefile for Qualcomm Venus driver
>> -venus-core-objs += core.o helpers.o firmware.o \
>> +venus-core-objs += ../firmware.o
>> +
>> +venus-core-objs += core.o helpers.o firmware_no_tz.o \
>>              hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \
>>              hfi_parser.o pm_helpers.o dbgfs.o \
>>              hfi_platform.o hfi_platform_v4.o \
>> diff --git a/drivers/media/platform/qcom/venus/core.c
>> b/drivers/media/platform/qcom/vcodec/venus/core.c
>> similarity index 91%
>> rename from drivers/media/platform/qcom/venus/core.c
>> rename to drivers/media/platform/qcom/vcodec/venus/core.c
>> index 9cffe97..56d9a53 100644
>> --- a/drivers/media/platform/qcom/venus/core.c
>> +++ b/drivers/media/platform/qcom/vcodec/venus/core.c
>> @@ -22,7 +22,8 @@
>> #include <media/v4l2-ioctl.h>
>> #include "core.h"
>> -#include "firmware.h"
>> +#include "../firmware.h"
>> +#include "firmware_no_tz.h"
>> #include "pm_helpers.h"
>> #include "hfi_venus_io.h"
>> @@ -86,6 +87,8 @@ static void venus_sys_error_handler(struct
>> work_struct *work)
>>       struct venus_core *core =
>>               container_of(work, struct venus_core, work.work);
>>       int ret, i, max_attempts = RPM_WAIT_FOR_IDLE_MAX_ATTEMPTS;
>> +    const struct venus_resources *res = core->res;
>> +    const char *fwpath = NULL;
>>       const char *err_msg = "";
>>       bool failed = false;
>> @@ -107,7 +110,10 @@ static void venus_sys_error_handler(struct
>> work_struct *work)
>>       mutex_lock(&core->lock);
>> -    venus_shutdown(core);
>> +    if (core->use_tz)
>> +        unload_fw(VENUS_PAS_ID);
>> +    else
>> +        unload_fw_no_tz(core);
>
> This is more than introducing helpers.
>
The new helpers are written to make the code generic for video drivers.
which requires changes in the calling function also.
>>       venus_coredump(core);
>> @@ -127,12 +133,39 @@ static void venus_sys_error_handler(struct
>> work_struct *work)
>>           failed = true;
>>       }
>> -    ret = venus_boot(core);
>> +    ret = of_property_read_string_index(core->dev->of_node,
>> "firmware-name", 0,
>> +                        &fwpath);
>> +    if (ret)
>> +        fwpath = core->res->fwname;
>> +
>> +    ret = load_fw(core->dev, fwpath, &core->fw.mem_phys,
>> &core->fw.mem_size,
>> +              VENUS_PAS_ID, core->use_tz);
>
> So, we had a nice local 'venus_boot'. Instead we now have a pile of code
> with non-generic prefixes, etc. If you are introducing helpers, please
> refrain from inlining of calling functions, etc. Just move the code to your
> helpers.
>
As mentioned in above comment, the common helpers are written to make the
code generic. I Will try to make it more clear, working on the same.
> NAK for the rest of the patch.
>
>>       if (ret && !failed) {
>> -        err_msg = "boot Venus";
>> +        err_msg = "load FW";
>>           failed = true;
>>       }
>> +    if (core->use_tz)
>> +        ret = auth_reset_fw(VENUS_PAS_ID);
>> +    else
>> +        ret = auth_reset_fw_no_tz(core, core->fw.mem_phys,
>> core->fw.mem_size);
>> +    if (ret && !failed) {
>> +        err_msg = "Auth and Reset";
>> +        failed = true;
>> +    }
>> +
>> +    if (core->use_tz && res->cp_size) {
>> +        ret = protect_secure_region(res->cp_start,
>> +                        res->cp_size,
>> +                        res->cp_nonpixel_start,
>> +                        res->cp_nonpixel_size,
>> +                        VENUS_PAS_ID);
>> +        if (ret && !failed) {
>> +            err_msg = "Protect CP Mem";
>> +            failed = true;
>> +        }
>> +    }
>> +
>>       ret = hfi_core_resume(core, true);
>>       if (ret && !failed) {
>>           err_msg = "resume HFI";
>> @@ -281,7 +314,9 @@ static irqreturn_t venus_isr_thread(int irq, void
>> *dev_id)
>> static int venus_probe(struct platform_device *pdev)
>> {
>> +    const struct venus_resources *res;
>>       struct device *dev = &pdev->dev;
>> +    const char *fwpath = NULL;
>>       struct venus_core *core;
>>       int ret;
>> @@ -362,14 +397,42 @@ static int venus_probe(struct platform_device *pdev)
>>       if (ret)
>>           goto err_runtime_disable;
>> -    ret = venus_firmware_init(core);
>> +    core->use_tz = use_tz(core->dev);
>> +
>> +    if (!core->use_tz) {
>> +        ret = init_fw_no_tz(core);
>> +        if (ret)
>> +            goto err_of_depopulate;
>> +    }
>> +
>> +    ret = of_property_read_string_index(dev->of_node, "firmware-name", 0,
>> +                        &fwpath);
>>       if (ret)
>> -        goto err_of_depopulate;
>> +        fwpath = core->res->fwname;
>> -    ret = venus_boot(core);
>> +    ret = load_fw(core->dev, fwpath, &core->fw.mem_phys,
>> &core->fw.mem_size,
>> +              VENUS_PAS_ID, core->use_tz);
>>       if (ret)
>>           goto err_firmware_deinit;
>> +    if (core->use_tz)
>> +        ret = auth_reset_fw(VENUS_PAS_ID);
>> +    else
>> +        ret = auth_reset_fw_no_tz(core, core->fw.mem_phys,
>> core->fw.mem_size);
>> +    if (ret)
>> +        goto err_firmware_deinit;
>> +
>> +    res = core->res;
>> +    if (core->use_tz && res->cp_size) {
>> +        ret = protect_secure_region(res->cp_start,
>> +                        res->cp_size,
>> +                        res->cp_nonpixel_start,
>> +                        res->cp_nonpixel_size,
>> +                        VENUS_PAS_ID);
>> +        if (ret)
>> +            goto err_firmware_deinit;
>> +    }
>> +
>>       ret = hfi_core_resume(core, true);
>>       if (ret)
>>           goto err_venus_shutdown;
>> @@ -399,9 +462,13 @@ static int venus_probe(struct platform_device *pdev)
>> err_dev_unregister:
>>       v4l2_device_unregister(&core->v4l2_dev);
>> err_venus_shutdown:
>> -    venus_shutdown(core);
>> +    if (core->use_tz)
>> +        unload_fw(VENUS_PAS_ID);
>> +    else
>> +        unload_fw_no_tz(core);
>> err_firmware_deinit:
>> -    venus_firmware_deinit(core);
>> +    if (!core->use_tz)
>> +        deinit_fw_no_tz(core);
>> err_of_depopulate:
>>       of_platform_depopulate(dev);
>> err_runtime_disable:
>> @@ -430,10 +497,15 @@ static void venus_remove(struct platform_device *pdev)
>>       ret = hfi_core_deinit(core, true);
>>       WARN_ON(ret);
>> -    venus_shutdown(core);
>> +    if (core->use_tz)
>> +        unload_fw(VENUS_PAS_ID);
>> +    else
>> +        unload_fw_no_tz(core);
>> +
>>       of_platform_depopulate(dev);
>> -    venus_firmware_deinit(core);
>> +    if (!core->use_tz)
>> +        deinit_fw_no_tz(core);
>>       pm_runtime_put_sync(dev);
>>       pm_runtime_disable(dev);
>> @@ -455,8 +527,12 @@ static void venus_core_shutdown(struct
>> platform_device *pdev)
>>       struct venus_core *core = platform_get_drvdata(pdev);
>>       pm_runtime_get_sync(core->dev);
>> -    venus_shutdown(core);
>> -    venus_firmware_deinit(core);
>> +    if (core->use_tz) {
>> +        unload_fw(VENUS_PAS_ID);
>> +    } else {
>> +        unload_fw_no_tz(core);
>> +        deinit_fw_no_tz(core);
>> +    }
>>       pm_runtime_put_sync(core->dev);
>> }
>> diff --git a/drivers/media/platform/qcom/venus/core.h
>> b/drivers/media/platform/qcom/vcodec/venus/core.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/core.h
>> rename to drivers/media/platform/qcom/vcodec/venus/core.h
>> diff --git a/drivers/media/platform/qcom/venus/dbgfs.c
>> b/drivers/media/platform/qcom/vcodec/venus/dbgfs.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/dbgfs.c
>> rename to drivers/media/platform/qcom/vcodec/venus/dbgfs.c
>> diff --git a/drivers/media/platform/qcom/venus/dbgfs.h
>> b/drivers/media/platform/qcom/vcodec/venus/dbgfs.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/dbgfs.h
>> rename to drivers/media/platform/qcom/vcodec/venus/dbgfs.h
>> diff --git a/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
>> b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
>> new file mode 100644
>> index 0000000..9dca6e23
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
>> @@ -0,0 +1,194 @@
>> +// SPDX-License-Identifier: GPL-2.0-only
>> +/*
>> + * Copyright (C) 2017 Linaro Ltd.
>> + */
>> +
>> +#include <linux/iommu.h>
>> +#include <linux/of_device.h>
>> +#include "core.h"
>> +#include "firmware_no_tz.h"
>> +#include "hfi_venus_io.h"
>> +
>> +#define VENUS_FW_MEM_SIZE        (6 * SZ_1M)
>> +#define VENUS_FW_START_ADDR        0x0
>> +
>> +int init_fw_no_tz(struct venus_core *core)
>> +{
>> +    struct platform_device_info info;
>> +    struct iommu_domain *iommu_dom;
>> +    struct platform_device *pdev;
>> +    struct device_node *np;
>> +    int ret;
>> +
>> +    np = of_get_child_by_name(core->dev->of_node, "video-firmware");
>> +
>> +    memset(&info, 0, sizeof(info));
>> +    info.fwnode = &np->fwnode;
>> +    info.parent = core->dev;
>> +    info.name = np->name;
>> +    info.dma_mask = DMA_BIT_MASK(32);
>> +
>> +    pdev = platform_device_register_full(&info);
>> +    if (IS_ERR(pdev)) {
>> +        of_node_put(np);
>> +        return PTR_ERR(pdev);
>> +    }
>> +
>> +    pdev->dev.of_node = np;
>> +
>> +    ret = of_dma_configure(&pdev->dev, np, true);
>> +    if (ret) {
>> +        dev_err(core->dev, "dma configure fail\n");
>> +        goto err_unregister;
>> +    }
>> +
>> +    core->fw.dev = &pdev->dev;
>> +
>> +    iommu_dom = iommu_domain_alloc(&platform_bus_type);
>> +    if (!iommu_dom) {
>> +        dev_err(core->fw.dev, "Failed to allocate iommu domain\n");
>> +        ret = -ENOMEM;
>> +        goto err_unregister;
>> +    }
>> +
>> +    ret = iommu_attach_device(iommu_dom, core->fw.dev);
>> +    if (ret) {
>> +        dev_err(core->fw.dev, "could not attach device\n");
>> +        goto err_iommu_free;
>> +    }
>> +
>> +    core->fw.iommu_domain = iommu_dom;
>> +
>> +    of_node_put(np);
>> +
>> +    return 0;
>> +
>> +err_iommu_free:
>> +    iommu_domain_free(iommu_dom);
>> +err_unregister:
>> +    platform_device_unregister(pdev);
>> +    of_node_put(np);
>> +    return ret;
>> +}
>> +
>> +void deinit_fw_no_tz(struct venus_core *core)
>> +{
>> +    struct iommu_domain *iommu;
>> +
>> +    if (!core->fw.dev)
>> +        return;
>> +
>> +    iommu = core->fw.iommu_domain;
>> +
>> +    iommu_detach_device(iommu, core->fw.dev);
>> +
>> +    if (iommu) {
>> +        iommu_domain_free(iommu);
>> +        iommu = NULL;
>> +    }
>> +
>> +    platform_device_unregister(to_platform_device(core->fw.dev));
>> +}
>> +
>> +static void reset_cpu_no_tz(struct venus_core *core)
>> +{
>> +    u32 fw_size = core->fw.mapped_mem_size;
>> +    void __iomem *wrapper_base;
>> +
>> +    if (IS_IRIS2_1(core))
>> +        wrapper_base = core->wrapper_tz_base;
>> +    else
>> +        wrapper_base = core->wrapper_base;
>> +
>> +    writel(0, wrapper_base + WRAPPER_FW_START_ADDR);
>> +    writel(fw_size, wrapper_base + WRAPPER_FW_END_ADDR);
>> +    writel(0, wrapper_base + WRAPPER_CPA_START_ADDR);
>> +    writel(fw_size, wrapper_base + WRAPPER_CPA_END_ADDR);
>> +    writel(fw_size, wrapper_base + WRAPPER_NONPIX_START_ADDR);
>> +    writel(fw_size, wrapper_base + WRAPPER_NONPIX_END_ADDR);
>> +
>> +    if (IS_IRIS2_1(core)) {
>> +        /* Bring XTSS out of reset */
>> +        writel(0, wrapper_base + WRAPPER_TZ_XTSS_SW_RESET);
>> +    } else {
>> +        writel(0x0, wrapper_base + WRAPPER_CPU_CGC_DIS);
>> +        writel(0x0, wrapper_base + WRAPPER_CPU_CLOCK_CONFIG);
>> +
>> +        /* Bring ARM9 out of reset */
>> +        writel(0, wrapper_base + WRAPPER_A9SS_SW_RESET);
>> +    }
>> +}
>> +
>> +void set_hw_state_no_tz(struct venus_core *core, bool resume)
>> +{
>> +    if (resume) {
>> +        reset_cpu_no_tz(core);
>> +    } else {
>> +        if (IS_IRIS2_1(core))
>> +            writel(WRAPPER_XTSS_SW_RESET_BIT,
>> +                   core->wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
>> +        else
>> +            writel(WRAPPER_A9SS_SW_RESET_BIT,
>> +                   core->wrapper_base + WRAPPER_A9SS_SW_RESET);
>> +    }
>> +}
>> +
>> +int auth_reset_fw_no_tz(struct venus_core *core, phys_addr_t mem_phys,
>> +            size_t mem_size)
>> +{
>> +    struct iommu_domain *iommu;
>> +    struct device *dev;
>> +    int ret;
>> +
>> +    dev = core->fw.dev;
>> +    if (!dev)
>> +        return -EPROBE_DEFER;
>> +
>> +    iommu = core->fw.iommu_domain;
>> +    core->fw.mapped_mem_size = mem_size;
>> +
>> +    ret = iommu_map(iommu, VENUS_FW_START_ADDR, mem_phys, mem_size,
>> +            IOMMU_READ | IOMMU_WRITE | IOMMU_PRIV, GFP_KERNEL);
>> +    if (ret) {
>> +        dev_err(dev, "could not map video firmware region\n");
>> +        return ret;
>> +    }
>> +
>> +    reset_cpu_no_tz(core);
>> +
>> +    return 0;
>> +}
>> +
>> +void unload_fw_no_tz(struct venus_core *core)
>> +{
>> +    const size_t mapped = core->fw.mapped_mem_size;
>> +    struct iommu_domain *iommu;
>> +    size_t unmapped;
>> +    u32 reg;
>> +    struct device *dev = core->fw.dev;
>> +    void __iomem *wrapper_base = core->wrapper_base;
>> +    void __iomem *wrapper_tz_base = core->wrapper_tz_base;
>> +
>> +    if (IS_IRIS2_1(core)) {
>> +        /* Assert the reset to XTSS */
>> +        reg = readl(wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
>> +        reg |= WRAPPER_XTSS_SW_RESET_BIT;
>> +        writel(reg, wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
>> +    } else {
>> +        /* Assert the reset to ARM9 */
>> +        reg = readl(wrapper_base + WRAPPER_A9SS_SW_RESET);
>> +        reg |= WRAPPER_A9SS_SW_RESET_BIT;
>> +        writel(reg, wrapper_base + WRAPPER_A9SS_SW_RESET);
>> +    }
>> +
>> +    iommu = core->fw.iommu_domain;
>> +
>> +    if (core->fw.mapped_mem_size && iommu) {
>> +        unmapped = iommu_unmap(iommu, VENUS_FW_START_ADDR, mapped);
>> +
>> +        if (unmapped != mapped)
>> +            dev_err(dev, "failed to unmap firmware\n");
>> +        else
>> +            core->fw.mapped_mem_size = 0;
>> +    }
>> +}
>> diff --git a/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
>> b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
>> new file mode 100644
>> index 0000000..5f008ef
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
>> @@ -0,0 +1,19 @@
>> +/* SPDX-License-Identifier: GPL-2.0-only */
>> +/*
>> + * Copyright (C) 2017 Linaro Ltd.
>> + */
>> +#ifndef __FIRMWARE_NO_TZ_H__
>> +#define __FIRMWARE_NO_TZ_H__
>> +
>> +struct device;
>> +
>> +#define VENUS_PAS_ID    9
>> +
>> +int init_fw_no_tz(struct venus_core *core);
>> +void deinit_fw_no_tz(struct venus_core *core);
>> +int auth_reset_fw_no_tz(struct venus_core *core, phys_addr_t mem_phys,
>> +            size_t mem_size);
>> +void unload_fw_no_tz(struct venus_core *core);
>> +void set_hw_state_no_tz(struct venus_core *core, bool resume);
>> +
>> +#endif
>> diff --git a/drivers/media/platform/qcom/venus/helpers.c
>> b/drivers/media/platform/qcom/vcodec/venus/helpers.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/helpers.c
>> rename to drivers/media/platform/qcom/vcodec/venus/helpers.c
>> diff --git a/drivers/media/platform/qcom/venus/helpers.h
>> b/drivers/media/platform/qcom/vcodec/venus/helpers.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/helpers.h
>> rename to drivers/media/platform/qcom/vcodec/venus/helpers.h
>> diff --git a/drivers/media/platform/qcom/venus/hfi.c
>> b/drivers/media/platform/qcom/vcodec/venus/hfi.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi.c
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi.c
>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>> b/drivers/media/platform/qcom/vcodec/venus/hfi.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi.h
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi.h
>> diff --git a/drivers/media/platform/qcom/venus/hfi_cmds.c
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_cmds.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_cmds.c
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_cmds.c
>> diff --git a/drivers/media/platform/qcom/venus/hfi_cmds.h
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_cmds.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_cmds.h
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_cmds.h
>> diff --git a/drivers/media/platform/qcom/venus/hfi_helper.h
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_helper.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_helper.h
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_helper.h
>> diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.c
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_msgs.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_msgs.c
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_msgs.c
>> diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.h
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_msgs.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_msgs.h
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_msgs.h
>> diff --git a/drivers/media/platform/qcom/venus/hfi_parser.c
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_parser.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_parser.c
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_parser.c
>> diff --git a/drivers/media/platform/qcom/venus/hfi_parser.h
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_parser.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_parser.h
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_parser.h
>> diff --git a/drivers/media/platform/qcom/venus/hfi_plat_bufs.h
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_plat_bufs.h
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h
>> diff --git a/drivers/media/platform/qcom/venus/hfi_plat_bufs_v6.c
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_plat_bufs_v6.c
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c
>> diff --git a/drivers/media/platform/qcom/venus/hfi_platform.c
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_platform.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_platform.c
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform.c
>> diff --git a/drivers/media/platform/qcom/venus/hfi_platform.h
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_platform.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_platform.h
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform.h
>> diff --git a/drivers/media/platform/qcom/venus/hfi_platform_v4.c
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_platform_v4.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_platform_v4.c
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform_v4.c
>> diff --git a/drivers/media/platform/qcom/venus/hfi_platform_v6.c
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_platform_v6.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_platform_v6.c
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform_v6.c
>> diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
>> similarity index 99%
>> rename from drivers/media/platform/qcom/venus/hfi_venus.c
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
>> index f9437b6..5a68db9 100644
>> --- a/drivers/media/platform/qcom/venus/hfi_venus.c
>> +++ b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c
>> @@ -13,11 +13,12 @@
>> #include <linux/slab.h>
>> #include "core.h"
>> +#include "../firmware.h"
>> #include "hfi_cmds.h"
>> #include "hfi_msgs.h"
>> #include "hfi_venus.h"
>> #include "hfi_venus_io.h"
>> -#include "firmware.h"
>> +#include "firmware_no_tz.h"
>> #define HFI_MASK_QHDR_TX_TYPE        0xff000000
>> #define HFI_MASK_QHDR_RX_TYPE        0x00ff0000
>> @@ -635,7 +636,10 @@ static int venus_power_off(struct venus_hfi_device
>> *hdev)
>>       if (!hdev->power_enabled)
>>           return 0;
>> -    ret = venus_set_hw_state_suspend(hdev->core);
>> +    if (hdev->core->use_tz)
>> +        ret = set_hw_state(false);
>> +    else
>> +        set_hw_state_no_tz(hdev->core, false);
>>       if (ret)
>>           return ret;
>> @@ -655,7 +659,13 @@ static int venus_power_on(struct venus_hfi_device
>> *hdev)
>>       if (hdev->power_enabled)
>>           return 0;
>> -    ret = venus_set_hw_state_resume(hdev->core);
>> +    if (hdev->core->use_tz) {
>> +        ret = set_hw_state(true);
>> +        if (ret == -EINVAL)
>> +            ret = 0;
>> +    } else {
>> +        set_hw_state_no_tz(hdev->core, true);
>> +    }
>>       if (ret)
>>           goto err;
>> @@ -668,7 +678,10 @@ static int venus_power_on(struct venus_hfi_device
>> *hdev)
>>       return 0;
>> err_suspend:
>> -    venus_set_hw_state_suspend(hdev->core);
>> +    if (hdev->core->use_tz)
>> +        set_hw_state(false);
>> +    else
>> +        set_hw_state_no_tz(hdev->core, false);
>> err:
>>       hdev->power_enabled = false;
>>       return ret;
>> diff --git a/drivers/media/platform/qcom/venus/hfi_venus.h
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_venus.h
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus.h
>> diff --git a/drivers/media/platform/qcom/venus/hfi_venus_io.h
>> b/drivers/media/platform/qcom/vcodec/venus/hfi_venus_io.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/hfi_venus_io.h
>> rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus_io.h
>> diff --git a/drivers/media/platform/qcom/venus/pm_helpers.c
>> b/drivers/media/platform/qcom/vcodec/venus/pm_helpers.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/pm_helpers.c
>> rename to drivers/media/platform/qcom/vcodec/venus/pm_helpers.c
>> diff --git a/drivers/media/platform/qcom/venus/pm_helpers.h
>> b/drivers/media/platform/qcom/vcodec/venus/pm_helpers.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/pm_helpers.h
>> rename to drivers/media/platform/qcom/vcodec/venus/pm_helpers.h
>> diff --git a/drivers/media/platform/qcom/venus/vdec.c
>> b/drivers/media/platform/qcom/vcodec/venus/vdec.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/vdec.c
>> rename to drivers/media/platform/qcom/vcodec/venus/vdec.c
>> diff --git a/drivers/media/platform/qcom/venus/vdec.h
>> b/drivers/media/platform/qcom/vcodec/venus/vdec.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/vdec.h
>> rename to drivers/media/platform/qcom/vcodec/venus/vdec.h
>> diff --git a/drivers/media/platform/qcom/venus/vdec_ctrls.c
>> b/drivers/media/platform/qcom/vcodec/venus/vdec_ctrls.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/vdec_ctrls.c
>> rename to drivers/media/platform/qcom/vcodec/venus/vdec_ctrls.c
>> diff --git a/drivers/media/platform/qcom/venus/venc.c
>> b/drivers/media/platform/qcom/vcodec/venus/venc.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/venc.c
>> rename to drivers/media/platform/qcom/vcodec/venus/venc.c
>> diff --git a/drivers/media/platform/qcom/venus/venc.h
>> b/drivers/media/platform/qcom/vcodec/venus/venc.h
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/venc.h
>> rename to drivers/media/platform/qcom/vcodec/venus/venc.h
>> diff --git a/drivers/media/platform/qcom/venus/venc_ctrls.c
>> b/drivers/media/platform/qcom/vcodec/venus/venc_ctrls.c
>> similarity index 100%
>> rename from drivers/media/platform/qcom/venus/venc_ctrls.c
>> rename to drivers/media/platform/qcom/vcodec/venus/venc_ctrls.c
>> diff --git a/drivers/media/platform/qcom/venus/firmware.c
>> b/drivers/media/platform/qcom/venus/firmware.c
>> deleted file mode 100644
>> index fe7da2b..0000000
>> --- a/drivers/media/platform/qcom/venus/firmware.c
>> +++ /dev/null
>> @@ -1,363 +0,0 @@
>> -// SPDX-License-Identifier: GPL-2.0-only
>> -/*
>> - * Copyright (C) 2017 Linaro Ltd.
>> - */
>> -
>> -#include <linux/device.h>
>> -#include <linux/firmware.h>
>> -#include <linux/kernel.h>
>> -#include <linux/iommu.h>
>> -#include <linux/io.h>
>> -#include <linux/of.h>
>> -#include <linux/of_address.h>
>> -#include <linux/of_reserved_mem.h>
>> -#include <linux/platform_device.h>
>> -#include <linux/of_device.h>
>> -#include <linux/firmware/qcom/qcom_scm.h>
>> -#include <linux/sizes.h>
>> -#include <linux/soc/qcom/mdt_loader.h>
>> -
>> -#include "core.h"
>> -#include "firmware.h"
>> -#include "hfi_venus_io.h"
>> -
>> -#define VENUS_PAS_ID            9
>> -#define VENUS_FW_MEM_SIZE        (6 * SZ_1M)
>> -#define VENUS_FW_START_ADDR        0x0
>> -
>> -static void venus_reset_cpu(struct venus_core *core)
>> -{
>> -    u32 fw_size = core->fw.mapped_mem_size;
>> -    void __iomem *wrapper_base;
>> -
>> -    if (IS_IRIS2_1(core))
>> -        wrapper_base = core->wrapper_tz_base;
>> -    else
>> -        wrapper_base = core->wrapper_base;
>> -
>> -    writel(0, wrapper_base + WRAPPER_FW_START_ADDR);
>> -    writel(fw_size, wrapper_base + WRAPPER_FW_END_ADDR);
>> -    writel(0, wrapper_base + WRAPPER_CPA_START_ADDR);
>> -    writel(fw_size, wrapper_base + WRAPPER_CPA_END_ADDR);
>> -    writel(fw_size, wrapper_base + WRAPPER_NONPIX_START_ADDR);
>> -    writel(fw_size, wrapper_base + WRAPPER_NONPIX_END_ADDR);
>> -
>> -    if (IS_IRIS2_1(core)) {
>> -        /* Bring XTSS out of reset */
>> -        writel(0, wrapper_base + WRAPPER_TZ_XTSS_SW_RESET);
>> -    } else {
>> -        writel(0x0, wrapper_base + WRAPPER_CPU_CGC_DIS);
>> -        writel(0x0, wrapper_base + WRAPPER_CPU_CLOCK_CONFIG);
>> -
>> -        /* Bring ARM9 out of reset */
>> -        writel(0, wrapper_base + WRAPPER_A9SS_SW_RESET);
>> -    }
>> -}
>> -
>> -int venus_set_hw_state(struct venus_core *core, bool resume)
>> -{
>> -    int ret;
>> -
>> -    if (core->use_tz) {
>> -        ret = qcom_scm_set_remote_state(resume, 0);
>> -        if (resume && ret == -EINVAL)
>> -            ret = 0;
>> -        return ret;
>> -    }
>> -
>> -    if (resume) {
>> -        venus_reset_cpu(core);
>> -    } else {
>> -        if (IS_IRIS2_1(core))
>> -            writel(WRAPPER_XTSS_SW_RESET_BIT,
>> -                   core->wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
>> -        else
>> -            writel(WRAPPER_A9SS_SW_RESET_BIT,
>> -                   core->wrapper_base + WRAPPER_A9SS_SW_RESET);
>> -    }
>> -
>> -    return 0;
>> -}
>> -
>> -static int venus_load_fw(struct venus_core *core, const char *fwname,
>> -             phys_addr_t *mem_phys, size_t *mem_size)
>> -{
>> -    const struct firmware *mdt;
>> -    struct reserved_mem *rmem;
>> -    struct device_node *node;
>> -    struct device *dev;
>> -    ssize_t fw_size;
>> -    void *mem_va;
>> -    int ret;
>> -
>> -    *mem_phys = 0;
>> -    *mem_size = 0;
>> -
>> -    dev = core->dev;
>> -    node = of_parse_phandle(dev->of_node, "memory-region", 0);
>> -    if (!node) {
>> -        dev_err(dev, "no memory-region specified\n");
>> -        return -EINVAL;
>> -    }
>> -
>> -    rmem = of_reserved_mem_lookup(node);
>> -    of_node_put(node);
>> -    if (!rmem) {
>> -        dev_err(dev, "failed to lookup reserved memory-region\n");
>> -        return -EINVAL;
>> -    }
>> -
>> -    ret = request_firmware(&mdt, fwname, dev);
>> -    if (ret < 0)
>> -        return ret;
>> -
>> -    fw_size = qcom_mdt_get_size(mdt);
>> -    if (fw_size < 0) {
>> -        ret = fw_size;
>> -        goto err_release_fw;
>> -    }
>> -
>> -    *mem_phys = rmem->base;
>> -    *mem_size = rmem->size;
>> -
>> -    if (*mem_size < fw_size || fw_size > VENUS_FW_MEM_SIZE) {
>> -        ret = -EINVAL;
>> -        goto err_release_fw;
>> -    }
>> -
>> -    mem_va = memremap(*mem_phys, *mem_size, MEMREMAP_WC);
>> -    if (!mem_va) {
>> -        dev_err(dev, "unable to map memory region %pa size %#zx\n",
>> mem_phys, *mem_size);
>> -        ret = -ENOMEM;
>> -        goto err_release_fw;
>> -    }
>> -
>> -    if (core->use_tz)
>> -        ret = qcom_mdt_load(dev, mdt, fwname, VENUS_PAS_ID,
>> -                    mem_va, *mem_phys, *mem_size, NULL);
>> -    else
>> -        ret = qcom_mdt_load_no_init(dev, mdt, fwname, VENUS_PAS_ID,
>> -                        mem_va, *mem_phys, *mem_size, NULL);
>> -
>> -    memunmap(mem_va);
>> -err_release_fw:
>> -    release_firmware(mdt);
>> -    return ret;
>> -}
>> -
>> -static int venus_boot_no_tz(struct venus_core *core, phys_addr_t mem_phys,
>> -                size_t mem_size)
>> -{
>> -    struct iommu_domain *iommu;
>> -    struct device *dev;
>> -    int ret;
>> -
>> -    dev = core->fw.dev;
>> -    if (!dev)
>> -        return -EPROBE_DEFER;
>> -
>> -    iommu = core->fw.iommu_domain;
>> -    core->fw.mapped_mem_size = mem_size;
>> -
>> -    ret = iommu_map(iommu, VENUS_FW_START_ADDR, mem_phys, mem_size,
>> -            IOMMU_READ | IOMMU_WRITE | IOMMU_PRIV, GFP_KERNEL);
>> -    if (ret) {
>> -        dev_err(dev, "could not map video firmware region\n");
>> -        return ret;
>> -    }
>> -
>> -    venus_reset_cpu(core);
>> -
>> -    return 0;
>> -}
>> -
>> -static int venus_shutdown_no_tz(struct venus_core *core)
>> -{
>> -    const size_t mapped = core->fw.mapped_mem_size;
>> -    struct iommu_domain *iommu;
>> -    size_t unmapped;
>> -    u32 reg;
>> -    struct device *dev = core->fw.dev;
>> -    void __iomem *wrapper_base = core->wrapper_base;
>> -    void __iomem *wrapper_tz_base = core->wrapper_tz_base;
>> -
>> -    if (IS_IRIS2_1(core)) {
>> -        /* Assert the reset to XTSS */
>> -        reg = readl(wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
>> -        reg |= WRAPPER_XTSS_SW_RESET_BIT;
>> -        writel(reg, wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET);
>> -    } else {
>> -        /* Assert the reset to ARM9 */
>> -        reg = readl(wrapper_base + WRAPPER_A9SS_SW_RESET);
>> -        reg |= WRAPPER_A9SS_SW_RESET_BIT;
>> -        writel(reg, wrapper_base + WRAPPER_A9SS_SW_RESET);
>> -    }
>> -
>> -    iommu = core->fw.iommu_domain;
>> -
>> -    if (core->fw.mapped_mem_size && iommu) {
>> -        unmapped = iommu_unmap(iommu, VENUS_FW_START_ADDR, mapped);
>> -
>> -        if (unmapped != mapped)
>> -            dev_err(dev, "failed to unmap firmware\n");
>> -        else
>> -            core->fw.mapped_mem_size = 0;
>> -    }
>> -
>> -    return 0;
>> -}
>> -
>> -int venus_boot(struct venus_core *core)
>> -{
>> -    struct device *dev = core->dev;
>> -    const struct venus_resources *res = core->res;
>> -    const char *fwpath = NULL;
>> -    phys_addr_t mem_phys;
>> -    size_t mem_size;
>> -    int ret;
>> -
>> -    if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
>> -        (core->use_tz && !qcom_scm_is_available()))
>> -        return -EPROBE_DEFER;
>> -
>> -    ret = of_property_read_string_index(dev->of_node, "firmware-name", 0,
>> -                        &fwpath);
>> -    if (ret)
>> -        fwpath = core->res->fwname;
>> -
>> -    ret = venus_load_fw(core, fwpath, &mem_phys, &mem_size);
>> -    if (ret) {
>> -        dev_err(dev, "fail to load video firmware\n");
>> -        return -EINVAL;
>> -    }
>> -
>> -    core->fw.mem_size = mem_size;
>> -    core->fw.mem_phys = mem_phys;
>> -
>> -    if (core->use_tz)
>> -        ret = qcom_scm_pas_auth_and_reset(VENUS_PAS_ID);
>> -    else
>> -        ret = venus_boot_no_tz(core, mem_phys, mem_size);
>> -
>> -    if (ret)
>> -        return ret;
>> -
>> -    if (core->use_tz && res->cp_size) {
>> -        /*
>> -         * Clues for porting using downstream data:
>> -         * cp_start = 0
>> -         * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and
>> not size!
>> -         *   This works, as the non-secure context bank is placed
>> -         *   contiguously right after the Content Protection region.
>> -         *
>> -         * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0]
>> -         * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1]
>> -         */
>> -        ret = qcom_scm_mem_protect_video_var(res->cp_start,
>> -                             res->cp_size,
>> -                             res->cp_nonpixel_start,
>> -                             res->cp_nonpixel_size);
>> -        if (ret) {
>> -            qcom_scm_pas_shutdown(VENUS_PAS_ID);
>> -            dev_err(dev, "set virtual address ranges fail (%d)\n",
>> -                ret);
>> -            return ret;
>> -        }
>> -    }
>> -
>> -    return 0;
>> -}
>> -
>> -int venus_shutdown(struct venus_core *core)
>> -{
>> -    int ret;
>> -
>> -    if (core->use_tz)
>> -        ret = qcom_scm_pas_shutdown(VENUS_PAS_ID);
>> -    else
>> -        ret = venus_shutdown_no_tz(core);
>> -
>> -    return ret;
>> -}
>> -
>> -int venus_firmware_init(struct venus_core *core)
>> -{
>> -    struct platform_device_info info;
>> -    struct iommu_domain *iommu_dom;
>> -    struct platform_device *pdev;
>> -    struct device_node *np;
>> -    int ret;
>> -
>> -    np = of_get_child_by_name(core->dev->of_node, "video-firmware");
>> -    if (!np) {
>> -        core->use_tz = true;
>> -        return 0;
>> -    }
>> -
>> -    memset(&info, 0, sizeof(info));
>> -    info.fwnode = &np->fwnode;
>> -    info.parent = core->dev;
>> -    info.name = np->name;
>> -    info.dma_mask = DMA_BIT_MASK(32);
>> -
>> -    pdev = platform_device_register_full(&info);
>> -    if (IS_ERR(pdev)) {
>> -        of_node_put(np);
>> -        return PTR_ERR(pdev);
>> -    }
>> -
>> -    pdev->dev.of_node = np;
>> -
>> -    ret = of_dma_configure(&pdev->dev, np, true);
>> -    if (ret) {
>> -        dev_err(core->dev, "dma configure fail\n");
>> -        goto err_unregister;
>> -    }
>> -
>> -    core->fw.dev = &pdev->dev;
>> -
>> -    iommu_dom = iommu_domain_alloc(&platform_bus_type);
>> -    if (!iommu_dom) {
>> -        dev_err(core->fw.dev, "Failed to allocate iommu domain\n");
>> -        ret = -ENOMEM;
>> -        goto err_unregister;
>> -    }
>> -
>> -    ret = iommu_attach_device(iommu_dom, core->fw.dev);
>> -    if (ret) {
>> -        dev_err(core->fw.dev, "could not attach device\n");
>> -        goto err_iommu_free;
>> -    }
>> -
>> -    core->fw.iommu_domain = iommu_dom;
>> -
>> -    of_node_put(np);
>> -
>> -    return 0;
>> -
>> -err_iommu_free:
>> -    iommu_domain_free(iommu_dom);
>> -err_unregister:
>> -    platform_device_unregister(pdev);
>> -    of_node_put(np);
>> -    return ret;
>> -}
>> -
>> -void venus_firmware_deinit(struct venus_core *core)
>> -{
>> -    struct iommu_domain *iommu;
>> -
>> -    if (!core->fw.dev)
>> -        return;
>> -
>> -    iommu = core->fw.iommu_domain;
>> -
>> -    iommu_detach_device(iommu, core->fw.dev);
>> -
>> -    if (core->fw.iommu_domain) {
>> -        iommu_domain_free(iommu);
>> -        core->fw.iommu_domain = NULL;
>> -    }
>> -
>> -    platform_device_unregister(to_platform_device(core->fw.dev));
>> -}
>> diff --git a/drivers/media/platform/qcom/venus/firmware.h
>> b/drivers/media/platform/qcom/venus/firmware.h
>> deleted file mode 100644
>> index aaccd84..0000000
>> --- a/drivers/media/platform/qcom/venus/firmware.h
>> +++ /dev/null
>> @@ -1,26 +0,0 @@
>> -/* SPDX-License-Identifier: GPL-2.0-only */
>> -/*
>> - * Copyright (C) 2017 Linaro Ltd.
>> - */
>> -#ifndef __VENUS_FIRMWARE_H__
>> -#define __VENUS_FIRMWARE_H__
>> -
>> -struct device;
>> -
>> -int venus_firmware_init(struct venus_core *core);
>> -void venus_firmware_deinit(struct venus_core *core);
>> -int venus_boot(struct venus_core *core);
>> -int venus_shutdown(struct venus_core *core);
>> -int venus_set_hw_state(struct venus_core *core, bool suspend);
>> -
>> -static inline int venus_set_hw_state_suspend(struct venus_core *core)
>> -{
>> -    return venus_set_hw_state(core, false);
>> -}
>> -
>> -static inline int venus_set_hw_state_resume(struct venus_core *core)
>> -{
>> -    return venus_set_hw_state(core, true);
>> -}
>> -
>> -#endif
>

2023-12-20 08:03:00

[permalink] [raw]

Subject: Re: [PATCH v2 01/34] media: introduce common helpers for video firmware handling

On 12/19/2023 6:56 PM, Dmitry Baryshkov wrote:
> On Tue, 19 Dec 2023 at 13:40, Bryan O'Donoghue
> <[email protected]> wrote:
>>
>> On 18/12/2023 11:31, Dikshita Agarwal wrote:
>>> Re-organize the video driver code by introducing a new folder
>>> 'vcodec' and placing 'venus' driver code inside that.
>>>
>>> Introduce common helpers for trustzone based firmware
>>> load/unload etc. which are placed in common folder
>>> i.e. 'vcodec'.
>>> Use these helpers in 'venus' driver. These helpers will be
>>> used by 'iris' driver as well which is introduced later
>>> in this patch series.
>>>
>>> Signed-off-by: Dikshita Agarwal <[email protected]>
>>> ---
>>
>> This is a very large patch, I think it needs to be broken up into
>> smaller chunks.
>>
>> #1 Introduce common helper functions
>> #2 Use common helper functions
>
> This will make it harder to review. It's usually preferred to have a
> single 'move' patch instead of two (add + remove). But I definitely
> agree that the size of the patch is big. Somewhat it is related to the
> fact that this doesn't only introduce helpers, but also reshuffles the
> rest of the code.
>
Thanking along the same lines as Dmitry's, we wanted to show the usage of
these common helpers in iris driver as well, hence we added these patches
as part of this series.
I can send these patches 1-3 as separate series and mark the dependency of
iris series to that if you would prefer that way.

Also, as mentioned in comments in previous patches, to make the common
helper code generic, moving just the code from venus to vcodec was not
sufficient and more changes were needed in calling functions of venus as well.
>>
>> Its alot of code to try to eat in the one go.
>>
>> Could you consider making patches 1-3 a standalone series to reduce the
>> amount of code to review here ?
>
> This sounds like a good idea.
> >>
>> * 77e7025529d7c - (HEAD -> linux-stable-master-23-12-18-iris-v2) media:
>> iris: add power management for encoder (21 hours ago)
>>
>> * ceb6a6f023fd3 - (tag: v6.7-rc6, linux-stable/master) Linux 6.7-rc6 (2
>> days ago)
>>
>> git diff ceb6a6f023fd3 | wc -l
>>
>> 21243
>>
>> Also I feel it wouild give more time for the changes to "digest" though
>> upstream users and to find any unintended bugs.
>>
>>> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
>>> + size_t *mem_size, u32 pas_id, bool use_tz)
>>> +{
>>> + const struct firmware *firmware = NULL;
>>> + struct reserved_mem *rmem;
>>> + struct device_node *node;
>>> + void *mem_virt = NULL;
>>> + ssize_t fw_size = 0;
>>> + int ret;
>>> +
>>> + if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
>>> + (use_tz && !qcom_scm_is_available()))
>>> + return -EPROBE_DEFER;
>>> +
>>> + if (!fw_name || !(*fw_name))
>>> + return -EINVAL;
>>
>> The parameter check should come before the qcom_scm_is_available()
>>
>> No matter how many times you -EPROBE_DEFER -EINVAL is still -EINVAL.
>>
Sure, will check and do the needful.
>>> +
>>> + *mem_phys = 0;
>>> + *mem_size = 0;
>>
>> I don't think you need this, you don't appear to use these variables
>> before you assign them below.
>>
That's true, will fix.
>>
>>> +
>>> + *mem_phys = rmem->base;
>>> + *mem_size = rmem->size;
>>
>>> +
>>> +int auth_reset_fw(u32 pas_id)
>>> +{
>>> + return qcom_scm_pas_auth_and_reset(pas_id);
>>> +}
>>> +
>>> +void unload_fw(u32 pas_id)
>>> +{
>>> + qcom_scm_pas_shutdown(pas_id);
>>> +}
>>> +
>>
>> Do these wrapper functions add anything ? Some kind of validity check on
>> the pas_id otherwise I'm not sure these are justified.
>>
>>> +int set_hw_state(bool resume)
>>> +{
>>> + return qcom_scm_set_remote_state(resume, 0);
>>> +}
Will check more on this and do required changes.
>>> diff --git a/drivers/media/platform/qcom/vcodec/firmware.h b/drivers/media/platform/qcom/vcodec/firmware.h
>>> new file mode 100644
>>> index 0000000..7d410a8
>>> --- /dev/null
>>
>> ---
>> bod
>>
>>
>
>

2023-12-20 08:06:37

[permalink] [raw]

Subject: Re: [PATCH v2 09/34] media: iris: initialize shared queues for host and firmware communication

On 12/19/2023 3:03 AM, Konrad Dybcio wrote:
>
>
> On 12/18/23 12:32, Dikshita Agarwal wrote:
>> Shared queues are used for communication between driver and firmware.
>> There are 3 types of queues:
>> Command queue - driver to write any command to firmware.
>> Message queue - firmware to send any response to driver.
>> Debug queue - firmware to write debug message.
>>
>> Above queues are initialized and configured to firmware during probe.
>>
>> Signed-off-by: Dikshita Agarwal <[email protected]>
>> ---
> [...]
>
>> +    ret = hfi_queue_init(core->dev, &core->iface_q_table, &core->sfr,
>> +                 &core->command_queue, &core->message_queue,
>> +                 &core->debug_queue, core);
>> +    if (ret) {
>> +        dev_err_probe(core->dev, ret,
>> +                  "%s: interface queues init failed\n", __func__);
>> +        goto err_vdev_unreg;
>> +    }
>> +
>>       return ret;
> Like before, you're suppose to return dev_err_probe
Sure, will fix all such instances.
>
> Konrad

2023-12-20 08:06:59

[permalink] [raw]

Subject: Re: [PATCH v2 09/34] media: iris: initialize shared queues for host and firmware communication

On 12/19/2023 12:16 AM, Dmitry Baryshkov wrote:
> On 18/12/2023 13:32, Dikshita Agarwal wrote:
>> Shared queues are used for communication between driver and firmware.
>> There are 3 types of queues:
>> Command queue - driver to write any command to firmware.
>> Message queue - firmware to send any response to driver.
>> Debug queue - firmware to write debug message.
>>
>> Above queues are initialized and configured to firmware during probe.
>>
>> Signed-off-by: Dikshita Agarwal <[email protected]>
>> ---
>> drivers/media/platform/qcom/vcodec/iris/Makefile   | 2 ++
>> .../media/platform/qcom/vcodec/iris/iris_core.h    | 11 ++++++++
>> .../media/platform/qcom/vcodec/iris/iris_probe.c   | 31
>> ++++++++++++++++++++++
>> 3 files changed, 44 insertions(+)
>>
>> diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile
>> b/drivers/media/platform/qcom/vcodec/iris/Makefile
>> index 12a74de..59798e5d 100644
>> --- a/drivers/media/platform/qcom/vcodec/iris/Makefile
>> +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile
>> @@ -1,3 +1,5 @@
>> +iris-objs += ../hfi_queue.o
>> +
>> iris-objs += iris_probe.o \
>>                resources.o \
>>                iris_state.o
>> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
>> b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
>> index 56a5b7d..77124f9 100644
>> --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h
>> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h
>> @@ -9,6 +9,7 @@
>> #include <linux/types.h>
>> #include <media/v4l2-device.h>
>> +#include "../hfi_queue.h"
>> #include "iris_state.h"
>> /**
>> @@ -30,6 +31,11 @@
>>    * @reset_tbl: table of iris reset clocks
>>    * @reset_count: count of iris reset clocks
>>    * @state: current state of core
>> + * @iface_q_table: Interface queue table memory
>> + * @command_queue: shared interface queue to send commands to firmware
>> + * @message_queue: shared interface queue to receive responses from
>> firmware
>> + * @debug_queue: shared interface queue to receive debug info from firmware
>> + * @sfr: SFR register memory
>>    */
>> struct iris_core {
>> @@ -49,6 +55,11 @@ struct iris_core {
>>       struct reset_info            *reset_tbl;
>>       u32                    reset_count;
>>       enum iris_core_state            state;
>> +    struct mem_desc                iface_q_table;
>> +    struct iface_q_info            command_queue;
>> +    struct iface_q_info            message_queue;
>> +    struct iface_q_info            debug_queue;
>> +    struct mem_desc                sfr;
>> };
>> #endif
>> diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
>> b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
>> index 7bb9c92..fd349a3 100644
>> --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
>> +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c
>> @@ -7,6 +7,7 @@
>> #include <linux/of_device.h>
>> #include <linux/platform_device.h>
>> +#include "../hfi_queue.h"
>> #include "iris_core.h"
>> #include "resources.h"
>> @@ -50,6 +51,10 @@ static void iris_remove(struct platform_device *pdev)
>>       if (!core)
>>           return;
>> +    hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr,
>> +             &core->command_queue, &core->message_queue,
>> +             &core->debug_queue);
>> +
>>       video_unregister_device(core->vdev_dec);
>>       v4l2_device_unregister(&core->v4l2_dev);
>> @@ -59,6 +64,7 @@ static int iris_probe(struct platform_device *pdev)
>> {
>>       struct device *dev = &pdev->dev;
>>       struct iris_core *core;
>> +    u64 dma_mask;
>>       int ret;
>>       core = devm_kzalloc(&pdev->dev, sizeof(*core), GFP_KERNEL);
>> @@ -91,8 +97,33 @@ static int iris_probe(struct platform_device *pdev)
>>       platform_set_drvdata(pdev, core);
>> +    /*
>> +     * Specify the max value of address space, which can be used
>> +     * for buffer transactions.
>> +     */
>> +    dma_mask = DMA_BIT_MASK(32);
>> +    dma_mask &= ~BIT(29);
>> +
>> +    ret = dma_set_mask_and_coherent(dev, dma_mask);
>> +    if (ret)
>> +        goto err_vdev_unreg;
>> +
>> +    dma_set_max_seg_size(&pdev->dev, (unsigned int)DMA_BIT_MASK(32));
>> +    dma_set_seg_boundary(&pdev->dev, (unsigned long)DMA_BIT_MASK(64));
>
> This isn't related to queues.
>
Using this DMA mask, we define the upper limit of memory to be used to
allocate the queues, hence this is added with this patch.
>> +
>> +    ret = hfi_queue_init(core->dev, &core->iface_q_table, &core->sfr,
>> +                 &core->command_queue, &core->message_queue,
>> +                 &core->debug_queue, core);
>> +    if (ret) {
>> +        dev_err_probe(core->dev, ret,
>> +                  "%s: interface queues init failed\n", __func__);
>> +        goto err_vdev_unreg;
>> +    }
>> +
>>       return ret;
>> +err_vdev_unreg:
>> +    iris_unregister_video_device(core);
>> err_v4l2_unreg:
>>       v4l2_device_unregister(&core->v4l2_dev);
>>
>

2023-12-20 08:12:44

[permalink] [raw]

Subject: Re: [PATCH v2 01/34] media: introduce common helpers for video firmware handling

On Wed, 20 Dec 2023 at 10:01, Dikshita Agarwal
<[email protected]> wrote:
>
>
>
> On 12/18/2023 11:54 PM, Dmitry Baryshkov wrote:
> > On 18/12/2023 13:31, Dikshita Agarwal wrote:
> >> Re-organize the video driver code by introducing a new folder
> >> 'vcodec' and placing 'venus' driver code inside that.
> >>
> >> Introduce common helpers for trustzone based firmware
> >> load/unload etc. which are placed in common folder
> >> i.e. 'vcodec'.
> >> Use these helpers in 'venus' driver. These helpers will be
> >> used by 'iris' driver as well which is introduced later
> >> in this patch series.
> >
> > But why do you need to move the venus driver to subdir?
>
> Currently venus driver is present in drivers/media/platform/qcom folder
> which also has camss folder. We introduced vcodec to keep common code and
> moved venus inside that, to indicate that the common code is for vcodec
> drivers i.e venus and iris. Keeping this in qcom folder would mean, common
> code will be used for camss only which is not the case here.

you can have .../platform/qcom/camss, .../platform/qcom/vcodec-common,
.../platform/qcom/venus and .../platform/qcom/iris.

If you were to use build helpers in a proper kernel module, this would
be more obvious.

> >
> >>
> >> Signed-off-by: Dikshita Agarwal <[email protected]>
> >> ---
> >> drivers/media/platform/qcom/Kconfig | 2 +-
> >> drivers/media/platform/qcom/Makefile | 2 +-
> >> drivers/media/platform/qcom/vcodec/firmware.c | 147 +++++++++
> >> drivers/media/platform/qcom/vcodec/firmware.h | 21 ++
> >> .../media/platform/qcom/{ => vcodec}/venus/Kconfig | 0
> >> .../platform/qcom/{ => vcodec}/venus/Makefile | 4 +-
> >> .../media/platform/qcom/{ => vcodec}/venus/core.c | 102 +++++-
> >> .../media/platform/qcom/{ => vcodec}/venus/core.h | 0
> >> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.c | 0
> >> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.h | 0
> >> .../platform/qcom/vcodec/venus/firmware_no_tz.c | 194 +++++++++++
> >> .../platform/qcom/vcodec/venus/firmware_no_tz.h | 19 ++
> >> .../platform/qcom/{ => vcodec}/venus/helpers.c | 0
> >> .../platform/qcom/{ => vcodec}/venus/helpers.h | 0
> >> .../media/platform/qcom/{ => vcodec}/venus/hfi.c | 0
> >> .../media/platform/qcom/{ => vcodec}/venus/hfi.h | 0
> >> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.c | 0
> >> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.h | 0
> >> .../platform/qcom/{ => vcodec}/venus/hfi_helper.h | 0
> >> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.c | 0
> >> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.h | 0
> >> .../platform/qcom/{ => vcodec}/venus/hfi_parser.c | 0
> >> .../platform/qcom/{ => vcodec}/venus/hfi_parser.h | 0
> >> .../qcom/{ => vcodec}/venus/hfi_plat_bufs.h | 0
> >> .../qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c | 0
> >> .../qcom/{ => vcodec}/venus/hfi_platform.c | 0
> >> .../qcom/{ => vcodec}/venus/hfi_platform.h | 0
> >> .../qcom/{ => vcodec}/venus/hfi_platform_v4.c | 0
> >> .../qcom/{ => vcodec}/venus/hfi_platform_v6.c | 0
> >> .../platform/qcom/{ => vcodec}/venus/hfi_venus.c | 21 +-
> >> .../platform/qcom/{ => vcodec}/venus/hfi_venus.h | 0
> >> .../qcom/{ => vcodec}/venus/hfi_venus_io.h | 0
> >> .../platform/qcom/{ => vcodec}/venus/pm_helpers.c | 0
> >> .../platform/qcom/{ => vcodec}/venus/pm_helpers.h | 0
> >> .../media/platform/qcom/{ => vcodec}/venus/vdec.c | 0
> >> .../media/platform/qcom/{ => vcodec}/venus/vdec.h | 0
> >> .../platform/qcom/{ => vcodec}/venus/vdec_ctrls.c | 0
> >> .../media/platform/qcom/{ => vcodec}/venus/venc.c | 0
> >> .../media/platform/qcom/{ => vcodec}/venus/venc.h | 0
> >> .../platform/qcom/{ => vcodec}/venus/venc_ctrls.c | 0
> >> drivers/media/platform/qcom/venus/firmware.c | 363
> >> ---------------------
> >> drivers/media/platform/qcom/venus/firmware.h | 26 --
> >> 42 files changed, 492 insertions(+), 409 deletions(-)
> >> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.c
> >> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.h
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/Kconfig (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/Makefile (83%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.c (91%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.h (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.c (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.h (100%)
> >> create mode 100644
> >> drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
> >> create mode 100644
> >> drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.c (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.h (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.c (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.h (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.c (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.h (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_helper.h (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.c (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.h (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.c (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.h (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs.h
> >> (100%)
> >> rename drivers/media/platform/qcom/{ =>
> >> vcodec}/venus/hfi_plat_bufs_v6.c (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.c
> >> (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.h
> >> (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v4.c
> >> (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v6.c
> >> (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.c (99%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.h (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus_io.h
> >> (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.c (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.h (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.c (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.h (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec_ctrls.c (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.c (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.h (100%)
> >> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc_ctrls.c (100%)
> >> delete mode 100644 drivers/media/platform/qcom/venus/firmware.c
> >> delete mode 100644 drivers/media/platform/qcom/venus/firmware.h
> >>
> >> diff --git a/drivers/media/platform/qcom/Kconfig
> >> b/drivers/media/platform/qcom/Kconfig
> >> index cc5799b..e94142f 100644
> >> --- a/drivers/media/platform/qcom/Kconfig
> >> +++ b/drivers/media/platform/qcom/Kconfig
> >> @@ -3,4 +3,4 @@
> >> comment "Qualcomm media platform drivers"
> >> source "drivers/media/platform/qcom/camss/Kconfig"
> >> -source "drivers/media/platform/qcom/venus/Kconfig"
> >> +source "drivers/media/platform/qcom/vcodec/venus/Kconfig"
> >> diff --git a/drivers/media/platform/qcom/Makefile
> >> b/drivers/media/platform/qcom/Makefile
> >> index 4f055c3..3d2d82b 100644
> >> --- a/drivers/media/platform/qcom/Makefile
> >> +++ b/drivers/media/platform/qcom/Makefile
> >> @@ -1,3 +1,3 @@
> >> # SPDX-License-Identifier: GPL-2.0-only
> >> obj-y += camss/
> >> -obj-y += venus/
> >> +obj-y += vcodec/venus/
> >> diff --git a/drivers/media/platform/qcom/vcodec/firmware.c
> >> b/drivers/media/platform/qcom/vcodec/firmware.c
> >> new file mode 100644
> >> index 0000000..dbc220a
> >> --- /dev/null
> >> +++ b/drivers/media/platform/qcom/vcodec/firmware.c
> >> @@ -0,0 +1,147 @@
> >> +// SPDX-License-Identifier: GPL-2.0-only
> >> +/*
> >> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
> >> reserved.
> >> + */
> >> +
> >> +#include <linux/device.h>
> >> +#include <linux/dma-mapping.h>
> >> +#include <linux/firmware.h>
> >> +#include <linux/kernel.h>
> >> +#include <linux/iommu.h>
> >> +#include <linux/of_device.h>
> >> +#include <linux/firmware/qcom/qcom_scm.h>
> >> +#include <linux/of_reserved_mem.h>
> >> +#include <linux/platform_device.h>
> >> +#include <linux/soc/qcom/mdt_loader.h>
> >> +
> >> +#include "firmware.h"
> >> +
> >> +bool use_tz(struct device *core_dev)
> >
> > All these functions must get some sane prefix. Otherwise a generic 'use_tz'
> > function is too polluting for the global namespace.
> >
> I understand, will check and do the needful.
> >> +{
> >> + struct device_node *np;
> >> +
> >> + np = of_get_child_by_name(core_dev->of_node, "video-firmware");
> >> + if (!np)
> >> + return true;
> >> +
> >> + return false;
> >> +}
> >> +
> >> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
> >> + u32 cp_nonpixel_size, u32 pas_id)
> >> +{
> >> + int ret;
> >> + /*
> >> + * Clues for porting using downstream data:
> >> + * cp_start = 0
> >> + * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and not size!
> >> + * This works, as the non-secure context bank is placed
> >> + * contiguously right after the Content Protection region.
> >> + *
> >> + * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0]
> >> + * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1]
> >> + */
> >> + ret = qcom_scm_mem_protect_video_var(cp_start,
> >> + cp_size,
> >> + cp_nonpixel_start,
> >> + cp_nonpixel_size);
> >> + if (ret)
> >> + qcom_scm_pas_shutdown(pas_id);
> >> +
> >> + return ret;
> >> +}
> >> +
> >> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
> >> + size_t *mem_size, u32 pas_id, bool use_tz)
> >> +{
> >> + const struct firmware *firmware = NULL;
> >> + struct reserved_mem *rmem;
> >> + struct device_node *node;
> >> + void *mem_virt = NULL;
> >> + ssize_t fw_size = 0;
> >> + int ret;
> >> +
> >> + if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
> >
> > Why? Can you just depend on it?
> >
> Sure, Will check this and get back.
> >> + (use_tz && !qcom_scm_is_available()))
> >> + return -EPROBE_DEFER;
> >> +
> >> + if (!fw_name || !(*fw_name))
> >> + return -EINVAL;
> >> +
> >> + *mem_phys = 0;
> >> + *mem_size = 0;
> >> +
> >> + node = of_parse_phandle(dev->of_node, "memory-region", 0);
> >> + if (!node) {
> >> + dev_err(dev, "no memory-region specified\n");
> >> + return -EINVAL;
> >> + }
> >> +
> >> + rmem = of_reserved_mem_lookup(node);
> >> + of_node_put(node);
> >> + if (!rmem) {
> >> + dev_err(dev, "failed to lookup reserved memory-region\n");
> >> + return -EINVAL;
> >> + }
> >> +
> >> + ret = request_firmware(&firmware, fw_name, dev);
> >> + if (ret) {
> >> + dev_err(dev, "%s: failed to request fw \"%s\", error %d\n",
> >> + __func__, fw_name, ret);
> >> + return ret;
> >> + }
> >> +
> >> + fw_size = qcom_mdt_get_size(firmware);
> >> + if (fw_size < 0) {
> >> + ret = fw_size;
> >> + dev_err(dev, "%s: out of bound fw image fw size: %ld\n",
> >> + __func__, fw_size);
> >> + goto err_release_fw;
> >> + }
> >> +
> >> + *mem_phys = rmem->base;
> >> + *mem_size = rmem->size;
> >> +
> >> + if (*mem_size < fw_size) {
> >> + ret = -EINVAL;
> >> + goto err_release_fw;
> >> + }
> >> +
> >> + mem_virt = memremap(*mem_phys, *mem_size, MEMREMAP_WC);
> >> + if (!mem_virt) {
> >> + dev_err(dev, "unable to remap fw memory region %pa size %#zx\n",
> >> + mem_phys, *mem_size);
> >> + goto err_release_fw;
> >> + }
> >> +
> >> + if (use_tz)
> >> + ret = qcom_mdt_load(dev, firmware, fw_name, pas_id, mem_virt,
> >> + *mem_phys, *mem_size, NULL);
> >> + else
> >> + ret = qcom_mdt_load_no_init(dev, firmware, fw_name, pas_id,
> >> mem_virt,
> >> + *mem_phys, *mem_size, NULL);
> >> + if (ret) {
> >> + dev_err(dev, "%s: error %d loading fw \"%s\"\n",
> >> + __func__, ret, fw_name);
> >> + }
> >> +
> >> + memunmap(mem_virt);
> >> +err_release_fw:
> >> + release_firmware(firmware);
> >> + return ret;
> >> +}
> >> +
> >> +int auth_reset_fw(u32 pas_id)
> >> +{
> >> + return qcom_scm_pas_auth_and_reset(pas_id);
> >> +}
> >> +
> >> +void unload_fw(u32 pas_id)
> >> +{
> >> + qcom_scm_pas_shutdown(pas_id);
> >> +}
> >> +
> >> +int set_hw_state(bool resume)
> >> +{
> >> + return qcom_scm_set_remote_state(resume, 0);
> >> +}
> >> diff --git a/drivers/media/platform/qcom/vcodec/firmware.h
> >> b/drivers/media/platform/qcom/vcodec/firmware.h
> >> new file mode 100644
> >> index 0000000..7d410a8
> >> --- /dev/null
> >> +++ b/drivers/media/platform/qcom/vcodec/firmware.h
> >> @@ -0,0 +1,21 @@
> >> +/* SPDX-License-Identifier: GPL-2.0-only */
> >> +/*
> >> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
> >> reserved.
> >> + */
> >> +
> >> +#ifndef _FIRMWARE_H_
> >> +#define _FIRMWARE_H_
> >> +
> >> +#include <linux/device.h>
> >> +#include <linux/types.h>
> >> +
> >> +bool use_tz(struct device *core_dev);
> >> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
> >> + size_t *mem_size, u32 pas_id, bool use_tz);
> >> +int auth_reset_fw(u32 pas_id);
> >> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
> >> + u32 cp_nonpixel_size, u32 pas_id);
> >> +void unload_fw(u32 pas_id);
> >> +int set_hw_state(bool resume);
> >> +
> >> +#endif
> >> diff --git a/drivers/media/platform/qcom/venus/Kconfig
> >> b/drivers/media/platform/qcom/vcodec/venus/Kconfig
> >> similarity index 100%
> >> rename from drivers/media/platform/qcom/venus/Kconfig
> >> rename to drivers/media/platform/qcom/vcodec/venus/Kconfig
> >> diff --git a/drivers/media/platform/qcom/venus/Makefile
> >> b/drivers/media/platform/qcom/vcodec/venus/Makefile
> >> similarity index 83%
> >> rename from drivers/media/platform/qcom/venus/Makefile
> >> rename to drivers/media/platform/qcom/vcodec/venus/Makefile
> >> index 91ee6be..f6f3a88 100644
> >> --- a/drivers/media/platform/qcom/venus/Makefile
> >> +++ b/drivers/media/platform/qcom/vcodec/venus/Makefile
> >> @@ -1,7 +1,9 @@
> >> # SPDX-License-Identifier: GPL-2.0
> >> # Makefile for Qualcomm Venus driver
> >> -venus-core-objs += core.o helpers.o firmware.o \
> >> +venus-core-objs += ../firmware.o
> >> +
> >> +venus-core-objs += core.o helpers.o firmware_no_tz.o \
> >> hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \
> >> hfi_parser.o pm_helpers.o dbgfs.o \
> >> hfi_platform.o hfi_platform_v4.o \
> >> diff --git a/drivers/media/platform/qcom/venus/core.c
> >> b/drivers/media/platform/qcom/vcodec/venus/core.c
> >> similarity index 91%
> >> rename from drivers/media/platform/qcom/venus/core.c
> >> rename to drivers/media/platform/qcom/vcodec/venus/core.c
> >> index 9cffe97..56d9a53 100644
> >> --- a/drivers/media/platform/qcom/venus/core.c
> >> +++ b/drivers/media/platform/qcom/vcodec/venus/core.c
> >> @@ -22,7 +22,8 @@
> >> #include <media/v4l2-ioctl.h>
> >> #include "core.h"
> >> -#include "firmware.h"
> >> +#include "../firmware.h"
> >> +#include "firmware_no_tz.h"
> >> #include "pm_helpers.h"
> >> #include "hfi_venus_io.h"
> >> @@ -86,6 +87,8 @@ static void venus_sys_error_handler(struct
> >> work_struct *work)
> >> struct venus_core *core =
> >> container_of(work, struct venus_core, work.work);
> >> int ret, i, max_attempts = RPM_WAIT_FOR_IDLE_MAX_ATTEMPTS;
> >> + const struct venus_resources *res = core->res;
> >> + const char *fwpath = NULL;
> >> const char *err_msg = "";
> >> bool failed = false;
> >> @@ -107,7 +110,10 @@ static void venus_sys_error_handler(struct
> >> work_struct *work)
> >> mutex_lock(&core->lock);
> >> - venus_shutdown(core);
> >> + if (core->use_tz)
> >> + unload_fw(VENUS_PAS_ID);
> >> + else
> >> + unload_fw_no_tz(core);
> >
> > This is more than introducing helpers.
> >
> The new helpers are written to make the code generic for video drivers.
> which requires changes in the calling function also.
> >> venus_coredump(core);
> >> @@ -127,12 +133,39 @@ static void venus_sys_error_handler(struct
> >> work_struct *work)
> >> failed = true;
> >> }
> >> - ret = venus_boot(core);
> >> + ret = of_property_read_string_index(core->dev->of_node,
> >> "firmware-name", 0,
> >> + &fwpath);
> >> + if (ret)
> >> + fwpath = core->res->fwname;
> >> +
> >> + ret = load_fw(core->dev, fwpath, &core->fw.mem_phys,
> >> &core->fw.mem_size,
> >> + VENUS_PAS_ID, core->use_tz);
> >
> > So, we had a nice local 'venus_boot'. Instead we now have a pile of code
> > with non-generic prefixes, etc. If you are introducing helpers, please
> > refrain from inlining of calling functions, etc. Just move the code to your
> > helpers.
> >
> As mentioned in above comment, the common helpers are written to make the
> code generic. I Will try to make it more clear, working on the same.

First, you move the code, then you make it generic. Or vice versa.
First you split the code, then you move it. Don't do both in the same
patch.

> > NAK for the rest of the patch.

--
With best wishes
Dmitry

2023-12-20 08:14:39

[permalink] [raw]

Subject: Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver

Hi,

On 12/20/2023 1:07 PM, Dmitry Baryshkov wrote:
> On Wed, 20 Dec 2023 at 08:32, Vikash Garodia <[email protected]> wrote:
>>
>> Hi Dmitry,
>>
>> On 12/19/2023 12:08 AM, Dmitry Baryshkov wrote:
>>> On 18/12/2023 13:31, Dikshita Agarwal wrote:
>>>> This patch series introduces support for Qualcomm new video acceleration
>>>> hardware architecture, used for video stream decoding/encoding. This driver
>>>> is based on new communication protocol between video hardware and application
>>>> processor.
>>>
>>> This doesn't answer one important point, you have been asked for v1. What is the
>>> actual change point between Venus and Iris? What has been changed so much that
>>> it demands a separate driver. This is the main question for the cover letter,
>>> which has not been answered so far.
>>>
>>> From what I see from you bindings, the hardware is pretty close to what we see
>>> in the latest venus generations. I asssme that there was a change in the vcodec
>>> inteface to the firmware and other similar changes. Could you please point out,
>>> which parts of Venus driver do no longer work or are not applicable for sm8550
>>
>> The motivation behind having a separate IRIS driver was discussed earlier in [1]
>> In the same discussion, it was ellaborated on how the impact would be with
>> change in the new firmware interface and other video layers in the driver. I can
>> add this in cover letter in the next revision.
>
> Ok. So the changes cover the HFI interface. Is that correct?
Change wise, yes.

>> We see some duplication of code and to handle the same, the series brings in a
>> common code reusability between iris and venus. Aligning the common peices of
>> venus and iris will be a work in progress, once we land the base driver for iris.
>
> This is not how it usually works. Especially not with the patches you
> have posted.
>
> I have the following suggestion how this story can continue:
> You can _start_ by reworking venus driver, separating the HFI /
> firmware / etc interface to an internal interface in the driver. Then
> implement Iris as a plug in for that interface. I might be mistaken
> here, but I think this is the way how this can be beneficial for both
> the video en/decoding on both old and new platforms.

HFI/firmware interface is already confined to HFI layer in the existing venus
driver. We explained in the previous discussion [1], on how the HFI change
impacts the other layers by taking example of a DRC usecase. Please have a look
considering the usecase and the impact it brings to other layers in the driver.

[1] https://lore.kernel.org/lkml/[email protected]/
> Short rationale:
> The venus driver has a history of supported platforms. There is
> already some kind of buffer management in place. Both developers and
> testers have spent their effort on finding issues there. Sending new
> driver means that we have to spend the same amount of efforts on this.
> Moreover, even from the porter point of view. You are creating new
> bindings for the new hardware. Which do not follow the
> venus-common.yaml. And they do not follow the defined bindings for the
> recent venus platforms. Which means that as a developer I have to care
> about two different ways to describe nearly the same hardware.>> Again qualcomm video team does not have a plan to support sm8550/x1e80100 on
>> venus as the changes are too interleaved to absorb in venus driver. And there is
>> significant interest in community to start validating video driver on sm8550 or
>> x1e80100.
>>
>> [1] https://lore.kernel.org/lkml/[email protected]/
>>
>> Regards,
>> Vikash
>
>
>

2023-12-20 08:16:08

by Krzysztof Kozlowski

[permalink] [raw]

Subject: Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver

On 20/12/2023 07:32, Vikash Garodia wrote:
>> From what I see from you bindings, the hardware is pretty close to what we see
>> in the latest venus generations. I asssme that there was a change in the vcodec
>> inteface to the firmware and other similar changes. Could you please point out,
>> which parts of Venus driver do no longer work or are not applicable for sm8550
>
> The motivation behind having a separate IRIS driver was discussed earlier in [1]
> In the same discussion, it was ellaborated on how the impact would be with
> change in the new firmware interface and other video layers in the driver. I can
> add this in cover letter in the next revision.
>
> We see some duplication of code and to handle the same, the series brings in a
> common code reusability between iris and venus. Aligning the common peices of
> venus and iris will be a work in progress, once we land the base driver for iris.
>
> Again qualcomm video team does not have a plan to support sm8550/x1e80100 on

If you want it to get merged, then create such plan, please.

> venus as the changes are too interleaved to absorb in venus driver. And there is
> significant interest in community to start validating video driver on sm8550 or
> x1e80100.

Community does not want duplicated drivers leading to maintenance costs.
Your approach to this is not how upstreaming process works.

Best regards,
Krzysztof

2023-12-20 08:23:54

[permalink] [raw]

Subject: Re: [PATCH v2 29/34] media: iris: implement power management

On 12/19/2023 5:54 PM, Konrad Dybcio wrote:
> On 18.12.2023 12:32, Dikshita Agarwal wrote:
>> From: Vikash Garodia <[email protected]>
>>
>> Hardware specific power sequence include programming specific
>> sequence of registers. Implements this sequence for iris3.
>>
>> Introduce video device suspend and resume using runtime PM.
>>
>> Also, implement iris3 specific calculation for clock and
>> bus bandwidth which depends on hardware configuration,
>> codec format, resolution and frame rate.
>>
>> Signed-off-by: Vikash Garodia <[email protected]>
>> Signed-off-by: Dikshita Agarwal <[email protected]>
>> ---
> [...]
>
>> @@ -97,6 +103,7 @@ static int iris_probe(struct platform_device *pdev)
> return dev_err_probe, drop __func__
Sure
>
> [...]
>
>> + ret = readl_relaxed_poll_timeout(base_addr + reg, val, ((val & mask) == exp_val),
>> + sleep_us, timeout_us);
>> + /*
>> + * Memory barrier to make sure value is read correctly from the
>> + * register.
>> + */
>> + rmb();
> just drop _relaxed and return directly
>
Will check and do the needful.
>> +
>> + return ret;
>> +}
> [...]
>
>> + ret = read_register_with_poll_timeout(core, AON_WRAPPER_MVP_NOC_RESET_ACK,
>> + 0x3, 0x3, 200, 2000);
> that looks like a lot of bits/bitfields that deserve #defining in this
> and some other functions
>
Will check this.
> Konrad

2023-12-20 08:27:39

[permalink] [raw]

Subject: Re: [PATCH v2 27/34] media: iris: implement vb2 ops for buf_queue and firmware response

On 12/19/2023 5:51 PM, Konrad Dybcio wrote:
> On 18.12.2023 12:32, Dikshita Agarwal wrote:
>> Implement vb2 ops for buf queue. Below are the different
>> buffer attributes:
>> BUF_ATTR_DEFERRED - buffer queued by client but not submitted
>> to firmware.
>> BUF_ATTR_READ_ONLY - processed buffer received from firmware
>> as read only. These buffers are held in firmware as reference
>> for future frame processing.
>> BUF_ATTR_PENDING_RELEASE - buffers requested to be released
>> from firmware.
>> BUF_ATTR_QUEUED - buffers submitted to firmware.
>> BUF_ATTR_DEQUEUED - buffers received from firmware.
>> BUF_ATTR_BUFFER_DONE - buffers sent back to vb2.
>>
>> Buffers are submitted and received via HFI_CMD_BUFFER.
>> Firmware associates below flags during buffer response:
>> HFI_BUF_FW_FLAG_RELEASE_DONE - buffer released in firmware.
>> HFI_BUF_FW_FLAG_READONLY - buffer used as reference in firmware.
>>
>> Input buffers dequeued from firmware are sent directly to vb2.
>>
>> Output buffers if read only, are sent to vb2 and also maintained
>> in read only list. If the same read only buffer is received form
>> client, HFI_BUF_HOST_FLAG_READONLY is attached to the buffer and
>> submitted to firmware. Once the buffer is received from firmware
>> as non read only, it is removed from read only list.
>>
>> Signed-off-by: Dikshita Agarwal <[email protected]>
>> ---
> [...]
>
>>
>> +enum iris_buffer_flags {
>> + BUF_FLAG_KEYFRAME = 0x00000008,
>> + BUF_FLAG_PFRAME = 0x00000010,
>> + BUF_FLAG_BFRAME = 0x00000020,
>> + BUF_FLAG_ERROR = 0x00000040,
> BIT(3), 4, 5, 6?
Sure, will update.
>
> Konrad
>

2023-12-20 08:30:17

[permalink] [raw]

Subject: Re: [PATCH v2 20/34] media: iris: add video hardware internal buffer count and size calculation

On 12/19/2023 5:36 PM, Bryan O'Donoghue wrote:
> On 18/12/2023 11:32, Dikshita Agarwal wrote:
>> drivers/media/platform/qcom/vcodec/iris/Makefile | 1 +
>> .../media/platform/qcom/vcodec/iris/iris_buffer.c | 48 ++
>> .../media/platform/qcom/vcodec/iris/iris_common.h | 1 +
>> .../media/platform/qcom/vcodec/iris/iris_core.h | 2 +
>> .../media/platform/qcom/vcodec/iris/iris_helpers.c | 13 +
>> .../media/platform/qcom/vcodec/iris/iris_helpers.h | 4 +
>> .../platform/qcom/vcodec/iris/iris_instance.h | 2 +
>> .../media/platform/qcom/vcodec/iris/iris_vdec.c | 1 +
>> .../media/platform/qcom/vcodec/iris/vpu_common.h | 8 +
>> .../media/platform/qcom/vcodec/iris/vpu_iris3.c | 6 +
>> .../platform/qcom/vcodec/iris/vpu_iris3_buffer.c | 201 +++++
>> .../platform/qcom/vcodec/iris/vpu_iris3_buffer.h | 845 +++++++++++++++++++++
>
> So, unless the code you are adding effects existing upstream venus, I think
> it would be a bit easier to review if you squashed down changes that
> pertain to Iris only.
>
> For example this patch seems to relate to Iris only, so why is it a
> progressive change within your series.
>
> Similar comment for "add vb2 streaming and buffer ops" and other "add
> feature x" patches in this series.
>
> If the change is contained to your own codebase, then progressive changes
> are more noise than content.
>
> Please try to squash down changes - to reduce the number of patches and the
> total LOC being proposed here.
>
As requested in V1, we separated out the iris driver code in functional
patches. Also, as commented in 1st patch, we are OK to have the common
helpers as separate series.

Patch 6 onward all the code is contained to iris driver only.
Could you please elaborate, what do you mean by squashing down the changes?
which patches you want us to squash?
> ---
> bod

2023-12-20 08:33:43

[permalink] [raw]

Subject: Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver

Hi Krzysztof,

On 12/20/2023 1:45 PM, Krzysztof Kozlowski wrote:
> On 20/12/2023 07:32, Vikash Garodia wrote:
>>> From what I see from you bindings, the hardware is pretty close to what we see
>>> in the latest venus generations. I asssme that there was a change in the vcodec
>>> inteface to the firmware and other similar changes. Could you please point out,
>>> which parts of Venus driver do no longer work or are not applicable for sm8550
>>
>> The motivation behind having a separate IRIS driver was discussed earlier in [1]
>> In the same discussion, it was ellaborated on how the impact would be with
>> change in the new firmware interface and other video layers in the driver. I can
>> add this in cover letter in the next revision.
>>
>> We see some duplication of code and to handle the same, the series brings in a
>> common code reusability between iris and venus. Aligning the common peices of
>> venus and iris will be a work in progress, once we land the base driver for iris.
>>
>> Again qualcomm video team does not have a plan to support sm8550/x1e80100 on
>
> If you want it to get merged, then create such plan, please.
>
>> venus as the changes are too interleaved to absorb in venus driver. And there is
>> significant interest in community to start validating video driver on sm8550 or
>> x1e80100.
>
> Community does not want duplicated drivers leading to maintenance costs.
> Your approach to this is not how upstreaming process works.
If you go over the pseudo code which i explained in [1], you can see that the
maintenance cost would be higher if try to impose the new interface in existing
driver. Video team, alongwith Stan, have considered the aspect of maintenance
before starting on this new driver.

[1] https://lore.kernel.org/lkml/[email protected]/

Regards,
Vikash

2023-12-20 08:38:04

[permalink] [raw]

Subject: Re: [PATCH v2 17/34] media: iris: implement vb2_ops queue setup

On 12/19/2023 5:26 PM, Konrad Dybcio wrote:
> On 18.12.2023 12:32, Dikshita Agarwal wrote:
>> Implement queue_setup vb2_ops.
>> Calculate the buffer count and buffer size as par video
>> hardware requirement and updates to client.
>> Also, allocate the video driver buffers for output and
>> capture plane.
>>
>> Signed-off-by: Dikshita Agarwal <[email protected]>
>> ---
> [...]
>
>> +static int input_min_count(struct iris_inst *inst)
>> +{
>> + return MIN_BUFFERS;
>> +}
> Why is this a function?
>
Some extra checks will be performed to calculate the input buffers for
encoder, hence it was made a function. May be I can directly use the macro
at this point and introduce the function later in the patch with encoder
patches.
>> +
>> +static int output_min_count(struct iris_inst *inst)
>> +{
>> + int output_min_count;
>> +
>> + switch (inst->codec) {
>> + case H264:
>> + case HEVC:
>> + output_min_count = 4;
>> + break;
>> + case VP9:
>> + output_min_count = 9;
>> + break;
>> + default:
>> + output_min_count = 4;
>> + break;
>> + }
>
> switch (inst->codec) {
> case VP9:
> return 9;
> case H264:
> case HEVC:
> default:
> return 4;
> }
>
Right, this is better, will make the change.
>> +
>> + return output_min_count;
>> +}
>> +
>> +int iris_get_buf_min_count(struct iris_inst *inst,
>> + enum iris_buffer_type buffer_type)
>> +{
>> + switch (buffer_type) {
>> + case BUF_INPUT:
>> + return input_min_count(inst);
>> + case BUF_OUTPUT:
>> + return output_min_count(inst);
>> + default:
>> + return 0;
>> + }
>> +}
>> +
>> +static u32 input_buffer_size(struct iris_inst *inst)
>> +{
>> + u32 base_res_mbs = NUM_MBS_4k;
>> + u32 frame_size, num_mbs;
>> + struct v4l2_format *f;
>> + u32 div_factor = 1;
>> + u32 codec;
>> +
>> + f = inst->fmt_src;
>> + codec = f->fmt.pix_mp.pixelformat;
>> +
>> + num_mbs = get_mbpf(inst);
>> + if (num_mbs > NUM_MBS_4k) {
>> + div_factor = 4;
>> + base_res_mbs = inst->cap[MBPF].value;
>> + } else {
>> + base_res_mbs = NUM_MBS_4k;
>> + if (codec == V4L2_PIX_FMT_VP9)
>> + div_factor = 1;
>> + else
>> + div_factor = 2;
>> + }
>> +
>> + frame_size = base_res_mbs * MB_IN_PIXEL * 3 / 2 / div_factor;
> that's a bit magic..
>
This is the formula to calculate video frame size, no magic here :)
>> +
>> + /* multiply by 10/8 (1.25) to get size for 10 bit case */
> misaligned
>
Oh! wasn't caught by check-patch.
Thanks for pointing this, Will fix.
>
> Konrad

2023-12-20 08:43:49

[permalink] [raw]

Subject: Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver

On Wed, 20 Dec 2023 at 10:14, Vikash Garodia <[email protected]> wrote:
>
> Hi,
>
> On 12/20/2023 1:07 PM, Dmitry Baryshkov wrote:
> > On Wed, 20 Dec 2023 at 08:32, Vikash Garodia <[email protected]> wrote:
> >>
> >> Hi Dmitry,
> >>
> >> On 12/19/2023 12:08 AM, Dmitry Baryshkov wrote:
> >>> On 18/12/2023 13:31, Dikshita Agarwal wrote:
> >>>> This patch series introduces support for Qualcomm new video acceleration
> >>>> hardware architecture, used for video stream decoding/encoding. This driver
> >>>> is based on new communication protocol between video hardware and application
> >>>> processor.
> >>>
> >>> This doesn't answer one important point, you have been asked for v1. What is the
> >>> actual change point between Venus and Iris? What has been changed so much that
> >>> it demands a separate driver. This is the main question for the cover letter,
> >>> which has not been answered so far.
> >>>
> >>> From what I see from you bindings, the hardware is pretty close to what we see
> >>> in the latest venus generations. I asssme that there was a change in the vcodec
> >>> inteface to the firmware and other similar changes. Could you please point out,
> >>> which parts of Venus driver do no longer work or are not applicable for sm8550
> >>
> >> The motivation behind having a separate IRIS driver was discussed earlier in [1]
> >> In the same discussion, it was ellaborated on how the impact would be with
> >> change in the new firmware interface and other video layers in the driver. I can
> >> add this in cover letter in the next revision.
> >
> > Ok. So the changes cover the HFI interface. Is that correct?
> Change wise, yes.
>
> >> We see some duplication of code and to handle the same, the series brings in a
> >> common code reusability between iris and venus. Aligning the common peices of
> >> venus and iris will be a work in progress, once we land the base driver for iris.
> >
> > This is not how it usually works. Especially not with the patches you
> > have posted.
> >
> > I have the following suggestion how this story can continue:
> > You can _start_ by reworking venus driver, separating the HFI /
> > firmware / etc interface to an internal interface in the driver. Then
> > implement Iris as a plug in for that interface. I might be mistaken
> > here, but I think this is the way how this can be beneficial for both
> > the video en/decoding on both old and new platforms.
>
> HFI/firmware interface is already confined to HFI layer in the existing venus
> driver. We explained in the previous discussion [1], on how the HFI change
> impacts the other layers by taking example of a DRC usecase. Please have a look
> considering the usecase and the impact it brings to other layers in the driver.

I have looked at it. And I still see huge change in the interface
side, but it doesn't tell me about the hardware changes.

Have you evaluated the other opportunity?

To have a common platform interface and firmware-specific backend?

You have already done a part of it, but from a different perspective.
You have tried to move common code out of the driver. Instead we are
asking you to do a different thing. Move non-common code within the
driver. Then add your code on top of that.

>
> [1] https://lore.kernel.org/lkml/[email protected]/
> > Short rationale:
> > The venus driver has a history of supported platforms. There is
> > already some kind of buffer management in place. Both developers and
> > testers have spent their effort on finding issues there. Sending new
> > driver means that we have to spend the same amount of efforts on this.
> > Moreover, even from the porter point of view. You are creating new
> > bindings for the new hardware. Which do not follow the
> > venus-common.yaml. And they do not follow the defined bindings for the
> > recent venus platforms. Which means that as a developer I have to care
> > about two different ways to describe nearly the same hardware.>> Again qualcomm video team does not have a plan to support sm8550/x1e80100 on
> >> venus as the changes are too interleaved to absorb in venus driver. And there is
> >> significant interest in community to start validating video driver on sm8550 or
> >> x1e80100.
> >>
> >> [1] https://lore.kernel.org/lkml/[email protected]/
> >>
> >> Regards,
> >> Vikash
> >
> >
> >

--
With best wishes
Dmitry

2023-12-20 08:54:07

[permalink] [raw]

Subject: Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver

On 12/20/2023 2:09 PM, Dmitry Baryshkov wrote:
> On Wed, 20 Dec 2023 at 10:14, Vikash Garodia <[email protected]> wrote:
>>
>> Hi,
>>
>> On 12/20/2023 1:07 PM, Dmitry Baryshkov wrote:
>>> On Wed, 20 Dec 2023 at 08:32, Vikash Garodia <[email protected]> wrote:
>>>>
>>>> Hi Dmitry,
>>>>
>>>> On 12/19/2023 12:08 AM, Dmitry Baryshkov wrote:
>>>>> On 18/12/2023 13:31, Dikshita Agarwal wrote:
>>>>>> This patch series introduces support for Qualcomm new video acceleration
>>>>>> hardware architecture, used for video stream decoding/encoding. This driver
>>>>>> is based on new communication protocol between video hardware and application
>>>>>> processor.
>>>>>
>>>>> This doesn't answer one important point, you have been asked for v1. What is the
>>>>> actual change point between Venus and Iris? What has been changed so much that
>>>>> it demands a separate driver. This is the main question for the cover letter,
>>>>> which has not been answered so far.
>>>>>
>>>>> From what I see from you bindings, the hardware is pretty close to what we see
>>>>> in the latest venus generations. I asssme that there was a change in the vcodec
>>>>> inteface to the firmware and other similar changes. Could you please point out,
>>>>> which parts of Venus driver do no longer work or are not applicable for sm8550
>>>>
>>>> The motivation behind having a separate IRIS driver was discussed earlier in [1]
>>>> In the same discussion, it was ellaborated on how the impact would be with
>>>> change in the new firmware interface and other video layers in the driver. I can
>>>> add this in cover letter in the next revision.
>>>
>>> Ok. So the changes cover the HFI interface. Is that correct?
>> Change wise, yes.
>>
>>>> We see some duplication of code and to handle the same, the series brings in a
>>>> common code reusability between iris and venus. Aligning the common peices of
>>>> venus and iris will be a work in progress, once we land the base driver for iris.
>>>
>>> This is not how it usually works. Especially not with the patches you
>>> have posted.
>>>
>>> I have the following suggestion how this story can continue:
>>> You can _start_ by reworking venus driver, separating the HFI /
>>> firmware / etc interface to an internal interface in the driver. Then
>>> implement Iris as a plug in for that interface. I might be mistaken
>>> here, but I think this is the way how this can be beneficial for both
>>> the video en/decoding on both old and new platforms.
>>
>> HFI/firmware interface is already confined to HFI layer in the existing venus
>> driver. We explained in the previous discussion [1], on how the HFI change
>> impacts the other layers by taking example of a DRC usecase. Please have a look
>> considering the usecase and the impact it brings to other layers in the driver.
>
> I have looked at it. And I still see huge change in the interface
> side, but it doesn't tell me about the hardware changes.

I hope you noticed how the common layers like decoder, response, state layers
are impacted in handling one of usecase. Now add that to all the different
scenarios like seek, drain, DRC during seek, DRC during drain, etc.

> Have you evaluated the other opportunity?
>
> To have a common platform interface and firmware-specific backend?
>
> You have already done a part of it, but from a different perspective.
> You have tried to move common code out of the driver. Instead we are
> asking you to do a different thing. Move non-common code within the
> driver. Then add your code on top of that.

For common platform - yes, we are bringing in common stuff like PIL.
Other than that, abstraction to firmware interface is not that confined to one
layer. It spreads over decoder/encoder/common layers. Now when majority of the
layers/code is different, we planned to make it in parallel to venus and have a
common layer having common things to both iris and venus.

>>
>> [1] https://lore.kernel.org/lkml/[email protected]/
>>> Short rationale:
>>> The venus driver has a history of supported platforms. There is
>>> already some kind of buffer management in place. Both developers and
>>> testers have spent their effort on finding issues there. Sending new
>>> driver means that we have to spend the same amount of efforts on this.
>>> Moreover, even from the porter point of view. You are creating new
>>> bindings for the new hardware. Which do not follow the
>>> venus-common.yaml. And they do not follow the defined bindings for the
>>> recent venus platforms. Which means that as a developer I have to care
>>> about two different ways to describe nearly the same hardware.>> Again qualcomm video team does not have a plan to support sm8550/x1e80100 on
>>>> venus as the changes are too interleaved to absorb in venus driver. And there is
>>>> significant interest in community to start validating video driver on sm8550 or
>>>> x1e80100.
>>>>
>>>> [1] https://lore.kernel.org/lkml/[email protected]/
>>>>
>>>> Regards,
>>>> Vikash
>>>
>>>
>>>
>
>
>

2023-12-20 09:53:27

[permalink] [raw]

Subject: Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver

On Wed, 20 Dec 2023 at 10:53, Vikash Garodia <[email protected]> wrote:
>
> On 12/20/2023 2:09 PM, Dmitry Baryshkov wrote:
> > On Wed, 20 Dec 2023 at 10:14, Vikash Garodia <[email protected]> wrote:
> >>
> >> Hi,
> >>
> >> On 12/20/2023 1:07 PM, Dmitry Baryshkov wrote:
> >>> On Wed, 20 Dec 2023 at 08:32, Vikash Garodia <[email protected]> wrote:
> >>>>
> >>>> Hi Dmitry,
> >>>>
> >>>> On 12/19/2023 12:08 AM, Dmitry Baryshkov wrote:
> >>>>> On 18/12/2023 13:31, Dikshita Agarwal wrote:
> >>>>>> This patch series introduces support for Qualcomm new video acceleration
> >>>>>> hardware architecture, used for video stream decoding/encoding. This driver
> >>>>>> is based on new communication protocol between video hardware and application
> >>>>>> processor.
> >>>>>
> >>>>> This doesn't answer one important point, you have been asked for v1. What is the
> >>>>> actual change point between Venus and Iris? What has been changed so much that
> >>>>> it demands a separate driver. This is the main question for the cover letter,
> >>>>> which has not been answered so far.
> >>>>>
> >>>>> From what I see from you bindings, the hardware is pretty close to what we see
> >>>>> in the latest venus generations. I asssme that there was a change in the vcodec
> >>>>> inteface to the firmware and other similar changes. Could you please point out,
> >>>>> which parts of Venus driver do no longer work or are not applicable for sm8550
> >>>>
> >>>> The motivation behind having a separate IRIS driver was discussed earlier in [1]
> >>>> In the same discussion, it was ellaborated on how the impact would be with
> >>>> change in the new firmware interface and other video layers in the driver. I can
> >>>> add this in cover letter in the next revision.
> >>>
> >>> Ok. So the changes cover the HFI interface. Is that correct?
> >> Change wise, yes.
> >>
> >>>> We see some duplication of code and to handle the same, the series brings in a
> >>>> common code reusability between iris and venus. Aligning the common peices of
> >>>> venus and iris will be a work in progress, once we land the base driver for iris.
> >>>
> >>> This is not how it usually works. Especially not with the patches you
> >>> have posted.
> >>>
> >>> I have the following suggestion how this story can continue:
> >>> You can _start_ by reworking venus driver, separating the HFI /
> >>> firmware / etc interface to an internal interface in the driver. Then
> >>> implement Iris as a plug in for that interface. I might be mistaken
> >>> here, but I think this is the way how this can be beneficial for both
> >>> the video en/decoding on both old and new platforms.
> >>
> >> HFI/firmware interface is already confined to HFI layer in the existing venus
> >> driver. We explained in the previous discussion [1], on how the HFI change
> >> impacts the other layers by taking example of a DRC usecase. Please have a look
> >> considering the usecase and the impact it brings to other layers in the driver.
> >
> > I have looked at it. And I still see huge change in the interface
> > side, but it doesn't tell me about the hardware changes.
>
> I hope you noticed how the common layers like decoder, response, state layers
> are impacted in handling one of usecase. Now add that to all the different
> scenarios like seek, drain, DRC during seek, DRC during drain, etc.

Yes, for sure.

>
> > Have you evaluated the other opportunity?
> >
> > To have a common platform interface and firmware-specific backend?
> >
> > You have already done a part of it, but from a different perspective.
> > You have tried to move common code out of the driver. Instead we are
> > asking you to do a different thing. Move non-common code within the
> > driver. Then add your code on top of that.
>
> For common platform - yes, we are bringing in common stuff like PIL.
> Other than that, abstraction to firmware interface is not that confined to one
> layer. It spreads over decoder/encoder/common layers. Now when majority of the
> layers/code is different, we planned to make it in parallel to venus and have a
> common layer having common things to both iris and venus.

My suggestion still holds. Start with this common platform code.
Rather than arguing back and forth, could you please perform an
experiment on the current venus driver and move firmware interface to
subdirs, leaving just the platform / PIL / formats / etc in place?
This will at least allow us to determine whether it is a feasible
concept or not.

>
> >>
> >> [1] https://lore.kernel.org/lkml/[email protected]/
> >>> Short rationale:
> >>> The venus driver has a history of supported platforms. There is
> >>> already some kind of buffer management in place. Both developers and
> >>> testers have spent their effort on finding issues there. Sending new
> >>> driver means that we have to spend the same amount of efforts on this.
> >>> Moreover, even from the porter point of view. You are creating new
> >>> bindings for the new hardware. Which do not follow the
> >>> venus-common.yaml. And they do not follow the defined bindings for the
> >>> recent venus platforms. Which means that as a developer I have to care
> >>> about two different ways to describe nearly the same hardware.>> Again qualcomm video team does not have a plan to support sm8550/x1e80100 on
> >>>> venus as the changes are too interleaved to absorb in venus driver. And there is
> >>>> significant interest in community to start validating video driver on sm8550 or
> >>>> x1e80100.
> >>>>
> >>>> [1] https://lore.kernel.org/lkml/[email protected]/
> >>>>
> >>>> Regards,
> >>>> Vikash
> >>>
> >>>
> >>>
> >
> >
> >

--
With best wishes
Dmitry

2023-12-20 17:11:12

by Abhinav Kumar

[permalink] [raw]

Subject: Re: [PATCH v2 01/34] media: introduce common helpers for video firmware handling

Hi Dmitry

On 12/20/2023 12:12 AM, Dmitry Baryshkov wrote:
> On Wed, 20 Dec 2023 at 10:01, Dikshita Agarwal
> <[email protected]> wrote:
>>
>>
>>
>> On 12/18/2023 11:54 PM, Dmitry Baryshkov wrote:
>>> On 18/12/2023 13:31, Dikshita Agarwal wrote:
>>>> Re-organize the video driver code by introducing a new folder
>>>> 'vcodec' and placing 'venus' driver code inside that.
>>>>
>>>> Introduce common helpers for trustzone based firmware
>>>> load/unload etc. which are placed in common folder
>>>> i.e. 'vcodec'.
>>>> Use these helpers in 'venus' driver. These helpers will be
>>>> used by 'iris' driver as well which is introduced later
>>>> in this patch series.
>>>
>>> But why do you need to move the venus driver to subdir?
>>
>> Currently venus driver is present in drivers/media/platform/qcom folder
>> which also has camss folder. We introduced vcodec to keep common code and
>> moved venus inside that, to indicate that the common code is for vcodec
>> drivers i.e venus and iris. Keeping this in qcom folder would mean, common
>> code will be used for camss only which is not the case here.
>
> you can have .../platform/qcom/camss, .../platform/qcom/vcodec-common,
> .../platform/qcom/venus and .../platform/qcom/iris.
>
> If you were to use build helpers in a proper kernel module, this would
> be more obvious.
>

Although your suggestion is good in terms of avoiding moving venus, I
think the location of venus was wrong to begin with. There should have
always been a vcodec (or similar) folder for venus/iris as that will
establish the boundaries of camss and video sub-system in a cleaner way

I like the mediatek separation that way as it makes the boundaries clear:

drivers/media/platform/mediatek$ ls
jpeg Kconfig Makefile mdp mdp3 vcodec vpu

So I think that this re-org of venus into a vcodec had to happen at some
point. Its just that it aligned with iris addition.

>>>
>>>>
>>>> Signed-off-by: Dikshita Agarwal <[email protected]>
>>>> ---
>>>> drivers/media/platform/qcom/Kconfig | 2 +-
>>>> drivers/media/platform/qcom/Makefile | 2 +-
>>>> drivers/media/platform/qcom/vcodec/firmware.c | 147 +++++++++
>>>> drivers/media/platform/qcom/vcodec/firmware.h | 21 ++
>>>> .../media/platform/qcom/{ => vcodec}/venus/Kconfig | 0
>>>> .../platform/qcom/{ => vcodec}/venus/Makefile | 4 +-
>>>> .../media/platform/qcom/{ => vcodec}/venus/core.c | 102 +++++-
>>>> .../media/platform/qcom/{ => vcodec}/venus/core.h | 0
>>>> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.c | 0
>>>> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.h | 0
>>>> .../platform/qcom/vcodec/venus/firmware_no_tz.c | 194 +++++++++++
>>>> .../platform/qcom/vcodec/venus/firmware_no_tz.h | 19 ++
>>>> .../platform/qcom/{ => vcodec}/venus/helpers.c | 0
>>>> .../platform/qcom/{ => vcodec}/venus/helpers.h | 0
>>>> .../media/platform/qcom/{ => vcodec}/venus/hfi.c | 0
>>>> .../media/platform/qcom/{ => vcodec}/venus/hfi.h | 0
>>>> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.c | 0
>>>> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.h | 0
>>>> .../platform/qcom/{ => vcodec}/venus/hfi_helper.h | 0
>>>> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.c | 0
>>>> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.h | 0
>>>> .../platform/qcom/{ => vcodec}/venus/hfi_parser.c | 0
>>>> .../platform/qcom/{ => vcodec}/venus/hfi_parser.h | 0
>>>> .../qcom/{ => vcodec}/venus/hfi_plat_bufs.h | 0
>>>> .../qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c | 0
>>>> .../qcom/{ => vcodec}/venus/hfi_platform.c | 0
>>>> .../qcom/{ => vcodec}/venus/hfi_platform.h | 0
>>>> .../qcom/{ => vcodec}/venus/hfi_platform_v4.c | 0
>>>> .../qcom/{ => vcodec}/venus/hfi_platform_v6.c | 0
>>>> .../platform/qcom/{ => vcodec}/venus/hfi_venus.c | 21 +-
>>>> .../platform/qcom/{ => vcodec}/venus/hfi_venus.h | 0
>>>> .../qcom/{ => vcodec}/venus/hfi_venus_io.h | 0
>>>> .../platform/qcom/{ => vcodec}/venus/pm_helpers.c | 0
>>>> .../platform/qcom/{ => vcodec}/venus/pm_helpers.h | 0
>>>> .../media/platform/qcom/{ => vcodec}/venus/vdec.c | 0
>>>> .../media/platform/qcom/{ => vcodec}/venus/vdec.h | 0
>>>> .../platform/qcom/{ => vcodec}/venus/vdec_ctrls.c | 0
>>>> .../media/platform/qcom/{ => vcodec}/venus/venc.c | 0
>>>> .../media/platform/qcom/{ => vcodec}/venus/venc.h | 0
>>>> .../platform/qcom/{ => vcodec}/venus/venc_ctrls.c | 0
>>>> drivers/media/platform/qcom/venus/firmware.c | 363
>>>> ---------------------
>>>> drivers/media/platform/qcom/venus/firmware.h | 26 --
>>>> 42 files changed, 492 insertions(+), 409 deletions(-)
>>>> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.c
>>>> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.h
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/Kconfig (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/Makefile (83%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.c (91%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.h (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.c (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.h (100%)
>>>> create mode 100644
>>>> drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
>>>> create mode 100644
>>>> drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.c (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.h (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.c (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.h (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.c (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.h (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_helper.h (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.c (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.h (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.c (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.h (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs.h
>>>> (100%)
>>>> rename drivers/media/platform/qcom/{ =>
>>>> vcodec}/venus/hfi_plat_bufs_v6.c (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.c
>>>> (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.h
>>>> (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v4.c
>>>> (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v6.c
>>>> (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.c (99%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.h (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus_io.h
>>>> (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.c (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.h (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.c (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.h (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec_ctrls.c (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.c (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.h (100%)
>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc_ctrls.c (100%)
>>>> delete mode 100644 drivers/media/platform/qcom/venus/firmware.c
>>>> delete mode 100644 drivers/media/platform/qcom/venus/firmware.h
>>>>
>>>> diff --git a/drivers/media/platform/qcom/Kconfig
>>>> b/drivers/media/platform/qcom/Kconfig
>>>> index cc5799b..e94142f 100644
>>>> --- a/drivers/media/platform/qcom/Kconfig
>>>> +++ b/drivers/media/platform/qcom/Kconfig
>>>> @@ -3,4 +3,4 @@
>>>> comment "Qualcomm media platform drivers"
>>>> source "drivers/media/platform/qcom/camss/Kconfig"
>>>> -source "drivers/media/platform/qcom/venus/Kconfig"
>>>> +source "drivers/media/platform/qcom/vcodec/venus/Kconfig"
>>>> diff --git a/drivers/media/platform/qcom/Makefile
>>>> b/drivers/media/platform/qcom/Makefile
>>>> index 4f055c3..3d2d82b 100644
>>>> --- a/drivers/media/platform/qcom/Makefile
>>>> +++ b/drivers/media/platform/qcom/Makefile
>>>> @@ -1,3 +1,3 @@
>>>> # SPDX-License-Identifier: GPL-2.0-only
>>>> obj-y += camss/
>>>> -obj-y += venus/
>>>> +obj-y += vcodec/venus/
>>>> diff --git a/drivers/media/platform/qcom/vcodec/firmware.c
>>>> b/drivers/media/platform/qcom/vcodec/firmware.c
>>>> new file mode 100644
>>>> index 0000000..dbc220a
>>>> --- /dev/null
>>>> +++ b/drivers/media/platform/qcom/vcodec/firmware.c
>>>> @@ -0,0 +1,147 @@
>>>> +// SPDX-License-Identifier: GPL-2.0-only
>>>> +/*
>>>> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
>>>> reserved.
>>>> + */
>>>> +
>>>> +#include <linux/device.h>
>>>> +#include <linux/dma-mapping.h>
>>>> +#include <linux/firmware.h>
>>>> +#include <linux/kernel.h>
>>>> +#include <linux/iommu.h>
>>>> +#include <linux/of_device.h>
>>>> +#include <linux/firmware/qcom/qcom_scm.h>
>>>> +#include <linux/of_reserved_mem.h>
>>>> +#include <linux/platform_device.h>
>>>> +#include <linux/soc/qcom/mdt_loader.h>
>>>> +
>>>> +#include "firmware.h"
>>>> +
>>>> +bool use_tz(struct device *core_dev)
>>>
>>> All these functions must get some sane prefix. Otherwise a generic 'use_tz'
>>> function is too polluting for the global namespace.
>>>
>> I understand, will check and do the needful.
>>>> +{
>>>> + struct device_node *np;
>>>> +
>>>> + np = of_get_child_by_name(core_dev->of_node, "video-firmware");
>>>> + if (!np)
>>>> + return true;
>>>> +
>>>> + return false;
>>>> +}
>>>> +
>>>> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
>>>> + u32 cp_nonpixel_size, u32 pas_id)
>>>> +{
>>>> + int ret;
>>>> + /*
>>>> + * Clues for porting using downstream data:
>>>> + * cp_start = 0
>>>> + * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and not size!
>>>> + * This works, as the non-secure context bank is placed
>>>> + * contiguously right after the Content Protection region.
>>>> + *
>>>> + * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0]
>>>> + * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1]
>>>> + */
>>>> + ret = qcom_scm_mem_protect_video_var(cp_start,
>>>> + cp_size,
>>>> + cp_nonpixel_start,
>>>> + cp_nonpixel_size);
>>>> + if (ret)
>>>> + qcom_scm_pas_shutdown(pas_id);
>>>> +
>>>> + return ret;
>>>> +}
>>>> +
>>>> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
>>>> + size_t *mem_size, u32 pas_id, bool use_tz)
>>>> +{
>>>> + const struct firmware *firmware = NULL;
>>>> + struct reserved_mem *rmem;
>>>> + struct device_node *node;
>>>> + void *mem_virt = NULL;
>>>> + ssize_t fw_size = 0;
>>>> + int ret;
>>>> +
>>>> + if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
>>>
>>> Why? Can you just depend on it?
>>>
>> Sure, Will check this and get back.
>>>> + (use_tz && !qcom_scm_is_available()))
>>>> + return -EPROBE_DEFER;
>>>> +
>>>> + if (!fw_name || !(*fw_name))
>>>> + return -EINVAL;
>>>> +
>>>> + *mem_phys = 0;
>>>> + *mem_size = 0;
>>>> +
>>>> + node = of_parse_phandle(dev->of_node, "memory-region", 0);
>>>> + if (!node) {
>>>> + dev_err(dev, "no memory-region specified\n");
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + rmem = of_reserved_mem_lookup(node);
>>>> + of_node_put(node);
>>>> + if (!rmem) {
>>>> + dev_err(dev, "failed to lookup reserved memory-region\n");
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + ret = request_firmware(&firmware, fw_name, dev);
>>>> + if (ret) {
>>>> + dev_err(dev, "%s: failed to request fw \"%s\", error %d\n",
>>>> + __func__, fw_name, ret);
>>>> + return ret;
>>>> + }
>>>> +
>>>> + fw_size = qcom_mdt_get_size(firmware);
>>>> + if (fw_size < 0) {
>>>> + ret = fw_size;
>>>> + dev_err(dev, "%s: out of bound fw image fw size: %ld\n",
>>>> + __func__, fw_size);
>>>> + goto err_release_fw;
>>>> + }
>>>> +
>>>> + *mem_phys = rmem->base;
>>>> + *mem_size = rmem->size;
>>>> +
>>>> + if (*mem_size < fw_size) {
>>>> + ret = -EINVAL;
>>>> + goto err_release_fw;
>>>> + }
>>>> +
>>>> + mem_virt = memremap(*mem_phys, *mem_size, MEMREMAP_WC);
>>>> + if (!mem_virt) {
>>>> + dev_err(dev, "unable to remap fw memory region %pa size %#zx\n",
>>>> + mem_phys, *mem_size);
>>>> + goto err_release_fw;
>>>> + }
>>>> +
>>>> + if (use_tz)
>>>> + ret = qcom_mdt_load(dev, firmware, fw_name, pas_id, mem_virt,
>>>> + *mem_phys, *mem_size, NULL);
>>>> + else
>>>> + ret = qcom_mdt_load_no_init(dev, firmware, fw_name, pas_id,
>>>> mem_virt,
>>>> + *mem_phys, *mem_size, NULL);
>>>> + if (ret) {
>>>> + dev_err(dev, "%s: error %d loading fw \"%s\"\n",
>>>> + __func__, ret, fw_name);
>>>> + }
>>>> +
>>>> + memunmap(mem_virt);
>>>> +err_release_fw:
>>>> + release_firmware(firmware);
>>>> + return ret;
>>>> +}
>>>> +
>>>> +int auth_reset_fw(u32 pas_id)
>>>> +{
>>>> + return qcom_scm_pas_auth_and_reset(pas_id);
>>>> +}
>>>> +
>>>> +void unload_fw(u32 pas_id)
>>>> +{
>>>> + qcom_scm_pas_shutdown(pas_id);
>>>> +}
>>>> +
>>>> +int set_hw_state(bool resume)
>>>> +{
>>>> + return qcom_scm_set_remote_state(resume, 0);
>>>> +}
>>>> diff --git a/drivers/media/platform/qcom/vcodec/firmware.h
>>>> b/drivers/media/platform/qcom/vcodec/firmware.h
>>>> new file mode 100644
>>>> index 0000000..7d410a8
>>>> --- /dev/null
>>>> +++ b/drivers/media/platform/qcom/vcodec/firmware.h
>>>> @@ -0,0 +1,21 @@
>>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>>> +/*
>>>> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
>>>> reserved.
>>>> + */
>>>> +
>>>> +#ifndef _FIRMWARE_H_
>>>> +#define _FIRMWARE_H_
>>>> +
>>>> +#include <linux/device.h>
>>>> +#include <linux/types.h>
>>>> +
>>>> +bool use_tz(struct device *core_dev);
>>>> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
>>>> + size_t *mem_size, u32 pas_id, bool use_tz);
>>>> +int auth_reset_fw(u32 pas_id);
>>>> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
>>>> + u32 cp_nonpixel_size, u32 pas_id);
>>>> +void unload_fw(u32 pas_id);
>>>> +int set_hw_state(bool resume);
>>>> +
>>>> +#endif
>>>> diff --git a/drivers/media/platform/qcom/venus/Kconfig
>>>> b/drivers/media/platform/qcom/vcodec/venus/Kconfig
>>>> similarity index 100%
>>>> rename from drivers/media/platform/qcom/venus/Kconfig
>>>> rename to drivers/media/platform/qcom/vcodec/venus/Kconfig
>>>> diff --git a/drivers/media/platform/qcom/venus/Makefile
>>>> b/drivers/media/platform/qcom/vcodec/venus/Makefile
>>>> similarity index 83%
>>>> rename from drivers/media/platform/qcom/venus/Makefile
>>>> rename to drivers/media/platform/qcom/vcodec/venus/Makefile
>>>> index 91ee6be..f6f3a88 100644
>>>> --- a/drivers/media/platform/qcom/venus/Makefile
>>>> +++ b/drivers/media/platform/qcom/vcodec/venus/Makefile
>>>> @@ -1,7 +1,9 @@
>>>> # SPDX-License-Identifier: GPL-2.0
>>>> # Makefile for Qualcomm Venus driver
>>>> -venus-core-objs += core.o helpers.o firmware.o \
>>>> +venus-core-objs += ../firmware.o
>>>> +
>>>> +venus-core-objs += core.o helpers.o firmware_no_tz.o \
>>>> hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \
>>>> hfi_parser.o pm_helpers.o dbgfs.o \
>>>> hfi_platform.o hfi_platform_v4.o \
>>>> diff --git a/drivers/media/platform/qcom/venus/core.c
>>>> b/drivers/media/platform/qcom/vcodec/venus/core.c
>>>> similarity index 91%
>>>> rename from drivers/media/platform/qcom/venus/core.c
>>>> rename to drivers/media/platform/qcom/vcodec/venus/core.c
>>>> index 9cffe97..56d9a53 100644
>>>> --- a/drivers/media/platform/qcom/venus/core.c
>>>> +++ b/drivers/media/platform/qcom/vcodec/venus/core.c
>>>> @@ -22,7 +22,8 @@
>>>> #include <media/v4l2-ioctl.h>
>>>> #include "core.h"
>>>> -#include "firmware.h"
>>>> +#include "../firmware.h"
>>>> +#include "firmware_no_tz.h"
>>>> #include "pm_helpers.h"
>>>> #include "hfi_venus_io.h"
>>>> @@ -86,6 +87,8 @@ static void venus_sys_error_handler(struct
>>>> work_struct *work)
>>>> struct venus_core *core =
>>>> container_of(work, struct venus_core, work.work);
>>>> int ret, i, max_attempts = RPM_WAIT_FOR_IDLE_MAX_ATTEMPTS;
>>>> + const struct venus_resources *res = core->res;
>>>> + const char *fwpath = NULL;
>>>> const char *err_msg = "";
>>>> bool failed = false;
>>>> @@ -107,7 +110,10 @@ static void venus_sys_error_handler(struct
>>>> work_struct *work)
>>>> mutex_lock(&core->lock);
>>>> - venus_shutdown(core);
>>>> + if (core->use_tz)
>>>> + unload_fw(VENUS_PAS_ID);
>>>> + else
>>>> + unload_fw_no_tz(core);
>>>
>>> This is more than introducing helpers.
>>>
>> The new helpers are written to make the code generic for video drivers.
>> which requires changes in the calling function also.
>>>> venus_coredump(core);
>>>> @@ -127,12 +133,39 @@ static void venus_sys_error_handler(struct
>>>> work_struct *work)
>>>> failed = true;
>>>> }
>>>> - ret = venus_boot(core);
>>>> + ret = of_property_read_string_index(core->dev->of_node,
>>>> "firmware-name", 0,
>>>> + &fwpath);
>>>> + if (ret)
>>>> + fwpath = core->res->fwname;
>>>> +
>>>> + ret = load_fw(core->dev, fwpath, &core->fw.mem_phys,
>>>> &core->fw.mem_size,
>>>> + VENUS_PAS_ID, core->use_tz);
>>>
>>> So, we had a nice local 'venus_boot'. Instead we now have a pile of code
>>> with non-generic prefixes, etc. If you are introducing helpers, please
>>> refrain from inlining of calling functions, etc. Just move the code to your
>>> helpers.
>>>
>> As mentioned in above comment, the common helpers are written to make the
>> code generic. I Will try to make it more clear, working on the same.
>
> First, you move the code, then you make it generic. Or vice versa.
> First you split the code, then you move it. Don't do both in the same
> patch.
>
>>> NAK for the rest of the patch.
>
>

2023-12-20 18:56:04

by Abhinav Kumar

[permalink] [raw]

Subject: Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver

Hi Dmitry and Krzysztof

On 12/20/2023 1:52 AM, Dmitry Baryshkov wrote:
> On Wed, 20 Dec 2023 at 10:53, Vikash Garodia <[email protected]> wrote:
>>
>> On 12/20/2023 2:09 PM, Dmitry Baryshkov wrote:
>>> On Wed, 20 Dec 2023 at 10:14, Vikash Garodia <[email protected]> wrote:
>>>>
>>>> Hi,
>>>>
>>>> On 12/20/2023 1:07 PM, Dmitry Baryshkov wrote:
>>>>> On Wed, 20 Dec 2023 at 08:32, Vikash Garodia <[email protected]> wrote:
>>>>>>
>>>>>> Hi Dmitry,
>>>>>>
>>>>>> On 12/19/2023 12:08 AM, Dmitry Baryshkov wrote:
>>>>>>> On 18/12/2023 13:31, Dikshita Agarwal wrote:
>>>>>>>> This patch series introduces support for Qualcomm new video acceleration
>>>>>>>> hardware architecture, used for video stream decoding/encoding. This driver
>>>>>>>> is based on new communication protocol between video hardware and application
>>>>>>>> processor.
>>>>>>>
>>>>>>> This doesn't answer one important point, you have been asked for v1. What is the
>>>>>>> actual change point between Venus and Iris? What has been changed so much that
>>>>>>> it demands a separate driver. This is the main question for the cover letter,
>>>>>>> which has not been answered so far.
>>>>>>>
>>>>>>> From what I see from you bindings, the hardware is pretty close to what we see
>>>>>>> in the latest venus generations. I asssme that there was a change in the vcodec
>>>>>>> inteface to the firmware and other similar changes. Could you please point out,
>>>>>>> which parts of Venus driver do no longer work or are not applicable for sm8550
>>>>>>
>>>>>> The motivation behind having a separate IRIS driver was discussed earlier in [1]
>>>>>> In the same discussion, it was ellaborated on how the impact would be with
>>>>>> change in the new firmware interface and other video layers in the driver. I can
>>>>>> add this in cover letter in the next revision.
>>>>>
>>>>> Ok. So the changes cover the HFI interface. Is that correct?
>>>> Change wise, yes.
>>>>
>>>>>> We see some duplication of code and to handle the same, the series brings in a
>>>>>> common code reusability between iris and venus. Aligning the common peices of
>>>>>> venus and iris will be a work in progress, once we land the base driver for iris.
>>>>>
>>>>> This is not how it usually works. Especially not with the patches you
>>>>> have posted.
>>>>>
>>>>> I have the following suggestion how this story can continue:
>>>>> You can _start_ by reworking venus driver, separating the HFI /
>>>>> firmware / etc interface to an internal interface in the driver. Then
>>>>> implement Iris as a plug in for that interface. I might be mistaken
>>>>> here, but I think this is the way how this can be beneficial for both
>>>>> the video en/decoding on both old and new platforms.
>>>>
>>>> HFI/firmware interface is already confined to HFI layer in the existing venus
>>>> driver. We explained in the previous discussion [1], on how the HFI change
>>>> impacts the other layers by taking example of a DRC usecase. Please have a look
>>>> considering the usecase and the impact it brings to other layers in the driver.
>>>
>>> I have looked at it. And I still see huge change in the interface
>>> side, but it doesn't tell me about the hardware changes.
>>
>> I hope you noticed how the common layers like decoder, response, state layers
>> are impacted in handling one of usecase. Now add that to all the different
>> scenarios like seek, drain, DRC during seek, DRC during drain, etc.
>
> Yes, for sure.
>
>>
>>> Have you evaluated the other opportunity?
>>>
>>> To have a common platform interface and firmware-specific backend?
>>>
>>> You have already done a part of it, but from a different perspective.
>>> You have tried to move common code out of the driver. Instead we are
>>> asking you to do a different thing. Move non-common code within the
>>> driver. Then add your code on top of that.
>>
>> For common platform - yes, we are bringing in common stuff like PIL.
>> Other than that, abstraction to firmware interface is not that confined to one
>> layer. It spreads over decoder/encoder/common layers. Now when majority of the
>> layers/code is different, we planned to make it in parallel to venus and have a
>> common layer having common things to both iris and venus.
>
> My suggestion still holds. Start with this common platform code.
> Rather than arguing back and forth, could you please perform an
> experiment on the current venus driver and move firmware interface to
> subdirs, leaving just the platform / PIL / formats / etc in place?
> This will at least allow us to determine whether it is a feasible
> concept or not.
>

Your comments are valid. The platform driver piece and some other pieces
still are common between venus and iris despite this initial effort of
moving common pieces out. I have also seen this from whatever I have
reviewed.

Video team also acknowledges this part and internally I think they did
evaluate this option and their feedback was, the more and more they
changed, they were touching pretty much every file of venus.

The missing piece i think in all this discussion is that in addition to
the forward channel, the reverse channel of HFI, based on which the rest
of the video state machine changes should also be considered.

So even with respect to the code layout, its not just the forward
communication but the backwards communication of fw--->hfi--->codec is
becoming a challenge as the venus layers seem to only work with the hfi
of venus.

For adding support for the new HFI events/communication, it was getting
harder to extend venus.

What I certainly acknowledge is now with iris whether this can be dealt
with better for future chipsets to avoid another re-write for another
HFI as that will be too much of maintenance cost.

I will let the video team comment on this part.

Thanks

Abhinav

>>
>>>>
>>>> [1] https://lore.kernel.org/lkml/[email protected]/
>>>>> Short rationale:
>>>>> The venus driver has a history of supported platforms. There is
>>>>> already some kind of buffer management in place. Both developers and
>>>>> testers have spent their effort on finding issues there. Sending new
>>>>> driver means that we have to spend the same amount of efforts on this.
>>>>> Moreover, even from the porter point of view. You are creating new
>>>>> bindings for the new hardware. Which do not follow the
>>>>> venus-common.yaml. And they do not follow the defined bindings for the
>>>>> recent venus platforms. Which means that as a developer I have to care
>>>>> about two different ways to describe nearly the same hardware.>> Again qualcomm video team does not have a plan to support sm8550/x1e80100 on
>>>>>> venus as the changes are too interleaved to absorb in venus driver. And there is
>>>>>> significant interest in community to start validating video driver on sm8550 or
>>>>>> x1e80100.
>>>>>>
>>>>>> [1] https://lore.kernel.org/lkml/[email protected]/
>>>>>>
>>>>>> Regards,
>>>>>> Vikash
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>
>
>

2023-12-20 20:51:38

by Nicolas Dufresne

[permalink] [raw]

Subject: Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver

Hi,

Le lundi 18 décembre 2023 à 17:01 +0530, Dikshita Agarwal a écrit :
> This patch series introduces support for Qualcomm new video acceleration
> hardware architecture, used for video stream decoding/encoding. This driver
> is based on new communication protocol between video hardware and application
> processor.
> This driver comes with below capabilities:
> - V4L2 complaint video driver with M2M and STREAMING capability.
> - Supports H264, H265, VP9 decoders.
> - Supports H264, H265 encoders.
> This driver comes with below features:
> - Centralized resource and memory management.
> - Centralized management of core and instance states.
> - Defines platform specific capabilities and features. As a results, it provides
> a single point of control to enable/disable a given feature depending on
> specific platform capabilities.
> - Handles hardware interdependent configurations. For a given configuration from
> client, the driver checks for hardware dependent configuration/s and updates
> the same.
> - Handles multiple complex video scenarios involving state transitions - DRC,
> Drain, Seek, back to back DRC, DRC during Drain sequence, DRC during Seek
> sequence.
> - Introduces a flexible way for driver to subscribe for any property with
> hardware. Hardware would inform driver with those subscribed property with any
> change in value.
> - Introduces performance (clock and bus) model based on new hardware
> architecture.
> - Introduces multi thread safe design to handle communication between client and
> hardware.
> - Adapts encoder quality improvements available in new hardware architecture.
> - Implements asynchronous communication with hardware to achieve better
> experience in low latency usecases.
> - Supports multi stage hardware architecture for encode/decode.
> - Output and capture planes are controlled independently. Thereby providing a
> way to reconfigure individual plane.
> - Hardware packetization layer supports synchronization between configuration
> packet and data packet.
> - Introduces a flexibility to receive a hardware response for a given command
> packet.
> - Native hardware support of LAST flag which is mandatory to align with port
> reconfiguration and DRAIN sequence as per V4L guidelines.
> - Native hardware support for drain sequence.
>
> Changes done since v1[1]:
> - Patches are created as logical split instead of file by file.
> - Extracted common functionality between venus and iris driver and placed them
> under common folder vcodec.
> - Flattened directory structure.
> - Restructured the code in a simplified layer architecture.
> - Implemented runtime PM, and dropped explicit power collapse thread in driver.
> - Moved to standard kernel log print api.
> - Simplified the bus and clock calculation logic.
> - Removed debug features like dma tracker, frame stats, debugfs.
> - Merged v4l2 and vidc layer as vidc in driver.
> - Removed separate probe for context bank.
> - Use of_device_get_match_data to get platform data.
> - Replaced complex macros with inline functions.
> - Migrated venus to iris names.
> - Addressed many other comments received on [1].
>
> [1]: https://patchwork.kernel.org/project/linux-media/list/?series=770581
>
> Patch 1, adds dt binding for iris driver.
>
> Patches 2-4, are introducing vcodec folder and moving common APIs like fw
> load/unload, buffer size calcualtions, read/write to shared queues to common
> files which are being used by both venus and iris drivers.
> This also moves the venus driver folder to vcodec folder.
>
> Patch 5, adds the maintainers for iris driver.
>
> Patch 6-29, adds core iris driver and enable decoder.
>
> Patch 30-34, enable encoder functionality in iris driver.
>
> Static tools like checkpatch, smatch, dt_binding_check, sparse and Coccinelle
> run successfully with this driver.
>
> The output of v4l2-compliance test looks like:
>
> v4l2-compliance SHA: not available, 64 bits
>
> Cannot open device /dev/vido0, exiting.
> root@qemuarm64:/usr/bin# ./v4l2-compliance -d /dev/video0
> v4l2-compliance SHA: not available, 64 bits
>
> Compliance test for iris_driver device /dev/video0:
>
> Driver Info:
> Driver name : iris_driver
> Card type : iris_decoder
> Bus info : platform:iris_bus
> Driver version : 6.6.0
> Capabilities : 0x84204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x04204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
>
> Required ioctls:
> test VIDIOC_QUERYCAP: OK
>
> Allow for multiple opens:
> test second /dev/video0 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> fail: ../../../v4l-utils-1.18.1/utils/v4l2-compliance/v4l2-compliance.cpp(724): !ok
> test for unlimited opens: FAIL

The test says:
/*
* There should *not* be an artificial limit to the number
* of open()s you can do on a V4L2 device node. So test whether
* you can open a device node at least 100 times.
*
* And please don't start rejecting opens in your driver at 101!
* There really shouldn't be a limit in the driver.
*
* If there are resource limits, then check against those limits
* where they are actually needed.
*/

So might have been acceptable for old venus, but not for iris since its a new
driver. Discussion and solution is needed. This is, I suppose, limiting the
number of decode sessions ? I personally find that method easier for userspace
then let say failing at streamon or something, but clearly a discussion need to
take place here as this is failing the compliance.

>
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 0 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls:
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> test VIDIOC_QUERYCTRL: OK
> test VIDIOC_G/S_CTRL: OK
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 48 Private Controls: 0
>
> Format ioctls:
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK (Not Supported)
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> test VIDIOC_S_FMT: OK
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> test Cropping: OK
> test Composing: OK
> test Scaling: OK
>
> Codec ioctls:
> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK
>
> Buffer ioctls:
> testVIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> test VIDIOC_EXPBUF: OK (Not Supported)
> test Requests: OK (Not Supported)
>
> Total for iris_driver device /dev/video0: 44, Succeeded: 43, Failed: 1, Warnings: 0
>
> Compliance test for iris_driver device /dev/video1:
>
> Driver Info:
> Driver name : iris_driver
> Card type : iris_encoder
> Bus info : platform:iris_bus
> Driver version : 6.6.0
> Capabilities : 0x84204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x04204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
>
> Required ioctls:
> test VIDIOC_QUERYCAP: OK
>
> Allow for multiple opens:
> test second /dev/video1 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> fail: ../../../v4l-utils-1.18.1/utils/v4l2-compliance/v4l2-compliance.cpp(724): !ok
> test for unlimited opens: FAIL
>
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 0 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls:
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> test VIDIOC_QUERYCTRL: OK
> test VIDIOC_G/S_CTRL: OK
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 48 Private Controls: 0
>
> Format ioctls:
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> test VIDIOC_S_FMT: OK
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> test Cropping: OK
> test Composing: OK
> test Scaling: OK (Not Supported)
>
> Codec ioctls:
> test VIDIOC_(TRY_)ENCODER_CMD: OK
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>
> Buffer ioctls:
> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> test VIDIOC_EXPBUF: OK (Not Supported)
> test Requests: OK (Not Supported)
>
> Total for iris_driver device /dev/video1: 44, Succeeded: 43, Failed: 1, Warnings: 0
>
> Dikshita Agarwal (27):
> media: introduce common helpers for video firmware handling
> media: introduce common helpers for queues handling
> media: introduce common helpers for buffer size calculation

These helper make it possible to introduce regressions, did you run compliance
tests against a venus based system to make sure ?

On top of this, initial submission and core modification to CODEC drivers are
now expected to be accompanied with fluster [0] results. This will help later
with integration testing enablement into KernelCI and will show that you made
sure the new driver is performing fairly well and is comparable to venus
(results available in Kernel CI, let me know if you have a hard time finding the
results).

[0] https://github.com/fluendo/fluster kernel CI uses GStreamer decoder, but
ffmpeg support is also available in the test runner.

> dt-bindings: media: Add sm8550 dt schema
> media: MAINTAINERS: Add Qualcomm Iris video accelerator driver
> media: iris: register video device to platform driver
> media: iris: initialize power resources
> media: iris: introduce state machine for iris core
> media: iris: initialize shared queues for host and firmware
> communication
> media: iris: add PIL functionality for video firmware
> media: iris: introduce packetization layer for creating HFI packets
> media: iris: add video processing unit(VPU) specific register handling
> media: iris: introduce platform specific capabilities for core and
> instance
> media: iris: add handling for interrupt service routine(ISR) invoked
> by hardware
> media: iris: implement iris v4l2_ctrl_ops and prepare capabilities
> media: iris: implement vb2_ops queue setup
> media: iris: implement HFI to queue and release buffers
> media: iris: add video hardware internal buffer count and size
> calculation
> media: iris: implement internal buffer management
> media: iris: introduce instance states
> media: iris: implement vb2 streaming ops on capture and output planes
> media: iris: implement vb2 ops for buf_queue and firmware response
> media: iris: register video encoder device to platform driver
> media: iris: add platform specific instance capabilities for encoder
> media: iris: implement iris v4l2 ioctl ops supported by encoder
> media: iris: add vb2 streaming and buffer ops for encoder
> media: iris: add power management for encoder
>
> Vikash Garodia (7):
> media: iris: implement iris v4l2 file ops
> media: iris: introduce and implement iris vb2 mem ops
> media: iris: implement iris v4l2 ioctl ops supported by decoder
> media: iris: subscribe input and output properties to firmware
> media: iris: subscribe src change and handle firmware responses
> media: iris: add instance sub states and implement DRC and Drain
> sequence
> media: iris: implement power management
>
> .../bindings/media/qcom,sm8550-iris.yaml | 177 ++
> MAINTAINERS | 11 +
> drivers/media/platform/qcom/Kconfig | 3 +-
> drivers/media/platform/qcom/Makefile | 3 +-
> drivers/media/platform/qcom/vcodec/buffers.c | 103 ++
> drivers/media/platform/qcom/vcodec/buffers.h | 15 +
> drivers/media/platform/qcom/vcodec/firmware.c | 147 ++
> drivers/media/platform/qcom/vcodec/firmware.h | 21 +
> drivers/media/platform/qcom/vcodec/hfi_queue.c | 258 +++
> drivers/media/platform/qcom/vcodec/hfi_queue.h | 129 ++
> drivers/media/platform/qcom/vcodec/iris/Kconfig | 13 +
> drivers/media/platform/qcom/vcodec/iris/Makefile | 26 +
> .../media/platform/qcom/vcodec/iris/hfi_defines.h | 386 +++++
> .../media/platform/qcom/vcodec/iris/iris_buffer.c | 833 +++++++++
> .../media/platform/qcom/vcodec/iris/iris_buffer.h | 65 +
> .../media/platform/qcom/vcodec/iris/iris_common.h | 147 ++
> .../media/platform/qcom/vcodec/iris/iris_core.c | 110 ++
> .../media/platform/qcom/vcodec/iris/iris_core.h | 121 ++
> .../media/platform/qcom/vcodec/iris/iris_ctrls.c | 1782 ++++++++++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_ctrls.h | 71 +
> .../media/platform/qcom/vcodec/iris/iris_helpers.c | 1099 ++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_helpers.h | 68 +
> drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 917 ++++++++++
> drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 47 +
> .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 729 ++++++++
> .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 118 ++
> .../platform/qcom/vcodec/iris/iris_hfi_response.c | 1097 ++++++++++++
> .../platform/qcom/vcodec/iris/iris_hfi_response.h | 20 +
> .../platform/qcom/vcodec/iris/iris_instance.h | 106 ++
> .../media/platform/qcom/vcodec/iris/iris_power.c | 178 ++
> .../media/platform/qcom/vcodec/iris/iris_power.h | 15 +
> .../media/platform/qcom/vcodec/iris/iris_probe.c | 357 ++++
> .../media/platform/qcom/vcodec/iris/iris_state.c | 453 +++++
> .../media/platform/qcom/vcodec/iris/iris_state.h | 78 +
> drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 457 +++++
> drivers/media/platform/qcom/vcodec/iris/iris_vb2.h | 28 +
> .../media/platform/qcom/vcodec/iris/iris_vdec.c | 1211 +++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_vdec.h | 25 +
> .../media/platform/qcom/vcodec/iris/iris_venc.c | 948 +++++++++++
> .../media/platform/qcom/vcodec/iris/iris_venc.h | 27 +
> .../media/platform/qcom/vcodec/iris/iris_vidc.c | 1419 ++++++++++++++++
> .../media/platform/qcom/vcodec/iris/iris_vidc.h | 15 +
> .../platform/qcom/vcodec/iris/platform_common.c | 29 +
> .../platform/qcom/vcodec/iris/platform_common.h | 323 ++++
> .../platform/qcom/vcodec/iris/platform_sm8550.c | 1190 +++++++++++++
> .../media/platform/qcom/vcodec/iris/resources.c | 506 ++++++
> .../media/platform/qcom/vcodec/iris/resources.h | 44 +
> .../media/platform/qcom/vcodec/iris/vpu_common.c | 158 ++
> .../media/platform/qcom/vcodec/iris/vpu_common.h | 52 +
> .../media/platform/qcom/vcodec/iris/vpu_iris3.c | 568 +++++++
> .../media/platform/qcom/vcodec/iris/vpu_iris3.h | 13 +
> .../platform/qcom/vcodec/iris/vpu_iris3_buffer.c | 395 +++++
> .../platform/qcom/vcodec/iris/vpu_iris3_buffer.h | 1469 ++++++++++++++++
> .../platform/qcom/vcodec/iris/vpu_iris3_power.c | 148 ++
> .../platform/qcom/vcodec/iris/vpu_iris3_power.h | 13 +
> .../media/platform/qcom/{ => vcodec}/venus/Kconfig | 0
> .../platform/qcom/{ => vcodec}/venus/Makefile | 5 +-
> .../media/platform/qcom/{ => vcodec}/venus/core.c | 102 +-
> .../media/platform/qcom/{ => vcodec}/venus/core.h | 0
> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.h | 0
> .../platform/qcom/vcodec/venus/firmware_no_tz.c | 194 +++
> .../platform/qcom/vcodec/venus/firmware_no_tz.h | 19 +
> .../platform/qcom/{ => vcodec}/venus/helpers.c | 172 +-
> .../platform/qcom/{ => vcodec}/venus/helpers.h | 2 +-
> .../media/platform/qcom/{ => vcodec}/venus/hfi.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/hfi.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_helper.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.h | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_parser.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_parser.h | 0
> .../qcom/{ => vcodec}/venus/hfi_plat_bufs.h | 4 +-
> .../qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c | 10 +-
> .../qcom/{ => vcodec}/venus/hfi_platform.c | 0
> .../qcom/{ => vcodec}/venus/hfi_platform.h | 0
> .../qcom/{ => vcodec}/venus/hfi_platform_v4.c | 0
> .../qcom/{ => vcodec}/venus/hfi_platform_v6.c | 0
> .../platform/qcom/{ => vcodec}/venus/hfi_venus.c | 521 +-----
> .../platform/qcom/{ => vcodec}/venus/hfi_venus.h | 0
> .../qcom/{ => vcodec}/venus/hfi_venus_io.h | 0
> .../platform/qcom/{ => vcodec}/venus/pm_helpers.c | 0
> .../platform/qcom/{ => vcodec}/venus/pm_helpers.h | 0
> .../media/platform/qcom/{ => vcodec}/venus/vdec.c | 5 +-
> .../media/platform/qcom/{ => vcodec}/venus/vdec.h | 0
> .../platform/qcom/{ => vcodec}/venus/vdec_ctrls.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/venc.c | 0
> .../media/platform/qcom/{ => vcodec}/venus/venc.h | 0
> .../platform/qcom/{ => vcodec}/venus/venc_ctrls.c | 0
> drivers/media/platform/qcom/venus/firmware.c | 363 ----
> drivers/media/platform/qcom/venus/firmware.h | 26 -
> 93 files changed, 19175 insertions(+), 989 deletions(-)
> create mode 100644 Documentation/devicetree/bindings/media/qcom,sm8550-iris.yaml
> create mode 100644 drivers/media/platform/qcom/vcodec/buffers.c
> create mode 100644 drivers/media/platform/qcom/vcodec/buffers.h
> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.c
> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.h
> create mode 100644 drivers/media/platform/qcom/vcodec/hfi_queue.c
> create mode 100644 drivers/media/platform/qcom/vcodec/hfi_queue.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/Kconfig
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/Makefile
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/hfi_defines.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_buffer.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_buffer.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_common.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_core.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_core.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_helpers.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_helpers.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_instance.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_power.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_power.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_probe.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_state.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_state.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vb2.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vb2.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_venc.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_venc.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_common.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_common.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/resources.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/resources.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_common.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_common.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c
> create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.h
> rename drivers/media/platform/qcom/{ => vcodec}/venus/Kconfig (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/Makefile (75%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.c (91%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.h (100%)
> create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
> create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.c (90%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.h (98%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_helper.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs.h (94%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c (99%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v4.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v6.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.c (73%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus_io.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.c (99%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec_ctrls.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.c (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.h (100%)
> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc_ctrls.c (100%)
> delete mode 100644 drivers/media/platform/qcom/venus/firmware.c
> delete mode 100644 drivers/media/platform/qcom/venus/firmware.h
>

2023-12-20 20:56:44

[permalink] [raw]

Subject: Re: [PATCH v2 01/34] media: introduce common helpers for video firmware handling

Hi Abhinav,

On Wed, 20 Dec 2023 at 19:10, Abhinav Kumar <[email protected]> wrote:
>
> Hi Dmitry
>
> On 12/20/2023 12:12 AM, Dmitry Baryshkov wrote:
> > On Wed, 20 Dec 2023 at 10:01, Dikshita Agarwal
> > <[email protected]> wrote:
> >>
> >>
> >>
> >> On 12/18/2023 11:54 PM, Dmitry Baryshkov wrote:
> >>> On 18/12/2023 13:31, Dikshita Agarwal wrote:
> >>>> Re-organize the video driver code by introducing a new folder
> >>>> 'vcodec' and placing 'venus' driver code inside that.
> >>>>
> >>>> Introduce common helpers for trustzone based firmware
> >>>> load/unload etc. which are placed in common folder
> >>>> i.e. 'vcodec'.
> >>>> Use these helpers in 'venus' driver. These helpers will be
> >>>> used by 'iris' driver as well which is introduced later
> >>>> in this patch series.
> >>>
> >>> But why do you need to move the venus driver to subdir?
> >>
> >> Currently venus driver is present in drivers/media/platform/qcom folder
> >> which also has camss folder. We introduced vcodec to keep common code and
> >> moved venus inside that, to indicate that the common code is for vcodec
> >> drivers i.e venus and iris. Keeping this in qcom folder would mean, common
> >> code will be used for camss only which is not the case here.
> >
> > you can have .../platform/qcom/camss, .../platform/qcom/vcodec-common,
> > .../platform/qcom/venus and .../platform/qcom/iris.
> >
> > If you were to use build helpers in a proper kernel module, this would
> > be more obvious.
> >
>
> Although your suggestion is good in terms of avoiding moving venus, I
> think the location of venus was wrong to begin with. There should have
> always been a vcodec (or similar) folder for venus/iris as that will
> establish the boundaries of camss and video sub-system in a cleaner way
>
> I like the mediatek separation that way as it makes the boundaries clear:
>
> drivers/media/platform/mediatek$ ls
> jpeg Kconfig Makefile mdp mdp3 vcodec vpu
>
> So I think that this re-org of venus into a vcodec had to happen at some
> point. Its just that it aligned with iris addition.

Then it should be a clean separate step with its own justification.

>
> >>>
> >>>>
> >>>> Signed-off-by: Dikshita Agarwal <[email protected]>
> >>>> ---
> >>>> drivers/media/platform/qcom/Kconfig | 2 +-
> >>>> drivers/media/platform/qcom/Makefile | 2 +-
> >>>> drivers/media/platform/qcom/vcodec/firmware.c | 147 +++++++++
> >>>> drivers/media/platform/qcom/vcodec/firmware.h | 21 ++
> >>>> .../media/platform/qcom/{ => vcodec}/venus/Kconfig | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/Makefile | 4 +-
> >>>> .../media/platform/qcom/{ => vcodec}/venus/core.c | 102 +++++-
> >>>> .../media/platform/qcom/{ => vcodec}/venus/core.h | 0
> >>>> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.c | 0
> >>>> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.h | 0
> >>>> .../platform/qcom/vcodec/venus/firmware_no_tz.c | 194 +++++++++++
> >>>> .../platform/qcom/vcodec/venus/firmware_no_tz.h | 19 ++
> >>>> .../platform/qcom/{ => vcodec}/venus/helpers.c | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/helpers.h | 0
> >>>> .../media/platform/qcom/{ => vcodec}/venus/hfi.c | 0
> >>>> .../media/platform/qcom/{ => vcodec}/venus/hfi.h | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.c | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.h | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/hfi_helper.h | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.c | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.h | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/hfi_parser.c | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/hfi_parser.h | 0
> >>>> .../qcom/{ => vcodec}/venus/hfi_plat_bufs.h | 0
> >>>> .../qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c | 0
> >>>> .../qcom/{ => vcodec}/venus/hfi_platform.c | 0
> >>>> .../qcom/{ => vcodec}/venus/hfi_platform.h | 0
> >>>> .../qcom/{ => vcodec}/venus/hfi_platform_v4.c | 0
> >>>> .../qcom/{ => vcodec}/venus/hfi_platform_v6.c | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/hfi_venus.c | 21 +-
> >>>> .../platform/qcom/{ => vcodec}/venus/hfi_venus.h | 0
> >>>> .../qcom/{ => vcodec}/venus/hfi_venus_io.h | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/pm_helpers.c | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/pm_helpers.h | 0
> >>>> .../media/platform/qcom/{ => vcodec}/venus/vdec.c | 0
> >>>> .../media/platform/qcom/{ => vcodec}/venus/vdec.h | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/vdec_ctrls.c | 0
> >>>> .../media/platform/qcom/{ => vcodec}/venus/venc.c | 0
> >>>> .../media/platform/qcom/{ => vcodec}/venus/venc.h | 0
> >>>> .../platform/qcom/{ => vcodec}/venus/venc_ctrls.c | 0
> >>>> drivers/media/platform/qcom/venus/firmware.c | 363
> >>>> ---------------------
> >>>> drivers/media/platform/qcom/venus/firmware.h | 26 --
> >>>> 42 files changed, 492 insertions(+), 409 deletions(-)
> >>>> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.c
> >>>> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.h
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/Kconfig (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/Makefile (83%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.c (91%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.h (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.c (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.h (100%)
> >>>> create mode 100644
> >>>> drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
> >>>> create mode 100644
> >>>> drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.c (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.h (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.c (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.h (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.c (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.h (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_helper.h (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.c (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.h (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.c (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.h (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs.h
> >>>> (100%)
> >>>> rename drivers/media/platform/qcom/{ =>
> >>>> vcodec}/venus/hfi_plat_bufs_v6.c (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.c
> >>>> (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.h
> >>>> (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v4.c
> >>>> (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v6.c
> >>>> (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.c (99%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.h (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus_io.h
> >>>> (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.c (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.h (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.c (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.h (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec_ctrls.c (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.c (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.h (100%)
> >>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc_ctrls.c (100%)
> >>>> delete mode 100644 drivers/media/platform/qcom/venus/firmware.c
> >>>> delete mode 100644 drivers/media/platform/qcom/venus/firmware.h
> >>>>
> >>>> diff --git a/drivers/media/platform/qcom/Kconfig
> >>>> b/drivers/media/platform/qcom/Kconfig
> >>>> index cc5799b..e94142f 100644
> >>>> --- a/drivers/media/platform/qcom/Kconfig
> >>>> +++ b/drivers/media/platform/qcom/Kconfig
> >>>> @@ -3,4 +3,4 @@
> >>>> comment "Qualcomm media platform drivers"
> >>>> source "drivers/media/platform/qcom/camss/Kconfig"
> >>>> -source "drivers/media/platform/qcom/venus/Kconfig"
> >>>> +source "drivers/media/platform/qcom/vcodec/venus/Kconfig"
> >>>> diff --git a/drivers/media/platform/qcom/Makefile
> >>>> b/drivers/media/platform/qcom/Makefile
> >>>> index 4f055c3..3d2d82b 100644
> >>>> --- a/drivers/media/platform/qcom/Makefile
> >>>> +++ b/drivers/media/platform/qcom/Makefile
> >>>> @@ -1,3 +1,3 @@
> >>>> # SPDX-License-Identifier: GPL-2.0-only
> >>>> obj-y += camss/
> >>>> -obj-y += venus/
> >>>> +obj-y += vcodec/venus/
> >>>> diff --git a/drivers/media/platform/qcom/vcodec/firmware.c
> >>>> b/drivers/media/platform/qcom/vcodec/firmware.c
> >>>> new file mode 100644
> >>>> index 0000000..dbc220a
> >>>> --- /dev/null
> >>>> +++ b/drivers/media/platform/qcom/vcodec/firmware.c
> >>>> @@ -0,0 +1,147 @@
> >>>> +// SPDX-License-Identifier: GPL-2.0-only
> >>>> +/*
> >>>> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
> >>>> reserved.
> >>>> + */
> >>>> +
> >>>> +#include <linux/device.h>
> >>>> +#include <linux/dma-mapping.h>
> >>>> +#include <linux/firmware.h>
> >>>> +#include <linux/kernel.h>
> >>>> +#include <linux/iommu.h>
> >>>> +#include <linux/of_device.h>
> >>>> +#include <linux/firmware/qcom/qcom_scm.h>
> >>>> +#include <linux/of_reserved_mem.h>
> >>>> +#include <linux/platform_device.h>
> >>>> +#include <linux/soc/qcom/mdt_loader.h>
> >>>> +
> >>>> +#include "firmware.h"
> >>>> +
> >>>> +bool use_tz(struct device *core_dev)
> >>>
> >>> All these functions must get some sane prefix. Otherwise a generic 'use_tz'
> >>> function is too polluting for the global namespace.
> >>>
> >> I understand, will check and do the needful.
> >>>> +{
> >>>> + struct device_node *np;
> >>>> +
> >>>> + np = of_get_child_by_name(core_dev->of_node, "video-firmware");
> >>>> + if (!np)
> >>>> + return true;
> >>>> +
> >>>> + return false;
> >>>> +}
> >>>> +
> >>>> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
> >>>> + u32 cp_nonpixel_size, u32 pas_id)
> >>>> +{
> >>>> + int ret;
> >>>> + /*
> >>>> + * Clues for porting using downstream data:
> >>>> + * cp_start = 0
> >>>> + * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and not size!
> >>>> + * This works, as the non-secure context bank is placed
> >>>> + * contiguously right after the Content Protection region.
> >>>> + *
> >>>> + * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0]
> >>>> + * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1]
> >>>> + */
> >>>> + ret = qcom_scm_mem_protect_video_var(cp_start,
> >>>> + cp_size,
> >>>> + cp_nonpixel_start,
> >>>> + cp_nonpixel_size);
> >>>> + if (ret)
> >>>> + qcom_scm_pas_shutdown(pas_id);
> >>>> +
> >>>> + return ret;
> >>>> +}
> >>>> +
> >>>> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
> >>>> + size_t *mem_size, u32 pas_id, bool use_tz)
> >>>> +{
> >>>> + const struct firmware *firmware = NULL;
> >>>> + struct reserved_mem *rmem;
> >>>> + struct device_node *node;
> >>>> + void *mem_virt = NULL;
> >>>> + ssize_t fw_size = 0;
> >>>> + int ret;
> >>>> +
> >>>> + if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
> >>>
> >>> Why? Can you just depend on it?
> >>>
> >> Sure, Will check this and get back.
> >>>> + (use_tz && !qcom_scm_is_available()))
> >>>> + return -EPROBE_DEFER;
> >>>> +
> >>>> + if (!fw_name || !(*fw_name))
> >>>> + return -EINVAL;
> >>>> +
> >>>> + *mem_phys = 0;
> >>>> + *mem_size = 0;
> >>>> +
> >>>> + node = of_parse_phandle(dev->of_node, "memory-region", 0);
> >>>> + if (!node) {
> >>>> + dev_err(dev, "no memory-region specified\n");
> >>>> + return -EINVAL;
> >>>> + }
> >>>> +
> >>>> + rmem = of_reserved_mem_lookup(node);
> >>>> + of_node_put(node);
> >>>> + if (!rmem) {
> >>>> + dev_err(dev, "failed to lookup reserved memory-region\n");
> >>>> + return -EINVAL;
> >>>> + }
> >>>> +
> >>>> + ret = request_firmware(&firmware, fw_name, dev);
> >>>> + if (ret) {
> >>>> + dev_err(dev, "%s: failed to request fw \"%s\", error %d\n",
> >>>> + __func__, fw_name, ret);
> >>>> + return ret;
> >>>> + }
> >>>> +
> >>>> + fw_size = qcom_mdt_get_size(firmware);
> >>>> + if (fw_size < 0) {
> >>>> + ret = fw_size;
> >>>> + dev_err(dev, "%s: out of bound fw image fw size: %ld\n",
> >>>> + __func__, fw_size);
> >>>> + goto err_release_fw;
> >>>> + }
> >>>> +
> >>>> + *mem_phys = rmem->base;
> >>>> + *mem_size = rmem->size;
> >>>> +
> >>>> + if (*mem_size < fw_size) {
> >>>> + ret = -EINVAL;
> >>>> + goto err_release_fw;
> >>>> + }
> >>>> +
> >>>> + mem_virt = memremap(*mem_phys, *mem_size, MEMREMAP_WC);
> >>>> + if (!mem_virt) {
> >>>> + dev_err(dev, "unable to remap fw memory region %pa size %#zx\n",
> >>>> + mem_phys, *mem_size);
> >>>> + goto err_release_fw;
> >>>> + }
> >>>> +
> >>>> + if (use_tz)
> >>>> + ret = qcom_mdt_load(dev, firmware, fw_name, pas_id, mem_virt,
> >>>> + *mem_phys, *mem_size, NULL);
> >>>> + else
> >>>> + ret = qcom_mdt_load_no_init(dev, firmware, fw_name, pas_id,
> >>>> mem_virt,
> >>>> + *mem_phys, *mem_size, NULL);
> >>>> + if (ret) {
> >>>> + dev_err(dev, "%s: error %d loading fw \"%s\"\n",
> >>>> + __func__, ret, fw_name);
> >>>> + }
> >>>> +
> >>>> + memunmap(mem_virt);
> >>>> +err_release_fw:
> >>>> + release_firmware(firmware);
> >>>> + return ret;
> >>>> +}
> >>>> +
> >>>> +int auth_reset_fw(u32 pas_id)
> >>>> +{
> >>>> + return qcom_scm_pas_auth_and_reset(pas_id);
> >>>> +}
> >>>> +
> >>>> +void unload_fw(u32 pas_id)
> >>>> +{
> >>>> + qcom_scm_pas_shutdown(pas_id);
> >>>> +}
> >>>> +
> >>>> +int set_hw_state(bool resume)
> >>>> +{
> >>>> + return qcom_scm_set_remote_state(resume, 0);
> >>>> +}
> >>>> diff --git a/drivers/media/platform/qcom/vcodec/firmware.h
> >>>> b/drivers/media/platform/qcom/vcodec/firmware.h
> >>>> new file mode 100644
> >>>> index 0000000..7d410a8
> >>>> --- /dev/null
> >>>> +++ b/drivers/media/platform/qcom/vcodec/firmware.h
> >>>> @@ -0,0 +1,21 @@
> >>>> +/* SPDX-License-Identifier: GPL-2.0-only */
> >>>> +/*
> >>>> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
> >>>> reserved.
> >>>> + */
> >>>> +
> >>>> +#ifndef _FIRMWARE_H_
> >>>> +#define _FIRMWARE_H_
> >>>> +
> >>>> +#include <linux/device.h>
> >>>> +#include <linux/types.h>
> >>>> +
> >>>> +bool use_tz(struct device *core_dev);
> >>>> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
> >>>> + size_t *mem_size, u32 pas_id, bool use_tz);
> >>>> +int auth_reset_fw(u32 pas_id);
> >>>> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
> >>>> + u32 cp_nonpixel_size, u32 pas_id);
> >>>> +void unload_fw(u32 pas_id);
> >>>> +int set_hw_state(bool resume);
> >>>> +
> >>>> +#endif
> >>>> diff --git a/drivers/media/platform/qcom/venus/Kconfig
> >>>> b/drivers/media/platform/qcom/vcodec/venus/Kconfig
> >>>> similarity index 100%
> >>>> rename from drivers/media/platform/qcom/venus/Kconfig
> >>>> rename to drivers/media/platform/qcom/vcodec/venus/Kconfig
> >>>> diff --git a/drivers/media/platform/qcom/venus/Makefile
> >>>> b/drivers/media/platform/qcom/vcodec/venus/Makefile
> >>>> similarity index 83%
> >>>> rename from drivers/media/platform/qcom/venus/Makefile
> >>>> rename to drivers/media/platform/qcom/vcodec/venus/Makefile
> >>>> index 91ee6be..f6f3a88 100644
> >>>> --- a/drivers/media/platform/qcom/venus/Makefile
> >>>> +++ b/drivers/media/platform/qcom/vcodec/venus/Makefile
> >>>> @@ -1,7 +1,9 @@
> >>>> # SPDX-License-Identifier: GPL-2.0
> >>>> # Makefile for Qualcomm Venus driver
> >>>> -venus-core-objs += core.o helpers.o firmware.o \
> >>>> +venus-core-objs += ../firmware.o
> >>>> +
> >>>> +venus-core-objs += core.o helpers.o firmware_no_tz.o \
> >>>> hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \
> >>>> hfi_parser.o pm_helpers.o dbgfs.o \
> >>>> hfi_platform.o hfi_platform_v4.o \
> >>>> diff --git a/drivers/media/platform/qcom/venus/core.c
> >>>> b/drivers/media/platform/qcom/vcodec/venus/core.c
> >>>> similarity index 91%
> >>>> rename from drivers/media/platform/qcom/venus/core.c
> >>>> rename to drivers/media/platform/qcom/vcodec/venus/core.c
> >>>> index 9cffe97..56d9a53 100644
> >>>> --- a/drivers/media/platform/qcom/venus/core.c
> >>>> +++ b/drivers/media/platform/qcom/vcodec/venus/core.c
> >>>> @@ -22,7 +22,8 @@
> >>>> #include <media/v4l2-ioctl.h>
> >>>> #include "core.h"
> >>>> -#include "firmware.h"
> >>>> +#include "../firmware.h"
> >>>> +#include "firmware_no_tz.h"
> >>>> #include "pm_helpers.h"
> >>>> #include "hfi_venus_io.h"
> >>>> @@ -86,6 +87,8 @@ static void venus_sys_error_handler(struct
> >>>> work_struct *work)
> >>>> struct venus_core *core =
> >>>> container_of(work, struct venus_core, work.work);
> >>>> int ret, i, max_attempts = RPM_WAIT_FOR_IDLE_MAX_ATTEMPTS;
> >>>> + const struct venus_resources *res = core->res;
> >>>> + const char *fwpath = NULL;
> >>>> const char *err_msg = "";
> >>>> bool failed = false;
> >>>> @@ -107,7 +110,10 @@ static void venus_sys_error_handler(struct
> >>>> work_struct *work)
> >>>> mutex_lock(&core->lock);
> >>>> - venus_shutdown(core);
> >>>> + if (core->use_tz)
> >>>> + unload_fw(VENUS_PAS_ID);
> >>>> + else
> >>>> + unload_fw_no_tz(core);
> >>>
> >>> This is more than introducing helpers.
> >>>
> >> The new helpers are written to make the code generic for video drivers.
> >> which requires changes in the calling function also.
> >>>> venus_coredump(core);
> >>>> @@ -127,12 +133,39 @@ static void venus_sys_error_handler(struct
> >>>> work_struct *work)
> >>>> failed = true;
> >>>> }
> >>>> - ret = venus_boot(core);
> >>>> + ret = of_property_read_string_index(core->dev->of_node,
> >>>> "firmware-name", 0,
> >>>> + &fwpath);
> >>>> + if (ret)
> >>>> + fwpath = core->res->fwname;
> >>>> +
> >>>> + ret = load_fw(core->dev, fwpath, &core->fw.mem_phys,
> >>>> &core->fw.mem_size,
> >>>> + VENUS_PAS_ID, core->use_tz);
> >>>
> >>> So, we had a nice local 'venus_boot'. Instead we now have a pile of code
> >>> with non-generic prefixes, etc. If you are introducing helpers, please
> >>> refrain from inlining of calling functions, etc. Just move the code to your
> >>> helpers.
> >>>
> >> As mentioned in above comment, the common helpers are written to make the
> >> code generic. I Will try to make it more clear, working on the same.
> >
> > First, you move the code, then you make it generic. Or vice versa.
> > First you split the code, then you move it. Don't do both in the same
> > patch.
> >
> >>> NAK for the rest of the patch.
> >
> >

--
With best wishes
Dmitry

2023-12-20 21:04:22

by Abhinav Kumar

[permalink] [raw]

Subject: Re: [PATCH v2 01/34] media: introduce common helpers for video firmware handling

On 12/20/2023 12:56 PM, Dmitry Baryshkov wrote:
> Hi Abhinav,
>
> On Wed, 20 Dec 2023 at 19:10, Abhinav Kumar <[email protected]> wrote:
>>
>> Hi Dmitry
>>
>> On 12/20/2023 12:12 AM, Dmitry Baryshkov wrote:
>>> On Wed, 20 Dec 2023 at 10:01, Dikshita Agarwal
>>> <[email protected]> wrote:
>>>>
>>>>
>>>>
>>>> On 12/18/2023 11:54 PM, Dmitry Baryshkov wrote:
>>>>> On 18/12/2023 13:31, Dikshita Agarwal wrote:
>>>>>> Re-organize the video driver code by introducing a new folder
>>>>>> 'vcodec' and placing 'venus' driver code inside that.
>>>>>>
>>>>>> Introduce common helpers for trustzone based firmware
>>>>>> load/unload etc. which are placed in common folder
>>>>>> i.e. 'vcodec'.
>>>>>> Use these helpers in 'venus' driver. These helpers will be
>>>>>> used by 'iris' driver as well which is introduced later
>>>>>> in this patch series.
>>>>>
>>>>> But why do you need to move the venus driver to subdir?
>>>>
>>>> Currently venus driver is present in drivers/media/platform/qcom folder
>>>> which also has camss folder. We introduced vcodec to keep common code and
>>>> moved venus inside that, to indicate that the common code is for vcodec
>>>> drivers i.e venus and iris. Keeping this in qcom folder would mean, common
>>>> code will be used for camss only which is not the case here.
>>>
>>> you can have .../platform/qcom/camss, .../platform/qcom/vcodec-common,
>>> .../platform/qcom/venus and .../platform/qcom/iris.
>>>
>>> If you were to use build helpers in a proper kernel module, this would
>>> be more obvious.
>>>
>>
>> Although your suggestion is good in terms of avoiding moving venus, I
>> think the location of venus was wrong to begin with. There should have
>> always been a vcodec (or similar) folder for venus/iris as that will
>> establish the boundaries of camss and video sub-system in a cleaner way
>>
>> I like the mediatek separation that way as it makes the boundaries clear:
>>
>> drivers/media/platform/mediatek$ ls
>> jpeg Kconfig Makefile mdp mdp3 vcodec vpu
>>
>> So I think that this re-org of venus into a vcodec had to happen at some
>> point. Its just that it aligned with iris addition.
>
> Then it should be a clean separate step with its own justification.
>

Yes, I am fine with this first going as a separate series to move venus
into vcodec and anyway there was a separate request to make patches 1-3
into another series from Bryan.

So this can first be posted separately.

>>
>>>>>
>>>>>>
>>>>>> Signed-off-by: Dikshita Agarwal <[email protected]>
>>>>>> ---
>>>>>> drivers/media/platform/qcom/Kconfig | 2 +-
>>>>>> drivers/media/platform/qcom/Makefile | 2 +-
>>>>>> drivers/media/platform/qcom/vcodec/firmware.c | 147 +++++++++
>>>>>> drivers/media/platform/qcom/vcodec/firmware.h | 21 ++
>>>>>> .../media/platform/qcom/{ => vcodec}/venus/Kconfig | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/Makefile | 4 +-
>>>>>> .../media/platform/qcom/{ => vcodec}/venus/core.c | 102 +++++-
>>>>>> .../media/platform/qcom/{ => vcodec}/venus/core.h | 0
>>>>>> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.c | 0
>>>>>> .../media/platform/qcom/{ => vcodec}/venus/dbgfs.h | 0
>>>>>> .../platform/qcom/vcodec/venus/firmware_no_tz.c | 194 +++++++++++
>>>>>> .../platform/qcom/vcodec/venus/firmware_no_tz.h | 19 ++
>>>>>> .../platform/qcom/{ => vcodec}/venus/helpers.c | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/helpers.h | 0
>>>>>> .../media/platform/qcom/{ => vcodec}/venus/hfi.c | 0
>>>>>> .../media/platform/qcom/{ => vcodec}/venus/hfi.h | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.c | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/hfi_cmds.h | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/hfi_helper.h | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.c | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/hfi_msgs.h | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/hfi_parser.c | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/hfi_parser.h | 0
>>>>>> .../qcom/{ => vcodec}/venus/hfi_plat_bufs.h | 0
>>>>>> .../qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c | 0
>>>>>> .../qcom/{ => vcodec}/venus/hfi_platform.c | 0
>>>>>> .../qcom/{ => vcodec}/venus/hfi_platform.h | 0
>>>>>> .../qcom/{ => vcodec}/venus/hfi_platform_v4.c | 0
>>>>>> .../qcom/{ => vcodec}/venus/hfi_platform_v6.c | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/hfi_venus.c | 21 +-
>>>>>> .../platform/qcom/{ => vcodec}/venus/hfi_venus.h | 0
>>>>>> .../qcom/{ => vcodec}/venus/hfi_venus_io.h | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/pm_helpers.c | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/pm_helpers.h | 0
>>>>>> .../media/platform/qcom/{ => vcodec}/venus/vdec.c | 0
>>>>>> .../media/platform/qcom/{ => vcodec}/venus/vdec.h | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/vdec_ctrls.c | 0
>>>>>> .../media/platform/qcom/{ => vcodec}/venus/venc.c | 0
>>>>>> .../media/platform/qcom/{ => vcodec}/venus/venc.h | 0
>>>>>> .../platform/qcom/{ => vcodec}/venus/venc_ctrls.c | 0
>>>>>> drivers/media/platform/qcom/venus/firmware.c | 363
>>>>>> ---------------------
>>>>>> drivers/media/platform/qcom/venus/firmware.h | 26 --
>>>>>> 42 files changed, 492 insertions(+), 409 deletions(-)
>>>>>> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.c
>>>>>> create mode 100644 drivers/media/platform/qcom/vcodec/firmware.h
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/Kconfig (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/Makefile (83%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.c (91%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/core.h (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.c (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.h (100%)
>>>>>> create mode 100644
>>>>>> drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c
>>>>>> create mode 100644
>>>>>> drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.c (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.h (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.c (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.h (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.c (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.h (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_helper.h (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.c (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.h (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.c (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.h (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs.h
>>>>>> (100%)
>>>>>> rename drivers/media/platform/qcom/{ =>
>>>>>> vcodec}/venus/hfi_plat_bufs_v6.c (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.c
>>>>>> (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.h
>>>>>> (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v4.c
>>>>>> (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v6.c
>>>>>> (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.c (99%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.h (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus_io.h
>>>>>> (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.c (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.h (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.c (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.h (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec_ctrls.c (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.c (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.h (100%)
>>>>>> rename drivers/media/platform/qcom/{ => vcodec}/venus/venc_ctrls.c (100%)
>>>>>> delete mode 100644 drivers/media/platform/qcom/venus/firmware.c
>>>>>> delete mode 100644 drivers/media/platform/qcom/venus/firmware.h
>>>>>>
>>>>>> diff --git a/drivers/media/platform/qcom/Kconfig
>>>>>> b/drivers/media/platform/qcom/Kconfig
>>>>>> index cc5799b..e94142f 100644
>>>>>> --- a/drivers/media/platform/qcom/Kconfig
>>>>>> +++ b/drivers/media/platform/qcom/Kconfig
>>>>>> @@ -3,4 +3,4 @@
>>>>>> comment "Qualcomm media platform drivers"
>>>>>> source "drivers/media/platform/qcom/camss/Kconfig"
>>>>>> -source "drivers/media/platform/qcom/venus/Kconfig"
>>>>>> +source "drivers/media/platform/qcom/vcodec/venus/Kconfig"
>>>>>> diff --git a/drivers/media/platform/qcom/Makefile
>>>>>> b/drivers/media/platform/qcom/Makefile
>>>>>> index 4f055c3..3d2d82b 100644
>>>>>> --- a/drivers/media/platform/qcom/Makefile
>>>>>> +++ b/drivers/media/platform/qcom/Makefile
>>>>>> @@ -1,3 +1,3 @@
>>>>>> # SPDX-License-Identifier: GPL-2.0-only
>>>>>> obj-y += camss/
>>>>>> -obj-y += venus/
>>>>>> +obj-y += vcodec/venus/
>>>>>> diff --git a/drivers/media/platform/qcom/vcodec/firmware.c
>>>>>> b/drivers/media/platform/qcom/vcodec/firmware.c
>>>>>> new file mode 100644
>>>>>> index 0000000..dbc220a
>>>>>> --- /dev/null
>>>>>> +++ b/drivers/media/platform/qcom/vcodec/firmware.c
>>>>>> @@ -0,0 +1,147 @@
>>>>>> +// SPDX-License-Identifier: GPL-2.0-only
>>>>>> +/*
>>>>>> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
>>>>>> reserved.
>>>>>> + */
>>>>>> +
>>>>>> +#include <linux/device.h>
>>>>>> +#include <linux/dma-mapping.h>
>>>>>> +#include <linux/firmware.h>
>>>>>> +#include <linux/kernel.h>
>>>>>> +#include <linux/iommu.h>
>>>>>> +#include <linux/of_device.h>
>>>>>> +#include <linux/firmware/qcom/qcom_scm.h>
>>>>>> +#include <linux/of_reserved_mem.h>
>>>>>> +#include <linux/platform_device.h>
>>>>>> +#include <linux/soc/qcom/mdt_loader.h>
>>>>>> +
>>>>>> +#include "firmware.h"
>>>>>> +
>>>>>> +bool use_tz(struct device *core_dev)
>>>>>
>>>>> All these functions must get some sane prefix. Otherwise a generic 'use_tz'
>>>>> function is too polluting for the global namespace.
>>>>>
>>>> I understand, will check and do the needful.
>>>>>> +{
>>>>>> + struct device_node *np;
>>>>>> +
>>>>>> + np = of_get_child_by_name(core_dev->of_node, "video-firmware");
>>>>>> + if (!np)
>>>>>> + return true;
>>>>>> +
>>>>>> + return false;
>>>>>> +}
>>>>>> +
>>>>>> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
>>>>>> + u32 cp_nonpixel_size, u32 pas_id)
>>>>>> +{
>>>>>> + int ret;
>>>>>> + /*
>>>>>> + * Clues for porting using downstream data:
>>>>>> + * cp_start = 0
>>>>>> + * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and not size!
>>>>>> + * This works, as the non-secure context bank is placed
>>>>>> + * contiguously right after the Content Protection region.
>>>>>> + *
>>>>>> + * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0]
>>>>>> + * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1]
>>>>>> + */
>>>>>> + ret = qcom_scm_mem_protect_video_var(cp_start,
>>>>>> + cp_size,
>>>>>> + cp_nonpixel_start,
>>>>>> + cp_nonpixel_size);
>>>>>> + if (ret)
>>>>>> + qcom_scm_pas_shutdown(pas_id);
>>>>>> +
>>>>>> + return ret;
>>>>>> +}
>>>>>> +
>>>>>> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
>>>>>> + size_t *mem_size, u32 pas_id, bool use_tz)
>>>>>> +{
>>>>>> + const struct firmware *firmware = NULL;
>>>>>> + struct reserved_mem *rmem;
>>>>>> + struct device_node *node;
>>>>>> + void *mem_virt = NULL;
>>>>>> + ssize_t fw_size = 0;
>>>>>> + int ret;
>>>>>> +
>>>>>> + if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) ||
>>>>>
>>>>> Why? Can you just depend on it?
>>>>>
>>>> Sure, Will check this and get back.
>>>>>> + (use_tz && !qcom_scm_is_available()))
>>>>>> + return -EPROBE_DEFER;
>>>>>> +
>>>>>> + if (!fw_name || !(*fw_name))
>>>>>> + return -EINVAL;
>>>>>> +
>>>>>> + *mem_phys = 0;
>>>>>> + *mem_size = 0;
>>>>>> +
>>>>>> + node = of_parse_phandle(dev->of_node, "memory-region", 0);
>>>>>> + if (!node) {
>>>>>> + dev_err(dev, "no memory-region specified\n");
>>>>>> + return -EINVAL;
>>>>>> + }
>>>>>> +
>>>>>> + rmem = of_reserved_mem_lookup(node);
>>>>>> + of_node_put(node);
>>>>>> + if (!rmem) {
>>>>>> + dev_err(dev, "failed to lookup reserved memory-region\n");
>>>>>> + return -EINVAL;
>>>>>> + }
>>>>>> +
>>>>>> + ret = request_firmware(&firmware, fw_name, dev);
>>>>>> + if (ret) {
>>>>>> + dev_err(dev, "%s: failed to request fw \"%s\", error %d\n",
>>>>>> + __func__, fw_name, ret);
>>>>>> + return ret;
>>>>>> + }
>>>>>> +
>>>>>> + fw_size = qcom_mdt_get_size(firmware);
>>>>>> + if (fw_size < 0) {
>>>>>> + ret = fw_size;
>>>>>> + dev_err(dev, "%s: out of bound fw image fw size: %ld\n",
>>>>>> + __func__, fw_size);
>>>>>> + goto err_release_fw;
>>>>>> + }
>>>>>> +
>>>>>> + *mem_phys = rmem->base;
>>>>>> + *mem_size = rmem->size;
>>>>>> +
>>>>>> + if (*mem_size < fw_size) {
>>>>>> + ret = -EINVAL;
>>>>>> + goto err_release_fw;
>>>>>> + }
>>>>>> +
>>>>>> + mem_virt = memremap(*mem_phys, *mem_size, MEMREMAP_WC);
>>>>>> + if (!mem_virt) {
>>>>>> + dev_err(dev, "unable to remap fw memory region %pa size %#zx\n",
>>>>>> + mem_phys, *mem_size);
>>>>>> + goto err_release_fw;
>>>>>> + }
>>>>>> +
>>>>>> + if (use_tz)
>>>>>> + ret = qcom_mdt_load(dev, firmware, fw_name, pas_id, mem_virt,
>>>>>> + *mem_phys, *mem_size, NULL);
>>>>>> + else
>>>>>> + ret = qcom_mdt_load_no_init(dev, firmware, fw_name, pas_id,
>>>>>> mem_virt,
>>>>>> + *mem_phys, *mem_size, NULL);
>>>>>> + if (ret) {
>>>>>> + dev_err(dev, "%s: error %d loading fw \"%s\"\n",
>>>>>> + __func__, ret, fw_name);
>>>>>> + }
>>>>>> +
>>>>>> + memunmap(mem_virt);
>>>>>> +err_release_fw:
>>>>>> + release_firmware(firmware);
>>>>>> + return ret;
>>>>>> +}
>>>>>> +
>>>>>> +int auth_reset_fw(u32 pas_id)
>>>>>> +{
>>>>>> + return qcom_scm_pas_auth_and_reset(pas_id);
>>>>>> +}
>>>>>> +
>>>>>> +void unload_fw(u32 pas_id)
>>>>>> +{
>>>>>> + qcom_scm_pas_shutdown(pas_id);
>>>>>> +}
>>>>>> +
>>>>>> +int set_hw_state(bool resume)
>>>>>> +{
>>>>>> + return qcom_scm_set_remote_state(resume, 0);
>>>>>> +}
>>>>>> diff --git a/drivers/media/platform/qcom/vcodec/firmware.h
>>>>>> b/drivers/media/platform/qcom/vcodec/firmware.h
>>>>>> new file mode 100644
>>>>>> index 0000000..7d410a8
>>>>>> --- /dev/null
>>>>>> +++ b/drivers/media/platform/qcom/vcodec/firmware.h
>>>>>> @@ -0,0 +1,21 @@
>>>>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>>>>> +/*
>>>>>> + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights
>>>>>> reserved.
>>>>>> + */
>>>>>> +
>>>>>> +#ifndef _FIRMWARE_H_
>>>>>> +#define _FIRMWARE_H_
>>>>>> +
>>>>>> +#include <linux/device.h>
>>>>>> +#include <linux/types.h>
>>>>>> +
>>>>>> +bool use_tz(struct device *core_dev);
>>>>>> +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys,
>>>>>> + size_t *mem_size, u32 pas_id, bool use_tz);
>>>>>> +int auth_reset_fw(u32 pas_id);
>>>>>> +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start,
>>>>>> + u32 cp_nonpixel_size, u32 pas_id);
>>>>>> +void unload_fw(u32 pas_id);
>>>>>> +int set_hw_state(bool resume);
>>>>>> +
>>>>>> +#endif
>>>>>> diff --git a/drivers/media/platform/qcom/venus/Kconfig
>>>>>> b/drivers/media/platform/qcom/vcodec/venus/Kconfig
>>>>>> similarity index 100%
>>>>>> rename from drivers/media/platform/qcom/venus/Kconfig
>>>>>> rename to drivers/media/platform/qcom/vcodec/venus/Kconfig
>>>>>> diff --git a/drivers/media/platform/qcom/venus/Makefile
>>>>>> b/drivers/media/platform/qcom/vcodec/venus/Makefile
>>>>>> similarity index 83%
>>>>>> rename from drivers/media/platform/qcom/venus/Makefile
>>>>>> rename to drivers/media/platform/qcom/vcodec/venus/Makefile
>>>>>> index 91ee6be..f6f3a88 100644
>>>>>> --- a/drivers/media/platform/qcom/venus/Makefile
>>>>>> +++ b/drivers/media/platform/qcom/vcodec/venus/Makefile
>>>>>> @@ -1,7 +1,9 @@
>>>>>> # SPDX-License-Identifier: GPL-2.0
>>>>>> # Makefile for Qualcomm Venus driver
>>>>>> -venus-core-objs += core.o helpers.o firmware.o \
>>>>>> +venus-core-objs += ../firmware.o
>>>>>> +
>>>>>> +venus-core-objs += core.o helpers.o firmware_no_tz.o \
>>>>>> hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \
>>>>>> hfi_parser.o pm_helpers.o dbgfs.o \
>>>>>> hfi_platform.o hfi_platform_v4.o \
>>>>>> diff --git a/drivers/media/platform/qcom/venus/core.c
>>>>>> b/drivers/media/platform/qcom/vcodec/venus/core.c
>>>>>> similarity index 91%
>>>>>> rename from drivers/media/platform/qcom/venus/core.c
>>>>>> rename to drivers/media/platform/qcom/vcodec/venus/core.c
>>>>>> index 9cffe97..56d9a53 100644
>>>>>> --- a/drivers/media/platform/qcom/venus/core.c
>>>>>> +++ b/drivers/media/platform/qcom/vcodec/venus/core.c
>>>>>> @@ -22,7 +22,8 @@
>>>>>> #include <media/v4l2-ioctl.h>
>>>>>> #include "core.h"
>>>>>> -#include "firmware.h"
>>>>>> +#include "../firmware.h"
>>>>>> +#include "firmware_no_tz.h"
>>>>>> #include "pm_helpers.h"
>>>>>> #include "hfi_venus_io.h"
>>>>>> @@ -86,6 +87,8 @@ static void venus_sys_error_handler(struct
>>>>>> work_struct *work)
>>>>>> struct venus_core *core =
>>>>>> container_of(work, struct venus_core, work.work);
>>>>>> int ret, i, max_attempts = RPM_WAIT_FOR_IDLE_MAX_ATTEMPTS;
>>>>>> + const struct venus_resources *res = core->res;
>>>>>> + const char *fwpath = NULL;
>>>>>> const char *err_msg = "";
>>>>>> bool failed = false;
>>>>>> @@ -107,7 +110,10 @@ static void venus_sys_error_handler(struct
>>>>>> work_struct *work)
>>>>>> mutex_lock(&core->lock);
>>>>>> - venus_shutdown(core);
>>>>>> + if (core->use_tz)
>>>>>> + unload_fw(VENUS_PAS_ID);
>>>>>> + else
>>>>>> + unload_fw_no_tz(core);
>>>>>
>>>>> This is more than introducing helpers.
>>>>>
>>>> The new helpers are written to make the code generic for video drivers.
>>>> which requires changes in the calling function also.
>>>>>> venus_coredump(core);
>>>>>> @@ -127,12 +133,39 @@ static void venus_sys_error_handler(struct
>>>>>> work_struct *work)
>>>>>> failed = true;
>>>>>> }
>>>>>> - ret = venus_boot(core);
>>>>>> + ret = of_property_read_string_index(core->dev->of_node,
>>>>>> "firmware-name", 0,
>>>>>> + &fwpath);
>>>>>> + if (ret)
>>>>>> + fwpath = core->res->fwname;
>>>>>> +
>>>>>> + ret = load_fw(core->dev, fwpath, &core->fw.mem_phys,
>>>>>> &core->fw.mem_size,
>>>>>> + VENUS_PAS_ID, core->use_tz);
>>>>>
>>>>> So, we had a nice local 'venus_boot'. Instead we now have a pile of code
>>>>> with non-generic prefixes, etc. If you are introducing helpers, please
>>>>> refrain from inlining of calling functions, etc. Just move the code to your
>>>>> helpers.
>>>>>
>>>> As mentioned in above comment, the common helpers are written to make the
>>>> code generic. I Will try to make it more clear, working on the same.
>>>
>>> First, you move the code, then you make it generic. Or vice versa.
>>> First you split the code, then you move it. Don't do both in the same
>>> patch.
>>>
>>>>> NAK for the rest of the patch.
>>>
>>>
>
>
>

2023-12-20 21:25:19

[permalink] [raw]

Subject: Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver

Hi Abhinav,

On Wed, 20 Dec 2023 at 20:55, Abhinav Kumar <[email protected]> wrote:
>
> Hi Dmitry and Krzysztof
>
> On 12/20/2023 1:52 AM, Dmitry Baryshkov wrote:
> > On Wed, 20 Dec 2023 at 10:53, Vikash Garodia <[email protected]> wrote:
> >>
> >> On 12/20/2023 2:09 PM, Dmitry Baryshkov wrote:
> >>> On Wed, 20 Dec 2023 at 10:14, Vikash Garodia <[email protected]> wrote:
> >>>>
> >>>> Hi,
> >>>>
> >>>> On 12/20/2023 1:07 PM, Dmitry Baryshkov wrote:
> >>>>> On Wed, 20 Dec 2023 at 08:32, Vikash Garodia <[email protected]> wrote:
> >>>>>>
> >>>>>> Hi Dmitry,
> >>>>>>
> >>>>>> On 12/19/2023 12:08 AM, Dmitry Baryshkov wrote:
> >>>>>>> On 18/12/2023 13:31, Dikshita Agarwal wrote:
> >>>>>>>> This patch series introduces support for Qualcomm new video acceleration
> >>>>>>>> hardware architecture, used for video stream decoding/encoding. This driver
> >>>>>>>> is based on new communication protocol between video hardware and application
> >>>>>>>> processor.
> >>>>>>>
> >>>>>>> This doesn't answer one important point, you have been asked for v1. What is the
> >>>>>>> actual change point between Venus and Iris? What has been changed so much that
> >>>>>>> it demands a separate driver. This is the main question for the cover letter,
> >>>>>>> which has not been answered so far.
> >>>>>>>
> >>>>>>> From what I see from you bindings, the hardware is pretty close to what we see
> >>>>>>> in the latest venus generations. I asssme that there was a change in the vcodec
> >>>>>>> inteface to the firmware and other similar changes. Could you please point out,
> >>>>>>> which parts of Venus driver do no longer work or are not applicable for sm8550
> >>>>>>
> >>>>>> The motivation behind having a separate IRIS driver was discussed earlier in [1]
> >>>>>> In the same discussion, it was ellaborated on how the impact would be with
> >>>>>> change in the new firmware interface and other video layers in the driver. I can
> >>>>>> add this in cover letter in the next revision.
> >>>>>
> >>>>> Ok. So the changes cover the HFI interface. Is that correct?
> >>>> Change wise, yes.
> >>>>
> >>>>>> We see some duplication of code and to handle the same, the series brings in a
> >>>>>> common code reusability between iris and venus. Aligning the common peices of
> >>>>>> venus and iris will be a work in progress, once we land the base driver for iris.
> >>>>>
> >>>>> This is not how it usually works. Especially not with the patches you
> >>>>> have posted.
> >>>>>
> >>>>> I have the following suggestion how this story can continue:
> >>>>> You can _start_ by reworking venus driver, separating the HFI /
> >>>>> firmware / etc interface to an internal interface in the driver. Then
> >>>>> implement Iris as a plug in for that interface. I might be mistaken
> >>>>> here, but I think this is the way how this can be beneficial for both
> >>>>> the video en/decoding on both old and new platforms.
> >>>>
> >>>> HFI/firmware interface is already confined to HFI layer in the existing venus
> >>>> driver. We explained in the previous discussion [1], on how the HFI change
> >>>> impacts the other layers by taking example of a DRC usecase. Please have a look
> >>>> considering the usecase and the impact it brings to other layers in the driver.
> >>>
> >>> I have looked at it. And I still see huge change in the interface
> >>> side, but it doesn't tell me about the hardware changes.
> >>
> >> I hope you noticed how the common layers like decoder, response, state layers
> >> are impacted in handling one of usecase. Now add that to all the different
> >> scenarios like seek, drain, DRC during seek, DRC during drain, etc.
> >
> > Yes, for sure.
> >
> >>
> >>> Have you evaluated the other opportunity?
> >>>
> >>> To have a common platform interface and firmware-specific backend?
> >>>
> >>> You have already done a part of it, but from a different perspective.
> >>> You have tried to move common code out of the driver. Instead we are
> >>> asking you to do a different thing. Move non-common code within the
> >>> driver. Then add your code on top of that.
> >>
> >> For common platform - yes, we are bringing in common stuff like PIL.
> >> Other than that, abstraction to firmware interface is not that confined to one
> >> layer. It spreads over decoder/encoder/common layers. Now when majority of the
> >> layers/code is different, we planned to make it in parallel to venus and have a
> >> common layer having common things to both iris and venus.
> >
> > My suggestion still holds. Start with this common platform code.
> > Rather than arguing back and forth, could you please perform an
> > experiment on the current venus driver and move firmware interface to
> > subdirs, leaving just the platform / PIL / formats / etc in place?
> > This will at least allow us to determine whether it is a feasible
> > concept or not.
> >
>
> Your comments are valid. The platform driver piece and some other pieces
> still are common between venus and iris despite this initial effort of
> moving common pieces out. I have also seen this from whatever I have
> reviewed.
>
> Video team also acknowledges this part and internally I think they did
> evaluate this option and their feedback was, the more and more they
> changed, they were touching pretty much every file of venus.

This is fine from my POV. Splitting the the common functionality also
touched a significant part of the venus driver in a pretty bad way.

> The missing piece i think in all this discussion is that in addition to
> the forward channel, the reverse channel of HFI, based on which the rest
> of the video state machine changes should also be considered.
>
> So even with respect to the code layout, its not just the forward
> communication but the backwards communication of fw--->hfi--->codec is
> becoming a challenge as the venus layers seem to only work with the hfi
> of venus.

Again, this is a question of the platform backend part. Nobody
questions that newer platforms have their own driver interface.
We are not asking your team to add if(new) { foo; } else { bar; }
conditions all over the code. Instead we have been asking to split all
platform specifics to the 'backend'. Compare this with mdp4, mdp5 and
dpu backends of the single drm/msm frontend. Or with a similar
approach found in other DRM drivers. Adding new driver is frequently
unjustified, instead the existing driver gets extended to support
different generations.

> For adding support for the new HFI events/communication, it was getting
> harder to extend venus.
>
> What I certainly acknowledge is now with iris whether this can be dealt
> with better for future chipsets to avoid another re-write for another
> HFI as that will be too much of maintenance cost.

Another approach, which might also be fine, if that looks better from
your point of view. The current venus driver might have issues with
the internal interfaces. Or it might be not suitable for the backends.
In this case, can we please get older platforms also supported by the
iris driver? This will be a very simple case then: the old deprecated
driver, being phased out and removed, and a new one, which is being
phased in.

The problem is very simple from my side. I do not want to end up in a
situation where we have to change platform code for both drivers if
there is any common issue. Neither would I like for venus and iris
bindings to diverge more than necessary. And as you have seen from my
comments, the iris bindings already do not follow the venus example.

> I will let the video team comment on this part.
>
> Thanks
>
> Abhinav
>
>
>
>
>
> >>
> >>>>
> >>>> [1] https://lore.kernel.org/lkml/[email protected]/
> >>>>> Short rationale:
> >>>>> The venus driver has a history of supported platforms. There is
> >>>>> already some kind of buffer management in place. Both developers and
> >>>>> testers have spent their effort on finding issues there. Sending new
> >>>>> driver means that we have to spend the same amount of efforts on this.
> >>>>> Moreover, even from the porter point of view. You are creating new
> >>>>> bindings for the new hardware. Which do not follow the
> >>>>> venus-common.yaml. And they do not follow the defined bindings for the
> >>>>> recent venus platforms. Which means that as a developer I have to care
> >>>>> about two different ways to describe nearly the same hardware.>> Again qualcomm video team does not have a plan to support sm8550/x1e80100 on
> >>>>>> venus as the changes are too interleaved to absorb in venus driver. And there is
> >>>>>> significant interest in community to start validating video driver on sm8550 or
> >>>>>> x1e80100.
> >>>>>>
> >>>>>> [1] https://lore.kernel.org/lkml/[email protected]/
> >>>>>>
> >>>>>> Regards,
> >>>>>> Vikash

--
With best wishes
Dmitry

2024-02-29 15:09:53

[permalink] [raw]

Subject: Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver

Hello All,

On 12/18/2023 5:01 PM, Dikshita Agarwal wrote:
> This patch series introduces support for Qualcomm new video acceleration
> hardware architecture, used for video stream decoding/encoding. This driver
> is based on new communication protocol between video hardware and application
> processor.
> This driver comes with below capabilities:
> - V4L2 complaint video driver with M2M and STREAMING capability.
> - Supports H264, H265, VP9 decoders.
> - Supports H264, H265 encoders.
> This driver comes with below features:
> - Centralized resource and memory management.
> - Centralized management of core and instance states.
> - Defines platform specific capabilities and features. As a results, it provides
> a single point of control to enable/disable a given feature depending on
> specific platform capabilities.
> - Handles hardware interdependent configurations. For a given configuration from
> client, the driver checks for hardware dependent configuration/s and updates
> the same.
> - Handles multiple complex video scenarios involving state transitions - DRC,
> Drain, Seek, back to back DRC, DRC during Drain sequence, DRC during Seek
> sequence.
> - Introduces a flexible way for driver to subscribe for any property with
> hardware. Hardware would inform driver with those subscribed property with any
> change in value.
> - Introduces performance (clock and bus) model based on new hardware
> architecture.
> - Introduces multi thread safe design to handle communication between client and
> hardware.
> - Adapts encoder quality improvements available in new hardware architecture.
> - Implements asynchronous communication with hardware to achieve better
> experience in low latency usecases.
> - Supports multi stage hardware architecture for encode/decode.
> - Output and capture planes are controlled independently. Thereby providing a
> way to reconfigure individual plane.
> - Hardware packetization layer supports synchronization between configuration
> packet and data packet.
> - Introduces a flexibility to receive a hardware response for a given command
> packet.
> - Native hardware support of LAST flag which is mandatory to align with port
> reconfiguration and DRAIN sequence as per V4L guidelines.
> - Native hardware support for drain sequence.

1. We considered enhancing the venus driver to support iris functionality but
concluded that the result would essentially be two parallel drivers implemented
in the same folder.
2. Although the underlying hardware for venus and iris are quite similar, but
there are other factors which brings the need of new driver:
a. the host firmware interface (HFI) changed between the two. Venus supports
HFI protocol 1.0 while iris supports HFI protocol 2.0.
b. unlike HFI protocol 1.0, HFI protocol 2.0 is better aligned to V4L2 APIs.
c. iris driver modularize platforms, hardware variants, and states to extend
it for any upcoming SOC. Thereby more extendable to newer SOCs in future.
d. Iris also supports many advanced features.
3. Based on the comments received on posted iris driver [1], we evaluated it
further and to better align with the upstream standard and practices, we
acknowledge that even though iris driver is the way forward, it would be ideal
to bring in the Venus hardwares enabled in this driver.
4. Hence, we decided to rework iris driver to add support of HFI protocol 1.0 as
well.
5. Initially we would start with adding support for one HFI protocol 1.0 based
platform which would be SM8250.
6. SM8250 enablement on iris driver would be incremental, starting with basic
decode for H264 codec.
7. In long-term, all venus supported platforms would be migrated to iris.
8. Iris driver and venus driver will co-exist till remaining supported targets
also gets migrated to iris driver.
9. We would continue to support and maintain venus driver during this phased out
approach.

Please share your thoughts on the above proposal.

[1]
https://patchwork.kernel.org/project/linux-media/cover/[email protected]/#25643473

Regards,
Vikash

2024-03-12 10:38:17

by Hans Verkuil

[permalink] [raw]

Subject: Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver

Hi Vikash,

On 2/29/24 16:09, Vikash Garodia wrote:
> Hello All,
>
> On 12/18/2023 5:01 PM, Dikshita Agarwal wrote:
>> This patch series introduces support for Qualcomm new video acceleration
>> hardware architecture, used for video stream decoding/encoding. This driver
>> is based on new communication protocol between video hardware and application
>> processor.
>> This driver comes with below capabilities:
>> - V4L2 complaint video driver with M2M and STREAMING capability.
>> - Supports H264, H265, VP9 decoders.
>> - Supports H264, H265 encoders.
>> This driver comes with below features:
>> - Centralized resource and memory management.
>> - Centralized management of core and instance states.
>> - Defines platform specific capabilities and features. As a results, it provides
>> a single point of control to enable/disable a given feature depending on
>> specific platform capabilities.
>> - Handles hardware interdependent configurations. For a given configuration from
>> client, the driver checks for hardware dependent configuration/s and updates
>> the same.
>> - Handles multiple complex video scenarios involving state transitions - DRC,
>> Drain, Seek, back to back DRC, DRC during Drain sequence, DRC during Seek
>> sequence.
>> - Introduces a flexible way for driver to subscribe for any property with
>> hardware. Hardware would inform driver with those subscribed property with any
>> change in value.
>> - Introduces performance (clock and bus) model based on new hardware
>> architecture.
>> - Introduces multi thread safe design to handle communication between client and
>> hardware.
>> - Adapts encoder quality improvements available in new hardware architecture.
>> - Implements asynchronous communication with hardware to achieve better
>> experience in low latency usecases.
>> - Supports multi stage hardware architecture for encode/decode.
>> - Output and capture planes are controlled independently. Thereby providing a
>> way to reconfigure individual plane.
>> - Hardware packetization layer supports synchronization between configuration
>> packet and data packet.
>> - Introduces a flexibility to receive a hardware response for a given command
>> packet.
>> - Native hardware support of LAST flag which is mandatory to align with port
>> reconfiguration and DRAIN sequence as per V4L guidelines.
>> - Native hardware support for drain sequence.
>
> 1. We considered enhancing the venus driver to support iris functionality but
> concluded that the result would essentially be two parallel drivers implemented
> in the same folder.
> 2. Although the underlying hardware for venus and iris are quite similar, but
> there are other factors which brings the need of new driver:
> a. the host firmware interface (HFI) changed between the two. Venus supports
> HFI protocol 1.0 while iris supports HFI protocol 2.0.
> b. unlike HFI protocol 1.0, HFI protocol 2.0 is better aligned to V4L2 APIs.
> c. iris driver modularize platforms, hardware variants, and states to extend
> it for any upcoming SOC. Thereby more extendable to newer SOCs in future.
> d. Iris also supports many advanced features.
> 3. Based on the comments received on posted iris driver [1], we evaluated it
> further and to better align with the upstream standard and practices, we
> acknowledge that even though iris driver is the way forward, it would be ideal
> to bring in the Venus hardwares enabled in this driver.
> 4. Hence, we decided to rework iris driver to add support of HFI protocol 1.0 as
> well.
> 5. Initially we would start with adding support for one HFI protocol 1.0 based
> platform which would be SM8250.
> 6. SM8250 enablement on iris driver would be incremental, starting with basic
> decode for H264 codec.
> 7. In long-term, all venus supported platforms would be migrated to iris.
> 8. Iris driver and venus driver will co-exist till remaining supported targets
> also gets migrated to iris driver.
> 9. We would continue to support and maintain venus driver during this phased out
> approach.
>
> Please share your thoughts on the above proposal.

I think this is a reasonable approach: the venus driver is quite old and it was
created when we were at more-or-less the same time also developing better codec
frameworks. So it is not quite up-to-date with all the latest requirements.

Starting with a clean slate for the iris driver and then add support for venus
platforms to the iris driver makes sense.

Just one serious concern: who will do the venus platform migration? Are there resources
available to do that work? Or is this just wishful thinking?

Regards,

Hans

>
> [1]
> https://patchwork.kernel.org/project/linux-media/cover/[email protected]/#25643473
>
> Regards,
> Vikash
>

2024-03-15 13:51:33

[permalink] [raw]

Subject: Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver

Hi Hans,

On 3/12/2024 4:07 PM, Hans Verkuil wrote:
> Hi Vikash,
>
> On 2/29/24 16:09, Vikash Garodia wrote:
>> Hello All,
>>
>> On 12/18/2023 5:01 PM, Dikshita Agarwal wrote:
>>> This patch series introduces support for Qualcomm new video acceleration
>>> hardware architecture, used for video stream decoding/encoding. This driver
>>> is based on new communication protocol between video hardware and application
>>> processor.
>>> This driver comes with below capabilities:
>>> - V4L2 complaint video driver with M2M and STREAMING capability.
>>> - Supports H264, H265, VP9 decoders.
>>> - Supports H264, H265 encoders.
>>> This driver comes with below features:
>>> - Centralized resource and memory management.
>>> - Centralized management of core and instance states.
>>> - Defines platform specific capabilities and features. As a results, it provides
>>> a single point of control to enable/disable a given feature depending on
>>> specific platform capabilities.
>>> - Handles hardware interdependent configurations. For a given configuration from
>>> client, the driver checks for hardware dependent configuration/s and updates
>>> the same.
>>> - Handles multiple complex video scenarios involving state transitions - DRC,
>>> Drain, Seek, back to back DRC, DRC during Drain sequence, DRC during Seek
>>> sequence.
>>> - Introduces a flexible way for driver to subscribe for any property with
>>> hardware. Hardware would inform driver with those subscribed property with any
>>> change in value.
>>> - Introduces performance (clock and bus) model based on new hardware
>>> architecture.
>>> - Introduces multi thread safe design to handle communication between client and
>>> hardware.
>>> - Adapts encoder quality improvements available in new hardware architecture.
>>> - Implements asynchronous communication with hardware to achieve better
>>> experience in low latency usecases.
>>> - Supports multi stage hardware architecture for encode/decode.
>>> - Output and capture planes are controlled independently. Thereby providing a
>>> way to reconfigure individual plane.
>>> - Hardware packetization layer supports synchronization between configuration
>>> packet and data packet.
>>> - Introduces a flexibility to receive a hardware response for a given command
>>> packet.
>>> - Native hardware support of LAST flag which is mandatory to align with port
>>> reconfiguration and DRAIN sequence as per V4L guidelines.
>>> - Native hardware support for drain sequence.
>>
>> 1. We considered enhancing the venus driver to support iris functionality but
>> concluded that the result would essentially be two parallel drivers implemented
>> in the same folder.
>> 2. Although the underlying hardware for venus and iris are quite similar, but
>> there are other factors which brings the need of new driver:
>> a. the host firmware interface (HFI) changed between the two. Venus supports
>> HFI protocol 1.0 while iris supports HFI protocol 2.0.
>> b. unlike HFI protocol 1.0, HFI protocol 2.0 is better aligned to V4L2 APIs.
>> c. iris driver modularize platforms, hardware variants, and states to extend
>> it for any upcoming SOC. Thereby more extendable to newer SOCs in future.
>> d. Iris also supports many advanced features.
>> 3. Based on the comments received on posted iris driver [1], we evaluated it
>> further and to better align with the upstream standard and practices, we
>> acknowledge that even though iris driver is the way forward, it would be ideal
>> to bring in the Venus hardwares enabled in this driver.
>> 4. Hence, we decided to rework iris driver to add support of HFI protocol 1.0 as
>> well.
>> 5. Initially we would start with adding support for one HFI protocol 1.0 based
>> platform which would be SM8250.
>> 6. SM8250 enablement on iris driver would be incremental, starting with basic
>> decode for H264 codec.
>> 7. In long-term, all venus supported platforms would be migrated to iris.
>> 8. Iris driver and venus driver will co-exist till remaining supported targets
>> also gets migrated to iris driver.
>> 9. We would continue to support and maintain venus driver during this phased out
>> approach.
>>
>> Please share your thoughts on the above proposal.
>
> I think this is a reasonable approach: the venus driver is quite old and it was
> created when we were at more-or-less the same time also developing better codec
> frameworks. So it is not quite up-to-date with all the latest requirements.
>
> Starting with a clean slate for the iris driver and then add support for venus
> platforms to the iris driver makes sense.
Appreciate your time to review the proposal.
>
> Just one serious concern: who will do the venus platform migration? Are there resources
> available to do that work? Or is this just wishful thinking?
To initiate the venus platform migration, iris driver would be enabled with all
SOCs under HFI_V6XX which are currently supported in venus driver. In doing so,
iris driver, posted earlier [1] is being reworked to have cleaner abstraction to
invoke HFI 1.0 protocol (venus variants) and would be applicable for all venus
platforms. With this, remaining platform migration would be done with some
incremental efforts (platform specific handling).
We would like to get help from community and co-work in enabling the migration
for remaining venus platforms in a phased out approach.

Regards,
Vikash

[1]
https://patchwork.kernel.org/project/linux-media/cover/[email protected]/#25643473

2024-04-12 07:34:59

by Hyunjun Ko

[permalink] [raw]

Subject: Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver

Hi,

I'm trying this series of patches for enabling /dev/video0 on sm8650 hdk but failed.
After modprobing, lsmod says just like the follwoing:

root@hdk8650:/lib/modules/6.7.0-rc3+# lsmod
Module Size Used by
iris 110592 -2
v4l2_mem2mem 20480 -2
videobuf2_memops 12288 -2
videobuf2_v4l2 20480 -2
videobuf2_common 45056 -2
videodev 176128 -2

This series looks for sm8550 device though, my question is that this series have been tested on the device (sm8650 hdk)? or do you expect this should work on it?

2024-04-12 13:53:13

[permalink] [raw]

Subject: Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver

On 12/04/2024 08:13, Hyunjun Ko wrote:
> Hi,
>
> I'm trying this series of patches for enabling /dev/video0 on sm8650 hdk but failed.
> After modprobing, lsmod says just like the follwoing:
>
> root@hdk8650:/lib/modules/6.7.0-rc3+# lsmod
> Module Size Used by
> iris 110592 -2
> v4l2_mem2mem 20480 -2
> videobuf2_memops 12288 -2
> videobuf2_v4l2 20480 -2
> videobuf2_common 45056 -2
> videodev 176128 -2
>
>
> This series looks for sm8550 device though, my question is that this series have been tested on the device (sm8650 hdk)? or do you expect this should work on it?
>

Different SoCs so despite how close they look in version numbers, you'd
expect at a minimum some clock and/or power-domain churn - even if there
is a 1:1 mapping in register numbers and offsets.

We wouldn't ordinarily expect to be able to be able to move SoC versions
so easily - 8550 and 8650 have similar SoC version numbers but, this is
not necessarily an indicator of silicon IP block version reuse.

---
bod

2024-05-16 07:59:14

by Hyunjun Ko

[permalink] [raw]

Subject: Re: [PATCH v2 00/34] Qualcomm video encoder and decoder driver

Hi,

Thanks for this series of patches. I successfully adjusted these patches and tried to test video features with gstreamer or ffmpeg.
But I found this provides staetful interfaces while I need stateless, which might cause an issue for my side..

My question is do you have any plan to implement stateless interfaces or already you have somewhere?

2024-05-16 08:10:49