Currently, the VPU in the i.MQ8MQ is appearing as one codec, but in reality, it's
two IP blocks called G1 and G2. There is initialization code in VPU code to
pull some clocks, resets and other features which has been integrated into
the vpu-blk-ctrl for the i.MX8M Mini and a similar method can be used to make
the VPU codec's operate as stand-alone cores without having to know the details
of each other or the quirks unique to the i.MX8MQ, so the remaining code can be
left more generic.
This series was started by Lucas Stach, and picked up by me so some patches have
his s-o-b and mine where I might have changed a few minor items. It's in an RFC state
because the G2 VP9 operations appear to hang, but the parent code from which I started doesn't
appear to show VP9 support, and it looks like it should.
since the g-streamer and media trees are in a constant state of change, this series is based on
git://linuxtv.org/hverkuil/media_tree.git for-v5.17e
Adam Ford (2):
media: hantro: split i.MX8MQ G1 and G2 code
arm64: dts: imx8mq: Split i.MX8MQ G1 and G2 with vpu-blk-ctrl
Lucas Stach (3):
dt-bindings: power: imx8mq: add defines for VPU blk-ctrl domains
dt-bindings: soc: add binding for i.MX8MQ VPU blk-ctrl
soc: imx: imx8m-blk-ctrl: add i.MX8MQ VPU blk-ctrl
.../soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml | 71 +++++++++++
arch/arm64/boot/dts/freescale/imx8mq.dtsi | 69 ++++++----
drivers/soc/imx/imx8m-blk-ctrl.c | 67 ++++++++++
drivers/staging/media/hantro/hantro_drv.c | 4 +-
drivers/staging/media/hantro/hantro_hw.h | 2 +-
drivers/staging/media/hantro/imx8m_vpu_hw.c | 119 +++---------------
include/dt-bindings/power/imx8mq-power.h | 3 +
7 files changed, 205 insertions(+), 130 deletions(-)
create mode 100644 Documentation/devicetree/bindings/soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml
--
2.32.0
From: Lucas Stach <[email protected]>
This adds the defines for the power domains provided by the VPU
blk-ctrl on the i.MX8MQ.
Signed-off-by: Lucas Stach <[email protected]>
---
include/dt-bindings/power/imx8mq-power.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/include/dt-bindings/power/imx8mq-power.h b/include/dt-bindings/power/imx8mq-power.h
index 8a513bd9166e..9f7d0f1e7c32 100644
--- a/include/dt-bindings/power/imx8mq-power.h
+++ b/include/dt-bindings/power/imx8mq-power.h
@@ -18,4 +18,7 @@
#define IMX8M_POWER_DOMAIN_MIPI_CSI2 9
#define IMX8M_POWER_DOMAIN_PCIE2 10
+#define IMX8MQ_VPUBLK_PD_G1 0
+#define IMX8MQ_VPUBLK_PD_G2 1
+
#endif
--
2.32.0
From: Lucas Stach <[email protected]>
This adds the DT binding for the i.MX8MQ VPU blk-ctrl.
Signed-off-by: Lucas Stach <[email protected]>
Signed-off-by: Adam Ford <[email protected]>
---
.../soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml | 71 +++++++++++++++++++
1 file changed, 71 insertions(+)
create mode 100644 Documentation/devicetree/bindings/soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml
diff --git a/Documentation/devicetree/bindings/soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml b/Documentation/devicetree/bindings/soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml
new file mode 100644
index 000000000000..b8cfad0d7a1e
--- /dev/null
+++ b/Documentation/devicetree/bindings/soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml
@@ -0,0 +1,71 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: NXP i.MX8MQ VPU blk-ctrl
+
+maintainers:
+ - Lucas Stach <[email protected]>
+
+description:
+ The i.MX8MQ VPU blk-ctrl is a top-level peripheral providing access to
+ the NoC and ensuring proper power sequencing of the VPU peripherals
+ located in the VPU domain of the SoC.
+
+properties:
+ compatible:
+ items:
+ - const: fsl,imx8mq-vpu-blk-ctrl
+
+ reg:
+ maxItems: 1
+
+ '#power-domain-cells':
+ const: 1
+
+ power-domains:
+ minItems: 3
+ maxItems: 3
+
+ power-domain-names:
+ items:
+ - const: bus
+ - const: g1
+ - const: g2
+
+ clocks:
+ minItems: 2
+ maxItems: 2
+
+ clock-names:
+ items:
+ - const: g1
+ - const: g2
+
+required:
+ - compatible
+ - reg
+ - power-domains
+ - power-domain-names
+ - clocks
+ - clock-names
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/imx8mq-clock.h>
+ #include <dt-bindings/power/imx8mq-power.h>
+
+ vpu_blk_ctrl: blk-ctrl@38320000 {
+ compatible = "fsl,imx8mq-vpu-blk-ctrl", "syscon";
+ reg = <0x38320000 0x100>;
+ power-domains = <&pgc_vpu>, <&pgc_vpu>, <&pgc_vpu>;
+ power-domain-names = "bus", "g1", "g2";
+ clocks = <&clk IMX8MQ_CLK_VPU_G1_ROOT>,
+ <&clk IMX8MQ_CLK_VPU_G2_ROOT>;
+ clock-names = "g1", "g2";
+ #power-domain-cells = <1>;
+ };
--
2.32.0
From: Lucas Stach <[email protected]>
This adds the necessary bits to drive the VPU blk-ctrl on the i.MX8MQ, to
avoid putting more of this functionality into the decoder driver.
Signed-off-by: Lucas Stach <[email protected]>
---
drivers/soc/imx/imx8m-blk-ctrl.c | 67 ++++++++++++++++++++++++++++++++
1 file changed, 67 insertions(+)
diff --git a/drivers/soc/imx/imx8m-blk-ctrl.c b/drivers/soc/imx/imx8m-blk-ctrl.c
index 519b3651d1d9..66c879bcc66b 100644
--- a/drivers/soc/imx/imx8m-blk-ctrl.c
+++ b/drivers/soc/imx/imx8m-blk-ctrl.c
@@ -14,6 +14,7 @@
#include <linux/clk.h>
#include <dt-bindings/power/imx8mm-power.h>
+#include <dt-bindings/power/imx8mq-power.h>
#define BLK_SFT_RSTN 0x0
#define BLK_CLK_EN 0x4
@@ -498,6 +499,69 @@ static const struct imx8m_blk_ctrl_data imx8mm_disp_blk_ctl_dev_data = {
.num_domains = ARRAY_SIZE(imx8mm_disp_blk_ctl_domain_data),
};
+static int imx8mq_vpu_power_notifier(struct notifier_block *nb,
+ unsigned long action, void *data)
+{
+ struct imx8m_blk_ctrl *bc = container_of(nb, struct imx8m_blk_ctrl,
+ power_nb);
+
+ if (action != GENPD_NOTIFY_ON && action != GENPD_NOTIFY_PRE_OFF)
+ return NOTIFY_OK;
+
+ /*
+ * The ADB in the VPUMIX domain has no separate reset and clock
+ * enable bits, but is ungated and reset together with the VPUs. The
+ * reset and clock enable inputs to the ADB is a logical OR of the
+ * VPU bits. To avoid putting a improperly reset decoder on the bus
+ * while making the ADB work, we release G2 from reset, but only
+ * enable clocks to G1.
+ */
+ regmap_update_bits(bc->regmap, BLK_SFT_RSTN, BIT(0) | BIT(1), BIT(0));
+ regmap_set_bits(bc->regmap, BLK_CLK_EN, BIT(1));
+
+ if (action == GENPD_NOTIFY_ON) {
+ /*
+ * On power up we have no software backchannel to the GPC to
+ * wait for the ADB handshake to happen, so we just delay for a
+ * bit. On power down the GPC driver waits for the handshake.
+ */
+ udelay(5);
+
+ /* set "fuse" bits to enable the VPUs */
+ regmap_set_bits(bc->regmap, 0x8, 0xffffffff);
+ regmap_set_bits(bc->regmap, 0xc, 0xffffffff);
+ regmap_set_bits(bc->regmap, 0x10, 0xffffffff);
+ }
+
+ return NOTIFY_OK;
+}
+
+static const struct imx8m_blk_ctrl_domain_data imx8mq_vpu_blk_ctl_domain_data[] = {
+ [IMX8MQ_VPUBLK_PD_G1] = {
+ .name = "vpublk-g1",
+ .clk_names = (const char *[]){ "g1", },
+ .num_clks = 1,
+ .gpc_name = "g1",
+ .rst_mask = BIT(1),
+ .clk_mask = BIT(1),
+ },
+ [IMX8MQ_VPUBLK_PD_G2] = {
+ .name = "vpublk-g2",
+ .clk_names = (const char *[]){ "g2", },
+ .num_clks = 1,
+ .gpc_name = "g2",
+ .rst_mask = BIT(0),
+ .clk_mask = BIT(0),
+ },
+};
+
+static const struct imx8m_blk_ctrl_data imx8mq_vpu_blk_ctl_dev_data = {
+ .max_reg = 0x14,
+ .power_notifier_fn = imx8mq_vpu_power_notifier,
+ .domains = imx8mq_vpu_blk_ctl_domain_data,
+ .num_domains = ARRAY_SIZE(imx8mq_vpu_blk_ctl_domain_data),
+};
+
static const struct of_device_id imx8m_blk_ctrl_of_match[] = {
{
.compatible = "fsl,imx8mm-vpu-blk-ctrl",
@@ -505,6 +569,9 @@ static const struct of_device_id imx8m_blk_ctrl_of_match[] = {
}, {
.compatible = "fsl,imx8mm-disp-blk-ctrl",
.data = &imx8mm_disp_blk_ctl_dev_data
+ } ,{
+ .compatible = "fsl,imx8mq-vpu-blk-ctrl",
+ .data = &imx8mq_vpu_blk_ctl_dev_data
} ,{
/* Sentinel */
}
--
2.32.0
With the Hantro G1 and G2 now setup to run independently, update
the device tree to enable it again. This requires the vpu-blk-ctrl
node to be configured, and the clock-parents and clock-rates for
the various VPU's to be moved into the pgc_vpu because they cannot
get re-parented once enabled, and the pgc_vpu is the highest
in the chain.
Signed-off-by: Adam Ford <[email protected]>
---
arch/arm64/boot/dts/freescale/imx8mq.dtsi | 69 +++++++++++++++--------
1 file changed, 45 insertions(+), 24 deletions(-)
This clock rates and parents were moved to the pgc_vpu node because the vpu-blk-ctrl is
expecting certain clocks to be operational, and they cannot be re-parented once they are
runing.
The datasheet and the TRM both show different potential clock rates for the G2 CODEC.
It could be as low as 300MHZ or as high as 650 depending on what document is read, and
the operating voltage of the hardware. Because of the uncertainty, the clock rates
proposed here match those in NXP's 5.10 Release.
https://source.codeaurora.org/external/imx/linux-imx/tree/arch/arm64/boot/dts/freescale/imx8mq.dtsi?h=lf-5.10.y
diff --git a/arch/arm64/boot/dts/freescale/imx8mq.dtsi b/arch/arm64/boot/dts/freescale/imx8mq.dtsi
index 972766b67a15..0d3053bbfb2c 100644
--- a/arch/arm64/boot/dts/freescale/imx8mq.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mq.dtsi
@@ -711,7 +711,21 @@ pgc_gpu: power-domain@5 {
pgc_vpu: power-domain@6 {
#power-domain-cells = <0>;
reg = <IMX8M_POWER_DOMAIN_VPU>;
- clocks = <&clk IMX8MQ_CLK_VPU_DEC_ROOT>;
+ clocks = <&clk IMX8MQ_CLK_VPU_DEC_ROOT>,
+ <&clk IMX8MQ_CLK_VPU_G1_ROOT>,
+ <&clk IMX8MQ_CLK_VPU_G2_ROOT>;
+ assigned-clocks = <&clk IMX8MQ_CLK_VPU_G1>,
+ <&clk IMX8MQ_CLK_VPU_G2>,
+ <&clk IMX8MQ_CLK_VPU_BUS>,
+ <&clk IMX8MQ_VPU_PLL_BYPASS>;
+ assigned-clock-parents = <&clk IMX8MQ_VPU_PLL_OUT>,
+ <&clk IMX8MQ_VPU_PLL_OUT>,
+ <&clk IMX8MQ_SYS1_PLL_800M>,
+ <&clk IMX8MQ_VPU_PLL>;
+ assigned-clock-rates = <600000000>,
+ <600000000>,
+ <800000000>,
+ <0>;
};
pgc_disp: power-domain@7 {
@@ -1432,30 +1446,37 @@ usb3_phy1: usb-phy@382f0040 {
status = "disabled";
};
- vpu: video-codec@38300000 {
- compatible = "nxp,imx8mq-vpu";
- reg = <0x38300000 0x10000>,
- <0x38310000 0x10000>,
- <0x38320000 0x10000>;
- reg-names = "g1", "g2", "ctrl";
- interrupts = <GIC_SPI 7 IRQ_TYPE_LEVEL_HIGH>,
- <GIC_SPI 8 IRQ_TYPE_LEVEL_HIGH>;
- interrupt-names = "g1", "g2";
+ vpu_g1: video-codec@38300000 {
+ compatible = "nxp,imx8mq-vpu-g1";
+ reg = <0x38300000 0x10000>;
+ reg-names = "g1";
+ interrupts = <GIC_SPI 7 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "g1";
+ clocks = <&clk IMX8MQ_CLK_VPU_G1_ROOT>;
+ clock-names = "g1";
+ power-domains = <&vpu_blk_ctrl IMX8MQ_VPUBLK_PD_G1>;
+ };
+
+ vpu_g2: video-codec@38310000 {
+ compatible = "nxp,imx8mq-vpu-g2";
+ reg = <0x38310000 0x10000>;
+ reg-names = "g2";
+ interrupts = <GIC_SPI 8 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "g2";
+ clocks = <&clk IMX8MQ_CLK_VPU_G2_ROOT>;
+ clock-names = "g2";
+ power-domains = <&vpu_blk_ctrl IMX8MQ_VPUBLK_PD_G2>;
+ };
+
+ vpu_blk_ctrl: blk-ctrl@38320000 {
+ compatible = "fsl,imx8mq-vpu-blk-ctrl", "syscon";
+ reg = <0x38320000 0x100>;
+ power-domains = <&pgc_vpu>, <&pgc_vpu>, <&pgc_vpu>;
+ power-domain-names = "bus", "g1", "g2";
clocks = <&clk IMX8MQ_CLK_VPU_G1_ROOT>,
- <&clk IMX8MQ_CLK_VPU_G2_ROOT>,
- <&clk IMX8MQ_CLK_VPU_DEC_ROOT>;
- clock-names = "g1", "g2", "bus";
- assigned-clocks = <&clk IMX8MQ_CLK_VPU_G1>,
- <&clk IMX8MQ_CLK_VPU_G2>,
- <&clk IMX8MQ_CLK_VPU_BUS>,
- <&clk IMX8MQ_VPU_PLL_BYPASS>;
- assigned-clock-parents = <&clk IMX8MQ_VPU_PLL_OUT>,
- <&clk IMX8MQ_VPU_PLL_OUT>,
- <&clk IMX8MQ_SYS1_PLL_800M>,
- <&clk IMX8MQ_VPU_PLL>;
- assigned-clock-rates = <600000000>, <600000000>,
- <800000000>, <0>;
- power-domains = <&pgc_vpu>;
+ <&clk IMX8MQ_CLK_VPU_G2_ROOT>;
+ clock-names = "g1", "g2";
+ #power-domain-cells = <1>;
};
pcie0: pcie@33800000 {
--
2.32.0
The VPU in the i.MX8MQ is really the combination of Hantro G1 and
Hantro G2. With the updated vpu-blk-ctrl, the power domains system
can enable and disable them separately as well as pull them out of
reset. This simplifies the code and lets them run independently.
Signed-off-by: Adam Ford <[email protected]>
---
drivers/staging/media/hantro/hantro_drv.c | 4 +-
drivers/staging/media/hantro/hantro_hw.h | 2 +-
drivers/staging/media/hantro/imx8m_vpu_hw.c | 119 +++-----------------
3 files changed, 19 insertions(+), 106 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
index ab2467998d29..d803252a5aba 100644
--- a/drivers/staging/media/hantro/hantro_drv.c
+++ b/drivers/staging/media/hantro/hantro_drv.c
@@ -608,8 +608,8 @@ static const struct of_device_id of_hantro_match[] = {
{ .compatible = "rockchip,rk3399-vpu", .data = &rk3399_vpu_variant, },
#endif
#ifdef CONFIG_VIDEO_HANTRO_IMX8M
- { .compatible = "nxp,imx8mq-vpu", .data = &imx8mq_vpu_variant, },
- { .compatible = "nxp,imx8mq-vpu-g2", .data = &imx8mq_vpu_g2_variant },
+ { .compatible = "nxp,imx8mq-vpu-g1", .data = &imx8mq_vpu_g1_variant, },
+ { .compatible = "nxp,imx8mq-vpu-g2", .data = &imx8mq_vpu_g2_variant, },
#endif
#ifdef CONFIG_VIDEO_HANTRO_SAMA5D4
{ .compatible = "microchip,sama5d4-vdec", .data = &sama5d4_vdec_variant, },
diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
index cff817ca8d22..122b83a16663 100644
--- a/drivers/staging/media/hantro/hantro_hw.h
+++ b/drivers/staging/media/hantro/hantro_hw.h
@@ -299,8 +299,8 @@ enum hantro_enc_fmt {
ROCKCHIP_VPU_ENC_FMT_UYVY422 = 3,
};
+extern const struct hantro_variant imx8mq_vpu_g1_variant;
extern const struct hantro_variant imx8mq_vpu_g2_variant;
-extern const struct hantro_variant imx8mq_vpu_variant;
extern const struct hantro_variant px30_vpu_variant;
extern const struct hantro_variant rk3036_vpu_variant;
extern const struct hantro_variant rk3066_vpu_variant;
diff --git a/drivers/staging/media/hantro/imx8m_vpu_hw.c b/drivers/staging/media/hantro/imx8m_vpu_hw.c
index 1a43f6fceef9..c9f6e8472258 100644
--- a/drivers/staging/media/hantro/imx8m_vpu_hw.c
+++ b/drivers/staging/media/hantro/imx8m_vpu_hw.c
@@ -13,67 +13,6 @@
#include "hantro_g1_regs.h"
#include "hantro_g2_regs.h"
-#define CTRL_SOFT_RESET 0x00
-#define RESET_G1 BIT(1)
-#define RESET_G2 BIT(0)
-
-#define CTRL_CLOCK_ENABLE 0x04
-#define CLOCK_G1 BIT(1)
-#define CLOCK_G2 BIT(0)
-
-#define CTRL_G1_DEC_FUSE 0x08
-#define CTRL_G1_PP_FUSE 0x0c
-#define CTRL_G2_DEC_FUSE 0x10
-
-static void imx8m_soft_reset(struct hantro_dev *vpu, u32 reset_bits)
-{
- u32 val;
-
- /* Assert */
- val = readl(vpu->ctrl_base + CTRL_SOFT_RESET);
- val &= ~reset_bits;
- writel(val, vpu->ctrl_base + CTRL_SOFT_RESET);
-
- udelay(2);
-
- /* Release */
- val = readl(vpu->ctrl_base + CTRL_SOFT_RESET);
- val |= reset_bits;
- writel(val, vpu->ctrl_base + CTRL_SOFT_RESET);
-}
-
-static void imx8m_clk_enable(struct hantro_dev *vpu, u32 clock_bits)
-{
- u32 val;
-
- val = readl(vpu->ctrl_base + CTRL_CLOCK_ENABLE);
- val |= clock_bits;
- writel(val, vpu->ctrl_base + CTRL_CLOCK_ENABLE);
-}
-
-static int imx8mq_runtime_resume(struct hantro_dev *vpu)
-{
- int ret;
-
- ret = clk_bulk_prepare_enable(vpu->variant->num_clocks, vpu->clocks);
- if (ret) {
- dev_err(vpu->dev, "Failed to enable clocks\n");
- return ret;
- }
-
- imx8m_soft_reset(vpu, RESET_G1 | RESET_G2);
- imx8m_clk_enable(vpu, CLOCK_G1 | CLOCK_G2);
-
- /* Set values of the fuse registers */
- writel(0xffffffff, vpu->ctrl_base + CTRL_G1_DEC_FUSE);
- writel(0xffffffff, vpu->ctrl_base + CTRL_G1_PP_FUSE);
- writel(0xffffffff, vpu->ctrl_base + CTRL_G2_DEC_FUSE);
-
- clk_bulk_disable_unprepare(vpu->variant->num_clocks, vpu->clocks);
-
- return 0;
-}
-
/*
* Supported formats.
*/
@@ -209,27 +148,6 @@ static irqreturn_t imx8m_vpu_g2_irq(int irq, void *dev_id)
return IRQ_HANDLED;
}
-static int imx8mq_vpu_hw_init(struct hantro_dev *vpu)
-{
- vpu->ctrl_base = vpu->reg_bases[vpu->variant->num_regs - 1];
-
- return 0;
-}
-
-static void imx8m_vpu_g1_reset(struct hantro_ctx *ctx)
-{
- struct hantro_dev *vpu = ctx->dev;
-
- imx8m_soft_reset(vpu, RESET_G1);
-}
-
-static void imx8m_vpu_g2_reset(struct hantro_ctx *ctx)
-{
- struct hantro_dev *vpu = ctx->dev;
-
- imx8m_soft_reset(vpu, RESET_G2);
-}
-
/*
* Supported codec ops.
*/
@@ -237,19 +155,16 @@ static void imx8m_vpu_g2_reset(struct hantro_ctx *ctx)
static const struct hantro_codec_ops imx8mq_vpu_codec_ops[] = {
[HANTRO_MODE_MPEG2_DEC] = {
.run = hantro_g1_mpeg2_dec_run,
- .reset = imx8m_vpu_g1_reset,
.init = hantro_mpeg2_dec_init,
.exit = hantro_mpeg2_dec_exit,
},
[HANTRO_MODE_VP8_DEC] = {
.run = hantro_g1_vp8_dec_run,
- .reset = imx8m_vpu_g1_reset,
.init = hantro_vp8_dec_init,
.exit = hantro_vp8_dec_exit,
},
[HANTRO_MODE_H264_DEC] = {
.run = hantro_g1_h264_dec_run,
- .reset = imx8m_vpu_g1_reset,
.init = hantro_h264_dec_init,
.exit = hantro_h264_dec_exit,
},
@@ -258,14 +173,12 @@ static const struct hantro_codec_ops imx8mq_vpu_codec_ops[] = {
static const struct hantro_codec_ops imx8mq_vpu_g2_codec_ops[] = {
[HANTRO_MODE_HEVC_DEC] = {
.run = hantro_g2_hevc_dec_run,
- .reset = imx8m_vpu_g2_reset,
.init = hantro_hevc_dec_init,
.exit = hantro_hevc_dec_exit,
},
[HANTRO_MODE_VP9_DEC] = {
.run = hantro_g2_vp9_dec_run,
.done = hantro_g2_vp9_dec_done,
- .reset = imx8m_vpu_g2_reset,
.init = hantro_vp9_dec_init,
.exit = hantro_vp9_dec_exit,
},
@@ -275,7 +188,7 @@ static const struct hantro_codec_ops imx8mq_vpu_g2_codec_ops[] = {
* VPU variants.
*/
-static const struct hantro_irq imx8mq_irqs[] = {
+static const struct hantro_irq imx8mq_g1_irqs[] = {
{ "g1", imx8m_vpu_g1_irq },
};
@@ -283,10 +196,12 @@ static const struct hantro_irq imx8mq_g2_irqs[] = {
{ "g2", imx8m_vpu_g2_irq },
};
-static const char * const imx8mq_clk_names[] = { "g1", "g2", "bus" };
-static const char * const imx8mq_reg_names[] = { "g1", "g2", "ctrl" };
+static const char * const imx8mq_g1_clk_names[] = { "g1" };
+static const char * const imx8mq_g1_reg_names[] = { "g1" };
+static const char * const imx8mq_g2_clk_names[] = { "g2" };
+static const char * const imx8mq_g2_reg_names[] = { "g2" };
-const struct hantro_variant imx8mq_vpu_variant = {
+const struct hantro_variant imx8mq_vpu_g1_variant = {
.dec_fmts = imx8m_vpu_dec_fmts,
.num_dec_fmts = ARRAY_SIZE(imx8m_vpu_dec_fmts),
.postproc_fmts = imx8m_vpu_postproc_fmts,
@@ -295,14 +210,12 @@ const struct hantro_variant imx8mq_vpu_variant = {
.codec = HANTRO_MPEG2_DECODER | HANTRO_VP8_DECODER |
HANTRO_H264_DECODER,
.codec_ops = imx8mq_vpu_codec_ops,
- .init = imx8mq_vpu_hw_init,
- .runtime_resume = imx8mq_runtime_resume,
- .irqs = imx8mq_irqs,
- .num_irqs = ARRAY_SIZE(imx8mq_irqs),
- .clk_names = imx8mq_clk_names,
- .num_clocks = ARRAY_SIZE(imx8mq_clk_names),
- .reg_names = imx8mq_reg_names,
- .num_regs = ARRAY_SIZE(imx8mq_reg_names)
+ .irqs = imx8mq_g1_irqs,
+ .num_irqs = ARRAY_SIZE(imx8mq_g1_irqs),
+ .clk_names = imx8mq_g1_clk_names,
+ .num_clocks = ARRAY_SIZE(imx8mq_g1_clk_names),
+ .reg_names = imx8mq_g1_reg_names,
+ .num_regs = ARRAY_SIZE(imx8mq_g1_reg_names),
};
const struct hantro_variant imx8mq_vpu_g2_variant = {
@@ -314,10 +227,10 @@ const struct hantro_variant imx8mq_vpu_g2_variant = {
.postproc_ops = &hantro_g2_postproc_ops,
.codec = HANTRO_HEVC_DECODER | HANTRO_VP9_DECODER,
.codec_ops = imx8mq_vpu_g2_codec_ops,
- .init = imx8mq_vpu_hw_init,
- .runtime_resume = imx8mq_runtime_resume,
.irqs = imx8mq_g2_irqs,
.num_irqs = ARRAY_SIZE(imx8mq_g2_irqs),
- .clk_names = imx8mq_clk_names,
- .num_clocks = ARRAY_SIZE(imx8mq_clk_names),
+ .clk_names = imx8mq_g2_clk_names,
+ .num_clocks = ARRAY_SIZE(imx8mq_g2_clk_names),
+ .reg_names = imx8mq_g2_reg_names,
+ .num_regs = ARRAY_SIZE(imx8mq_g2_reg_names),
};
--
2.32.0
On Sun, Dec 5, 2021 at 12:16 PM Adam Ford <[email protected]> wrote:
>
> Currently, the VPU in the i.MQ8MQ is appearing as one codec, but in reality, it's
> two IP blocks called G1 and G2. There is initialization code in VPU code to
> pull some clocks, resets and other features which has been integrated into
> the vpu-blk-ctrl for the i.MX8M Mini and a similar method can be used to make
> the VPU codec's operate as stand-alone cores without having to know the details
> of each other or the quirks unique to the i.MX8MQ, so the remaining code can be
> left more generic.
>
> This series was started by Lucas Stach, and picked up by me so some patches have
> his s-o-b and mine where I might have changed a few minor items. It's in an RFC state
> because the G2 VP9 operations appear to hang, but the parent code from which I started doesn't
> appear to show VP9 support, and it looks like it should.
>
> since the g-streamer and media trees are in a constant state of change, this series is based on
> git://linuxtv.org/hverkuil/media_tree.git for-v5.17e
>
I forgot to post Fluster results.
Before the patches to this branch:
2gst-main] root@localhost:~/gstreamer/fluster# ./fluster.py list -c |grep -i v4l
GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for
GStreamer 1.0... ❌
GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for
GStreamer 1.0... ✔️
GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for
GStreamer 1.0... ❌
GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for
GStreamer 1.0... ❌
GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌
GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for
GStreamer 1.0... ✔️
GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for
GStreamer 1.0... ❌
GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for
GStreamer 1.0... ❌
[gst-main] root@localhost:~/gstreamer/fluster#
There was no VP9 support.
./fluster.py run -dGStreamer-H.264-V4L2SL-Gst1.0
GStreamer-H.264-V4L2SL-Gst1.0
Ran 90/135 tests successfully in 58.787 secs
AFTER this series, two decoders appear:
[ 15.919137] hantro-vpu 38300000.video-codec: registered
nxp,imx8mq-vpu-g1-dec as /dev/video0
[ 15.983579] hantro-vpu 38310000.video-codec: registered
nxp,imx8mq-vpu-g2-dec as /dev/video1
VP9 was listed:
GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for
GStreamer 1.0... ❌
GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for
GStreamer 1.0... ✔️
GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for
GStreamer 1.0... ❌
GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for
GStreamer 1.0... ❌
GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌
GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for
GStreamer 1.0... ✔️
GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for
GStreamer 1.0... ✔️
GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for
GStreamer 1.0... ❌
./fluster.py run -d GStreamer-VP8-V4L2SL-Gst1.0
Ran 55/61 tests successfully in 8.565 secs
./fluster.py run -dGStreamer-H.264-V4L2SL-Gst1.0
Ran 90/135 tests successfully in 60.269 secs
Same results for H.264
VP9 Hangs, where it didn't even appear as available before:
./fluster.py run -d GStreamer-VP9-V4L2SL-Gst1.0
****************************************************************************************************
Running test suite VP9-TEST-VECTORS with decoder GStreamer-VP9-V4L2SL-Gst1.0
Using 4 parallel job(s)
****************************************************************************************************
[TEST SUITE ] (DECODER ) TEST VECTOR
... RESULT
----------------------------------------------------------------------
[VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0)
vp90-2-00-quantizer-00.webm ... Success
[VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0)
vp90-2-00-quantizer-01.webm ... Success
[VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0)
vp90-2-00-quantizer-02.webm ... Succes
<hang > - and yes, 'Success' didnt' finish writing to the serial port.
> Adam Ford (2):
> media: hantro: split i.MX8MQ G1 and G2 code
> arm64: dts: imx8mq: Split i.MX8MQ G1 and G2 with vpu-blk-ctrl
>
> Lucas Stach (3):
> dt-bindings: power: imx8mq: add defines for VPU blk-ctrl domains
> dt-bindings: soc: add binding for i.MX8MQ VPU blk-ctrl
> soc: imx: imx8m-blk-ctrl: add i.MX8MQ VPU blk-ctrl
>
> .../soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml | 71 +++++++++++
> arch/arm64/boot/dts/freescale/imx8mq.dtsi | 69 ++++++----
> drivers/soc/imx/imx8m-blk-ctrl.c | 67 ++++++++++
> drivers/staging/media/hantro/hantro_drv.c | 4 +-
> drivers/staging/media/hantro/hantro_hw.h | 2 +-
> drivers/staging/media/hantro/imx8m_vpu_hw.c | 119 +++---------------
> include/dt-bindings/power/imx8mq-power.h | 3 +
> 7 files changed, 205 insertions(+), 130 deletions(-)
> create mode 100644 Documentation/devicetree/bindings/soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml
>
> --
> 2.32.0
>
On Sun, 05 Dec 2021 12:16:15 -0600, Adam Ford wrote:
> From: Lucas Stach <[email protected]>
>
> This adds the DT binding for the i.MX8MQ VPU blk-ctrl.
>
> Signed-off-by: Lucas Stach <[email protected]>
> Signed-off-by: Adam Ford <[email protected]>
> ---
> .../soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml | 71 +++++++++++++++++++
> 1 file changed, 71 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml
>
My bot found errors running 'make DT_CHECKER_FLAGS=-m dt_binding_check'
on your patch (DT_CHECKER_FLAGS is new in v5.13):
yamllint warnings/errors:
dtschema/dtc warnings/errors:
/builds/robherring/linux-dt-review/Documentation/devicetree/bindings/soc/imx/fsl,imx8mq-vpu-blk-ctrl.example.dt.yaml: blk-ctrl@38320000: compatible: ['fsl,imx8mq-vpu-blk-ctrl', 'syscon'] is too long
From schema: /builds/robherring/linux-dt-review/Documentation/devicetree/bindings/soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml
/builds/robherring/linux-dt-review/Documentation/devicetree/bindings/soc/imx/fsl,imx8mq-vpu-blk-ctrl.example.dt.yaml: blk-ctrl@38320000: compatible: Additional items are not allowed ('syscon' was unexpected)
From schema: /builds/robherring/linux-dt-review/Documentation/devicetree/bindings/soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml
doc reference errors (make refcheckdocs):
See https://patchwork.ozlabs.org/patch/1563759
This check can fail if there are any dependencies. The base for a patch
series is generally the most recent rc1.
If you already ran 'make dt_binding_check' and didn't see the above
error(s), then make sure 'yamllint' is installed and dt-schema is up to
date:
pip3 install dtschema --upgrade
Please check and re-submit.
Le dimanche 05 décembre 2021 à 13:51 -0600, Adam Ford a écrit :
> On Sun, Dec 5, 2021 at 12:16 PM Adam Ford <[email protected]> wrote:
> >
> > Currently, the VPU in the i.MQ8MQ is appearing as one codec, but in reality, it's
> > two IP blocks called G1 and G2. There is initialization code in VPU code to
> > pull some clocks, resets and other features which has been integrated into
> > the vpu-blk-ctrl for the i.MX8M Mini and a similar method can be used to make
> > the VPU codec's operate as stand-alone cores without having to know the details
> > of each other or the quirks unique to the i.MX8MQ, so the remaining code can be
> > left more generic.
> >
> > This series was started by Lucas Stach, and picked up by me so some patches have
> > his s-o-b and mine where I might have changed a few minor items. It's in an RFC state
> > because the G2 VP9 operations appear to hang, but the parent code from which I started doesn't
> > appear to show VP9 support, and it looks like it should.
> >
> > since the g-streamer and media trees are in a constant state of change, this series is based on
> > git://linuxtv.org/hverkuil/media_tree.git for-v5.17e
> >
>
> I forgot to post Fluster results.
>
> Before the patches to this branch:
>
> 2gst-main] root@localhost:~/gstreamer/fluster# ./fluster.py list -c |grep -i v4l
>
> GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for
> GStreamer 1.0... ❌
> GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for
> GStreamer 1.0... ✔️
> GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for
> GStreamer 1.0... ❌
> GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for
> GStreamer 1.0... ❌
> GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌
> GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for
> GStreamer 1.0... ✔️
> GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for
> GStreamer 1.0... ❌
> GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for
> GStreamer 1.0... ❌
> [gst-main] root@localhost:~/gstreamer/fluster#
>
> There was no VP9 support.
>
> ./fluster.py run -dGStreamer-H.264-V4L2SL-Gst1.0
> GStreamer-H.264-V4L2SL-Gst1.0
> Ran 90/135 tests successfully in 58.787 secs
>
> AFTER this series, two decoders appear:
>
> [ 15.919137] hantro-vpu 38300000.video-codec: registered
> nxp,imx8mq-vpu-g1-dec as /dev/video0
> [ 15.983579] hantro-vpu 38310000.video-codec: registered
> nxp,imx8mq-vpu-g2-dec as /dev/video1
>
> VP9 was listed:
>
> GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for
> GStreamer 1.0... ❌
> GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for
> GStreamer 1.0... ✔️
> GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for
> GStreamer 1.0... ❌
> GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for
> GStreamer 1.0... ❌
> GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌
> GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for
> GStreamer 1.0... ✔️
> GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for
> GStreamer 1.0... ✔️
> GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for
> GStreamer 1.0... ❌
>
> ./fluster.py run -d GStreamer-VP8-V4L2SL-Gst1.0
> Ran 55/61 tests successfully in 8.565 secs
>
>
> ./fluster.py run -dGStreamer-H.264-V4L2SL-Gst1.0
> Ran 90/135 tests successfully in 60.269 secs
>
> Same results for H.264
>
> VP9 Hangs, where it didn't even appear as available before:
>
> ./fluster.py run -d GStreamer-VP9-V4L2SL-Gst1.0
> ****************************************************************************************************
> Running test suite VP9-TEST-VECTORS with decoder GStreamer-VP9-V4L2SL-Gst1.0
> Using 4 parallel job(s)
> ****************************************************************************************************
>
> [TEST SUITE ] (DECODER ) TEST VECTOR
> ... RESULT
> ----------------------------------------------------------------------
> [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0)
> vp90-2-00-quantizer-00.webm ... Success
> [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0)
> vp90-2-00-quantizer-01.webm ... Success
> [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0)
> vp90-2-00-quantizer-02.webm ... Succes
>
> <hang > - and yes, 'Success' didnt' finish writing to the serial port.
Looks like hope to me ! Do you get further with -j 1 (one concurrent decode) ?
>
>
>
> > Adam Ford (2):
> > media: hantro: split i.MX8MQ G1 and G2 code
> > arm64: dts: imx8mq: Split i.MX8MQ G1 and G2 with vpu-blk-ctrl
> >
> > Lucas Stach (3):
> > dt-bindings: power: imx8mq: add defines for VPU blk-ctrl domains
> > dt-bindings: soc: add binding for i.MX8MQ VPU blk-ctrl
> > soc: imx: imx8m-blk-ctrl: add i.MX8MQ VPU blk-ctrl
> >
> > .../soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml | 71 +++++++++++
> > arch/arm64/boot/dts/freescale/imx8mq.dtsi | 69 ++++++----
> > drivers/soc/imx/imx8m-blk-ctrl.c | 67 ++++++++++
> > drivers/staging/media/hantro/hantro_drv.c | 4 +-
> > drivers/staging/media/hantro/hantro_hw.h | 2 +-
> > drivers/staging/media/hantro/imx8m_vpu_hw.c | 119 +++---------------
> > include/dt-bindings/power/imx8mq-power.h | 3 +
> > 7 files changed, 205 insertions(+), 130 deletions(-)
> > create mode 100644 Documentation/devicetree/bindings/soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml
> >
> > --
> > 2.32.0
> >
On Mon, Dec 6, 2021 at 2:51 PM Nicolas Dufresne <[email protected]> wrote:
>
> Le dimanche 05 décembre 2021 à 13:51 -0600, Adam Ford a écrit :
> > On Sun, Dec 5, 2021 at 12:16 PM Adam Ford <[email protected]> wrote:
> > >
> > > Currently, the VPU in the i.MQ8MQ is appearing as one codec, but in reality, it's
> > > two IP blocks called G1 and G2. There is initialization code in VPU code to
> > > pull some clocks, resets and other features which has been integrated into
> > > the vpu-blk-ctrl for the i.MX8M Mini and a similar method can be used to make
> > > the VPU codec's operate as stand-alone cores without having to know the details
> > > of each other or the quirks unique to the i.MX8MQ, so the remaining code can be
> > > left more generic.
> > >
> > > This series was started by Lucas Stach, and picked up by me so some patches have
> > > his s-o-b and mine where I might have changed a few minor items. It's in an RFC state
> > > because the G2 VP9 operations appear to hang, but the parent code from which I started doesn't
> > > appear to show VP9 support, and it looks like it should.
> > >
> > > since the g-streamer and media trees are in a constant state of change, this series is based on
> > > git://linuxtv.org/hverkuil/media_tree.git for-v5.17e
> > >
> >
> > I forgot to post Fluster results.
> >
> > Before the patches to this branch:
> >
> > 2gst-main] root@localhost:~/gstreamer/fluster# ./fluster.py list -c |grep -i v4l
> >
> > GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for
> > GStreamer 1.0... ❌
> > GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for
> > GStreamer 1.0... ✔️
> > GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for
> > GStreamer 1.0... ❌
> > GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for
> > GStreamer 1.0... ❌
> > GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌
> > GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for
> > GStreamer 1.0... ✔️
> > GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for
> > GStreamer 1.0... ❌
> > GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for
> > GStreamer 1.0... ❌
> > [gst-main] root@localhost:~/gstreamer/fluster#
> >
> > There was no VP9 support.
> >
> > ./fluster.py run -dGStreamer-H.264-V4L2SL-Gst1.0
> > GStreamer-H.264-V4L2SL-Gst1.0
> > Ran 90/135 tests successfully in 58.787 secs
> >
> > AFTER this series, two decoders appear:
> >
> > [ 15.919137] hantro-vpu 38300000.video-codec: registered
> > nxp,imx8mq-vpu-g1-dec as /dev/video0
> > [ 15.983579] hantro-vpu 38310000.video-codec: registered
> > nxp,imx8mq-vpu-g2-dec as /dev/video1
> >
> > VP9 was listed:
> >
> > GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for
> > GStreamer 1.0... ❌
> > GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for
> > GStreamer 1.0... ✔️
> > GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for
> > GStreamer 1.0... ❌
> > GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for
> > GStreamer 1.0... ❌
> > GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌
> > GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for
> > GStreamer 1.0... ✔️
> > GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for
> > GStreamer 1.0... ✔️
> > GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for
> > GStreamer 1.0... ❌
> >
> > ./fluster.py run -d GStreamer-VP8-V4L2SL-Gst1.0
> > Ran 55/61 tests successfully in 8.565 secs
> >
> >
> > ./fluster.py run -dGStreamer-H.264-V4L2SL-Gst1.0
> > Ran 90/135 tests successfully in 60.269 secs
> >
> > Same results for H.264
> >
> > VP9 Hangs, where it didn't even appear as available before:
> >
> > ./fluster.py run -d GStreamer-VP9-V4L2SL-Gst1.0
> > ****************************************************************************************************
> > Running test suite VP9-TEST-VECTORS with decoder GStreamer-VP9-V4L2SL-Gst1.0
> > Using 4 parallel job(s)
> > ****************************************************************************************************
> >
> > [TEST SUITE ] (DECODER ) TEST VECTOR
> > ... RESULT
> > ----------------------------------------------------------------------
> > [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0)
> > vp90-2-00-quantizer-00.webm ... Success
> > [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0)
> > vp90-2-00-quantizer-01.webm ... Success
> > [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0)
> > vp90-2-00-quantizer-02.webm ... Succes
> >
> > <hang > - and yes, 'Success' didnt' finish writing to the serial port.
>
> Looks like hope to me ! Do you get further with -j 1 (one concurrent decode) ?
Anything after the first attempt appears to fail. I haven't figured
out if it's the G2 code or the power domain stuff. Since VP9 didn't
work I migrated it to use the vpu-blk-ctrl, I can't really compare to
know if it was a regression or an improvement.
:-)
adam
>
> >
> >
> >
> > > Adam Ford (2):
> > > media: hantro: split i.MX8MQ G1 and G2 code
> > > arm64: dts: imx8mq: Split i.MX8MQ G1 and G2 with vpu-blk-ctrl
> > >
> > > Lucas Stach (3):
> > > dt-bindings: power: imx8mq: add defines for VPU blk-ctrl domains
> > > dt-bindings: soc: add binding for i.MX8MQ VPU blk-ctrl
> > > soc: imx: imx8m-blk-ctrl: add i.MX8MQ VPU blk-ctrl
> > >
> > > .../soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml | 71 +++++++++++
> > > arch/arm64/boot/dts/freescale/imx8mq.dtsi | 69 ++++++----
> > > drivers/soc/imx/imx8m-blk-ctrl.c | 67 ++++++++++
> > > drivers/staging/media/hantro/hantro_drv.c | 4 +-
> > > drivers/staging/media/hantro/hantro_hw.h | 2 +-
> > > drivers/staging/media/hantro/imx8m_vpu_hw.c | 119 +++---------------
> > > include/dt-bindings/power/imx8mq-power.h | 3 +
> > > 7 files changed, 205 insertions(+), 130 deletions(-)
> > > create mode 100644 Documentation/devicetree/bindings/soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml
> > >
> > > --
> > > 2.32.0
> > >
>
On Mon, Dec 6, 2021 at 4:30 PM Adam Ford <[email protected]> wrote:
>
> On Mon, Dec 6, 2021 at 2:51 PM Nicolas Dufresne <[email protected]> wrote:
> >
> > Le dimanche 05 décembre 2021 à 13:51 -0600, Adam Ford a écrit :
> > > On Sun, Dec 5, 2021 at 12:16 PM Adam Ford <[email protected]> wrote:
> > > >
> > > > Currently, the VPU in the i.MQ8MQ is appearing as one codec, but in reality, it's
> > > > two IP blocks called G1 and G2. There is initialization code in VPU code to
> > > > pull some clocks, resets and other features which has been integrated into
> > > > the vpu-blk-ctrl for the i.MX8M Mini and a similar method can be used to make
> > > > the VPU codec's operate as stand-alone cores without having to know the details
> > > > of each other or the quirks unique to the i.MX8MQ, so the remaining code can be
> > > > left more generic.
> > > >
> > > > This series was started by Lucas Stach, and picked up by me so some patches have
> > > > his s-o-b and mine where I might have changed a few minor items. It's in an RFC state
> > > > because the G2 VP9 operations appear to hang, but the parent code from which I started doesn't
> > > > appear to show VP9 support, and it looks like it should.
> > > >
> > > > since the g-streamer and media trees are in a constant state of change, this series is based on
> > > > git://linuxtv.org/hverkuil/media_tree.git for-v5.17e
> > > >
> > >
> > > I forgot to post Fluster results.
> > >
> > > Before the patches to this branch:
> > >
> > > 2gst-main] root@localhost:~/gstreamer/fluster# ./fluster.py list -c |grep -i v4l
> > >
> > > GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for
> > > GStreamer 1.0... ❌
> > > GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for
> > > GStreamer 1.0... ✔️
> > > GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for
> > > GStreamer 1.0... ❌
> > > GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for
> > > GStreamer 1.0... ❌
> > > GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌
> > > GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for
> > > GStreamer 1.0... ✔️
> > > GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for
> > > GStreamer 1.0... ❌
> > > GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for
> > > GStreamer 1.0... ❌
> > > [gst-main] root@localhost:~/gstreamer/fluster#
> > >
> > > There was no VP9 support.
> > >
> > > ./fluster.py run -dGStreamer-H.264-V4L2SL-Gst1.0
> > > GStreamer-H.264-V4L2SL-Gst1.0
> > > Ran 90/135 tests successfully in 58.787 secs
> > >
> > > AFTER this series, two decoders appear:
> > >
> > > [ 15.919137] hantro-vpu 38300000.video-codec: registered
> > > nxp,imx8mq-vpu-g1-dec as /dev/video0
> > > [ 15.983579] hantro-vpu 38310000.video-codec: registered
> > > nxp,imx8mq-vpu-g2-dec as /dev/video1
> > >
> > > VP9 was listed:
> > >
> > > GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for
> > > GStreamer 1.0... ❌
> > > GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for
> > > GStreamer 1.0... ✔️
> > > GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for
> > > GStreamer 1.0... ❌
> > > GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for
> > > GStreamer 1.0... ❌
> > > GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌
> > > GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for
> > > GStreamer 1.0... ✔️
> > > GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for
> > > GStreamer 1.0... ✔️
> > > GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for
> > > GStreamer 1.0... ❌
> > >
> > > ./fluster.py run -d GStreamer-VP8-V4L2SL-Gst1.0
> > > Ran 55/61 tests successfully in 8.565 secs
> > >
> > >
> > > ./fluster.py run -dGStreamer-H.264-V4L2SL-Gst1.0
> > > Ran 90/135 tests successfully in 60.269 secs
> > >
> > > Same results for H.264
> > >
> > > VP9 Hangs, where it didn't even appear as available before:
> > >
> > > ./fluster.py run -d GStreamer-VP9-V4L2SL-Gst1.0
> > > ****************************************************************************************************
> > > Running test suite VP9-TEST-VECTORS with decoder GStreamer-VP9-V4L2SL-Gst1.0
> > > Using 4 parallel job(s)
> > > ****************************************************************************************************
> > >
> > > [TEST SUITE ] (DECODER ) TEST VECTOR
> > > ... RESULT
> > > ----------------------------------------------------------------------
> > > [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0)
> > > vp90-2-00-quantizer-00.webm ... Success
> > > [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0)
> > > vp90-2-00-quantizer-01.webm ... Success
> > > [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0)
> > > vp90-2-00-quantizer-02.webm ... Succes
> > >
> > > <hang > - and yes, 'Success' didnt' finish writing to the serial port.
> >
> > Looks like hope to me ! Do you get further with -j 1 (one concurrent decode) ?
>
> Anything after the first attempt appears to fail. I haven't figured
> out if it's the G2 code or the power domain stuff. Since VP9 didn't
> work I migrated it to use the vpu-blk-ctrl, I can't really compare to
> know if it was a regression or an improvement.
>
> :-)
>
I managed to get the vpu-blk-ctrl updated to address the G2 hanging.
./fluster.py run -d GStreamer-VP9-V4L2SL-Gst1.0
Ran 144/303 tests successfully in 224.847 secs
Since I couldn't get it to work before the split, I am not sure if
this is good or not. I also wasn't sure if I needed to allocate more
CMA for this.
If/when people can give me some feedback on the RFC's, I'll submit an
updated formal patch for both the i.MQ8MQ and the i.MQ8M Mini as one
series based on "for-v5.17e" from
git://linuxtv.org/hverkuil/media_tree.git
I wasn't sure if I needed to break up the migration from a combined
VPU to split G1 and G2. I might need some help/suggestions on the
YAML, because I struggle with that part a little, but I'd like the 8MM
to use the 8MQ dt-binding yaml file since they are so similar.
I hope to have some more time this week because my weekends are
completely booked for the rest of the year, and leave for London, on
23 Dec 2021 and I won't return until 5 Jan 2022.
> adam
>
> >
> > >
> > >
> > >
> > > > Adam Ford (2):
> > > > media: hantro: split i.MX8MQ G1 and G2 code
> > > > arm64: dts: imx8mq: Split i.MX8MQ G1 and G2 with vpu-blk-ctrl
> > > >
> > > > Lucas Stach (3):
> > > > dt-bindings: power: imx8mq: add defines for VPU blk-ctrl domains
> > > > dt-bindings: soc: add binding for i.MX8MQ VPU blk-ctrl
> > > > soc: imx: imx8m-blk-ctrl: add i.MX8MQ VPU blk-ctrl
> > > >
> > > > .../soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml | 71 +++++++++++
> > > > arch/arm64/boot/dts/freescale/imx8mq.dtsi | 69 ++++++----
> > > > drivers/soc/imx/imx8m-blk-ctrl.c | 67 ++++++++++
> > > > drivers/staging/media/hantro/hantro_drv.c | 4 +-
> > > > drivers/staging/media/hantro/hantro_hw.h | 2 +-
> > > > drivers/staging/media/hantro/imx8m_vpu_hw.c | 119 +++---------------
> > > > include/dt-bindings/power/imx8mq-power.h | 3 +
> > > > 7 files changed, 205 insertions(+), 130 deletions(-)
> > > > create mode 100644 Documentation/devicetree/bindings/soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml
> > > >
> > > > --
> > > > 2.32.0
> > > >
> >