This is the v7 version of this series adding hantro g1 video decoder
support for RK3588.
RK3588 has Hantro G1 video decoder known as VDPU121 in TRM of RK3588 which
is capable to decode MPEG2/H.264/VP8 up to 1920x1088. This vpu ip is also
found in RK3568.
Test results from fluster can be found from thread of v3[1].
Changes in v7:
- Change compatible string from vdpu121 to vpu121 to match the real
hardware
- Change the devicetree node vdpu_mmu to vpu_mmu because this mmu is
also used by vepu121 jpeg encoder
- Link to v6: https://lore.kernel.org/all/[email protected]/
Changes in v6:
- Apply dt-bindings first
- Collect missing commit tags of old versions
- Specify the base commit suggested by Diederik
- Link to v5: https://lore.kernel.org/all/[email protected]/
Changes in v5:
- Add missing interrupt-names to devicetree node
- Rebase devicetree patch based on next-20240412
- Link to v4: https://lore.kernel.org/all/[email protected]/
Changes in v4:
- Change compatible string to rockchip,rk3588-vdpu121
- Link to v3: https://lore.kernel.org/all/[email protected]/
Changes in v3:
- Drop code in hantro_drv.c because hantro g1 vpu in rk3588 is compatible
with the one in rk3568, only adding devicetree node should work.
- Change devicetree node name to video-codec@fdb50000 to match the reg
address.
- Add dt-bindings rockchip,rk3588-vpu compatible with rockchip,rk3568-vpu
- Link to v2: https://lore.kernel.org/all/[email protected]
Changes in v2:
- Fix alphabetical order in patch1 and patch3
- Sort device tree node by bus-address
- Drop rk3588_vpu_variant fron v1 because that is exactly the same as rk3568_vpu_variant
- Link to v1: https://lore.kernel.org/all/[email protected]
[1]https://lore.kernel.org/all/[email protected]/
Jianfeng Liu (2):
dt-bindings: media: rockchip-vpu: Add rk3588 vpu121 compatible string
arm64: dts: rockchip: Add Hantro G1 VPU support for RK3588
.../bindings/media/rockchip-vpu.yaml | 3 +++
arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 21 +++++++++++++++++++
2 files changed, 24 insertions(+)
base-commit: b0a2c79c6f3590b74742cbbc76687014d47972d8
--
2.34.1
Enable Hantro G1 video decoder in RK3588's devicetree.
Tested with FFmpeg v4l2_request code taken from [1]
with MPEG2, H.264 and VP8 samples.
[1] https://github.com/LibreELEC/LibreELEC.tv/blob/master/packages/multimedia/ffmpeg/patches/v4l2-request/ffmpeg-001-v4l2-request.patch
Signed-off-by: Jianfeng Liu <[email protected]>
Tested-by: Hugh Cole-Baker <[email protected]>
Reviewed-by: Sebastian Reichel <[email protected]>
---
arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
index b0a59ec51..da10df7d6 100644
--- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi
@@ -1135,6 +1135,27 @@ power-domain@RK3588_PD_SDMMC {
};
};
+ vpu: video-codec@fdb50000 {
+ compatible = "rockchip,rk3588-vpu121", "rockchip,rk3568-vpu";
+ reg = <0x0 0xfdb50000 0x0 0x800>;
+ interrupts = <GIC_SPI 119 IRQ_TYPE_LEVEL_HIGH 0>;
+ interrupt-names = "vdpu";
+ clocks = <&cru ACLK_VPU>, <&cru HCLK_VPU>;
+ clock-names = "aclk", "hclk";
+ iommus = <&vpu_mmu>;
+ power-domains = <&power RK3588_PD_VDPU>;
+ };
+
+ vpu_mmu: iommu@fdb50800 {
+ compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
+ reg = <0x0 0xfdb50800 0x0 0x40>;
+ interrupts = <GIC_SPI 118 IRQ_TYPE_LEVEL_HIGH 0>;
+ clock-names = "aclk", "iface";
+ clocks = <&cru ACLK_VPU>, <&cru HCLK_VPU>;
+ power-domains = <&power RK3588_PD_VDPU>;
+ #iommu-cells = <0>;
+ };
+
av1d: video-codec@fdc70000 {
compatible = "rockchip,rk3588-av1-vpu";
reg = <0x0 0xfdc70000 0x0 0x800>;
--
2.34.1
Add Hantro G1 VPU compatible string for RK3588.
RK3588 has the same Hantro G1 ip as RK3568, which are both
known as VDPU121 in TRM of RK3568 and RK3588.
Note that this VPU also has a jpeg encoder, which is one part of
the five VEPU121 jpeg encoders on RK3588. So this VPU121 shoud be
the same as vpu@ff650000 on RK3399. But we don't use the compatible
string rk3399-vpu because of two reasons:
1, rk3399-vpu has disabled H264 decoding because RK3399 also has
rkvdec to support 4K H264 decoding. And we need H264 decoding because
rkvdec2 on rk3588 for H264 decoding is not supported now.
2, There are five VEPU121 jpeg encoders, but the kernel driver can't
do scheduling. So it's better to disable the VEPU121 jpeg encoder
at the moment.
Signed-off-by: Jianfeng Liu <[email protected]>
Acked-by: Conor Dooley <[email protected]>
Reviewed-by: Sebastian Reichel <[email protected]>
---
Documentation/devicetree/bindings/media/rockchip-vpu.yaml | 3 +++
1 file changed, 3 insertions(+)
diff --git a/Documentation/devicetree/bindings/media/rockchip-vpu.yaml b/Documentation/devicetree/bindings/media/rockchip-vpu.yaml
index c57e1f488..2710bb2fb 100644
--- a/Documentation/devicetree/bindings/media/rockchip-vpu.yaml
+++ b/Documentation/devicetree/bindings/media/rockchip-vpu.yaml
@@ -31,6 +31,9 @@ properties:
- items:
- const: rockchip,rk3228-vpu
- const: rockchip,rk3399-vpu
+ - items:
+ - const: rockchip,rk3588-vpu121
+ - const: rockchip,rk3568-vpu
reg:
maxItems: 1
--
2.34.1
Sorry,
[...]
> > + - const: rockchip,rk3568-vpu
>
> Sorry to come that late, but I'm noticing a big mistake here. You said you are
> enabling VDPU121, the JPEG decoder. But we don't have a JPEG decoder driver
> mainline, is there some patches missing ?
>
> Nicolas
Ignore this part, just didn't read carefully. This is about getting H264, VP8
and MPEG2 out of these extra cores of course. I still would like to know how we
will express the grouping of these four cores, so a driver can know they are
identical G1 cores and not bound to be time sliced with an H1 core like the
fifth one? I want to see a plan that will work and will not cause headache for
future work on fully utilizing the HW resources.
Nicolas
Hi,
Le mardi 30 avril 2024 à 10:40 +0800, Jianfeng Liu a écrit :
> Add Hantro G1 VPU compatible string for RK3588.
> RK3588 has the same Hantro G1 ip as RK3568, which are both
> known as VDPU121 in TRM of RK3568 and RK3588.
>
> Note that this VPU also has a jpeg encoder, which is one part of
> the five VEPU121 jpeg encoders on RK3588. So this VPU121 shoud be
> the same as vpu@ff650000 on RK3399. But we don't use the compatible
> string rk3399-vpu because of two reasons:
> 1, rk3399-vpu has disabled H264 decoding because RK3399 also has
> rkvdec to support 4K H264 decoding. And we need H264 decoding because
> rkvdec2 on rk3588 for H264 decoding is not supported now.
> 2, There are five VEPU121 jpeg encoders, but the kernel driver can't
> do scheduling. So it's better to disable the VEPU121 jpeg encoder
> at the moment.
>
> Signed-off-by: Jianfeng Liu <[email protected]>
> Acked-by: Conor Dooley <[email protected]>
> Reviewed-by: Sebastian Reichel <[email protected]>
> ---
> Documentation/devicetree/bindings/media/rockchip-vpu.yaml | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/Documentation/devicetree/bindings/media/rockchip-vpu.yaml b/Documentation/devicetree/bindings/media/rockchip-vpu.yaml
> index c57e1f488..2710bb2fb 100644
> --- a/Documentation/devicetree/bindings/media/rockchip-vpu.yaml
> +++ b/Documentation/devicetree/bindings/media/rockchip-vpu.yaml
> @@ -31,6 +31,9 @@ properties:
> - items:
> - const: rockchip,rk3228-vpu
> - const: rockchip,rk3399-vpu
> + - items:
> + - const: rockchip,rk3588-vpu121
> + - const: rockchip,rk3568-vpu
Sorry to come that late, but I'm noticing a big mistake here. You said you are
enabling VDPU121, the JPEG decoder. But we don't have a JPEG decoder driver
mainline, is there some patches missing ?
Nicolas
>
> reg:
> maxItems: 1
> --
> 2.34.1
>